External validation of cerebral aneurysm rupture probability model with data from two patient cohorts

Background For a treatment decision of unruptured cerebral aneurysms, physicians and patients need to weigh the risk of treatment against the risk of hemorrhagic stroke caused by aneurysm rupture. The aim of this study was to externally evaluate a recently developed statistical aneurysm rupture probability model, which could potentially support such treatment decisions. Methods Segmented image data and patient information obtained from two patient cohorts including 203 patients with 249 aneurysms were used for patient-specific computational fluid dynamics simulations and subsequent evaluation of the statistical model in terms of accuracy, discrimination, and goodness of fit. The model’s performance was further compared to a similarity-based approach for rupture assessment by identifying aneurysms in the training cohort that were similar in terms of hemodynamics and shape compared to a given aneurysm from the external cohorts. Results When applied to the external data, the model achieved a good discrimination and goodness of fit (area under the receiver operating characteristic curve AUC = 0.82), which was only slightly reduced compared to the optimism-corrected AUC in the training population (AUC = 0.84). The accuracy metrics indicated a small decrease in accuracy compared to the training data (misclassification error of 0.24 vs. 0.21). The model’s prediction accuracy was improved when combined with the similarity approach (misclassification error of 0.14). Conclusions The model’s performance measures indicated a good generalizability for data acquired at different clinical institutions. Combining the model-based and similarity-based approach could further improve the assessment and interpretation of new cases, demonstrating its potential use for clinical risk assessment.