A causal framework for assessing the transportability of clinical prediction models

BACKGROUND: Machine learning promises to support the diagnosis of dementia and Alzheimer's Disease, but may not perform well in new settings. We present a framework to assess the transportability of models predicting cognitive impairment in external settings with different demographics. METHODS: We mapped and quantified relationships between variables associated with cognitive impairment using causal graphs, structural equation models, and data from the ADNI study. These estimates generated datasets for training and validating prediction models. We measured transportability to external settings with interventions on age, APOE e4, and sex, using calibration metric differences. RESULTS: Models predicting with causes of the outcome were 1.3-12.8 times more transportable than those predicting with consequences. Logistic and lasso models had better calibration in internal validation settings than random forest and boosted models. DISCUSSION: Applying a framework considering causal relationships is crucial to assess transportability. Future research could investigate more interventions and methods to quantify causal relationships in risk prediction.