Abstract LB152: The Critical Assessment of Genome Interpretation: A community experiment that informs use of methods for germline cancer variant impact prediction

Abstract Interpretation of genomic variation plays an essential role in the analysis of cancer and monogenic disease, with applications ranging from basic research to clinical decisions. Yet the field lacks a clear consensus on the appropriate level of confidence to place in variant impact and interpretation methods, for both well-established oncogenes as well as less understood genes. The Critical Assessment of Genome Interpretation (CAGI, \'kā-jē\) is a community experiment to objectively assess computational methods for predicting the phenotypic impacts of genomic variation. CAGI participants are provided genetic variants and make blind predictions of resulting phenotype. Independent assessors evaluate the predictions by comparing them against experimental and clinical data. Six CAGI editions involving over 60 experiments have taken place to date. Results have been described in two special issues of Human Mutation (vol. 38(9) and vol. 40(9)). About a quarter of CAGI experiments have involved genes implicated in cancer, including variants in BRCA1, BRCA2, CHEK2, the MRN complex (RAD50, MRE11, and NBS1), FXN, NSMCE2 (coding for SUMO ligase), NPM-ALK CDKN2A, PTEN, TPMT, STK11, p16 and TP53. In the ENIGMA consortium challenge, a total of 326 germline BRCA1/BRCA2 missense and in-frame indel variants were provided. The most successful predictions achieved a ROC AUC of 0.94 and were derived from a metapredictor that explicitly considered the population frequency of each variant, trained on cancer data, and included the role of altered splicing. In challenges exploring the role of protein stability in cancer progression, biophysical and structure-based methods are among the top performers and have provided insights into underlying molecular mechanisms. This was the case for the p16 challenge involving variants of unknown significance in familial malignant melanoma, the PTEN and TPMT challenges that measured variant effect on protein stability via intracellular abundance in a high-throughput assay, and the p53 rescue challenge in which predictors managed to successfully identify a handful of double mutants, among a set of 14,668 total variants, that reactivated the damaged protein. Cancer challenges involving breast cancer pharmacogenomics, case-controls, splicing and polygenic risk scores (PRS) have been moderately successful but show promise for the future. A key finding from CAGI is that for most missense challenges, including cancer, it is possible to relate phenotype values to a pathogenicity threshold, and so deduce potential performance in a clinical setting. The results suggest that computational methods are generally more reliable than recognized in the current clinical use guidelines, warranting a reevaluation of their role in clinical variant interpretation. Results from the ENIGMA and other BRCA challenges support this observation but more data are needed. Overall, CAGI has helped establishing the state of art in genome interpretation, encouraged new methodological developments, and informed the clinical application of computational predictors. Detailed information about CAGI may be found at https://genomeinterpretation.org. Citation Format: Constantina Bakolitsa, Shantanu Jain, Gaia Andreoletti, Roger A. Hoskins, Predrag Radivojac, John Moult, Steven E. Brenner, CAGI Participants. The Critical Assessment of Genome Interpretation: A community experiment that informs use of methods for germline cancer variant impact prediction [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2022; 2022 Apr 8-13. Philadelphia (PA): AACR; Cancer Res 2022;82(12_Suppl):Abstract nr LB152.