PD36-02 EVALUATING AI ASSISTANCE IN PROSTATE BPMRI INTERPRETATION: A MULTI-READER STUDY

You have accessJournal of UrologySurgical Technology & Simulation: Artificial Intelligence III (PD36)1 May 2024PD36-02 EVALUATING AI ASSISTANCE IN PROSTATE BPMRI INTERPRETATION: A MULTI-READER STUDY David G. Gelikman, Enis C. Yilmaz, Stephanie A. Harmon, Julie Y. An, Sena Azamat, Yan Mee Law, Daniel J. A. Margolis, Jamie Marko, Valeria Panebianco, Sonia Gaur, Marco Bicchetti, Erich P. Huang, Sandeep Gurram, Joanna H. Shih, Peter L. Choyke, Bradford J. Wood, Peter A. Pinto, and Baris Turkbey David G. GelikmanDavid G. Gelikman , Enis C. YilmazEnis C. Yilmaz , Stephanie A. HarmonStephanie A. Harmon , Julie Y. AnJulie Y. An , Sena AzamatSena Azamat , Yan Mee LawYan Mee Law , Daniel J. A. MargolisDaniel J. A. Margolis , Jamie MarkoJamie Marko , Valeria PanebiancoValeria Panebianco , Sonia GaurSonia Gaur , Marco BicchettiMarco Bicchetti , Erich P. HuangErich P. Huang , Sandeep GurramSandeep Gurram , Joanna H. ShihJoanna H. Shih , Peter L. ChoykePeter L. Choyke , Bradford J. WoodBradford J. Wood , Peter A. PintoPeter A. Pinto , and Baris TurkbeyBaris Turkbey View All Author Informationhttps://doi.org/10.1097/01.JU.0001008916.72488.6a.02AboutPDF ToolsAdd to favoritesDownload CitationsTrack CitationsPermissionsReprints ShareFacebookLinked InTwitterEmail Abstract INTRODUCTION AND OBJECTIVE: Interpretation of biparametric magnetic resonance imaging (bpMRI) for prostate cancer is subject to significant inter-reader variability. Artificial intelligence (AI) has the potential to enhance diagnostic accuracy and consistency among radiologists. This study aims to evaluate the effectiveness of a deep-learning AI model as a first reader in assisting radiologists of varied experience in interpreting prostate bpMRI. METHODS: Six radiologists (3 prostate-focused and 3 generalists) from different institutions each evaluated 120 prostate bpMRIs, of which 80 were from cases with pathologically confirmed prostate cancer and 40 were controls. In 60 of these scans, readers used AI assistance using a first-reader method in which they were only allowed to accept or reject lesions that were detected on AI prediction maps without reporting any additional lesions. The remaining 60 cases were read without AI. We conducted a patient-level analysis to compare sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy for lesion detection at bpMRI with and without AI. Inter-reader agreement on lesion measurement and PI-RADS scores was also evaluated. RESULTS: The patient cohort had a median age of 63 years (IQR, 57-68) and prostate-specific antigen of 7.1 ng/mL (IQR, 5.1-10.8). AI assistance varied in effectiveness across all readers, with sensitivity ranging from 83%-93%, specificity from 16%-90%, and accuracy from 68%-87%. Without AI, the ranges were 82%-98% for sensitivity, 19%-81% for specificity, and 70%-87% for accuracy, Table 1. One prostate-focused reader improved accuracy by 10% with AI and one general radiologist showed a 5% improvement. AI-assistance resulted in a slight, non-significant reduction in the median absolute difference in largest lesion dimension measurements (1.56 mm with AI vs. 2.27 mm without; p=.373). The quadratic weighted Cohen's kappa indicated a slight improvement in PI-RADS score agreement from 0.438 to 0.459 with AI. CONCLUSIONS: AI assistance in the interpretation of bpMRI can improve accuracy for some readers. Despite a slight improvement in measurement agreement and PI-RADS scores, the varied impact of AI on different readers calls for further investigation into how AI tools can best complement radiologists in an effective and consistent manner. Source of Funding: Intramural Research Program of the NCI, NIH © 2024 by American Urological Association Education and Research, Inc.FiguresReferencesRelatedDetails Volume 211Issue 5SMay 2024Page: e792 Advertisement Copyright & Permissions© 2024 by American Urological Association Education and Research, Inc.Metrics Author Information David G. Gelikman More articles by this author Enis C. Yilmaz More articles by this author Stephanie A. Harmon More articles by this author Julie Y. An More articles by this author Sena Azamat More articles by this author Yan Mee Law More articles by this author Daniel J. A. Margolis More articles by this author Jamie Marko More articles by this author Valeria Panebianco More articles by this author Sonia Gaur More articles by this author Marco Bicchetti More articles by this author Erich P. Huang More articles by this author Sandeep Gurram More articles by this author Joanna H. Shih More articles by this author Peter L. Choyke More articles by this author Bradford J. Wood More articles by this author Peter A. Pinto More articles by this author Baris Turkbey More articles by this author Expand All Advertisement PDF downloadLoading ...