Prediction of cancer treatment response from histopathology images for a broad set of treatments and indications

Abstract Background: Advances in artificial intelligence have paved the way for predicting cancer patients’ survival and response to treatment from hematoxylin and eosin (H&E)-stained tumor slides. Extant approaches do so either via prediction of actionable mutations and gene fusions, or directly from the H&E images by training a task-specific model using large treatment outcome data. Methods: Here we present the first approach for predicting patient response to multiple targeted treatments and immunotherapies directly from H&E slides. It is founded on two conceptual steps: (1) First, we developed DeepPT, a deep-learning framework that trains on formalin-fixed, paraffin-embedded (FFPE) TCGA whole slide images and their corresponding gene expression profiles to predict wide-scale tumor gene expression from the slides. (2) Second, we apply ENLIGHT, a published approach that predicts individual responses to a wide range of targeted and immunotherapies based on the tumor biopsy measured transcriptomics. Here we apply ENLIGHT to predict patient treatment response from the DeepPT predicted expression values instead of those measured directly from the tumor, notably, without any further training or adaptation of ENLIGHT. Results: First, we find that DeepPT generalizes well to predicting gene expression in all 13 TCGA cohorts tested in cross-validation and importantly, in two independent unseen breast and brain cancer datasets. DeepPT outperforms HE2RNA, a state-of-the-art algorithm for the same task. Second, we demonstrate that the combined DeepPT/ENLIGHT pipeline (termed ENLIGHT-DeepPT) successfully predicts true responders using the original ENLIGHT decision threshold with odds ratios of 1.5 - 4.5, increasing the baseline response rates by 15-85% among predicted responders in five independent unseen cohorts of diverse cancer types and treatments. Remarkably, in one dataset where matched data was available, ENLIGHT-DeepPT has similar performance to that obtained by a supervised learning algorithm that was recently published in Nature, trained on the same cohort. Conclusions: We present for the first time a general framework for predicting patient response to a broad array of targeted and checkpoint therapies from histopathological images, without reliance on treatment outcome data, which are yet scarce and challenging to obtain. Importantly, ENLIGHT-DeepPT offers clinicians real-time treatment recommendations when one cannot wait for sequencing results. Also, and due to its very low cost, we very much hope that it will facilitate the advent of precision oncology in developing countries. Citation Format: Danh-Tai Hoang, Gal Dinstag, Leandro C. Hermida, Doreen S. Ben-Zvi, Efrat Elis, Katherine Caley, Sanju Sinha, Neelam Sinha, Christopher H. Dampier, Tuvik Beker, Kenneth Aldape, Ranit Aharonov, Eric A. Stone, Eytan Ruppin. Prediction of cancer treatment response from histopathology images for a broad set of treatments and indications. [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2023; Part 1 (Regular and Invited Abstracts); 2023 Apr 14-19; Orlando, FL. Philadelphia (PA): AACR; Cancer Res 2023;83(7_Suppl):Abstract nr 4355.