Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

A deep-learning framework to predict cancer treatment response from histopathology images through imputed transcriptomics

Abstract

Advances in artificial intelligence have paved the way for leveraging hematoxylin and eosin-stained tumor slides for precision oncology. We present ENLIGHT–DeepPT, an indirect two-step approach consisting of (1) DeepPT, a deep-learning framework that predicts genome-wide tumor mRNA expression from slides, and (2) ENLIGHT, which predicts response to targeted and immune therapies from the inferred expression values. We show that DeepPT successfully predicts transcriptomics in all 16 The Cancer Genome Atlas cohorts tested and generalizes well to two independent datasets. ENLIGHT–DeepPT successfully predicts true responders in five independent patient cohorts involving four different treatments spanning six cancer types, with an overall odds ratio of 2.28 and a 39.5% increased response rate among predicted responders versus the baseline rate. Notably, its prediction accuracy, obtained without any training on the treatment data, is comparable to that achieved by directly predicting the response from the images, which requires specific training on the treatment evaluation cohorts.

This is a preview of subscription content, access via your institution

Access options

Buy this article

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Study overview.
Fig. 2: DeepPT prediction of gene expression from H&E slides.
Fig. 3: Gene set enrichment analysis identifying pathways whose predicted gene expression correlates with TIL abundance.
Fig. 4: Comparison of the correlation of survival association in terms of log(HR) for three proliferation signatures based on actual and predicted expression.
Fig. 5: Predicting treatment response from H&E slides.

Similar content being viewed by others

Data availability

TCGA histological images and their corresponding gene expression profiles were downloaded from the Genomic Data Commons Data Portal (https://portal.gdc.cancer.gov). The TransNEO-breast dataset is available at https://ega-archive.org/studies/EGAS00001004582. The NCI-brain and bintrafusp alfa cohorts were obtained from the NCI. The trastuzumab1 cohort is a subset of the TransNEO-breast dataset. Trastuzumab2 cohort data were downloaded from The Cancer Imaging Archive database (https://www.cancerimagingarchive.net). The ALKi dataset was made available by the University of Colorado, and the PARPi dataset was made available by Sheba Medical Center. Restrictions apply to the distribution of these data, which were used under license for the current study. Access to the ALKi dataset may be requested from S.P. (sharon.pine@cuanschutz.edu). Access to the PARPi dataset may be requested from T.G. (Talia.Golan@sheba.health.gov.il). All DeepPT-predicted expression and relevant response data, along with the code to calculate performance measures, are available via GitHub at https://github.com/PangeaResearch/enlight-deeppt-data. Source data are provided with this paper. All other data supporting the findings of this study are available from the corresponding authors upon reasonable request.

Code availability

The DeepPT framework is available for academic research purposes via Zenodo at https://doi.org/10.5281/zenodo.11125591 (ref. 61). ENLIGHT scores, given expression profiles (either measured directly from the tumor or predicted from slides), can be calculated using a web service at https://ems.pangeabiomed.com/. Any additional information is available from the corresponding authors upon request.

References

  1. Golub, T. R. et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science 286, 531–537 (1999).

    Article  CAS  PubMed  Google Scholar 

  2. Curtis, C. et al. The genomic and transcriptomic architecture of 2,000 breast tumours reveals novel subgroups. Nature 486, 346–352 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Doroshow, D. B. & Doroshow, J. H. Genomics and the history of precision oncology. Surg. Oncol. Clin. N. Am. 29, 35–49 (2020).

    Article  PubMed  Google Scholar 

  4. Rosenthal, J. et al. Building tools for machine learning and artificial intelligence in cancer research: best practices and a case study with the PathML toolkit for computational pathology. Mol. Cancer Res. 20, 202–206 (2022).

    Article  CAS  PubMed  Google Scholar 

  5. Ström, P. et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. 21, 222–232 (2020).

    Article  PubMed  Google Scholar 

  6. Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer 1, 800–810 (2020).

    Article  CAS  PubMed  Google Scholar 

  7. Noorbakhsh, J. et al. Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat. Commun. 11, 6367 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Kim, J. et al. Unsupervised discovery of tissue architecture in multiplexed imaging. Nat. Methods 19, 1653–1661 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Coudray, N. et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Swiderska-Chadaj, Z. et al. Learning to detect lymphocytes in immunohistochemistry with deep learning. Med. Image Anal. 58, 101547 (2019).

    Article  PubMed  Google Scholar 

  11. Yu, K.-H. et al. Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks. J. Am. Med. Inform. Assoc. 27, 757–769 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  12. Couture, H. D. et al. Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. NPJ Breast Cancer 4, 30 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  13. Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Xu, H. et al. Spatial heterogeneity and organization of tumor mutation burden with immune infiltrates within tumors based on whole slide images correlated with patient survival in bladder cancer. J. Pathol. Inform. 13, 100105 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  15. Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer 1, 789–799 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Qu, H. et al. Genetic mutation and biological pathway prediction based on whole slide images in breast carcinoma using deep learning. NPJ Precis. Oncol. 5, 87 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. Schaumberg, A. J., Rubin, M. A. & Fuchs, T. J. H&E-stained whole slide image deep learning predicts SPOP mutation state in prostate cancer. Preprint at bioRxiv https://doi.org/10.1101/064279 (2018).

  18. Tsou, P. & Wu, C.-J. Mapping driver mutations to histopathological subtypes in papillary thyroid carcinoma: applying a deep convolutional neural network. J. Clin. Med. 8, 1675 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Chang, P. et al. Deep-learning convolutional neural networks accurately classify genetic mutations in gliomas. AJNR Am. J. Neuroradiol. 39, 1201–1207 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Kather, J. N. et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25, 1054–1056 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Kim, R. H. et al. A deep learning approach for rapid mutational screening in melanoma. Preprint at bioRxiv https://doi.org/10.1101/610311 (2019).

  22. Chen, M. et al. Classification and mutation prediction based on histopathology H&E images in liver cancer using deep learning. NPJ Precis. Oncol. 4, 14 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Ghaffari Laleh, N. et al. Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology. Med. Image Anal. 79, 102474 (2022).

    Article  PubMed  Google Scholar 

  24. Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  25. Cheng, J. et al. Integrative analysis of histopathological images and genomic data predicts clear cell renal cell carcinoma prognosis. Cancer Res. 77, e91–e100 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Beck, A. H. et al. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci. Transl. Med. 3, 108ra113 (2011).

    Article  PubMed  Google Scholar 

  27. Bulten, W. et al. Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21, 233–241 (2020).

    Article  PubMed  Google Scholar 

  28. Courtiol, P. et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nat. Med. 25, 1519–1525 (2019).

    Article  CAS  PubMed  Google Scholar 

  29. Boehm, K. M. et al. Multimodal data integration using machine learning improves risk stratification of high-grade serous ovarian cancer. Nat. Cancer 3, 723–733 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Chen, Y. et al. Computational pathology improves risk stratification of a multi-gene assay for early stage ER+ breast cancer. NPJ Breast Cancer 9, 40 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  31. Zheng, H., Momeni, A., Cedoz, P.-L., Vogel, H. & Gevaert, O. Whole slide images reflect DNA methylation patterns of human tumors. NPJ Genom. Med. 5, 11 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Wang, H. et al. Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. J. Med. Imaging (Bellingham) 1, 034003 (2014).

    Article  PubMed  Google Scholar 

  33. Turkki, R., Linder, N., Kovanen, P. E., Pellinen, T. & Lundin, J. Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples. J. Pathol. Inform. 7, 38 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  34. Saltz, J. et al. Spatial organization and molecular correlation of tumor-infiltrating lymphocytes using deep learning on pathology images. Cell Rep. 23, 181–193 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Page, D. B. et al. Spatial analyses of immune cell infiltration in cancer: current methods and future directions: a report of the International Immuno-Oncology Biomarker Working Group on Breast Cancer. J. Pathol. 260, 514–532 (2023).

    Article  CAS  PubMed  Google Scholar 

  36. Wang, X. et al. Spatial interplay patterns of cancer nuclei and tumor-infiltrating lymphocytes (TILs) predict clinical benefit for immune checkpoint inhibitors. Sci. Adv. 8, eabn3966 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Hu, J. et al. Using deep learning to predict anti-PD-1 response in melanoma and lung cancer patients from histopathology images. Transl. Oncol. 14, 100921 (2021).

    Article  CAS  PubMed  Google Scholar 

  38. Johannet, P. et al. Using machine learning algorithms to predict immunotherapy response in patients with advanced melanoma. Clin. Cancer Res. 27, 131–140 (2021).

    Article  CAS  PubMed  Google Scholar 

  39. Zhang, F. et al. Predicting treatment response to neoadjuvant chemoradiotherapy in local advanced rectal cancer by biopsy digital pathology image features. Clin. Transl. Med. 10, e110 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  40. Honomichl, N. HER2 and trastuzumab treatment response H&E slides with tumor ROI annotations (HER2 tumor ROIs). The Cancer Imaging Archive https://doi.org/10.7937/E65C-AM96 (2022).

  41. He, B. et al. Integrating spatial gene expression and breast tumour morphology via deep learning. Nat. Biomed. Eng. 4, 827–834 (2020).

    Article  CAS  PubMed  Google Scholar 

  42. Wang, Y. et al. Predicting molecular phenotypes from histopathology images: a transcriptome-wide expression–morphology analysis in breast cancer. Cancer Res. 81, 5115–5126 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Monjo, T., Koido, M., Nagasawa, S., Suzuki, Y. & Kamatani, Y. Efficient prediction of a spatial transcriptomics profile better characterizes breast cancer tissue sections without costly experimentation. Sci. Rep. 12, 4133 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  44. Schmauch, B. et al. A deep learning model to predict RNA-Seq expression of tumours from whole slide images. Nat. Commun. 11, 3877 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Levy-Jurgenson, A., Tekpli, X., Kristensen, V. N. & Yakhini, Z. Spatial transcriptomics inferred from pathology whole-slide images links tumor heterogeneity to survival in breast and lung cancer. Sci. Rep. 10, 18802 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Alsaafin, A., Safarpoor, A., Sikaroudi, M., Hipp, J. D. & Tizhoosh, H. R. Learning to predict RNA sequence expressions from whole slide images with applications for search and classification. Commun. Biol. 6, 304 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Dinstag, G. et al. Clinically oriented prediction of patient response to targeted and immunotherapies from the tumor transcriptome. Med 4, 15–30 (2023).

    Article  CAS  PubMed  Google Scholar 

  48. Sammut, S.-J. et al. Multi-omic machine learning predictor of breast cancer therapy response. Nature 601, 623–629 (2022).

    Article  CAS  PubMed  Google Scholar 

  49. Zheng, Y. et al. Digital profiling of cancer transcriptomes from histology images with grouped vision attention. Preprint at bioRxiv https://doi.org/10.1101/2023.09.28.560068 (2023).

  50. Hanahan, D. & Weinberg, R. A. Hallmarks of cancer: the next generation. Cell 144, 646–674 (2011).

    Article  CAS  PubMed  Google Scholar 

  51. Iorio, F. et al. Pathway-based dissection of the genomic heterogeneity of cancer hallmarks’ acquisition with SLAPenrich. Sci. Rep. 8, 6713 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  52. Subramanian, A. et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proc. Natl Acad. Sci. USA 102, 15545–15550 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Richardsen, E. et al. Evaluation of the proliferation marker Ki-67 in a large prostatectomy cohort. PLoS ONE 12, e0186852 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  54. Whitfield, M. L., George, L. K., Grant, G. D. & Perou, C. M. Common markers of proliferation. Nat. Rev. Cancer 6, 99–106 (2006).

    Article  CAS  PubMed  Google Scholar 

  55. Liberzon, A. et al. The Molecular Signatures Database (MSigDB) hallmark gene set collection. Cell Syst. 1, 417–425 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Farahmand, S. et al. Deep learning trained on hematoxylin and eosin tumor region of interest predicts HER2 status and trastuzumab treatment response in HER2+ breast cancer. Mod. Pathol. 35, 44–51 (2022).

    Article  CAS  PubMed  Google Scholar 

  57. Strauss, J. et al. Bintrafusp alfa, a bifunctional fusion protein targeting TGF-β and PD-L1, in patients with human papillomavirus-associated malignancies. J. Immunother. Cancer 8, e001395 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  58. Kanopoulos, N., Vasanthavada, N. & Baker, R. L. Design of an image edge detection filter using the Sobel operator. IEEE J. Solid-State Circuits 23, 358–367 (1988).

    Article  Google Scholar 

  59. Macenko, M. et al. A method for normalizing histology slides for quantitative analysis. in 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro 1107–1110 (IEEE, 2009); https://doi.org/10.1109/ISBI.2009.5193250

  60. He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (IEEE, 2016); https://doi.org/10.1109/cvpr.2016.90

  61. Hoang, D.-T. et al. DeepPT: a deep learning model for predicting transcriptomics from histopathology images. Zenodo https://doi.org/10.5281/zenodo.11125591 (2024).

Download references

Acknowledgements

This work was partially supported by grant DP190103402 from the Australian Research Council (D.-T.H., E.A.S.) and by the Intramural Research Program of the National Institutes of Health (NIH), National Cancer Institute (NCI), Center for Cancer Research (CCR) (E.D.S., S.S., N.S., E.R.). This work used the supercomputational resources of the Australian National Computational Infrastructure (AUNCI) and the Australian National University Merit Allocation Scheme (ANUMAS). D.-T.H. would like to thank T. Bui for the helpful discussion.

Author information

Authors and Affiliations

Authors

Contributions

D.-T.H., S.S., E.A.S. and E.R. initialed the study. D.-T.H. developed the DeepPT framework and performed predictions of gene expression from H&E slides. G.D. performed pathway enrichment analysis, conducted the analysis of prognostic signatures and applied ENLIGHT to predict patient response from the predicted gene expression, with assistance from D.S.B.-Z. and E.E. L.C.H. assisted with data curation and gene expression preprocessing. D.-T.H. and E.D.S. analyzed the correlation between the predicted gene expression and the abundance of tumor-infiltrating lymphocytes. E.D.S. conducted the gene set enrichment analysis. K.C. assisted with image preprocessing. S.-J.S., C.H.D., C.S., T.P., A.R., W.L., J.S., S.B., C.A., J.R., T.G., S.W., A.G.S., S.R.P., C.C., J.L.G. and K.A. provided external and clinical datasets. N.S. and P.J. interpreted the results and provided feedback on the study. T.B., K.A., R.A., E.A.S. and E.R. supervised the study. D.-T.H., G.D., R.A. and E.R. wrote the paper with assistance and feedback from all the other coauthors.

Corresponding authors

Correspondence to Danh-Tai Hoang, Eric A. Stone or Eytan Ruppin.

Ethics declarations

Competing interests

D.-T.H., E.A.S., E.R., G.D., R.A. and T.B. are listed as inventors on a patent (application no. 63/349,829, United States, 2022) filed based on the methodology outlined in this study. G.D., D.S.B., E.E., T.B. and R.A. are employees of Pangea Biomed. E.R. is a cofounder of Medaware, Metabomed and Pangea Biomed (divested from the latter). E.R. serves as a non-paid scientific consultant to Pangea Biomed under a collaboration agreement between Pangea Biomed and the NCI. The other authors declare no competing interests.

Peer review

Peer review information

Nature Cancer thanks Ulysses Balis and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Model architecture in detail and training strategies.

(a) The feature compression subnetwork consists of an input layer of 2,048 neurons, a bottleneck of 512 neurons, and an output layer of 2,048 neurons. (b) The MLP regression subnetwork consists of an input layer of 512 neurons, a hidden layer of 512 neurons, and an output layer with the number of neurons reflecting the number of genes. (c) In the ensemble learning strategy (bagging), five models were trained independently with five internal training-validation splits; these five model predictions were averaged to make the final prediction. (d) In the model selection strategy, the ‘best’ model with the highest performance on the validation set was chosen to make predictions on the test set. Of note, DeepPT uses ensemble learning.

Extended Data Fig. 2 The distribution of correlations between the predicted and actual gene expression values across the cohort samples.

The violin plots depict the correlations between the predicted and measured expression values across the cohort samples obtained by HE2RNA (light pink) and DeepPT (light blue) for all genes (a), the top 1,000 genes (b), the top 2,000 genes (c), and the top 3,000 genes (d) with the highest correlations. The results presented in this figure were measured by the mean of 5 folds, as reported in44. Except for this figure, all other results presented in this study were measured across the entire test samples, consistent with the approach used in49. P-values were calculated using the one-sided Mann-Whitney U test. In violin plots, the central mark is the median. The number of patients in each cohort is shown in parentheses.

Source data

Extended Data Fig. 3 Difference between histopathological features extracted from TCGA-Breast tiles and TransNEO-Breast tiles.

UMAP visualization of 2,048 histopathological features that were extracted by using pre-trained ResNet50 CNN. 4,000 image tiles from each dataset were selected randomly to illustrate. Each point represents each feature vector of one image tile.

Extended Data Fig. 4 Comparisons of ENLIGHT-DeepPT with other methods.

(a) Performance of ENLIGHT-DeepPT (light blue bars) and the respective drug target(s) expression (gray bars). (b) Performance of ENLIGHT-DeepPT when using the same methodology described in47 (light blue bars) and a version of ENLIGHT-DeepPT that incorporates the target expression in the scoring method for antibodies (gray bars). (c) Performance of ENLIGHT-DeepPT when using the same methodology described in47 to generate genetic interaction networks that constitutes ENLIGHT’s predictive biomarkers (light blue bars) and a revised methodology (gray bars) where we restricted ENLIGHT’s biomarker to only include genes that showed high positive correlation (R > 0.4) between actual and DeepPT-predicted values among the respective TCGA cohort (that is, according to the cancer type of each of the five drug response datasets). Results are shown for each of the three datasets where antibody drugs were used and the aggregation of them. Odds Ratio (OR) for each dataset were obtained by using the same clinical decision threshold that has been previously established in47. The number of patients in each cohort is shown in parentheses.

Source data

Extended Data Fig. 5 Histograms of the number of tiles per slide by cohort.

The number of tiles in each slide image from TCGA and NCI-Brain datasets ranges from 100 to 8,000 (a-e), while the number of tiles in each TransNEO-Breast slide image is much smaller, ranging from 100 to 1,000 (f).

Source data

Extended Data Fig. 6 Histogram of median expression over slides.

The median expression over samples of each gene commonly varies from 10 to 100,000 for every dataset considered in this study (a to f).

Source data

Extended Data Fig. 7 The benefit of the autoencoder module.

(a-d) Difference between ResNet features and AutoEncoder features. Histograms of median and standard deviation of ResNet features (a-b) and AutoEncoder features (c-d). The TCGA-BRCA cohort was selected as an example. (e-f) Model performance on external validation datasets. The violin plots depict the distribution of Pearson correlations between the predicted and experimentally measured expression values across the cohort samples for the top 1,000 genes with the highest correlation. The bar charts indicate the number of genes exhibiting Pearson correlations between the predicted and measured expression values across the cohort samples that are above 0.4. The results are presented separately for each external validation set, TransNeo-Breast (e) (n = 160 patients) and NCI-Brain (f) (n = 226 patients). P-values were calculated using the one-sided Mann-Whitney U test. In violin plots, the central mark is the median.

Source data

Extended Data Fig. 8 The benefit of ensemble learning.

The violin plots depict the distribution of Pearson correlation for the top 1,000 genes, and the bar charts indicate the number of genes exhibiting Pearson correlations between the predicted and measured expression values across the cohort samples above 0.4. The results were obtained from model selection strategy (gray) and ensemble learning strategy (light blue). P-values were calculated using the one-sided Mann-Whitney U test. In violin plots, the central mark is the median. The number of patients in each cohort is shown in parentheses.

Source data

Supplementary information

Reporting Summary

Supplementary Tables

Supplementary Table 1. Number of slides, number of patients and number of selected tiles for each cohort. Supplementary Table 2. Details of the five datasets analyzed using ENLIGHT–DeepPT. Supplementary Table 3. AUC and accuracy for the five datasets analyzed using ENLIGHT–DeepPT.

Source data

Source Data Fig. 2

Statistical source data.

Source Data Fig. 3

Statistical source data.

Source Data Fig. 4

Statistical source data.

Source Data Fig. 5

Statistical source data.

Source Data Extended Data Fig. 2

Statistical source data.

Source Data Extended Data Fig. 4

Statistical source data.

Source Data Extended Data Fig. 5

Statistical source data.

Source Data Extended Data Fig. 6

Statistical source data.

Source Data Extended Data Fig. 7

Statistical source data.

Source Data Extended Data Fig. 8

Statistical source data.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoang, DT., Dinstag, G., Shulman, E.D. et al. A deep-learning framework to predict cancer treatment response from histopathology images through imputed transcriptomics. Nat Cancer (2024). https://doi.org/10.1038/s43018-024-00793-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s43018-024-00793-2

Search

Quick links

Nature Briefing: Cancer

Sign up for the Nature Briefing: Cancer newsletter — what matters in cancer research, free to your inbox weekly.

Get what matters in cancer research, free to your inbox weekly. Sign up for Nature Briefing: Cancer