Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Deep learning trained on hematoxylin and eosin tumor region of Interest predicts HER2 status and trastuzumab treatment response in HER2+ breast cancer

Abstract

The current standard of care for many patients with HER2-positive breast cancer is neoadjuvant chemotherapy in combination with anti-HER2 agents, based on HER2 amplification as detected by in situ hybridization (ISH) or protein immunohistochemistry (IHC). However, hematoxylin & eosin (H&E) tumor stains are more commonly available, and accurate prediction of HER2 status and anti-HER2 treatment response from H&E would reduce costs and increase the speed of treatment selection. Computational algorithms for H&E have been effective in predicting a variety of cancer features and clinical outcomes, including moderate success in predicting HER2 status. In this work, we present a novel convolutional neural network (CNN) approach able to predict HER2 status with increased accuracy over prior methods. We trained a CNN classifier on 188 H&E whole slide images (WSIs) manually annotated for tumor Regions of interest (ROIs) by our pathology team. Our classifier achieved an area under the curve (AUC) of 0.90 in cross-validation of slide-level HER2 status and 0.81 on an independent TCGA test set. Within slides, we observed strong agreement between pathologist annotated ROIs and blinded computational predictions of tumor regions / HER2 status. Moreover, we trained our classifier on pre-treatment samples from 187 HER2+ patients that subsequently received trastuzumab therapy. Our classifier achieved an AUC of 0.80 in a five-fold cross validation. Our work provides an H&E-based algorithm that can predict HER2 status and trastuzumab response in breast cancer at an accuracy that may benefit clinical evaluations.

This is a preview of subscription content

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Datasets and study design for HER2 status and Trastuzumab response classification.
Fig. 2: HER2 status classification using unannotated slides.
Fig. 3: HER2 tumor status classification using annotated slides.
Fig. 4: Representative H&E slides from TCGA test set and their predicted heatmaps.
Fig. 5: Trastuzumab response prediction.

Data availability

Data are available upon request.

Code availability

Codes are available upon request.

References

  1. 1.

    Woo, J. W. et al. The updated 2018 American Society of Clinical Oncology/College of American Pathologists guideline on human epidermal growth factor receptor 2 interpretation in breast cancer: comparison with previous guidelines and clinical significance of the proposed in situ hybridization groups. Hum. Pathol. 98, 10–21 (2020).

    CAS  PubMed  Article  Google Scholar 

  2. 2.

    Press, M. F. et al. HER-2/neu gene amplification characterized by fluorescence in situ hybridization: poor prognosis in node-negative breast carcinomas. J. Clin. Oncol. 15, 2894–2904 (1997).

    CAS  PubMed  Article  Google Scholar 

  3. 3.

    Tandon, A. K., Clark, G. M., Chamness, G. C., Ullrich, A. & McGuire, W. L. HER-2/neu oncogene protein and prognosis in breast cancer. J. Clin. Oncol. 7, 1120–1128 (1989).

    CAS  PubMed  Article  Google Scholar 

  4. 4.

    Hayes, D. F. HER2 and breast cancer - a phenomenal success story. N. Engl. J. Med. 381, 1284–1286 (2019).

    PubMed  Article  Google Scholar 

  5. 5.

    Andersson, M. et al. Phase III randomized study comparing docetaxel plus trastuzumab with vinorelbine plus trastuzumab as first-line therapy of metastatic or locally advanced human epidermal growth factor receptor 2–positive breast cancer: the HERNATA study. J. Clin. Orthod. 29, 264–271 (2011).

    CAS  Google Scholar 

  6. 6.

    Pivot, X. et al. CEREBEL (EGF111438): a phase iii, randomized, open-label study of lapatinib plus capecitabine versus trastuzumab plus capecitabine in patients with human epidermal growth factor receptor 2–positive metastatic breast cancer. J. Clin. Orthod. 33, 1564–1573 (2015).

    CAS  Google Scholar 

  7. 7.

    Valero, V. et al. Multicenter phase III} randomized trial comparing docetaxel and trastuzumab with docetaxel, carboplatin, and trastuzumab as first-line chemotherapy for patients with HER2-gene-amplified metastatic breast cancer (BCIRG 007 study): two highly active th. J. Clin. Oncol. 29, 149–156 (2011).

    CAS  PubMed  Article  Google Scholar 

  8. 8.

    Slamon, D. J. et al. Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2. N. Engl. J. Med. 344, 783–792 (2001).

    CAS  PubMed  Article  Google Scholar 

  9. 9.

    Rugo, H. S. et al. Effect of a proposed trastuzumab biosimilar compared with trastuzumab on overall response rate in patients with ERBB2 (HER2)-positive metastatic breast cancer: a randomized clinical trial. JAMA 317, 37–47 (2017).

    CAS  PubMed  Article  Google Scholar 

  10. 10.

    Urruticoechea, A. et al. Randomized phase iii trial of trastuzumab plus capecitabine with or without pertuzumab in patients with human epidermal growth factor receptor 2-positive metastatic breast cancer who experienced disease progression during or after trastuzumab-based Th. J. Clin. Oncol. 35, 3030–3038 (2017).

    CAS  PubMed  Article  Google Scholar 

  11. 11.

    Gianni, L. et al. AVEREL: a randomized phase III Trial evaluating bevacizumab in combination with docetaxel and trastuzumab as first-line therapy for HER2-positive locally recurrent/metastatic breast cancer. J. Clin. Oncol. 31, 1719–1725 (2013).

    CAS  PubMed  Article  Google Scholar 

  12. 12.

    Baselga, J. et al. Pertuzumab plus trastuzumab plus docetaxel for metastatic breast cancer. N. Engl. J. Med. 366, 109–119 (2012).

    CAS  PubMed  Article  Google Scholar 

  13. 13.

    Gelmon, K. A. et al. Lapatinib or trastuzumab plus taxane therapy for human epidermal growth factor receptor 2–positive advanced breast cancer: final results of NCIC CTG MA.31. J. Clin. Orthod. 33, 1574–1583 (2015).

    CAS  Google Scholar 

  14. 14.

    Blackwell, K. L. et al. Randomized study of Lapatinib alone or in combination with trastuzumab in women with ErbB2-positive, trastuzumab-refractory metastatic breast cancer. J. Clin. Oncol. 28, 1124–1130 (2010).

    CAS  PubMed  Article  Google Scholar 

  15. 15.

    Advani, P. P., Crozier, J. A. & Perez, E. A. HER2 testing and its predictive utility in anti-HER2 breast cancer therapy. Biomark Med. 9, 35–49 (2015).

    CAS  PubMed  Article  Google Scholar 

  16. 16.

    Woo, J. W. et al. The updated 2018 American Society of Clinical Oncology/College of American Pathologists guideline on human epidermal growth factor receptor 2 interpretation in breast cancer: comparison with previous guidelines and clinical significance of the proposed. Hum. Pathol. 98, 10–21 (2020).

    CAS  PubMed  Article  Google Scholar 

  17. 17.

    Wolff, A. C. et al. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer. Arch. Pathol. Lab. Med. 131, 18–43 (2007).

    CAS  PubMed  Article  Google Scholar 

  18. 18.

    Wolff, A. C. et al. Human epidermal growth factor receptor 2 testing in breast cancer: American Society of Clinical Oncology/College of American pathologists clinical practice guideline focused update. Arch. Pathol. Lab. Med. 142, 1364–1382 (2018).

    PubMed  Article  Google Scholar 

  19. 19.

    Jiang, Y., Yang, M., Wang, S., Li, X. & Sun, Y. Emerging role of deep learning-based artificial intelligence in tumor pathology. Cancer Commun. 40, 154–166 (2020).

    Article  Google Scholar 

  20. 20.

    Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017).

    Article  Google Scholar 

  21. 21.

    Szegedy C. et al. Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). (2015) p. 1–9.

  22. 22.

    He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016). p. 770–778.

  23. 23.

    Noorbakhsh, J. et al. Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat. Commun. 11, 1–14 (2020).

    Article  Google Scholar 

  24. 24.

    Coudray, N. et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).

    CAS  PubMed  Article  Google Scholar 

  25. 25.

    Wei, J. W. et al. Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks. Sci. Rep. 9, 1–8 (2019).

    Google Scholar 

  26. 26.

    Liu Y., et al. Detecting cancer metastases on gigapixel pathology images. Preprint at https://arxiv.org/abs/1703.02442 (2017).

  27. 27.

    Coudray, N. & Tsirigos, A. Deep learning links histology, molecular signatures and prognosis in cancer. Nat. Cancer 1, 755–757 (2020).

    Article  Google Scholar 

  28. 28.

    Yang, Y., Fang, Q. & Shen, H.-B. Predicting gene regulatory interactions based on spatial gene expression data and deep learning. PLOS Comput. Biol. 15, e1007324 (2019).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  29. 29.

    Yu K.-H. et al. Classifying non-small cell lung cancer histopathology types and transcriptomic subtypes using convolutional neural networks. J. Am. Med. Inform. Assoc. 27, 757–769 (2020).

  30. 30.

    Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl. Acad. Sci. USA 115, E2970–E2979 (2018).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  31. 31.

    Braman N. et al. Deep learning-based prediction of response to HER2-targeted neoadjuvant chemotherapy from pre-treatment dynamic breast MRI: A multi-institutional validation study. Preprint at https://arxiv.org/abs/2001.08570 (2020).

  32. 32.

    Naik, N. et al. Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains. Nat. Commun. 11, 1–8 (2020).

    Article  Google Scholar 

  33. 33.

    Rawat, R. R. et al. Deep learned tissue “fingerprints” classify breast cancers by ER/PR/Her2 status from H&E images. Sci. Rep. 10, 1–13 (2020).

    Article  Google Scholar 

  34. 34.

    Bychkov, D. et al. Deep learning identifies morphological features in breast cancer predictive of cancer ERBB2 status and trastuzumab treatment efficacy. Sci. Rep. 11, 4037 (2021).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  35. 35.

    Aperio ImageScope - pathology slide viewing software: Leica Biosystems. https://www.leicabiosystems.com/digital-pathology/manage/aperio-imagescope/.

  36. 36.

    Zanjani F. G., Zinger S., Bejnordi B. E., van der Laak J. A., de With P. H. N. Histopathology stain-color normalization using deep generative models. 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherlands (2018).

  37. 37.

    Szegedy C. et al. Going deeper with convolutions. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition. (IEEE Computer Society, 2015). p. 1–9.

  38. 38.

    Efron B. et al. An introduction to the bootstrap, 1st ed. (Taylor & Francis Group, 1994).

  39. 39.

    Pedregosa F. et al. others. Scikit-learn: Machine learning in Python. JMLR. 2825-2830 (2011).

  40. 40.

    Carpenter, J. E., Marsh, D., Mariasegaram, M. & Clarke, C. L. The Australian Breast Cancer Tissue Bank (ABCTB). Open J Bioresour. 1, e1 (2014).

    Article  Google Scholar 

  41. 41.

    Lundin, J., Lundin, M., Isola, J. & Joensuu, H. A web-based system for individualised survival estimation in breast cancer. BMJ 326, 29 (2003).

    PubMed  PubMed Central  Article  Google Scholar 

  42. 42.

    Joensuu, H. et al. Adjuvant docetaxel or vinorelbine with or without trastuzumab for breast cancer. N. Engl. J. Med. 354, 809–820 (2006).

    CAS  PubMed  Article  Google Scholar 

  43. 43.

    Howard F. M. et al. The impact of digital histopathology batch effect on deep learning model accuracy and bias. Preprint at https://www.biorxiv.org/content/10.1101/2020.12.03.410845v1 (2020).

  44. 44.

    Wang Y. Y., Chang S. C., Wu L. W., Tsai S. T., Sun Y. N. A color-based approach for automated segmentation in tumor tissue classification. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology (2007) p. 6576–6579.

  45. 45.

    Alsubaie, N., Trahearn, N., Raza, S. E. A., Snead, D. & Rajpoot, N. M. Stain deconvolution using statistical analysis of multi-resolution stain colour representation. PLoS One 12, e0169875 (2017).

    PubMed  PubMed Central  Article  Google Scholar 

  46. 46.

    Sha L., Schonfeld D., Sethi A. Color normalization of histology slides using graph regularized sparse NMF. In: Gurcan M. N., Tomaszewski J. E., eds. Medical Imaging 2017: Digital Pathology (SPIE, 2017). p. 1014010.

Download references

Funding

J.H.C. acknowledges support from NCI grants R01CA230031 and P30CA034196. E.S.R. acknowledges support from Grant #IRG 17-172-57 from the American Cancer Society. D.L.R. acknowledges support from the Breast Cancer Research Foundation BCRF20-138. S.F. acknowledges support from UMass Boston College of Sciences and Mathematics (CSM) Dean’s Doctoral Research Fellowship.

Author information

Affiliations

Authors

Contributions

S.F. developed the classifiers, analyzed the Yale and TCGA data, and drafted the paper. A.F. performed quality control and analyzed the Yale data. F.S.A. performed quality control and analyzed the Yale data. D.R. provided pathological evaluations and oversaw the project. J.H.C. oversaw the project and drafted the paper. E.S.R. generated the data, annotated the data, provided pathological evaluations and oversaw the project. K.Z. led the project and finalized the paper.

Corresponding authors

Correspondence to Jeffrey H. Chuang, Emily Reisenbichler or Kourosh Zarringhalam.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethics approval

All tissues and data were retrieved under permission from the Yale Human Investigation Committee protocol #9505008219 to D.L.R.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Farahmand, S., Fernandez, A.I., Ahmed, F.S. et al. Deep learning trained on hematoxylin and eosin tumor region of Interest predicts HER2 status and trastuzumab treatment response in HER2+ breast cancer. Mod Pathol (2021). https://doi.org/10.1038/s41379-021-00911-w

Download citation

Search

Quick links