The current standard of care for many patients with HER2-positive breast cancer is neoadjuvant chemotherapy in combination with anti-HER2 agents, based on HER2 amplification as detected by in situ hybridization (ISH) or protein immunohistochemistry (IHC). However, hematoxylin & eosin (H&E) tumor stains are more commonly available, and accurate prediction of HER2 status and anti-HER2 treatment response from H&E would reduce costs and increase the speed of treatment selection. Computational algorithms for H&E have been effective in predicting a variety of cancer features and clinical outcomes, including moderate success in predicting HER2 status. In this work, we present a novel convolutional neural network (CNN) approach able to predict HER2 status with increased accuracy over prior methods. We trained a CNN classifier on 188 H&E whole slide images (WSIs) manually annotated for tumor Regions of interest (ROIs) by our pathology team. Our classifier achieved an area under the curve (AUC) of 0.90 in cross-validation of slide-level HER2 status and 0.81 on an independent TCGA test set. Within slides, we observed strong agreement between pathologist annotated ROIs and blinded computational predictions of tumor regions / HER2 status. Moreover, we trained our classifier on pre-treatment samples from 187 HER2+ patients that subsequently received trastuzumab therapy. Our classifier achieved an AUC of 0.80 in a five-fold cross validation. Our work provides an H&E-based algorithm that can predict HER2 status and trastuzumab response in breast cancer at an accuracy that may benefit clinical evaluations.
This is a preview of subscription content
Subscribe to Journal
Get full journal access for 1 year
only $9.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Data are available upon request.
Codes are available upon request.
Woo, J. W. et al. The updated 2018 American Society of Clinical Oncology/College of American Pathologists guideline on human epidermal growth factor receptor 2 interpretation in breast cancer: comparison with previous guidelines and clinical significance of the proposed in situ hybridization groups. Hum. Pathol. 98, 10–21 (2020).
Press, M. F. et al. HER-2/neu gene amplification characterized by fluorescence in situ hybridization: poor prognosis in node-negative breast carcinomas. J. Clin. Oncol. 15, 2894–2904 (1997).
Tandon, A. K., Clark, G. M., Chamness, G. C., Ullrich, A. & McGuire, W. L. HER-2/neu oncogene protein and prognosis in breast cancer. J. Clin. Oncol. 7, 1120–1128 (1989).
Hayes, D. F. HER2 and breast cancer - a phenomenal success story. N. Engl. J. Med. 381, 1284–1286 (2019).
Andersson, M. et al. Phase III randomized study comparing docetaxel plus trastuzumab with vinorelbine plus trastuzumab as first-line therapy of metastatic or locally advanced human epidermal growth factor receptor 2–positive breast cancer: the HERNATA study. J. Clin. Orthod. 29, 264–271 (2011).
Pivot, X. et al. CEREBEL (EGF111438): a phase iii, randomized, open-label study of lapatinib plus capecitabine versus trastuzumab plus capecitabine in patients with human epidermal growth factor receptor 2–positive metastatic breast cancer. J. Clin. Orthod. 33, 1564–1573 (2015).
Valero, V. et al. Multicenter phase III} randomized trial comparing docetaxel and trastuzumab with docetaxel, carboplatin, and trastuzumab as first-line chemotherapy for patients with HER2-gene-amplified metastatic breast cancer (BCIRG 007 study): two highly active th. J. Clin. Oncol. 29, 149–156 (2011).
Slamon, D. J. et al. Use of chemotherapy plus a monoclonal antibody against HER2 for metastatic breast cancer that overexpresses HER2. N. Engl. J. Med. 344, 783–792 (2001).
Rugo, H. S. et al. Effect of a proposed trastuzumab biosimilar compared with trastuzumab on overall response rate in patients with ERBB2 (HER2)-positive metastatic breast cancer: a randomized clinical trial. JAMA 317, 37–47 (2017).
Urruticoechea, A. et al. Randomized phase iii trial of trastuzumab plus capecitabine with or without pertuzumab in patients with human epidermal growth factor receptor 2-positive metastatic breast cancer who experienced disease progression during or after trastuzumab-based Th. J. Clin. Oncol. 35, 3030–3038 (2017).
Gianni, L. et al. AVEREL: a randomized phase III Trial evaluating bevacizumab in combination with docetaxel and trastuzumab as first-line therapy for HER2-positive locally recurrent/metastatic breast cancer. J. Clin. Oncol. 31, 1719–1725 (2013).
Baselga, J. et al. Pertuzumab plus trastuzumab plus docetaxel for metastatic breast cancer. N. Engl. J. Med. 366, 109–119 (2012).
Gelmon, K. A. et al. Lapatinib or trastuzumab plus taxane therapy for human epidermal growth factor receptor 2–positive advanced breast cancer: final results of NCIC CTG MA.31. J. Clin. Orthod. 33, 1574–1583 (2015).
Blackwell, K. L. et al. Randomized study of Lapatinib alone or in combination with trastuzumab in women with ErbB2-positive, trastuzumab-refractory metastatic breast cancer. J. Clin. Oncol. 28, 1124–1130 (2010).
Advani, P. P., Crozier, J. A. & Perez, E. A. HER2 testing and its predictive utility in anti-HER2 breast cancer therapy. Biomark Med. 9, 35–49 (2015).
Woo, J. W. et al. The updated 2018 American Society of Clinical Oncology/College of American Pathologists guideline on human epidermal growth factor receptor 2 interpretation in breast cancer: comparison with previous guidelines and clinical significance of the proposed. Hum. Pathol. 98, 10–21 (2020).
Wolff, A. C. et al. American Society of Clinical Oncology/College of American Pathologists guideline recommendations for human epidermal growth factor receptor 2 testing in breast cancer. Arch. Pathol. Lab. Med. 131, 18–43 (2007).
Wolff, A. C. et al. Human epidermal growth factor receptor 2 testing in breast cancer: American Society of Clinical Oncology/College of American pathologists clinical practice guideline focused update. Arch. Pathol. Lab. Med. 142, 1364–1382 (2018).
Jiang, Y., Yang, M., Wang, S., Li, X. & Sun, Y. Emerging role of deep learning-based artificial intelligence in tumor pathology. Cancer Commun. 40, 154–166 (2020).
Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 84–90 (2017).
Szegedy C. et al. Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). (2015) p. 1–9.
He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2016). p. 770–778.
Noorbakhsh, J. et al. Deep learning-based cross-classifications reveal conserved spatial behaviors within tumor histological images. Nat. Commun. 11, 1–14 (2020).
Coudray, N. et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).
Wei, J. W. et al. Pathologist-level classification of histologic patterns on resected lung adenocarcinoma slides with deep neural networks. Sci. Rep. 9, 1–8 (2019).
Liu Y., et al. Detecting cancer metastases on gigapixel pathology images. Preprint at https://arxiv.org/abs/1703.02442 (2017).
Coudray, N. & Tsirigos, A. Deep learning links histology, molecular signatures and prognosis in cancer. Nat. Cancer 1, 755–757 (2020).
Yang, Y., Fang, Q. & Shen, H.-B. Predicting gene regulatory interactions based on spatial gene expression data and deep learning. PLOS Comput. Biol. 15, e1007324 (2019).
Yu K.-H. et al. Classifying non-small cell lung cancer histopathology types and transcriptomic subtypes using convolutional neural networks. J. Am. Med. Inform. Assoc. 27, 757–769 (2020).
Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl. Acad. Sci. USA 115, E2970–E2979 (2018).
Braman N. et al. Deep learning-based prediction of response to HER2-targeted neoadjuvant chemotherapy from pre-treatment dynamic breast MRI: A multi-institutional validation study. Preprint at https://arxiv.org/abs/2001.08570 (2020).
Naik, N. et al. Deep learning-enabled breast cancer hormonal receptor status determination from base-level H&E stains. Nat. Commun. 11, 1–8 (2020).
Rawat, R. R. et al. Deep learned tissue “fingerprints” classify breast cancers by ER/PR/Her2 status from H&E images. Sci. Rep. 10, 1–13 (2020).
Bychkov, D. et al. Deep learning identifies morphological features in breast cancer predictive of cancer ERBB2 status and trastuzumab treatment efficacy. Sci. Rep. 11, 4037 (2021).
Aperio ImageScope - pathology slide viewing software: Leica Biosystems. https://www.leicabiosystems.com/digital-pathology/manage/aperio-imagescope/.
Zanjani F. G., Zinger S., Bejnordi B. E., van der Laak J. A., de With P. H. N. Histopathology stain-color normalization using deep generative models. 1st Conference on Medical Imaging with Deep Learning (MIDL 2018), Amsterdam, The Netherlands (2018).
Szegedy C. et al. Going deeper with convolutions. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition. (IEEE Computer Society, 2015). p. 1–9.
Efron B. et al. An introduction to the bootstrap, 1st ed. (Taylor & Francis Group, 1994).
Pedregosa F. et al. others. Scikit-learn: Machine learning in Python. JMLR. 2825-2830 (2011).
Carpenter, J. E., Marsh, D., Mariasegaram, M. & Clarke, C. L. The Australian Breast Cancer Tissue Bank (ABCTB). Open J Bioresour. 1, e1 (2014).
Lundin, J., Lundin, M., Isola, J. & Joensuu, H. A web-based system for individualised survival estimation in breast cancer. BMJ 326, 29 (2003).
Joensuu, H. et al. Adjuvant docetaxel or vinorelbine with or without trastuzumab for breast cancer. N. Engl. J. Med. 354, 809–820 (2006).
Howard F. M. et al. The impact of digital histopathology batch effect on deep learning model accuracy and bias. Preprint at https://www.biorxiv.org/content/10.1101/2020.12.03.410845v1 (2020).
Wang Y. Y., Chang S. C., Wu L. W., Tsai S. T., Sun Y. N. A color-based approach for automated segmentation in tumor tissue classification. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology (2007) p. 6576–6579.
Alsubaie, N., Trahearn, N., Raza, S. E. A., Snead, D. & Rajpoot, N. M. Stain deconvolution using statistical analysis of multi-resolution stain colour representation. PLoS One 12, e0169875 (2017).
Sha L., Schonfeld D., Sethi A. Color normalization of histology slides using graph regularized sparse NMF. In: Gurcan M. N., Tomaszewski J. E., eds. Medical Imaging 2017: Digital Pathology (SPIE, 2017). p. 1014010.
J.H.C. acknowledges support from NCI grants R01CA230031 and P30CA034196. E.S.R. acknowledges support from Grant #IRG 17-172-57 from the American Cancer Society. D.L.R. acknowledges support from the Breast Cancer Research Foundation BCRF20-138. S.F. acknowledges support from UMass Boston College of Sciences and Mathematics (CSM) Dean’s Doctoral Research Fellowship.
The authors declare no competing interests.
All tissues and data were retrieved under permission from the Yale Human Investigation Committee protocol #9505008219 to D.L.R.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Farahmand, S., Fernandez, A.I., Ahmed, F.S. et al. Deep learning trained on hematoxylin and eosin tumor region of Interest predicts HER2 status and trastuzumab treatment response in HER2+ breast cancer. Mod Pathol (2021). https://doi.org/10.1038/s41379-021-00911-w