Article | Published:

Using deep convolutional neural networks to identify and classify tumor-associated stroma in diagnostic breast biopsies


The breast stromal microenvironment is a pivotal factor in breast cancer development, growth and metastases. Although pathologists often detect morphologic changes in stroma by light microscopy, visual classification of such changes is subjective and non-quantitative, limiting its diagnostic utility. To gain insights into stromal changes associated with breast cancer, we applied automated machine learning techniques to digital images of 2387 hematoxylin and eosin stained tissue sections of benign and malignant image-guided breast biopsies performed to investigate mammographic abnormalities among 882 patients, ages 40–65 years, that were enrolled in the Breast Radiology Evaluation and Study of Tissues (BREAST) Stamp Project. Using deep convolutional neural networks, we trained an algorithm to discriminate between stroma surrounding invasive cancer and stroma from benign biopsies. In test sets (928 whole-slide images from 330 patients), this algorithm could distinguish biopsies diagnosed as invasive cancer from benign biopsies solely based on the stromal characteristics (area under the receiver operator characteristics curve = 0.962). Furthermore, without being trained specifically using ductal carcinoma in situ as an outcome, the algorithm detected tumor-associated stroma in greater amounts and at larger distances from grade 3 versus grade 1 ductal carcinoma in situ. Collectively, these results suggest that algorithms based on deep convolutional neural networks that evaluate only stroma may prove useful to classify breast biopsies and aid in understanding and evaluating the biology of breast lesions.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.


  1. 1.

    Dupont WD, Parl FF, Hartmann WH, et al. Breast cancer risk associated with proliferative breast disease and atypical hyperplasia. Cancer. 1993;71:1258–65.

  2. 2.

    Quail DF, Joyce JA. Microenvironmental regulation of tumor progression and metastasis. Nat Med. 2013;19:1423–37.

  3. 3.

    Egeblad M, Nakasone ES, Werb Z. Tumors as organs: complex tissues that interface with the entire organism. Dev Cell. 2010;18:884–901.

  4. 4.

    Joyce JA, Pollard JW. Microenvironmental regulation of metastasis. Nat Rev Cancer. 2009;9:239–52.

  5. 5.

    Provenzano PP, Eliceiri KW, Campbell JM, et al. Collagen reorganization at the tumor-stromal interface facilitates local invasion. BMC Med. 2006;4:38.

  6. 6.

    Orimo A, Gupta PB, Sgroi DC, et al. Stromal fibroblasts present in invasive human breast carcinomas promote tumor growth and angiogenesis through elevated SDF-1/CXCL12 secretion. Cell. 2005;121:335–48.

  7. 7.

    Rønnov-Jessen L, Petersen OW, Koteliansky VE, Bissell MJ. The origin of the myofibroblasts in breast cancer. Recapitulation of tumor environment in culture unravels diversity and implicates converted fibroblasts and recruited smooth muscle cells. J Clin Invest. 1995;95:859–73.

  8. 8.

    Tlsty TD, Hein PW. Know thy neighbor: Stromal cells can contribute oncogenic signals. Curr Opin Genet Dev. 2001;11:54–9.

  9. 9.

    Ma X-J, Dahiya S, Richardson E, Erlander M, Sgroi DC. Gene expression profiling of the tumor microenvironment during breast cancer progression. Breast Cancer Res. 2009;11:R7.

  10. 10.

    Lopez-Garcia MA, Geyer FC, Lacroix-Triki M, Marchió C, Reis-Filho JS. Breast cancer precursors revisited: molecular features and progression pathways. Histopathology. 2010;57:171–92.

  11. 11.

    Salgado R, Denkert C, Demaria S, et al. The evaluation of tumor-infiltrating lymphocytes (TILs) in breast cancer: recommendations by an International TILs Working Group 2014. Ann Oncol. 2015;26:259–71.

  12. 12.

    Beck AH, Sangoi AR, Leung S, et al. Systematic Analysis of breast cancer morphology uncovers stromal features associated with survival. Sci Transl Med. 2011;3:108ra13.

  13. 13.

    LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521:436–44.

  14. 14.

    Litjens G, Kooi T, Ehteshami Bejnordi B, et al. A survey on deep learning in medical image analysis. Med Image Anal. 2017;42:60–88.

  15. 15.

    Ehteshami Bejnordi B, Veta M, Johannes van Diest P, et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA. 2017;318:2199–210.

  16. 16.

    Gierach GL, Geller BM, Shepherd JA, et al. Comparison of mammographic density assessed as volumes and areas among women undergoing diagnostic image-guided breast biopsy. Cancer Epidemiol Biomark Prev. 2014;23:2338–48.

  17. 17.

    Gierach GL, Patel DA, Pfeiffer RM, et al. Relationship of terminal duct lobular unit involution of the breast with area and volume mammographic densities. Cancer Prev Res. 2016;9:149–58.

  18. 18.

    Pinder SE. Ductal carcinoma in situ (DCIS): Pathological features, differential diagnosis, prognostic factors and specimen evaluation. Mod Pathol. 2010;23:S8–S13.

  19. 19.

    Ehteshami Bejnordi B, Linz J, Glass B, et al. Deep learning-based assessment of tumor-associated stroma for diagnosing breast cancer in histopathology images. Proceedings of the 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI), IEEE: Melbourne, VIC; 2017;929–32.

  20. 20.

    Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014:14091556.

  21. 21.

    Russakovsky O, Deng J, Su H, et al. ImageNet Large scale visual recognition challenge. Int J Comput Vision. 2015;115:211–52.

  22. 22.

    Breiman L. Random forests. Mach Learn. 2001;45:5–32.

  23. 23.

    Okabe A, Boots B, Sugihara K. Spatial tessellations: concepts and applications of Voronoi diagrams. John Wiley & Sons, Inc., Chichester, New York, Brisbane, Toronto and Singapore; 1992. p. 532.

  24. 24.

    Glasziou P. The statistical evaluation of medical tests for classification and prediction. Stat Med. 2005;24:2582.

  25. 25.

    Efron B. Bootstrap methods: another look at the Jackknife. Ann Stat. 1979;7:1–26.

  26. 26.

    Robin X, Turck N, Hainard A, et al. pROC: An open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinforma. 2011;12:77.

  27. 27.

    Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148:839–43.

  28. 28.

    Naik S, Doyle S, Agner S, et al. Automated gland and nuclei segmentation for grading of prostate and breast cancer histopathology. Proceedings of the 2008 IEEE 5th International Symposium on Biomedical Imaging (ISBI), IEEE: Paris, 2008;284–7.

  29. 29.

    Ehteshami Bejnordi B, Litjens G, Hermsen M, Karssemeijer N, A. W. M. van der Laak J. A multi-scale superpixel classification approach to the detection of regions of interest in whole slide histopathology images. Proceedings of the SPIE Medical Imaging 2015: Digital Pathology, SPIE Orlando, Florida; 2015;94200H.

  30. 30.

    Ehteshami Bejnordi B, Balkenhol M, Litjens G, et al. Automated detection of DCIS in whole-slide H&E stained breast histopathology images. IEEE Trans Med Imaging. 2016;35:2141–50.

  31. 31.

    Dundar MM, Badve S, Bilgin G, et al. Computerized classification of intraductal breast lesions using histopathological images. IEEE Trans Biomed Eng. 2011;58:1977–84.

  32. 32.

    Dong F, Irshad H, Oh E-Y, et al. Computational pathology to discriminate benign from malignant intraductal proliferations of the breast. PLOS ONE. 2014;9:e114885.

  33. 33.

    Ehteshami Bejnordi B, Zuidhof G, Balkenhol M, et al. Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images. J Med Imaging. 2017;4:044504.

  34. 34.

    Solin LJ, Kurtz J, Fourquet A, et al. Fifteen-year results of breast-conserving surgery and definitive breast irradiation for the treatment of ductal carcinoma in situ of the breast. J Clin Oncol. 1996;14:754–63.

  35. 35.

    Silverstein MJ, Lagios MD, Groshen S, et al. The influence of margin width on local control of ductal carcinoma in situ of the breast. N Engl J Med. 1999;340:1455–61.

  36. 36.

    Benson JR, Jatoi I, Toi M. Treatment of low-risk ductal carcinoma in situ: is nothing better than something? Lancet Oncol. 2016;17:e442–e51.

  37. 37.

    Wallis MG, Clements K, Kearins O, et al. The effect of DCIS grade on rate, type and time to recurrence after 15 years of follow-up of screen-detected DCIS. Br J Cancer. 2012;106:1611–7.

  38. 38.

    Lagios MD, Margolin FR, Westdahl PR, Rose MR. Mammographically detected duct carcinoma in situ. Frequency of local recurrence following tylectomy and prognostic effect of nuclear grade on local recurrence. Cancer. 1989;63:618–24.

  39. 39.

    Bagnall MJC, Evans AJ, Wilson ARM, et al. Predicting invasion in mammographically detected microcalcification. Clin Radiol. 2001;56:828–32.

  40. 40.

    Silver SA, Tavassoli FA. Mammary ductal carcinoma in situ with microinvasion. Cancer. 1998;82:2382–90.

  41. 41.

    Bombonati A, Sgroi DC. The molecular pathology of breast cancer progression. J Pathol. 2011;223:307–17.

  42. 42.

    Francis A, Bartlett J, Billingham L, et al. The LORIS trial: a multicentre, randomized phase III trial of standard surgery versus active monitoring in women with newly diagnosed low risk ductal carcinoma in situ. Cancer Res. 2013;73:OT2-3-01.

  43. 43.

    Elshof LE, Tryfonidis K, Slaets L, et al. Feasibility of a prospective, randomised, open-label, international multicentre, phase III, non-inferiority trial to assess the safety of active surveillance for low risk ductal carcinoma in situ—The LORD study. Eur J Cancer. 2015;51:1497–510.

  44. 44.

    Youngwirth L, Boughey J, Hwang E. Surgery versus monitoring and endocrine therapy for low-risk DCIS: The COMET Trial. Bull Am Coll Surg. 2017;102:62–3.

  45. 45.

    Felix AS, Lenz P, Pfeiffer RM, et al. Relationships between mammographic density, tissue microvessel density, and breast biopsy diagnosis. Breast Cancer Res. 2016;18:88.

Download references


This project was funded in part by the Intramural Research Program of the National Cancer Institute, National Institutes of Health, Bethesda, Maryland and a competitive award to MES and LAB funded through the sale of breast cancer awareness postage stamps. The authors wish to acknowledge the financial support by the European Union FP7 funded VPHPRISM project under the grant agreement n601040. Pamela Vacek and Donald Weaver are currently funded under a U01 exploring stromal contributions to tumor progression (U01 CA196383).

Author information

Conflict of interest

The authors declare that they have no conflict of interest.

Correspondence to Jeroen A. W. M. van der Laak.

Electronic supplementary material

  1. Supplementary material

  2. Supplementary Table 1

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7