Deep learning (DL) is a powerful methodology for the recognition and classification of tissue structures in digital pathology. Its performance in prostate cancer pathology is still under intensive investigation. Here we develop DL-based models for the detection of prostate cancer tissue in whole-slide images based on a large high-quality annotated training dataset and a modern state-of-the-art convolutional network architecture (NASNetLarge). The overall accuracy of our model for tumour detection in two validation cohorts is comparable to that of pathologists and reaches 97.3% in a native version and more than 98% using the suggested DL-based augmentation strategies. As a second step, we suggest a new biologically meaningful DL-based algorithm for Gleason grading of prostatic adenocarcinomas with high, human-level performance in prognostic stratification of patients when tested in several well-characterized validation cohorts. Furthermore, we determine the optimal minimal tumour size (real size of approximately 560 × 560 µm) for robust Gleason grading representative of the whole tumour focus. Our approach is realized in the unified digital pathology pipeline, which delivers all the relevant tumour metrics for a pathology report.
Subscribe to Journal
Get full journal access for 1 year
only $8.67 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
The whole-slide images used for algorithm development (training dataset) are publicly available through GDC Data Portal of the National Cancer Institute (The Cancer Genome Atlas Project; http://portal.gdc.cancer.gov). The validation datasets (image patches of tumour and benign classes) generated and analysed during the current study are available for academical use only from public repository https://zenodo.org/deposit/3825933. Any usage for publications should be consented by corresponding authors. All other data may be obtained upon request to the authors.
The source code used in this study is available at https://github.com/gagarin37/deep_learning_pca.
Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V. & Madabhushi, A. Artificial intelligence in digital pathology—new tools for diagnosis and precision oncology. Nat. Rev. Clin. Oncol. 16, 703–715 (2019).
Coudray, N. et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).
Kather, J. N. et al. Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 16, e1002730 (2019).
Kather, J. N. et al. Deep learning can predict microsatellite instability directly from histology in gastrointestinal cancer. Nat. Med. 25, 1054–1056 (2019).
Bychkov, D. et al. Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci. Rep. 8, 3395 (2018).
Couture, H. D. et al. Image analysis with deep learning to predict breast cancer grade, ER status, histologic subtype, and intrinsic subtype. npj Breast Cancer 4, 30 (2018).
Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019).
Ehteshami Bejnordi, B. et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318, 2199–2210 (2017).
Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).
Nagpal, K. et al. Development and validation of a deep learning algorithm for improving Gleason scoring of prostate cancer. npj Digit. Med. 2, 48 (2019).
Bulten, W. et al. Epithelium segmentation using deep learning in H&E-stained prostate specimens with immunohistochemistry as reference standard. Sci. Rep. 9, 864 (2019).
Chen, C. M., Huang, Y. S., Fang, P. W., Liang, C. W. & Chang, R. F. A computer-aided diagnosis system for differentiation and delineation of malignant regions on whole-slide prostate histopathology image using spatial statistics and multidimensional DenseNet. Med. Phys. 47, 1021–1033 (2019).
Poojitha, U. P. & Lal Sharma, S. Hybrid unified deep learning network for highly precise Gleason grading of prostate cancer. In Proc. Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 899–903 (IEEE, 2019).
Li, J. et al. A multi-scale U-Net for semantic segmentation of histological images from radical prostatectomies. AMIA Annu. Symp. Proc. 2017, 1140–1148 (2017).
Li, J. et al. An EM-based semi-supervised deep learning approach for semantic segmentation of histopathological images from radical prostatectomies. Comput. Med. Imaging Graph. 69, 125–133 (2018).
Li, W. et al. Path R-CNN for prostate cancer diagnosis and gleason grading of histological images. IEEE Trans. Med. Imaging 38, 945–954 (2019).
Lucas, M. et al. Deep learning for automatic Gleason pattern classification for grade group determination of prostate biopsies. Virchows Arch. 475, 77–83 (2019).
Ren, J. et al. Recurrence analysis on prostate cancer patients with Gleason score 7 using integrated histopathology whole-slide images and genomic data through deep neural networks. J. Med. Imaging 5, 047501 (2018).
Arvaniti, E. et al. Automated Gleason grading of prostate cancer tissue microarrays via deep learning. Sci. Rep. 8, 12054 (2018).
Singh, M. et al. Gland segmentation in prostate histopathological images. J. Med. Imaging 4, 027501 (2017).
Jiménez del Toro, O. et al. Convolutional neural networks for an automatic classification of prostate tissue slides with high-grade Gleason score. Proc. SPIE 10140, 101400O (2017).
Bulten, W. et al. Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study. Lancet Oncol. 21, 233–241 (2020).
Ström, P. et al. Artificial intelligence for diagnosis and grading of prostate cancer in biopsies: a population-based, diagnostic study. Lancet Oncol. 21, 222–232 (2020).
Komura, D. & Ishikawa, S. Machine learning methods for histopathological image analysis. Comput. Struct. Biotechnol. J. 16, 34–42 (2018).
Gertych, A. et al. Convolutional neural networks can accurately distinguish four histologic growth patterns of lung adenocarcinoma in digital slides. Sci. Rep. 9, 1483 (2019).
Nir, G. et al. Automatic grading of prostate cancer in digitized histopathology images: learning from multiple experts. Med. Image Anal. 50, 167–180 (2018).
Nir, G. et al. Comparison of artificial intelligence techniques to evaluate performance of a classifier for automatic grading of prostate cancer from digitized histopathologic images. JAMA Netw. Open 2, e190442 (2019).
Li, J. et al. An attention-based multi-resolution model for prostate whole slide imageclassification and localization. Preprint at https://arxiv.org/abs/1905.13208 (2019).
Gertych, A. et al. Machine learning approaches to analyze histological images of tissues from radical prostatectomies. Comput. Med. Imaging Graph. 46 (Pt 2), 197–208 (2015).
Litjens, G. et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6, 26286 (2016).
Zoph, B., Vasudevan, V., Shlens, J. & Le, Q. V. Learning transferable architectures for scalable image recognition. Preprinit at https://arxiv.org/abs/1707.07012 (2017).
Tolkach, Y., Thomann, S. & Kristiansen, G. Three-dimensional reconstruction of prostate cancer architecture with serial immunohistochemical sections: hallmarks of tumour growth, tumour compartmentalisation, and implications for grading and heterogeneity. Histopathology 72, 1051–1059 (2018).
van Royen, M. E. et al. Three-dimensional microscopic analysis of clinical prostate specimens. Histopathology 69, 985–992 (2016).
Macenko, M. et al. A method for normalizing histology slides for quantitative analysis. In IEEE International Symposium on Biomedical Imaging: From Nano to Macro 1107–1110 (IEEE, 2009).
Tolkach, Y. & Kristiansen, G. Cribriform and glomeruloid acinar adenocarcinoma of the prostate—an intratumoural intraductal carcinoma? Histopathology 74, 804–808 (2019).
Humphrey, P. A., Moch, H., Cubilla, A. L., Ulbright, T. M. & Reuter, V. E. The 2016 WHO classification of tumours of the urinary System and male genital organs—part B: prostate and bladder tumours. Eur. Urol. 70, 106–119 (2016).
The GPU card for this study was donated by the NVIDIA corporation (a GPU academic grant programme). No funding was received for this study.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Study design and setup of single study steps, principles of patch generation (including number of patches generated in training and validation datasets in thousands of unique non-intersecting patches, k), approaches to training and validation. For details to certain steps see Methods.
Three classes were used for training (benign glandular, benign non-glandular, tumour) with stain normalization. NASNetLarge architecture was used with addition of three new layers for classification (flatten, fully connected, and classification layers). Probability of being benign could be summarized from output probabilities of two benign classes. Training principles and parameters are presented in boxes.
a. Prostate cancer (PCA) architecture is not just a rough mix of glands with Gleason patterns (GP) 3, 4, and 5. In three dimensions, carcinoma is a tree, where single GPs do not exist, being a continuum. b. Our Gleason grading algorithm is based on the understanding of GPs being a continuum. During processing, a tumour region is being cut into patches that are then separately analyzed by convnet (trained on pure GPs, see Materials and Methods). Probabilities of different classes (GP3, GP4, GP5) are counted for every single patch and summarized for the whole tumour region and not used for nomination of every patch to a discrete GP (as in classical paradigm).
About this article
Cite this article
Tolkach, Y., Dohmgörgen, T., Toma, M. et al. High-accuracy prostate cancer pathology using deep learning. Nat Mach Intell 2, 411–418 (2020). https://doi.org/10.1038/s42256-020-0200-7