Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

A new era: artificial intelligence and machine learning in prostate cancer


Artificial intelligence (AI) — the ability of a machine to perform cognitive tasks to achieve a particular goal based on provided data — is revolutionizing and reshaping our health-care systems. The current availability of ever-increasing computational power, highly developed pattern recognition algorithms and advanced image processing software working at very high speeds has led to the emergence of computer-based systems that are trained to perform complex tasks in bioinformatics, medical imaging and medical robotics. Accessibility to ‘big data’ enables the ‘cognitive’ computer to scan billions of bits of unstructured information, extract the relevant information and recognize complex patterns with increasing confidence. Computer-based decision-support systems based on machine learning (ML) have the potential to revolutionize medicine by performing complex tasks that are currently assigned to specialists to improve diagnostic accuracy, increase efficiency of throughputs, improve clinical workflow, decrease human resource costs and improve treatment choices. These characteristics could be especially helpful in the management of prostate cancer, with growing applications in diagnostic imaging, surgical interventions, skills training and assessment, digital pathology and genomics. Medicine must adapt to this changing world, and urologists, oncologists, radiologists and pathologists, as high-volume users of imaging and pathology, need to understand this burgeoning science and acknowledge that the development of highly accurate AI-based decision-support applications of ML will require collaboration between data scientists, computer researchers and engineers.

Key points

  • Applications of machine learning (ML) to prostate cancer care are rapidly growing owing to the many technological platforms involved in its diagnosis, prognosis and treatment.

  • In diagnostic imaging, ML is applied to perform low-level image analysis tasks such as prostate segmentation and fusion of different modalities (for example MRI, CT and ultrasonography) and high-level inference and prediction tasks such as prostate cancer detection and characterization.

  • ML algorithms are able to enhance prostate cancer treatment by augmenting the surgeon’s display with information such as cancer localization during robotic procedures and other image-guided interventions and could be used towards autonomous manipulation of tools for assistance in the operating room.

  • Computer-assisted diagnosis of prostate cancer in histopathological slides could be achieved by ML in order to optimize accuracy, reproducibility and throughput and to further enhance health-care delivery by enabling the use of customized precision-care pathways.

  • ML methods are used to identify genes or groups of genes for which expression specificity to predict outcomes of prostate cancer is high and could be used for screening, developing diagnostic tools, determining optimal individualized treatment and producing targeted drug regimens.

  • Collaboration between urologists, data scientists, computer researchers and engineers is required to ensure that artificial intelligence (AI)-based decision-support applications are properly trained, operated and regulated.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Fig. 1: SVM learning enables us to solve classification problems — that is, to find a separation or a decision boundary or a classifier line between patterns.
Fig. 2: A handcrafted feature-based ML approach versus a DL approach applied to a digital pathology image.
Fig. 3: Prostate segmentation in ultrasonography34 and MRI29 using DL.
Fig. 4: ML technique to detect prostate cancer and create a cancer probability map.
Fig. 5: The da Vinci console camera view, augmented by two Tile-pro images.
Fig. 6: Illustration of the probabilistic approach to prostate cancer grading based on multi-expert training.
Fig. 7: Results of a cross validation study to compare an automatic classifier with pathologists’ grading.
Fig. 8: Synthetic patches generated using GANs and autoencoders


  1. Russell, S. J. & Norvig, P. Artificial Intelligence: A Modern Approach 3rd edn (Prentice Hall, 2009).

  2. McGinnis, D. What is the fourth industrial revolution? Salesforce (2018).

  3. Hodges, A. Beyond Turing’s machines. Science 336, 163–164 (2012).

    CAS  PubMed  Article  Google Scholar 

  4. Friedman, T. L. Thank You for Being Late: An Optimist’s Guide to Thriving in the Age of Accelerations 38–39 (Picador, 2017).

  5. Darcy, A. M., Louie, A. K. & Roberts, L. W. Machine learning and the profession of medicine. JAMA 315, 551–552 (2016).

    CAS  PubMed  Article  Google Scholar 

  6. Duda, R. O., Hart, P. E. & Stork, D. G. Pattern Classification (John Wiley & Sons, 2001).

  7. Bishop, C. M. Pattern Recognition and Machine Learning (Springer-Verlag New York, 2006).

  8. Nelder, J. A. & Baker, R. J. in Encyclopedia of Statistical Sciences 2nd edn (eds Kotz, S., Read, C. B., Balakrishnan, N., Vidakovic, B. & Johnson, N. L.) (John Wiley & Sons, 2004).

  9. Breiman, L. Random forests. Mach. Learn. 45, 5–32 (2001).

    Article  Google Scholar 

  10. Khurd, P. et al. Computer-aided Gleason grading of prostate cancer histopathological images using texton forests. Proc. IEEE Int. Symp. Biomed. Imaging 2010, 636–639 (2010).

    Google Scholar 

  11. Doyle, S., Feldman, M. D., Shih, N., Tomaszewski, J. & Madabhushi, A. Cascaded discrimination of normal, abnormal, and confounder classes in histopathology: Gleason grading of prostate cancer. BMC Bioinformatics 13, 282 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

  12. Gorelick, L. et al. Prostate histopathology: learning tissue component histograms for cancer detection and classification. IEEE Trans Med. Imaging 32, 1804–1818 (2013).

    PubMed  Article  Google Scholar 

  13. Jolliffe, I. in Encyclopedia of Statistics in Behavioral Science Vol. 1 (eds Everitt, B. & Howell, D.) (John Wiley & Sons, 2005).

  14. Goodfellow, I., Bengio, Y., Courville, A. & Bengio, Y. Deep Learning (MIT Press, 2016).

  15. Seligson, D. B. et al. Global histone modification patterns predict risk of prostate cancer recurrence. Nature 435, 1262 (2005). This study uses unsupervised learning techniques to identify markers of recurrence of prostate cancer.

    CAS  PubMed  Article  Google Scholar 

  16. Thananjeyan, B. et al. in IEEE International Conference on Robotics and Automation (ICRA) 2371–2378 (IEEE, 2017).

  17. Russ, J. C. The Image Processing Handbook 6th edn (CRC Press, 2016).

  18. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436 (2015). This paper provides an overview of DL and its many applications by three pioneers in the field.

    CAS  PubMed  Article  Google Scholar 

  19. Bishop, C. M. Neural Networks for Pattern Recognition (Oxford Univ. Press, 1995).

  20. LeCun, Y. et al. Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551 (1989).

    Article  Google Scholar 

  21. Litjens, G. et al. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017). This article reviews the major DL concepts pertinent to medical image analysis and summarizes over 300 contributions to the field.

    PubMed  Article  Google Scholar 

  22. Shen, D., Wu, G. & Suk, H. I. Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248 (2017).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  23. Suzuki, K. Overview of deep learning in medical imaging. Radiol. Phys. Technol. 10, 257–273 (2017).

    PubMed  Article  Google Scholar 

  24. Alam, I. S. et al. Emerging intraoperative imaging modalities to improve surgical precision. Mol. Imaging Biol. 20, 705–715 (2018).

    PubMed  Article  Google Scholar 

  25. Angermueller, C., Pärnamaa, T., Parts, L. & Stegle, O. Deep learning for computational biology. Mol. Systems Biol. 12, 878 (2016).

    Article  Google Scholar 

  26. Litjens, G. et al. Evaluation of prostate segmentation algorithms for MRI: the PROMISE12 challenge. Med. Image Anal. 18, 359–373 (2014).

    PubMed  Article  Google Scholar 

  27. Madabhushi, A. & Lee, G. Image analysis and machine learning in digital pathology: challenges and opportunities. Med. Image Anal. 33, 170–175 (2016). This paper reviews both handcrafted feature extraction and DL approaches for histopathological image analysis and discusses digital pathology as a bridge between radiology and genomics.

    PubMed  PubMed Central  Article  Google Scholar 

  28. Nir, G. et al. Comparison of artificial intelligence techniques to evaluate performance of a classifier for automatic grading of prostate cancer from digitized histopathologic images. JAMA Netw. Open 2, e190442 (2019).

    PubMed  PubMed Central  Article  Google Scholar 

  29. Karimi, D., Samei, G., Kesch, C., Nir, G. & Salcudean, S. E. Prostate segmentation in MRI using a convolutional neural network architecture and training strategy based on statistical shape models. Int. J. Comput. Assist. Radiol. Surg. 13, 1211–1219 (2018).

    PubMed  Article  Google Scholar 

  30. Cootes, T. F., Taylor, C. J., Cooper, D. H. & Graham, J. Active shape models-their training and application. Comput. Vis. Image Underst. 61, 38–59 (1995).

    Article  Google Scholar 

  31. Milletari, F., Rothberg, A., Jia, J. & Sofka, M. in International Conference on Medical Image Computing and Computer-Assisted Intervention 161–168 (Springer, 2017).

  32. Dice, L. R. Measures of the amount of ecologic association between species. Ecology 26, 297–302 (1945).

    Article  Google Scholar 

  33. Sørensen, T. A method of establishing groups of equal amplitude in plant sociology based on similarity of species and its application to analyses of the vegetation on Danish commons. Biol. Skr. 5, 1–34 (1948).

    Google Scholar 

  34. Zeng, Q. et al. Prostate segmentation in transrectal ultrasound using magnetic resonance imaging priors. Int. J. Comput. Assist. Radiol. Surg. 13, 749–757 (2018).

    PubMed  Article  Google Scholar 

  35. Anas, E. M. A., Mousavi, P. & Abolmaesumi, P. A deep learning approach for real time prostate segmentation in freehand ultrasound guided biopsy. Med. Image Anal. 48, 107–116 (2018).

    PubMed  Article  Google Scholar 

  36. Hu, Y. et al. Weakly-supervised convolutional neural networks for multimodal image registration. Med. Image Anal. 49, 1–13 (2018).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  37. Litjens, G., Debats, O., Barentsz, J., Karssemeijer, N. & Huisman, H. Computer-aided detection of prostate cancer in MRI. IEEE Trans. Med. Imaging 33, 1083–1092 (2014).

    PubMed  Article  Google Scholar 

  38. Moradi, M. et al. Multiparametric MRI maps for detection and grading of dominant prostate tumors. J. Magn. Reson. Imaging 35, 1403–1413 (2012).

    PubMed  PubMed Central  Article  Google Scholar 

  39. Liu, S., Zheng, H., Feng, Y. & Li, W. Prostate cancer diagnosis using deep learning with 3D multiparametric MRI. Proc. SPIE Int. Soc. Opt. Eng. 10134, 1013428 (2017).

    Google Scholar 

  40. Karimi, D. & Ruan, D. in International Conference on Medical Image Computing and Computer-Assisted Intervention 391–398 (Springer, 2017).

  41. Lemaître, G. et al. Computer-aided detection and diagnosis for prostate cancer based on mono and multi-parametric MRI: a review. Comput. Biol. Med. 60, 8–31 (2015).

    PubMed  Article  Google Scholar 

  42. Shiradkar, R. et al. Radiomics based targeted radiotherapy planning (Rad-TRaP): a computational framework for prostate cancer treatment planning with MRI. Radiat. Oncol. 11, 148 (2016).

    PubMed  PubMed Central  Article  Google Scholar 

  43. Boussion, N., Valeri, A., Malhaire, J. & Visvikis, D. Predicting the number of seeds in LDR prostate brachytherapy using machine learning and 320 patients [abstract PO-0897]. Radiother. Oncol. 127, S477–S478 (2018).

    Article  Google Scholar 

  44. Kalan, S. et al. History of robotic surgery. J. Robot. Surg. 4, 141–147 (2010).

    PubMed  Article  Google Scholar 

  45. Kassahun, Y. et al. Surgical robotics beyond enhanced dexterity instrumentation: a survey of machine learning techniques and their role in intelligent and autonomous surgical actions. Int. J. Comput. Assist. Radiol. Surg. 11, 553–568 (2016). This article discusses current and future ML applications in surgical robotics.

    PubMed  Article  Google Scholar 

  46. Yip, M. & Das, N. Robot autonomy for surgery. Preprint at arXiv (2017).

  47. Yang, G. Z. et al. Medical robotics—regulatory, ethical, and legal considerations for increasing levels of autonomy. Sci. Robot 2, 8638 (2017).

    Article  Google Scholar 

  48. Ji, J. J., Krishnan, S., Patel, V., Fer, D. & Goldberg, K. in 2018 IEEE 14th International Conference on Automation Science and Engineering (CASE) 35–42 (IEEE, 2018).

  49. Mohareri, O. et al. Intraoperative registered transrectal ultrasound guidance for robot-assisted laparoscopic radical prostatectomy. J. Urol. 193, 302–312 (2015).

    PubMed  Article  Google Scholar 

  50. Mohareri, O. et al. in International Conference on Medical Image Computing and Computer-Assisted Intervention 497–504 (Springer, 2015).

  51. Samei, G. et al. Real-time FEM-based registration of 3D to 2.5 D transrectal ultrasound images. IEEE Trans. Med. Imaging 37, 1877–1886 (2018).

    PubMed  Article  Google Scholar 

  52. Teber, D. et al. Augmented reality: a new tool to improve surgical accuracy during laparoscopic partial nephrectomy? Preliminary in vitro and in vivo results. Eur. Urol. 56, 332–338 (2009).

    PubMed  Article  Google Scholar 

  53. Goldenberg, M. & Lee, J. Y. Surgical education, simulation and simulators-updating the concept of validity. Curr. Urol. Rep. 19, 52 (2018).

    PubMed  Article  Google Scholar 

  54. Sarikaya, D., Corso, J. & Guru, K. Detection and localization of robotic tools in robot-assisted surgery videos using deep neural networks for region proposal and detection. IEEE Trans. Med. Imaging 36, 1542–1549 (2017).

    PubMed  Article  Google Scholar 

  55. Hung, A. J., Chen, J. & Gill, I. S. Automated performance metrics and machine learning algorithms to measure surgeon performance and anticipate clinical outcomes in robotic surgery. JAMA Surg. 153, 770–771 (2018).

    PubMed  Article  PubMed Central  Google Scholar 

  56. Pantanowitz, L. Digital images and the future of digital pathology. J. Pathol. Inform. 1, 15 (2010).

    PubMed  PubMed Central  Article  Google Scholar 

  57. Huisman, A., Looijen, A., van den Brink, S. M. & van Diest, P. J. Creation of a fully digital pathology slide archive by high-volume tissue slide scanning. Hum. Pathol. 41, 751–757 (2010).

    PubMed  Article  Google Scholar 

  58. Thompson, D. et al. Multimegapixel images in histopathology. Anal. Quant. Cytol. Histol 23, 169–177 (2001).

    CAS  PubMed  Google Scholar 

  59. Arevalo, J. et al. Histopathology image representation for automatic analysis: a state-of-the-art review. Revista Med. 22, 79–91 (2014).

    Article  Google Scholar 

  60. Mosquera-Lopez, C., Agaian, S., Velez-Hoyos, A. & Thompson, I. Computer aided prostate cancer diagnosis from digitized histopathology: a review on texture-based systems. IEEE Rev. Biomed. Eng. 8, 98–113 (2015).

    PubMed  Article  Google Scholar 

  61. Bhargava, R. & Madabhushi, A. Emerging themes in image informatics and molecular analysis for digital pathology. Ann. Rev. Biomed. Eng. 18, 387–412 (2016).

    CAS  Article  Google Scholar 

  62. Jothi, J. A. & Rajam, M. A. A survey on automated cancer diagnosis from histopathology images. Artif. Intell. Rev. 48, 31–81 (2017).

    Article  Google Scholar 

  63. Chen, P. H. C. et al. An augmented reality microscope for real-time automated detection of cancer. Google (2018).

  64. Montironi, R., Cheng, L., Lopez-Beltran, A. & Scarpelli, M. Quantitative image analysis on histologic virtual slides for prostate pathology diagnosis, response to chemopreventive agents, and prognosis. Eur. Urol. Focus 3, 467–469 (2017).

    PubMed  Article  Google Scholar 

  65. Diamond, J., Anderson, N. H., Bartels, P. H., Montironi, R. & Hamilton, P. W. The use of morphological characteristics and texture analysis in the identification of tissue composition in prostatic neoplasia. Hum. Pathol. 35, 1121–1131 (2004).

    PubMed  Article  Google Scholar 

  66. Janowczyk, A. & Madabhushi, A. Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J. Pathol. Inform. 7, 29 (2016).

    PubMed  PubMed Central  Article  Google Scholar 

  67. Litjens, G. et al. Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci. Rep. 6, 26286 (2016).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  68. Källén, H., Molin, J., Heyden, A., Lundström, C. & Åström, K. in 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) 1163–1167 (IEEE, 2016).

  69. Rezaeilouyeh, H., Mollahosseini, A. & Mahoor, M. H. Microscopic medical image classification framework via deep learning and shearlet transform. J. Med. Imaging (Bellingham) 3, 044501 (2016).

    Article  Google Scholar 

  70. Zhou, N., Fedorov, A., Fennessy, F., Kikinis, R. & Gao, Y. Large scale digital prostate pathology image analysis combining feature extraction and deep neural network. Preprint at arXiv 02678 (2017).

  71. Bejnordi, B. E. et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318, 2199–2210 (2017).

    Article  Google Scholar 

  72. Allsbrook, W. C. Jr. et al. Interobserver reproducibility of Gleason grading of prostatic carcinoma: urologic pathologists. Hum. Pathol. 32, 74–80 (2001).

    PubMed  Article  Google Scholar 

  73. Nir, G. et al. Automatic grading of prostate cancer in digitized histopathology images: learning from multiple experts. Med. Image Anal. 50, 167–180 (2018).

    PubMed  Article  Google Scholar 

  74. Kononen, J. et al. Tissue microarrays for high-throughput molecular profiling of tumor specimens. Nat. Med. 4, 844–847 (1998).

    CAS  PubMed  Article  Google Scholar 

  75. Radford, A., Metz, L. & Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. Preprint at arXiv (2015).

  76. Hou, L. et al. Unsupervised histopathology image synthesis. Preprint at arXiv (2017).

  77. Senaras, C., Sahiner, B., Tozbikian, G., Lozanski, G. & Gurcan, M. N. Creating synthetic digital slides using conditional generative adversarial networks: application to Ki67 staining. Proc. SPIE Int. Soc. Opt. Eng. 10581, 1058103 (2018).

    Google Scholar 

  78. Ross, A. E., D’amico, A. V. & Freedland, S. J. Which, when and why? Rational use of tissue-based molecular testing in localized prostate cancer. Prostate Cancer Prostatic Dis. 19, 1–16 (2016).

    CAS  PubMed  Article  Google Scholar 

  79. Libbrecht, M. W. & Noble, W. S. Machine learning applications in genetics and genomics. Nat. Rev. Genet. 16, 321–332 (2015). This paper provides an overview of ML applications for the analysis of genome sequencing data sets, including the annotation of sequence elements and epigenetic, proteomic or metabolomic data.

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  80. Karnes, R. J. et al. Validation of a genomic classifier that predicts metastasis following radical prostatectomy in an at risk patient population. J. Urol. 190, 2047–2053 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  81. Nguyen, P. L. et al. Ability of a genomic classifier to predict metastasis and prostate cancer-specific mortality after radiation or surgery based on needle biopsy specimens. Eur. Urol. 72, 845–852 (2017).

    PubMed  Article  Google Scholar 

  82. Lee, S. et al. Machine learning on a genome-wide association study to predict late genitourinary toxicity after prostate radiation therapy. Int. J. Radiat. Oncol. Biol. Phys. 101, 128–135 (2018).

    PubMed  PubMed Central  Article  Google Scholar 

  83. Lee, G. et al. Supervised multi-view canonical correlation analysis (sMVCCA): integrating histologic and proteomic features for predicting recurrent prostate cancer. IEEE Trans. Med. Imaging 34, 284–297 (2015).

    PubMed  Article  Google Scholar 

  84. Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  85. Ren, J., Karagoz, K., Gatza, M., Foran, D. J. & Qi, X. Differentiation among prostate cancer patients with Gleason score of 7 using histopathology image and genomic data. Proc. SPIE Int. Soc. Opt. Eng. 10579, 1057904 (2018).

    PubMed  PubMed Central  Google Scholar 

  86. Rubin, M. A. & Demichelis, F. The genomics of prostate cancer: emerging understanding with technologic advances. Mod. Pathol. 31, S1–S11 (2018).

    PubMed  Article  Google Scholar 

  87. Quotes. “Computers are incredibly fast, accurate, and stupid. Human beings are incredibly slow, inaccurate, and brilliant. Together they are powerful beyond imagination”. (2019).

  88. Kristiansen, G. Markers of clinical utility in the differential diagnosis and prognosis of prostate cancer. Mod. Pathol. 31, S143–S155 (2018).

    PubMed  Article  Google Scholar 

Download references


This work was funded by a Prostate Cancer Canada grant (D2016-1352), by the Canadian Institutes of Health Research (MOP-142439) and by the C. A. Laszlo Chair of S.E.S. G.N. is a recipient of a Prostate Cancer Canada Postdoctoral Research Fellowship Award (PDF2016-1338).

Reviewer information

Nature Reviews Urology thanks O. Vermesh and Q. Wang for their contribution to the peer review of this work.

Author information

Authors and Affiliations



All authors researched data for the article, made substantial contributions to discussions of content and reviewed and edited the manuscript before submission. S.L.G. and G.N. wrote the manuscript.

Corresponding author

Correspondence to S. Larry Goldenberg.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.



A classifier in machine learning refers to the placing of a new observation into the appropriate category among those categories that were based on trained data sets of known observations.

Ground truth

Labels or annotations that were determined by an expert, considered to be the correct targets and used in training, testing and evaluating machine learning algorithms.

Naive Bayes classification

A supervised learning method that is based on a probabilistic approach and relies on Bayes’ theorem. The learning process involves parameter estimation of the probability distributions based on the data. For a new sample, the class with the maximum probability would be predicted on the basis of the probability of its features.

Support vector machines

(SVMs). Supervised learning methods for classification that learn the optimal ‘separation’ between the features of each class. The predicted class of a new sample would be based on the ‘region’ in feature space that the sample features occupy.

Random forests

A supervised learning method for classification that is based on decision trees. It consists of multiple trees, each with a random subset of the features, and tries to optimize the split values of the branches.

k-Means clustering

An unsupervised learning method for clustering. The algorithm iteratively assigns each data sample into one of k classes on the basis of the ‘distances’ between features.

Principal component analysis

An unsupervised learning method for dimensionality reduction.


Types of neural networks that are trained to encode an input into a lower dimensionality such that the reverse decoder can reconstruct the encoded sample as similar as possible to the original input. These models can be used for unsupervised learning of the most descriptive features of the data, for example, for dimensionality reduction.

Artificial neural networks

A collection of units that are connected to each other (typically) as layers and inspired by biological neural networks in the brain. Each unit, also referred to as an ‘artificial neuron’, has an output that is a function of the weighted sum of its multiple inputs and is ‘activated’ if that sum is higher than a threshold (bias). Given the data as the input to the first layer, the weights and biases of the neurons are optimized to match the output of the last layer with some target (for example, minimizing a classification error).

Convolutional neural network

(CNN). A type of artificial neural network in which a neuron in a layer is connected to a few adjacent neurons from the previous layer, and the next neuron in that layer is connected to the next adjacent neurons from the previous layer and so on. Such a network architecture is commonly used in learning vision tasks such as image classification.

‘Leave-patient-out’ validation

An approach, also known as a k-fold cross-validation, to evaluate the performance of a classifier by training it on all data except the samples of one or more patient or patients and then testing it on the left outpatient or outpatients. The process can be repeated over all patients, each time with a different (subset of) patient or patients left out, and averaging the results.

Statistical shape modelling

A representation of a set of shapes by modelling their geometry with a typically small number of parameters that control their main modes of variation and are derived using statistical methods.

Sørensen–Dice similarity coefficient

A value that measures the similarity between two sets, with a value of zero when the two sets are unique and a value of one when they completely overlap. If the sets are pixels within two shapes, the Sørensen–Dice similarity coefficient can measure the amount of overlap of the shapes.

Recurrent neural network

(RNN). A type of artificial neural network in which neurons are connected to other neurons at previous time steps and can, therefore, learn temporal patterns in sequential data.

Dynamic contrast-enhanced

(DCE). A modality of MRI that measures parameters of tissue perfusion (ktrans) in the presence of a contrast agent.

Diffusion tensor imaging

(DTI). An MRI technique that maps the diffusion of water molecules in the tissue.

Generative adversarial networks

(GANs). Types of artificial neural networks in which there are two paths of layers: one that generates samples from a random input and another that tests the similarity of the generated samples to real samples. The generative network is, therefore, trained to generate samples that mimic the real samples. Such networks are used to generate synthetic images that are visually similar to real images of their class.


When a machine learning model is trained to perform well on a limited data set and performs worse when it is applied to new data, it is said to be overfitted to those data. In order to avoid overfitting, a data set that is large enough to represent the real-world diversity should be used for training and the training should be stopped before fully converging.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Goldenberg, S., Nir, G. & Salcudean, S.E. A new era: artificial intelligence and machine learning in prostate cancer. Nat Rev Urol 16, 391–403 (2019).

Download citation

  • Published:

  • Issue Date:

  • DOI:

Further reading


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing