Credit: YAY Media AS / Alamy Stock Vector

Histopathology is the current ‘gold standard’ for establishing cancer diagnosis. Whereas the interpretation of tissue slides is subjective, the advent of clinical genomics has contributed to refining patient stratification and clinical decision-making through the identification of actionable tumour vulnerabilities. However, both histological assessment and sequencing approaches are time consuming and costly, and they often yield inconsistent results across institutions. Enter big data and artificial intelligence (AI), with the premise that, if enough information is available, predictive models can be generated that achieve clinical-grade automated diagnoses and facilitate clinicians’ workflows for accurate diagnosis.

The past decade has witnessed a surge in the amount of digitalized clinical data, including electronic health records, genomics and digital biomedical images. Perhaps because of its standardized high-quality data-collection protocols and fewer inherent missing-data challenges than in other clinical data types, medical imaging has emerged as a frontier of success for AI in medicine. Computer vision has enabled some of the most striking studies predicting molecular and pathological cues from digitalized images. The level of information that can be retrieved from digital images surpasses human capacity in terms of speed as well as the ability to infer and detect subtle traits hidden to human perception.

In 2017, Esteva et al. published a landmark study in the field of computer vision applied to cancer detection. The authors used a large dataset of digital images of skin conditions to train and validate a deep convolutional neural network able to discriminate between benign and malignant lesions with accuracy similar to that of trained dermatologists. These findings sparked further studies aimed at understanding whether additional features beyond the classification of tumour versus normal tissue—a task that can be easily conducted visually by trained pathologists—could be ascertained from histological patterns. Proof-of-concept work by Coudray et al. provided the first solid evidence that deep learning on tumour histology slides can predict driver mutations and classify tumour subtypes. Follow-up work by Campanella et al. demonstrated that high diagnostic accuracy can be achieved without a need for prior data curation, if the datasets are sufficiently large, and studies by Fu et al. and Kather et al. expanded the breadth of applicability of these algorithms in detecting molecular footprints and prognostic correlates across multiple cancers.

Perhaps the most relevant functionality of AI in cancer histopathology is its integration in clinical workflows for improving patient management. For example, pioneering work by Bejnordi et al. exploring the clinical value of AI showed that deep-learning models can achieve accurate detection of lymph node metastases in breast cancer, rivalling human performance. Another pragmatic clinical application of medical AI has enabled near-real-time diagnosis of patients in the operating room. A prospective clinical trial by Hollon et al. has shown that an AI-powered system can deliver accurate diagnoses for patients undergoing surgery for brain cancer, thus underscoring the potential of these models to increase the speed and accuracy of clinical decisions.

Future studies are needed to ensure that this level of augmented perception is not available only to institutions that can afford expensive medical appliances and software. Esteva et al. paved the way to affordable access to AI-assisted differential classification of skin lesions through the use of regular cell-phone cameras. Other efforts by Chen et al. have enabled real-time integration through augmented-reality approaches which, through overlaying an AI system with a microscope field of view, facilitate automated histological annotation in places lacking access to trained pathologists. These methods have extended the real-world applications of AI a step further in the diagnosis for prostate cancer, and they may be expanded to other tumour types and diseases that often require specialized histological evaluation.

Most studies to date have been correlative and based on retrospective analysis of controlled data sets. These have provided the foundations for AI in healthcare applications and have illustrated new possibilities for enhancing the performance of trained pathologists. To fully realize their potential, AI systems must crucially be tested in controlled prospective studies to accurately evaluate their clinical value—a setting for which specific guidelines have been elaborated upon by Liu et al.

Computer vision and digital pathology are only the beginning, and AI systems are being explored in multiple additional settings. The convergence of big data and AI in oncology provides a unique interface to explore the full capabilities of these approaches for the benefit of patients. We look forward to the next 10 years of groundbreaking discoveries in this exciting area of research.

Further reading

Coudray, N. et al. Classification and mutation prediction from non–small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).

Campanella, G. et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat. Med. 25, 1301–1309 (2019).

Fu, Y. et al. Pan-cancer computational histopathology reveals mutations, tumor composition and prognosis. Nat. Cancer 1, 800–810 (2020).

Kather, J. N. et al. Pan-cancer image-based detection of clinically actionable genetic alterations. Nat. Cancer 1, 789–799 (2020).

Bejnordi, B. E. et al. Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 22, 2199–2210 (2017).

Hollon, T. C. et al. Near real-time intraoperative brain tumor diagnosis using stimulated Raman histology and deep neural networks. Nat. Med. 26, 52–58 (2020).

Chen, P. C. et al. An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis. Nat. Med. 25, 1453–1457 (2019).

Liu, X. et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nat. Med. 26, 1364–1374 (2020).