Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Intelligent feature engineering and ontological mapping of brain tumour histomorphologies by deep learning

Nature Machine Intelligence volume 1, pages 316–321 (2019)Cite this article

Subjects

A Publisher Correction to this article was published on 10 December 2019

This article has been updated

Abstract

Deep learning is an emerging transformative tool in diagnostic medicine, yet limited access and the interpretability of learned parameters hinders widespread adoption. Here we have generated a diverse repository of 838,644 histopathologic images and used them to optimize and discretize learned representations into 512-dimensional feature vectors. Importantly, we show that individual machine-engineered features correlate with salient human-derived morphologic constructs and ontological relationships. Deciphering the overlap between human and machine reasoning may aid in eliminating biases and improving automation and accountability for artificial intelligence-assisted medicine.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Dataset development and deep learning strategy.
Fig. 2: Ontological arrangement of brain tumour histopathologic patterns by deep learning.
Fig. 3: Correlation and prediction of histologic patterns using individual cluster-enriched DLFs.
Fig. 4: Testing of individual DLFs and DLFVs on an independent cohort of cases.

Similar content being viewed by others

Data availability

The full image training datasets used to generate the DLFVs are available on Bitbucket (https://bitbucket.org/diamandislabii/faust-feature-vectors-2019) (https://doi.org/10.5281/zenodo.3234829). External testing images generated by the TCGA Research Network (http://cancergenome.nih.gov/) are also publicly available without restrictions.

Code availability

Code and both the general and 1p19q-codeletion trained CNN models are available on Bitbucket (https://bitbucket.org/diamandislabii/faust-feature-vectors-2019 and https://doi.org/10.5281/zenodo.3234829). A stand-alone computing capsule is available on Code Ocean for Feature Vector Extraction and Clustering (https://doi.org/10.24433/CO.3573560.v2) and for feature activation map (FAM) generation (https://doi.org/10.24433/CO.7749421.v1).

Change history

  • 10 December 2019

    An amendment to this paper has been published and can be accessed via a link at the top of the paper.

References

  1. Lotz, J. M., Primack, J. & Madau, P. A new nonparametric approach to galaxy morphological classification. Astron. J. 128, 163–182 (2004).

    Article  Google Scholar 

  2. Murphy, W. J. et al. Molecular phylogenetics and the origins of placental mammals. Nature 409, 614–618 (2001).

    Article  Google Scholar 

  3. Louis, D. N. et al. The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 131, 803–820 (2016).

    Article  Google Scholar 

  4. Nicholson, A. G. et al. Inter-observer variation between pathologists in diffuse parenchymal lung disease. Thorax 59, 500–505 (2004).

    Article  Google Scholar 

  5. Gilles, F. H. et al. Pathologist interobserver variability of histologic features in childhood brain tumors: results from the CCG-945 study. Pediatr. Dev. Pathol. 11, 108–117 (2008).

    Article  Google Scholar 

  6. Allard, F. D. et al. Intraobserver and interobserver variability in the assessment of dysplasia in ampullary mucosal biopsies. Am. J. Surg. Pathol. 42, 1095–1100 (2018).

    Article  Google Scholar 

  7. Naghavi, M. et al. Global, regional and national age-sex specific mortality for 264 causes of death, 1980–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 390, 1151–1210 (2017).

    Article  Google Scholar 

  8. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  Google Scholar 

  9. Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

    Article  Google Scholar 

  10. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

    Article  Google Scholar 

  11. Coudray, N. et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).

    Article  Google Scholar 

  12. Servick, K. Brain scientists dive into deep neural networks. Science 361, 1177 (2018).

    Article  Google Scholar 

  13. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Preprint at http://arxiv.org/abs/1409.1556 (2014).

  14. Holzinger, A. et al. Causability and explainabilty of artificial intelligence in medicine. WIRES Data Min. Knowl. Discov. 9, e1312 (2019).

    Article  Google Scholar 

  15. Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at http://arxiv.org/abs/1702.08608(2017).

  16. Samek, W., Wiegand, T. & Müller, K.-R. Explainable artificial intelligence: understanding, visualizing and interpreting deep learning models. Preprint at http://arxiv.org/abs/1708.08296 (2017).

  17. Faust, K. et al. Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction. BMC Bioinformatics 19, 173 (2018).

    Article  Google Scholar 

  18. Cancer Genome Atlas Research Network et al.Comprehensive, integrative genomic analysis of diffuse lower-grade gliomas. N. Engl. J. Med. 372, 2481–2498 (2015).

    Article  Google Scholar 

Download references

Acknowledgements

Funding support for this research and trainees is provided by the Princess Margaret Cancer Foundation, an Adam Coules Research Grant, the Department of Laboratory Medicine and Pathology at the University Health Network, the Brain Tumour Foundation of Canada, the American Society of Clinical Oncology Career Development Award (ASCO-CDA) and The Brain Tumour Charity Expanding Theories Research Grant Program (GN-000560). We thank P. Boutros for critical feedback and suggestions.

Author information

Authors and Affiliations

  1. Princess Margaret Cancer Centre, MacFeeters–Hamilton Brain Tumour Centre, Toronto, Ontario, Canada

    Kevin Faust, Sudarshan Bala, Alessia Portante, Ugljesa Djuric & Phedias Diamandis

  2. Department of Computer Science, University of Toronto, Toronto, Ontario, Canada

    Kevin Faust

  3. Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada

    Randy van Ommeren & Phedias Diamandis

  4. Laboratory Medicine Program, University Health Network, Toronto, Ontario, Canada

    Raniah Al Qawahmed & Phedias Diamandis

Authors
  1. Kevin Faust
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sudarshan Bala
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Randy van Ommeren
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alessia Portante
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Raniah Al Qawahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ugljesa Djuric
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Phedias Diamandis
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

K.F., U.D. and P.D. conceived the idea and approach. K.F. developed the computational workflow. S.B., R.A.Q., A.P., P.D. and R.v.O. reviewed and selected the included cases and annotated and generated the image dataset. K.F., U.D., R.v.O. and P.D. wrote the manuscript, with input from all other authors.

Corresponding author

Correspondence to Phedias Diamandis.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Faust, K., Bala, S., van Ommeren, R. et al. Intelligent feature engineering and ontological mapping of brain tumour histomorphologies by deep learning. Nat Mach Intell 1, 316–321 (2019). https://doi.org/10.1038/s42256-019-0068-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-019-0068-6

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing