Intelligent feature engineering and ontological mapping of brain tumour histomorphologies by deep learning

Abstract

Deep learning is an emerging transformative tool in diagnostic medicine, yet limited access and the interpretability of learned parameters hinders widespread adoption. Here we have generated a diverse repository of 838,644 histopathologic images and used them to optimize and discretize learned representations into 512-dimensional feature vectors. Importantly, we show that individual machine-engineered features correlate with salient human-derived morphologic constructs and ontological relationships. Deciphering the overlap between human and machine reasoning may aid in eliminating biases and improving automation and accountability for artificial intelligence-assisted medicine.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Dataset development and deep learning strategy.
Fig. 2: Ontological arrangement of brain tumour histopathologic patterns by deep learning.
Fig. 3: Correlation and prediction of histologic patterns using individual cluster-enriched DLFs.
Fig. 4: Testing of individual DLFs and DLFVs on an independent cohort of cases.

Data availability

The full image training datasets used to generate the DLFVs are available on Bitbucket (https://bitbucket.org/diamandislabii/faust-feature-vectors-2019) (https://doi.org/10.5281/zenodo.3234829). External testing images generated by the TCGA Research Network (http://cancergenome.nih.gov/) are also publicly available without restrictions.

Code availability

Code and both the general and 1p19q-codeletion trained CNN models are available on Bitbucket (https://bitbucket.org/diamandislabii/faust-feature-vectors-2019 and https://doi.org/10.5281/zenodo.3234829). A stand-alone computing capsule is available on Code Ocean for Feature Vector Extraction and Clustering (https://doi.org/10.24433/CO.3573560.v2) and for feature activation map (FAM) generation (https://doi.org/10.24433/CO.7749421.v1).

Change history

  • 10 December 2019

    An amendment to this paper has been published and can be accessed via a link at the top of the paper.

References

  1. 1.

    Lotz, J. M., Primack, J. & Madau, P. A new nonparametric approach to galaxy morphological classification. Astron. J. 128, 163–182 (2004).

    Article  Google Scholar 

  2. 2.

    Murphy, W. J. et al. Molecular phylogenetics and the origins of placental mammals. Nature 409, 614–618 (2001).

    Article  Google Scholar 

  3. 3.

    Louis, D. N. et al. The 2016 World Health Organization classification of tumors of the central nervous system: a summary. Acta Neuropathol. 131, 803–820 (2016).

    Article  Google Scholar 

  4. 4.

    Nicholson, A. G. et al. Inter-observer variation between pathologists in diffuse parenchymal lung disease. Thorax 59, 500–505 (2004).

    Article  Google Scholar 

  5. 5.

    Gilles, F. H. et al. Pathologist interobserver variability of histologic features in childhood brain tumors: results from the CCG-945 study. Pediatr. Dev. Pathol. 11, 108–117 (2008).

    Article  Google Scholar 

  6. 6.

    Allard, F. D. et al. Intraobserver and interobserver variability in the assessment of dysplasia in ampullary mucosal biopsies. Am. J. Surg. Pathol. 42, 1095–1100 (2018).

    Article  Google Scholar 

  7. 7.

    Naghavi, M. et al. Global, regional and national age-sex specific mortality for 264 causes of death, 1980–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet 390, 1151–1210 (2017).

    Article  Google Scholar 

  8. 8.

    LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  Google Scholar 

  9. 9.

    Mobadersany, P. et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc. Natl Acad. Sci. USA 115, E2970–E2979 (2018).

    Article  Google Scholar 

  10. 10.

    Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017).

    Article  Google Scholar 

  11. 11.

    Coudray, N. et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559–1567 (2018).

    Article  Google Scholar 

  12. 12.

    Servick, K. Brain scientists dive into deep neural networks. Science 361, 1177 (2018).

    Article  Google Scholar 

  13. 13.

    Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Preprint at http://arxiv.org/abs/1409.1556 (2014).

  14. 14.

    Holzinger, A. et al. Causability and explainabilty of artificial intelligence in medicine. WIRES Data Min. Knowl. Discov. 9, e1312 (2019).

    Article  Google Scholar 

  15. 15.

    Doshi-Velez, F. & Kim, B. Towards a rigorous science of interpretable machine learning. Preprint at http://arxiv.org/abs/1702.08608(2017).

  16. 16.

    Samek, W., Wiegand, T. & Müller, K.-R. Explainable artificial intelligence: understanding, visualizing and interpreting deep learning models. Preprint at http://arxiv.org/abs/1708.08296 (2017).

  17. 17.

    Faust, K. et al. Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction. BMC Bioinformatics 19, 173 (2018).

    Article  Google Scholar 

  18. 18.

    Cancer Genome Atlas Research Network et al.Comprehensive, integrative genomic analysis of diffuse lower-grade gliomas. N. Engl. J. Med. 372, 2481–2498 (2015).

    Article  Google Scholar 

Download references

Acknowledgements

Funding support for this research and trainees is provided by the Princess Margaret Cancer Foundation, an Adam Coules Research Grant, the Department of Laboratory Medicine and Pathology at the University Health Network, the Brain Tumour Foundation of Canada, the American Society of Clinical Oncology Career Development Award (ASCO-CDA) and The Brain Tumour Charity Expanding Theories Research Grant Program (GN-000560). We thank P. Boutros for critical feedback and suggestions.

Author information

Affiliations

Authors

Contributions

K.F., U.D. and P.D. conceived the idea and approach. K.F. developed the computational workflow. S.B., R.A.Q., A.P., P.D. and R.v.O. reviewed and selected the included cases and annotated and generated the image dataset. K.F., U.D., R.v.O. and P.D. wrote the manuscript, with input from all other authors.

Corresponding author

Correspondence to Phedias Diamandis.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Faust, K., Bala, S., van Ommeren, R. et al. Intelligent feature engineering and ontological mapping of brain tumour histomorphologies by deep learning. Nat Mach Intell 1, 316–321 (2019). https://doi.org/10.1038/s42256-019-0068-6

Download citation

Further reading