Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Human-level recognition of blast cells in acute myeloid leukaemia with convolutional neural networks

Nature Machine Intelligence volume 1, pages 538–544 (2019)Cite this article

Subjects

A preprint version of the article is available at bioRxiv.

Abstract

Reliable recognition of malignant white blood cells is a key step in the diagnosis of haematologic malignancies such as acute myeloid leukaemia. Microscopic morphological examination of blood cells is usually performed by trained human examiners, making the process tedious, time-consuming and hard to standardize. Here, we compile an annotated image dataset of over 18,000 white blood cells, use it to train a convolutional neural network for leukocyte classification and evaluate the network’s performance by comparing to inter- and intra-expert variability. The network classifies the most important cell types with high accuracy. It also allows us to decide two clinically relevant questions with human-level performance: (1) if a given cell has blast character and (2) if it belongs to the cell types normally present in non-pathological blood smears. Our approach holds the potential to be used as a classification aid for examining much larger numbers of cells in a smear than can usually be done by a human expert. This will allow clinicians to recognize malignant cell populations with lower prevalence at an earlier stage of the disease.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Data handling workflow.
Fig. 2: Classification of 18,000 single-cell images into an 18-class scheme.
Fig. 3: Human-level network performance in single-cell classification, pixel-wise attention and binary decision tasks.

Similar content being viewed by others

Data availability

The full single-cell image dataset and corresponding annotations are publicly available at The Cancer Imaging Archive (TCIA): https://doi.org/10.7937/tcia.2019.36f5o9ld 43.

Code availability

Code for the network trained in this study and network weights for one fold are available on CodeOcean, together with a subset of the single-cell image data used to test the network: https://codeocean.com/capsule/9068249/tree/v1 44.

References

  1. Bain, B. J. Diagnosis from the blood smear. N. Engl. J. Med.353, 498–507 (2005).

    Article  Google Scholar 

  2. Tkachuk, D. C. & Hirschmann, J. V. Wintrobe’s Atlas of Clinical Hematology (Lippincott Raven, 2006).

  3. Theml, H., Diem, H. & Haferlach, T. Color Atlas of Hematology (Thieme, 2004).

  4. Döhner, H. et al. Diagnosis and management of AML in adults: 2017 ELN recommendations from an international expert panel. Blood129, 424–447 (2017).

    Article  Google Scholar 

  5. Swerdlow, S. H. et al. (eds) WHO Classification of Tumours of Haematopoietic and Lymphoid Tissues 4th edn (International Agency for Research on Cancer, 2017).

  6. Arber, D. A. et al. The 2016 revision to the World Health Organization classification of myeloid neoplasms and acute leukemia. Blood127, 2391–2405 (2016).

    Article  Google Scholar 

  7. Bennett, J. M. et al. Proposed revised criteria for the classification of acute myeloid leukemia. A report of the French–American–British Cooperative Group. Ann. Intern. Med.103, 620–625 (1985).

    Article  Google Scholar 

  8. Font, P. et al. Inter-observer variance with the diagnosis of myelodysplastic syndromes (MDS) following the 2008 WHO classification. Ann. Hematol.92, 19–24 (2013).

    Article  Google Scholar 

  9. Font, P. et al. Interobserver variance in myelodysplastic syndromes with less than 5% bone marrow blasts: unilineage vs. multilineage dysplasia and reproducibility of the threshold of 2% blasts. Ann. Hematol.94, 565–573 (2015).

    Article  Google Scholar 

  10. Fuentes-Arderiu, X. & Dot-Bach, D. Measurement uncertainty in manual differential leukocyte counting. Clin. Chem. Lab. Med.47, 112–115 (2009).

    Article  Google Scholar 

  11. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).

  12. Rawat, W. & Wang, Z. Deep convolutional neural networks for image classification: a comprehensive review. Neural Comput.29, 2352–2449 (2017).

    Article  MathSciNet  MATH  Google Scholar 

  13. Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vision115, 211–252 (2015).

    Article  MathSciNet  Google Scholar 

  14. Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature542, 115–118 (2017).

    Article  Google Scholar 

  15. Eulenberg, P. et al. Reconstructing cell cycle and disease progression using deep learning. Nat. Commun.8, 463 (2017).

    Article  Google Scholar 

  16. Janowczyk, A. & Madabhushi, A. Deep learning for digital pathology image analysis: a comprehensive tutorial with selected use cases. J. Pathol. Inform.7, 29 (2016).

    Article  Google Scholar 

  17. Fuchs, T. J. & Buhmann, J. M. Computational pathology: challenges and promises for tissue analysis. Comput. Med. Imaging Graph.35, 515–530 (2011).

    Article  Google Scholar 

  18. Albarqouni, S. et al. AggNet: deep learning from crowds for mitosis detection in breast cancer histology images. IEEE Trans. Med. Imaging35, 1313–1321 (2016).

    Article  Google Scholar 

  19. Levenson, R. M., Fornari, A. & Loda, M. Multispectral imaging and pathology: seeing and doing more. Expert Opin. Med. Diagn.2, 1067–1081 (2008).

    Article  Google Scholar 

  20. Gertych, A. et al. Machine learning approaches to analyze histological images of tissues from radical prostatectomies. Comput. Med. Imaging Graph.46, 197–208 (2015).

    Article  Google Scholar 

  21. Bigorra, L., Merino, A., Alférez, S. & Rodellar, J. Feature analysis and automatic identification of leukemic lineage blast cells and reactive lymphoid cells from peripheral blood cell images. J. Clin. Lab. Anal.31, e22024 (2017).

    Article  Google Scholar 

  22. Krappe, S., Wittenberg, T., Haferlach, T. & Munzenmayer, C. Automated morphological analysis of bone marrow cells in microscopic images for diagnosis of leukemia: nucleus–plasma separation and cell classification using a hierarchical tree model of hematopoesis. Proc. SPIE9785, 97853C (2016).

    Google Scholar 

  23. Scotti, F. Automatic morphological analysis for acute leukemia identification in peripheral blood microscope images. In Computational Intelligence for Measurement Systems and Applications (CIMSA) 96–101 (IEEE, 2005).

  24. Mohapatra, S., Patra, D. & Satpathy, S. An ensemble classifier system for early diagnosis of acute lymphoblastic leukemia in blood microscopic images. Neural Comput. Appl.24, 1887–1904 (2014).

    Article  Google Scholar 

  25. Greenspan, H., van Ginneken, B. & Summers, R. M. Deep learning in medical imaging: overview and future promise of an exciting new technique. IEEE Trans. Med. Imaging35, 1153–1159 (2016).

    Article  Google Scholar 

  26. Shen, D., Wu, G. & Suk, H. Deep learning in medical image analysis. Ann. Rev. Biomed. Eng.19, 221–248 (2017).

    Article  Google Scholar 

  27. Choi, J. W. et al. White blood cell differential count of maturation stages in bone marrow smear using dual-stage convolutional neural networks. PLoS One12, e0189259 (2017).

    Article  Google Scholar 

  28. Kainz, P., Burgsteiner, H., Asslaber, M. & Ahammer, H. Training echo state networks for rotation-invariant bone marrow cell classification. Neural Comput. Appl.28, 1277–1292 (2017).

    Article  Google Scholar 

  29. Su, M.-C., Cheng, C.-Y. & Wang, P.-C. A neural-network-based approach to white blood cell classification. Sci. World J.2014, 796371 (2014).

    Google Scholar 

  30. Macawile, M. J., Quiñones, V. V., Ballado, A., Cruz, J. D. & Caya, M. V. White blood cell classification and counting using convolutional neural network. In 2018 3rd International Conference on Control and Robotics Engineering (ICCRE) 259–263 (IEEE, 2018).

  31. Keohane, E. M., Smith, L. & Walenga, J. M. Rodak’s Hematology—Clinical Principles and Applications 5th edn (Elsevier, 2016).

  32. Xie, S., Girshick, R., Dollár, P., Tu, Z. & He, K. Aggregated residual transformations for deep neural networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 5987–5995 (IEEE, 2017).

  33. Dietz, M. ResNeXt implementation for Keras. GitHub Gist https://gist.githubusercontent.com/mjdietzx/ (2017).

  34. Chollet, F. et al. Keras 2.0. Keras https://keras.io (2017).

  35. Bychkov, D. et al. Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci. Rep.8, 3395 (2018).

    Article  Google Scholar 

  36. Simonyan, K., Vedaldi, A. & Zisserman, A. Deep inside convolutional networks: visualising image classification models and saliency maps. Preprint at https://arxiv.org/abs/1312.6034 (2013).

  37. Mandrekar, J. N. Receiver operating characteristic curve in diagnostic test assessment. J. Thorac. Oncol.5, 1315–1316 (2010).

    Article  Google Scholar 

  38. Hosmer, D. & Lemeshow, S. Applied Logistic Regression 2nd edn (Wiley, 2000).

  39. Xing, F. & Yang, L. Robust nucleus/cell detection and segmentation in digital pathology and microscopy images: a comprehensive review. IEEE Rev. Biomed. Eng.9, 234–263 (2016).

    Article  Google Scholar 

  40. Cuevas, E. et al. White blood cell segmentation by circle detection using electromagnetism-like optimization. Comput. Math. Methods Med.2013, 395071 (2013).

    MathSciNet  Google Scholar 

  41. Alomari, Y. M., Abdullah, S. N. H. S., Azma, R. Z. & Omar, K. Automatic detection and quantification of WBCs and RBCs using iterative structured circle detection algorithm. Comput. Math. Methods Med.2014, 979302 (2014).

    Article  MATH  Google Scholar 

  42. He, K., Gkioxari, G., Dollár, P. & Girshick, R. Mask R-CNN. In Proceedings of the International Conference on Computer Vision (ICCV) 2980–2988 (IEEE, 2017).

  43. Matek, C., Schwarz, S., Spiekermann, K. & Marr, C. A single-cell morphological dataset of leukocytes from AML patients and non-malignant controls (AML-Cytomorphology_LMU). TCAI https://doi.org/10.7937/tcia.2019.36f5o9ld (2019).

  44. Matek, C., Schwarz, S., Spiekermann, K. & Marr, C. A neural network for classifying leukocyte images from blood smears. CodeOcean https://codeocean.com/capsule/9068249/tree/v1 (2019).

Download references

Acknowledgements

We thank N. Chlis for comments on the manuscript, K. Metzeler for helpful discussions and A. Holzäpfel for contributions to the annotation task. This work was supported by the German Research Foundation DFG within the Collaborative Research Center SFB 1243. C. Matek acknowledges support from Deutsche José Carreras-Leukämie Stiftung.

Author information

Author notes

  1. Joint corresponding authors: Karsten Spiekermann, Carsten Marr.

  2. These authors jointly supervised this work: Karsten Spiekermann, Carsten Marr.

Authors and Affiliations

  1. Institute of Computational Biology, Helmholtz Zentrum München – German Research Center for Environmental Health, Neuherberg, Germany

    Christian Matek & Carsten Marr

  2. Laboratory of Leukemia Diagnostics, Department of Medicine III, University Hospital, LMU Munich, Munich, Germany

    Christian Matek, Simone Schwarz & Karsten Spiekermann

  3. German Cancer Consortium (DKTK), Heidelberg, Germany

    Karsten Spiekermann

  4. German Cancer Research Center (DKFZ), Heidelberg, Germany

    Karsten Spiekermann

Authors
  1. Christian Matek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Simone Schwarz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karsten Spiekermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Carsten Marr
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C. Matek, C. Marr and K.S. conceived the initial idea. C. Matek selected the cohort, digitized blood smears, wrote annotation software, and trained and evaluated the network. S.S. contributed to selecting the cohort and annotated the image data. C. Matek, C. Marr and K.S. interpreted data and wrote the paper. All authors approved the manuscript.

Corresponding authors

Correspondence to Karsten Spiekermann or Carsten Marr.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Matek, C., Schwarz, S., Spiekermann, K. et al. Human-level recognition of blast cells in acute myeloid leukaemia with convolutional neural networks. Nat Mach Intell 1, 538–544 (2019). https://doi.org/10.1038/s42256-019-0101-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-019-0101-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing