Detection of anaemia from retinal fundus images via deep learning

Abstract

Owing to the invasiveness of diagnostic tests for anaemia and the costs associated with screening for it, the condition is often undetected. Here, we show that anaemia can be detected via machine-learning algorithms trained using retinal fundus images, study participant metadata (including race or ethnicity, age, sex and blood pressure) or the combination of both data types (images and study participant metadata). In a validation dataset of 11,388 study participants from the UK Biobank, the metadata-only, fundus-image-only and combined models predicted haemoglobin concentration (in g dl–1) with mean absolute error values of 0.73 (95% confidence interval: 0.72–0.74), 0.67 (0.66–0.68) and 0.63 (0.62–0.64), respectively, and with areas under the receiver operating characteristic curve (AUC) values of 0.74 (0.71–0.76), 0.87 (0.85–0.89) and 0.88 (0.86–0.89), respectively. For 539 study participants with self-reported diabetes, the combined model predicted haemoglobin concentration with a mean absolute error of 0.73 (0.68–0.78) and anaemia an AUC of 0.89 (0.85–0.93). Automated anaemia screening on the basis of fundus images could particularly aid patients with diabetes undergoing regular retinal imaging and for whom anaemia can increase morbidity and mortality risks.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Bland–Altman plot for predicted and measured Hb.
Fig. 2: Prediction of anaemia classifications.
Fig. 3: Effects of masking parts of the image on the prediction of anaemia and moderate anaemia.
Fig. 4: Effects of removing high-frequency information using Gaussian blur on the prediction of anaemia and moderate anaemia.
Fig. 5: Examples applying different explanation techniques to generate saliency maps highlighting the regions the model focuses on when predicting anaemia.

Data availability

The data supporting the findings of this study are available, with restrictions, from the UK Biobank24.

Code availability

The machine-learning models were developed by using standard model libraries and scripts in TensorFlow56. Custom code was specific to our computing infrastructure and mainly used for data input/output and parallelization across computers.

Change history

  • 12 February 2020

    An amendment to this paper has been published and can be accessed via a link at the top of the paper.

References

  1. 1.

    McLean, E., Cogswell, M., Egli, I., Wojdyla, D. & de Benoist, B. Worldwide prevalence of anaemia, WHO Vitamin and Mineral Nutrition Information System, 1993–2005. Public Health Nutr. 12, 444–454 (2008).

  2. 2.

    Stevens, G. A. et al. Global, regional, and national trends in haemoglobin concentration and prevalence of total and severe anaemia in children and pregnant and non-pregnant women for 1995–2011: a systematic analysis of population-representative data. Lancet Glob. Health 1, e16–e25 (2013).

  3. 3.

    Stoltzfus, R. J. Iron-deficiency anemia: reexamining the nature and magnitude of the public health problem. Summary: implications for research and programs. J. Nutr. 131, 697S–701S (2001).

  4. 4.

    Milman, N. Anemia—still a major health problem in many parts of the world! Ann. Hematol. 90, 369–377 (2011).

  5. 5.

    Smith, R. E. Jr. The clinical and economic burden of anemia. Am. J. Manag. Care 16 (Suppl.), S59–S66 (2010).

  6. 6.

    Shah, N., Osea, E. A. & Martinez, G. J. Accuracy of noninvasive hemoglobin and invasive point-of-care hemoglobin testing compared with a laboratory analyzer. Int. J. Lab. Hematol. 36, 56–61 (2014).

  7. 7.

    Kalantri, A., Karambelkar, M., Joshi, R., Kalantri, S. & Jajoo, U. Accuracy and reliability of pallor for detecting anaemia: a hospital-based diagnostic accuracy study. PLoS ONE 5, e8545 (2010).

  8. 8.

    Kasper, D. L. et al. Harrison’s Principles of Internal Medicine (McGraw Hill Professional, 2006).

  9. 9.

    Mannino, R. G. et al. Smartphone app for non-invasive detection of anemia using only patient-sourced photos. Nat. Commun. 9, 4924 (2018).

  10. 10.

    Barker, S. J. & Badal, J. J. The measurement of dyshemoglobins and total hemoglobin by pulse oximetry. Curr. Opin. Anaesthesiol. 21, 805–810 (2008).

  11. 11.

    Pinto, M. et al. The new noninvasive occlusion spectroscopy hemoglobin measurement method: a reliable and easy anemia screening test for blood donors. Transfusion 53, 766–769 (2013).

  12. 12.

    Wittenmeier, E. et al. Comparison of the gold standard of hemoglobin measurement with the clinical standard (BGA) and noninvasive hemoglobin measurement (SpHb) in small children: a prospective diagnostic observational study. Paediatr. Anaesth. 25, 1046–1053 (2015).

  13. 13.

    Posey, W. M. C. The ocular manifestations of anemia. JAMA XXIX, 169–171 (1897).

  14. 14.

    Aisen, M. L., Bacon, B. R., Goodman, A. M. & Chester, E. M. Retinal abnormalities associated with anemia. Arch. Ophthalmol. 101, 1049–1052 (1983).

  15. 15.

    LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

  16. 16.

    Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402–2410 (2016).

  17. 17.

    Krause, J. et al. Grader variability and the importance of reference standards for evaluating machine learning models for diabetic retinopathy. Ophthalmology 125, 1264–1272 (2018).

  18. 18.

    Ting, D. S. W. et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA 318, 2211–2223 (2017).

  19. 19.

    Liu, S. et al. A deep learning-based algorithm identifies glaucomatous discs using monoscopic fundus photographs. Ophthalmol. Glaucoma 1, 15–22 (2018).

  20. 20.

    Christopher, M. et al. Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs. Sci. Rep. 8, 16685 (2018).

  21. 21.

    Li, Z. et al. Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology 125, 1199–1206 (2018).

  22. 22.

    Varadarajan, A. V. et al. Deep learning for predicting refractive error from retinal fundus images. Invest. Ophthalmol. Vis. Sci. 59, 2861–2868 (2018).

  23. 23.

    Poplin, R. et al. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat. Biomed. Eng. 2, 158–164 (2018).

  24. 24.

    Sudlow, C. et al. UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age. PLoS Med. 12, e1001779 (2015).

  25. 25.

    Szegedy, C., Ioffe, S., Vanhoucke, V. & Alemi, A. A. Inception-v4, inception-ResNet and the impact of residual connections on learning. In Proc. of the 31st AAAI Conference on Artificial Intelligence 4278–4284 (AAAI Press, 2017).

  26. 26.

    Bland, J. M. & Altman, D. G. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1, 307–310 (1986).

  27. 27.

    Selvaraju, R. R. et al. Grad-CAM: visual explanations from deep networks via gradient-based localization.International Journal of Computer Vision https://doi.org/10.1007/s11263-019-01228-7 (2019).

  28. 28.

    Sundararajan, M., Taly, A. & Yan, Q. Axiomatic attribution for deep networks. In Proc. of the 34th International Conference on Machine Learning 3319–3328 (Microtome Publishing, 2017).

  29. 29.

    Smilkov, D., Thorat, N., Kim, B., Viégas, F. & Wattenberg, M. SmoothGrad: removing noise by adding noise. Preprint at https://arxiv.org/abs/1706.03825 (2017).

  30. 30.

    Springenberg, J. T., Dosovitskiy, A., Brox, T. & Riedmiller, M. Striving for simplicity: the all convolutional net. Preprint at https://arxiv.org/abs/1412.6806 (2014).

  31. 31.

    Bland, J. M. & Altman, D. G. Measuring agreement in method comparison studies. Stat. Methods Med. Res. 8, 135–160 (1999).

  32. 32.

    Barker, S. J., Shander, A. & Ramsay, M. A. Continuous noninvasive hemoglobin monitoring: a measured response to a critical review. Anesth. Analg. 122, 565–572 (2016).

  33. 33.

    Elliott, P. & Peakman, T. C. The UK Biobank sample handling and storage protocol for the collection, processing and archiving of human blood and urine. Int. J. Epidemiol. 37, 234–244 (2008).

  34. 34.

    Gehring, H. et al. Accuracy of point-of-care-testing (POCT) for determining hemoglobin concentrations. Acta Anaesthesiol. Scand. 46, 980–986 (2002).

  35. 35.

    Hiscock, R., Kumar, D. & Simmons, S. W. Systematic review and meta-analysis of method comparison studies of Masimo pulse co-oximeters (Radical-7TM or Pronto-7TM) and HemoCue® absorption spectrometers (B-Hemoglobin or 201+) with laboratory haemoglobin estimation. Anaesth. Intensive Care 43, 341–350 (2015).

  36. 36.

    Kim, S.-H. et al. Accuracy of continuous noninvasive hemoglobin monitoring: a systematic review and meta-analysis. Anesth. Analg. 119, 332–346 (2014).

  37. 37.

    Tsan, G. L. et al. Assessment of diabetic teleretinal imaging program at the Portland Department of Veterans Affairs Medical Center. J. Rehabil. Res. Dev. 52, 193–200 (2015).

  38. 38.

    Conlin, P. R. et al. Nonmydriatic teleretinal imaging improves adherence to annual eye examinations in patients with diabetes. J. Rehabil. Res. Dev. 43, 733–740 (2006).

  39. 39.

    Garg, S., Jani, P. D., Kshirsagar, A. V., King, B. & Chaum, E. Telemedicine and retinal imaging for improving diabetic retinopathy evaluation. Arch. Intern. Med. 172, 1677–1678 (2012).

  40. 40.

    Jones, S. C. et al. Prevalence and nature of anaemia in a prospective, population-based sample of people with diabetes: Teesside anaemia in diabetes (TAD) study. Diabet. Med. 27, 655–659 (2010).

  41. 41.

    Thomas, M. C., MacIsaac, R. J., Tsalamandris, C., Power, D. & Jerums, G. Unrecognized anemia in patients with diabetes: a cross-sectional survey. Diabetes Care 26, 1164–1169 (2003).

  42. 42.

    Wright, J. A., Oddy, M. J. & Richards, T. Presence and characterisation of anaemia in diabetic foot ulceration. Anemia 2014, 104214 (2014).

  43. 43.

    AlDallal, S. M. & Jena, N. Prevalence of anemia in type 2 diabetic patients. J. Hematol. 7, 57–61 (2018).

  44. 44.

    Mehdi, U. & Toto, R. D. Anemia, diabetes, and chronic kidney disease. Diabetes Care 32, 1320–1326 (2009).

  45. 45.

    Kliger, A. S. et al. KDOQI US commentary on the 2012 KDIGO clinical practice guideline for anemia in CKD. Am. J. Kidney Dis. 62, 849–859 (2013).

  46. 46.

    Davis, M. D. et al. Risk factors for high-risk proliferative diabetic retinopathy and severe visual loss: early treatment diabetic retinopathy study report #18. Invest. Ophthalmol. Vis. Sci. 39, 233–252 (1998).

  47. 47.

    Taylor-Phillips, S. et al. Extending the diabetic retinopathy screening interval beyond 1 year: systematic review. Br. J. Ophthalmol. 100, 105–114 (2016).

  48. 48.

    Owsley, C. et al. Diabetes eye screening in urban settings serving minority populations: detection of diabetic retinopathy and other ocular findings using telemedicine. JAMA Ophthalmol. 133, 174–181 (2015).

  49. 49.

    Scanlon, P. H. The English national screening programme for diabetic retinopathy 2003–2016. Acta Diabetol. 54, 515–525 (2017).

  50. 50.

    Das, T. & Pappuru, R. R. Telemedicine in diabetic retinopathy: access to rural India. Indian J. Ophthalmol. 64, 84–86 (2016).

  51. 51.

    American Diabetes Association. Standards of medical care in diabetes—2018 Abridged for primary care providers. Clin. Diabetes 36, 14–37 (2018).

  52. 52.

    Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N. & Folk, J. C. Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices. NPJ Digit. Med. 1, 39 (2018).

  53. 53.

    Tran, K., Mendel, T. A., Holbrook, K. L. & Yates, P. A. Construction of an inexpensive, hand-held fundus camera through modification of a consumer ‘point-and-shoot’ camera. Invest. Opthalmol. Vis. Sci. 53, 7600–7607 (2012).

  54. 54.

    Firat, P. G., Demirel, E. E., Dikci, S., Kuku, I. & Genc, O. Evaluation of iron deficiency anemia frequency as a risk factor in glaucoma. Anemia 2018, 1456323 (2018).

  55. 55.

    WHO. Haemoglobin concentrations for the diagnosis of anaemia and assessment of severity. WHO https://www.who.int/vmnis/indicators/haemoglobin.pdf (2011).

  56. 56.

    Abadi, M. et al. TensorFlow: a system for large-scale machine learning. In Proc. of the 12th USENIX Symposium on Operating Systems Design and Implementation 265–283 (USENIX Association, 2016).

  57. 57.

    Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Proc. of the 25th Conference on Advances in Neural Information Processing Systems 1097–1105 (Curran Associates, 2012).

  58. 58.

    Sutskever, I., Martens, J., Dahl, G. & Hinton, G. On the importance of initialization and momentum in deep learning. In Proc. of the 30th International Conference on Machine Learning 1139–1147 (Microtome Publishing, 2013).

  59. 59.

    Priya, G. et al. Accurate, Large Minibatch SGD: training ImageNet in 1 hour. Preprint at https://arxiv.org/abs/1706.02677 (2017).

  60. 60.

    Jouppi, N. P. et al. In-datacenter performance analysis of a tensor processing unit. In Proc. of the 44th Annual International Symposium on Computer Architecture 1–12 (ACM New York, 2017).

  61. 61.

    Caruana, R., Lawrence, S. & Giles, L. Overftting in neural nets: backpropagation, conjugate gradient, and early stopping. In Proc. of the 13th Conference on Advances in Neural Information Processing Systems 381–387 (MIT Press, 2001).

  62. 62.

    Opitz, D. & Maclin, R. Popular ensemble methods: an empirical study. J. Artif. Intell. Res. 11, 169–198 (1999).

Download references

Acknowledgements

This research was conducted using the UK Biobank Resource under application number 17643. The work of A.H. was done via Advanced Clinical, San Francisco, USA. The authors thank C. Angermueller from Google Research for his engineering contributions and A. Zaidi, A. Narayanaswamy, C. Chen, J. Krause and R. Sayres from Google Research for their advice and assistance with reviewing the manuscript.

Author information

A.M., G.S.C., L.P., D.R.W., N.H. and A.V.V. designed the research. A.M., L.P. and A.V.V. acquired data from the UK Biobank. A.M. executed the research and analysed the data. S.V. conducted the model explanation analysis. A.M., Y.L. and A.V.V. interpreted the results. A.M., A.H., Y.L. and N.H. prepared the manuscript. All authors contributed to manuscript revision and approved the submitted version.

Correspondence to Akinori Mitani.

Ethics declarations

Competing interests

The authors are employees of Google and own Alphabet stock or are working at Google. A.M., A.V.V., L.P. and D.R.W. are inventors on a patent applied by Google related to this work (current status: pending).

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary Table 1 and Supplementary Figs. 1–8.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Mitani, A., Huang, A., Venugopalan, S. et al. Detection of anaemia from retinal fundus images via deep learning. Nat Biomed Eng 4, 18–27 (2020). https://doi.org/10.1038/s41551-019-0487-z

Download citation

Further reading