Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Validation of diagnostic accuracy of retinal image grading by trained non-ophthalmologist grader for detecting diabetic retinopathy and diabetic macular edema

Eye volume 37, pages 1577–1582 (2023)Cite this article

Subjects

Abstract

Purpose

To validate the fundus image grading results by a trained grader (Non-ophthalmologist) and an ophthalmologist grader for detecting diabetic retinopathy (DR) and diabetic macular oedema (DMO) against fundus examination by a retina specialist (gold standard).

Methods

A prospective diagnostic accuracy study was conducted using 2002 non-mydriatic colour fundus images from 1001 patients aged ≥40 years. Using the Aravind Diabetic Retinopathy Evaluation Software (ADRES) images were graded by both a trained non-ophthalmologist grader (grader-1) and an ophthalmologist (grader-2). Sensitivity, specificity, positive predictive value and negative predictive value were calculated for grader-1 and grader-2 against the grading results by an independent retina specialist who performed dilated fundus examination for every study participant.

Results

Out of 1001 patients included, 42% were women and the mean ± (SD) age was 55.8 (8.39) years. For moderate or worse DR, the sensitivity and specificity for grading by grader-1 with respect to the gold standard was 66.9% and 91.0% respectively and the same for the ophthalmologist was 83.6% and 80.3% respectively. For referable DMO, grader-1 and grader-2 had a sensitivity of 74.6% and 85.6% respectively and a specificity of 83.7% and 79.8% respectively.

Conclusions

Our results demonstrate good level of accuracy for the fundus image grading performed by a trained non-ophthalmologist which was comparable with the grading by an ophthalmologist. Engaging trained non-ophthalmologists potentially can enhance the efficiency of DR diagnosis using fundus images. Further study with multiple non-ophthalmologist graders is needed to verify the results and strategies to improve agreement for DMO diagnosis are needed.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Flow chart describing the study procedure.

Similar content being viewed by others

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

References

  1. Sivaprasad S, Gupta B, Crosby-Nwaobi R, Evans J. Prevalence of diabetic retinopathy in various ethnic groups: a worldwide perspective. Surv Ophthalmol. 2012;57:347–70. https://doi.org/10.1016/j.survophthal.2012.01.004.

    Article  PubMed  Google Scholar 

  2. Recommendations in Prevention of Blindness from Diabetes Mellitus: report of a WHO consultation. Geneva, Switzerland 2005 World Health Organization.

  3. Yau JWY, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, et al. Global prevalence and major risk factors of diabetic retinopathy for the meta-analysis for eye disease (meta-eye) study group. Diabetes Care. 2012;35:556–64.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Anjana RM, Deepa M, Pradeepa R, Mahanta J, Narain K, Das HK, et al. ICMR–INDIAB Collaborative Study Group. Prevalence of diabetes and prediabetes (impaired fasting glucose and/or impaired glucose tolerance) in urban and rural India: phase I results of the Indian Council of Medical Research-INdia DIABetes (ICMR-INDIAB) study. Diabetologia. 2011;54:3022–7. https://doi.org/10.1007/s00125-011-2291-5.

    Article  CAS  PubMed  Google Scholar 

  5. Luhar S, Timæus IM, Jones R, Cunningham S, Patel SA, Kinra S, et al. Forecasting the prevalence of overweight and obesity in India to 2040. PLoS One. 2020;15:e0229438. https://doi.org/10.1371/journal.pone.0229438.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Guidelines for Diabetic Eye Care Adapted from ICO guideline to suit the Indian context; VISION 2020: The Right to Sight-INDIA Programme; 2015.

  7. Rani PK, Raman R, Sharma V, Mahuli SV, Tarigopala A, Sudhir RR, et al. Analysis of a comprehensive diabetic retinopathy screening model for rural and urban diabetics in developing countries. Br J Ophthalmol. 2007;91:1425–9.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Narendran V, John RK, Raghuram A, Ravindran RD, Nirmalan PK, Thulasiraj RD. Diabetic retinopathy among self-reported diabetics in southern India: a population based assessment. Br J Ophthalmol. 2002;86:1014–8.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Namperumalsamy P, Kim R, Vignesh TP, Nithya N, Royes J, Gijo T, et al. Prevalence and risk factors for diabetic retinopathy: a population-based assessment from Theni District, south India. Br J Ophthalmol. 2009;93:429–34.

    CAS  PubMed  Google Scholar 

  10. Rema M, Premkumar S, Anitha B, Deepa R, Pradeepa R, Mohan V. Prevalence of diabetic retinopathy in urban India: the Chennai Urban Rural Epidemiology Study (CURES) eye study. Invest Ophthalmol Vis Sci. 2005;46:2328–33.

    Article  PubMed  Google Scholar 

  11. Ramasamy K, Raman R, Tandon M. Current state of care for diabetic retinopathy in India. Curr Diab Rep. 2013;13:460–8. https://doi.org/10.1007/s11892-013-0388-6.

    Article  PubMed  Google Scholar 

  12. Liesenfeld B, Kohner E, Piehlmeier W, Kluthe S, Aldington S, Porta M, et al. A telemedical approach to the screening of diabetic retinopathy digital fundus photography. Diabetes Care. 2000;23:345–48.

    Article  CAS  PubMed  Google Scholar 

  13. Rachapelle S, Legood R, Alavi Y, Lindfield R, Sharma T, Kuper H, et al. The cost-utility of telemedicine to screen for diabetic retinopathy in India. Ophthalmology. 2013;120:566–73.

    Article  PubMed  Google Scholar 

  14. Joseph S, Ravilla T, Bassett K. Gender issues in a cataract surgical population in South India. Ophthalmic Epidemiol. 2013;20:96–101.

    Article  PubMed  Google Scholar 

  15. Ruamviboonsuk P, Teerasuwanajak K, Tiensuwan M, Yuttitham K, Thai Screening for Diabetic Retinopathy Study Group. Interobserver agreement in the interpretation of singlefield digital fundus images for diabetic retinopathy screening. Ophthalmology 2006;113:826e832.

    Article  Google Scholar 

  16. Li Z, Wu C, Olayiwola JN, Hilaire DS, Huang JJ. Telemedicine-based digital retinal imaging vs standard ophthalmologic evaluation for the assessment of diabetic retinopathy. Conn Med. 2012;76:85–90.

    PubMed  Google Scholar 

  17. Gangaputra S, Lovato JF, Hubbard L, Davis MD, Esser BA, Ambrosius WT, et al. Comparison of standardized clinical classification with fundus photograph grading for the assessment of diabetic retinopathy and diabetic macular edema severity. Retina 2013;33:1393e1399.

    Article  Google Scholar 

  18. Scanlon PH. The English National Screening Programme for diabetic retinopathy 2003–16. Acta Diabetol. 2017;54:515–25. https://doi.org/10.1007/s00592-017-0974-1.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Kumar S, Kumar G, Velu S, Pardhan S, Sivaprasad S, Ruamviboonsuk P, et al. Patient and provider perspectives on barriers to screening for diabetic retinopathy: an exploratory study from southern India. BMJ Open 2020;10:e037277 https://doi.org/10.1136/bmjopen-2020-037277.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Van Allen Z, Dogba MJ, Brent MH, Bach C, Grimshaw JM, Ivers NM, et al. Barriers to and enablers of attendance at diabetic retinopathy screening experienced by immigrants to Canada from multiple cultural and linguistic minority groups. Diabet Med. 2021;38:e14429. https://doi.org/10.1111/dme.14429.

    Article  PubMed  Google Scholar 

  21. Kashim RM, Newton P, Ojo O. Diabetic retinopathy screening: a systematic review on patients’ non-attendance. Int J Environ Res Public Health 2018;15:157. https://doi.org/10.3390/ijerph15010157.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Piyasena MMPN, Murthy GVS, Yip JLY, Gilbert C, Zuurmond M, Peto T, et al. Systematic review on barriers and enablers for access to diabetic retinopathy screening services in different income settings. PLoS One 2019;14:e0198979. https://doi.org/10.1371/journal.pone.0198979.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Teo ZL, Tham YC, Yu M, Cheng CY, Wong TY, Sabanayagam C. Do we have enough ophthalmologists to manage vision-threatening diabetic retinopathy? A global perspective. Eye 2020;34:1255–61. https://doi.org/10.1038/s41433-020-0776-5.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Klein R, Klein BE, Magli YL, Brothers RJ, Meuer SM, Moss SE, et al. An alternative method of grading diabetic retinopathy. Ophthalmology 1986;93:1183–7. https://doi.org/10.1016/s0161-6420(86)33606-6.

    Article  CAS  PubMed  Google Scholar 

  25. Rajalakshmi R, Prathiba V, Arulmalar S, Usha M, Review of retinal cameras for global coverage of diabetic retinopathy screening. Eye. 202; 35:162–72.

  26. American Academy of Ophthalmology; International clinical diabetic retinopathy disease severity scale, detailed table. http://www.icoph.org/dynamic/attachments/resources/diabeticretinopathy-detail.pdf.

  27. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159e174.

    Article  Google Scholar 

  28. Bhargava M, Cheung CY, Sabanayagam C, Kawasaki R, Harper CA, Lamoureux EL, et al. Accuracy of diabetic retinopathy screening by trained non-physician graders using non-mydriatic fundus camera. Singap Med J. 2012;53:715–9.

    Google Scholar 

  29. McKenna M, Chen T, McAneney H, Vázquez Membrillo MA, Jin L, Xiao W, et al. Accuracy of trained rural ophthalmologists versus non-medical image graders in the diagnosis of diabetic retinopathy in rural China. Br J Ophthalmol. 2018;102:1471–6.

    Article  PubMed  Google Scholar 

  30. Public Health England. NHS diabetic eye screening programme overview of patientpathway, grading pathway, surveillance pathways and referral pathways, 2017.

  31. Conlin PR, Fisch BM, Cavallerano AA, Cavallerano JD, Bursell S-E, Aiello LM. Nonmydriatic teleretinal imaging improves adherence to annual eye examinations in patients with diabetes. J Rehabil Res Dev. 2006;43:733–40. https://doi.org/10.1682/JRRD.2005.07.0117.

    Article  PubMed  Google Scholar 

  32. Mansberger SL, Gleitsmann K, Gardiner S, Sheppler C, Demirel S, Wooten K, et al. Comparing the effectiveness of telemedicine and traditional surveillance in providing diabetic retinopathy screening examinations: a randomized controlled trial. Telemed J E Health. 2013;19:942–8. https://doi.org/10.1089/tmj.2012.0313.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Liu SL, Mahon LW, Klar NS, Schulz DC, Gonder JR, Hramiak IM, et al. A randomised trial of non-mydriatic ultra-wide field retinal imaging versus usual care to screen for diabetic eye disease: rationale and protocol for the Clearsight trial. BMJ Open. 2017;7:e015382. https://doi.org/10.1136/bmjopen-2016-015382.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Silva PaoloS, Cavallerano JerryD, Sun JenniferK, Soliman AhmedZ, Aiello LloydM, Lloyd Paul Aiello. Peripheral lesions identified by mydriatic ultrawide field imaging: distribution and potential impact on diabetic retinopathy severity. Ophthalmology. 2013;ume 120:2587–95. https://doi.org/10.1016/j.ophtha.2013.05.004.

    Article  Google Scholar 

  35. Darwish DY, Patel SN, Gao Y, Bhat P, Chau FY, Lim JI, et al. Diagnostic accuracy and reliability of retinal pathology using the Forus 3nethra fundus camera compared to ultra wide-field imaging. Eye. 2019;33:856–7. https://doi.org/10.1038/s41433-019-0339-9.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Trevethan R. Sensitivity, specificity, and predictive values: foundations, pliabilities, and pitfalls in research and practice. Front Public Health 2017. 2017;5:307. https://doi.org/10.3389/fpubh.2017.00307.

    Article  Google Scholar 

  37. Gulshan V, Rajan RP, Widner K, Wu D, Wubbels P, Rhodes T, et al. Performance of a deep-learning algorithm vs manual grading for detecting diabetic retinopathy in India. JAMA Ophthalmol 2019;137:987–93. https://doi.org/10.1001/jamaophthalmol.2019.2004.

    Article  PubMed  PubMed Central  Google Scholar 

  38. FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems https://www.fda.gov/news-events/press-announcements/fda-permits-marketing-artificial-intelligence-based-device-detect-certain-diabetes-related-eye (Accessed 9 July, 2021).

  39. Heydon P, Egan C, Bolter L, Chambers R, Anderson J, Aldington S, et al. Prospective evaluation of an artificial intelligence-enabled algorithm for automated diabetic retinopathy screening of 30 000 patients. Br J Ophthalmol 2021;105:723–8. https://doi.org/10.1136/bjophthalmol-2020-316594.

    Article  PubMed  Google Scholar 

  40. Srinivasan NK, John D, Rebekah G, Kujur ES, Paul P, John SS. Diabetes and diabetic retinopathy: knowledge, attitude, practice (KAP) among diabetic patients in a tertiary eye care centre. J Clin Diagn Res. 2017;11:NC01–NC07.

    PubMed  PubMed Central  Google Scholar 

  41. Gilbert CE, Babu RG, Gudlavalleti AS, Anchala R, Shukla R, Ballabh PH, et al. Eye care infrastructure and human resources for managing diabetic retinopathy in India: The India 11-city 9-state study. Indian J Endocrinol Metab. 2016;20:S3–S10.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Roglic G Global report on diabetes. Geneva, Switzerland: World Health Organization, 2016.

Download references

Acknowledgements

The authors acknowledge the cooperation and support from the staff and management of Aravind Eye Hospital, Madurai, India during the study.

Funding

This study was internally funded by the Aravind Eye Care System and the personnel dealing with funds allocation did not have any influence on the study procedures or interpretation of the results.

Author information

Authors and Affiliations

  1. Lions Aravind Institute of Community Ophthalmology, Aravind Eye Care System, Madurai, India

    Sanil Joseph & Soundarya Venkatachalam

  2. Centre for Eye Research Australia, Royal Victorian Eye and Ear Hospital, East Melbourne, Vic, Australia

    Sanil Joseph

  3. Department of Surgery (Ophthalmology), The University of Melbourne, Melbourne, Vic, Australia

    Sanil Joseph

  4. Aravind Eye Hospital and Postgraduate Institute of Ophthalmology, Madurai, India

    Renu P. Rajan, Balagiri Sundar & Ramasamy Kim

  5. Department of Ophthalmology, , Massachusetts Eye and Ear and Harvard Medical School; Schepens Eye Research Institute, Boston, MA, USA

    John H. Kempen

  6. MCM Eye Unit, MyungSung Christian Medical Center (MCM) Multispecialty Hospital and MyungSung Medical School, Addis Ababa, Ethiopia

    John H. Kempen

  7. Department of Ophthalmology, Faculty of Medicine, Addis Ababa University, Addis Ababa, Ethiopia

    John H. Kempen

Authors
  1. Sanil Joseph
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Renu P. Rajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Balagiri Sundar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Soundarya Venkatachalam
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. John H. Kempen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ramasamy Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SJ, RK and RPR were responsible for conception, design and implementation of the study. SV contributed to data management and manuscript preparation. SJ and BS were responsible for data analysis and interpretation of the results. SJ led preparation of the manuscript with contribution from all co-authors. JHK did a thorough review of the final draft of the manuscript and contributed substantially in finalizing it.

Corresponding author

Correspondence to Ramasamy Kim.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Joseph, S., Rajan, R.P., Sundar, B. et al. Validation of diagnostic accuracy of retinal image grading by trained non-ophthalmologist grader for detecting diabetic retinopathy and diabetic macular edema. Eye 37, 1577–1582 (2023). https://doi.org/10.1038/s41433-022-02190-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41433-022-02190-4

Search

Advanced search

Quick links