Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Brief Communication
  • Published:

Appropriateness of ophthalmic symptoms triage by a popular online artificial intelligence chatbot

Eye volume 37, pages 3692–3693 (2023)Cite this article

Subjects

To determine the importance of symptoms, patients send messages to providers and search the internet for answers. Although professional advice is ideal, delays exist between when a patient message is received and when a response can be sent. A popular online chatbot released by OpenAI in November 2022 has garnered widespread attention in academia and social media based on its ability to simulate natural language and demonstrate intelligence [1, 2]. ChatGPT is free, open-access, responds in real-time, provides human-like responses, and even provides appropriate cardiovascular disease prevention recommendations [3]. We evaluated the ability of this artificial intelligence (AI) to respond to prompts concerning common ocular symptoms.

This study was performed in February-March 2023. We scripted ten prompts reflective of common patient messages relating to conditions including posterior vitreous detachment, retinal tear and detachment, ocular surface disease, exudative age-related macular degeneration (eAMD), and post-intravitreal injection pain and redness. Each question was submitted in three individual chats to chatGPT and the responses were recorded. Each set of responses was graded individually and separately by five retinal specialists. Reviewers evaluated each set as “precise” or “imprecise” based on similarity in content across the three answers, and either “suitable” or “unsuitable” based on whether answers reflected adequate clinical responses to real-world scenarios. If any of the responses contained inaccurate or irrelevant content, the set was deemed “unsuitable.” Each set was given the majority reviewer grade for precision and suitability.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Data availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on request.

References

  1. ChatGPT: optimizing language models for dialogue. Accessed February, 2023. https://openai.com/blog/chatgpt.

  2. Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor RA, et al. How Does ChatGPT perform on the united states medical licensing examination? the implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9:e45312.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Sarraju A, Bruemmer D, Van Iterson E, Cho L, Rodriguez F, Laffin L. Appropriateness of cardiovascular disease prevention recommendations obtained from a popular online chat-based artificial intelligence model. JAMA [Internet]. 2023;329:842. Available from: https://jamanetwork.com/journals/jama/fullarticle/2801244.

  4. National and Regional Projections of Supply and Demand for Surgical Specialty Practitioners: 2013-2025 [Internet]. U.S. Department of Health and Human Services Health Resources and Services Administration Bureau of Health Workforce National Center for Health Workforce Analysis; 2016. Available from: https://bhw.hrsa.gov/sites/default/files/bureau-health-workforce/data-research/surgical-specialty-report.pdf.

  5. Linzer M, Jin JO, Shah P, Stillman M, Brown R, Poplau S, et al. Trends in clinician burnout with associated mitigating and aggravating factors during the COVID-19 pandemic. JAMA Health Forum. 2022;3:e224163.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Scheie Eye Institute, Department of Ophthalmology, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA

    Jonathan C. Tsui, Michael B. Wong, Benjamin J. Kim, Albert M. Maguire, Drew Scoles, Brian L. VanderBeek & Alexander J. Brucker

Authors
  1. Jonathan C. Tsui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael B. Wong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamin J. Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Albert M. Maguire
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Drew Scoles
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Brian L. VanderBeek
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Alexander J. Brucker
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JCT had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. MBW was responsible for reviewing the data and statistical analysis. BJK, AMM, DS, and AJB were responsible for data and critical review of the manuscript. BVB was responsible for data, critical review of the manuscript, and statistical analysis. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Jonathan C. Tsui.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tsui, J.C., Wong, M.B., Kim, B.J. et al. Appropriateness of ophthalmic symptoms triage by a popular online artificial intelligence chatbot. Eye 37, 3692–3693 (2023). https://doi.org/10.1038/s41433-023-02556-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41433-023-02556-2

This article is cited by

Search

Advanced search

Quick links