Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

AI-generated text may have a role in evidence-based medicine

Nature Medicine volume 29, pages 1593–1594 (2023)Cite this article

Subjects

Evidence-based medicine (EBM) requires the retrieval and ranking of relevant evidence by epistemological strength, to identify the most appropriate evidence to inform guidelines and policies, with a preference for robust evidence from randomized clinical trials (RCTs), systematic reviews, and meta-analyses. The explosive growth of the scientific literature and the emergence of new sources of evidence, including social media, case reports, and large-scale observational studies, as well as the free-text nature of this large body of evidence, collectively make it difficult to appraise and select the best available evidence.

Large language models (LLMs), exemplified by ChatGPT1,2, are already affecting document generation, including creating legal documents, news, and medical writing. ChatGPT can interpret the context of prompts and generate grammatically correct and semantically meaningful answers, compose essays indistinguishable from those written by humans, and author captivating medical research abstracts3. In our own experience, ChatGPT can summarize pre-written systematic reviews. However, ChatGPT tends to miss important attributes in the summary, such as failing to refer to short-term or long-term outcomes that often have varying risks. In addition, the content generated by ChatGPT has been reported to contain factual errors, incorrect data, and misrepresentations4. ChatGPT was not explicitly trained for EBM and is not intended to be used for this crucial task5. Nevertheless, there are opportunities to use LLMs such as ChatGPT in EBM as long as pitfalls can be avoided (Table 1).

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Stokel-Walker, C. Nature https://doi.org/10.1038/d41586-022-04397-7 (2022).

    Article  PubMed  Google Scholar 

  2. Stokel-Walker, C. et al. Nature 614, 214–216 (2023).

    Article  CAS  PubMed  Google Scholar 

  3. Gao, C. A. et al. npj Digit. Med. 6, 1–5 (2023).

  4. Eva, A. M. Nature 614, 224–226 (2023).

    Article  Google Scholar 

  5. Zhavoronkov, A. Nat. Med. 29, 532 (2023).

    Article  CAS  PubMed  Google Scholar 

  6. Hedderich, M. A. et al. In Proc. 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies 2545–2568 (2021).

  7. Elliot, J. et al. Nature 600, 383–385 (2021).

    Article  Google Scholar 

  8. Shekelle, P. G. et al. JAMA 286, 1461–1467 (2001).

    Article  CAS  PubMed  Google Scholar 

  9. Goldstein, A. et al. J. Am. Med. Inform. Assoc. 24, 1192–1203 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA

    Yifan Peng

  2. Department of Neurology, Dell Medical School, University of Texas at Austin, Austin, TX, USA

    Justin F. Rousseau

  3. Department of Population Health, Dell Medical School, University of Texas at Austin, Austin, TX, USA

    Justin F. Rousseau

  4. Department of Biomedical Informatics, Vagelos College of Physicians and Surgeons, Columbia University, New York, NY, USA

    Edward H. Shortliffe & Chunhua Weng

Authors
  1. Yifan Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Justin F. Rousseau
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edward H. Shortliffe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chunhua Weng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yifan Peng, Justin F. Rousseau, Edward H. Shortliffe or Chunhua Weng.

Ethics declarations

Competing interests

The authors declare no competing interests

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, Y., Rousseau, J.F., Shortliffe, E.H. et al. AI-generated text may have a role in evidence-based medicine. Nat Med 29, 1593–1594 (2023). https://doi.org/10.1038/s41591-023-02366-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-023-02366-9

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing