Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

Response to correspondence regarding “Analysis of large-language model versus human performance for genetics questions”

European Journal of Human Genetics volume 32, pages 379–380 (2024)Cite this article

Subjects

We appreciate the interesting commentaries on our recent work regarding ChatGPT and genetics questions [1, 2]. It is encouraging that our paper has garnered attention. This is perhaps unsurprising, since tools—we use that term very intentionally—like ChatGPT are receiving so much scrutiny in the scientific and lay press, as well as via government scrutiny.

Without responding to all points, we did want to briefly emphasize a few thoughts here. First, and perhaps most importantly, we want to be careful to mention that we did not claim that ChatGPT understands genetics as a clinician (or other human) would. Our introduction section states that “LLMs like ChatGPT use a specific type of DL called a transformer. The model is trained on a large text dataset from sources like books, articles, and websites, and learns to predict the next word in a set of words following a prompt like a question”. Here, our intention indicates that LLMs, including ChatGPT, are based on the transformer architecture, which is a mathematical model. Hence, being built from a mathematical model, ChatGPT is meant to predict the next set of words conditioned on a user’s text-based input. We agree that it is not accurate to say that ChatGPT can understand existing ideas and/or extrapolate new ideas like a person. In the discussion section, we further wrote “ChatGPT performed better with memorization versus critical thinking questions. This aligns with the consensus online and the theory of the LLM, where the observation is that AI only learns the distribution of words and is not better than humans at performing tasks traditionally considered in the realm of critical thinking”.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Duong D, Solomon BD. Analysis of large-language model versus human performance for genetics questions. Eur J Hum Genet. 2023. https://doi.org/10.1038/s41431-023-01396-8. Epub ahead of print.

  2. Emmert-Streib F. Can ChatGPT understand genetics? Eur J Hum Genet. 2023. https://doi.org/10.1038/s41431-023-01419-4. Epub ahead of print.

  3. Singhal K, Azizi S, Tu T, Mahdavi S, Wei J, Won Chung H, et al. Large language models encode clinical knowledge. arXiv preprint. arXiv:221213138. 2022.

  4. Solomon BD. Medical genetics and genomics: questions for board review. Hoboken: Wiley; 2022.

Download references

Funding

This work was supported by the Intramural Research Program of the National Human Genome Research Institute, National Institutes of Health.

Author information

Authors and Affiliations

  1. Medical Genomics Unit, Medical Genetics Branch, National Human Genome Research Institute, Bethesda, MD, USA

    Dat Duong & Benjamin D. Solomon

Authors
  1. Dat Duong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benjamin D. Solomon
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin D. Solomon.

Ethics declarations

Competing interests

The authors receive salary and research support from the intramural program of the National Human Genome Research Institute. Benjamin D. Solomon is the co-Editor-in-Chief of the American Journal of Medical Genetics, and published some of the questions mentioned in the original study in a book, as well as others [4]. Both editing/publishing activities are conducted as an approved outside activity, separate from his US Government role.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Duong, D., Solomon, B.D. Response to correspondence regarding “Analysis of large-language model versus human performance for genetics questions”. Eur J Hum Genet 32, 379–380 (2024). https://doi.org/10.1038/s41431-023-01444-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41431-023-01444-3

Search

Advanced search

Quick links