Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

Response to commentary on: assessing ChatGPT’s ability to answer questions pertaining to erectile dysfunction: can our patients trust it?

International Journal of Impotence Research (2024)Cite this article

Subjects

In our manuscript titled “Assessing ChatGPT’s Ability to Answer Questions Pertaining to Erectile Dysfunction: Can Our Patients Trust It?” we extensively queried ChatGPT’s ability to accurately and comprehensively respond to common patient erectile dysfunction (ED) questions [1]. Our results showed mixed accuracy, with reading levels far more advanced than what is recommended for the average patient. As a whole, we concluded that in the future, with further refinement, ChatGPT may be a useful adjunct to the urologist’s counseling on ED, but should not serve as primary patient education. This conclusion has been corroborated by other similar studies examining the performance of ChatGPT on answering andrology-related questions [2].

We appreciated the commentary by Hershenhouse et al. and agree with their assessment of our findings [3]. As they stated, any further iterations of ChatGPT or other large language models should be critically assessed using standard parameters. The authors proposed a 5-item quantitative analysis for assessing response quality: Accuracy, Completeness, Clarity, Readability, and Understandability, Empathy. While the exact specifications of each of these parameters require further explanation, the essence was captured by our initial investigation. Accuracy and Completeness were assessed by two blinded reviewers. Clarity, Readability, and Understandability were graded using the Gunning Fog Index and assigned reading scores. Empathy was also quantitatively graded by two blinded reviewers. Additionally, we concur that further assessment of ChatGPT and related platforms will require increased transparency of the sources used to derive generated responses to ensure their accuracy and legitimacy.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Razdan S, Siegal AR, Brewer Y, Sljivich M, Valenzuela RJ. Assessing ChatGPT’s ability to answer questions pertaining to erectile dysfunction: can our patients trust it? Int J Impot Res. 2023. https://doi.org/10.1038/s41443-023-00797-z.

    Article  PubMed  Google Scholar 

  2. Caglar U, Yildiz O, Ozervarli MF, Aydin R, Sarilar O, Ozgor F, et al. Assessing the performance of chat generative pretrained transformer (ChatGPT) in answering andrology-related questions. Urol Res Pract. 2023;49:365–9. https://doi.org/10.5152/tud.2023.23171

    Article  PubMed  PubMed Central  Google Scholar 

  3. Hershenhouse JS, Cacciamani GE. Comment on: Assessing ChatGPT’s ability to answer questions pertaining to erectile dysfunction. Int J Impot Res. 2024. https://doi.org/10.1038/s41443-023-00821-2 (in pre-print)

    Article  PubMed  Google Scholar 

  4. Pan A, Musheyev D, Loeb S, Kabarriti AE. Quality of erectile dysfunction information from ChatGPT and other artificial intelligence chatbots. BJU Int. 2023. https://doi.org/10.1111/bju.16209. Published online November 24

    Article  PubMed  Google Scholar 

  5. Fode M, Nolsøe AB, Jacobsen FM, Russo GI, Ostergren PB, S Jensen CF, et al. Quality of information in YouTube videos on erectile dysfunction. Sex Med. 2020;8:408–13.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Dubin JM, Aguiar JA, Lin JS, Greenberg DR, Keeter MK, Fantus RJ, et al. The broad reach and inaccuracy of men’s health information on social media: analysis of TikTok and Instagram. Int J Impot Res. 2022;19:1–5.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Urology, Icahn School of Medicine at Mount Sinai Hospital, New York, NY, 10029, USA

    Shirin Razdan & Robert J. Valenzuela

Authors
  1. Shirin Razdan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert J. Valenzuela
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SR: conceptualization, data curation, formal analysis, original draft writing, review and editing. RJV: conceptualization, review and editing, supervision.

Corresponding author

Correspondence to Shirin Razdan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Razdan, S., Valenzuela, R.J. Response to commentary on: assessing ChatGPT’s ability to answer questions pertaining to erectile dysfunction: can our patients trust it?. Int J Impot Res (2024). https://doi.org/10.1038/s41443-024-00823-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41443-024-00823-8

Search

Advanced search

Quick links