Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Can AI chatbots accurately answer patient questions regarding vasectomies?

International Journal of Impotence Research (2024)Cite this article

Subjects

Abstract

Artificial Intelligence (AI) has revolutionized the healthcare industry. There have been limited studies assessing AI model efficacy and accuracy in urology. To our knowledge, there is a lack in research looking at one of the most common urological procedures: the vasectomy. Ten frequently asked questions regarding vasectomies were individually entered into three different AI sources (ChatGPT, Bard & Bing) using free interfaces available to consumers. The responses were critically analyzed by three urologists and graded on a scale of 1 to 4 for clarity, accuracy, and evidence-based information, with 1 being the best and 4 being the worst. ChatGPT had the best average rating per question at 1.367, followed by Bard at 2.167 and Bing at 1.800(p = 0.000083). ChatGPT was found to provide significantly more satisfactory answers than both Bard (p = 0.00005) and Bing (p = 0.03988). The difference between Bard and Bing however was found to be insignificant (p = 0.09651). Overall, our study shows that AI Chatbots may provide mostly accurate information on frequently asked questions regarding vasectomies and is a reasonable resource for patients interested in the procedure to use. ChatGPT is the most accurate and concise of the chatbots assessed.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. 2023;29:1930–40.

    Article  CAS  PubMed  Google Scholar 

  2. Shewale R. ChatGPT Statistics For 2024 (Users Demographics & Facts). 2024. https://www.demandsage.com/chatgpt-statistics/#:~:text=According%20to%20the%20most%20recent,visits%20recorded%20in%20December%202022.

  3. Walker D, Adebajo A, Bukhari M. The benefits and challenges of providing patient education digitally. Rheumatology. 2020;59:3591–2.

    Article  PubMed  Google Scholar 

  4. Kianian R, Hu M-YY, Lavold AJ, Andino JJ, Morrison JC, Eleswarapu SV, et al. Patient-Directed Vasectomy Information: How Readable Is It? World J Mens Health. 2024;42:408–14. https://doi.org/10.5534/wjmh.230033.

    Article  PubMed  Google Scholar 

  5. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health. 2023;2:e0000198.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Sallam M. ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare. 2023;11:887. https://doi.org/10.3390/healthcare11060887.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Pan A, Musheyev D, Loeb S, Kabarriti AE. Quality of erectile dysfunction information from ChatGPT and other artificial intelligence chatbots. BJU Int. 2024;133:152–4.

    Article  PubMed  Google Scholar 

  8. Pan A, Musheyev D, Bockelman D, Loeb S, Kabarriti AE. Assessment of Artificial Intelligence Chatbot Responses to Top Searched Queries About Cancer. JAMA Oncol. 2023;9:1437–40.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Dubin JM, Aguiar JA, Lin JS, Greenberg DR, Keeter MK, Fantus RJ, et al. The broad reach and inaccuracy of men’s health information on social media: analysis of TikTok and Instagram. Int J Impot Res. 2024;36:256–60.

    Article  PubMed  Google Scholar 

  10. Mika AP, Martin JR, Engstrom SM, Polkowski GG, Wilson JM. Assessing ChatGPT Responses to Common Patient Questions Regarding Total Hip Arthroplasty. JBJS. 2023;105:1519–26. https://doi.org/10.2106/JBJS.23.00209.

    Article  Google Scholar 

  11. Bole R, Lundy SD, Pei E, Bajic P, Parekh N, Vij SC. Rising vasectomy volume following reversal of federal protections for abortion rights in the United States. Int J Impot Res. 2023;36:265–8.

    Article  PubMed  Google Scholar 

  12. ChatGPT Statistics 2024: How Many People Use ChatGPT? 2024. https://backlinko.com/chatgpt-stats. Accessed 6 Jul 2024.

  13. Tyson A, Pasquini G, Spencer A, Funk C. 60% of Americans Would Be Uncomfortable With Provider Relying on AI in Their Own Health Care. 2023. https://policycommons.net/artifacts/3453213/60-of-americans-would-be-uncomfortable-with-provider-relying-on-ai-in-their-own-health-care/4253539/.

  14. Ayers JW, Poliak A, Dredze M, Leas EC, Zhu Z, Kelley JB, et al. Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum. JAMA Intern Med. 2023;183:589–96. https://doi.org/10.1001/jamainternmed.2023.1838.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Almaazmi S, Simsekler MCE, Henschel A, Qazi A, Marbouh D, Luqman RAMA. Evaluating Drivers of the Patient Experience Triangle: Stress, Anxiety, and Frustration. Int J Environ Res Public Health. 2023;20:5384. https://doi.org/10.3390/ijerph20075384.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Duong D. Time to embrace Dr. Google? CMAJ. 2021;193:571–2. https://doi.org/10.1503/cmaj.1095935.

    Article  Google Scholar 

  17. Griffith DM. Promoting Men’s Health Equity. Am J Mens Health. 2020;14. https://doi.org/10.1177/1557988320980184.

  18. Razdan S, Siegal AR, Brewer Y, Sljivich M, Valenzuela RJ. Assessing ChatGPT’s ability to answer questions pertaining to erectile dysfunction: can our patients trust it? Int J Impot Res. 2023;1–7. https://doi.org/10.1038/s41443-023-00797-z.

Download references

Acknowledgements

Drs. Kevin Chu, MD, Utsav Bansal, MD and Justin M. Dubin, MD, gave critical advice and direction for this manuscript.

Author information

Authors and Affiliations

  1. Charles E. Schmidt Florida Atlantic University College of Medicine, Boca Raton, FL, USA

    Edwin Mouhawasse, Christopher W. Haff, Preet Kumar & Benjamin Lack

  2. Advanced Urology, Los Angeles, CA, USA

    Kevin Chu

  3. Division of Urology, Department of Surgery, Memorial Healthcare System, Hollywood, FL, 33021, USA

    Utsav Bansal & Justin M. Dubin

Authors
  1. Edwin Mouhawasse
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christopher W. Haff
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Preet Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Benjamin Lack
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kevin Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Utsav Bansal
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Justin M. Dubin
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

EM, CH, PK, BL, JMD conceived/designed the work that led to submission, acquired data, and drafted/revised the manuscript. KC, UB, JMD played an important role in interpreting the results. JMD approved the final version.

Corresponding author

Correspondence to Edwin Mouhawasse.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mouhawasse, E., Haff, C.W., Kumar, P. et al. Can AI chatbots accurately answer patient questions regarding vasectomies?. Int J Impot Res (2024). https://doi.org/10.1038/s41443-024-00970-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41443-024-00970-y

Search

Advanced search

Quick links