Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Frequently asked questions on erectile dysfunction: evaluating artificial intelligence answers with expert mentorship

International Journal of Impotence Research (2024)Cite this article

Subjects

Abstract

The present study assessed the accuracy of artificiaI intelligence-generated responses to frequently asked questions on erectile dysfunction. A cross-sectional analysis involved 56 erectile dysfunction-related questions searched on Google, categorized into nine sections: causes, diagnosis, treatment options, treatment complications, protective measures, relationship with other illnesses, treatment costs, treatment with herbal agents, and appointments. Responses from ChatGPT 3.5, ChatGPT 4, and BARD were evaluated by two experienced urology experts using the F1 and global quality scores (GQS) for accuracy, relevance, and comprehensibility. ChatGPT 3.5 and ChatGPT 4 achieved higher GQS than BARD in categories such as causes (4.5 ± 0.54, 4.5 ± 0.51, 3.15 ± 1.01, respectively, p < 0.001), treatment options (4.35 ± 0.6, 4.5 ± 0.43, 2.71 ± 1.38, respectively, p < 0.001), protective measures (5.0 ± 0, 5.0 ± 0, 4 ± 0.5, respectively, p = 0.013), relationships with other illnesses (4.58 ± 0.58, 4.83 ± 0.25, 3.58 ± 0.8, respectively, p = 0.006), and treatment with herbal agents (3 ± 0.61, 3.33 ± 0.83, 1.8 ± 1.09, respectively, p = 0.043). F1 scores in categories: causes (1), diagnosis (0.857), treatment options (0.726), and protective measures (1), indicated their alignment with the guidelines. There was no significant difference between ChatGPT 3.5 and ChatGPT 4 regarding answer quality, but both outperformed BARD in the GQS. These results emphasize the need to continually enhance and validate AI-generated medical information, underscoring the importance of artificiaI intelligence systems in delivering reliable information on erectile dysfunction.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Data availability

The data are freely available upon reasonable request from the corresponding authors.

References

  1. NIH Consensus Conference. Impotence. NIH Consensus Development Panel on Impotence. JAMA 1993;270:83–90.

  2. Corona G, Lee DM, Forti G, O’Connor DB, Maggi M, O’Neill TW, et al. Age-related changes in general and sexual health in middle-aged and older men: results from the European Male Ageing Study (EMAS). J Sex Med. 2010;7:1362–80.

    Article  PubMed  Google Scholar 

  3. Braun MH, Sommer F, Haupt G, Mathers MJ, Reifenrath B, Engelmann UH. Lower urinary tract symptoms and erectile dysfunction: co-morbidity or typical ‘aging male’ symptoms? Results of the ‘Cologne male survey’. Eur Urol. 2003;44:588–94.

    Article  CAS  PubMed  Google Scholar 

  4. McCarthy J, Minsky ML, Rochester N, Shannon CE. A proposal for the Dartmouth Summer Research project on artificial intelligence - August 31, 1955. AI Magazine. 2006;27:12–14

  5. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, et al. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2:230–43.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Xiong Y, Zhang Y, Zhang F, Wu C, Qin F, Yuan J. Applications of artificial intelligence in the diagnosis and prediction of erectile dysfunction: a narrative review. Int J Impot Res. 2023;35. https://doi.org/10.1038/S41443-022-00528-W.

  7. Yahya N, Ebert MA, Bulsara M, House MJ, Kennedy A, Joseph DJ, et al. Statistical-learning strategies generate only modestly performing predictive models for urinary symptoms following external beam radiotherapy of the prostate: a comparison of conventional and machine-learning methods. Med Phys. 2016;43:2040–52.

    Article  PubMed  Google Scholar 

  8. Kelly BS, Judge C, Bollard SM, Clifford SM, Healy GM, Aziz A, et al. Radiology artificial intelligence: a systematic review and evaluation of methods (RAISE). Eur Radiol. 2022;32:7998–8007.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Suarez-Ibarrola R, Hein S, Reis G, Gratzke C, Miernik A. Current and future applications of machine and deep learning in urology: a review of the literature on urolithiasis, renal cell carcinoma, and bladder and prostate cancer. World J Urol. 2020;38:2329–47.

    Article  PubMed  Google Scholar 

  10. Mohareri O, Ischia J, Black PC, Schneider C, Lobo J, Goldenberg L, et al. Intraoperative registered transrectal ultrasound guidance for robot-assisted laparoscopic radical prostatectomy. J Urol. 2015;193:302–12.

    Article  PubMed  Google Scholar 

  11. Coskun B, Ocakoglu G, Yetemen M, Kaygisiz O. Can ChatGPT, an artificial intelligence language model, provide accurate and high-quality patient information on prostate cancer? Urology. 2023. https://doi.org/10.1016/J.UROLOGY.2023.05.040.

  12. Kianian R, Carter M, Finkelshtein I, Eleswarapu SV, Kachroo N. Application of artificial intelligence to patient-targeted health information on kidney stone disease. J Renal Nutr. 2023. https://doi.org/10.1053/J.JRN.2023.10.002.

  13. Lin H, Zhao L, Wu H, Cao M, Jiang H. Sexual life and medication taking behaviours in young men: an online survey of 92 620 respondents in China. Int J Clin Pract. 2020;74. https://doi.org/10.1111/IJCP.13417.

  14. Bernard A, Langille M, Hughes S, Rose C, Leddin D, Veldhuyzen Van Zanten S. A systematic review of patient inflammatory bowel disease information resources on the World Wide Web. Am J Gastroenterol. 2007;102:2070–7.

    Article  PubMed  Google Scholar 

  15. Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks. Inf Process Manag. 2009;45:427–37.

    Article  Google Scholar 

  16. ChatGPT. OpenAI. https://openai.com/research. Accessed 24 July 2023.

  17. Gemini. https://gemini.google.com/app. Accessed 12 Feb 2024.

  18. Salonia A, Bettocchi C, Boeri L, Capogrosso P, Carvalho J, Cilesiz NC. et al. European Association of Urology guidelines on sexual and reproductive health-2021 update: male sexual dysfunction. Eur Urol. 2021;80:333–57.

    Article  PubMed  Google Scholar 

  19. OpenAI. GPT-4 Technical Report. 2023. https://arxiv.org/abs/2303.08774v3. Accessed 4 Nov 2023.

  20. Bard (chatbot) - Wikipedia. https://en.wikipedia.org/wiki/Bard_(chatbot). Accessed 1 Aug 2023.

  21. Flach PA, Kull M. Precision-recall-gain curves: PR analysis done right. Advances in neural information processing systems. 2015;28. http://www.cs. Accessed 31 Oct 2023.

  22. Hicks SA, Strümke I, Thambawita V, Hammou M, Riegler MA, Halvorsen P, et al. On evaluation metrics for medical applications of artificial intelligence. Sci Rep. 2022;12. https://doi.org/10.1038/S41598-022-09954-8.

  23. Nayan M, Salari K, Bozzo A, Ganglberger W, Lu G, Carvalho F, et al. A machine learning approach to predict progression on active surveillance for prostate cancer. Urol Oncol Semin Orig Investig. 2022;40:161.e1–161.e7.

    Google Scholar 

  24. Huang H, Xin F, Lim Y, Gu GT, Han MJ, Hao A, et al. Natural language processing in urology: automated extraction of clinical information from histopathology reports of uro-oncology procedures. Heliyon. 2023;9:2405–8440.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Urology, University of Gaziantep, Gaziantep, Turkey

    Muharrem Baturu & Omer Bayrak

  2. Department of Urology, Hasan Kalyoncu University, Gaziantep, Turkey

    Mehmet Solakhan

  3. Department of Biostatistics, University of Gaziantep, Gaziantep, Turkey

    Tanyeli Guneyligil Kazaz

Authors
  1. Muharrem Baturu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehmet Solakhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tanyeli Guneyligil Kazaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Omer Bayrak
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Concept – ÖB, MB; Design – ÖB, MS, MB; Supervision – ÖB, Resources – MS, MB; Materials-MB, ÖB, MS; Data Collection and/or Processing – MB, T; Analysis and/or Interpretation – TK, MB; Literature Search MS, ÖB, MB; Writing Manuscript – ÖB, MB; Critical Review – ÖB.

Corresponding author

Correspondence to Omer Bayrak.

Ethics declarations

Competing interests

The authors declare no competing interests.

Ethical approval

This study did not involve human or animal subjects; therefore, ethical committee approval was not required.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Baturu, M., Solakhan, M., Kazaz, T.G. et al. Frequently asked questions on erectile dysfunction: evaluating artificial intelligence answers with expert mentorship. Int J Impot Res (2024). https://doi.org/10.1038/s41443-024-00898-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1038/s41443-024-00898-3

Search

Advanced search

Quick links