Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals

Nature Medicine volume 27, pages 582–584 (2021)Cite this article

Subjects

A comprehensive overview of medical AI devices approved by the US Food and Drug Administration sheds new light on limitations of the evaluation process that can mask vulnerabilities of devices when they are deployed on patients.

Medical artificial-intelligence (AI) algorithms are being increasingly proposed for the assessment and care of patients. Although the academic community has started to develop reporting guidelines for AI clinical trials1,2,3, there are no established best practices for evaluating commercially available algorithms to ensure their reliability and safety. The path to safe and robust clinical AI requires that important regulatory questions be addressed. Are medical devices able to demonstrate performance that can be generalized to the entire intended population? Are commonly faced shortcomings of AI (overfitting to training data, vulnerability to data shifts, and bias against underrepresented patient subgroups) adequately quantified and addressed?

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Breakdown of 130 FDA-approved medical AI devices by body area.

References

  1. Liu, X. et al. Nat. Med. 26, 1364–1374 (2020).

    Article  CAS  Google Scholar 

  2. Norgeot, B. et al. Nat. Med. 26, 1320–1324 (2020).

    Article  CAS  Google Scholar 

  3. Cruz Rivera, S., Liu, X. & Chan, A. W. et al. Nat. Med. 26, 1351–1363 (2020).

    Article  CAS  Google Scholar 

  4. U.S. Food & Drug Administration Center for Devices and Radiological Health. https://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm514737.pdf (October 2017).

  5. U.S. Food & Drug Administration Center for Devices and Radiological Health. https://www.fda.gov/media/145022/download (January 2021).

  6. U.S. Food & Drug Administration. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMN/denovo.cfm (accessed 14 December 2020).

  7. U.S. Food & Drug Administration. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfPMA/pma.cfm (accessed 14 December 2020).

  8. U.S. Food & Drug Administration. https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/cfpmn/pmn.cfm (accessed 14 December 2020).

  9. Benjamens, S., Dhunnoo, P. & Meskó, B. Npj Digit. Med. 3, 1–8 (2020).

    Article  Google Scholar 

  10. American College of Radiology. https://www.acrdsi.org/DSI-Services/FDA-Cleared-AI-Algorithms (accessed 29 November 2020).

  11. Kaushal, A., Altman, R. & Langlotz, C. J. Am. Med. Assoc. 324, 1212–1213 (2020).

    Article  Google Scholar 

  12. Wang, X. et al. 2017 IEEE Conference on Computer Vision and Pattern Recognition 3462–3471 (Institute of Electrical and Electronics Engineers, 2017).

  13. Irvin, J. et al. The Thirty-Third AAAI Conference on Artificial Intelligence 590–597 (Association for the Advancement of Artificial Intelligence, 2019).

  14. Johnson, A. E. W. et al. Sci. Data 6, 317 (2019).

    Article  Google Scholar 

  15. Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K.Q. 2017 IEEE Conference on Computer Vision and Pattern Recognition 4700–4708 (Institute of Electrical and Electronics Engineers, 2017).

  16. Seyyed-Kalantari, L., Liu, G., McDermott, M., Chen, I. Y. & Ghassemi, M. Pac. Symp. Biocomput. 26, 232–243 (2021).

    PubMed  Google Scholar 

  17. U.S. Food & Drug Administration. https://www.fda.gov/medical-devices/device-advice-comprehensive-regulatory-assistance/postmarket-requirements-devices (2018).

  18. Ferryman, K. J. Am. Med. Inform. Assoc. 27, 2016–2019 (2020).

    Article  Google Scholar 

Download references

Acknowledgements

J.Z. is supported by the National Science Foundation (CCF 1763191 and CAREER 1942926), the US National Institutes of Health (P30AG059307 and U01MH098953) and grants from the Silicon Valley Foundation and the Chan-Zuckerberg Initiative. R.D. is supported by the US National Institutes of Health (T32 5T32AR007422-38). Our compiled database of approved medical AI devices, analysis code, and models used for the case study are all available at https://ericwu09.github.io/medical-ai-evaluation.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Stanford University, Stanford, CA, USA

    Eric Wu & James Zou

  2. Department of Biomedical Data Science, Stanford University, Stanford, CA, USA

    Kevin Wu, Roxana Daneshjou & James Zou

  3. Department of Dermatology, Stanford University, Stanford, CA, USA

    Roxana Daneshjou

  4. Department of Cardiology, Smidt Heart Institute, Cedars-Sinai Medical Center, Los Angeles, CA, USA

    David Ouyang

  5. Stanford Law School, Stanford University, Stanford, CA, USA

    Daniel E. Ho

  6. Chan-Zuckerberg Biohub, San Francisco, CA, USA

    James Zou

Authors
  1. Eric Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roxana Daneshjou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. David Ouyang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniel E. Ho
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. James Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

E.W., K.W. and J.Z. designed the study. E.W., K.W. conducted research with help from R.D. and D.O. All the authors contributed to interpretation of the results and writing of the manuscript.

Corresponding author

Correspondence to James Zou.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, E., Wu, K., Daneshjou, R. et al. How medical AI devices are evaluated: limitations and recommendations from an analysis of FDA approvals. Nat Med 27, 582–584 (2021). https://doi.org/10.1038/s41591-021-01312-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-021-01312-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing