Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

Not all AI health tools with regulatory authorization are clinically validated

Nature Medicine (2024)Cite this article

Subjects

An Author Correction to this article was published on 16 September 2024

Devices that lack adequate clinical validation pose risks for patient care. A new validation standard is proposed to evaluate FDA authorization as an indication of clinical effectiveness in medical AI.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Validation methods for FDA-authorized AI devices by specialty.
Fig. 2: Validation methods for FDA-authorized AI devices over time.

References

  1. Murphy, K. et al. BMC Med. Ethics 22, 14 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  2. Kim, J. Y. et al. In FAccT '23: Proc. 2023 ACM Conference on Fairness, Accountability, and Transparency 1396–1417 (2023).

  3. Geis, J. R. et al. Can. Assoc. Radiol. J. 70, 329–334 (2019).

    Article  PubMed  Google Scholar 

  4. Evans, B. J. J. Law Biosci. 10, lsad001 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Vellido, A. Kidney Dis. 5, 11–17 (2018).

    Article  Google Scholar 

  6. Center for Devices and Radiological Health. FDA https://go.nature.com/4dDbUct (6 September 2023).

  7. US Food and Drug Administration. https://go.nature.com/3YN8Civ (14 October 2016).

  8. US Food and Drug Administration. https://go.nature.com/3LYd32i (September 2023).

  9. US Food and Drug Administration. https://go.nature.com/3SOXHRw (accessed 17 September 2023).

  10. Lyell, D., Wang, Y., Coiera, E. & Magrabi, F. J. Am. Med. Inform. Assoc. 30, 1227–1236 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  11. US Food and Drug Administration. https://go.nature.com/3YwMt87 (accessed 14 June 2023).

  12. Kalantaridis, C. & Küttim, M. J. Technol. Transfer 46, 1487–1513 (2021).

    Article  Google Scholar 

  13. MIT News Office. https://go.nature.com/3LVzBkb (7 September 2017).

  14. Sandhu, S. et al. Patterns 4, 100710 (2023).

    Article  PubMed  PubMed Central  Google Scholar 

  15. Tierney, W. M., Meslin, E. M. & Kroenke, K. J. Gen. Intern. Med. 31, 228–233 (2016).

    Article  PubMed  Google Scholar 

  16. Harris, P., Winney, I., Ashhurst-Smith, C., O'Brien, M. & Graves, S. Pathology 44, 583–585 (2012).

    Article  PubMed  Google Scholar 

  17. Wu, E. et al. Nat. Med. 27, 582–584 (2021).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

For their consultation, comments and mentorship, the authors thank M. R. Patel, W. Ratliff, Z. Hussain, J. Castillo, I. Yüksel, H. Chen, P. Guo, H. McInnis, J. E. Hancock, B. J. Evans and A. Merlo.

Author information

Author notes

  1. These authors contributed equally: Adonis Abdullah, Ying Fang.

Authors and Affiliations

  1. UNC School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Sammy Chouffani El Fassi, Ying Fang & Naya Kayali

  2. Duke Heart Center, Duke University School of Medicine, Durham, NC, USA

    Sammy Chouffani El Fassi

  3. Portfolio Credit Strategy, Ally Bank, Charlotte, NC, USA

    Adonis Abdullah

  4. University of South Carolina School of Medicine, Columbia, SC, USA

    Ying Fang

  5. Department of Pharmacology, University of Oxford, Oxford, UK

    Sarabesh Natarajan

  6. Columbia University Vagelos College of Physicians and Surgeons, Columbia University, New York, NY, USA

    Sarabesh Natarajan

  7. Department of Biomedical Engineering, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Awab Bin Masroor

  8. ECU School of Dental Medicine, East Carolina University, Greenville, NC, USA

    Naya Kayali

  9. University of Miami Miller School of Medicine, University of Miami, Miami, FL, USA

    Simran Prakash

  10. Department of Social Medicine, School of Medicine, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

    Gail E. Henderson

Authors
  1. Sammy Chouffani El Fassi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adonis Abdullah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sarabesh Natarajan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Awab Bin Masroor
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Naya Kayali
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Simran Prakash
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Gail E. Henderson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sammy Chouffani El Fassi.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chouffani El Fassi, S., Abdullah, A., Fang, Y. et al. Not all AI health tools with regulatory authorization are clinically validated. Nat Med (2024). https://doi.org/10.1038/s41591-024-03203-3

Download citation

  • Published:

  • DOI: https://doi.org/10.1038/s41591-024-03203-3

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing