Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Medical imaging data science competitions should report dataset demographics and evaluate for bias

Nature Medicine volume 29, pages 1038–1039 (2023)Cite this article

Subjects

Artificial intelligence (AI) has promise for automating the diagnosis of diseases from medical imaging, with some deep learning models demonstrating performance comparable to that of expert physicians1. However, AI models have also demonstrated biases due to the under-representation of certain demographic groups in the datasets used to develop them2, which have the potential to exacerbate pre-existing racial and gender disparities in clinical care2,3. Identifying potential biases in the early stages of dataset and model development should be standard practice.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

References

  1. Pan, I. et al. Am. J. Roentgenol. 213, 568–574 (2019).

    Article  Google Scholar 

  2. Larrazabal, A. J. et al. Proc. Natl Acad. Sci. USA 117, 12592–12594 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Seyyed-Kalantari, L. et al. Nat. Med. 27, 2176–2182 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Howard, F. M. et al. Nat. Commun. 12, 4423 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Daneshjou, R. et al. Sci. Adv. 8, eabq6147 (2022).

    Article  PubMed  PubMed Central  Google Scholar 

  6. Beheshtian, E. et al. Radiology 306, 220505 (2022).

    Article  Google Scholar 

  7. Tolkachev, A. et al. IEEE J. Biomed. Health Inform. 25, 1660–1672 (2021).

    Article  PubMed  Google Scholar 

  8. Chen, J. et al. In Proc. Conference on Fairness, Accountability, and Transparency 339–348 (Association for Computing Machinery, 2019).

  9. Bharti, B. et al. Preprint at arXiv, https://doi.org/10.48550/arXiv.2207.12497 (2022).

Download references

Author information

Authors and Affiliations

  1. University of Maryland Medical Intelligent Imaging (UM2ii) Center, Department of Diagnostic Radiology and Nuclear Medicine, University of Maryland School of Medicine, Baltimore, MD, USA

    Sean P. Garin, Vishwa S. Parekh & Paul H. Yi

  2. Department of Biomedical Engineering, Johns Hopkins University, Baltimore, MD, USA

    Jeremias Sulam

Authors
  1. Sean P. Garin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vishwa S. Parekh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeremias Sulam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paul H. Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul H. Yi.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Garin, S.P., Parekh, V.S., Sulam, J. et al. Medical imaging data science competitions should report dataset demographics and evaluate for bias. Nat Med 29, 1038–1039 (2023). https://doi.org/10.1038/s41591-023-02264-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-023-02264-0

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing