Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Matters Arising
  • Published:

Reply to: ‘Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms’ and ‘Confounding factors need to be accounted for in assessing bias by machine learning algorithms’

Nature Medicine volume 28, pages 1161–1162 (2022)Cite this article

Subjects

The Original Article was published on 16 June 2022

The Original Article was published on 16 June 2022

replying to Bernhardt, M., Jones, C. & Glocker, B. Nature Medicine https://doi.org/10.1038/s41591-022-01846-8 (2022) and Mukherjee, P. at al. Nature Medicine https://doi.org/10.1038/s41591-022-01847-7 (2022)

In our Article1 we demonstrate that widespread, state-of-the-art machine learning (ML) techniques for diagnostic models over chest radiographs produce results that are biased to predict too frequently that patients in select minority subgroups are healthy when, in fact, they are not. This kind of bias in particular, which in our work we refer to as ‘underdiagnosis bias’, presents both major social and ethical concerns because underdiagnosis can lead to inappropriate or delayed care. In their comments, Bernhardt et al.2 and Mukherjee et al.3 both argue that the utility of our work is limited given the difficulty in fully uncovering the true cause of the disparities we demonstrate in these models. We thank both Bernhardt et al. and Mukherjee et al. and welcome increased discussion around these important questions. Furthermore, we agree with these authors in that understanding the root cause of bias in ML applications is a key step towards eliminating that bias. However, we believe that an even more fundamental step is noting that this bias exists in current best practices, and this is where our work provides its value. In particular, we have shown that following the standard approach4 to training diagnostic models over several heavily used chest X-ray datasets5,6,7 yields results that show significant underdiagnosis rates specifically for historically underserved patients, both at the subpopulation level (for example, Hispanic patients) and at the intersectional level (for example, Hispanic female patients)1. While we agree that further investigation is needed to discover the true, underlying cause of these biases, our work does demonstrate that these biases exist and that this problem needs to be addressed before any of the many prominent ML diagnostic models4 under research could progress to clinical deployment. Below we offer further, specific commentary on the important questions raised by Bernhardt et al. and Mukherjee et al.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Seyyed-Kalantari, L. et al. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021).

    Article  CAS  Google Scholar 

  2. Bernhardt, M., Jones, C. & Glocker, B. Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms. Preprint at https://arxiv.org/abs/2201.07856 (2022).

  3. Mukherjee, P. et al. Confounding factors need to be accounted for in assessing bias by machine learning algorithms (2022).

  4. Rajpurkar, P. et al. CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. Preprint at https://arxiv.org/abs/1711.05225 (2017).

  5. Irvin, J. et al. CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In Proc. AAAI Conf. Artif. Intell. 33, 590–597 (AAAI 2019).

  6. Johnson, A. E. W. et al. MIMIC-CXR: a de-identified publicly available database of chest radiographs with free-text reports. Sci. Data 6, 317 (2019).

    Article  Google Scholar 

  7. Wang, X. et al. ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 3462–3471 (IEEE, 2017); https://doi.org/10.1109/CVPR.2017.369

  8. Smit, A. et al. CheXbert: combining automatic labelers and expert annotations for accurate radiology report labeling using BERT. In Proc. of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) 1500–1519 (EMNLP 2020).

  9. Seyyed-Kalantari, L. et al. CheXclusion: fairness gaps in deep chest X-ray classifiers. In Pacific Symposium on Biocomputing 2021 (eds Altman, R. B. et al.) 232–243 (World Scientific Publishing, 2021).

  10. Buolamwini, J. & Gebru, T. Gender shades: intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research https://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf (2018).

Download references

Author information

Authors and Affiliations

  1. University of Toronto, Toronto, Ontario, Canada

    Laleh Seyyed-Kalantari

  2. Vector Institute, Toronto, Ontario, Canada

    Laleh Seyyed-Kalantari

  3. Massachusetts Institute of Technology, Cambridge, MA, USA

    Haoran Zhang, Matthew B. A. McDermott, Irene Y. Chen & Marzyeh Ghassemi

Authors
  1. Laleh Seyyed-Kalantari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haoran Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew B. A. McDermott
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Irene Y. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marzyeh Ghassemi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

L.S.-K., H.Z., M.B.A.M., I.Y.C. and M.G. have substantially contributed to the underlying research and drafting of the paper.

Corresponding author

Correspondence to Laleh Seyyed-Kalantari.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seyyed-Kalantari, L., Zhang, H., McDermott, M.B.A. et al. Reply to: ‘Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms’ and ‘Confounding factors need to be accounted for in assessing bias by machine learning algorithms’. Nat Med 28, 1161–1162 (2022). https://doi.org/10.1038/s41591-022-01854-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-022-01854-8

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing