Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Reply to: ‘Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms’ and ‘Confounding factors need to be accounted for in assessing bias by machine learning algorithms’

Nature Medicine volume 28, pages 1161–1162 (2022)Cite this article

Subjects

The Original Article was published on 16 June 2022

The Original Article was published on 16 June 2022

replying to Bernhardt, M., Jones, C. & Glocker, B. Nature Medicine https://doi.org/10.1038/s41591-022-01846-8 (2022) and Mukherjee, P. at al. Nature Medicine https://doi.org/10.1038/s41591-022-01847-7 (2022)

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

Buy

All prices are NET prices.

References

  1. Seyyed-Kalantari, L. et al. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021).

    CAS  Article  Google Scholar 

  2. Bernhardt, M., Jones, C. & Glocker, B. Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms. Preprint at https://arxiv.org/abs/2201.07856 (2022).

  3. Mukherjee, P. et al. Confounding factors need to be accounted for in assessing bias by machine learning algorithms (2022).

  4. Rajpurkar, P. et al. CheXNet: radiologist-level pneumonia detection on chest X-rays with deep learning. Preprint at https://arxiv.org/abs/1711.05225 (2017).

  5. Irvin, J. et al. CheXpert: a large chest radiograph dataset with uncertainty labels and expert comparison. In Proc. AAAI Conf. Artif. Intell. 33, 590–597 (AAAI 2019).

  6. Johnson, A. E. W. et al. MIMIC-CXR: a de-identified publicly available database of chest radiographs with free-text reports. Sci. Data 6, 317 (2019).

    Article  Google Scholar 

  7. Wang, X. et al. ChestX-ray8: hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 3462–3471 (IEEE, 2017); https://doi.org/10.1109/CVPR.2017.369

  8. Smit, A. et al. CheXbert: combining automatic labelers and expert annotations for accurate radiology report labeling using BERT. In Proc. of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) 1500–1519 (EMNLP 2020).

  9. Seyyed-Kalantari, L. et al. CheXclusion: fairness gaps in deep chest X-ray classifiers. In Pacific Symposium on Biocomputing 2021 (eds Altman, R. B. et al.) 232–243 (World Scientific Publishing, 2021).

  10. Buolamwini, J. & Gebru, T. Gender shades: intersectional accuracy disparities in commercial gender classification. Proceedings of Machine Learning Research https://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf (2018).

Download references

Author information

Authors and Affiliations

  1. University of Toronto, Toronto, Ontario, Canada

    Laleh Seyyed-Kalantari

  2. Vector Institute, Toronto, Ontario, Canada

    Laleh Seyyed-Kalantari

  3. Massachusetts Institute of Technology, Cambridge, MA, USA

    Haoran Zhang, Matthew B. A. McDermott, Irene Y. Chen & Marzyeh Ghassemi

Authors
  1. Laleh Seyyed-Kalantari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haoran Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matthew B. A. McDermott
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Irene Y. Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marzyeh Ghassemi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

L.S.-K., H.Z., M.B.A.M., I.Y.C. and M.G. have substantially contributed to the underlying research and drafting of the paper.

Corresponding author

Correspondence to Laleh Seyyed-Kalantari.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Seyyed-Kalantari, L., Zhang, H., McDermott, M.B.A. et al. Reply to: ‘Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms’ and ‘Confounding factors need to be accounted for in assessing bias by machine learning algorithms’. Nat Med 28, 1161–1162 (2022). https://doi.org/10.1038/s41591-022-01854-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41591-022-01854-8

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing