Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Federated disentangled representation learning for unsupervised brain anomaly detection

Nature Machine Intelligence volume 4, pages 685–695 (2022)Cite this article

Subjects

A preprint version of the article is available at Research Square.

Abstract

With the advent of deep learning and increasing use of brain MRIs, a great amount of interest has arisen in automated anomaly segmentation to improve clinical workflows; however, it is time-consuming and expensive to curate medical imaging. Moreover, data are often scattered across many institutions, with privacy regulations hampering its use. Here we present FedDis to collaboratively train an unsupervised deep convolutional autoencoder on 1,532 healthy magnetic resonance scans from four different institutions, and evaluate its performance in identifying pathologies such as multiple sclerosis, vascular lesions, and low- and high-grade tumours/glioblastoma on a total of 538 volumes from six different institutions. To mitigate the statistical heterogeneity among different institutions, we disentangle the parameter space into global (shape) and local (appearance). Four institutes jointly train shape parameters to model healthy brain anatomical structures. Every institute trains appearance parameters locally to allow for client-specific personalization of the global domain-invariant features. We have shown that our collaborative approach, FedDis, improves anomaly segmentation results by 99.74% for multiple sclerosis, 83.33% for vascular lesions and 40.45% for tumours over locally trained models without the need for annotations or sharing of private local data. We found out that FedDis is especially beneficial for institutes that share both healthy and anomaly data, improving their local model performance by up to 227% for multiple sclerosis lesions and 77% for brain tumours.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Architecture overview.
Fig. 2: Quantitative results.
Fig. 3: Qualitative results.
Fig. 4: Effect of disease severity on performance.
Fig. 5: Insights into FedDis.

Similar content being viewed by others

Data availability

Most of the datasets used in this study are publicly available and can be downloaded after signing a Data Usage Agreement. The OASIS dataset is available at https://www.oasis-brains.org; the ADNI-S and ADNI-P datasets are available at http://adni.loni.usc.edu/data-samples/access-data/; the MSLUB dataset is available at http://lit.fe.uni-lj.si/tools.php?lang=eng; the MSISBI dataset is available at https://smart-stats-tools.org/lesion-challenge-2015; the WMH dataset is available at https://wmh.isi.uu.nl; and the BRATS 2018 dataset is available at https://www.med.upenn.edu/sbia/brats2018/data.html. For KRI, MSKRI and GBKRI, all patients were part of in-house observational cohorts, some of which were prospective (MSKRI; with patient consent), whereas the others were retrospective (without patient consent). For all patients, our local IRB approved the use of imaging data for research purposes after anonymization. As several patients were part of retrospective cohorts without explicit patient consent, these data cannot be shared as mandated by our IRB. For the prospective cohort data can be shared through Benedikt Wiestler (http://b.wiestler@tum.de) on reasonable request and signing of data transfer agreements, pending approval by our IRB and data protection officer.

Code availability

The code is publicly available at ref. 52.

References

  1. Klawiter, E. C. Current and new directions in MRI in multiple sclerosis. Continuum 19, 1058–1073 (2013).

  2. Soltaninejad, M. et al. Automated brain tumour detection and segmentation using superpixel-based extremely randomized trees in FLAIR MRI. Int. J. Comput. Assist. Radiol. Surg. 12, 183–203 (2017).

  3. Sun, C., Shrivastava, A., Singh, S. & Gupta, A. Revisiting unreasonable effectiveness of data in deep learning era. In Proc. IEEE International Conference on Computer Vision 843–852 (IEEE, 2017).

  4. Rieke, N. et al. The future of digital health with federated learning. NPJ Digit. Med. 3, 1–7 (2020).

    Article  Google Scholar 

  5. Kaissis, G. A., Makowski, M. R., Rückert, D. & Braren, R. F. Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2, 305–311 (2020).

    Article  Google Scholar 

  6. McMahan, B., Moore, E., Ramage, D., Hampson, S. & Arcas, B. A. y. Communication-efficient learning of deep networks from decentralized data. In Artificial Intelligence and Statistics 1273–1282 (PMLR, 2017).

  7. Collaborative learning without sharing data. Nat. Mach. Intell. 3, 459–459 (2021).

  8. Dou, Q. et al. Federated deep learning for detecting COVID-19 lung abnormalities in CT: a privacy-preserving multinational validation study. NPJ Digit. Med. 4, 1–11 (2021).

    Article  Google Scholar 

  9. Sheller, M. J. et al. Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci. Rep. 10, 1–12 (2020).

    Article  Google Scholar 

  10. Li, X. et al. Multi-site fMRI analysis using privacy-preserving federated learning and domain adaptation: ABIDE results. Med. Image Anal. 65, 101765 (2020).

    Article  Google Scholar 

  11. Li, D., Kar, A., Ravikumar, N., Frangi, A. F. & Fidler, S. Federated simulation for medical imaging. In International Conference on Medical Image Computing and Computer-Assisted Intervention 159–168 (Springer, 2020).

  12. Sarma, K. V. et al. Federated learning improves site performance in multicenter deep learning without data sharing. J. Am. Med. Inform. Assoc. 12, 1259-1264 (2021).

  13. Yang, D. et al. Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China, Italy, Japan. Med. Image Anal. 70, 101992 (2021).

  14. Albarqouni, S. et al. Domain adaptation and representation transfer, and distributed and collaborative learning. In Second MICCAI Workshop, DART 2020, and First MICCAI Workshop, DCL 2020, Held in Conjunction with MICCAI 2020, Lima, Peru, October 4-8, 2020, Proceedings Vol. 12444 (Springer Nature, 2020).

  15. Bdair, T., Navab, N. & Albarqouni, S. FedPerl: semi-supervised peer learning for skin lesion classification. In International Conference on Medical Image Computing and Computer-Assisted Intervention (Springer, 2021).

  16. Campello, V. M. et al. Multi-centre, multi-vendor and multi-disease cardiac segmentation: the M&Ms challenge. IEEE Trans. Med. Imaging 40, 3543–35541 (2021).

  17. Biberacher, V. et al. Intra-and interscanner variability of magnetic resonance imaging based volumetry in multiple sclerosis. Neuroimage 142, 188–197 (2016).

    Article  Google Scholar 

  18. Andreux, M., du Terrail, J. O., Beguier, C. & Tramel, E. W. Siloed federated learning for multi-centric histopathology datasets. In Domain Adaptation and Representation Transfer, and Distributed and Collaborative Learning 129–139 (Springer, 2020).

  19. Higgins, I. et al. Beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework (ICLR, 2016).

  20. Bercea, C. I., Wiestler, B., Rueckert, D. & Albarqouni, S. FedDis: disentangled federated learning for unsupervised brain pathology segmentation. Preprint at https://arxiv.org/abs/2103.03705 (2021).

  21. Chartsias, A. et al. Disentangled representation learning in cardiac image analysis. Med. Image Anal. 58, 101535 (2019).

    Article  Google Scholar 

  22. Locatello, F. et al. Challenging common assumptions in the unsupervised learning of disentangled representations. In International Conference on Machine Learning Vol. 97 (PMLR, 2019).

  23. Sarhan, M. H., Navab, N., Eslami, A. & Albarqouni, S. Fairness by learning orthogonal disentangled representations. In European Conference on Computer Vision 746–761 (Springer, 2020).

  24. Baur, C., Denner, S., Wiestler, B., Navab, N. & Albarqouni, S. Autoencoders for unsupervised anomaly segmentation in brain MR images: a comparative study. Med. Image Anal. 101952 (2021).

  25. Chen, X., You, S., Tezcan, K. C. & Konukoglu, E. Unsupervised lesion detection via image restoration with a normative prior. In Proc. Machine Learning Research Vol. 102 (PMLR, 2020).

  26. Pinaya, W. H. L. et al. Unsupervised brain anomaly detection and segmentation with transformers. Preprint at https://arxiv.org/abs/2102.11650 (2021).

  27. Baur, C. et al. Modeling healthy anatomy with artificial intelligence for unsupervised anomaly detection in brain MRI. Radiol. Artif. Intell. 3, e190169 (2021).

    Article  Google Scholar 

  28. Pathak, D., Krähenbühl, P., Donahue, J., Darrell, T. & Efros, A. A. Context encoders: feature learning by inpainting. Preprint at https://arxiv.org/abs/1604.07379 (2016).

  29. Zimmerer, D., Kohl, S. A. A., Petersen, J., Isensee, F. & Maier-Hein, K. H. Context-encoding variational autoencoder for unsupervised anomaly detection. Preprint at https://arxiv.org/abs/1812.05941 (2018).

  30. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K. & Galstyan, A. A survey on bias and fairness in machine learning. ACM Comput. Surv. 54, 1–35 (2019).

  31. van Hespen, K. M. et al. An anomaly detection approach to identify chronic brain infarcts on MRI. Sci. Rep. 11, 1–10 (2021).

    Google Scholar 

  32. Heer, M., Postels, J., Chen, X., Konukoglu, E. & Albarqouni, S. The OOD blind spot of unsupervised anomaly detection. In Proc. Machine Learning Research 286–300 (PMLR, 2021).

  33. Konukoglu, E., Glocker, B. & Initiative, A. D. N. et al. Reconstructing subject-specific effect maps. NeuroImage 181, 521–538 (2018).

    Article  Google Scholar 

  34. Dilokthanakul, N. et al. Deep unsupervised clustering with gaussian mixture variational autoencoders. Preprint at https://arxiv.org/abs/1611.02648 (2016).

  35. You, S., Tezcan, K. C., Chen, X. & Konukoglu, E. Unsupervised lesion detection via image restoration with a normative prior. In Cardoso, M. J. et al. (eds.) Proceedings of The 2nd International Conference on Medical Imaging with Deep Learning Vol. 102 of Proceedings of Machine Learning Research 540–556 (PMLR, 2019).

  36. Xie, C., Huang, K., Chen, P.-Y. & Li, B. Dba: Distributed backdoor attacks against federated learning. In International Conference on Learning Representations (2019).

  37. Lyu, L. et al. Privacy and robustness in federated learning: attacks and defenses. Preprint at https://arxiv.org/abs/2012.06337 (2020).

  38. Sun, J. et al. Soteria: Provable defense against privacy leakage in federated learning from representation perspective. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 9311–9319 (IEEE, CVF, 2021).

  39. Vincent, P. et al. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010).

  40. Ronneberger, O., Fischer, P. & Brox, T. U-Net: convolutional networks for biomedical image segmentation. In Lecture Notes in Computer Science (eds Frangi, A. et al.) Vol. 9351 (Springer, 2015).

  41. LaMontagne, P. J. et al. OASIS-3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and Alzheimer disease. Preprint at https://www.medrxiv.org/content/10.1101/2019.12.13.19014902v1 (2019).

  42. Weiner, M. et al. The Alzheimer’s Disease Neuroimaging Initiative 3: continued innovation for clinical trial improvement. Alzheimers Dement. 13, 561–571 (2016).

  43. Lesjak, Z. et al. A novel public MR image dataset of multiple sclerosis patients with lesion segmentations based on multi-rater consensus. Neuroinformatics 16, 51–63 (2018).

  44. Carass, A. et al. Longitudinal multiple sclerosis lesion segmentation data resource. Data Brief 12, 346–350 (2017).

    Article  Google Scholar 

  45. Kuijf, H. J. et al. Standardized assessment of automatic segmentation of white matter hyperintensities and results of the WMH segmentation challenge. IEEE Trans. Med. Imaging 38, 2556–2568 (2019).

    Article  Google Scholar 

  46. Menze, B. et al. The multimodal brain tumor image segmentation benchmark (BRATS). IEEE Trans. Med. Imaging 34, 1993–2034 (2015).

  47. Bakas, S. et al. Advancing The Cancer Genome Atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci Data 4, 170117 (2017).

  48. Bakas, S. et al. Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge (Univ. Cambridge, 2019).

  49. Rohlfing, T., Zahr, N. M., Sullivan, E. V. & Pfefferbaum, A. The SRI24 multichannel atlas of normal adult human brain structure. Hum. Brain Mapp. 31, 798–819 (2010).

    Article  Google Scholar 

  50. Iglesias, J. E., Liu, C.-Y., Thompson, P. M. & Tu, Z. Robust brain extraction across datasets and comparison with publicly available methods. IEEE Trams. Med. Imaging 30, 1617–1634 (2011).

    Article  Google Scholar 

  51. Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13, 600–612 (2004).

    Article  Google Scholar 

  52. Albarqouni, S. albarqounilab/feddis-nmi: Feddis_v0.1-alpha (Zenodo, 2022); https://doi.org/10.5281/zenodo.6604161

Download references

Acknowledgements

We would like to thank our clinical partners at Klinikum rechts der Isar, Munich, for generously providing their data.

Author information

Authors and Affiliations

  1. Helmholtz AI, Helmholtz Munich, Neuherberg, Germany

    Cosmin I. Bercea & Shadi Albarqouni

  2. Faculty of Informatics, Technical University of Munich, Garching, Germany

    Cosmin I. Bercea, Daniel Rueckert & Shadi Albarqouni

  3. Department of Neuroradiology, School of Medicine, Klinikum rechts der Isar, Munich, Germany

    Benedikt Wiestler

  4. Department of Computing, Imperial College London, London, UK

    Daniel Rueckert

  5. Clinic for Diagnostic and Interventional Radiology, University Hospital Bonn, Bonn, Germany

    Shadi Albarqouni

Authors
  1. Cosmin I. Bercea
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Benedikt Wiestler
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Rueckert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shadi Albarqouni
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

C.B. contributed to the methodology, software, formal analysis, investigation, visualization and writing of the original draft. B.W. contributed to the data curation, resources, and reviewed and edited the manuscript. D.R contributed to supervision, and reviwed and edited the manuscript. S.A. contributed to supervision, conceptualization, methodology, formal analysis, investigation, resources and project administration, and reviewed and edited the manuscript. All authors proof-read and accepted the final version of the manuscript.

Corresponding author

Correspondence to Shadi Albarqouni.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Machine Intelligence thanks Seung Hong Choi, Bjoern Menze and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Cleaning.

The top row shows original healthy samples from OASIS, ADNI-S, and ADNI-P and the bottom row shows the cleaned images. Given the advanced age of the healthy participants in the federated training, some hyperintensities, for example, because of prevalence of small vessel disease, may occur and be considered normal. We therefore clean the ground truth used for the reconstruction by in-painting over these hyperintense regions ( > 98th percentile) with the mean intensity value of the brain slice (50th percentile).

Extended Data Table 1 Datasets. The first set of datasets for healthy patients was mainly used for training and validation. The rest was used for anomaly detection and pathology delineation. Details about patients’ demography, scanners, resolution, and imaging parameters are reported
Full size table

Supplementary information

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bercea, C.I., Wiestler, B., Rueckert, D. et al. Federated disentangled representation learning for unsupervised brain anomaly detection. Nat Mach Intell 4, 685–695 (2022). https://doi.org/10.1038/s42256-022-00515-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s42256-022-00515-2

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing