Abstract
Multiplexed imaging enables the simultaneous spatial profiling of dozens of biological molecules in tissues at single-cell resolution. Extracting biologically relevant information, such as the spatial distribution of cell phenotypes from multiplexed tissue imaging data, involves a number of computational tasks, including image segmentation, feature extraction and spatially resolved single-cell analysis. Here, we present an end-to-end workflow for multiplexed tissue image processing and analysis that integrates previously developed computational tools to enable these tasks in a user-friendly and customizable fashion. For data quality assessment, we highlight the utility of napari-imc for interactively inspecting raw imaging data and the cytomapper R/Bioconductor package for image visualization in R. Raw data preprocessing, image segmentation and feature extraction are performed using the steinbock toolkit. We showcase two alternative approaches for segmenting cells on the basis of supervised pixel classification and pretrained deep learning models. The extracted single-cell data are then read, processed and analyzed in R. The protocol describes the use of community-established data containers, facilitating the application of R/Bioconductor packages for dimensionality reduction, single-cell visualization and phenotyping. We provide instructions for performing spatially resolved single-cell analysis, including community analysis, cellular neighborhood detection and cell–cell interaction testing using the imcRtools R/Bioconductor package. The workflow has been previously applied to imaging mass cytometry data, but can be easily adapted to other highly multiplexed imaging technologies. This protocol can be implemented by researchers with basic bioinformatics training, and the analysis of the provided dataset can be completed within 5–6 h. An extended version is available at https://bodenmillergroup.github.io/IMCDataAnalysis/.
Key points
-
The protocol describes the analysis of data generated by highly multiplexed tissue imaging approaches, such as imaging mass cytometry. The presented workflow includes steps for imaging data visualization, data preprocessing, image segmentation, single-cell feature extraction, reading data into R, spillover correction, quality control, cell phenotyping and spatially resolved single-cell analysis.
-
The software packages used include napari, steinbock, DeepCell/Mesmer, Ilastik, CellProfiler, cytomapper and imcRtools.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Code availability
The steinbock toolkit is available at https://github.com/BodenmillerGroup/steinbock. The pixel classification-based pipeline is available at https://github.com/BodenmillerGroup/ImcSegmentationPipeline. The cytomapper R/Bioconductor package is available at https://bioconductor.org/packages/cytomapper. The imcRtools R/Bioconductor package is available at https://bioconductor.org/packages/imcRtools. The napari-imc napari plugin is available at https://github.com/BodenmillerGroup/napari-imc. The readimc Python package is available at https://github.com/BodenmillerGroup/readimc. An extensive workflow on processing and analyzing multiplexed imaging data can be accessed at https://bodenmillergroup.github.io/IMCDataAnalysis/. The script reproducing the analysis presented in this protocol is available at https://github.com/BodenmillerGroup/IMCDataAnalysis/tree/main/publication.
Change history
18 October 2023
In the HTML version of the article initially published, there were some formatting errors in the presentation of the code which have now been corrected.
References
Lin, J.-R. et al. Highly multiplexed immunofluorescence imaging of human tissues and tumors using t-CyCIF and conventional optical microscopes. eLife 7, e31657 (2018).
Gut, G., Herrmann, M. D. & Pelkmans, L. Multiplexed protein maps link subcellular organization to cellular states. Science 361, eaar7042 (2018).
Radtke, A. J. et al. IBEX: a versatile multiplex optical imaging approach for deep phenotyping and spatial analysis of cells in complex tissues. Proc. Natl Acad. Sci. USA 117, 33455–33465 (2020).
Saka, S. K. et al. Immuno-SABER enables highly multiplexed and amplified protein imaging in tissues. Nat. Biotechnol. 37, 1080–1090 (2019).
Goltsev, Y. et al. Deep profiling of mouse splenic architecture with CODEX multiplexed imaging. Cell 174, 968–981.e15 (2018).
Giesen, C. et al. Highly multiplexed imaging of tumor tissues with subcellular resolution by mass cytometry. Nat. Methods 11, 417–422 (2014).
Angelo, M. et al. Multiplexed ion beam imaging of human breast tumors. Nat. Med. 20, 436–442 (2014).
Jackson, H. W. et al. The single-cell pathology landscape of breast cancer. Nature 578, 615–620 (2020).
Ali, H. R. et al. Imaging mass cytometry and multiplatform genomics define the phenogenomic landscape of breast cancer. Nat. Cancer 1, 163–175 (2020).
Schürch, C. M. et al. Coordinated cellular neighborhoods orchestrate antitumoral immunity at the colorectal cancer invasive front. Cell 183, 838 (2020).
Hoch, T. et al. Multiplexed imaging mass cytometry of the chemokine milieus in melanoma characterizes features of the response to immunotherapy. Sci. Immunol. 7, eabk1692 (2022).
Jiang, S. et al. Combined protein and nucleic acid imaging reveals virus-dependent B cell and macrophage immunosuppression of tissue microenvironments. Immunity 55, 1118–1134.e8 (2022).
Rendeiro, A. F. et al. The spatial landscape of lung pathology during COVID-19 progression. Nature 593, 564–569 (2021).
Mitamura, Y. et al. Cutaneous and systemic hyperinflammation drives maculopapular drug exanthema in severely ill COVID-19 patients. Allergy 77, 595–608 (2022).
Damond, N. et al. A map of human type 1 diabetes progression by imaging mass cytometry. Cell Metab. 29, 755–768.e5 (2019).
Ferrian, S. et al. Multiplexed imaging reveals an IFN-γ-driven inflammatory state in nivolumab-associated gastritis. Cell Rep. Med. 2, 100419 (2021).
Amezquita, R. A. et al. Orchestrating single-cell analysis with Bioconductor. Nat. Methods 17, 137–145 (2020).
Schapiro, D. et al. histoCAT: analysis of cell phenotypes and interactions in multiplex image cytometry data. Nat. Methods 14, 873–876 (2017).
Bhate, S. S., Barlow, G. L., Schürch, C. M. & Nolan, G. P. Tissue schematics map the specialization of immune tissue motifs and their appropriation by tumors. Cell Syst. 13, 109–130.e6 (2022).
napari contributors. napari: a multi-dimensional image viewer for python. https://doi.org/10.5281/zenodo.3555620 (2019).
Berg, S. et al. ilastik: interactive machine learning for (bio)image analysis. Nat. Methods 16, 1226–1232 (2019).
Stirling, D. R. et al. CellProfiler 4: improvements in speed, utility and usability. BMC Bioinforma. 22, 433 (2021).
Greenwald, N. F. et al. Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning. Nat. Biotechnol. 40, 555–565 (2022).
Chevrier, S. et al. Compensation of signal spillover in suspension and imaging mass cytometry. Cell Syst. 6, 612–620.e5 (2018).
Eling, N., Damond, N., Hoch, T. & Bodenmiller, B. cytomapper: an R/bioconductor package for visualisation of highly multiplexed imaging data. Bioinformatics 36, 5706–5708 (2020).
Bankhead, P. et al. QuPath: open source software for digital pathology image analysis. Sci. Rep. 7, 16878 (2017).
Schulz, D. et al. Simultaneous multiplexed imaging of mRNA and proteins with subcellular resolution in breast cancer tissue samples by mass cytometry. Cell Syst. 6, 531 (2018).
Righelli, D. et al. SpatialExperiment: infrastructure for spatially resolved transcriptomics data in R using Bioconductor. Bioinformatics 38, 3128–3131 (2022).
Bunis, D. G., Andrews, J., Fragiadakis, G. K., Burt, T. D. & Sirota, M. dittoSeq: universal user-friendly single-cell and bulk RNA sequencing visualization toolkit. Bioinformatics 36, 5535–5536 (2020).
Haghverdi, L., Lun, A. T. L., Morgan, M. D. & Marioni, J. C. Batch effects in single-cell RNA-sequencing data are corrected by matching mutual nearest neighbors. Nat. Biotechnol. 36, 421–427 (2018).
Tan, Y. et al. Single-cell analysis of endometriosis reveals a coordinated transcriptional programme driving immunotolerance and angiogenesis across eutopic and ectopic tissues. Nat. Cell Biol. 24, 1306–1318 (2022).
Canete, N. P. et al. spicyR: spatial analysis of in situ cytometry data in R. Bioinformatics 38, 3099–3105 (2022).
Siu, J. H. Y. et al. Two subsets of human marginal zone B cells resolved by global analysis of lymphoid tissues and blood. Sci. Immunol. 7, eabm9060 (2022).
Singh, N. et al. Development of a 2-dimensional atlas of the human kidney with imaging mass cytometry. JCI Insight 4, e129477 (2019).
Lotsberg, M. L. et al. Intrinsic differences in spatiotemporal organization and stromal cell interactions between isogenic lung cancer cells of epithelial and mesenchymal phenotypes revealed by high-dimensional single-cell analysis of heterotypic 3D spheroid models. Front. Oncol. 12, 818437 (2022).
Lohoff, T. et al. Integration of spatial and single-cell transcriptomic data elucidates mouse organogenesis. Nat. Biotechnol. 40, 74–85 (2022).
Virshup, I., Rybakov, S., Theis, F. J., Angerer, P. & Alexander Wolf, F. anndata: annotated data. Preprint at bioRxiv https://doi.org/10.1101/2021.12.16.473007 (2021).
Street, K. et al. Slingshot: cell lineage and pseudotime inference for single-cell transcriptomics. BMC Genomics 19, 477 (2018).
Robinson, M. D., McCarthy, D. J. & Smyth, G. K. edgeR: a Bioconductor package for differential expression analysis of digital gene expression data. Bioinformatics 26, 139–140 (2010).
Stoltzfus, C. R. et al. CytoMAP: a spatial analysis toolbox reveals features of myeloid cell organization in lymphoid tissues. Cell Rep. 31, 107523 (2020).
Czech, E., Aksoy, B. A., Aksoy, P. & Hammerbacher, J. Cytokit: a single-cell analysis toolkit for high dimensional fluorescent microscopy imaging. BMC Bioinforma. 20, 448 (2019).
Keller, M. S. et al. Vitessce: a framework for integrative visualization of multi-modal and spatially-resolved single-cell data. Preprint at OSF Preprints https://doi.org/10.31219/osf.io/y8thv (2021).
van Maldegem, F. et al. Characterisation of tumour microenvironment remodelling following oncogene inhibition in preclinical studies with imaging mass cytometry. Nat. Commun. 12, 5906 (2021).
Schmidt, U., Weigert, M., Broaddus, C. & Myers, G. Cell detection with star-convex polygons. MICCAI 2018, 265–273 (2018).
Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18, 100–106 (2021).
Somarakis, A., Van Unen, V., Koning, F., Lelieveldt, B. & Hollt, T. ImaCytE: visual exploration of cellular micro-environments for imaging mass cytometry data. IEEE Trans. Vis. Comput. Graph. 27, 98–110 (2021).
Gentleman, R. C. et al. Bioconductor: open software development for computational biology and bioinformatics. Genome Biol. 5, R80 (2004).
Palla, G. et al. Squidpy: a scalable framework for spatial omics analysis. Nat. Methods 19, 171–178 (2022).
Dries, R. et al. Giotto: a toolbox for integrative analysis and visualization of spatial expression data. Genome Biol. 22, 78 (2021).
McKinley, E. T. et al. MIRIAM: a machine and deep learning single‐cell segmentation and quantification pipeline for multi‐dimensional tissue images. Cytom. Part A 101, 521–528 (2022).
Eng, J. et al. A framework for multiplex imaging optimization and reproducible analysis. Commun. Biol. 5, 438 (2022).
Schapiro, D. et al. MCMICRO: a scalable, modular image-processing pipeline for multiplexed tissue imaging. Nat. Methods 19, 311–315 (2022).
Levine, J. H. et al. Data-driven phenotypic dissection of AML reveals progenitor-like cells that correlate with prognosis. Cell 162, 184–197 (2015).
Nirmal, A. J. et al. The spatial landscape of progression and immunoediting in primary melanoma at single-cell resolution. Cancer Discov. 12, 1518–1541 (2022).
Hickey, J. W. et al. Spatial mapping of protein composition and tissue organization: a primer for multiplexed antibody-based imaging. Nat. Methods 19, 284–295 (2022).
Baker, E. A. G., Schapiro, D., Dumitrascu, B., Vickovic, S. & Regev, A. In silico tissue generation and power analysis for spatial omics. Nat. Methods 20, 424–431 (2023).
Bost, P., Schulz, D., Engler, S., Wasserfall, C. & Bodenmiller, B. Optimizing multiplexed imaging experimental design through tissue spatial segregation estimation. Nat. Methods 20, 418–423 (2023).
McCarthy, D. J., Campbell, K. R., Lun, A. T. L. & Wills, Q. F. Scater: pre-processing, quality control, normalization and visualization of single-cell RNA-seq data in R. Bioinformatics 33, 1179–1186 (2017).
Lun, A. T. L., McCarthy, D. J. & Marioni, J. C. A step-by-step workflow for low-level analysis of single-cell RNA-seq data with Bioconductor. F1000Res. 5, 2122 (2016).
Hong, H., Liechti, R. & Morfouace, M. Yes, IMMUcan! Unraveling the human tumor microenvironment. Immunol. Oncol. Insights 02, 301–309 (2021).
Catena, R., Montuenga, L. M. & Bodenmiller, B. Ruthenium counterstaining for imaging mass cytometry. J. Pathol. 244, 479–484 (2018).
Lu, P. et al. IMC-Denoise: a content aware denoising pipeline to enhance imaging mass cytometry. Nat. Commun. 14, 1601 (2023).
Muhlich, J. L. et al. Stitching and registering highly multiplexed whole-slide images of tissues and tumors using ASHLAR. Bioinformatics 38, 4613–4621 (2022).
Keren, L. et al. A structured tumor-immune microenvironment in triple negative breast cancer revealed by multiplexed ion beam imaging. Cell 174, 1373–1387.e19 (2018).
Ji, A. L. et al. Multimodal analysis of composition and spatial architecture in human squamous cell carcinoma. Cell 182, 1661–1662 (2020).
Selega, A. & Campbell, K. R. Multi-objective Bayesian optimization with heuristic objectives for biomedical and molecular data analysis workflows. Preprint at bioRxiv https://doi.org/10.1101/2022.06.08.495370 (2022).
Baranski, A. et al. MAUI (MBI Analysis User Interface)—an image processing pipeline for multiplexed mass based imaging. PLoS Comput. Biol. 17, e1008887 (2021).
Lin, J.-R., Fallahi-Sichani, M., Chen, J.-Y. & Sorger, P. K. Cyclic immunofluorescence (CycIF), a highly multiplexed method for single-cell imaging. Curr. Protoc. Chem. Biol. 8, 251–264 (2016).
Crowell H, Zanotelli V, Chevrier S, Robinson M. CATALYST: Cytometry dATa anALYSis Tools. https://github.com/HelenaLC/CATALYST (2023).
Kuett, L. et al. Three-dimensional imaging mass cytometry for highly multiplexed molecular and cellular mapping of tissues and the tumor microenvironment. Nat. Cancer 3, 122–133 (2022).
Korsunsky, I. et al. Fast, sensitive and accurate integration of single-cell data with Harmony. Nat. Methods 16, 1289–1296 (2019).
Stuart, T. et al. Comprehensive integration of single-cell data. Cell 177, 1888–1902.e21 (2019).
Quintelier, K. et al. Analyzing high-dimensional cytometry data using FlowSOM. Nat. Protoc. 16, 3775–3801 (2021).
Blondel, V. D., Guillaume, J.-L., Lambiotte, R. & Lefebvre, E. Fast unfolding of communities in large networks. J. Stat. Mech. Theory Exp. 2008, P10008 (2008).
Patrick, E. et al. Spatial analysis for highly multiplexed imaging data to identify tissue microenvironments. Cytom. A https://doi.org/10.1002/cyto.a.24729 (2023).
Shannon, P. et al. Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome Res. 13, 2498–2504 (2003).
Acknowledgements
We thank the Bodenmiller laboratory for helpful feedback and discussions, and the IMMUcan consortium for allowing us to use their data. Specifically, we would like to thank J.R. Fischer and T. Hoch for code contributions to the imcRtools package, as well as R. Casanova and N. Damond for testing the steinbock toolkit in Python code. We would further like to thank N. de Souza for critically commenting on the manuscript. Finally, we would like to express our appreciation of the larger bioimage analysis and open-source software communities for providing supportive feedback and packages critical to the development of the presented workflow. J.W. was funded by the CRUK IMAXT Grand Challenge and by two Chan Zuckerberg Initiative (CZI) napari Plugin Accelerator Grants (2021-239869(5022), 2021-239940(5022)). B.B. was supported by the European Research Council (ERC) under the European Union’s Horizon 2020 framework, ERC-2019-CoG: 866074 – Precision Motifs, a SNF Project Grant, a Promedica foundation grant and the CRUK IMAXT Grand Challenge. N.E. was funded by the European Union’s Horizon 2020 research and innovation program under Marie Sklodowska-Curie Actions grant agreement no. 892225.
Author information
Authors and Affiliations
Contributions
J.W. developed readimc, napari-imc and the steinbock toolkit, and contributed to the pixel classification-based segmentation pipeline. V.R.T.Z. developed the pixel classification-based segmentation pipeline and contributed to the spillover correction and the interaction testing approaches. D.S. implemented the cellular neighborhood detection approach. L.M. wrote the spatial community and spatial context-related functions. M.D. generated the data and provided experimental information for the spillover correction approach. N.E. developed the cytomapper and imcRtools packages and contributed to the pixel classification-based segmentation pipeline. J.W., B.B. and N.E. conceived the study. J.W., B.B. and N.E. wrote the manuscript. All authors approve of the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Protocols thanks Ellis Patrick and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Related links
Key references using this protocol
Hoch, T. et al. Sci. Immunol. 7, eabk1692 (2022): https://doi.org/10.1126/sciimmunol.abk1692
Jackson, H. W. et al. Nature 578, 615–620 (2020): https://doi.org/10.1038/s41586-019-1876-x
Damond, N. et al. Cell Metab. 29, 755–768.e5 (2019): https://doi.org/10.1016/j.cmet.2018.11.014
Ali, H. R. et al. Nat. Cancer 1, 163–175 (2020): https://doi.org/10.1038/s43018-020-0026-6
Schulz, D. et al. Cell Syst. 6, 531 (2018): https://doi.org/10.1016/j.cels.2017.12.001
Supplementary information
Supplementary Information
Supplementary Notes 1 and 2.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Windhager, J., Zanotelli, V.R.T., Schulz, D. et al. An end-to-end workflow for multiplexed image processing and analysis. Nat Protoc 18, 3565–3613 (2023). https://doi.org/10.1038/s41596-023-00881-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41596-023-00881-0