Abstract
We describe a predictive imaging modality created by 'fusing' two distinct technologies: imaging mass spectrometry (IMS) and microscopy. IMS-generated molecular maps, rich in chemical information but having coarse spatial resolution, are combined with optical microscopy maps, which have relatively low chemical specificity but high spatial information. The resulting images combine the advantages of both technologies, enabling prediction of a molecular distribution both at high spatial resolution and with high chemical specificity. Multivariate regression is used to model variables in one technology, using variables from the other technology. We demonstrate the potential of image fusion through several applications: (i) 'sharpening' of IMS images, which uses microscopy measurements to predict ion distributions at a spatial resolution that exceeds that of measured ion images by ten times or more; (ii) prediction of ion distributions in tissue areas that were not measured by IMS; and (iii) enrichment of biological signals and attenuation of instrumental artifacts, revealing insights not easily extracted from either microscopy or IMS individually.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Weissleder, R. Scaling down imaging: molecular mapping of cancer in mice. Nat. Rev. Cancer 2, 11–18 (2002).
Massoud, T.F. & Gambhir, S.S. Molecular imaging in living subjects: seeing fundamental biological processes in a new light. Genes Dev. 17, 545–580 (2003).
Jahn, K.A. et al. Correlative microscopy: providing new understanding in the biomedical and plant sciences. Micron 43, 565–582 (2012).
Jacobs, R.E. & Cherry, S.R. Complementary emerging techniques: high-resolution PET and MRI. Curr. Opin. Neurobiol. 11, 621–629 (2001).
Chughtai, S. et al. A multimodal mass spectrometry imaging approach for the study of musculoskeletal tissues. Int. J. Mass Spectrom. 325–327, 150–160 (2012).
Smith, C. Two microscopes are better than one. Nature 492, 293–297 (2012).
Caplan, J., Niethammer, M., Taylor, R.M. II. & Czymmek, K.J. The power of correlative microscopy: multi-modal, multi-scale, multi-dimensional. Curr. Opin. Struct. Biol. 21, 686–693 (2011).
Modla, S. & Czymmek, K.J. Correlative microscopy: a powerful tool for exploring neurological cells and tissues. Micron 42, 773–792 (2011).
Townsend, D.W. A combined PET/CT scanner: the choices. J. Nucl. Med. 42, 533–534 (2001).
Townsend, D.W., Beyer, T. & Blodgett, T.M. PET/CT scanners: a hardware approach to image fusion. Semin. Nucl. Med. 33, 193–204 (2003).
Masyuko, R., Lanni, E.J., Sweedler, J.V. & Bohn, P.W. Correlated imaging - a grand challenge in chemical analysis. Analyst 138, 1924–1939 (2013).
Bocklitz, T.W. et al. Deeper understanding of biological tissue: quantitative correlation of MALDI-TOF and Raman imaging. Anal. Chem. 85, 10829–10834 (2013).
Clarke, F.C. et al. Chemical image fusion. The synergy of FT-NIR and Raman mapping microscopy to enable a more complete visualization of pharmaceutical formulations. Anal. Chem. 73, 2213–2220 (2001).
Judenhofer, M.S. et al. Simultaneous PET-MRI: a new approach for functional and morphological imaging. Nat. Med. 14, 459–465 (2008).
Glenn, D.R. et al. Correlative light and electron microscopy using cathodoluminescence from nanoparticles with distinguishable colours. Sci. Rep. 2, 865 (2012).
Josephson, L., Kircher, M.F., Mahmood, U., Tang, Y. & Weissleder, R. Near-infrared fluorescent nanoparticles as combined MR/optical imaging probes. Bioconjug. Chem. 13, 554–560 (2002).
Blum, R.S. & Liu, Z. Multi-sensor Image Fusion and Its Applications (CRC Press, 2005).
Bretschneider, T. & Kao, O. in Proc. 1st Online Symp. Electron. Eng. 1–8 (2000).
Pohl, C. & Van Genderen, J.L. Multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19, 823–854 (1998).
Price, J.C. Combining multispectral data of differing spatial resolution. IEEE Trans. Geosci. Rem. Sens. 37, 1199–1203 (1999).
Simone, G., Farina, A., Morabito, F.C., Serpico, S.B. & Bruzzone, L. Image fusion techniques for remote sensing applications. Inf. Fusion 3, 3–15 (2002).
Gaemperli, O. et al. Cardiac image fusion from stand-alone SPECT and CT: clinical experience. J. Nucl. Med. 48, 696–703 (2007).
Li, H. et al. Object recognition in brain CT-scans: knowledge-based fusion of data from multiple feature extractors. IEEE Trans. Med. Imaging 14, 212–229 (1995).
Yang, L., Guo, B.L. & Ni, W. Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72, 203–211 (2008).
Varshney, P.K. et al. in Proc. Int. Conf. Image Proc. 3, 532–536 (IEEE, 1999).
Caprioli, R.M., Farmer, T.B. & Gile, J. Molecular imaging of biological samples: localization of peptides and proteins using MALDI-TOF MS. Anal. Chem. 69, 4751–4760 (1997).
Stoeckli, M., Chaurand, P., Hallahan, D.E. & Caprioli, R.M. Imaging mass spectrometry: a new technology for the analysis of protein expression in mammalian tissues. Nat. Med. 7, 493–496 (2001).
Amstalden van Hove, E.R., Smith, D.F. & Heeren, R. A concise review of mass spectrometry imaging. J. Chromatogr. A 1217, 3946–3954 (2010).
Chaurand, P. Imaging mass spectrometry of thin tissue sections: a decade of collective efforts. J. Proteomics 75, 4883–4892 (2012).
Norris, J.L. & Caprioli, R.M. Analysis of tissue specimens by matrix-assisted laser desorption/ionization imaging mass spectrometry in biological and clinical research. Chem. Rev. 113, 2309–2342 (2013).
Murphy, R.C., Hankin, J.A. & Barkley, R.M. Imaging of lipid species by MALDI mass spectrometry. J. Lipid Res. 50, S317–S322 (2009).
Van de Plas, R. Tissue Based Proteomics and Biomarker Discovery – Multivariate Data Mining Strategies for Mass Spectral Imaging. PhD thesis, KU Leuven (2010).
Andersson, M., Andren, P. & Caprioli, R.M. in Neuroproteomics (ed. Azalte, O.) Ch. 7, 115–134 (CRC Press, 2009).
Franck, J. et al. MALDI mass spectrometry imaging of proteins exceeding 30,000 daltons. Med. Sci. Monit. 16, BR293–BR299 (2010).
Bradshaw, R., Bleay, S., Wolstenholme, R., Clench, M.R. & Francese, S. Towards the integration of matrix assisted laser desorption ionisation mass spectrometry imaging into the current fingermark examination workflow. Forensic Sci. Int. 232, 111–124 (2013).
Chavez, P.S. Jr., Sides, S.C. & Anderson, J.A. Comparison of three different methods to merge multiresolution and multispectral data- Landsat TM and SPOT panchromatic. Photogramm. Eng. Remote Sensing 57, 295–303 (1991).
Garguet-Duport, B., Girel, J., Chassery, J.-M. & Patou, G. The use of multiresolution analysis and wavelets transform for merging SPOT panchromatic and multispectral image data. Photogramm. Eng. Remote Sensing 62, 1057–1066 (1996).
Lee, J. & Lee, C. Fast and efficient panchromatic sharpening. IEEE Trans. Geosci. Remote Sens. 48, 155–163 (2010).
Draper, N.R., Smith, H. & Pownell, E. Applied Regression Analysis 1st edn. (Wiley, 1966).
Wold, S., Sjöström, M. & Eriksson, L. PLS-regression: a basic tool of chemometrics. Chemom. Intell. Lab. Syst. 58, 109–130 (2001).
Acknowledgements
This work was supported by the US National Institutes of Health grants NIH/NIGMS R01 GM058008-15 and NIH/NIGMS P41 GM103391-04. R.V.d.P. thanks E. Waelkens for his encouragement and support.
Author information
Authors and Affiliations
Contributions
R.V.d.P. conceived of and developed the methodology, designed experiments, analyzed and interpreted data and wrote the manuscript; J.Y. designed experiments, acquired data and edited the manuscript; J.S. designed experiments, acquired data, performed identifications and edited the manuscript; R.M.C. designed experiments, interpreted data, revised the manuscript and is principal investigator for the grants that fund this research.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–21, Supplementary Table 1, Supplementary Notes 1–4, Supplementary Results and Supplementary Discussion (PDF 100622 kb)
Supplementary Data
Supplementary Data for Case Study 1 This zip-file contains images related to Step 2 of the model building and evaluation phase, the mapping of transformed IMS and microscopy measurement sets to each other. The figures depict the IMS-to-microscopy weighted permutation mapping function (at the microscopy spatial resolution) and the integer mapping weights defined by it. (ZIP 1546 kb)
Rights and permissions
About this article
Cite this article
Van de Plas, R., Yang, J., Spraggins, J. et al. Image fusion of mass spectrometry and microscopy: a multimodality paradigm for molecular tissue mapping. Nat Methods 12, 366–372 (2015). https://doi.org/10.1038/nmeth.3296
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nmeth.3296
This article is cited by
-
Spatial probabilistic mapping of metabolite ensembles in mass spectrometry imaging
Nature Communications (2023)
-
Spatially resolved multi-omics highlights cell-specific metabolic remodeling and interactions in gastric cancer
Nature Communications (2023)
-
A super-resolution strategy for mass spectrometry imaging via transfer learning
Nature Machine Intelligence (2023)
-
Application of infrared matrix-assisted laser desorption electrospray ionization mass spectrometry for morphine imaging in brain tissue
Analytical and Bioanalytical Chemistry (2023)
-
Advances and prospects for the Human BioMolecular Atlas Program (HuBMAP)
Nature Cell Biology (2023)