We describe a predictive imaging modality created by 'fusing' two distinct technologies: imaging mass spectrometry (IMS) and microscopy. IMS-generated molecular maps, rich in chemical information but having coarse spatial resolution, are combined with optical microscopy maps, which have relatively low chemical specificity but high spatial information. The resulting images combine the advantages of both technologies, enabling prediction of a molecular distribution both at high spatial resolution and with high chemical specificity. Multivariate regression is used to model variables in one technology, using variables from the other technology. We demonstrate the potential of image fusion through several applications: (i) 'sharpening' of IMS images, which uses microscopy measurements to predict ion distributions at a spatial resolution that exceeds that of measured ion images by ten times or more; (ii) prediction of ion distributions in tissue areas that were not measured by IMS; and (iii) enrichment of biological signals and attenuation of instrumental artifacts, revealing insights not easily extracted from either microscopy or IMS individually.
At a glance
- Scaling down imaging: molecular mapping of cancer in mice. Nat. Rev. Cancer 2, 11–18 (2002).
- Molecular imaging in living subjects: seeing fundamental biological processes in a new light. Genes Dev. 17, 545–580 (2003). &
- Correlative microscopy: providing new understanding in the biomedical and plant sciences. Micron 43, 565–582 (2012). et al.
- Complementary emerging techniques: high-resolution PET and MRI. Curr. Opin. Neurobiol. 11, 621–629 (2001). &
- A multimodal mass spectrometry imaging approach for the study of musculoskeletal tissues. Int. J. Mass Spectrom. 325–327, 150–160 (2012). et al.
- Two microscopes are better than one. Nature 492, 293–297 (2012).
- The power of correlative microscopy: multi-modal, multi-scale, multi-dimensional. Curr. Opin. Struct. Biol. 21, 686–693 (2011). , , &
- Correlative microscopy: a powerful tool for exploring neurological cells and tissues. Micron 42, 773–792 (2011). &
- A combined PET/CT scanner: the choices. J. Nucl. Med. 42, 533–534 (2001).
- PET/CT scanners: a hardware approach to image fusion. Semin. Nucl. Med. 33, 193–204 (2003). , &
- Correlated imaging - a grand challenge in chemical analysis. Analyst 138, 1924–1939 (2013). , , &
- Deeper understanding of biological tissue: quantitative correlation of MALDI-TOF and Raman imaging. Anal. Chem. 85, 10829–10834 (2013). et al.
- Chemical image fusion. The synergy of FT-NIR and Raman mapping microscopy to enable a more complete visualization of pharmaceutical formulations. Anal. Chem. 73, 2213–2220 (2001). et al.
- Simultaneous PET-MRI: a new approach for functional and morphological imaging. Nat. Med. 14, 459–465 (2008). et al.
- Correlative light and electron microscopy using cathodoluminescence from nanoparticles with distinguishable colours. Sci. Rep. 2, 865 (2012). et al.
- Near-infrared fluorescent nanoparticles as combined MR/optical imaging probes. Bioconjug. Chem. 13, 554–560 (2002). , , , &
- Multi-sensor Image Fusion and Its Applications (CRC Press, 2005). &
- Proc. 1st Online Symp. Electron. Eng. 1–8 (2000). & in
- Multisensor image fusion in remote sensing: concepts, methods and applications. Int. J. Remote Sens. 19, 823–854 (1998). &
- Combining multispectral data of differing spatial resolution. IEEE Trans. Geosci. Rem. Sens. 37, 1199–1203 (1999).
- Image fusion techniques for remote sensing applications. Inf. Fusion 3, 3–15 (2002). , , , &
- Cardiac image fusion from stand-alone SPECT and CT: clinical experience. J. Nucl. Med. 48, 696–703 (2007). et al.
- Object recognition in brain CT-scans: knowledge-based fusion of data from multiple feature extractors. IEEE Trans. Med. Imaging 14, 212–229 (1995). et al.
- Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform. Neurocomputing 72, 203–211 (2008). , &
- Proc. Int. Conf. Image Proc. 3, 532–536 (IEEE, 1999). et al. in
- Molecular imaging of biological samples: localization of peptides and proteins using MALDI-TOF MS. Anal. Chem. 69, 4751–4760 (1997). , &
- Imaging mass spectrometry: a new technology for the analysis of protein expression in mammalian tissues. Nat. Med. 7, 493–496 (2001). , , &
- A concise review of mass spectrometry imaging. J. Chromatogr. A 1217, 3946–3954 (2010). , &
- Imaging mass spectrometry of thin tissue sections: a decade of collective efforts. J. Proteomics 75, 4883–4892 (2012).
- Analysis of tissue specimens by matrix-assisted laser desorption/ionization imaging mass spectrometry in biological and clinical research. Chem. Rev. 113, 2309–2342 (2013). &
- Imaging of lipid species by MALDI mass spectrometry. J. Lipid Res. 50, S317–S322 (2009). , &
- Tissue Based Proteomics and Biomarker Discovery – Multivariate Data Mining Strategies for Mass Spectral Imaging. PhD thesis, KU Leuven (2010).
- Neuroproteomics (ed. Azalte, O.) Ch. 7, 115–134 (CRC Press, 2009). , & in
- MALDI mass spectrometry imaging of proteins exceeding 30,000 daltons. Med. Sci. Monit. 16, BR293–BR299 (2010). et al.
- Towards the integration of matrix assisted laser desorption ionisation mass spectrometry imaging into the current fingermark examination workflow. Forensic Sci. Int. 232, 111–124 (2013). , , , &
- Comparison of three different methods to merge multiresolution and multispectral data- Landsat TM and SPOT panchromatic. Photogramm. Eng. Remote Sensing 57, 295–303 (1991). , &
- The use of multiresolution analysis and wavelets transform for merging SPOT panchromatic and multispectral image data. Photogramm. Eng. Remote Sensing 62, 1057–1066 (1996). , , &
- Fast and efficient panchromatic sharpening. IEEE Trans. Geosci. Remote Sens. 48, 155–163 (2010). &
- Applied Regression Analysis 1st edn. (Wiley, 1966). , &
- PLS-regression: a basic tool of chemometrics. Chemom. Intell. Lab. Syst. 58, 109–130 (2001). , &
- Supplementary Text and Figures (100,250 KB)
Supplementary Figures 1–21, Supplementary Table 1, Supplementary Notes 1–4, Supplementary Results and Supplementary Discussion
- Supplementary Data (1,583 KB)
Supplementary Data for Case Study 1
This zip-file contains images related to Step 2 of the model building and evaluation phase, the mapping of transformed IMS and microscopy measurement sets to each other. The figures depict the IMS-to-microscopy weighted permutation mapping function (at the microscopy spatial resolution) and the integer mapping weights defined by it.