Compressive hyperspectral time-resolved wide-field fluorescence lifetime imaging

Journal name:
Nature Photonics
Year published:
Published online

Spectrally resolved fluorescence lifetime imaging1, 2, 3 and spatial multiplexing1, 4, 5 have offered information content and collection-efficiency boosts in microscopy, but efficient implementations for macroscopic applications are still lacking. An imaging platform based on time-resolved structured light and hyperspectral single-pixel detection has been developed to perform quantitative macroscopic fluorescence lifetime imaging (MFLI) over a large field of view (FOV) and multiple spectral bands simultaneously. The system makes use of three digital micromirror device (DMD)-based spatial light modulators (SLMs) to generate spatial optical bases and reconstruct N by N images over 16 spectral channels with a time-resolved capability (∼40 ps temporal resolution) using fewer than N2 optical measurements. We demonstrate the potential of this new imaging platform by quantitatively imaging near-infrared (NIR) Förster resonance energy transfer (FRET) both in vitro and in vivo. The technique is well suited for quantitative hyperspectral lifetime imaging with a high sensitivity and paves the way for many important biomedical applications.

At a glance


  1. System diagram and typical hyperspectral TPSF example.
    Figure 1: System diagram and typical hyperspectral TPSF example.

    a, Diagram of the time-resolved hyperspectral single-pixel imager with transmission mode (formed by DMD I and III) and reflectance mode (formed by DMD II and III). The NIR CCD and mirror are only used during calibration. The laser-power control and fibre-coupling modules are not shown. b, Visualizations of an acquired data cube (top left, measurements under the first 100 Hadamard patterns acquired by the system) and hyperspectral TPSFs (right) under the illumination of pattern no. 40 (bottom left). PC, photon count; a.u., arbitrary units.

  2. Results of in vitro study.
    Figure 2: Results of in vitro study.

    a, Well-plate sample intensity captured using the NIR CCD (rescaled to a 64 × 64 resolution). The layout of the plate in terms of the relative acceptor:donor (A:D) ratio from well no. 1 to no. 9 is as follows: 0:1, 0:0, 0:0, 1:1, 2:1, 3:1, 1:0, 2:0 and 3:0. Wells 2 and 3 are used as the control with PBS only. b,c, Reconstructed spectrally resolved intensity and mean-lifetime spatial maps via the single-pixel method. d,e, Spectrally resolved intensity and mean-lifetime distributions for each well. f, FD% estimated via a bi-exponential decay fitting of wells 1 and 4–6 at the 725 nm channel and comparison with results obtained using an ICCD-based imaging system. Equivalence tests show that their mean FD% values are within a ±6% absolute margin at the 0.05 significance level (P = 0.023). Error bars represent s.d.

  3. Results of the in vivo study.
    Figure 3: Results of the in vivo study.

    a, Fluorescence signals captured using the NIR CCD (FOV, 40 mm × 40 mm) at 4 h post-injection of NIR FRET-labelled transferrin. b,c, Examples of the spectrally resolved single-pixel spatial reconstruction of intensity (b) and mean lifetime (c) at 4 h post-injection for subject no. 1. d,e, Means and standard deviations of intensity (d) and lifetime (e) over the liver and the urinary bladder as retrieved from single-pixel processing (4 and 6 h post-injection of subject no. 1). f, FD% of the two organs at the donor emission peak channel for four subjects and comparison with the results obtained from the ICCD-based system at 4 h post-injection. Here ‘Lm’ and ‘Bm’ (m = 1, 2, 3 and 4) represent the liver and the urinary bladder, respectively, for subject no. m. Equivalence tests show that their mean FD% values are within a ±6% absolute margin at the 0.05 significance level (P = 0.007). Error bars represent s.d.


  1. Hanley, Q. S., Arndt-Jovin, D. J. & Jovin, T. M. Spectrally resolved fluorescence lifetime imaging microscopy. Appl. Spectrosc. 56, 155166 (2002).
  2. Bird, D. K., Eliceiri, K. W., Fan, C.-H. & White, J. G. Simultaneous two-photon spectral and lifetime fluorescence microscopy. Appl. Opt. 43, 51735182 (2004).
  3. Zhao, M., Li, Y. & Peng, L. Parallel excitation-emission multiplexed fluorescence lifetime confocal microscopy for live cell imaging. Opt. Express 22, 1022110232 (2014).
  4. Studer, V. et al. Compressive fluorescence microscopy for biological and hyperspectral imaging. Proc. Natl Acad. Sci. USA 109, E1679E1687 (2012).
  5. Mizuno, T. & Iwata, T. Hadamard-transform fluorescence-lifetime imaging. Opt. Express 24, 82028213 (2016).
  6. Roblyer, D. et al. Multispectral optical imaging device for in vivo detection of oral neoplasia. J. Biomed. Opt. 13, 024019 (2008).
  7. Lu, G. & Fei, B. Medical hyperspectral imaging: a review. J. Biomed. Opt. 19, 010901 (2014).
  8. Panasyuk, S. V. et al. Medical hyperspectral imaging to facilitate residual tumor identification during surgery. Cancer Biol. Ther. 6, 439446 (2007).
  9. Akbari, H., Kosugi, Y., Kojima, K. & Tanaka, N. Detection and analysis of the intestinal ischemia using visible and invisible hyperspectral imaging. IEEE Trans. Biomed. Eng. 57, 20112017 (2010).
  10. Marcu, L. Fluorescence lifetime techniques in medical applications. Ann. Biomed. Eng. 40, 304331 (2012).
  11. Suhling, K. et al. Fluorescence lifetime imaging (FLIM): basic concepts and some recent developments. Med. Photon. 27, 340 (2015).
  12. Sun, B. et al. 3D computational imaging with single-pixel detectors. Science 340, 844847 (2013).
  13. Pham, Q. D. & Hayasaki, Y. Optical frequency comb profilometry using a single-pixel camera composed of digital micromirror devices. Appl. Opt. 54, A39A44 (2015).
  14. Yu, W.-K., Yao, X.-R., Liu, X.-F., Li, L.-Z. & Zhai, G.-J. Three-dimensional single-pixel compressive reflectivity imaging based on complementary modulation. Appl. Opt. 54, 363367 (2015).
  15. Chan, W. L. et al. A single-pixel terahertz imaging system based on compressed sensing. Appl. Phys. Lett. 93, 121105 (2008).
  16. Withayachumnankul, W. & Abbott, D. Terahertz imaging: compressing onto a single pixel. Nat. Photon. 8, 593594 (2014).
  17. Ma, J. Single-pixel remote sensing. IEEE Geosci. Remote Sens. Lett. 6, 199203 (2009).
  18. Donoho, D. L. Compressed sensing. IEEE Trans. Inf. Theory 52, 12891306 (2006).
  19. Duarte, M. F. et al. Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25, 8391 (2008).
  20. Welsh, S. S. et al. Fast full-color computational imaging with single-pixel detectors. Opt. Express 21, 2306823074 (2013).
  21. Greenberg, J., Krishnamurthy, K. & Brady, D. Compressive single-pixel snapshot X-ray diffraction imaging. Opt. Lett. 39, 111114 (2014).
  22. Ma, G., Gallant, P. & McIntosh, L. Sensitivity characterization of a time-domain fluorescence imager: eXplore Optix. Appl. Opt. 46, 16501657 (2007).
  23. Zhao, L. et al. Spatial light modulator based active wide-field illumination for ex vivo and in vivo quantitative NIR FRET imaging. Biomed. Opt. Express 5, 944960 (2014).
  24. Streeter, L., Burling-Claridge, G. R., Cree, M. J. & Künnemeyer, R. Optical full Hadamard matrix multiplexing and noise effects. Appl. Opt. 48, 20782085 (2009).
  25. Mäkitalo, M. & Foi, A. Optimal inversion of the Anscombe transformation in low-count Poisson image denoising. IEEE Trans. Image Process. 20, 99109 (2011).
  26. Abe, K., Zhao, L., Periasamy, A., Intes, X. & Barroso, M. Non-invasive in vivo imaging of near infrared-labeled transferrin in breast cancer cells and tumors using fluorescence lifetime FRET. PLoS ONE 8, e80269 (2013).
  27. Qian, Z. M., Li, H., Sun, H. & Ho, K. Targeted drug delivery via the transferrin receptor-mediated endocytosis pathway. Pharmacol. Rev. 54, 561587 (2002).
  28. Pian, Q., Yao, R., Zhao, L. & Intes, X. Hyperspectral time-resolved wide-field fluorescence molecular tomography based on structured light and single-pixel detection. Opt. Lett. 40, 431434 (2015).
  29. Wessels, J. T. et al. In vivo imaging in experimental preclinical tumor research—a review. Cytom. A 71A, 542549 (2007).
  30. Christian, N. A. et al. In vivo dendritic cell tracking using fluorescence lifetime imaging and near-infrared-emissive polymersomes. Mol. Imaging Biol. 11, 167177 (2009).
  31. Hall, D. J. et al. Dynamic optical imaging of metabolic and NADPH oxidase-derived superoxide in live mouse brain using fluorescence lifetime unmixing. J. Cereb. Blood Flow Metab. 32, 2332 (2012).

Download references

Author information


  1. Biomedical Engineering Department, Rensselaer Polytechnic Institute, 110 8th Street, Troy, New York 12180, USA

    • Qi Pian,
    • Ruoyang Yao,
    • Nattawut Sinsuebphon &
    • Xavier Intes


X.I. conceived the original idea. Q.P. contributed to the platform instrumentation, control software design and experimental validation. R.Y. contributed to the pattern implementation, initial simulations and image reconstruction. N.S. contributed to the animal study protocol and experiments. All the authors contributed to the analysis of the experimental results and to writing the manuscript.

Competing financial interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to:

Author details

Supplementary information

PDF files

  1. Supplementary information (892 KB)

    Supplementary information

Additional data