Photography optics in the time dimension

Abstract

Ultrafast sensors and depth cameras are key enablers for imaging through complex geometries, through scattering, and beyond the line of sight. However, despite accelerating advances in imaging electronics and imaging applications, the optics of such cameras have been inherited from conventional low-speed photography cameras. This has limited ultrafast cameras and their applications to the design constraints of conventional optics. Here, we exploit time as an extra dimension in the optical design and demonstrate that by folding large spaces in time using time-resolved cavities, one can enable new camera capabilities without losing the targeted information. We demonstrate lens tube compression by an order of magnitude, together with ultrafast multi-zoom imaging and ultrafast multispectral imaging by time-folding the optical path at different regions of the imaging optics. Considering the vast variety of designs that could emerge by time-folding conventional imaging optics, we expect this technique to have a broad impact on time-resolved imaging and depth-sensing optics.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Experimental set-up and time-folded imaging overview.
Fig. 2: Spatial compression of imaging optics by time-folding the focal length.
Fig. 3: Multi-zoom ultrafast camera by time-folding the entrance lens.
Fig. 4: Multispectral ultrafast camera by wavelength-dependent time-folding.

References

  1. 1.

    Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012).

    Article  Google Scholar 

  2. 2.

    Gariepy, G., Tonolini, F., Henderson, R., Leach, J. & Faccio, D. Detection and tracking of moving objects hidden from view. Nat. Photon. 10, 23–27 (2016).

    ADS  Article  Google Scholar 

  3. 3.

    Chan, S., Warburton, R. E., Gariepy, G., Leach, J. & Faccio, D. Non-line-of-sight tracking of people at long range. Opt. Express 25, 10109 (2017).

    Article  Google Scholar 

  4. 4.

    Satat, G., Heshmat, B., Raviv, D. & Raskar, R. All photons imaging through volumetric scattering. Sci. Rep. 6, 33946 (2016).

    ADS  Article  Google Scholar 

  5. 5.

    Satat, G. et al. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nat. Commun. 6, 6796 (2015).

    Article  Google Scholar 

  6. 6.

    Puszka, A. et al. Time-resolved diffuse optical tomography using fast-gated single-photon avalanche diodes. Biomed. Opt. Express 4, 1351–1365 (2013).

    Article  Google Scholar 

  7. 7.

    Redo-Sanchez, A. et al. Terahertz time-gated spectral imaging for content extraction through layered structures. Nat. Commun. 7, 12665 (2016).

    ADS  Article  Google Scholar 

  8. 8.

    Heshmat, B., Lee, I. H. & Raskar, R. Optical brush: imaging through permuted probes. Sci. Rep. 6, 20217 (2016).

    ADS  Article  Google Scholar 

  9. 9.

    Bhandari, A., Bourquard, A., Izadi, S. & Raskar, R. Time-resolved image demixing. In IEEE International Conf. on Acoustics, Speech and Signal Processing (ICASSP) 4483–4487 (2016).

  10. 10.

    Gariepy, G. et al. Single-photon sensitive light-in-fight imaging. Nat. Commun. 6, 6021 (2015).

    Article  Google Scholar 

  11. 11.

    Buller, G. S. & Wallace, A. Ranging and three-dimensional imaging using time-correlated single-photon counting and point-by-point acquisition. IEEE J. Sel. Top. Quant. Electron. 13, 1006–1015 (2007).

    ADS  Article  Google Scholar 

  12. 12.

    Goda, K., Tsia, K. K. & Jalali, B. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature 458, 1145–1149 (2009).

    ADS  Article  Google Scholar 

  13. 13.

    Richardson, J. A., Grant, L. A. & Henderson, R. K. Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology. IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).

    ADS  Article  Google Scholar 

  14. 14.

    Pellegrini, S. et al. Design and performance of an InGaAs–InP single-photon avalanche diode detector. IEEE J. Quantum Electron. 42, 397–403 (2006).

    ADS  Article  Google Scholar 

  15. 15.

    Hadfield, R. H. Single-photon detectors for optical quantum information applications. Nat. Photon. 3, 696–705 (2009).

    ADS  Article  Google Scholar 

  16. 16.

    Zhao, Q.-Y. et al. Single-photon imager based on a superconducting nanowire delay line. Nat. Photon. 11, 247–251 (2017).

    ADS  Article  Google Scholar 

  17. 17.

    Gao, L., Liang, J., Li, C. & Wang, L. V. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature 516, 74–77 (2014).

    ADS  Article  Google Scholar 

  18. 18.

    Goda, K. & Jalali, B. Dispersive Fourier transformation for fast continuous single-shot measurements. Nat. Photon. 7, 102–112 (2013).

    ADS  Article  Google Scholar 

  19. 19.

    Ozaktas, H. M. & Mendlovic, D. Fractional Fourier optics. J. Opt. Soc. Am. A 12, 743 (1995).

    ADS  Article  Google Scholar 

  20. 20.

    Collins, S. A. Analysis of optical resonators involving focusing elements. Appl. Opt. 3, 1263 (1964).

    ADS  Article  Google Scholar 

  21. 21.

    Gigan, S., Lopez, L., Treps, N., Maître, A. & Fabre, C. Image transmission through a stable paraxial cavity. Phys. Rev. A 72, 023804 (2005).

    ADS  Article  Google Scholar 

  22. 22.

    Sultana, P., Takami, A., Matsumoto, T. & Tomita, M. Delayed optical images through coupled-resonator-induced transparency. Opt. Lett. 35, 3414 (2010).

    ADS  Article  Google Scholar 

  23. 23.

    Tomita, M., Sultana, P., Takami, A. & Matsumoto, T. Advanced and delayed images through an image resonator. Opt. Express 18, 12599 (2010).

    ADS  Article  Google Scholar 

  24. 24.

    Wu, D. et al. Ultra-fast lensless computational imaging through 5D frequency analysis of time-resolved light transport. Int. J. Comput. Vis. 110, 128–140 (2014).

    Article  Google Scholar 

  25. 25.

    Heshmat, B., Satat, G., Barsi, C. & Raskar, R. Single-shot ultrafast imaging using parallax-free alignment with a tilted lenslet array. In OSA CLEO: 2014 STu3E.7 (2014).

  26. 26.

    Camacho, R. M., Broadbent, C. J., Ali-Khan, I. & Howell, J. C. All-optical delay of images using slow light. Phys. Rev. Lett. 98, 043902 (2007).

    ADS  Article  Google Scholar 

  27. 27.

    Li, L., Wang, D., Liu, C. & Wang, Q.-H. Ultrathin zoom telescopic objective. Opt. Express 24, 18674–18684 (2016).

    ADS  Article  Google Scholar 

  28. 28.

    Bahrami, M. & Goncharov, A. V. All-spherical catadioptric telescope design for wide-field imaging. Appl. Opt. 49, 5705 (2010).

    ADS  Article  Google Scholar 

  29. 29.

    Siegman, A. E. Unstable optical resonators for laser applications. Proc. IEEE 53, 277–287 (1965).

    Article  Google Scholar 

  30. 30.

    Riazi, A., Gandhi, O. P. & Christensen, D. A. Imaging characteristics of a confocal cavity. Opt. Commun. 28, 163–165 (1979).

    ADS  Article  Google Scholar 

  31. 31.

    Gigan, S. et al. Continuous-wave phase-sensitive parametric image amplification. J. Mod. Opt. 53, 809–820 (2006).

    ADS  Article  Google Scholar 

  32. 32.

    Bergstein, D. A. et al. Resonant cavity imaging: a means toward high-throughput label-free protein detection. IEEE J. Sel. Top. Quantum Electron. 14, 131–139 (2008).

    ADS  Article  Google Scholar 

  33. 33.

    Bouchard, M. B., Chen, B. R., Burgess, S. A. & Hillman, E. M. C. Ultra-fast multispectral optical imaging of cortical oxygenation, blood flow, and intracellular calcium dynamics. Opt. Express 17, 15670–15678 (2009).

    ADS  Article  Google Scholar 

  34. 34.

    Ikoma, H., Heshmat, B., Wetzstein, G. & Raskar, R. Attenuation-corrected fluorescence spectra unmixing for spectroscopy and microscopy. Opt. Express 22, 19469–19483 (2014).

    ADS  Article  Google Scholar 

Download references

Acknowledgements

We thank M. Bawendi and his team members D. Franke and J. J. Yoo at the MIT Department of Chemistry for their help with preparation of fluorescent samples. We also thank A. Bhandari and K. Pulli for discussion of matrix representations and for reading the manuscript.

Author information

Affiliations

Authors

Contributions

B.H. proposed the idea, performed some of the experiments, analysed the data and supervised the project. M.T. performed some of the experiments and analysed the data. G.S. analysed the data and helped with the theoretical expressions. R.R. co-supervised the project. All authors contributed equally to manuscript preparation.

Corresponding author

Correspondence to Barmak Heshmat.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

41566_2018_234_MOESM2_ESM.mp4

Imaging with time-folded compressed lens tube.

41566_2018_234_MOESM3_ESM.mp4

Time-folding enables capturing the evolution of the wavefront in the optical system.

41566_2018_234_MOESM4_ESM.mp4

Turning a streak camera into a multispectral infrared imaging camera using time-folding.

41566_2018_234_MOESM5_ESM.mp4

Turning a streak camera into a multispectral fluorescence lifetime imaging (FLIM) camera using time-folding.

Supplementary Information

This file contains additional information on the modelling and sampling of the approach, discussion and Supplementary Figures.

Supplementary Video 1

Imaging with time-folded compressed lens tube.

Supplementary Video 2

Time-folding enables capturing the evolution of the wavefront in the optical system.

Supplementary Video 3

Turning a streak camera into a multispectral infrared imaging camera using time-folding.

Supplementary Video 4

Turning a streak camera into a multispectral fluorescence lifetime imaging (FLIM) camera using time-folding.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Heshmat, B., Tancik, M., Satat, G. et al. Photography optics in the time dimension. Nature Photon 12, 560–566 (2018). https://doi.org/10.1038/s41566-018-0234-0

Download citation

Further reading