Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Photography optics in the time dimension

Abstract

Ultrafast sensors and depth cameras are key enablers for imaging through complex geometries, through scattering, and beyond the line of sight. However, despite accelerating advances in imaging electronics and imaging applications, the optics of such cameras have been inherited from conventional low-speed photography cameras. This has limited ultrafast cameras and their applications to the design constraints of conventional optics. Here, we exploit time as an extra dimension in the optical design and demonstrate that by folding large spaces in time using time-resolved cavities, one can enable new camera capabilities without losing the targeted information. We demonstrate lens tube compression by an order of magnitude, together with ultrafast multi-zoom imaging and ultrafast multispectral imaging by time-folding the optical path at different regions of the imaging optics. Considering the vast variety of designs that could emerge by time-folding conventional imaging optics, we expect this technique to have a broad impact on time-resolved imaging and depth-sensing optics.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Experimental set-up and time-folded imaging overview.
Fig. 2: Spatial compression of imaging optics by time-folding the focal length.
Fig. 3: Multi-zoom ultrafast camera by time-folding the entrance lens.
Fig. 4: Multispectral ultrafast camera by wavelength-dependent time-folding.

Similar content being viewed by others

References

  1. Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012).

    Article  Google Scholar 

  2. Gariepy, G., Tonolini, F., Henderson, R., Leach, J. & Faccio, D. Detection and tracking of moving objects hidden from view. Nat. Photon. 10, 23–27 (2016).

    Article  ADS  Google Scholar 

  3. Chan, S., Warburton, R. E., Gariepy, G., Leach, J. & Faccio, D. Non-line-of-sight tracking of people at long range. Opt. Express 25, 10109 (2017).

    Article  Google Scholar 

  4. Satat, G., Heshmat, B., Raviv, D. & Raskar, R. All photons imaging through volumetric scattering. Sci. Rep. 6, 33946 (2016).

    Article  ADS  Google Scholar 

  5. Satat, G. et al. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nat. Commun. 6, 6796 (2015).

    Article  Google Scholar 

  6. Puszka, A. et al. Time-resolved diffuse optical tomography using fast-gated single-photon avalanche diodes. Biomed. Opt. Express 4, 1351–1365 (2013).

    Article  Google Scholar 

  7. Redo-Sanchez, A. et al. Terahertz time-gated spectral imaging for content extraction through layered structures. Nat. Commun. 7, 12665 (2016).

    Article  ADS  Google Scholar 

  8. Heshmat, B., Lee, I. H. & Raskar, R. Optical brush: imaging through permuted probes. Sci. Rep. 6, 20217 (2016).

    Article  ADS  Google Scholar 

  9. Bhandari, A., Bourquard, A., Izadi, S. & Raskar, R. Time-resolved image demixing. In IEEE International Conf. on Acoustics, Speech and Signal Processing (ICASSP) 4483–4487 (2016).

  10. Gariepy, G. et al. Single-photon sensitive light-in-fight imaging. Nat. Commun. 6, 6021 (2015).

    Article  Google Scholar 

  11. Buller, G. S. & Wallace, A. Ranging and three-dimensional imaging using time-correlated single-photon counting and point-by-point acquisition. IEEE J. Sel. Top. Quant. Electron. 13, 1006–1015 (2007).

    Article  ADS  Google Scholar 

  12. Goda, K., Tsia, K. K. & Jalali, B. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature 458, 1145–1149 (2009).

    Article  ADS  Google Scholar 

  13. Richardson, J. A., Grant, L. A. & Henderson, R. K. Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology. IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).

    Article  ADS  Google Scholar 

  14. Pellegrini, S. et al. Design and performance of an InGaAs–InP single-photon avalanche diode detector. IEEE J. Quantum Electron. 42, 397–403 (2006).

    Article  ADS  Google Scholar 

  15. Hadfield, R. H. Single-photon detectors for optical quantum information applications. Nat. Photon. 3, 696–705 (2009).

    Article  ADS  Google Scholar 

  16. Zhao, Q.-Y. et al. Single-photon imager based on a superconducting nanowire delay line. Nat. Photon. 11, 247–251 (2017).

    Article  ADS  Google Scholar 

  17. Gao, L., Liang, J., Li, C. & Wang, L. V. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature 516, 74–77 (2014).

    Article  ADS  Google Scholar 

  18. Goda, K. & Jalali, B. Dispersive Fourier transformation for fast continuous single-shot measurements. Nat. Photon. 7, 102–112 (2013).

    Article  ADS  Google Scholar 

  19. Ozaktas, H. M. & Mendlovic, D. Fractional Fourier optics. J. Opt. Soc. Am. A 12, 743 (1995).

    Article  ADS  Google Scholar 

  20. Collins, S. A. Analysis of optical resonators involving focusing elements. Appl. Opt. 3, 1263 (1964).

    Article  ADS  Google Scholar 

  21. Gigan, S., Lopez, L., Treps, N., Maître, A. & Fabre, C. Image transmission through a stable paraxial cavity. Phys. Rev. A 72, 023804 (2005).

    Article  ADS  Google Scholar 

  22. Sultana, P., Takami, A., Matsumoto, T. & Tomita, M. Delayed optical images through coupled-resonator-induced transparency. Opt. Lett. 35, 3414 (2010).

    Article  ADS  Google Scholar 

  23. Tomita, M., Sultana, P., Takami, A. & Matsumoto, T. Advanced and delayed images through an image resonator. Opt. Express 18, 12599 (2010).

    Article  ADS  Google Scholar 

  24. Wu, D. et al. Ultra-fast lensless computational imaging through 5D frequency analysis of time-resolved light transport. Int. J. Comput. Vis. 110, 128–140 (2014).

    Article  Google Scholar 

  25. Heshmat, B., Satat, G., Barsi, C. & Raskar, R. Single-shot ultrafast imaging using parallax-free alignment with a tilted lenslet array. In OSA CLEO: 2014 STu3E.7 (2014).

  26. Camacho, R. M., Broadbent, C. J., Ali-Khan, I. & Howell, J. C. All-optical delay of images using slow light. Phys. Rev. Lett. 98, 043902 (2007).

    Article  ADS  Google Scholar 

  27. Li, L., Wang, D., Liu, C. & Wang, Q.-H. Ultrathin zoom telescopic objective. Opt. Express 24, 18674–18684 (2016).

    Article  ADS  Google Scholar 

  28. Bahrami, M. & Goncharov, A. V. All-spherical catadioptric telescope design for wide-field imaging. Appl. Opt. 49, 5705 (2010).

    Article  ADS  Google Scholar 

  29. Siegman, A. E. Unstable optical resonators for laser applications. Proc. IEEE 53, 277–287 (1965).

    Article  Google Scholar 

  30. Riazi, A., Gandhi, O. P. & Christensen, D. A. Imaging characteristics of a confocal cavity. Opt. Commun. 28, 163–165 (1979).

    Article  ADS  Google Scholar 

  31. Gigan, S. et al. Continuous-wave phase-sensitive parametric image amplification. J. Mod. Opt. 53, 809–820 (2006).

    Article  ADS  Google Scholar 

  32. Bergstein, D. A. et al. Resonant cavity imaging: a means toward high-throughput label-free protein detection. IEEE J. Sel. Top. Quantum Electron. 14, 131–139 (2008).

    Article  ADS  Google Scholar 

  33. Bouchard, M. B., Chen, B. R., Burgess, S. A. & Hillman, E. M. C. Ultra-fast multispectral optical imaging of cortical oxygenation, blood flow, and intracellular calcium dynamics. Opt. Express 17, 15670–15678 (2009).

    Article  ADS  Google Scholar 

  34. Ikoma, H., Heshmat, B., Wetzstein, G. & Raskar, R. Attenuation-corrected fluorescence spectra unmixing for spectroscopy and microscopy. Opt. Express 22, 19469–19483 (2014).

    Article  ADS  Google Scholar 

Download references

Acknowledgements

We thank M. Bawendi and his team members D. Franke and J. J. Yoo at the MIT Department of Chemistry for their help with preparation of fluorescent samples. We also thank A. Bhandari and K. Pulli for discussion of matrix representations and for reading the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

B.H. proposed the idea, performed some of the experiments, analysed the data and supervised the project. M.T. performed some of the experiments and analysed the data. G.S. analysed the data and helped with the theoretical expressions. R.R. co-supervised the project. All authors contributed equally to manuscript preparation.

Corresponding author

Correspondence to Barmak Heshmat.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

This file contains additional information on the modelling and sampling of the approach, discussion and Supplementary Figures.

Supplementary Video 1

Imaging with time-folded compressed lens tube.

Supplementary Video 2

Time-folding enables capturing the evolution of the wavefront in the optical system.

Supplementary Video 3

Turning a streak camera into a multispectral infrared imaging camera using time-folding.

Supplementary Video 4

Turning a streak camera into a multispectral fluorescence lifetime imaging (FLIM) camera using time-folding.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Heshmat, B., Tancik, M., Satat, G. et al. Photography optics in the time dimension. Nature Photon 12, 560–566 (2018). https://doi.org/10.1038/s41566-018-0234-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41566-018-0234-0

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing