Abstract
Ultrafast sensors and depth cameras are key enablers for imaging through complex geometries, through scattering, and beyond the line of sight. However, despite accelerating advances in imaging electronics and imaging applications, the optics of such cameras have been inherited from conventional low-speed photography cameras. This has limited ultrafast cameras and their applications to the design constraints of conventional optics. Here, we exploit time as an extra dimension in the optical design and demonstrate that by folding large spaces in time using time-resolved cavities, one can enable new camera capabilities without losing the targeted information. We demonstrate lens tube compression by an order of magnitude, together with ultrafast multi-zoom imaging and ultrafast multispectral imaging by time-folding the optical path at different regions of the imaging optics. Considering the vast variety of designs that could emerge by time-folding conventional imaging optics, we expect this technique to have a broad impact on time-resolved imaging and depth-sensing optics.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012).
Gariepy, G., Tonolini, F., Henderson, R., Leach, J. & Faccio, D. Detection and tracking of moving objects hidden from view. Nat. Photon. 10, 23–27 (2016).
Chan, S., Warburton, R. E., Gariepy, G., Leach, J. & Faccio, D. Non-line-of-sight tracking of people at long range. Opt. Express 25, 10109 (2017).
Satat, G., Heshmat, B., Raviv, D. & Raskar, R. All photons imaging through volumetric scattering. Sci. Rep. 6, 33946 (2016).
Satat, G. et al. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nat. Commun. 6, 6796 (2015).
Puszka, A. et al. Time-resolved diffuse optical tomography using fast-gated single-photon avalanche diodes. Biomed. Opt. Express 4, 1351–1365 (2013).
Redo-Sanchez, A. et al. Terahertz time-gated spectral imaging for content extraction through layered structures. Nat. Commun. 7, 12665 (2016).
Heshmat, B., Lee, I. H. & Raskar, R. Optical brush: imaging through permuted probes. Sci. Rep. 6, 20217 (2016).
Bhandari, A., Bourquard, A., Izadi, S. & Raskar, R. Time-resolved image demixing. In IEEE International Conf. on Acoustics, Speech and Signal Processing (ICASSP) 4483–4487 (2016).
Gariepy, G. et al. Single-photon sensitive light-in-fight imaging. Nat. Commun. 6, 6021 (2015).
Buller, G. S. & Wallace, A. Ranging and three-dimensional imaging using time-correlated single-photon counting and point-by-point acquisition. IEEE J. Sel. Top. Quant. Electron. 13, 1006–1015 (2007).
Goda, K., Tsia, K. K. & Jalali, B. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature 458, 1145–1149 (2009).
Richardson, J. A., Grant, L. A. & Henderson, R. K. Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology. IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).
Pellegrini, S. et al. Design and performance of an InGaAs–InP single-photon avalanche diode detector. IEEE J. Quantum Electron. 42, 397–403 (2006).
Hadfield, R. H. Single-photon detectors for optical quantum information applications. Nat. Photon. 3, 696–705 (2009).
Zhao, Q.-Y. et al. Single-photon imager based on a superconducting nanowire delay line. Nat. Photon. 11, 247–251 (2017).
Gao, L., Liang, J., Li, C. & Wang, L. V. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature 516, 74–77 (2014).
Goda, K. & Jalali, B. Dispersive Fourier transformation for fast continuous single-shot measurements. Nat. Photon. 7, 102–112 (2013).
Ozaktas, H. M. & Mendlovic, D. Fractional Fourier optics. J. Opt. Soc. Am. A 12, 743 (1995).
Collins, S. A. Analysis of optical resonators involving focusing elements. Appl. Opt. 3, 1263 (1964).
Gigan, S., Lopez, L., Treps, N., Maître, A. & Fabre, C. Image transmission through a stable paraxial cavity. Phys. Rev. A 72, 023804 (2005).
Sultana, P., Takami, A., Matsumoto, T. & Tomita, M. Delayed optical images through coupled-resonator-induced transparency. Opt. Lett. 35, 3414 (2010).
Tomita, M., Sultana, P., Takami, A. & Matsumoto, T. Advanced and delayed images through an image resonator. Opt. Express 18, 12599 (2010).
Wu, D. et al. Ultra-fast lensless computational imaging through 5D frequency analysis of time-resolved light transport. Int. J. Comput. Vis. 110, 128–140 (2014).
Heshmat, B., Satat, G., Barsi, C. & Raskar, R. Single-shot ultrafast imaging using parallax-free alignment with a tilted lenslet array. In OSA CLEO: 2014 STu3E.7 (2014).
Camacho, R. M., Broadbent, C. J., Ali-Khan, I. & Howell, J. C. All-optical delay of images using slow light. Phys. Rev. Lett. 98, 043902 (2007).
Li, L., Wang, D., Liu, C. & Wang, Q.-H. Ultrathin zoom telescopic objective. Opt. Express 24, 18674–18684 (2016).
Bahrami, M. & Goncharov, A. V. All-spherical catadioptric telescope design for wide-field imaging. Appl. Opt. 49, 5705 (2010).
Siegman, A. E. Unstable optical resonators for laser applications. Proc. IEEE 53, 277–287 (1965).
Riazi, A., Gandhi, O. P. & Christensen, D. A. Imaging characteristics of a confocal cavity. Opt. Commun. 28, 163–165 (1979).
Gigan, S. et al. Continuous-wave phase-sensitive parametric image amplification. J. Mod. Opt. 53, 809–820 (2006).
Bergstein, D. A. et al. Resonant cavity imaging: a means toward high-throughput label-free protein detection. IEEE J. Sel. Top. Quantum Electron. 14, 131–139 (2008).
Bouchard, M. B., Chen, B. R., Burgess, S. A. & Hillman, E. M. C. Ultra-fast multispectral optical imaging of cortical oxygenation, blood flow, and intracellular calcium dynamics. Opt. Express 17, 15670–15678 (2009).
Ikoma, H., Heshmat, B., Wetzstein, G. & Raskar, R. Attenuation-corrected fluorescence spectra unmixing for spectroscopy and microscopy. Opt. Express 22, 19469–19483 (2014).
Acknowledgements
We thank M. Bawendi and his team members D. Franke and J. J. Yoo at the MIT Department of Chemistry for their help with preparation of fluorescent samples. We also thank A. Bhandari and K. Pulli for discussion of matrix representations and for reading the manuscript.
Author information
Authors and Affiliations
Contributions
B.H. proposed the idea, performed some of the experiments, analysed the data and supervised the project. M.T. performed some of the experiments and analysed the data. G.S. analysed the data and helped with the theoretical expressions. R.R. co-supervised the project. All authors contributed equally to manuscript preparation.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
This file contains additional information on the modelling and sampling of the approach, discussion and Supplementary Figures.
Supplementary Video 1
Imaging with time-folded compressed lens tube.
Supplementary Video 2
Time-folding enables capturing the evolution of the wavefront in the optical system.
Supplementary Video 3
Turning a streak camera into a multispectral infrared imaging camera using time-folding.
Supplementary Video 4
Turning a streak camera into a multispectral fluorescence lifetime imaging (FLIM) camera using time-folding.
Rights and permissions
About this article
Cite this article
Heshmat, B., Tancik, M., Satat, G. et al. Photography optics in the time dimension. Nature Photon 12, 560–566 (2018). https://doi.org/10.1038/s41566-018-0234-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41566-018-0234-0
This article is cited by
-
Stray light characterization with ultrafast time-of-flight imaging
Scientific Reports (2021)
-
Ultrafast light field tomography for snapshot transient and non-line-of-sight imaging
Nature Communications (2021)
-
Simultaneous multiplane imaging with reverberation two-photon microscopy
Nature Methods (2020)
-
Electro-optic imaging enables efficient wide-field fluorescence lifetime microscopy
Nature Communications (2019)
-
Folding photography in the time domain
Nature Photonics (2018)