Ultrafast sensors and depth cameras are key enablers for imaging through complex geometries, through scattering, and beyond the line of sight. However, despite accelerating advances in imaging electronics and imaging applications, the optics of such cameras have been inherited from conventional low-speed photography cameras. This has limited ultrafast cameras and their applications to the design constraints of conventional optics. Here, we exploit time as an extra dimension in the optical design and demonstrate that by folding large spaces in time using time-resolved cavities, one can enable new camera capabilities without losing the targeted information. We demonstrate lens tube compression by an order of magnitude, together with ultrafast multi-zoom imaging and ultrafast multispectral imaging by time-folding the optical path at different regions of the imaging optics. Considering the vast variety of designs that could emerge by time-folding conventional imaging optics, we expect this technique to have a broad impact on time-resolved imaging and depth-sensing optics.
Subscribe to Journal
Get full journal access for 1 year
only $8.25 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012).
Gariepy, G., Tonolini, F., Henderson, R., Leach, J. & Faccio, D. Detection and tracking of moving objects hidden from view. Nat. Photon. 10, 23–27 (2016).
Chan, S., Warburton, R. E., Gariepy, G., Leach, J. & Faccio, D. Non-line-of-sight tracking of people at long range. Opt. Express 25, 10109 (2017).
Satat, G., Heshmat, B., Raviv, D. & Raskar, R. All photons imaging through volumetric scattering. Sci. Rep. 6, 33946 (2016).
Satat, G. et al. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nat. Commun. 6, 6796 (2015).
Puszka, A. et al. Time-resolved diffuse optical tomography using fast-gated single-photon avalanche diodes. Biomed. Opt. Express 4, 1351–1365 (2013).
Redo-Sanchez, A. et al. Terahertz time-gated spectral imaging for content extraction through layered structures. Nat. Commun. 7, 12665 (2016).
Heshmat, B., Lee, I. H. & Raskar, R. Optical brush: imaging through permuted probes. Sci. Rep. 6, 20217 (2016).
Bhandari, A., Bourquard, A., Izadi, S. & Raskar, R. Time-resolved image demixing. In IEEE International Conf. on Acoustics, Speech and Signal Processing (ICASSP) 4483–4487 (2016).
Gariepy, G. et al. Single-photon sensitive light-in-fight imaging. Nat. Commun. 6, 6021 (2015).
Buller, G. S. & Wallace, A. Ranging and three-dimensional imaging using time-correlated single-photon counting and point-by-point acquisition. IEEE J. Sel. Top. Quant. Electron. 13, 1006–1015 (2007).
Goda, K., Tsia, K. K. & Jalali, B. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature 458, 1145–1149 (2009).
Richardson, J. A., Grant, L. A. & Henderson, R. K. Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology. IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).
Pellegrini, S. et al. Design and performance of an InGaAs–InP single-photon avalanche diode detector. IEEE J. Quantum Electron. 42, 397–403 (2006).
Hadfield, R. H. Single-photon detectors for optical quantum information applications. Nat. Photon. 3, 696–705 (2009).
Zhao, Q.-Y. et al. Single-photon imager based on a superconducting nanowire delay line. Nat. Photon. 11, 247–251 (2017).
Gao, L., Liang, J., Li, C. & Wang, L. V. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature 516, 74–77 (2014).
Goda, K. & Jalali, B. Dispersive Fourier transformation for fast continuous single-shot measurements. Nat. Photon. 7, 102–112 (2013).
Ozaktas, H. M. & Mendlovic, D. Fractional Fourier optics. J. Opt. Soc. Am. A 12, 743 (1995).
Collins, S. A. Analysis of optical resonators involving focusing elements. Appl. Opt. 3, 1263 (1964).
Gigan, S., Lopez, L., Treps, N., Maître, A. & Fabre, C. Image transmission through a stable paraxial cavity. Phys. Rev. A 72, 023804 (2005).
Sultana, P., Takami, A., Matsumoto, T. & Tomita, M. Delayed optical images through coupled-resonator-induced transparency. Opt. Lett. 35, 3414 (2010).
Tomita, M., Sultana, P., Takami, A. & Matsumoto, T. Advanced and delayed images through an image resonator. Opt. Express 18, 12599 (2010).
Wu, D. et al. Ultra-fast lensless computational imaging through 5D frequency analysis of time-resolved light transport. Int. J. Comput. Vis. 110, 128–140 (2014).
Heshmat, B., Satat, G., Barsi, C. & Raskar, R. Single-shot ultrafast imaging using parallax-free alignment with a tilted lenslet array. In OSA CLEO: 2014 STu3E.7 (2014).
Camacho, R. M., Broadbent, C. J., Ali-Khan, I. & Howell, J. C. All-optical delay of images using slow light. Phys. Rev. Lett. 98, 043902 (2007).
Li, L., Wang, D., Liu, C. & Wang, Q.-H. Ultrathin zoom telescopic objective. Opt. Express 24, 18674–18684 (2016).
Bahrami, M. & Goncharov, A. V. All-spherical catadioptric telescope design for wide-field imaging. Appl. Opt. 49, 5705 (2010).
Siegman, A. E. Unstable optical resonators for laser applications. Proc. IEEE 53, 277–287 (1965).
Riazi, A., Gandhi, O. P. & Christensen, D. A. Imaging characteristics of a confocal cavity. Opt. Commun. 28, 163–165 (1979).
Gigan, S. et al. Continuous-wave phase-sensitive parametric image amplification. J. Mod. Opt. 53, 809–820 (2006).
Bergstein, D. A. et al. Resonant cavity imaging: a means toward high-throughput label-free protein detection. IEEE J. Sel. Top. Quantum Electron. 14, 131–139 (2008).
Bouchard, M. B., Chen, B. R., Burgess, S. A. & Hillman, E. M. C. Ultra-fast multispectral optical imaging of cortical oxygenation, blood flow, and intracellular calcium dynamics. Opt. Express 17, 15670–15678 (2009).
Ikoma, H., Heshmat, B., Wetzstein, G. & Raskar, R. Attenuation-corrected fluorescence spectra unmixing for spectroscopy and microscopy. Opt. Express 22, 19469–19483 (2014).
We thank M. Bawendi and his team members D. Franke and J. J. Yoo at the MIT Department of Chemistry for their help with preparation of fluorescent samples. We also thank A. Bhandari and K. Pulli for discussion of matrix representations and for reading the manuscript.
The authors declare no competing interests.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This file contains additional information on the modelling and sampling of the approach, discussion and Supplementary Figures.
Imaging with time-folded compressed lens tube.
Time-folding enables capturing the evolution of the wavefront in the optical system.
Turning a streak camera into a multispectral infrared imaging camera using time-folding.
Turning a streak camera into a multispectral fluorescence lifetime imaging (FLIM) camera using time-folding.
About this article
Cite this article
Heshmat, B., Tancik, M., Satat, G. et al. Photography optics in the time dimension. Nature Photon 12, 560–566 (2018). https://doi.org/10.1038/s41566-018-0234-0
Nature Communications (2021)
Nature Methods (2020)
Nature Communications (2019)
Nature Photonics (2018)