Article | Published:

Photography optics in the time dimension

Nature Photonicsvolume 12pages560566 (2018) | Download Citation

Abstract

Ultrafast sensors and depth cameras are key enablers for imaging through complex geometries, through scattering, and beyond the line of sight. However, despite accelerating advances in imaging electronics and imaging applications, the optics of such cameras have been inherited from conventional low-speed photography cameras. This has limited ultrafast cameras and their applications to the design constraints of conventional optics. Here, we exploit time as an extra dimension in the optical design and demonstrate that by folding large spaces in time using time-resolved cavities, one can enable new camera capabilities without losing the targeted information. We demonstrate lens tube compression by an order of magnitude, together with ultrafast multi-zoom imaging and ultrafast multispectral imaging by time-folding the optical path at different regions of the imaging optics. Considering the vast variety of designs that could emerge by time-folding conventional imaging optics, we expect this technique to have a broad impact on time-resolved imaging and depth-sensing optics.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nat. Commun. 3, 745 (2012).

  2. 2.

    Gariepy, G., Tonolini, F., Henderson, R., Leach, J. & Faccio, D. Detection and tracking of moving objects hidden from view. Nat. Photon. 10, 23–27 (2016).

  3. 3.

    Chan, S., Warburton, R. E., Gariepy, G., Leach, J. & Faccio, D. Non-line-of-sight tracking of people at long range. Opt. Express 25, 10109 (2017).

  4. 4.

    Satat, G., Heshmat, B., Raviv, D. & Raskar, R. All photons imaging through volumetric scattering. Sci. Rep. 6, 33946 (2016).

  5. 5.

    Satat, G. et al. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nat. Commun. 6, 6796 (2015).

  6. 6.

    Puszka, A. et al. Time-resolved diffuse optical tomography using fast-gated single-photon avalanche diodes. Biomed. Opt. Express 4, 1351–1365 (2013).

  7. 7.

    Redo-Sanchez, A. et al. Terahertz time-gated spectral imaging for content extraction through layered structures. Nat. Commun. 7, 12665 (2016).

  8. 8.

    Heshmat, B., Lee, I. H. & Raskar, R. Optical brush: imaging through permuted probes. Sci. Rep. 6, 20217 (2016).

  9. 9.

    Bhandari, A., Bourquard, A., Izadi, S. & Raskar, R. Time-resolved image demixing. In IEEE International Conf. on Acoustics, Speech and Signal Processing (ICASSP) 4483–4487 (2016).

  10. 10.

    Gariepy, G. et al. Single-photon sensitive light-in-fight imaging. Nat. Commun. 6, 6021 (2015).

  11. 11.

    Buller, G. S. & Wallace, A. Ranging and three-dimensional imaging using time-correlated single-photon counting and point-by-point acquisition. IEEE J. Sel. Top. Quant. Electron. 13, 1006–1015 (2007).

  12. 12.

    Goda, K., Tsia, K. K. & Jalali, B. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature 458, 1145–1149 (2009).

  13. 13.

    Richardson, J. A., Grant, L. A. & Henderson, R. K. Low dark count single-photon avalanche diode structure compatible with standard nanometer scale CMOS technology. IEEE Photon. Technol. Lett. 21, 1020–1022 (2009).

  14. 14.

    Pellegrini, S. et al. Design and performance of an InGaAs–InP single-photon avalanche diode detector. IEEE J. Quantum Electron. 42, 397–403 (2006).

  15. 15.

    Hadfield, R. H. Single-photon detectors for optical quantum information applications. Nat. Photon. 3, 696–705 (2009).

  16. 16.

    Zhao, Q.-Y. et al. Single-photon imager based on a superconducting nanowire delay line. Nat. Photon. 11, 247–251 (2017).

  17. 17.

    Gao, L., Liang, J., Li, C. & Wang, L. V. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature 516, 74–77 (2014).

  18. 18.

    Goda, K. & Jalali, B. Dispersive Fourier transformation for fast continuous single-shot measurements. Nat. Photon. 7, 102–112 (2013).

  19. 19.

    Ozaktas, H. M. & Mendlovic, D. Fractional Fourier optics. J. Opt. Soc. Am. A 12, 743 (1995).

  20. 20.

    Collins, S. A. Analysis of optical resonators involving focusing elements. Appl. Opt. 3, 1263 (1964).

  21. 21.

    Gigan, S., Lopez, L., Treps, N., Maître, A. & Fabre, C. Image transmission through a stable paraxial cavity. Phys. Rev. A 72, 023804 (2005).

  22. 22.

    Sultana, P., Takami, A., Matsumoto, T. & Tomita, M. Delayed optical images through coupled-resonator-induced transparency. Opt. Lett. 35, 3414 (2010).

  23. 23.

    Tomita, M., Sultana, P., Takami, A. & Matsumoto, T. Advanced and delayed images through an image resonator. Opt. Express 18, 12599 (2010).

  24. 24.

    Wu, D. et al. Ultra-fast lensless computational imaging through 5D frequency analysis of time-resolved light transport. Int. J. Comput. Vis. 110, 128–140 (2014).

  25. 25.

    Heshmat, B., Satat, G., Barsi, C. & Raskar, R. Single-shot ultrafast imaging using parallax-free alignment with a tilted lenslet array. In OSA CLEO: 2014 STu3E.7 (2014).

  26. 26.

    Camacho, R. M., Broadbent, C. J., Ali-Khan, I. & Howell, J. C. All-optical delay of images using slow light. Phys. Rev. Lett. 98, 043902 (2007).

  27. 27.

    Li, L., Wang, D., Liu, C. & Wang, Q.-H. Ultrathin zoom telescopic objective. Opt. Express 24, 18674–18684 (2016).

  28. 28.

    Bahrami, M. & Goncharov, A. V. All-spherical catadioptric telescope design for wide-field imaging. Appl. Opt. 49, 5705 (2010).

  29. 29.

    Siegman, A. E. Unstable optical resonators for laser applications. Proc. IEEE 53, 277–287 (1965).

  30. 30.

    Riazi, A., Gandhi, O. P. & Christensen, D. A. Imaging characteristics of a confocal cavity. Opt. Commun. 28, 163–165 (1979).

  31. 31.

    Gigan, S. et al. Continuous-wave phase-sensitive parametric image amplification. J. Mod. Opt. 53, 809–820 (2006).

  32. 32.

    Bergstein, D. A. et al. Resonant cavity imaging: a means toward high-throughput label-free protein detection. IEEE J. Sel. Top. Quantum Electron. 14, 131–139 (2008).

  33. 33.

    Bouchard, M. B., Chen, B. R., Burgess, S. A. & Hillman, E. M. C. Ultra-fast multispectral optical imaging of cortical oxygenation, blood flow, and intracellular calcium dynamics. Opt. Express 17, 15670–15678 (2009).

  34. 34.

    Ikoma, H., Heshmat, B., Wetzstein, G. & Raskar, R. Attenuation-corrected fluorescence spectra unmixing for spectroscopy and microscopy. Opt. Express 22, 19469–19483 (2014).

Download references

Acknowledgements

We thank M. Bawendi and his team members D. Franke and J. J. Yoo at the MIT Department of Chemistry for their help with preparation of fluorescent samples. We also thank A. Bhandari and K. Pulli for discussion of matrix representations and for reading the manuscript.

Author information

Affiliations

  1. Media Lab, Massachusetts Institute of Technology, Cambridge, MA, USA

    • Barmak Heshmat
    • , Guy Satat
    •  & Ramesh Raskar
  2. Computer Science and Electrical Engineering, Cambridge, MA, USA

    • Matthew Tancik

Authors

  1. Search for Barmak Heshmat in:

  2. Search for Matthew Tancik in:

  3. Search for Guy Satat in:

  4. Search for Ramesh Raskar in:

Contributions

B.H. proposed the idea, performed some of the experiments, analysed the data and supervised the project. M.T. performed some of the experiments and analysed the data. G.S. analysed the data and helped with the theoretical expressions. R.R. co-supervised the project. All authors contributed equally to manuscript preparation.

Competing interests

The authors declare no competing interests.

Corresponding author

Correspondence to Barmak Heshmat.

Supplementary information

  1. Supplementary Information

    This file contains additional information on the modelling and sampling of the approach, discussion and Supplementary Figures.

  2. Supplementary Video 1

    Imaging with time-folded compressed lens tube.

  3. Supplementary Video 2

    Time-folding enables capturing the evolution of the wavefront in the optical system.

  4. Supplementary Video 3

    Turning a streak camera into a multispectral infrared imaging camera using time-folding.

  5. Supplementary Video 4

    Turning a streak camera into a multispectral fluorescence lifetime imaging (FLIM) camera using time-folding.

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/s41566-018-0234-0

Further reading