Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Single-shot compressed ultrafast photography at one hundred billion frames per second

Abstract

The capture of transient scenes at high imaging speed has been long sought by photographers1,2,3,4, with early examples being the well known recording in 1878 of a horse in motion5 and the 1887 photograph of a supersonic bullet6. However, not until the late twentieth century were breakthroughs achieved in demonstrating ultrahigh-speed imaging (more than 105 frames per second)7. In particular, the introduction of electronic imaging sensors based on the charge-coupled device (CCD) or complementary metal–oxide–semiconductor (CMOS) technology revolutionized high-speed photography, enabling acquisition rates of up to 107 frames per second8. Despite these sensors’ widespread impact, further increasing frame rates using CCD or CMOS technology is fundamentally limited by their on-chip storage and electronic readout speed9. Here we demonstrate a two-dimensional dynamic imaging technique, compressed ultrafast photography (CUP), which can capture non-repetitive time-evolving events at up to 1011 frames per second. Compared with existing ultrafast imaging techniques, CUP has the prominent advantage of measuring an xyt (x, y, spatial coordinates; t, time) scene with a single camera snapshot, thereby allowing observation of transient events with temporal resolution as tens of picoseconds. Furthermore, akin to traditional photography, CUP is receive-only, and so does not need the specialized active illumination required by other single-shot ultrafast imagers2,3. As a result, CUP can image a variety of luminescent—such as fluorescent or bioluminescent—objects. Using CUP, we visualize four fundamental physical phenomena with single laser shots only: laser pulse reflection and refraction, photon racing in two media, and faster-than-light propagation of non-information (that is, motion that appears faster than the speed of light but cannot convey information). Given CUP’s capability, we expect it to find widespread applications in both fundamental and applied sciences, including biomedical research.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: CUP system configuration.
Figure 2: Characterization of CUP’s spatial frequency responses.
Figure 3: CUP of laser pulse reflection, refraction, and racing in different media.
Figure 4: CUP of faster-than-light propagation of non-information.

References

  1. Fuller, P. W. W. An introduction to high speed photography and photonics. Imaging Sci. J. 57, 293–302 (2009)

    Article  Google Scholar 

  2. Li, Z., Zgadzaj, R., Wang, X., Chang, Y.-Y. & Downer, M. C. Single-shot tomographic movies of evolving light-velocity objects. Nature Commun. 5, 3085 (2014)

    Article  ADS  Google Scholar 

  3. Nakagawa, K. et al. Sequentially timed all-optical mapping photography (STAMP). Nature Photon. 8, 695–700 (2014)

    Article  CAS  ADS  Google Scholar 

  4. Heshmat, B., Satat, G., Barsi, C. & Raskar, R. Single-shot ultrafast imaging using parallax-free alignment with a tilted lenslet array. CLEO Sci. Innov. http://dx.doi.org/10.1364/CLEO_SI.2014.STu3E.7 (2014)

  5. Munn, O. & Beach, A. A horse’s motion scientifically determined. Sci. Am. 39, 241 (1878)

    Google Scholar 

  6. Mach, E. & Salcher, P. Photographische Fixirung der durch Projectile in der Luft eingeleiteten Vorgänge. Ann. Phys. 268, 277–291 (1887)

    Article  Google Scholar 

  7. Nebeker, S. Rotating mirror cameras. High Speed Photogr. Photon. Newsl. 3, 31–37 (1983)

    Google Scholar 

  8. Kondo, Y. et al. Development of “HyperVision HPV-X” high-speed video camera. Shimadzu Rev. 69, 285–291 (2012)

    Google Scholar 

  9. El-Desouki, M. et al. CMOS image sensors for high speed applications. Sensors 9, 430–444 (2009)

    Article  Google Scholar 

  10. Guide. to Streak Camerashttp://www.hamamatsu.com/resources/pdf/sys/e_streakh.pdf (Hamamatsu Corp., Hamamatsu City, Japan, 2002)

  11. Velten, A., Lawson, E., Bardagjy, A., Bawendi, M. & Raskar, R. Slow art with a trillion frames per second camera. Proc. SIGGRAPH 44 (2011)

  12. Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging. Nature Commun. 3, 745 (2012)

    Article  ADS  Google Scholar 

  13. Solli, D. R., Ropers, C., Koonath, P. & Jalali, B. Optical rogue waves. Nature 450, 1054–1057 (2007)

    Article  CAS  ADS  Google Scholar 

  14. Eldar, Y. C. & Kutyniok, G. Compressed Sensing: Theory and Applications (Cambridge Univ. Press, 2012)

    Book  Google Scholar 

  15. Figueiredo, M. A. T., Nowak, R. D. & Wright, S. J. Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Top. Signal Process. 1, 586–597 (2007)

    Article  ADS  Google Scholar 

  16. Bioucas-Dias, J. M. & Figueiredo, M. A. T. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process. 16, 2992–3004 (2007)

    Article  ADS  MathSciNet  Google Scholar 

  17. Afonso, M. V., Bioucas-Dias, J. M. & Figueiredo, M. A. T. Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19, 2345–2356 (2010)

    Article  ADS  MathSciNet  Google Scholar 

  18. Wright, S. J., Nowak, R. D. & Figueiredo, M. A. T. Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57, 2479–2493 (2009)

    Article  ADS  MathSciNet  Google Scholar 

  19. Afonso, M. V., Bioucas-Dias, J. M. & Figueiredo, M. A. T. An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Trans. Image Process. 20, 681–695 (2011)

    Article  ADS  MathSciNet  Google Scholar 

  20. Einstein, A. Relativity: The Special and General Theory (Penguin, 1920)

    MATH  Google Scholar 

  21. Agrawal, S. & Vishwanath, S. Secrecy using compressive sensing. Proc. Inform. Theory Workshop, IEEE, 563–567 (2011)

  22. Orsdemir, A., Altun, H. O., Sharma, G. & Bocko, M. F. On the security and robustness of encryption via compressed sensing. Proc. Military Commun. Conf., IEEE, 1–7 (2008)

  23. Davis, D. L. Secure compressed imaging. US patent 5,907. 619 (1999)

  24. Borst, J. W. & Visser, A. J. W. G. Fluorescence lifetime imaging microscopy in life sciences. Meas. Sci. Technol. 21, 102002 (2010)

    Article  ADS  Google Scholar 

  25. Figueiredo, M. A. T., Nowak, R. D. & Wright, S. J. Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE Sel. Top. Signal Process. 1, 586–597 (2007)

    Article  ADS  Google Scholar 

  26. Bioucas-Dias, J. M. & Figueiredo, M. A. T. A new TwIST: two-step iterative shrinkage/thresholding algorithms for image restoration. IEEE Trans. Image Process. 16, 2992–3004 (2007)

    Article  ADS  MathSciNet  Google Scholar 

  27. Afonso, M. V., Bioucas-Dias, J. M. & Figueiredo, M. A. T. Fast image recovery using variable splitting and constrained optimization. IEEE Trans. Image Process. 19, 2345–2356 (2010)

    Article  ADS  MathSciNet  Google Scholar 

  28. Wright, S. J., Nowak, R. D. & Figueiredo, M. A. T. Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57, 2479–2493 (2009)

    Article  ADS  MathSciNet  Google Scholar 

  29. Afonso, M. V., Bioucas-Dias, J. M. & Figueiredo, M. A. T. An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems. IEEE Trans. Image Process. 20, 681–695 (2011)

    Article  ADS  MathSciNet  Google Scholar 

  30. Selanger, K. A., Falnes, J. & Sikkeland, T. Fluorescence lifetime studies of rhodamine 6G in methanol. J. Phys. Chem. 81, 1960–1963 (1977)

    Article  CAS  Google Scholar 

  31. Rentzepis, P. M., Topp, M. R., Jones, R. P. & Jortner, J. Picosecond emission spectroscopy of homogeneously broadened, electronically excited molecular states. Phys. Rev. Lett. 25, 1742–1745 (1970).

    Article  CAS  ADS  Google Scholar 

Download references

Acknowledgements

We thank N. Hagen for discussions and J. Ballard for a close reading of the manuscript. We also acknowledge Texas Instruments for providing the DLP device. This work was supported in part by National Institutes of Health grants DP1 EB016986 (NIH Director’s Pioneer Award) and R01 CA186567 (NIH Director’s Transformative Research Award).

Author information

Authors and Affiliations

Authors

Contributions

L.G. built the system, performed the experiments, analysed the data and prepared the manuscript. J.L. performed some of the experiments, analysed the data and prepared the manuscript. C.L. prepared the sample and performed some of the experiments. L.V.W. contributed to the conceptual system, experimental design and manuscript preparation.

Corresponding author

Correspondence to Lihong V. Wang.

Ethics declarations

Competing interests

L.V.W. has a financial interest in Microphotoacoustics, Inc. and Endra, Inc., which, however, did not support this work.

Extended data figures and tables

Extended Data Figure 1 CUP image formation model.

x, y, spatial coordinates; t, time; m, n, k, matrix indices; Im,n,k, input dynamic scene element; Cm,n, coded mask matrix element; Cm,n−kIm,n−k,k, encoded and sheared scene element; Em,n, image element energy measured by a 2D detector array; tmax, maximum recording time. See Methods for details.

Extended Data Figure 2

A temporally undispersed CCD image of the coded mask, which encodes the uniformly illuminated field with a pseudo-random binary pattern. The position of the mask and definition of the x′–y′ plane are shown in Fig. 1.

Extended Data Figure 3 Multicolour CUP.

a, Custom-built spectral separation unit. b, Representative temporal frames of a pulsed-laser-pumped fluorescence emission process. The pulsed pump laser and fluorescence emission are pseudo-coloured based on their peak emission wavelengths. To explicitly indicate the spatiotemporal pattern of this event, the CUP-reconstructed frames are overlaid with a static background image captured by a monochromatic CCD camera. All temporal frames of this event are provided in Supplementary Video 6. c, Time-lapse pump laser and fluorescence emission intensities averaged within the dashed box in b. The temporal responses of pump laser excitation and fluorescence decay are fitted to a Gaussian function and an exponential function, respectively. The recovered fluorescence lifetime of Rhodamine 6G is 3.8 ns. d, Event function describing the pulsed laser fluorescence excitation. e, Event function describing the fluorescence emission. f, Measured temporal PSF, with a full width at half maximum (FWHM) of 80 ps. Owing to reconstruction artefacts, the PSF has a side lobe and a shoulder extending over a range of 280 ps. g, Simulated temporal responses of event functions d and e after being convolved with the temporal PSF. The maxima of these two time-lapse signals are stretched by 200 ps. Scale bar, 10 mm.

Supplementary information

Light sweeping across a stripe pattern with varied spatial frequencies

Light sweeping across a stripe pattern with varied spatial frequencies. (MOV 107 kb)

Laser pulse reflection from a mirror

Laser pulse reflection from a mirror. (MOV 22 kb)

Laser pulse refraction at an air-resin interface

Laser pulse refraction at an air-resin interface (MOV 17 kb)

Laser pulse racing in different media

Laser pulse racing in different media. (MOV 23 kb)

Apparent faster-than-light phenomenon

Apparent faster-than-light phenomenon. (MOV 107 kb)

Pulsed-laser-pumped fluorescence emission

Pulsed-laser-pumped fluorescence emission. (MOV 679 kb)

PowerPoint slides

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, L., Liang, J., Li, C. et al. Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature 516, 74–77 (2014). https://doi.org/10.1038/nature14005

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature14005

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing