Many important scientific questions in physics, chemistry and biology require effective methodologies to spectroscopically probe ultrafast intra- and inter-atomic/molecular dynamics. However, current methods that extend into the femtosecond regime are capable of only point measurements or single-snapshot visualizations and thus lack the capability to perform ultrafast spectroscopic videography of dynamic single events. Here we present a laser-probe-based method that enables two-dimensional videography at ultrafast timescales (femtosecond and shorter) of single, non-repetitive events. The method is based on superimposing a structural code onto the illumination to encrypt a single event, which is then deciphered in a post-processing step. This coding strategy enables laser probing with arbitrary wavelengths/bandwidths to collect signals with indiscriminate spectral information, thus allowing for ultrafast videography with full spectroscopic capability. To demonstrate the high temporal resolution of our method, we present videography of light propagation with record high 200 femtosecond temporal resolution. The method is widely applicable for studying a multitude of dynamical processes in physics, chemistry and biology over a wide range of time scales. Because the minimum frame separation (temporal resolution) is dictated by only the laser pulse duration, attosecond-laser technology may further increase video rates by several orders of magnitude.
The ability to spectroscopically probe ultrafast events has been of vital importance for understanding fundamental scientific questions in biology, chemistry and physics1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11. Examples of such investigations include charge–charge interactions dynamics12, transition-state dynamics of chemical reactions7 and the photosynthetic process13. Traditionally, such phenomena have been studied with pump/probe arrangements, in which the time delay between the pump and probe are scanned sequentially; however, such methods are limited to statistical studies under reproducible conditions. Understanding ultrafast dynamic events under more arbitrary conditions, however, requires methods that can acquire a sequence of temporally and spatially resolved data from a single ultrafast event. Apart from the requirement of extremely high temporal resolution, such methods should ideally also be able to pump/probe and analyze light of arbitrary wavelengths. Although current state-of-the-art ultrafast video-sequence two-dimensional (2D) imaging methods are capable of capturing single events with frame rates up to 1 THz14, they lack the ability to illuminate samples at arbitrary wavelengths and/or to detect spectrally complex signals, thus preventing these methods from being compatible with most spectroscopic approaches to extract species-specific information15 or quantum state dynamics2.
Previously described methods for achieving rapid data acquisition have relied on either detectors with extremely high temporal resolution or schemes based on ultra-short laser probes. For example, recent advances in tomographic reconstruction16 and transillumination shadowgraphic imaging14 use the relatively large spectral bandwidth offered by femtosecond (fs) laser pulses for spatiotemporal distinction. However, although these probe-based methods14, 16 are able to achieve ultrafast video rates, their methodologies are based on spectrally dispersive detection, thus preventing the signal from having arbitrary spectral characteristics. Sequentially timed all-optical mapping photography (STAMP)14, which can provide video rates as low as 1 picosecond (ps), relies on probe pulses with distinctly separated wavelengths, thereby excluding unification with most spectroscopic techniques. The methodology presented by Li et al.16 also relies on a spectral separation detection scheme. However, that work was based on tomographic imaging, thus enabling three-dimensionally resolved videography.
An alternative videography method with potential for spectroscopic compatibility is the use of ultrafast detectors that temporally resolve signals. Such a detection scheme has been demonstrated by Gao et al.17, who have combined a streak camera18 and compressed ultrafast photography (CUP). In contrast to probe-based methods, CUP has significantly decreased (hardware-limited) temporal resolution of just above 30 ps17. In addition, streak cameras suffer from poor spatial resolution and quantum efficiency—crucial aspects for compressive imaging to avoid image artifacts19. Hence, no previously described state-of-the-art methods for 2D videography of dynamic single events are able to unify femtosecond temporal resolution with spectroscopic compatibility.
In this article, we present a laser probe-based method that unifies the aforementioned aspects and is capable of producing a video sequence of non-repetitive dynamic events at femtosecond timescales and beyond for spectroscopy. Our method, called Frequency Recognition Algorithm for Multiple Exposures (FRAME), allows acquisition of a series of laser-induced images at frame rates that are limited only by the laser pulse duration. Here we used femtosecond laser pulses to demonstrate videography with record high sub-ps (THz) temporal resolution, although attosecond laser pulses could, in principle, increase the frame rate into the PHz regime. The novelty of the FRAME concept lies in superimposing a structural code onto the illumination to encrypt a single event, which is then deciphered in the data post processing. Because each image in the video sequence is extracted by using a unique spatial code, the method does not rely on a specific optical wavelength or laser bandwidth, and hence can be used for spectroscopic measurements. Here we provide a proof-of-principle demonstration of our method by performing imaging light-in-flight—often regarded as the gold standard experiment for ultrafast imaging14, 16, 17, 20 at 5 THz, a timescale on which even light appears stationary on a macroscopic scale.
To further demonstrate the wide applicability and versatility of FRAME, we present spectroscopic videography measurements based on laser-induced fluorescence for resolving fluid dynamics in both liquid- and gas phase, thus illustrating how FRAME can capture motions occurring on vastly different timescales. Our findings demonstrate the potential to perform a wide range of scientific investigations of transient, non-repetitive events by expanding the dimensions of spatiotemporal information. Such capabilities enable (i) new measurement concepts (such as coherence lifetime imaging), (ii) visualization of dynamic processes (such as optical filamentation, plasma formation and photoinduced chemistry in heterogeneous organic compounds), (iii) technical possibilities to circumvent the complications of mechanical vibrations in ultrafast measurements21 and (iv) increased statistical sampling provided by cameras rather than single-point detectors.
Materials and methods
The FRAME methodology is graphically illustrated in Figure 1. The methodology is based on the simple premise (which is analogous to several methods used for digital image compression) that the power spectrum of an image is strongly shifted toward low spatial frequencies22. Figure 1a shows a uniformly illuminated sample together with its Fourier transform. Illuminating the same target with a sinusoidal modulation with a spatial frequency of effectively places a pair of ‘image copies’ of its object structures at in the Fourier domain (Figure 1b)—an approach often used in microscopy to improve depth resolution23 as well as for super-resolution imaging24. If instead the spatial frequency and/or the orientation of the modulation are altered for a number of illuminations, each respective ‘image copy’ may be strategically placed at other regions in reciprocal space (Figure 1c and 1d). After data acquisition, each frame (image copy) stored on-chip is accessed individually through a frequency-sensitive 2D spatial lock-in algorithm. In this process, each individual frame is first isolated in reciprocal space by means of a 2D frequency band-pass filter, thus effectively removing all other image copies. The dimensions of this band-pass filter set the final resolution of the extracted frames. The isolated data are then digitally transferred to the origin in the Fourier domain, thus transforming the modulation amplitude into a dc component. Application of an inverse Fourier transform on this filtered and rearranged data set transforms spatial frequencies into intensity values, thus revealing the image information that was stored at an offset location in reciprocal space. More details on the image analysis can be found in Supplementary Information.
From a practical perspective, the FRAME approach allows a single camera to accept a multiple number of 2D signals in one exposure. Fundamental limitations related to high electronic readout speeds that commonly restrict videography are thus circumvented, and consequently low-noise and high-sensitivity imaging sensors with a large dynamic range and high pixel resolution can be used. These features are important because all image copies may need to share pixels and thus also share the full well capacity (dynamic range) of the camera. For example, eight image copies whose spatial structures overlap would each have a 13-bit resolution if captured with a 16-bit sensor.
FRAME is adaptable for several optical configurations, such as for pump-probe-, wave mixing-, transmission- and back/side-scattering measurements. To demonstrate this versatility, we conducted four experiments using FRAME. The results demonstrating the spectroscopic capability of FRAME are shown in the insets in Figure 2. These results display planar laser-induced fluorescence videography as well as instantaneous volumetric measurements in liquid and gas. The gas dynamics displays the formaldehyde distribution in a turbulent flame, whereas the movement of a droplet, marked with dye, is captured in a liquid. These experiments were conducted with a side-scattering probing geometry, and experimental details from these measurements are further described in the Supplementary Information.
Femtosecond videography experiment
Our main proof-of-principle demonstration of FRAME focused on the extreme temporal resolution offered by this approach, which is higher than the resolution of previously reported methods for 2D videography. To demonstrate femtosecond videography, we performed a pump-probe measurement to monitor the propagation of a femtosecond laser pulse through a Kerr-sensitive medium (CS2) by recording with a pulse-to-pulse separation between the read (probe) pulses—defined here as temporal resolution—corresponding to a frame rate of up to 5 THz. In the experiment (lower panel in Figure 2), a 125 fs long Ti:Sapphire laser pulse at 800 nm, the pump pulse, was directed through the Kerr medium. A pulse train of four read pulses arrived at the Kerr medium with a direction of propagation orthogonal to the pump pulse. The Kerr gate uses two polarizers, rotated 90° relative to each other, thus preventing the read pulses from reaching the detector. However, when the arrival time for the pump pulse and a read pulse coincide within the Kerr medium, the birefringence caused by the pump pulse allows the overlapping part of the read pulse to be transmitted. The Kerr gate is, however, not binary; instead, its gate function is governed by the intensity profile of the pump beam and is thus superimposed onto the transmitted light. This effect was exploited here to visualize the pump beam as it propagates through the Kerr medium. The intensity profile of each read pulse was encoded with a unique spatial modulation structure, created by imaging the ±1 orders of diffraction of a Ronchi grating (20 lp mm−1) into the measurement volume. After exiting the Kerr medium, the transmitted signals were imaged onto a single camera chip (Andor Luca, Andor Technologies, Belfast, UK, 1002 × 1004 pixels).
The burst of read pulses was created by splitting a second output from the Ti:Sapphire laser by means of a beam-splitter arrangement. In this proof-of-concept experiment, the time of arrival for each read pulse was controlled via separate optical delay lines; however, both these and the encoding optics could, in principle, be replaced by a single photorefractive volume reflection hologram25. The spatial modulation can be formed by either projecting or imaging the grating into the probe volume. The latter approach (used in the current setup) is beneficial because potential issues caused by color dispersion (diffraction) of the read pulses are avoided. Experimental details are further presented in Supplementary Information.
Results and discussion
In Figure 3a, we present a conceptual example of a raw data video sequence that we recorded using the FRAME arrangement operated at relatively low frame rates of 150 GHz (for illustrative purposes). Despite being spatially overlapped, the FRAME algorithm is able to separate each read pulse individually (Figure 3b), thereby revealing structural details in the intensity profile of the pump beam (Figure 3c). The orientation relaxation of CS2 molecules is, however, expected to distort the image of the pump-beam profile; the loss of clarity of the diffraction pattern from the front aperture (Figure 3c), manifested as a ring along the rim of the pump-beam profile, may be an indication of this effect. Notably, the relaxation time of the Kerr effect cannot be separated from the motion of the pump pulse in the current data. We constructed a 3D model of the measurement to verify that this observation was caused by the relaxation of the Kerr effect; our experimental data were in good agreement with the model predictions (Figure 3c). Figure 3d displays a spatial representation of where (in space) the pump and read pulses interact (geometrical calculation based on a constant speed of light). Further illustrations of how the detected signal is created as the pulses interact in the probe volume are presented in the Supplementary Information.
The operation of FRAME does not inherently generate any beam dispersion and thus preserves the entire bandwidth of the laser. This characteristic allows FRAME to take full advantage of the short laser pulse duration to maximize frame rates and minimize exposure (interaction) times. Under conditions of no temporal overlap between individual frames, the pulse duration dictates the frame rate, and the maximum is achieved when the read pulses are arranged ‘back-to-back’. In the current study, the read pulse duration was ~125 fs, yielding a maximum frame rate of ~8 THz, which is best compared to the ~33 GHz frame rate achievable with CUP18 (despite the conceptual differences between the two approaches). However, the temporal resolution of FRAME, in principle, has no upper limit and higher frame rates could potentially be realized by overlapping individual frames in time; however, this approach would not necessarily allow faster events to be traced in practice, because the images in the sequence would not be temporally resolved. In addition, movements or structural changes occurring on a macroscopic level start to become negligible when frame rates of ~10 THz are approached. For example, in CS2, light travels ~23 μm in 125 fs26, corresponding to a distance of ~3 pixels on the detector chip in the setup used in this current experiment. Hence, although it would practically be possible to observe the propagation of the pump beam at even higher frame rates, the beam would appear stationary with the current spatial resolution. However, spatial resolution could be improved with high-resolution magnifying optics; for example, a microscopic FRAME arrangement could thus directly benefit from higher frame rates. In addition, some response times of quantized systems occur on femtosecond3, 4, 7 (or even attosecond1, 2) timescales, and tracking such dynamics would therefore require THz frame rates or higher.
Next, we demonstrated the temporal capabilities as well as the high image quality and low shot-to-shot fluctuations that can be realized with FRAME. Figure 4 shows three single-shot video sequences of the propagation of the pump pulse through the Kerr medium, which were captured using the FRAME setup at 1, 2.5 and 5 THz. Figure 4a–4c shows the unprocessed data as seen by the camera, and Figure 4d displays the Fourier transform of Figure 4a, wherein the object structures from each read pulse appear as isolated peaks symmetrically around the origin in the center. Most structural information resides near the origin, thus leaving the majority of the Fourier domain otherwise unexploited, with vacant space for even more image copies (Figure 1). An example of an image encoded with 16 frames is presented in Supplementary Fig. S10. Application of the frequency-sensitive algorithm extracts results from each read pulse (Figure 4f–4h). To the best of our knowledge, one-time events have not previously been tracked with such high temporal resolution (200 fs).
Finally, we demonstrated the ability to perform videography of a moving target undergoing transformation by using FRAME. Because both the pump- and read pulses propagated as they interacted, each separate frame did not, in fact, constitute a truly instantaneous representation. This ‘spatiotemporal effect’, which is somewhat analogous to motion blur, is not a direct concern for the data presented in Figures 3 and 4, because the beam profile is constant and is conserved throughout the video sequence. However, when visualizing dynamic objects that evolve at or near luminous speed, this unavoidable effect may become apparent. Because FRAME is capable of acquiring several images of a single event, interpolation schemes can be used to estimate the intermediate states of the complex dynamics of such objects. To demonstrate this capability, we used 3D interpolation to reconstruct the wave front of a diverging laser pump pulse as it propagated in the Kerr medium (Figure 5). Figure 5a shows a schematic of the path of the diverging pump beam through the measurement volume, and the spatially distributed data (similar to Figure 3d) are displayed from two different angles in Figure 5b and 5c. The outcome of the interpolation analysis, shown in Figure 5d, revealed the derived time evolution of the wave front from a single laser pulse while traversing the Kerr medium (in a plane perpendicular to the direction of propagation).
The FRAME system (Figures 3–5) used here had a final spatial resolution of ~15 lp mm−1 (over a field-of-view of 7 × 7 mm2). To improve a FRAME system, that is, by increasing the number of video frames and/or improving the spatial resolution of each frame, the Fourier domain of the detection system first must be expanded. Expanding the Fourier domain can be achieved by (i) increasing the number of pixels of the imaging sensor and (ii) improving the optical resolution accordingly. Such an optimized detection system would allow a larger set of frames, carried by low- and high-modulation frequencies, to be strategically positioned in reciprocal space, as exemplified in Figure 1d. Experimentally, this involves (i) a tailored set of gratings, each with a unique spatial frequency and orientation, (ii) an imaging sensor with a large number of pixels and (iii) a high-resolution optical system. An example of such a system is provided in the Supplementary Information. Notably, if spatial resolution is critical, then the frames should be maximally dispersed in reciprocal space to allow the use of large filter functions in the spatial lock-in algorithm.
To our knowledge, the reported methodology, FRAME, is the first to enable ultrafast 2D videography having spectroscopic compatibility with both high spatial and temporal resolution, down to at least femtosecond timescales. In contrast to existing image coding techniques18, 19 that rely on translation/sweeping solutions, FRAME is entirely light-based; a boost in frame rate by more than two orders of magnitude is demonstrated herein. Because FRAME is restricted to neither a specific wavelength nor any particular optical configuration, it is compatible with most light–matter interaction measurements using standard laboratory equipment, such as those based on absorption, scattering, fluorescence, polarization, wave mixing and coherence. Utilizing imaging sensors with large number of pixels in conjunction with high-resolution imaging optics would increase the image-storing capacity to allow a larger number of frames in the image sequence. Further, FRAME, together with the current progress within ultrafast science toward producing laser pulses in the attosecond regime27, could, in principle, offer video rates up to a thousand times faster than those presented herein.
The simple yet unique ability of FRAME to acquire several laser images in one recording also enables new measurement opportunities. For example, combining FRAME with currently existing laser methods for fluorescence lifetime imaging28 (FLI) of single events29 would allow all FLI data to be collected in one image recording. Finally, the ultimate purpose of single-event videography is to use the existing knowledge of ultrafast dynamics gained from studies of reduced systems to understand evolving global systems; we believe FRAME can facilitate achievement of this endeavor.
AE, EK, JB and MA designed the research. AE set up the experiment, and EK and AE performed the data collection. EB provided experimental insights and EK performed the data analysis. ZL and EK performed the flame measurements. EK and AE wrote the paper, with extensive suggestions from all other co-authors.
Schultze M, Fieβ M, Karpowicz N, Gagnon J, Korbman M et al. Delay in photoemission. Science 2010; 328: 1658–1662.
Beck AR, Neumark DM, Leone SR . Probing ultrafast dynamics with attosecond transient absorption. Chem Phys Lett 2015; 624: 119–130.
Zewail AH . Femtochemistry: atomic-scale dynamics of the chemical bond. J Phys Chem A 2000; 104: 5660–5694.
Sundström V . Femtobiology. Annu Rev Phys Chem 2008; 59: 53–77.
Berera R, van Grondelle R, Kennis JTM . Ultrafast transient absorption spectroscopy: principles and application to photosynthetic systems. Photosynth Res 2009; 101: 105–118.
Barty A, Boutet S, Bogan MJ, Hau-Riege S, Marchesini S et al. Ultrafast single-shot diffraction imaging of nanoscale dynamics. Nat Photonics 2008; 2: 415–419.
Motzkus M, Pedersen S, Zewail AH . Femtosecond real-time probing of reactions. 19. Nonlinear (DFWM) techniques for probing transition states of uni- and bimolecular reactions. J Phys Chem 1996; 100: 5620–5633.
Vos MH, Martin JL . Femtosecond processes in proteins. Biochim Biophys Acta 1999; 1411: 1–20.
Schwarts BJ, King JC, Zhang JZ, Harris CB . Direct femtosecond measurements of single collision dominated geminate recombination times of small molecules in liquids. Chem Phys Lett 1993; 203: 503–508.
Dantus M . Coherent nonlinear spectroscopy: from femtosecond dynamics to control. Annu Rev Phys Chem 2001; 52: 639–679.
Gallmann L, Cirelli C, Keller U . Attosecond science: recent highlights and future trends. Annu Rev Phys Chem 2012; 63: 447–469.
Huber R, Tauser F, Brodschelm A, Bichler M, Abstreiter G et al. How many-particle interactions develop after ultrafast excitation of an electron-hole plasma. Nature 2001; 414: 286–289.
Sundtröm V, Pullerits T, von Grondelle R . Photosynthetic light-harvesting: reconciling dynamics and structure of purple bacterial LH2 reveals function of photosynthetic unit. J Phys Chem B 1999; 103: 2327–2346.
Nakagawa K, Iwasaki A, Oishi Y, Horisaki R, Tsukamoto A et al. Sequentially timed all-optical mapping photography (STAMP). Nat Photonics 2014; 8: 695–700.
Mizutani Y, Kitagawa T . Direct observation of cooling of Heme upon photodissociation of carbonmonoxy myoglobin. Science 1997; 278: 443–446.
Li ZY, Zgadzaj R, Wang XM, Chang YY, Downer MC . Single-shot tomographic movies of evolving light-velocity objects. Nat Commun 2014; 5: 3085.
Gao L, Liang JY, Li CY, Wang LV . Single-shot compressed ultrafast photography at one hundred billion frames per second. Nature 2014; 516: 74–77.
Gord JR, Meyer TR, Roy S . Applications of ultrafast lasers for optical measurements in combusting flows. Annu Rev Anal Chem 2008; 1: 663–687.
Llull P, Liao XJ, Yuan X, Yang JB, Kittle D et al. Coded aperture compressive temporal imaging. Opt Express 2013; 21: 10526–10545.
Abrahamson N . Light-in-flight recording by holography. Opt Lett 1978; 3: 121–123.
Kim J, Kärtner FX . Attosecond-precision ultrafast photonics. Laser Photonics Rev 2010; 4: 432–456.
Gersho A, Gray R . Vector Quantization and Signal Compression. New York: Kluwer Academic; 1992.
Neil MAA, Juškaitis R, Wilson T . Method of obtaining optical sectioning by using structured light in a conventional microscope. Opt Lett 1997; 22: 1905–1907.
Gustafsson MGL . Nonlinear structured-illumination microscopy: wide-field fluorescence imaging with theoretically unlimited resolution. Proc Natl Acad Sci USA 2005; 102: 13081–13086.
Hill KB, Purchase KG, Brady DJ . Pulsed-image generation and detection. Opt Lett 1995; 20: 1201–1203.
Kedenburg S, Vieweg M, Gissibl T, Giessen H . Linear refractive index and absorption measurements of nonlinear optical liquids in the visible and near-infrared spectral region. Opt Mater Express 2012; 2: 1588–1611.
Feng XM, Gilbertson S, Mashiko H, Wang H, Khan SD et al. Generation of isolated attosecond pulses with 20 to 28 femtosecond lasers. Phys Rev Lett 2009; 103: 183901.
Suhling K, Hirvonen LM, Levitt JA, Chung PH, Tregidgo C et al. Fluorescence lifetime imaging. In: Ho AHP, Kim D, Somekh MG, editors. Handbook of Photonics for Biomedical Engineering. Dordrecht, The Netherlands: Springer; 2015, pp1–50.
Ehn A, Johansson O, Arvidsson A, Aldén M, Bood J . Single-laser shot fluorescence lifetime imaging on the nanosecond timescale using a Dual Image and Modeling Evaluation algorithm. Opt Express 2012; 20: 3043–3056.
We are grateful to the Swedish Energy Agency, the Knut and Alice Wallenberg Foundation and the Swedish Research Council (121892) for financial support. Funding from the European Research Council (ERC) through both the Advanced Grant ‘TUCLA’ and the Starting Grant ‘Spray-Imaging’ projects is also highly appreciated.
The authors declare no conflict of interest.
Note: Supplementary Information for this article can be found on the Light: Science & Applications’ website.
About this article
Cite this article
Ehn, A., Bood, J., Li, Z. et al. FRAME: femtosecond videography for atomic and molecular dynamics. Light Sci Appl 6, e17045 (2017). https://doi.org/10.1038/lsa.2017.45
- imaging techniques
- ultrafast photonics
- ultrafast spectroscopy
Single-shot imaging of microscopic dynamic scenes at 5 THz frame rates by time and spatial frequency multiplexing
Optics Express (2020)
Physical Review Letters (2019)
International Journal of Extreme Manufacturing (2019)
OSA Continuum (2019)
Optics Letters (2019)