Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Spectro-temporal encoded multiphoton microscopy and fluorescence lifetime imaging at kilohertz frame-rates

Abstract

Two-Photon Microscopy has become an invaluable tool for biological and medical research, providing high sensitivity, molecular specificity, inherent three-dimensional sub-cellular resolution and deep tissue penetration. In terms of imaging speeds, however, mechanical scanners still limit the acquisition rates to typically 10–100 frames per second. Here we present a high-speed non-linear microscope achieving kilohertz frame rates by employing pulse-modulated, rapidly wavelength-swept lasers and inertia-free beam steering through angular dispersion. In combination with a high bandwidth, single-photon sensitive detector, this enables recording of fluorescent lifetimes at speeds of 88 million pixels per second. We show high resolution, multi-modal - two-photon fluorescence and fluorescence lifetime (FLIM) – microscopy and imaging flow cytometry with a digitally reconfigurable laser, imaging system and data acquisition system. These high speeds should enable high-speed and high-throughput image-assisted cell sorting.

Introduction

The extension of regular one-photon fluorescence microscopy to non-linear two-photon microscopy (TPM)1 has led to important applications, e.g., in brain research, where the advantages of deeper tissue penetration and inherent three-dimensional sectioning capability enable recording neuronal activity to interrogate brain function in living mice2. Further, fluorescence lifetime imaging (FLIM)3 can probe internal biochemical interactions and external environment of molecules, e.g., for quantifying cellular energy metabolism in living cells4,5. The quadratic dependency on intensity of TPM favours laser-scanning over whole-field illumination, so typically raster-scanning with galvanometric mirrors is conducted. However, these mechanical mirrors are inertia limited to line-scan rates of <20 kHz. This limits the imaging frame rate, however faster frame-rates are desirable, e.g., for neuronal activity imaging at >1000 Hz frame-rates6,7,8. Therefore, new technologies were developed with faster acousto-optical scanners6,9, polygonial scanners10, parallelized multi-foci excitation11,12,13, optical scanning8 or sparse sampling7, just to name a few. For FLIM, recent developments also increased imaging speeds14 with techniques employing multifoci15 or widefield imaging16,17,18,19 or increased detection speed by analogue lifetime detection20,21,22 which permitted speeds up to video-rate23. Enhancing imaging speeds through spectral-encoded scanning has been successfully employed for confocal microscopy24, high-speed brightfield imaging25, quantitative phase imaging26 and more, especially by employing the time stretch technique25,26,27. However, these fast imaging approaches could not be used for fluorescence imaging, as both the emission spectrum and the fluorescent lifetime are independent on the excitation wavelength (Kasha’s rule), so the original spectral encoding is lost upon the fluorescence emission. We present a solution to this problem by further employing temporal encoding from a wavelength swept laser. This concept achieves spectro-temporal encoded imaging, where the wavelength is used for high-speed inertia-free point scanning and the temporal encoding for one-to-one mapping of the signal to the imaging pixels. This temporal encoding is just like in conventional raster-scanning or laser scanning microscopy. Spectro-temporal encoded imaging has unique advantages over other high-speed non-linear imaging approaches6,7,8,9,11,12,13 in terms of resolution, lifetime modality, compactness, flexibility, and fibre-based setup.

Here we report on a high-speed laser scanning technique for non-linear imaging using a rapidly wavelength swept laser in combination with a diffraction grating to achieve inertia-free, very rapid beam-scanning, orders of magnitude faster than mechanical scanners. We employ a high-speed swept source Fourier-domain Mode-locked (FDML) laser28 which is modulated to short pulses and amplified to high-peak powers. This FDML-MOPA laser was previously described in detail29. The swept wavelength output is sent onto a diffraction grating for line scanning (Fig. 1). Each pulse illuminates a distinct pixel both in time and in space (spectro-temporal encoding). The y-axis is scanned with a galvanometric mirror at typically 1 kHz speed (slow axis). At an FDML sweep rate of 342 kHz this achieves a frame rate of 2 kHz for a 256 × 170 pixel frame size and 88 MHz pulse repetition rate. A high-bandwidth detection at single-photon sensitivity enables recording of second harmonic generation (SHG), Two-Photon fluorescence and fluorescent lifetime imaging (FLIM) at speeds up to the excitation rate of 88 million pixels per second. This fast two-photon microscope, which we coin spectro-temporal laser imaging by diffracted excitation (SLIDE) microscope, can be digitally programmed to allow for adapting the pulse repetition rate to the fluorescent lifetimes. This is achieved through the direct pulse modulation through a 20 GHz bandwidth electro-optic modulator (EOM) (cf. Fig. 1 and our previous report29).

Results

Time bandwidth product in spectro-temporal laser imaging by diffractive excitation (SLIDE)

Spectro-temporal encoding with high-information density places rigorous requirements on the spectro-temporal bandwidth of the light source37. The wavelength sweep time ∆T is equal to the number of pixels n times the time between pulses ∆ti, governed by the information-interrogation induced latency (here: fluorescent decay time τi). Considering, for example, 256 horizontal (linescan) pixels and a typical total fluorescent decay time of 10 ns, this time calculates to ∆T = 2.56 µs. Assuming a spectral resolution of ∆λi = 100 pm for the diffractive mapping means that the light source needs to sweep over ∆λ = 25.6 nm in ∆T = 2.56 µs. A unique feature in SLIDE is that the spectro-temporal bandwidth scales quadratically with the number of pixels (in linescan):

$${\mathrm{Spectro}} - {\mathrm{temporal}}\,{\mathrm{bandwidth}}\,{\mathrm{M}}_{{\mathrm{ST}}} = \Delta {\mathrm{T}}\, \times \, \Delta \lambda = {\mathrm{n}}^{\mathrm{2}}\, \times\, \Delta \lambda _{\mathrm{i}}\, \times\, \Delta {\mathrm{t}}_{\mathrm{i}}$$
(1)

A wavelength tuning speed of tens of nm over few microseconds is beyond the reach of conventional tuneable lasers28,38. Although very fast tuning speeds can be achieved by chirping a supercontinuum pulse source in a dispersive medium as employed in time stretch techniques, achieving a time span of 2.56 µs is about three orders of magnitude beyond the reach of available dispersive elements (typically in the ns-regime). Furthermore, the spreading of energy due to the stretching would result in negligible peak powers and would prevent non-linear excitation.

Spectro-temporal stretch via an FDML laser solves this predicament37. The FDML laser provides a combination of large spectral span39, along with microseconds time span and narrow instantaneous linewidth. This type of laser has mainly been used for fast optical coherence tomography (OCT)28,38,40, semiconductor compressed pulse generation41 and non-linear stimulated Raman microscopy42 and recently inertia-free LiDAR37. Its low instantaneous linewidth allows us to achieve a high spatial resolution below 1 µm (see Supplementary Fig. 6), a feat that is not possible with chirped supercontinuum sources.

SLIDE: Experimental setup

The experimental setup of the SLIDE microscope is presented in Fig. 2. The whole system is electronically synchronized by an arbitrary waveform generator (AWG). It drives the FDML-MOPA laser including the pulse pattern, the y-axis galvo mirror and also the trigger and clock of the digitizer card. The modulated pulse length was measured to be 65 ps (Fig. 2g). This pulse width corresponds to a time-bandwidth calculated linewidth of 25 pm, so only a factor of two lower than the measured linewidth29. Raster scanning on the sample is achieved through spectro-temporal encoding along the x-axis and mechanical scanning on the y-axis (cf. Fig. 1b). It shall be noted that the used spectral bandwidth of 12 nm (Fig. 2e) lies well within most Two-Photon absorption bandwidths, so after absorption and excited state relaxation (Stokes shift), the fluorescence emission is equivalent for all pixels along the whole line (Kasha’s rule). The x-axis imaging resolution is given by the spectral resolution of the grating (67 pm, see methods) and the instantaneous laser linewidth (56 pm, Fig. 2g). In combination with the swept bandwidth of 12 nm, this should allow for 12 nm/0.067 pm ≈ 180 discernible spots and thus a lateral resolution of 100 µm/180 = 556 nm for the 60x objective (see Methods). We moderately oversample the x-axis by programming 256 pulses along the x-direction. We tested the imaging resolution achieved through spectro-temporal encoding by imaging 100 nm fluorescent beads (Supplementary Fig. 6) which yielded a resolution of 596 nm in x-direction (spectral scanning direction) and 455 nm in y-direction for the 60× NA 1.4 objective, i.e., close to the calculated resolution and sufficient for high-resolution imaging of sub-cellular features. We further verified that the fluorescence signal scales quadratically with the illumination power confirming Two-Photon origin of the signal. The instrument response function (IRF) of the detection was determined by SHG from a Urea sample and fitted to 1026 ps (Fig. 2h). The validity of the analogue lifetime detection approach was already confirmed in a previous work30, where we compared the recorded lifetime value of Rhodamine 6 G to a literature value in good agreement. This analogue detection enables rapid FLIM microscopy at even only a single pulse per pixel illumination (SP-FLIM)21, as discussed above.

Imaging performance

SLIDE imaging performance was assessed first by recording the field of view (FOV) on a resolution target (Fig. 3b). The x-axis was scanned with the spectro-temporal encoded scan while the y-axis was scanned with a galvanometric mirror (Fig. 3a). Figure 3c shows a two-photon microscopy image of a euglena gracilis microalgae and was recorded within 497 µs at 2 kHz frame rate. The time trace (Fig. 3d) of a single line illustrates the high SNR of up to 490 (peak SNR). This is due to the high photon counts achieved per excitation pulse (note that a single photon generates a voltage of 50 mV). By zooming in on two individual pixels, we see a difference in the transient fluorescence decay time (Fig. 3f). To this end, 3 × 3 spatial binning was performed to yield enough fluorescent photons for a mono-exponential fit43. By fitting with the convoluted IRF a FLIM image can be generated (Fig. 3e) revealing two different domains within the algae cells. The autofluorescent chloroplasts decay fast and are colour-coded red, while Nile Red, an exogenous fluorophore which was added to highlight lipid generation within the microalgae, has a longer decay time and is colour-coded green. Both imaging modalities TPM and FLIM are extracted from the same data which was acquired within 497 µs for this 256 × 170 pixel image. Further imaging examples can be found in the supplementary material (Supplementary Figs. 1, 2, 46).

SLIDE imaging flow cytometry

As a biological sample we imaged Euglena gracilis cells in SLIDE imaging flow cytometry mode, where autofluorescence from chloroplasts revealed a short fluorescence lifetime and Nile Red stained lipids show a longer lifetime (see also Fig. 3). Figure 5 presents four different snapshots of Euglena gracilis microalgae in flow where each image pair (Two-Photon & Two-Photon FLIM images) was acquired within 1 ms. The TPM images already show the high resolution which reveals sub-cellular morphology. Yet, through the FLIM images it is possible to discern chloroplasts from lipids through their fluorescent lifetime. The images have very high resolution and high SNR and, even though they were obtain at very high speeds in flow, they have quality comparable to regular one-photon fluorescence microscopy44. Excitation power was 30 mW on the sample and 5 × 5 pixel spatial binning was applied to ensure >100 photons per lifetime curve. Fluorescence lifetime can aid in discerning lipids as it is independent of concentration and signal height. This high-speed lipid content screening by SLIDE FLIM cytometry may help in purification of high lipid content microalgae for efficient biofuel production44.

Discussion

It shall be noted that the high speeds achieved by SLIDE require bright samples in order to detect many fluorescence photons per excitation pulse. In fact, the high speed of 88 MHz pixel rate was achieved for two-photon fluorescence imaging, yet for FLIM imaging pixel binning was applied to achieve the necessary photon numbers of >100 photons per pixel43. This blurs the lifetime (colour) resolution, without compromising the morphological resolution, which originates from the fluorescence intensity images. However, not all samples provide sufficient fluorophores in the focal volume, such as genetically encoded fluorescent proteins. We tested imaging of genetically encoded tdTomato fluorescent proteins in ex vivo mouse brain to test whether a kilohertz frame-rate could be achieved in imaging neuronal activity. However, even though we were able to obtain morphological images at 1 kHz rate (see Supplementary Figs. 1, 2, 4), we found that only single fluorescent proteins were present in the focal volume and the signal was saturated, as increasing the laser power did not yield a quadratic signal increase. This does not permit recording neuronal ensemble activity at the kHz speed of the SLIDE systems, as both fluorescence intensity changes or fluorescence lifetime changes require tens to hundreds of photons thus sacrificing the speed as a result of averaging. Therefore, biochemically engineered fast and bright fluorescent proteins are needed, which can also be expressed at high abundance without interfering with cellular behaviour, in order to enable kHz frame-rate imaging of neuronal activity for SLIDE or any other fast technique in the future.

It is noted that SLIDE microscopy employs higher average powers (~10–100 mW) compared to conventional two-photon microscopes (1–30 mW). However, we did not notice any sample damage. The higher powers are due to the fast acquisition rates obtained by SLIDE together with the relatively longer pulse length. It shall be noted that the two-photon excitation rate is given by ppeak2*tpulse30,45, so even with longer pulses similar TPM signal levels can be achieved if the duty cycle of the pulses is the same30,45. In SLIDE, the repetition rate is typically 88 MHz and thus duty cycles are normally higher than in femtosecond TPM systems (i.e. lower peak powers in SLIDE). In principle, however, same peak powers and thus same excitation rates can easily be achieved with SLIDE by simply digitally programming the required pulse repetition rate. This powerful pulse on demand feature of SLIDE can be used to trade speed for sensitivity when imaging dim fluorophores, i.e., with low two-photon absorption cross sections.

Interestingly, the excitation rates can also be scaled up by employing shorter pulses. Currently, the electronic pulse generation of 65 ps is the practical limiting factor, as the EOM bandwidth permits much shorter pulses. Furthermore, dispersion-based compression could be harnessed to achieve shorter pulses in the single-digit picosecond range41,46. To this end, a large time-bandwidth product is required, which however is achievable with FDML lasers37.

On the other hand, SLIDE profits from the longer pulses, since to increase photon numbers per single pulse it is advantageous to increase the pulse length rather than the peak power in order to avoid supra-quadratic scaling effects like photobleaching32,33 and photodamage35,45. This advantage of longer pulses was already reported before30,35,45,47 and was also discussed above. Another factor which reduces photodamage is the longer wavelengths of 1060 nm employed here48. Lastly, even though pulse energies become larger with longer pulses, photothermal effects on the sample are negligible in SLIDE microscopy as they are insignificantly small49 and dissipate in between frame scans.

The fast frame-scans also lead to a low effective repetition rate per pixel of only 1 kHz (i.e. half the bi-directional frame-rate) in SLIDE. These low repetition rates per pixel were reported to lead to advantageous relaxation of even long lived triplet states32, which can significantly increase photon numbers and thus further helps to speed up FLIM imaging rates.

The SLIDE system presented here has an excitation wavelength around 1060 nm, however future laser developments will target excitation wavelengths at 780 nm for autofluorescence applications4,5 and 940 nm for GCaMP-based imaging13.

Regarding sensitivity, Figs. 3 and 4 also show that SLIDE has sensitivity down to the single molecule level, so even samples where only a single fluorescent emitter is excited in the focal volume can be successfully imaged with SLIDE. In this sparse signal case, required photon counts can be reached either by spatial binning or, in applications where high spatial resolution FLIM is required5, by phase-locked detection and averaging. In SLIDE, frame averaging instead of pixel averaging can be performed, leading to low effective pulse repetition rate per pixel which can help increase signal levels6 (see above). The analogue detection for FLIM is furthermore compatible with existing two photon microscopes and can significantly increase FLIM imaging speeds19,20,21,23.

In conclusion, in this manuscript we presented the concept of high-speed SLIDE microscopy along with the experimental implementation and application in Two-Photon imaging flow cytometry with two imaging modalities, TPM and FLIM. High speeds of 342 kHz line scanning rates and 88 MHz pixel rates were presented. We presented high quality fluorescence lifetime imaging flow cytometry at very high speeds (>10,000 events per second). We believe that this high throughput and multi-modality enabled by SLIDE microscopy can lead to new insights into rapid biological processes and detection of rare events in applications like liquid biopsy or rare circulating tumour cell detection50. Further, the fiber-based setup of the SLIDE microscopy system may enable an endoscopic application to overcome the relatively shallow penetration depth of optical microscopy.

Methods

Wavelength-to-space mapping

Upon exiting the single-mode fibre, the light was collimated using an f = 37 mm lens (Thorlabs collimator F810APC-1064) followed by a beam-expander (f = 100 mm and f = 150 mm; Thorlabs LA1509-C and LA1433-C). This results in a beam diameter of 11.5 mm filling the 60× microscope objective aperture. The grating (Thorlabs GR25-1210) was positioned at a 30° angle (close to blaze angle), such that the first order was reflected at almost the incident direction in order to minimize ellipticity of the first-order diffraction beam. At 1200 lines/mm the grating only produced a 0 and +1 diffraction order and the first order power was maximized by adjusting the polarization on a polarization control paddle. The blazed grating ensured >80% power in the first order. The grating resolution is calculated to 67 pm. This fits well to the instantaneous linewidth of the FDML, which was measured for a single pulse to be 56 pm (cf. Fig. 2).

Microscopy setup

Two lenses were used to relay image the beams onto a galvanometric mirror for y-axis scanning. The galvo mirror (Cambridge Technology CT6215H) was driven synchronously, producing 170 lines at 2.012 kHz. A high NA, oil immersion microscope objective was used (Nikon Plan Apo 60× NA 1.4 oil) or a 40× water immersion objective (Nikon N40×-NIR - 40× Nikon CFI APO NIR Objective, 0.80 NA, 3.5 mm WD). The field-of-view (FOV) was determined by inserting a resolution target and recording the reflected excitation light on a CCD camera installed in the microscope, which was sensitive to the 1064 nm excitation light (cf. Fig. 3). A dichroic mirror (Thorlabs DMSP950R) in combination with an additional short-pass optical filter (Semrock FF01-750) transmits the Epi-generated signals to a hybrid photodetector (HPD, Hamamatsu R10467U-40) with high quantum efficiency (45%). The high time resolution of the HPD in combination with a fast digitizer (up to 4GS/s) leads to a fast IRF of only 1026 ps, measured by detecting the instantaneous signal of SHG in urea crystals (cf. Fig. 2).

Digitally synthesized waveforms

The whole system is driven by an arbitrary waveform generator (AWG, Tektronix AWG7052). This AWG provides all digitally synthesized driving waveforms, driving the FDML laser (Fabry-Pérot Filter waveform and 50% modulation of SOA for buffering), the galvo-mirror and also generates an external sample clock signal for the digitizer. The waveforms are digitally programmed and enable flexibility on the number of pulses per sweep, pulse width, pulse shape, pulse pattern and thus also the image sampling density.

Digitizer

As digitizers, either an oscilloscope (Tektronix DPO71604B) at 3.125 GSamples/s or a streaming ADC card (Innovative Integrations Andale X6GSPS and Alazartech ATS9373) with synchronously driven sample clock at 3196 MHz or 3940 MHz, respectively, were employed. The external sample clock was employed such that the data acquisition runs synchronously to the FDML laser and the pulse modulation and ensures sample-accurate fitting. In order to acquire large data sets, a streaming ADC in combination with a RAID-SSD array was employed to store the data.

Flow cytometry

In the flow cytometry recording, the flow-rate was set by two fundamental properties, namely the fluorescence lifetime and the imaging diffraction limit. The lifetime limits the repetition rate to ~100 MHz, while the diffraction limit is sampled at ~380 nm. Consequently, we employed 88 MHz repetition rate at 256 pulses per 2.92 µs linescan rate and 100 µm FOV. The flow rate was equally set to sample each line at 380 nm, i.e., 380 nm/2.92 µs = 0.13 m/s. The scale bars in the flow cytometry images were generated using the known 10 µm size of the Red-species bead to calibrate the actual flow speed. The Red bead was sampled with 18 lines, calculating to a line spacing of 556 nm. Using the line scan rate of 342 kHz, this calculates to a flow speed of ~0.2 m/s.

Data processing

For precise measurements, a deconvolution with the IRF was conducted in order to extract the fluorescent lifetimes (e.g. in Fig. 3 and Supplementary Fig. 4). However, this is time consuming, so for faster processing and qualitative results a tail-fitting algorithm was used. Often time, different species need to be discerned so a qualitative value is sufficient. The presence of multi-exponential behaviour was tested by checking for kinks in the slope of the logarithmic plot or checking for deviations between the mono-exponential fit and the measurement data (see e.g. Supplementary Fig. 4 where the mono-exponential fit yields a reliable fit). Photon numbers were calculated by dividing the area under the curve by the area of a single-photon event. For all images, the data was processed and images created in LabVIEW. The 2P-FLIM images were generated as HSL-images, where Hue was given by the lifetime-values, lightness by the integrated TPEF signal and saturation always set to maximum, i.e., the same for all images. For the TPEF images, the “Red Hot” or “Royal” look-up tables were applied in ImageJ. The plots were generated in GNUPlot and the figures produced using Inkscape.

Samples

The Pollen grain samples were ordered from Carolina (B690 slide). The fluorescent beads were ordered from ThermoFisher Scientific (# F8825, F8839, F8841, F8843, and F21012).

Experimental model systems: Euglena cell culture and fermentation

The E. gracilis cells used in the study are Euglena gracilis Z (NIES-48) strain procured from the Microbial Culture Collection at the National Institute for Environmental Studies (NIES), Japan. E. gracilis were cultured heterotrophically in 500 mL flasks using Koren-Hutner (KH) medium at a pH of 3.551. The cell cultures were maintained at 23 °C with a shaking rate of 120 strokes/min under continuous illumination of 100 μmol photons m−2 s−1. Cells were subjected to anaerobic fermentation to induce lipid accumulation. The fermentation was performed on cells in stationary phase by bubbling with nitrogen gas and incubating the flasks in the dark for three days.

Nile red staining of intracellular lipid droplets

The Nile red stock was prepared by dissolving original dye powder (Sigma) into 4 mL dimethyl sulfoxide (DMSO) to achieve a concentration of 15.9 mg/mL (50 mM). The stain stock solution was diluted 1,000 times with distilled water before use. The E. gracilis cells in the culture medium were washed with distilled water and resuspended in distilled water with a final concentration of 2 × 106 cells/mL. We mixed 15.9 μg/mL of nile red solution with E. gracilis cell suspension solution at a volume ratio of 1:1, which was followed by gentle vibration and incubation in the dark for 10 min. The final concentration of Nile red and E. gracilis cells were 7.95 μg/mL and 1 × 106 cells/mL, respectively. The E. gracilis cells were washed three times with distilled water and centrifugation (2000 × g, 1 min). The cells were resuspended in distilled water and protected from light prior to imaging.

Experimental model systems: mouse brain imaging

All experiments followed the U.S. National Institutes of Health guidelines for animal research, under an animal use protocol (ARC #2007-035) approved by the Chancellor’s Animal Research Committee and Office for Animal Research Oversight at the University of California, Los Angeles. Experiments in Supplementary materials used FVB.129P2 wild-type mice (JAX line 004828) injected with AAV1-pCAG-tdTomato (Addgene plasmid # 59462) into the right primary visual cortex. For the viral injections, mice were anaesthetized with isoflurane (5% induction and 1.5% maintenance) and placed on a stereotaxic surgery frame. Next, a small burr hole was drilled over the right primary visual cortex and ~50 nL of adeno associated virus (AAV) was injected 100–200 microns into the cortex. Following injections, the exposed skull was covered with dental cement. The mice were returned to their home cages and two weeks after viral injection, mice expressing TdTomato were perfused intracardially with 4% paraformaldehyde in phosphate buffer and their brains were extracted and glued on a petri dish, which was filled with PBS buffer for imaging on the inverted microscope. Neurons expressing TdTomato were imaged in Layer 2/3 at a depth of ~150–250 μm below the dura using a long working range 40× microscope objective (Nikon N40×-NIR - 40× Nikon CFI APO NIR Objective, 0.80 NA, 3.5 mm WD).

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

References

1. 1.

Denk, W., Strickler, J. & Webb, W. Two-photon laser scanning fluorescence microscopy. Science 248, 73–76 (1990).

2. 2.

Gong, Y. et al. High-speed recording of neural spikes in awake mice and flies with a fluorescent voltage sensor. Science 350, 1361–1366 (2015).

3. 3.

Lakowicz, J. R., Szmacinski, H., Nowaczyk, K., Berndt, K. W. & Johnson, M. Fluorescence lifetime imaging. Anal. Biochem. 202, 316–330 (1992).

4. 4.

Blacker, T. S. et al. Separating NADH and NADPH fluorescence in live cells and tissues using FLIM. Nat. Commun. 5, 3936 (2014).

5. 5.

Evers, M. et al. Enhanced quantification of metabolic activity for individual adipocytes by label-free FLIM. Sci. Rep. 8, 8757 (2018).

6. 6.

Chen, X., Leischner, U., Rochefort, N. L., Nelken, I. & Konnerth, A. Functional mapping of single spines in cortical neurons in vivo. Nature 475, 501–505 (2011).

7. 7.

Kazemipour, A. et al. Kilohertz frame-rate two-photon tomography. Nat. Methods 16, 778–786 (2019).

8. 8.

Wu, J. et al. Kilohertz in vivo imaging of neural activity. bioRxiv, 543058 (2019).

9. 9.

Lechleiter, J. D., Lin, D.-T. & Sieneart, I. Multi-photon laser scanning microscopy using an acoustic optical deflector. Biophysical J. 83, 2292–2299 (2002).

10. 10.

Kim, K. H., Buehler, C. & So, P. T. C. High-speed, two-photon scanning microscope. Appl. Opt. 38, 6004–6009 (1999).

11. 11.

Bewersdorf, J., Pick, R. & Hell, S. W. Multifocal multiphoton microscopy. Opt. Lett. 23, 655–657 (1998).

12. 12.

Kim, K. H. et al. Multifocal multiphoton microscopy based on multianode photomultiplier tubes. Opt. Express 15, 11658–11678 (2007).

13. 13.

Cheng, A., Goncalves, J. T., Golshani, P., Arisaka, K. & Portera-Cailliau, C. Simultaneous two-photon calcium imaging at different depths with spatiotemporal multiplexing. Nat. Methods 8, 139–142 (2011).

14. 14.

Liu, X. et al. Fast fluorescence lifetime imaging techniques: a review on challenge and development. J. Innovative Optical Health Sci. 12, 1930003 (2019).

15. 15.

Poland, S. P. et al. A high speed multifocal multiphoton fluorescence lifetime imaging microscope for live-cell FRET imaging. Biomed. Opt. Express 6, 277–296 (2015).

16. 16.

Agronskaia, A., Tertoolen, L. & Gerritsen, H. Fast fluorescence lifetime imaging of calcium in living cells. J. Biomed. Opt. 9, 1230–1237 (2004).

17. 17.

Krstajić, N. et al. 0.5 billion events per second time correlated single photon counting using CMOS SPAD arrays. Opt. Lett. 40, 4305–4308 (2015).

18. 18.

Raspe, M. et al. siFLIM: single-image frequency-domain FLIM provides fast and photon-efficient lifetime data. Nat. Methods 13, 501–504 (2016).

19. 19.

Bowman, A. J., Klopfer, B. B., Juffmann, T. & Kasevich, M. A. Electro-optic imaging enables efficient wide-field fluorescence lifetime microscopy. Nat. Commun. 10, 4561 (2019).

20. 20.

Giacomelli, M. G., Sheikine, Y., Vardeh, H., Connolly, J. L. & Fujimoto, J. G. Rapid imaging of surgical breast excisions using direct temporal sampling two photon fluorescent lifetime imaging. Biomed. Opt. Express 6, 4317–4325 (2015).

21. 21.

Eibl, M. et al. Single pulse two photon fluorescence lifetime imaging (SP-FLIM) with MHz pixel rate. Biomed. Opt. Express 8, 3132–3142 (2017).

22. 22.

Ryu, J. et al. Real-time visualization of two-photon fluorescence lifetime imaging microscopy using a wavelength-tunable femtosecond pulsed laser. Biomed. Opt. Express 9, 3449–3463 (2018).

23. 23.

Dow, X. Y., Sullivan, S. Z., Muir, R. D. & Simpson, G. J. Video-rate two-photon excited fluorescence lifetime imaging system with interleaved digitization. Opt. Lett. 40, 3296–3299 (2015).

24. 24.

Boudoux, C. et al. Rapid wavelength-swept spectrally encoded confocal microscopy. Opt. Express 13, 8214–8221 (2005).

25. 25.

Goda, K., Tsia, K. K. & Jalali, B. Serial time-encoded amplified imaging for real-time observation of fast dynamic phenomena. Nature 458, 1145–1149 (2009).

26. 26.

Chen, C. L. et al. Deep learning in label-free cell classification. Sci. Rep. 6, 21471 (2016).

27. 27.

Bhushan, A. S., Coppinger, F. & Jalali, B. Time-stretched analogue-to-digital conversion. Electron. Lett. 34, 839–841 (1998).

28. 28.

Huber, R., Wojtkowski, M. & Fujimoto, J. G. Fourier domain mode locking (FDML): a new laser operating regime and applications for optical coherence tomography. Opt. Express 14, 3225–3237 (2006).

29. 29.

Karpf, S. & Jalali, B. Fourier-domain mode-locked laser combined with a master-oscillator power amplifier architecture. Opt. Lett. 44, 1952–1955 (2019).

30. 30.

Karpf, S. et al. Two-photon microscopy using fiber-based nanosecond excitation. Biomed. Opt. Express 7, 2432–2440 (2016).

31. 31.

Yokoyama, H. et al. Two-photon bioimaging with picosecond optical pulses from a semiconductor laser. Opt. Express 14, 3467–3471 (2006).

32. 32.

Donnert, G., Eggeling, C. & Hell, S. W. Major signal increase in fluorescence microscopy through dark-state relaxation. Nat. Methods 4, 81–86 (2007).

33. 33.

Patterson, G. H. & Piston, D. W. Photobleaching in two-photon excitation microscopy. Biophysical J. 78, 2159–2162 (2000).

34. 34.

Débarre, D., Olivier, N., Supatto, W. & Beaurepaire, E. Mitigating phototoxicity during multiphoton microscopy of live Drosophila embryos in the 1.0–1.2 µm wavelength range. PLoS ONE 9, e104250 (2014).

35. 35.

Hopt, A. & Neher, E. Highly nonlinear photodamage in two-photon fluorescence microscopy. Biophysical J. 80, 2029–2036 (2001).

36. 36.

König, K., Becker, T. W., Fischer, P., Riemann, I. & Halbhuber, K. J. Pulse-length dependence of cellular response to intense near-infrared laser pulses in multiphoton microscopes. Opt. Lett. 24, 113–115 (1999).

37. 37.

Jiang, Y., Karpf, S. & Jalali, B. Time-stretch LiDAR as a spectrally scanned time-of-flight ranging camera. Nat. Photonics 14, 14–18 (2020).

38. 38.

Klein, T. & Huber, R. High-speed OCT light sources and systems [Invited]. Biomed. Opt. Express 8, 828–859 (2017).

39. 39.

Kolb, J. P., Pfeiffer, T., Eibl, M., Hakert, H. & Huber, R. High-resolution retinal swept source optical coherence tomography with an ultra-wideband Fourier-domain mode-locked laser at MHz A-scan rates. Biomed. Opt. Express 9, 120–130 (2018).

40. 40.

Wieser, W. et al. High definition live 3D-OCT in vivo: design and evaluation of a 4D OCT engine with 1 GVoxel/s. Biomed. Opt. Express 5, 2963–2977 (2014).

41. 41.

Eigenwillig, C. M. et al. Picosecond pulses from wavelength-swept continuous-wave Fourier domain mode-locked lasers. Nat. Commun. 4, 1848 (2013).

42. 42.

Karpf, S., Eibl, M., Wieser, W., Klein, T. & Huber, R. A time-encoded technique for fibre-based hyperspectral broadband stimulated Raman microscopy. Nat. Commun. 6, 6784 (2015).

43. 43.

Köllner, M. & Wolfrum, J. How many photons are necessary for fluorescence-lifetime measurements? Chem. Phys. Lett. 200, 199–204 (1992).

44. 44.

Yamada, K. et al. Efficient selective breeding of live oil-rich Euglena gracilis with fluorescence-activated cell sorting. Sci. Rep. 6, 26327 (2016).

45. 45.

Koester, H. J., Baur, D., Uhl, R. & Hell, S. W. Ca2+ fluorescence imaging with pico- and femtosecond two-photon excitation: signal and photodamage. Biophysical J. 77, 2226–2236 (1999).

46. 46.

Obrzud, E., Lecomte, S. & Herr, T. Temporal solitons in microresonators driven by optical pulses. Nat. Photonics 11, 600–607 (2017).

47. 47.

Bewersdorf, H. Picosecond pulsed two-photon imaging with repetition rates of 200 and 400 MHz. J. Microsc. 191, 28–38 (1998).

48. 48.

Chen, S.-Y. et al. In vivo virtual biopsy of human skin by using noninvasive higher harmonic generation microscopy. Sel. Top. Quantum Electron., IEEE J. 16, 478–492 (2010).

49. 49.

Schönle, A. & Hell, S. W. Heating by absorption in the focus of an objective lens. Opt. Lett. 23, 325–327 (1998).

50. 50.

He, W., Wang, H., Hartmann, L. C., Cheng, J.-X. & Low, P. S. In vivo quantitation of rare circulating tumor cells by multiphoton intravital flow cytometry. Proc. Natl Acad. Sci. USA 104, 11760–11765 (2007).

51. 51.

Koren, L. E. High-yield media for photosynthesizing Euglena gracilis Z. J. Protozool. 14, 17 (1967).

Acknowledgements

This research was sponsored in part by the National Institutes of Health grants 5R21GM107924-03 to B.J. and C.P.-C. and R21EB019645 to B.J., Cal-BRAIN grant 350050 (California Blueprint for Research to Advance Innovations in Neuroscience) to B.J. and C.P.-C., as well as grant W81XWH-14-1-0433 (USAMRMC, DOD), and NIH NICHD grant R01 HD054453 to C.P.-C. and by ImPACT Program of the Council of Science, Technology and Innovation (Cabinet office, Government of Japan) to C.R. and D.D.C. Sebastian Karpf gratefully acknowledges a postdoctoral research fellowship from the German Research Foundation (DFG, project KA 4354/1-1), the Juniorprofessorship with financial support by the state of Schleswig-Holstein (Excellence chair program by the universities Kiel and Luebeck) and funding from the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC 2167-390884018.

Author information

Authors

Contributions

S.K. conceived the idea, built the system and conducted the experiments. C.R. and D.D.C. provided the Euglena gracilis cells. C.R. and S.K. conducted the Euglena flow measurements. A.G, W.Z., A.S. and C.P.C. provided the mouse brain sample. C.P.C. and S.K. conducted the mouse brain imaging measurement. S.K. and B.J. conceived the digital image processing capabilities. B.J. and S.K. performed system analysis and wrote the manuscript. B.J. supervised the research.

Corresponding author

Correspondence to Sebastian Karpf.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review information Nature Communications thanks the anonymous reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

Karpf, S., Riche, C.T., Di Carlo, D. et al. Spectro-temporal encoded multiphoton microscopy and fluorescence lifetime imaging at kilohertz frame-rates. Nat Commun 11, 2062 (2020). https://doi.org/10.1038/s41467-020-15618-w

• Accepted:

• Published:

• Spatiotemporal sectioning of two-photon fluorescence ellipsoid with a CsPbBr3 nanosheet

• Yizhi Zhu
• , Jinping Chen
• , Qiannan Cui
• , Heng Guo
• , Zhuxin Li
• , Zengliang Shi
•  & Chunxiang Xu

Nano Research (2021)