Fast wide-field upconversion luminescence lifetime thermometry enabled by single-shot compressed ultrahigh-speed imaging

Photoluminescence lifetime imaging of upconverting nanoparticles is increasingly featured in recent progress in optical thermometry. Despite remarkable advances in photoluminescent temperature indicators, existing optical instruments lack the ability of wide-field photoluminescence lifetime imaging in real time, thus falling short in dynamic temperature mapping. Here, we report video-rate upconversion temperature sensing in wide field using single-shot photoluminescence lifetime imaging thermometry (SPLIT). Developed from a compressed-sensing ultrahigh-speed imaging paradigm, SPLIT first records wide-field luminescence intensity decay compressively in two views in a single exposure. Then, an algorithm, built upon the plug-and-play alternating direction method of multipliers, is used to reconstruct the video, from which the extracted lifetime distribution is converted to a temperature map. Using the core/shell NaGdF4:Er3+,Yb3+/NaGdF4 upconverting nanoparticles as the lifetime-based temperature indicators, we apply SPLIT in longitudinal wide-field temperature monitoring beneath a thin scattering medium. SPLIT also enables video-rate temperature mapping of a moving biological sample at single-cell resolution.

In practice, tm was computed by using the static letter "A" pattern. Supplementary Figs. 1a-b show the acquired images in View 1 and View 2. The co-registered View 1 image ( Supplementary Fig. 1c) and the View 2 image were used for SPLIT's image reconstruction.

Supplementary Note 2: Derivation of the SPLIT's reconstruction algorithm
In image reconstruction, the datacube of the dynamic scene is recovered by solving the minimization problem aided by regularizers 3 . In particular, the inverse problem [i.e., Equation (2) in Main Text] is first written as where , , and are primal variables. is the set of possible solutions in compliance with the spatial constraint 4 , which is generated by binarizing the image of View 1 (i.e., 1 ) with an appropriate intensity threshold that is determined by the Otsu's method 5
To retrieve the dynamic scene, the reconstruction algorithm sequentially updates primal variables, estimated solution +1 ( denotes the iteration time), dual variables and penalty parameters as well as evaluates the pre-set criteria, as following five steps.
Here, (0 < < 10 -3 ) is the pre-set tolerance value. These steps are repeated until both criteria in Step 5 are satisfied. The image reconstruction recovers the datacube of the dynamic scene.

Supplementary Note 3: Simulation results of the dual-view PnP-ADMM algorithm
To test the proposed dual-view plug-and-play alternating direction method of multipliers (PnP-ADMM) algorithm, we reconstructed a simulated dynamic scene-the intensity decay of a static  Fig. 2b).
Supplementary Fig. 2c presents the reconstructed normalized average intensity versus time, which has a good agreement with the pre-set intensity decay (black dashed line).

Supplementary Note 4: Details on the relationship between temperature and lifetime
The normalized area integration method is commonly used for calculating lifetime based on pulsed excitation 14 . Photoluminescence lifetime of UCNPs following pulsed excitation can be expressed by Here 2 ) represents the Gaussian excitation pulse with a pulse width of w .
( ) = ∑ ε exp( − ⁄ ) is used to represent the photoluminescence with multiple exponential decays, each of which has a lifetime and a proportion ε . " * " denotes convolution. Then, Supplementary Equation (10) becomes When w approaches to zero, which indicates the case of an ultrashort pulse, Supplementary Equation (11) becomes Following the established theory 15 , we defined the photoluminescence lifetime as = ∑ ε ∑ ε ⁄ . Considering that ∑ ε = 1, we have = L t .
The lifetime is linearly linked to the temperature by Here a denotes the absolute temperature sensitivity, and t denotes a constant. This derivation produces Equation (3) in Main Text.
In the SPLIT system, we used a continuous-wave laser and an optical chopper to generate excitation pulses. Although the chopper blade's slit width could approach zero for generating an ultrashort pulse duration, it demands a high laser power. Thus, a finite pulse width needs to be chosen to provide sufficient signal-to-noise ratio (SNRs) in measurements while still maintaining accurate lifetime calculation. In practice, we chose w = 50 μs, which was comparable to the values used in the literature 14 . Our calculation also showed that this pulse width induced a <0.3% calculation error for the 5.6-mm-thick-shell UCNPs that were mainly used in our experiments.
Thus, 50-μs pulse width allowed SPLIT to produce accurate temperature mapping results.

Synthesis of UCNPs
Core/shell NaGdF4: 2 mol% Er 3+ , 20 mol% Yb 3+ /NaGdF4 UCNPs were synthesized via the previously reported thermal decomposition method, with minor modifications to the synthesis procedure 16  In the second step, core/shell UCNPs of different shell thicknesses were prepared by epitaxial growth of the shell on the preformed cores via a multi-step hot-injection approach. First, we prepared Solution A by mixing approximately 1.5 mmol of core UCNPs (~21.6 mL) in a 100 mL three-neck round bottom flask together with 9.2 mL each of OA and ODE. Separately, we prepared Solution B by mixing 3 mmol of gadolinium trifluoroacetate (shelling) precursors with 3 mmol of sodium trifluoroacetate, and 10.5 mL each of OA and ODE. Both solutions were degassed under vacuum and magnetic stirring at 110 ºC for 30 minutes. After degassing, Solution A was back-filled with argon gas and the temperature was raised to 315 ºC. Solution B was then injected into the reaction vessel containing Solution A using a syringe and pump system at a 0.75 mL min -1 injection rate in three steps. After each ~7 mL injection step, the mixture was allowed to react for 60 minutes. A portion of core/shell UCNPs would be extracted before the next injection step: 15.6 mL after the first injection step for core/shell UCNPs with a 1.9 nm-thick shell and 19.2 mL after the second injection step for core/shell UCNPs with a 3.5 nm-thick shell.
Extractions were allowed to cool down to room temperature before transfer from glass syringe to Falcon centrifuge tube for subsequent washing. After the final injection step and a total of 180 minutes of reaction, the mixture (core/shell UCNPs with a 5.6 nm-thick shell) was cooled to room temperature under argon gas and magnetic stirring. All core/shell UCNPs were precipitated with ethanol and washed three times with hexane/acetone (1/4 v/v in each case), followed by centrifugation (with 5400 x g). Finally, all UCNPs were re-dispersed in hexane for further structural and optical characterization.

Structural characterization
The morphology and size distribution of the core/shell UCNPs were investigated by transmission electron microscopy (TEM, Philips, Tecnai 12). The particle size was determined from TEM images using ImageJ software with a minimum set size of 280 individual UCNPs per sample. The results are shown in Figure 2a. The crystallinity and phase of the core-only and core/shell UCNPs were determined via X-ray powder diffraction (XRD) analysis using a diffractometer (Bruker, D8 Advance) with CuKα radiation (Supplementary Fig. 3). The peaks in measured XRD spectra match the reference tabulated data (PDF# 01-080-8787). Along with the TEM images (i.e., Figure 2a), this result ensured that the fabricated UCNPs were of the hexagonal crystal phase.

Supplementary Note 6: Characterization of SPLIT's system sensitivity
To test the sensitivity of SPLIT, we monitored the reconstructed image quality while decreasing the laser power. The detection sensitivity of the SPLIT system was characterized by imaging photoluminescence intensity decay with various excitation power densities ( Supplementary Fig.   4). Transparency of the letter "P" covered the sample of UCNPs with a shell thickness of 5.6 nm.

Supplementary Note 7: Measurement of photoluminescence lifetimes of UCNPs using the time-correlated single-photon counting (TCSPC) technique
To ascertain our results, we used the standard TCSPC method (Edinburgh Instruments, FLS980, 70-µs excitation pulse) to measure photoluminescence decay of the 5.6 nm-thick-shell UCNPs dispersed in hexane. The measured intensity decay curve is shown in Supplementary Fig. 5.
Lifetime values acquired from the SPLIT and TCSPC measurements yielded a 6.9% mismatch.
This difference is attributed to different environments in which UCNPs were measured (dried powder for SPLIT and solution for TCSPC), different excitation pulse widths (50-µs for SPLIT and 70-µs for TCSPC), and different instrumental responses.

Supplementary Note 8: Comparison of reconstructed image quality
To quantitatively demonstrate the superiority of the dual-view PnP-ADMM algorithm employed in SPLIT's image reconstruction, we compared it with two other algorithms dominantly used in existing streak-camera-based single-shot ultrafast imaging-the single-view two-step iterative thresholding/shrinkage (TwIST) algorithm 21 and the dual-view TwIST algorithm [22][23][24] .
Specifically, we used the experimental data of the green emission of UCNPs with shell thicknesses of 1.9 nm, 3.5 nm, and 5.6 nm, covered by transparencies of letters "C", "A", and "N", respectively. Both Using the data shown in Figure 3e, r in the pre-set temperature range were quantified to be 0.39-0.43%•ºC -1 for the green emission and 0.52-0.60%•ºC -1 for the red emission ( Supplementary Fig.   7).
Finally, the thermal uncertainty 17 in SPLIT is calculated by where τ represents the uncertainty in the measured lifetimes. Supplementary Equation (15) shows that depends on both the UCNPs' performance (quantified by the relative sensitivity, r ) and experimental setup (that limits the normalized fluctuation of lifetimes, τ τ ). τ was characterized by repeating measurements using the SPLIT system under the same experimental conditions. Specifically, using the sample of the 5.6 nm shell thickness UCNPs at 20 ºC, we repeated the 2D lifetime measurements 60 times using the excitation power density of 0.4 W mm -2 and 0.06 W mm -2 , respectively. These measurements produced τ of 1.4-2.7 μs for the green emission and 2.2-4.0 μs for the red emission, respectively. With known values of | a | and by using Supplementary Equation (15), SPLIT's thermal uncertainty was calculated to be 0.7-1.4 ºC for the green emission and 0.9-1.7 ºC for the red emission.

Supplementary Note 10: Demonstration of SPLIT in biological environment
The UCNP sample with the shell thickness of 5.  Fig. 9).
To test SPLIT using a scattering medium with the presence of both light scattering and absorption, the UCNPs with the shell thickness of 5.6 nm were injected into a piece of fresh beef tissue, where we also inserted a 90 µm-diameter copper wire at the depth of 0.09 mm as a spatial feature ( Supplementary Fig. 10a). Myoglobin in the beef tissue, which has similar optical absorption properties to hemoglobin 18,19 , was used to mimic the absorption by blood. To evaluate the SPLIT's imaging ability at different depths, this phantom was covered by different additional fresh beef slices, so that the thicknesses from the surface to the copper wire were 0.09 mm, 0.34 mm, 0.55 mm, and 0.60 mm. SPLIT performed photoluminescence lifetime imaging at 20 kfps.  Supplementary Fig. 10f), which are greater than their counterpart of the chicken tissue of 26 cm -1 and 18 cm -1 . Because of its longer wavelength, the red emission has weaker scattering and weaker absorption by the myoglobin, which led to deeper penetration over the green emission for both types of scattering media. Finally, we analyzed the photoluminescence lifetimes for different thicknesses, and the results are shown in Supplementary Fig. 10g. The measured photoluminescence lifetimes for both emissions do not depend on the tissue thickness and hence excitation power density under the experimental conditions of our work. Lower excitation intensity, however, reduced the SNRs in the captured snapshots, which transfers to a larger standard deviation.

Supplementary Note 11: Preparation of the single-layer onion cells doped with UCNPs
For the onion cell experiments, UCNPs with a 5.6 nm-thick shell were first transferred to water via ligand exchange with citrate molecules. In a typical procedure, citrate-coated UCNPs were The two-phase (aqueous/organic) mixture was then poured into the separatory funnel, and the aqueous phase containing the UCNPs was isolated. The UCNPs were precipitated with acetone (1/3 v/v) via centrifugation (5400 x g) for 30 minutes. The obtained pellet was re-dispersed in 25 mL of 0.2 M trisodium citrate solution (pH 7-8) and left under stirring for an additional 2 hours.
UCNPs were then precipitated with acetone (1/3 v/v) via centrifugation (5400 x g) for 30 minutes and washed twice with a mixture of water/acetone (1/3 v/v). The citrate-coated UCNPs were redispersed in distilled water. The yellow household onion was used to peel single-layer sheets of onion cells, which were incubated in a solution of citrate-coated UCNPs (3 mg mL -1 ) for 24 hours.
After the incubation, single-layer onion cells were rinsed in distilled water and dried by gently tapping with a soft tissue paper, before being placed onto microscope slides for subsequent imaging experiments. Before lifetime imaging, the presence of UCNPs in single-layer onion cells was confirmed ( Supplementary Fig. 11a) with a bright-field microscope (Nikon, ECLIPSE Ti-S).
In addition, a reference photoluminescence intensity image was taken by a custom-built confocal imaging platform (Photon Etc.), equipped with pulsed femtosecond Ti: Sapphire laser (Spectra-Physics, Mai Tai DeepSee). Samples were excited and imaged epi-fluorescently through a 20×/0.40 NA objective lens (Nikon, CFI60 TU Plan Epi ELWD). Photoluminescence intensity was recorded by a low-noise CCD camera (Princeton Instruments, Pixis100). The upconversion emission images of static onion cells (Supplementary Fig. 11b) were obtained through raster scanning a 120×120 pixel map, each of which has the size of 2 µm and the integration time 0.2 seconds per pixel. The total time to form one lifetime map was 48 minutes.

Supplementary Note 12: Comparison between SPLIT and previous streak-camera-based modalities for 2D lifetime imaging
To articulate the difference between SPLIT and previous works on ultrafast imaging that used streak cameras, we summarize their technical specifications and applications in Supplementary   Table 1. To explain the details included in this table, we first detail the working principles of streak cameras and compressed ultrafast photography (CUP); then, we summarize technical specifications and applications of the existing imaging modalities.

Streak cameras for wide-field lifetime imaging
Streak cameras are highly suitable for 2D lifetime imaging. In its conventional operation, the field of view (FOV) of streak cameras is limited by an entrance slit with typical widths of 50-100 µm.
A sweeping unit deflects the time-of-arrival of the incident light signal along the axis perpendicular to the device's entrance slit. Depending on the mechanisms of the sweeping unit, streak cameras can be generally categorized into optoelectronic and mechanical types. In optoelectronic streak cameras ( Supplementary Fig. 12a), incident photons are first converted to photoelectrons by a photocathode. After acceleration, these photoelectrons are deflected by a time-varying voltage applied on a pair of sweep electrodes. Then, these photoelectrons are converted back to photons on a phosphor screen. Finally, the optical signal is imaged to an internal sensor. The optoelectronic streak camera can achieve a temporal resolution of up to 100 fs. Because of this ultrafast imaging ability, optoelectronic streak cameras have been used for imaging the emission of fluorescence that has lifetimes in the order of picoseconds and nanoseconds [20][21][22][23][24] . However, due to the photonto-photoelectron conversion by the photocathode, the quantum efficiency (QE) of the optoelectronic streak cameras is typically <15% for visible light. Besides, the space-charge effect in the electrostatic lens system imposes constraints in the spatial resolution (typically tens to hundreds of micrometers) and the dynamic range (e.g., <10 for certain femtosecond streak cameras). Both weaknesses severely limit the quality of acquired data.
Unlike optoelectronic streak cameras, a mechanical streak camera ( Supplementary Fig. 12b) usually uses a rotating mirror (e.g., a galvanometer scanner or a polygon mirror) to deflect the light.
Since the mechanical sweeping is much slower than the optoelectronic counterpart, this type of streak camera has tunable temporal resolutions typically from hundreds of nanoseconds to microseconds, which makes them highly suitable for lifetime imaging of luminescence processes on the order of microseconds and milliseconds, such as phosphorescence and parity forbidden 4f-4f transitions in lanthanide ions 25 . Moreover, its all-optical data acquisition allows flexibly implementing many high-sensitivity cameras [e.g., electron-multiplying (EM) CCD and scientific CMOS cameras, whose QEs can be >90% for visible light] to obtain superior SNRs in measurements. The all-optical operation also avoids the space-charge effect, which enables opticslimited spatial resolution and high dynamic range (e.g., >60,000 of the EMCCD camera used in this work). Finally, the mechanical streak camera is considerably more cost-efficient than the optoelectronic streak camera. Therefore, mechanical streak cameras are perfectly suitable for imaging microsecond-level emission from UCNPs.

Single-shot compressed temporal imaging for fast 2D lifetime mapping
Single-shot compressed temporal imaging is a novel computational imaging concept that enables 2D lifetime mapping in one acquisition. In the conventional operation of the streak camera, the entrance slit limits the imaging FOV to be one-dimensional (1D). To lift this limitation, compressed-sensing paradigms have been implemented with optoelectronic streak cameras. The resulted CUP technique [21][22][23][24] allows complete opening of the entrance slit for 2D ultrafast imaging in a single shot. CUP and its variants have been applied to single-shot fluorescence lifetime imaging 21,23,24 . In contrast, to our knowledge, single-shot compressed temporal imaging has not yet been applied to 2D imaging of microsecond-to-millisecond scale lifetimes, like those of UCNP emission. SPLIT thus marks the first technique in this category. It is also the first demonstration of single-shot photoluminescence lifetime-based temperature mapping in a 2D FOV. Compared to conventional line-scanning counterpart 25 , SPLIT has considerable advantages in light throughput and sample choices.

Performance comparison between SPLIT and line-scanning lifetime imaging
To experimentally demonstrate the advantages of SPLIT to the line-scanning confocal setup 25 , we imaged a moving photoluminescent sample ( Supplementary Fig. 13). The major experimental parameters (e.g., magnification ratio, camera's exposure time, and camera's frame rate) were kept the same as those of SPLIT. The UCNPs with the shell thickness of 5.6 nm were covered by a piece of transparency of letter "A". Loaded onto a translation stage, this sample moved downward at a speed of 0.8 mm s -1 . To perform line scanning, we placed a 200-µm-wide slit at the intermediate image plane (i.e., equivalently 50-µm-wide at the sample plane) to limit the FOV to 1D (Supplementary Fig. 13a). Attached to another translation stage, the slit was scanned in the direction at a speed of 2.8 mm s -1 . Using the green emission, this line-scanning confocal setup generated six 1D lifetime maps (Supplementary Fig. 13b). After stitching these results together, we obtained a 2D lifetime map as shown in Supplementary Fig. 13c. However, the stitched result inevitably suffers from the loss of spatial content due to the dark time between adjacent camera exposures. In the meantime, the map is distorted in the vertical direction due to the sample's movement, which proves the incapability of line-scanning-based techniques in measuring dynamic photoluminescent objects. As a comparison, we used SPLIT to image this sample under the same experimental conditions. Because of its single-shot imaging ability, SPLIT produced six 2D lifetime maps (Supplementary Fig. 13d). No image produced by the SPLIT system has any loss of spatial content or distortion. The results also clearly illustrate the downward movement of the letter "A". Therefore, SPLIT has unique advantages over the conventional scanning-based lifetime measurement in data throughput, measurement accuracy, and application scope.
It is also worth pointing out that from the perspective of optical instrumentation, SPLIT provides high-sensitivity cameras with ultrahigh imaging speeds in 2D FOV. In this regard, besides the single-shot wide-field photoluminescent lifetime mapping demonstrated in this work, the SPLIT system offers a generic imaging platform for many other studies. Potential future applications include optical voltage imaging of action potentials in neurons and high-throughput flow cytometry.

Supplementary Note 13: Comparison between SPLIT and thermal imaging
We used a thermal imaging camera (Yoseen, X384D) ( Supplementary Fig. 14a) and SPLIT ( Supplementary Fig. 14b) to image UCNPs covered by a metal mask of letters "rob" in lift-out grids (Ted Pella, 460-2031-S). Akin to the SPLIT system, a 4× magnification ratio was used for the thermal imaging camera. A blackbody radiator (Yoseen, YSHT-35) was used to heat this sample to 27 ºC. The images produced by these two methods are shown in Supplementary Figs. 14c-d and the selected line profiles are shown in Supplementary Figs. 14e-f. The edge contrast of the imaged letters using the thermal imaging camera is much worse than that using SPLIT.
Moreover, the thermal imaging result presents strong background due to the same temperature of the mask, whereas SPLIT keeps a clean background thanks to its optical sensing ability.
In another experiment, we loaded the metal mask on a translation stage. The mask was kept out of the FOV to keep its temperature at 18 °C (i.e., the room temperature in our laboratory). The UCNPs were still heated up by the blackbody radiator to 27 °C. The mask was quickly moved into the FOV, and the thermal imaging camera captured the images immediately ( Supplementary Fig.   14g). The thermal image and the selected line profiles are shown in Supplementary Figs. 14h-i.
Despite the slight improvement in contrast compared to Supplementary Figs. 14c and 14e, the image quality is still incomparable to the results produced by the SPLIT system ( Supplementary   Figs. 14d and 14f). Thus, compared to a thermal imaging camera, SPLIT supplies superior temperature mapping capability. Supplementary Fig. 7. Quantification of relative temperature sensitivities of the green and red emissions of the core/shell NaGdF4:Er 3+ ,Yb 3+ /NaGdF4 UCNPs with a 5.6 nm-thick shell.
Error bar: standard deviation. Supplementary Fig. 15. Illustration of the working principle of SPLIT.