Single-shot stereo-polarimetric compressed ultrafast photography for light-speed observation of high-dimensional optical transients with picosecond resolution

Simultaneous and efficient ultrafast recording of multiple photon tags contributes to high-dimensional optical imaging and characterization in numerous fields. Existing high-dimensional optical imaging techniques that record space and polarization cannot detect the photon’s time of arrival owing to the limited speeds of the state-of-the-art electronic sensors. Here, we overcome this long-standing limitation by implementing stereo-polarimetric compressed ultrafast photography (SP-CUP) to record light-speed high-dimensional events in a single exposure. Synergizing compressed sensing and streak imaging with stereoscopy and polarimetry, SP-CUP enables video-recording of five photon tags (x, y, z: space; t: time of arrival; and ψ: angle of linear polarization) at 100 billion frames per second with a picosecond temporal resolution. We applied SP-CUP to the spatiotemporal characterization of linear polarization dynamics in early-stage plasma emission from laser-induced breakdown. This system also allowed three-dimensional ultrafast imaging of the linear polarization properties of a single ultrashort laser pulse propagating in a scattering medium.


Introduction and Novelty
The manuscript introduces stereo-polarimetric compressed ultrafast photography (SP-CUP), an imaging system capable of recording high-dimensional events in a single exposure at speeds surpassing other single-shot high-dimensional cameras in the literature. The readout speed of current state-of-the-art sensors has limited single-shot temporal imaging to speeds significantly lower than the speed of light. However, SP-CUP bypasses this issue by implementing streak imaging. The system, however, also avoids inherent problems with streak imaging such as low sequence depth and a low field of view as well as producing high dimensional data by combining it with elements of stereoscopy, and polarimetry, and compressed sensing. The manuscript first describes the optical setup used to achieve these results in detail before moving on to a series of experiments that demonstrate their model's ability to capture multiple photon tags and generalizability to different applications. This method is novel in its advancement of existing CUP techniques by enabling simultaneous recording of polarization in three-dimensional space. It also improves upon current methods of single-shot ultrafast imaging techniques by producing a more favorable sequence depth, spatiotemporal resolution, and scalability in high dimensional imaging. The proposed method is clearly described in the context of alternate options and its novel improvements are outlined in detail.

Major Critiques
This manuscript makes it clear that SP-CUP can enable the collection of multiple photon tags in scattering media, and the recording of non-repeatable, single-shot events such as the polarization dynamics of plasma in laser-induced breakdown. The value that this device brings to the scientific community is evident, however, several aspects of this manuscript should be modified for it to match the caliber of other manuscripts published in Nature Communications. Firstly the paper mentions several times that SP-CUP synergizes compressed sensing and streak imaging with stereoscopy and polarimetry. Not only does this statement become redundant throughout the manuscript, but its implications are not fully explored. How streak imaging, stereoscopy, polarimetry, compressed sensing are integrated into the larger system is made clear, however, why these techniques, in particular, were selected and how they contribute to the advancements that the system has made should be fully fleshed out to fully orient the reader. Furthermore, the optical setup figure should be more integrated into the main body of text with perhaps less text description and more references to Figure 1 so that it's easier to visualize what the manuscript is referring to when describing each component.
The discussion section of the paper introduces more applications where SP-CUP could be implemented. This section is lengthy. Though applications discussed should be mentioned, without quantitative evidence for how SP-CUP performs in these environments, an in-depth write up lacks value to justify its length.
Overall, the manuscript in its current format is not yet suitable for publication without major modifications as suggested above. Should the authors consider addressing those comments, the value of SP-CUP can be made succinct and obvious to the scientific community, and I would gladly review again an updated version of the manuscript.

RESPONSES TO REVIEWERS
We sincerely appreciate the reviewers' timely effort in providing valuable comments of this manuscript, especially under the exceptional circumstance of COVID-19. We have adopted these comments to improve the quality of our manuscript. The point-by-point responses are listed as follows. All changes in the revised manuscript are highlighted in red.

REVIEWER 1 [Comment 0]
The authors present highly innovative work on a novel imaging platform capable of recording both stereoscopic and polarization information with picosecond temporal resolution. The method is based on the streak camera approach, which converts temporal resolution into a spatially sheared image. Although conventional streak cameras produce a one-dimensional image over time, the authors use compressive sensing methodology to produce 2-D image by imaging spatially sparse scenes. This work further pushes the frontier of ultra-fast imaging by capturing polarization information while also computing depth of the imaged objects.
The authors demonstrated that with this new and exciting technology, new physical phenomena, which have never been observed before, can be examined. In particular, polarization effects related to early-stage plasma are observed and reported for the first time in the literature.
These effects last for several hundreds of picoseconds and have never been studies before. These experiments demonstrated that this imaging technology will help uncover new basic science phenomena that have not been able to be observed before due to technology limitations. This work is highly innovative and transformative.

[Response 0]
We sincerely appreciate the reviewer's acknowledgment of the technical innovation carried in our work. We also thank the review for her/his recognition of the application of SP-CUP to observing new physical phenomena.
There are few minor comments that can help improve the quality of the manuscript:

[Comment 1]
1. The title of the paper suggests "real-time", which typically implies 30 to 100 frames per second temporal resolution. Since this technology is capturing events at over billion frames per second, the real-time does not accurately depicts the temporal capabilities of the system. I suggest including either picosecond resolution or billion frame-rate instead.

[Response 1]
In our manuscript, "real time" is defined, according to the Merriam-Webster dictionary, as "the actual time during which something takes place" [R1]. Therefore, the word is properly chosen for the title.
However, we agree with the reviewer that the word "real time" has different interpretations in the imaging community. Therefore, following the reviewer's suggestion, we have changed the title to "Single-shot stereo-polarimetric compressed ultrafast photography: Light-speed observation of high-dimensional optical transients with picosecond resolution". We also defined "real time" in the revised abstract.

[Comment 2]
2. Also the work focused on linear polarization while the title of the paper states "polarization".
It would be good to add linear to discern that circular polarization is not currently captured with the presented system.

[Response 2]
We have revised throughout the manuscript to clearly state that the polarization parameters that SP-CUP system captures are the angle of linear polarization (AoLP) and the degree of linear polarization (DoLP). The word "linear" has been added throughout the revised manuscript to emphasize the type of polarization that the system can measure. Finally, we have added a sentence in Summary about future work on extending SP-CUP to measure light with arbitrary polarization states (see Lines 323-325, Page 11, in the revised manuscript).
However, we decided to keep "stereo-polarimetric" in the title for generality and simplicity.
The current system can capture both depth and polarization information in addition to 2D ( , ) space and time. The reviewer is correct that our current system can only detect linear polarization, however, with modifications (e.g., by including more views with more different polarizers), an upgraded system may enable detection of the polarization state of any light.

[Comment 3]
3 3. The conclusion of the paper states that the angle and degree of linear polarization were accurately measured though error metric (or accuracy and precision) for polarization imaging is not presented. The authors should quantify the error in the reconstructed angel and degree of linear polarization for different light intensities and present it in the paper.

[Response 3]
The related data have been presented in our original manuscript. The error quantification of measured AoLPs in SP-CUP was conducted with the two proof-of-concept experiments (see

[Comment 4]
4. Related to the previous point, in the measured angles of polarization in the various experiments, the authors state the difference between the measured and actual values. What is this error due to and does the system need to be calibrated with respect to polarization (similar to the calibration used for stereoscopic imaging)? Discussion on this error should be included in the paper and how it can be mitigated.

[Response 4]
The errors in the calculation of AoLP and DoLP are mainly induced by the noise performance of the streak camera and by the reconstruction algorithm. The streak camera induces noise during the photon-to-photoelectron conversion in the photocathode, photoelectron-to-photo conversion on the phosphor screen, and the optical signal amplification inside the image intensifier. All these random noise sources cannot be calibrated. On the other hand, strong illumination intensity would aggravate the space-charge effect, which blurs the streak image. As a result, the SNR in the raw data is moderate. The noise in the raw data affects how accurately the reconstruction algorithm 4 allocates the signals to the correct spatiotemporal voxel in the reconstructed movie. Finally, the intensity error in the reconstructed movie is transferred to the inaccuracy in the calculation of AoLP and DoLP.
The mitigation of the errors in AoLP and DoLP can be performed by tackling these noise sources. Possible approaches include modifying the streak tube's design, using optical-streaking approaches, limiting the photon flux, and implementing new penalty terms and/or regularizers.
The related discussion has been included in Supplementary Note 6 (see Lines 918-935 on Page 43 in the revised manuscript).

[Comment 5]
5. In figure 3-c, the color scheme does not capture very well the temporal variations in the DOLP because they are mostly represented as grayscale. Since the AOP does not change, it would be better to represent the DOLP in jet scale and AOP as grayscale. This can help in better visualization of the ~30% drop of DOLP.

[Response 5]
Following the reviewer's suggestion, we have re-plotted Fig. 3c and Supplementary Movie 2 by using a jet colormap for DoLP and a grayscale colormap for AoLP.

REVIEWER 2 [Comment 0]
This is an interesting paper. I recommend publication with mandatory revisions, primarily aimed at clarifying the methods and the data.

[Response 0]
We sincerely thank the reviewer's support of our work.

[Comment 1]
The description of the image reconstruction given on pages 3-5 is not transparent enough for the general audience of Nature Communications. I recommend that the authors include a more intuitive description. Leaving the details of the methods to the Methods and Supplementary Materials is OK, but the reader should be able to have a fairly good understanding of how the method works based on the description in the main text. Perhaps the authors can start with an intuitive description of their basic CUP technique and then explain how they add the additional z and polarization dimensions?

[Response 1]
Following the reviewer's suggestions, we have modified the related content in Section 1 of Results.
We have added a more explanatory description of SP-CUP's image reconstruction to show how the method works. In particular, we have explained in detail (1)

[Comment 2]
Why is the inverse problem under-determined in this case, and how that affects the image retrieval?

[Response 2]
The under-determined nature of the inverse problem originates from the fact that the number of pixels in the measured data is much fewer than that in the high-dimensional datacube under observation. Under this situation, the sensing matrix does not have a full rank. Simple matrix 6 inversion cannot faithfully recover the value of each pixel in the datacube. Thus, the inverse problem is under-determined.
However, it has been shown that compressed sensing can accurately solve such an inverse problem via optimization. This salient feature of compressed sensing serves as the foundation of image reconstruction employed in SP-CUP. In a noise-free situation, compressed-sensing algorithms can reconstruct images with excellent image quality. The presence of noise compromises the reconstructed image quality. However, with a reasonable signal-to-noise ratio (SNR), the compressed-sensing algorithm can faithfully recover the scene.
To make our manuscript clearer, we have added more explanation in Section 1 of Results.
Please see them in Lines 116-130 on Pages 4-5 in the revised manuscript. We have also realized that the word "under-determined" may be confused with the "incapable of determining the values of reconstructed elements". Therefore, we have changed the wording to "ill-conditioned", which is also commonly used in the compressed sensing community [R2].

[Comment 3]
The authors need to include the estimates for the limitations of their imaging technique, within the implementation used here -the x,y,z,t resolutions and ranges, as well as the error of the measurement of the DoLP. What are the major limiting factors? That would give the reader an idea of what type of dynamic processes this technique can be used to image.

[Response 3]
In the original manuscript, the quantification of the spatial ( , , ) and temporal resolutions have been included, and the results are summarized in Supplementary Table S5 ( We have also analyzed the limitations of resolutions in , , , and in the revised manuscript. In general, the resolutions in , , and are limited by the encoding pixel size, the SNRs in the acquired data, and the overall magnification ratio of the SP-CUP system. The resolution in z is determined by the minimal distinguishable disparity, which is determined by the SNR in the acquired data. We have also added the analysis in the range limitation in , , , and . The range of ( , , ) is limited by the sensor size of the internal sensor and the overall magnification ratio of the imaging system. In the design of the SP-CUP system, the range of z is 7 limited by the depth of field of the camera lens. For detailed analysis of the , , , and resolutions and ranges, please see Line 816-848 on Pages 37-39 in Supplementary Note 5 in the revised manuscript.
The error of the measurement of the DoLP is mainly induced by the noise performance of the streak camera and by the reconstruction algorithm. Please refer to Response 4 for Reviewer 1.
We have experimentally quantified the error of DoLP. The results and analysis are presented in Supplementary Note 6 (see Lines 885-915 on Pages 41-43 in the revised manuscript).

[Comment 4]
In my opinion, including the description of the operation of a streak camera in the Supplementary seems unnecessary. That is a common knowledge.

[Response 4]
We agree with the reviewer that the operating principle of a streak camera is common knowledge to certain communities in physics. However, this knowledge may not be readily available to many other communities that are related to our work, including computational imaging, polarization optics, and three-dimensional imaging acquisition/processing. To make our manuscript a complete and self-referenced piece as well as considering the broad readership of Nature Communication, we decided to keep this section.

[Comment 5]
The author's example involving the polarization evolution of the emission by the ablation plume is interesting, if they are looking at a real effect. First, the ejecta they are showing are not "early- We thank the reviewer to provide this explanatory comment on our experiment. First, we would like to point out that polarized plasma emission is indeed a real effect. The plasma emission can be divided into two parts from the perspective of spectrum: (1) light from a series of discrete spectral lines and (2) a broadband continuum background. The reviewer is correct that the former, produced by fluorescence, is largely unpolarized. However, it has been shown in numerous literature that the latter is indeed polarized [R3, R4]. This characteristic sparkled the development of polarization-resolved laser-induced breakdown spectroscopy (PR-LIBS) to improve the SNR in measured data. More importantly, the emission of the broadband continuum background starts immediately after the laser excitation, while the series of discrete lines appear after tens of nanoseconds [R5]. This characteristic has inspired the development of time-gating-based PR-LIBS to further enhance the signal-to-background ratio [R6]. In our experiment, we imaged the plasma emission from 0 to 3 ns. Therefore, the majority of the light collected by the SP-CUP system is from the broadband continuum background, not from the discrete spectral lines. Thus, the light incident into the SP-CUP system is polarized.
Second, the word "early-stage" refers to the plasma emission, not plasma formation. The same expression has been used in many previous papers. Particularly for our work, we imaged the light from broadband continuum background in plasma emission, not from the discrete spectral lines. Therefore, the term that we used in the manuscript is accurate. However, we realized that in a few places in the original manuscript, "early-stage plasma emission" was not explicated written in certain context. In the revised manuscript, we have used "early-stage plasma emission" consistently throughout the text.
Third, following the reviewer's suggestion, we have conducted the error quantification of DoLP (see Lines 885-915 on Page 41-43 in Supplementary Note 6 in the revised manuscript).
Our results show that the SP-CUP system can offer a highly accurate measurement of DoLP. The decrease in DoLP is 0.25, which is much larger than the value of DoLP reconstruction accuracy (i.e., <0.044). This quantitative comparison supports our claim that the observed decrease in DoLP in the first 3 ns is a real effect.
Fourth, the values of DoLP measured by SP-CUP are supported by the literature. It is known that the polarization state of plasma emission highly depends on the specific experimental setups [R7], including the polarization, pulse width, and pulse energy of the laser beam, the composition, orientation, and pressure of sample, and the angle of observation. Despite these difficulties, DoLP values of 0.1-0.25 in PR-LIBS using a silicon sample (or a sample whose element has a similar atomic number to that of silicon) can be found in many references, e.g. [R8-R10]. These results are usually based on time-integrated measurement. Thus, the slightly higher DoLP in our measurements results from SP-CUP's picosecond temporal resolution that excludes the unpolarized emission in discrete spectral lines. In addition, we observed the DoLP's decrease over time. This trend well agrees with the fact that the polarized light from the broadband continuum background precedes the unpolarized light from the discrete spectral lines. Therefore, we focused on the presentation of experimental results rather than analyzing the underlying mechanisms. Rather, we hope the picosecond temporal resolution and single-shot capability of SP-CUP could contribute to further investigation of this interesting topic.

[Comment 6]
The relevant data shown in Figure 3(c) are not clear. Color coding in two dimensions (DoLP and psi) may not be a good representation of the data in this case. The plots look predominantly grey.
Perhaps the authors can change the color mapping or present the data in some other way to make the features they are talking about standout better?

[Response 6]
This comment echoes Comment 5 of Reviewer 1. In the revised manuscript, we have re-plotted AoLP can be supported by related literature (e.g., [R12-R14]). According to these papers, the unchanged AoLP could be caused by the Fresnel reflection of the plasma light by the sample or by the characteristic of the plasma itself. For the geometry used in our setup (see Fig. 3a in Main Text), the first potential reason can be ruled out. The second reason is possible because the anisotropic electron distribution function may decrease homogenously over time, resulting in a decreased DoLP and a maintained AoLP [R15]. However, a precise elucidation of this phenomenon is beyond our expertise and the scope of our paper. We hope that our result will provide experimental data for more theoretical investigation in the future.
We have reflected the above response in Discussion in the revised manuscript. Please see Lines 302-304 on Page 10.

Introduction and Novelty
The manuscript introduces stereo-polarimetric compressed ultrafast photography (SP-CUP), an imaging system capable of recording high-dimensional events in a single exposure at speeds surpassing other single-shot high-dimensional cameras in the literature. The readout speed of current state-of-the-art sensors has limited single-shot temporal imaging to speeds significantly lower than the speed of light. However, SP-CUP bypasses this issue by implementing streak imaging. The system, however, also avoids inherent problems with streak imaging such as low sequence depth and a low field of view as well as producing high dimensional data by combining it with elements of stereoscopy, and polarimetry, and compressed sensing. The manuscript first describes the optical setup used to achieve these results in detail before moving on to a series of experiments that demonstrate their model's ability to capture multiple photon tags and generalizability to different applications. This method is novel in its advancement of existing CUP techniques by enabling simultaneous recording of polarization in three-dimensional space.
It also improves upon current methods of single-shot ultrafast imaging techniques by producing a more favorable sequence depth, spatiotemporal resolution, and scalability in high dimensional imaging. The proposed method is clearly described in the context of alternate options and its novel improvements are outlined in detail.

Major Critiques
This manuscript makes it clear that SP-CUP can enable the collection of multiple photon tags in scattering media, and the recording of non-repeatable, single-shot events such as the polarization dynamics of plasma in laser-induced breakdown. The value that this device brings to the scientific community is evident, however, several aspects of this manuscript should be modified for it to match the caliber of other manuscripts published in Nature Communications.

[Response 0]
Dear Tessa, thank you for pointing out the technical novelty of SP-CUP and for your acknowledgment about the evident value that the SP-CUP system brings to the scientific community.

[Comment 1]
Firstly the paper mentions several times that SP-CUP synergizes compressed sensing and streak imaging with stereoscopy and polarimetry. Not only does this statement become redundant throughout the manuscript, but its implications are not fully explored. How streak imaging, stereoscopy, polarimetry, compressed sensing are integrated into the larger system is made clear, however, why these techniques, in particular, were selected and how they contribute to the advancements that the system has made should be fully fleshed out to fully orient the reader.

[Response 1]
We have deleted the repeated claims about the synergy in SP-CUP's system design in the revised manuscript. We have added more explanation about the underlying rationales to choose specific approaches of stereoscopy and polarimetry for the SP-CUP system from the aspects of functionality, compatibility, and flexibility. We have also associated these rationales with specific steps in image reconstruction to improve the clarification of the description. Please see this information in Lines 114-146 on Pages 4-5 in the revised manuscript.

[Comment 2]
Furthermore, the optical setup figure should be more integrated into the main body of text with perhaps less text description and more references to Figure 1 so that it's easier to visualize what the manuscript is referring to when describing each component.

[Response 2]
Following the reviewer's suggestion, we have shortened the caption of Fig. 1. We have added more details in the description of the system and refer to Fig. 1 as much as possible (see Lines 81-113 on Pages 3-4 in the revised manuscript).

[Comment 3]
The discussion section of the paper introduces more applications where SP-CUP could be implemented. This section is lengthy. Though applications discussed should be mentioned, without quantitative evidence for how SP-CUP performs in these environments, an in-depth write up lacks value to justify its length.

[Response 3]
We have significantly shortened the description of these potential applications in the revised manuscript. In particular, in Discussion, we have largely reduced the length of Section 1 and have deleted Section 3 of the original manuscript. These potential applications are briefly mentioned in Summary (see Lines 327-330 on Page 11 in the revised manuscript).

[Comment 4]
Overall, the manuscript in its current format is not yet suitable for publication without major modifications as suggested above. Should the authors consider addressing those comments, the value of SP-CUP can be made succinct and obvious to the scientific community, and I would gladly review again an updated version of the manuscript.

[Response 4]
We firmly believe that our revision has reflected the values of SP-CUP in a more straightforward means to the scientific community. We sincerely hope that this revised manuscript could satisfy your requirements for publishing our work.