Noise-tolerant single photon sensitive three-dimensional imager

Active imagers capable of reconstructing 3-dimensional (3D) scenes in the presence of strong background noise are highly desirable for many sensing and imaging applications. A key to this capability is the time-resolving photon detection that distinguishes true signal photons from the noise. To this end, quantum parametric mode sorting (QPMS) can achieve signal to noise exceeding by far what is possible with typical linear optics filters, with outstanding performance in isolating temporally and spectrally overlapping noise. Here, we report a QPMS-based 3D imager with exceptional detection sensitivity and noise tolerance. With only 0.0006 detected signal photons per pulse, we reliably reconstruct the 3D profile of an obscured scene, despite 34-fold spectral-temporally overlapping noise photons, within the 6 ps detection window (amounting to 113,000 times noise per 20 ns detection period). Our results highlight a viable approach to suppress background noise and measurement errors of single photon imager operation in high-noise environments.

In this manuscript, Rehain et al. demonstrate a single-photon level 3D imager based on quantum parametric mode sorting (QPMS). This imager allows for the faithful reconstruction of 3D objects even in the case of strong background noise, which renders standard approaches based on direct detection infeasible. QPMS is used to implement high spectral-temporal noise rejection by effectively implementing a single-mode detection. Every noise photon that is (partially) orthogonal to the detection mode is recorded with diminished probability. Hence, even in the case of noise being orders of magnitudes stronger than the reflected signal light, object reconstruction is possible. In addition, QPMS outperforms linear time-frequency filtering.
The manuscript is well written and the results are impressive. I am wondering, however, whether the paper contains enough genuine novelty to warrant publication in Nature Communications. The authors themselves have demonstrated that QPMS can beat linear filters roughly two years ago. In that sense, the performance of the 3D imager is not surprising. Also, using QPMS as basis for 3D imaging is indeed a brilliant idea, but I want to kindly ask the authors to comment on how this relates to ideas presented by Banaszek and co-workers earlier this year (Banaszek et al., "Approaching the ultimate capacity limit in deep-space optical communication", Proc. SPIE 10910, Free-Space Laser Communications XXXI, 109100A (4 March 2019); https://doi.org/10.1117/12.2506963). It seems to me that the idea of using QPMS for time-frequency filtering in LIDAR is very similar to the idea of using quantum pulse gating for noise rejection in deep space communications. I appreciate that this was only presented at a conference and thus is extremely hard to find. Also, I'm only asking for a statement on the similarities between the two approaches to help me make up my mind on the novelty of this manuscript.
There are a few other points I would like to raise and questions I would like to ask: --The authors compare their work to direct detection with InGaAs APDs throughout the manuscript, and state that the latter was limited by dark counts, timing resolution, and detector saturation. Can the authors provide any measurement results that support this claim? If not, can the authors comment on the details and assumptions of the underlying model? --What is the conversion efficiency of the upconverter used in the experiment? --In Figure 4, the authors recover a smoothed image from the raw image by applying a median filter. Maybe they could say a few more words about the parameters of the median filter? Also, it might help the broader readership if the authors explicitly stated that this was a standard method for removing noise from pictures in post processing. In conclusion, I think that this manuscript has the potential to be published in Nature Communications. It contains a good idea for a relevant application, the data is sufficient to support the claims of the authors and is well presented, and the work is interesting for a broad readership. If the authors convincingly lay out the novelty contained within this work, I am happy to reconsider the manuscript. At the current stage, I cannot, however, recommend publication in Nature Communications as I am not convinced that the manuscript does contain enough genuine novelty. Still, I want to commend the authors for making reading the manuscript a very joyful experience, and I am sure that they will be able to highlight the novelty more clearly.
Reviewer #2 (Remarks to the Author): The submitted manuscript introduces and demonstrates a 3D imager concept that uses quantum parametric mode sorting (QPMS) to greatly improve the ratio of detected signal to detected background photons, which I will call detected SBR. This improvement through optical means allows the imager to be effective even when the incident SBR is very low, including settings where the incident background flux would cause prohibitive dead time effects. Since the mitigation of low incident SBR is optical, it does not rely on assumptions about scene structure, as most computational imaging methods for mitigating low incident SBR would do. The avoidance of being limited by dead time effects is well-illustrated by an experiment in which a mannequin head is imaged through a wire mesh (openings about half the spot size diameter, if I have understood correctly) that is very close to the mannequin. Ambient light noise rejection is well-ilustrated in a separate experiment.
The authors demonstrate a method for separation of signal and ambient light photons that could have widespread application where it is not precluded by the time needed to scan the setting of a programmable optical delay line (ODL). Though the submitted manuscript depends heavily on its reference [22] (a Scientific Reports paper from the same group), I recommend publication in Nature Communications; the application of QPMS to lidar is a significant achievement beyond what is described in [22]. The submitting group's Frontiers in Optics / Laser Science 2018 abstract "Noise tolerant LIDAR via mode selective up-conversion detection" describes QPMS applied to lidar, but that is merely a short abstract. The biggest weakness of the manuscript is misplaced emphases, some of which are important enough that revision should be considered necessary for acceptance.
In my opinion, the manuscript is fundamentally about achieving "signal" flux measurement with fine time resolution that, because of QPMS, is highly robust to high flux (signal or ambient light) at neighboring time windows. The fine time resolution is illustrated through the experiment of Figure 2; the robustness to high signal flux at nearby earlier time windows is illustrated through the experiment of Figure 3; and the robustness to high ambient light flux at all time windows is illustrated through the experiment of Figure 4. This basic message of the paper is not clear because of putting "... a few backscattered photons ..." in the opening sentence, and "With only 0.0006 detected signal photons per pulse, we reliably reconstruct the 3D profile of an obscured scene, ..." later in the abstract. The low number of detected signal photons per pulse is almost irrelevant to your manuscript, and it gives the impression that your contributions are about overall photon efficiency in lidar (which it is not). Furthermore, highlighting that you have a low number of detected signal photons per pulse is not consistent with the many mentions of dead time. Low incident flux makes dead time irrelevant, and a reader might naturally multiply 0.0006 times 34 and think that even with signal and ambient light together, the flux is too low for dead time effects to be significant. Furthermore, a reader might naturally look at the factor of 34 and think that this is not radically impressive compared to the results in [Joshua Rapp and Vivek K Goyal, "A Few Photons Among Many: Unmixing Signal and Noise for Photon-Efficient Active Imaging," IEEE Trans. Computational Imaging, vol. 3, no. 3, pp. 445-459, September 2017] (which, with apologies for advertising my own work, is quite relevant here). Actually, if I am understanding correctly, the factor of 34 that you highlight in the abstract is extremely impressive because it is the ambient light to signal ratio within a 1 ns window. (If you were to compute a signal-to-background ratio over a full 20 ns repetition period, would it be 1 to 680?) In my opinion, the emphasis of the paper should be that the combination of QPMS and sweeping of a programmable ODL provides capabilities far beyond what is possible with sweeping of the time gating of an ordinary single-photon detector (both because of dead time and the fact that the detected SBR is no higher than the incident SBR). However, taking an entire millisecond for each ODL delay value causes the data collection to be quite slow, and this should be made plain. I feel it takes a bit too much work for the reader to extract this.

Stylistic suggestions and minor corrections:
"maximum-likely-value" is a very uncommon term compared to "maximum likelihood". I find it a bit suboptimal to use Fig. 2(d) in reference to the to variance of time-of-flight measurements before the paragraph that describes the experiment that yields Figure 2. Fig. 2(c) is used even one paragraph earlier, but this is less jarring because Fig. 2(c) is easier to interpret without understanding the rest of Figure 2.
There is a dimensional mismatch when you write that there is a variance of 0.5 ps. Do you mean to write standard deviation, or is this a variance in ps^2?
Is there a reason for the mismatch between "The dwell-time for each pixel is 1 ms per ODL sample" and the 0.5 ms dwell-time in the previous paragraph? The manuscript "Noise tolerant singlephoton sensitve 3D imager" from Rehain et al. report on a QPMS based 3D Si-APD imager . Using quantum parametric mode sorting (QPMS), they can suppress the effect of background noise in single photon counting range imaging in photon starved and noise polluted environments. To realize QPMS, they use a pump and probe setup with a 50 MHz mode locked laser. The method is applied to different senarios and show remarkeble results (for instance, 3d imaging of an artifical had behind an aluminium mesh). I see no reasons to request any changes and support publication as submitted. Reply: Thanks. We appreciate the findings that "The manuscript is well written and the results are 22 impressive," and that "using QPMS as basis for 3D imaging is indeed a brilliant idea." Indeed, while we 23 have previously established the principles of QPMS, it is employed here, for the first time, to achieve 24 unmatched 3D imaging capabilities that herald many practical applications of importance. This could be a 25 crucial step of transitioning a theory to real-life utilities that can impact a breadth of fields. 26 The significance and novelty of the present results are also affirmed by Reviewer 2 and 3, who 27 commented "The method is applied to different scenarios and show remarkable results," and that 28 "Though the submitted manuscript depends heavily on its reference [22] (a Scientific Reports paper from 29 the same group), I recommend publication in Nature Communications; the application of QPMS to lidar 30 is a significant achievement beyond what is described in [22]." 31 We conceptualized QPMS upon our previous demonstrations of mode-resolving photon counting 32 (MSPC) [Optics letters 38, 468, Optics Express 22 (23), 27942], which followed the pioneering idea of 33 quantum pulse gating (QPG) but utilized optical arbitrary waveforms instead of relying on waveguide 34 engineering. Yet, both QPG and MSPC were targeted at quantum communications and computing, by 35 discriminatively detecting single photons in a certain time-frequency mode out of several well-defined 36 modes for information coding. In contrast, QPMS was proposed and demonstrated to address a distinct 37 challenge facing LiDAR applications: selecting photon signals in a desirable mode mixed with broadband 38 background noise that resides in numerous, random modes. 39 Experimental wise, previous QPG and MSPC demonstrations achieved selective photon conversion 40 over two or several modes created in controlled waveforms. In contrast, we utilized QPMS to select signal 41 photons over strong noise in lots of random modes (white noise) and scored a huge (~40 dB) advantage in 42 detection signal to noise ratio. 43 Thus, while our basic idea shares some similarities with QPG and MSPC, the current work concerns 44 a different class of applications, and is the first experimental demonstration of QPMS-enabled 3D 45 imaging, realizing unmatched noise rejection and resolution that promise unprecedented LiDAR and 46 remote sensing capabilities. The same advantages shall assist deep space communications, as proposed by 47 Banaszek and co-workers. Our experiment could be viewed a validation of their proposed idea, and we 48 are interested in further studies in this field. 49 Changes: In the introduction, we have added: "…following the pioneering studies of quantum pulse 50 gating [25] and quantum optical arbitrary waveform generation and measurement [26]." 51 In the summary, we have added "The same advantage may assist deep space communications [ Reply: For the 1-ns time gated InGaAs APD, the dark count level was measured to be 1000 59 counts/second, the timing resolution was measured to be 500 ps and the detector saturates at 10 5 60 counts/second. In contrast, for the current QPMS detector, the dark count level was measured to be 250 61 counts/second, the timing resolution to be 9 ps, and the detector saturates at 2*10 7 counts/second. iii) In Figure 4, the authors recover a smoothed image from the raw image by applying a median filter. 71 Maybe they could say a few more words about the parameters of the median filter? Also, it might help the 72 broader readership if the authors explicitly stated that this was a standard method for removing noise 73 from pictures in post processing.? 74 Reply: A median filter was applied to remove erroneous pixels from the reconstructed image by 75 converting each pixel to the median value of its eight nearest neighbors. These noisy pixels are 76 singularities that manifest as sharp spikes in the image. The continuous nature of the target scene and the 77 sparsity of the noisy pixels make this a suitable application for median filtering. 78 Changes: We added the following to the manuscript: "For post-processing, a standard MATLAB median 79 filter is applied to smooth the reconstructed image by converting every pixel value to the median of its 80 eight nearest neighbors. This commonly used image processing tool is preferred over mean filtering 81 because it is robust against bias from outier values." 82 iv) Where does the chirp in the pulses in Supplementary Figure 3 originate from? Is it mainly the 84 propagation through the fibres? Or is it from the laser? Or the shaping?? 85 Reply: It is due to the flat top in the spectral profile of the commercial dense wavelength division 86 multiplexers (DWDM-200 GHz) used to carve out probe and pump pulses from a mode-locked fiber laser. 87

Changes:
We have added this explanation in the caption of Fig. 3