## Abstract

Computational imaging makes it possible to reconstruct hidden objects through random media and around corners, which is of fundamental importance in various fields. Despite recent advances, computational imaging has not been studied in certain types of random scenarios, such as tortuous corridors filled with random media. We refer to this category of complex environment as a ’random corridor’, and propose a reduced spatial- and ensemble-speckle intensity correlation (RSESIC) method to image a moving object obscured by a random corridor. Experimental results show that the method can reconstruct the image of a centimeter-sized hidden object with a sub-millimeter resolution by a low-cost digital camera. The imaging capability depends on three system parameters and can be characterized by the correlation fidelity (CF). Furthermore, the RSESIC method is able to recover the image of objects even for a single pixel containing the contribution of about 10^{2} speckle grains, which overcomes the theoretical limitation of traditional speckle imaging methods. Last but not least, when the power attenuation of speckle intensity leads to serious deterioration of CF, the image of hidden objects can still be reconstructed by the corrected intensity correlation.

## Introduction

Relying on its ability to reconstruct an object hidden beyond the direct line of sight of a camera, computational imaging has recently attracted more and more interest^{1,2,3}. The potential application of computational imaging ranges from robotic vision, autonomous driving, disaster relief, to medical imaging^{1}. In the typical imaging scene widely studied previously, non-line-of-sight imaging always requires a relay surface located in the field of view of both the hidden object and the camera^{3}. Non-line-of-sight methods, such as backprojection^{4,5,6,7}, light-cone transform^{8}, phasor field^{9,10}, computational periscope^{11}, and spatial coherence analysis^{12,13}, are able to reconstruct the images of objects obscured by corners, by using the spatiotemporal information of diffuse light reflected from the relay surface. New techniques and technology greatly improve the efficiency, economy, minimum resolution, and maximum working distance of non-line-of-sight imaging^{8,9,11,14,15}. On the other hand, methods such as speckle correlations^{16,17}, wavefront-shaping^{18}, and speckle deconvolution^{19} are capable of computational imaging through random media and around corners. In spite of recent advances, challenging but important computational imaging in another type of complex environment, ‘random corridor’ has received little attention. Random corridor represents a kind of tortuous corridor filled with random media, which often exists in rescue scenes such as dusty mines, smoke-filled ventilation ducts, and building corridors^{20}. Imaging rescue targets obscured by a random corridor can greatly improve rescue efficiency and protect rescue crews.

In this paper, we propose a reduced spatial- and ensemble-speckle intensity correlation (RSESIC) method to image a moving object obscured by a random corridor. Experimental results show that this method can reconstruct a sub-millimeter resolution image of a centimeter-sized hidden object with a low-cost digital camera, whether the object is obscured by a random corridor with one or two corners. A correlation fidelity (CF) is defined to characterize the imaging capability of the system, and the dependence of the imaging capability on three system parameters: subspace side length, camera pixel count, and size ratio of pixel to speckle are investigated. If these parameters are inappropriately chosen, the CF will be poor and even unacceptable, resulting in computational imaging failure. The parameter range in which the RSESIC method can correctly image the hidden object is shown. The deterioration of CF caused by speckle intensity power attenuation is also studied. Based on the experimental and theoretical analysis, we demonstrate that the image of hidden objects can still be reconstructed after additional correlation renormalization.

## Results

### Imaging in a random corridor with one corner

Figure 1a shows the schematic for an experimental imaging system. The hidden object is out of the camera’s line of sight and obscured by a random corridor. The random corridor is L-shaped white plexiglass with one corner, as shown in the insets. The black dotted line represents an open boundary of the random corridor. The relay surface is not present for this scenario, and thus the laser incident on the hidden object can only propagate into the camera’s field of view through the random corridor. The lens between the laser and the camera is used to defocus the beam to illuminate the entire object. The hidden object is a metal plate with three equally spaced slits mounted on an optical mobile platform (see Fig. 1e) and moves in the *XY* plane. The position vector of the hidden object is denoted as **r**_{o}. The displacement vector of the hidden object is denoted as Δ** ρ**, where \(| {{\Delta }}{{{{{{{\boldsymbol{\rho }}}}}}}}| =\sqrt{{{\Delta }}{x}^{2}+{{\Delta }}{y}^{2}}\). Δ

*x*and Δ

*y*are the projections of Δ

**on the X and Y axes, respectively. A series of speckle intensity patterns that varies with Δ**

*ρ***is recorded by a digital camera (Sony A6300).**

*ρ*We denote the raw speckle intensity images as *I*_{s}(**r**_{c}; **r**_{o} + Δ** ρ**), where

**r**

_{c}is the position vector on the sensor of the camera. Figure 1b shows

*I*

_{s}(

**r**

_{c};

**r**

_{o}+ Δ

**) when a hidden object (see Fig. 1e) is located at**

*ρ***r**

_{o}+ Δ

**.**

*ρ**x*

_{c}and

*z*

_{c}are projections of

**r**

_{c}on

*X*and

*Z*axes, respectively. The photographic parameters and image processing method are detailed in the “Methods” section. The raw speckle intensity image shown in Fig. 1b demonstrates that the transmitted light field illuminating the hidden object is completely randomized when it propagates into the camera’s field of view, indicating that the hidden object cannot be directly imaged.

In order to decode the information carried by the speckle intensity images, a spatial speckle intensity correlation (SSIC) method was introduced in the past^{21,22}. The method uses spatial average cross-correlation of *I*_{s}(**r**_{c}; **r**_{o} + Δ** ρ**) that varies with Δ

**to approximate the transmitted field autocorrelation of hidden objects**

*ρ**C*

_{ho}(Δ

**). When the object is hidden behind a thick random medium, the SSIC method accurately recovers**

*ρ**C*

_{ho}(Δ

**). Therefore, the image of the hidden object can be successfully reconstructed with**

*ρ**C*

_{ho}(Δ

**) and an iterative phase recovery algorithm**

*ρ*^{23}. However, when the object is obscured by a random corridor, the SSIC method fails to recover

*C*

_{ho}(Δ

**) and reconstruct hidden object’s image (experimental proof is provided in Supplementary Note 4).**

*ρ*Thus, we propose a RSESIC method to deal with the challenge of computational imaging hidden objects obscured by a random corridor. The RSESIC is defined as

where *I*_{rd} is the subspace reduced speckle intensity, \({\langle ...\rangle }_{{{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{c}}}}}}}}}}\) indicates spatial average and \({\langle ...\rangle }_{{{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{o}}}}}}}}}}\) indicates ensemble average. Spatial average means the average over **r**_{c}. Ensemble average means the average over **r**_{o} (more information is provided in the “Methods” section). \({\overline{I}}_{{{{{{{{\rm{rd}}}}}}}}}={\langle {I}_{{{{{{{{\rm{rd}}}}}}}}}({{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{c}}}}}}}}})\rangle }_{{{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{c}}}}}}}}}}\) and \({\sigma }_{{{{{{{{\rm{rd}}}}}}}}}=\sqrt{{\langle {I}_{{{{{{{{\rm{rd}}}}}}}}}^{2}({{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{c}}}}}}}}})\rangle }_{{{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{c}}}}}}}}}}-{\langle {I}_{{{{{{{{\rm{rd}}}}}}}}}({{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{c}}}}}}}}})\rangle }_{{{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{c}}}}}}}}}}^{2}}\) denote the mean and standard deviation of *I*_{rd}, respectively.

*I*_{rd} is obtained through the subspace reduction process shown in Fig. 1b. First, the subspace standard deviation of each speckle intensity *I*_{s}(**r**_{c}; **r**_{o} + Δ** ρ**) is calculated using the following definition:

Here 〈. . . 〉_{sub} represents the average of the *I*_{s} in a square subspace area on the camera sensor. The square subspace is centered at a given position vector **r**_{c}. The side length of the square subspace is denoted as *l*_{sub}. For convenience, we further define a reduced subspace length \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}={l}_{{{{{{{{\rm{sub}}}}}}}}}/{l}_{{{{{{{{\rm{pixel}}}}}}}}}\), where *l*_{pixel} represents the pixel side length of a camera. Then, *I*_{s} at each position is reduced by its own standard deviation *σ*_{s} to calculate the *I*_{rd},

Before the subspace reduction, the standard deviations of *I*_{s} at different positions are not equal. The *σ*_{s}(**r**_{c}; **r**_{o} + Δ** ρ**) plotted in Fig. 1b shows that the standard deviation of

*I*

_{s}varies rapidly with spatial position. Therefore,

*I*

_{s}(

**r**

_{c};

**r**

_{o}+ Δ

**) does not conform to Rayleigh statistics as a whole. The role of the subspace reduction is to obtain a reduced speckle intensity distribution**

*ρ**I*

_{rd}(

**r**

_{c};

**r**

_{o}+ Δ

**) in which the standard deviation of light intensity at each point is almost equal. After the subspace reduction,**

*ρ**I*

_{rd}(

**r**

_{c};

**r**

_{o}+ Δ

**) agrees with the Rayleigh distribution as a whole. So the RSESIC method is insensitive to the spatial position of the observed speckle intensity pattern or the shape of random medium.**

*ρ*The common point between RSESIC method and previous SSIC method is that they are both speckle intensity cross-correlation methods. However, unlike the SSIC method, the RSESIC method calculates the cross-correlation of *I*_{rd} and combines spatial and ensemble average. Subspace reduction enables RSESIC to image objects obscured by a random corridor. Ensemble averaging enables RSESIC to see through thicker random media.

In a random corridor with static random media, *C*_{rd}(Δ** ρ**) is approximately equivalent to

*C*

_{ho}(Δ

**), which allows the RSESIC method to be used to image hidden objects (Supplementary Note 1). The**

*ρ**C*

_{rd}(Δ

**) for the hidden object in Fig. 1e is plotted, as shown in Fig. 1c, clearly revealing characteristic patterns. The resolution of the patterns is determined by the product of the moving speed of the hidden object ∣**

*ρ***v**∣ and the time interval of two photographs Δ

*τ*. As mentioned before, the image of the hidden object can be reconstructed with

*C*

_{rd}(Δ

**) and an iterative phase recovery algorithm**

*ρ*^{23,24}(more information is provided in the “Methods” section). The reconstructed image of the hidden object is plotted in Fig. 1d, which clearly demonstrates that the size and geometry of the hidden object are accurately restored. Similarly, Fig. 1f–h exhibit the imaging result of another hidden object, but with a complex shape. This further confirms the imaging competence of the RSESIC method for a random corridor with one corner.

For the RSESIC method, the consistency between *C*_{rd}(Δ** ρ**) and

*C*

_{ho}(Δ

**), namely CF, determines the fidelity of the reconstructed image. The CF of the RSESIC method is dependent on three system parameters including reduced subspace length \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\), camera pixel count**

*ρ**N*and size ratio of pixel to speckle

*R*. If these parameters are inappropriately chosen, the CF will be poor and even unacceptable, resulting in failure to image hidden objects. We explore the dependence of CF on \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\),

*R*and

*N*for the range of the parameters in which the RSESIC method can image the hidden object.

The size ratio of pixel to speckle on a camera is defined as

Here *d*_{speckle} is the average diameter of the speckle grain on the camera sensor. *R* determines the ability of the camera to resolve a single speckle grain. When *R* ≪ 1, the morphology of the speckle grains can be clearly photographed by the camera. However, when *R* ≥ 1, the contribution of multiple speckle grains will overlap in one pixel, so the morphology information of speckle grains will be lost completely. In our experiment, *l*_{pixel} ≈ 3.90 μm, *d*_{speckle} is the product of the average diameter of the speckle grain on the surface of the white plexiglass and the overall magnification of the camera zoom lens. Theory and simulation show that the average diameter of the speckle grain on the surface of the random media is equal to a half wavelength^{25,26}. The overall magnification of the camera zoom lens in the experiment is approximately equal to 4.31. Therefore, *d*_{speckle} ≈ 1.36 μm and the minimum value of *R* is 2.87.

We apply Eq. (1) to calculate *C*_{rd}(Δ*x*) for different parameters \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\), *R* and *N*. The hidden object is a hollow metal plate shown in Fig. 1e and moves along the *X* direction. Since the camera position and the focal length of the zoom lens (50 mm) are fixed, *R* is increased by merging the pixels of the speckle intensity image. For example, *R* increases from 2.87 to 8.61 when nine pixels are merged into one.

We first show the influence of \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\), *R* and *N* on the CF by directly comparing *C*_{rd}(Δ*x*) and *C*_{ho}(Δ*x*) in Fig. 2a, where *C*_{ho}(Δ*x*) is numerically calculated from the transmitted field of the hidden object (Supplementary Note 2). It can be found from Fig. 2a that the CF is proportional to *N*, and inversely proportional to \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\) and *R*. The decrease of *N* gives rise to random fluctuations in *C*_{rd}(Δ*x*), and the increase of \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\) or *R* causes positive background correlation in *C*_{rd}(Δ*x*). This suggests that the influence of \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\) and *R* on *C*_{rd}(Δ*x*) are similar, but the effect from *N* is different.

The Euclidean distance^{27} is introduced to quantify the deviation between *C*_{rd}(Δ** ρ**) and

*C*

_{ho}(Δ

**). Therefore, the CF of the RSESIC method**

*ρ**d*(

*C*

_{ho},

*C*

_{rd}) can be quantitatively calculated by the following definitions:

where ∥. . . ∥_{2} represents the Euclidean norm. When *C*_{rd}(Δ** ρ**) =

*C*

_{ho}(Δ

**),**

*ρ**d*(

*C*

_{ho},

*C*

_{rd}) = 1. Otherwise, 0 <

*d*(

*C*

_{ho},

*C*

_{rd}) < 1. The values of

*d*(

*C*

_{ho},

*C*

_{rd}) calculated by Eq. (5) are shown in the legend in Fig. 2a, which directly reflect the dependence of CF on \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\),

*R*and

*N*. Since the curves of

*C*

_{rd}(Δ

*x*) and

*C*

_{ho}(Δ

*x*) almost overlap for

*d*(

*C*

_{ho},

*C*

_{rd}) ≥ 0.91, it is reasonable to choose

*d*(

*C*

_{ho},

*C*

_{rd}) = 0.91 as a lower threshold of the validity of the RSESIC method.

The color maps in Fig. 2b shows *d*(*C*_{ho}, *C*_{rd}) for various *R* and *N* when \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\) is equal to 100 and 10, respectively, which further demonstrates the dependence of the CF on \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\), *R* and *N*. The region surrounded by a contour line *d*(*C*_{ho}, *C*_{rd}) = 0.91 and two coordinate axes indicates the parameter range in which the RSESIC method can accurately recover *C*_{ho}(Δ*x*) and reconstruct the image of hidden objects. By comparing the two color maps, it can be found that reduction in \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\) significantly enlarges the optimal imaging region of *R* and *N*. We also notice that the RSESIC method even holds for *R* = 15 when \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}=10\) and *N* = 10^{4}. This result breaks through the limit (*R* < 1) of previous speckle correlation methods^{16,21,22}, and provides insight into speckle-based imaging methods. In addition, when \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}=10\) and *R* < 15, *C*_{ho}(Δ*x*) can be accurately recovered with only *N* = 10^{3} pixels, indicating that the imaging method is valid with a low-pixel camera.

### Imaging in a random corridor with two corners

When an object is hidden behind several corners in a random corridor, the scattering of the corners and random media makes the light propagate from the hidden object through a very complex path to the observation point. In this case, it is extremely difficult to simulate the propagation of light. Besides, the power attenuation of the scattered light significantly enhances the influence of noise. Therefore, imaging objects hidden in such random corridors is very challenging. However, the RSESIC method does not need to solve the model for light propagation, and can eliminate the influence of ambient noise on the speckle intensity cross-correlation through extrapolation, thus resolving the barriers to imaging (Supplementary Note 3).

The schematic diagram to image objects hidden in a random corridor with two corners is shown in Fig. 3a. The hidden object is the metal plate with three slits, which moves in the *XY* plane. The illustration shows the side and front view of the random corridor, which is a Z-shaped metal tube filled with white plexiglass. The first function of the metal tube is to increase the transmittance of incident light in the random corridor, so that the speckle intensity pattern behind the two corners can be detected by the camera. The second function of the metal tube is to enhance the scattering of incident light by the random corridor wall, so as to test the imaging performance of the RSESIC method under the coupling interference of the corridor wall and random medium.

We make a comparison between *C*_{rd}(Δ*x*) and *C*_{ho}(Δ*x*) in Fig. 3b. *C*_{rd}(Δ*x*) is calculated from the measured images of speckle intensity through the RSESIC method. Due to the influence of fast time-dependent noise, *C*_{rd}(Δ*x*) sharply drops at small displacement and significantly deviates from *C*_{ho}(Δ*x*), causing CF to decrease to *d*(*C*_{ho}, *C*_{rd}) = 0.55. Therefore, *C*_{rd} cannot be directly used to reconstruct the image of the hidden object. However, the positions of peaks and valleys in *C*_{rd}(Δ*x*) and *C*_{ho}(Δ*x*) are identical, which implies that *C*_{rd} still contains the information about *C*_{ho}. Fortunately, it is found that the adverse effect of ambient noise on the *C*_{rd} can be eliminated by extrapolation and normalization, and then obtain a correlation \({\widetilde{C}}_{{{{{{{{\rm{rd}}}}}}}}}\) approximately equivalent to *C*_{ho}. Here *C*_{rd}(Δ*x*, Δ*y*) is normalized by *C*_{rd}(0, 0) which is corrected by using extrapolation. The consistency between \({\widetilde{C}}_{{{{{{{{\rm{rd}}}}}}}}}({{\Delta }}x)\) and *C*_{ho}(Δ*x*) demonstrates that the RSESIC method can correctly restore the transmitted field autocorrelation of the hidden object behind two corners (see Fig. 3b). Then with the normalized \({\widetilde{C}}_{{{{{{{{\rm{rd}}}}}}}}}({{\Delta }}x,{{\Delta }}y)\) in Fig. 3c, the image of the hidden object was reconstructed by phase-retrieval, as shown in Fig. 3d. By comparing Fig.1e with Fig. 3d, it is clearly seen that the size and geometric characteristics of the hidden object are accurately restored, which confirms the ability of the RSESIC method to image in a random corridor with two corners.

In addition, the RSESIC method is further examined by imaging an object ‘S’ (see Fig. 4d) obscured by a random corridor with two corners. The area of ‘S’ is approximately equal to 30.79 mm^{2}, which is much smaller than that of the three equally spaced slits shown in Fig. 1e, 60 mm^{2}. Due to stronger attenuation of speckle intensity power, imaging of ‘S’ should be more difficult. The imaging results demonstrate that the hidden object can still be imaged robustly, as shown in Fig. 4.

## Discussion

When an object is obscured by a random corridor, the difficulty of imaging the object is that the ballistic light is blocked and the relay surface is absent. We propose the RSESIC method to cope with this computational imaging challenge. Experimental results demonstrate that the method can reconstruct the image of the object obscured by a random corridor, using only speckle intensity images recorded by a digital camera. Applicable conditions for the method are summarized as follows: the random medium is static, the hidden object radiates or scatters coherent light, and motion information of the object must be known. The method is not limited to the transmitted speckle imaging system shown in this paper, but also applied to reflective laser speckle imaging systems and two-pass scattering systems (experimental proof is provided in Supplementary Notes 6 and 7).

Being different from most existing non-line-of-sight imaging methods, the RSESIC method can image hidden objects behind multiple corners without the spatiotemporal information of diffuse light reflected from a relay surface^{3,8,9}, and the reconstructed image is independent of the measurement position of multiple scattering light. Therefore, the method is complementary to the existing non-line-of-sight imaging methods and will promote further research on computational imaging in more complex scenes. In addition, the RSESIC method restores the image from the intensity cross-correlation of multiple scattering light rather than from solving a light propagation model, so the computation and complexity of the method are independent of the thickness of the random media and the number of corners. The ability of the RSESIC method to image objects behind multiple corners makes it promising to be applied to many fields such as disaster relief and automatic driving. If the moving distance of the hidden object and the measurement time of the camera are large enough, the RSESIC method can image a hidden object of any size. The actual imaging resolution of the method is determined by the product of the moving speed ∣**v**∣ of the hidden object and the time interval of two photographs Δ*τ*.

We analyze the effects of \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\), *R* and *N* on the CF of the RSESIC method and use a color map to point out an applicable parameter range for the method. The optimal combination of these three parameters can ensure the CF of the RSESIC method. For instance, with appropriate \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\) and *N*, the RSESIC method can accurately recover *C*_{ho} and the image of the hidden object even for *R* = 15. This finding breaks through the limit (*R* < 1) of previous speckle correlation methods and proves that the speckle correlation method can image hidden objects when the speckle grains are not well resolved by a camera. Similarly, if \({\hat{l}}_{{{{{{{{\rm{sub}}}}}}}}}\) and *R* are appropriate, an average of *N* = 10^{3} camera pixels is enough to make *C*_{rd} and *C*_{ho} match well. This result indicates that RSESIC method is suitable for low pixel cameras and has low requirements for storage and computing.

When the transmitted light of the hidden object is strongly attenuated during the propagation, ambient noise causes the *C*_{rd} calculated by the RSESIC method to deviate significantly from *C*_{ho}. Therefore, the CF sharply deteriorates, resulting in imaging failure of hidden objects. Based on the experimental and theoretical analysis, we find that the mixture of correlation characteristic of fast time-varying ambient noise and the correlation of transmitted field of hidden objects leads to an abrupt decline in *C*_{rd}. We show that using an extrapolation method, the influence of noise in *C*_{rd} can be reduced, and then the image of hidden objects can be reconstructed. This confirms the adaptability of the RSESIC method in a noisy environment. We further confirm that the adaptability of the RSESIC method in a noisy environment makes it possible to see through a random medium thicker than that in the existing method (experimental proof is provided in Supplementary Note 5).

As a final note, the application of the RSESIC method is not only limited to imaging by laser, but also applicable for other types of coherent waves, such as acoustic waves, quantum waves and electromagnetic waves with multiple frequencies. Therefore, computational imaging with similar difficulties in other fields are expected to be addressed.

## Methods

### Experimental setup

The experimental system consists of a 632.991 nm laser source (Thorlabs, HRS015B), a lens, a hidden object, a random corridor and a digital camera (Sony A6300). The zoom lens of the digital camera is set to a focal length of 50 mm. The distance between the zoom lens and random media is 1 mm. A speckle intensity pattern in an ~5.45 mm × 3.64 mm region of the white plexiglass surface is imaged on the 14-bit CMOS sensor of the camera. For imaging objects hidden behind one (two) corner(s), the shooting parameters of the camera are set to exposure time = 1/30 (1/10) s, aperture factor = 5.6, and ISO = 500 (6400), respectively.

### Image processing

The photographs in RAW format were exported from the camera and further processed. Each photographic file is a 14-bit, 6024 × 4024-pixels raw image of which the color channels interleave according to the RGBG mode of a Bayer filter. Since the light emitted by the He–Ne laser is red, the data in the red channel of RAW files are extracted and used as the actual images of the speckle intensity. The size of speckle intensity image is 3012 × 2012-pixels.

### Spatial- and ensemble-speckle intensity correlation

In Eq. (1), \({\langle ...\rangle }_{{{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{c}}}}}}}}}}\) is the average over the pixels of recorded speckle intensity images. Each speckle intensity image contains 3012 × 2012 intensity values, so the maximum spatial average times is 6,060,144 which is much larger than that required by the RSESIC method. From the definition of Eq. (1), it can be explicitly seen that the cross-correlation of speckle intensity reflects the similarity between a series of evolutionary speckle intensity images *I*_{s}(**r**_{c}; **r**_{o} + Δ** ρ**) for a moving hidden object and the reference speckle intensity image

*I*

_{s}(

**r**

_{c};

**r**

_{o}). When Δ

**is given,**

*ρ**m*reference images of speckle intensity can be obtained by changing

**r**

_{o}, and then

*m*correlation values can be calculated. \({\langle ...\rangle }_{{{{{{{{{\bf{r}}}}}}}}}_{{{{{{{{\rm{o}}}}}}}}}}\) is the average of the

*m*correlation values. The intensity cross-correlations shown in Figs. 1, 3 and 4 are all calculated by 10

^{4}spatial averages and 9 ensemble average. That is to say, the image of the hidden object is reconstructed from the data of speckle intensity measured on an area of 100 × 100-pixels of the CMOS sensor.

### Image reconstruction of hidden objects

Autocorrelation of the transmitted field of a hidden object is defined as (Supplementary Note 1),

Here *E*_{ho} denotes the transmitted field of a hidden object, \({\hat{E}}_{{{{{{{{\rm{ho}}}}}}}}}\) denotes the reduced dimensionless transmitted field, which is defined as,

By the approximate relation *C*_{rd}(Δ** ρ**) ≈

*C*

_{ho}(Δ

**), we have**

*ρ*The Fourier amplitude \(| {\hat{E}}_{{{{{{{{\rm{ho}}}}}}}}}({k}_{{{{x}}}},{k}_{{{{y}}}})|\) of the reduced dimensionless transmitted field of a hidden object can be directly achieved. With substituting \(| {\hat{E}}_{{{{{{{{\rm{ho}}}}}}}}}({k}_{{{{x}}}},{k}_{{{{y}}}})|\) and appropriate real-space support domain into the iterative phase recovery algorithm, \({\hat{E}}_{{{{{{{{\rm{ho}}}}}}}}}(x,y)\) is reconstructed. Finally, \(| {\hat{E}}_{{{{{{{{\rm{ho}}}}}}}}}(x,y)|\) is the image of hidden object.

## Data availability

The data that support the findings of this study are available from the authors upon request.

## Code availability

The code that support the findings of this study are available from the authors upon request.

## References

Freund, I. Looking through walls and around corners.

*Physica A.***168**, 49–65 (1990).Altmann, Y. et al. Quantum-inspired computational imaging.

*Science***361**, eaat2298 (2018).Faccio, D., Velten, A. & Wetzstein, G. Non-line-of-sight imaging.

*Nat. Rev. Phys.***2**, 318–327 (2020).Gupta, O., Willwacher, T., Velten, A., Veeraraghavan, A. & Raskar, R. Reconstruction of hidden 3d shapes using diffuse reflections.

*Opt. Express***20**, 19096–19108 (2012).Velten, A. et al. Recovering three-dimensional shape around a corner using ultrafast time-of-flight imaging.

*Nat. Commun.***3**, 745–753 (2012).Buttafava, M., Zeman, J., Tosi, A., Eliceiri, K. & Velten, A. Non-line-of-sight imaging using a time-gated single photon avalanche diode.

*Opt. Express***23**, 20997–21011 (2015).Musarra, G. et al. Non-line-of-sight three-dimensional imaging with a single-pixel camera.

*Phys. Rev. Appl.***12**, 011002 (2019).O’Toole, M., Lindell, D. B. & Wetzstein, G. Confocal non-line-of-sight imaging based on the light-cone transform.

*Nature***555**, 338 (2018).Liu, X. et al. Non-line-of-sight imaging using phasor-field virtual waveoptics.

*Nature***572**, 620–623 (2019).Liu, X., Bauer, S. & Velten, A. Phasor field diffraction based reconstruction for fast non-line-of-sight imaging systems.

*Nat. Commun.***11**, 1645 (2020).Saunders, C., Murray-Bruce, J. & Goyal, V. K. Computational periscopy with an ordinary digital camera.

*Nature***565**, 472–475 (2019).Batarseh, M. et al. Passive sensing around the corner using spatial coherence.

*Nat. Commun.***9**, 3629 (2018).Starshynov, I., Ghafur, O., Fitches, J. & Faccio, D. Coherent control of light for non-line-of-sight imaging.

*Phys. Rev. Appl.***12**, 064045 (2019).Wang, B. et al. Non-line-of-sight imaging with picosecond temporal resolution.

*Phys. Rev. Lett.***127**, 053602 (2021).Wu, C. et al. Non-line-of-sight imaging over 1.43 km.

*Proc. Natl Acad. Sci. USA***118**, e2024468118 (2021).Katz, O., Heidmann, P., Fink, M. & Gigan, S. Non-invasive single-shot imaging through scattering layers and around corners via speckle correlations.

*Nat. Photonics***8**, 784–790 (2014).Shawn, D., Dennis, F. G. & Abbie, T. W. Imaging around corners in the mid-infrared using speckle correlations.

*Opt. Express***28**, 11051–11064 (2020).Katz, O., Small, E. & Silberberg, Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light.

*Nat. Photonics***6**, 549–553 (2012).Xu, X. et al. Imaging objects through scattering layers and around corners by retrieval of the scattered point spread function.

*Opt. Express***25**, 32829–32840 (2017).Waugh, W. L.

*Living with Hazards, Dealing with Disasters: An Introduction to Emergency Management*1st edn (Routledge, 1998).Newman, J. A. & Webb, K. J. Imaging optical fields through heavily scattering media.

*Phys. Rev. Lett.***113**, 263903 (2014).Webb, K. J. & Luo, Q. Theory of speckle intensity correlations over object position in a heavily scattering random medium.

*Phys. Rev. A***101**, 063827 (2020).Fienup, J. R. Reconstruction of a complex-valued object from the modulus of its Fourier transform using a support constraint.

*J. Opt. Soc. Am. A***4**, 118–123 (1987).Meihua, L. et al. Improving reconstruction of speckle correlation imaging by using a modified phase retrieval algorithm with the number of nonzero-pixels constraint.

*Appl. Opt.***58**, 473–478 (2019).Carminati, R. Subwavelength spatial correlations in near-field speckle patterns.

*Phys. Rev. A***81**, 053804 (2010).Shi, T., Li, Y., Shi, Q., Li, L. & Zheng, N. The Kullback–Leibler divergence evolution of randomized electromagnetic field.

*Opt. Commun.*126411 (2020).Michel, M. D. & Elena, D.

*Encyclopedia of Distances*4th edn (Springer-Verlag, Berlin, Heidelberg, 2016)

## Acknowledgements

We appreciate J.P. Johnson for a critical reading of this paper. This work was supported by the National Natural Science Foundation of China (NSF) (11974044) and National Key Research and Development Program of China (2016YFC1401001).

## Author information

### Authors and Affiliations

### Contributions

L.L., N.Z., and Q.S. conceived the idea, T.S. and L.L. designed the experiments and performed numerical simulations. T.S., H.C., and X.Z. conducted the experiments and analyzed the data. T.S. and N.Z. wrote the paper.

### Corresponding authors

## Ethics declarations

### Competing interests

The authors declare no competing interests.

## Additional information

**Publisher’s note** Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Supplementary information

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Shi, T., Li, L., Cai, H. *et al.* Computational imaging of moving objects obscured by a random corridor via speckle correlations.
*Nat Commun* **13**, 4081 (2022). https://doi.org/10.1038/s41467-022-31669-7

Received:

Accepted:

Published:

DOI: https://doi.org/10.1038/s41467-022-31669-7

## Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.