Abstract
Imaging through scattering media is still a formidable challenge with widespread applications ranging from biomedical imaging to remote sensing. Recent research progresses provide several feasible solutions, which are hampered by limited complexity of targets, invasiveness of data collection process and lack of robustness for reconstruction. In this paper, we show that the complex tobeobserved targets can be noninvasively reconstructed with fine details. Training targets, which can be directly reconstructed by speckle correlation and phase retrieval, are utilized as the input of the proposed speckle pattern estimation model, in which speckle modeling and constrained least square optimization are applied to estimate the distribution of the speckle pattern. Reconstructions for tobeobserved targets are realized by deconvoluting the estimated speckle pattern from the acquired integrated intensity matrices (IIMs). The qualities of reconstructed results are ensured by the stable statistical property and memory effect of laser speckle patterns. Experimental results show that the proposed method can reconstruct complex targets in high quality and the reconstruction performance is robust even much less data are acquired.
Introduction
Light scattering, caused by inhomogeneous refractive index presented in the transmission media, limits the resolution and the signaltonoise ratio of optical imaging^{1}. Random refraction in the scattering media encodes the spatial information of imaging targets into scrambled images and causes the speckle noise. Ballistic light based methods, such as optical coherent tomography^{2,3,4} and twophoton microscopy^{5,6}, extend the imaging depth by extracting photons that have not been scattered. Adaptive optics^{7} can correct the loworder deformations using deformable mirrors. However, when the scattering media diffuses nearly all the light going through it^{1,8}, the approaches mentioned above will be out of action. Target localization inside the scattering media can be realized by diffuse optical tomography^{9,10}, whose imaging resolution is limited by scattering depth. Some recent works^{11,12,13,14,15,16,17,18,19,20} based on memory effect^{21,22,23} provide novel imaging solutions, including wavefrontshaping^{11,12,13,14}, speckle correlation^{15,16,17}, bispectrum analysis^{18} and deconvolution^{19,20}, for strongly scattering situations. Controlled wavefrontshaping allows imaging and focusing through scattering media by modulating the degrees of freedom in the scattered waves^{24,25,26,27,28,29,30,31,32,33,34,35,36,37}. Furthermore, uptodate ‘guidestar’ mechanisms^{38,39,40,41,42} provide noninvasive feedback for intratissue focusing. Nevertheless, the imaging resolution is limited by the size of illumination speckle grain, and the field of view (FoV) for high lighttransmission enhancement is limited in the vicinity of the focusing spot. Speckle correlation based approaches, which include speckle scanning^{15,16} and singleshot^{17} techniques, collect the convolution of the target and the pointspread speckle pattern^{43,44}. The autocorrelation computation and Fourier transform are utilized to get the amplitude spectrum of targets, and phaseretrieval algorithms^{45,46} to realize reconstruction. However, the methods require highresolution angular sampling to weaken the effect caused by speckle noise. Also, the reconstruction quality is limited by the size of the illumination speckle grain, and the stability of phaseretrieval algorithm is restricted by the structural complexity of imaging targets. Bispectrum analysis based method^{18}, which can deterministically and unambiguously extract the Fourier phase of the target from scattered light pattern, also suffers from the problems caused by the stochastic perturbation of speckle pattern. Deconvolution based approaches can realize imaging with resolution beyond the diffraction limitation, while the main problem is how to acquire the pointspread speckle pattern. Pointspreadfunction (PSF) capturing based deconvolution method^{19,20} needs to put a point source behind the scattering media during the estimation process, whose invasiveness impedes the practicability for many real applications.
To noninvasively image complex targets through strongly scattering media with resolution beyond the limitation of illumination speckle grain lighted on the targets, a preliminary speckle pattern estimation and deconvolution method was proposed in our previous work^{47}. The method is extended in this paper by adding crosscorrelation matching method for restoring orientations and positions, deriving theoretical model for the speckle pattern estimation, analyzing the solution space and data reduction and comparing experimental results for imaging targets with different sizes and structural complexities. Imaging system based on speckle correlation^{15} is utilized to collect IIMs of the tobeobserved targets. Exploiting the frequencybounded characteristics of the speckle pattern^{48} and the convolutional relationship between the imaging target and the speckle pattern, speckle estimation based on the constrained least square optimization and speckle modeling is proposed to estimate the speckle pattern distribution. IIMs are aligned by the orientational and positional information recovered from the crosscorrelation matching method. The complexity in solving the speckle pattern estimation model and the quality of estimation using the training targets in different shapes with different combinations are discussed in detail. Deconvolution is performed to reconstruct targets using their corresponding IIMs and the estimated speckle pattern. Reconstruction using much less acquired IIM data is also analyzed to demonstrate the feasibility in reducing the data collection complexity. Compared with speckle correlation^{15,16,17} and bispectrum analysis^{18} based approaches, in which the stochastic perturbations in pointspread speckle pattern are considered as statistic noises, our method directly estimates the speckle pattern and reconstruct the target by deconvolution algorithm. The demand of highresolution angular sampling is relieved and it can recover highquality details of complex targets with directional information. Experimental results and discussions on realcaptured data are provided to verify the effectiveness of the proposed method. Compared with the existing approaches, the proposed method provides much higher reconstruction quality to much more complex tobeobserved targets with much larger FoV.
Principles and Methods
Experimental setup and denotations
The schematic diagram of data collection process is illustrated in Fig. 1. In the enclosed environment surrounded by the scattering media, several imaging targets with different structural complexities may appear in the imaging region. The targets that can be reconstructed by phaseretrieval algorithm, namely training targets, are denoted by \({O}_{s}^{i}(i=1,2,\cdots ,N)\). And the tobeobserved targets are denoted by O_{c}. Imaging system based on raster scanning and speckle correlation, which was first proposed by Bertolotti et al.^{15}, is utilized to collect the IIMs for the imaging targets. When imaging target is in the common imaging region behind the scattering media, the laser beam with incident angle of \(({\theta }_{x},{\theta }_{y})\) will generate speckle pattern lighting on the target through the scattering media. In case of the existence of memory effect^{21,22,23}, the pointspread speckle pattern can be approximately expressed by \(S(x{\theta }_{x}{d}_{1},y{\theta }_{y}{d}_{1})\), where d_{1} is the distance between the imaging target and the scattering media. The intensity of the speckle pattern transmitted and diffused by the scattering media, which encodes the information of the imaging target, is recorded by the photon detector. Imaging target can be generally denoted by \(O(x{r}^{(1)},y{r}^{(2)})\), where (x, y) is the coordinate on the object plane and \(({r}^{(1)},{r}^{(2)})\) is its spatial position relative to the center of the FoV. The recorded integrated intensity corresponding to a specific incident angle of laser beam, \(({\theta }_{x},{\theta }_{y})\), can be expressed as
where * denotes the convolution operator and IIM is the integrated intensity matrix. The pixel intensity of IIM corresponds to the total intensity of the image captured at a laser incident angle. The sample IIMs collected for \({O}_{s}^{1}\), \({O}_{c}\) and \({O}_{s}^{N}\) in Fig. 1(a–c) are shown in Fig. 1(d–f), respectively.
Speckle pattern estimation
From Eq. (1), it can be observed that once the pointspread speckle pattern that scans the imaging region is known, theoretically, the target can be reconstructed by deconvoluting S from IIM by^{47}
where \( {\mathcal F} \) and \({ {\mathcal F} }^{1}\) represent Fourier transformation and inverse Fourier transformation, respectively. However, in general, the pointspread speckle pattern is unknown, which can only be estimated.
To estimate the speckle pattern, the convolutional relationships among the speckle pattern, \(S\), training targets, \({O}_{s}^{i}(i=1,2,\cdots ,N)\), and their IIMs, \(II{M}_{s}^{i}(i=1,2,\cdots ,N)\), are exploited. Denoting the phaseretrieved results of training targets as \({\bar{O}}_{s}^{i}(i=1,2,\cdots ,N)\), the convolutional relationship defined in Eq. (1) can be converted to be
where \({e}_{i}\) represents the error caused by imaging noise and phaseretrieval algorithm. \({\bar{O}}_{s}^{i}\in {R}^{{M}_{x}\times {M}_{y}}\), \(II{M}_{s}^{i}\in {R}^{{K}_{x}\times {K}_{y}}\), \(S\in {R}^{({M}_{x}+{K}_{x}1)\times ({M}_{y}+{K}_{y}1)}\) and \({e}_{i}\in {R}^{{K}_{x}\times {K}_{y}}\), where \({M}_{x}\times {M}_{y}\) is the dimension of the imaging target and \({K}_{x}\times {K}_{y}\) is the number of sampling points in \(II{M}_{s}^{i}\). It is a linear system with \(({M}_{x}+{K}_{x}1)({M}_{y}+{K}_{y}1)\) variables and \(N{K}_{x}{K}_{y}\) equations. Solving s in Eq. (3) is an underdetermined problem and \({e}_{i}\) is also unknown. Considering the intensity distribution of speckle pattern is always positive and it presents frequencybounded characteristics inherited from Fourier transformation of the laser beam wavefront^{48}, the prior of speckle pattern distribution is introduced to build a constrained least square model for speckle estimation as^{47}
where \({\rm{\Delta }}\) constrains the variation between the intensity of two adjacent points in the speckle pattern. \({\rm{\Delta }}\) is inversely proportional to the size of the illumination speckle grain, and the speckle grain size is proportional to d_{1} and wavelength of the laser while inversely proportional to the diameter of laser beam^{48}. The speckle estimation model in Eq. (4) is a convex problem^{49}, which can be solved by the solvers of convex optimization problem (CVX^{50,51} is used in our experiments).
Phase retrieval and crosscorrelation matching for reconstruction of training targets
The phaseretrieval reconstruction of training target, denoted by \({\overline{O}}_{s}^{i}\), is used as the input of Eq. (4). To reconstruct \({\overline{O}}_{s}^{i}\), the sharply peaked property of the average autocorrelation of speckle pattern^{52} is exploited first. Through autocorrelation of IIM and plugging Eq. (1) into it, the Fourier magnitude spectrum of \({O}_{s}^{i}\) is given by^{15}
where * denotes the correlation operator. Then, phaseretrieval algorithm, Hybrid InputOutput and ErrorReduction^{15,45,46}, is applied to recover \({\overline{O}}_{s}^{i}\) via iteratively updating the phases with fixed magnitude spectrum.
After phase retrieval, the orientational and positional information of \({\overline{O}}_{s}^{i}\) is lost. Also, the training targets may not be spatially aligned in the imaging region. It leads to the spatial disconformity between the captured IIMs with the convolutions of \({\overline{O}}_{s}^{i}\) and S. Directly putting \({\overline{O}}_{s}^{i}\) and the captured \(II{M}_{s}^{i}\) into Eq. (4) will result in mismatches and cannot estimate the local spatial distributions of speckle pattern accurately. Thus, crosscorrelation matching is proposed to correct the orientations of \({\overline{O}}_{s}^{i}\) and to align \(II{M}_{s}^{i}\) in Eq. (4).
According to Eq. (1), the IIM of \({O}_{s}^{i}\) can be expressed as
where \(({r}_{i}^{(1)},{r}_{i}^{(2)})\) is the spatial position of \({O}_{s}^{i}\) relative to the center of FoV. The autocorrelation of speckle pattern is a sharply peaked function, which can be expressed by^{48}
where \(\delta \) denotes impulse function; B and C are the constants depending on the size of speckle grain. Applying crosscorrelation to the IIMs of two training targets \({O}_{s}^{i}\) and \({O}_{s}^{j}\), we have
For phaseretrieved result of \({O}_{s}^{i}\), i.e. \({\overline{O}}_{s}^{i}\), there are two possible orientations: the orientation of \({\overline{O}}_{s}^{i}\) is the same with that of \({O}_{s}^{i}\) as
and \({\overline{O}}_{s}^{i}\) is the flipped of \({O}_{s}^{i}\) as
Thus, by comparing the four possible orientation combinations of \({\overline{O}}_{s}^{i}\) and \({\overline{O}}_{s}^{j}\), the orientation which maximizes the correlation between the correlation of rotated phaseretrieved results and the correlation of IIMs is achieved by
As a result, the reconstructed training targets with correct orientations are obtained by
Since \({\ddot{O}}_{s}^{i}\) is an approximation of \({O}_{s}^{i}\) with some reconstruction errors, the spatial relative offsets between \({O}_{s}^{i}\) and \({O}_{s}^{j}\) can be estimated by
Then, \(II{M}_{s}^{i}(i=2,\cdots ,N)\) is aligned to \(II{M}_{s}^{1}\) by
Finally, the speckle estimation model in Eq. (4) is updated to be
which uses aligned IIMs and orientationcorrected phaseretrieved results to provide a theoretical optimization solution matching the original distribution of S.
Model Analysis
Training targets analysis
The speckle estimation performance of Eq. (15) is decided by the shape, size and the number of training targets. The increase in the number of training targets, the larger shape differences among training targets and the size reduction of each target will result in better estimation performance. However, the estimation qualities of entries in matrix S are not the same. The objective function of the estimation model, Eq. (15), is the least square solution of linear equations,
in which the entries of S in the central are included in more independent equations than those in the margins. Thus, the central entries have smaller solution spaces and higher probabilities to be solved with lower errors. Eq. (16) can also be discretely formed by
where
in which \(({\rm{\Delta }}{\theta }_{x},{\rm{\Delta }}{\theta }_{y})\) is the sampling interval of IIMs and the values of undefined elements in matrix A are zeros. The increase in the number of training targets with different shapes leads to the increase in the number of variables, denoted by \(NoV(A\chi =b)\), and the increase in the rank of A, denoted by \(rank(A)\). \(NoV(A\chi =b)\) equals to the number of nonzero columns in matrix A. Simulations in Fig. 2 compare the speckle pattern estimation performance using the training targets in different shapes with different combinations. Four training targets in size of 60 × 60 \(({M}_{x},{M}_{y}=60)\), as shown in Fig. 2(a–d), are selected. Speckle patterns at object plane are generated by simulating the wave propagation of random phased laser beam with Fresnel diffraction equation^{43}. Raster scanning process (corresponds to changing the incident angle of laser beam with the memory effect^{21,22,23}) is simulated to acquire the IIMs in size of 60 × 60 \(({K}_{x},{K}_{y}=60)\). To get rid of the reconstruction error caused by phaseretrieval algorithm, original training targets are utilized to estimate the speckle patterns using the proposed model in Eq. (15). Cropping the estimated speckle patterns in size of 60 × 60 in the center, i.e. the region lined in white, the peak signal to noise ratio (PSNR) is calculated using the simulated ground truth, as an example shown in Fig. 2(e), as the reference. 100 simulated speckle patterns are tested to average the PSNR for a robust result. As we can see from the results, the estimation quality is positively related to the number of the training targets used. For example, the estimation quality using two training targets is much higher than only using one training target, while much lower than using three or four training targets. Also, it can be found that the estimation quality is mainly determined by the rankvariable ratio, i.e. \(rank(A)/NoV(A\chi =b)\). When the number of training targets is increased, the increase in the number of independent equations, i.e. rank(A), is larger than that in the number of variables, i.e. \(NoV(A\chi =b)\). And it results in the increase of rankvariable ratio. Since the higher rankvariable ratio corresponds to the less underdetermined variables, the estimation quality can be improved obviously. The variation in the shape of the training targets also results in the change of the rankvariable ratio. If the targets with higher structural complexities can be reconstructed well by phase retrieval, the bigger structural differences among the training targets result in the higher rank(A), which benefits the speckle pattern estimation.
Reconstruction with less data
After getting the estimated speckle pattern \(\ddot{S}\) from Eq. (15), deconvolution can be executed for the acquired IIM to reconstruct an imaging target with much higher accuracy and robustness than the phaseretrieval methods. It is also observed that we can further reduce the data acquisition complexity for acceptable reconstruction quality. Considering that the collection of IIM by raster scanning and memory effect as proposed by Bertolotti et al.^{15} is a discrete sampling process, the discrete representation of IIM for a tobeobserved target can be expressed by updating Eq. (1) to
where \({\overline{IIM}}_{c}\), \({\overline{O}}_{c}\) and \(\overline{S}\) are the discrete sampling signals of \(II{M}_{c}({\theta }_{x},{\theta }_{y})\), \({O}_{c}(x{r}_{c}^{(1)},y{r}_{c}^{(2)})\) and \(S(x,y)\) with the sampling intervals as \(({\rm{\Delta }}{\theta }_{x},{\rm{\Delta }}{\theta }_{y})\), \(({\rm{\Delta }}{\theta }_{x}{d}_{1},{\rm{\Delta }}{\theta }_{y}{d}_{1})\) and \(({\rm{\Delta }}{\theta }_{x}{d}_{1},{\rm{\Delta }}{\theta }_{y}{d}_{1})\), respectively; \(({r}_{c}^{(1)},{r}_{c}^{(2)})\) is the position of the tobeobserved target.
As the sampling interval becomes to be \(\{(H{\rm{\Delta }}{\theta }_{x},H{\rm{\Delta }}{\theta }_{y})H\in {Z}^{+},H > 1\}\), corresponding to reducing the complexity in data acquisition by \({H}^{2}\) times, the collected IIM becomes to be
Thus, using the downsampled estimated speckle pattern, denoted by \(\overline{\ddot{S}}\), it is still possible to reconstruct the imaging target in lower resolution by deconvoluting \({\overline{IIM}}_{c}\) with \(\overline{\ddot{S}}\).
Results and Discussion
In this section, the performance of the proposed speckle pattern estimation model is demonstrated by testing on a real imaging system, in which two Edmund Optics 120grit groundglass diffusers^{15} are used as the scattering media. In this system, speckle scanning and recovery method proposed by Bertolotti et al.^{15} is used. The laser beam is controlled by two galvanometric scanners in vertical and horizontal directions separately to generate different incident angles at the scattering media. The details of the imaging system can be found in the electronic supplementary material. To reduce the noises and artifacts, RichardsonLucy deconvolution algorithm^{53,54} is applied for reconstruction.
Four training targets, as \({O}_{s}^{1},\cdots ,{O}_{s}^{4}\) shown in Fig. 3(a), are used in the experiments. Their sizes range from 20 microns to 25 microns. The distances between the targets and scattering media, \({d}_{1}\) in Eq. (1), are set as 5 millimeters. Scanning angles are divided evenly into 600 intervals, ranging from −\(2.1^\circ \) to \(2.1^\circ \) in both horizontal and vertical directions. Thus, the corresponding IIMs collected are shown in Fig. 3(b) as \(II{M}_{s}^{1},\cdots ,II{M}_{s}^{4}\). The reconstructed results of the training targets are shown in Fig. 3(c) as \({\overline{O}}_{s}^{1},\cdots ,{\overline{O}}_{s}^{4}\) via autocorrelation, Fourier transformation and phaseretrieval algorithm. Using the proposed crosscorrelation matching method, the orientation corrected reconstruction results of the training targets are shown in Fig. 3(d) as \({\ddot{O}}_{s}^{1},\cdots ,{\ddot{O}}_{s}^{4}\) and the spatially aligned IIMs are corrupted by the white line in Fig. 3(b) as \(I\dot{I}{\dot{M}}_{s}^{1},\cdots ,I\dot{I}{\dot{M}}_{s}^{4}\). Then, Eq. (15) is executed to estimate the speckle pattern by setting \({\rm{\Delta }}\) to be 0.3 and binarizing \({\ddot{O}}_{s}^{1},\cdots ,{\ddot{O}}_{s}^{4}\) to be those in Fig. 3(e) to reduce the computational complexity of solving Eq. (15). Figure 3(f) and (g) show the estimated speckle patterns using only two training targets and four training targets respectively. The ground truth of the speckle pattern is shown in Fig. 3(h). Comparing Fig. 3(f) and 3(g) with the ground truth, it can be found that both the two estimated speckle patterns are visually similar to the ground truth especially considering the lowfrequency features. The result estimated using four training targets preserves more highfrequency components, which may lead to reconstructing more details of the imaging targets by deconvolution.
Applying the estimated speckle pattern to the deconvolution process, the tobeobserved targets with complex structures can be reconstructed. The reconstruction quality is compared with those generated by phase retrieval^{15,45,46} to demonstrate the efficiency of the proposed method. As shown in Fig. 4(a), eight complex targets with different textural complexities and different number of subsections are tested. Their sizes range from 40 microns to 165 microns. IIMs of these targets are collected in the same way with the training targets. The reconstructed results are compared visually with the original images. Also, structural similarity index (SSIM)^{55} is calculated between the reconstructed results and the original images to evaluate the structural reconstruction capability objectively. SSIM closer to 1 corresponds to a higher reconstruction quality.
Figure 4(b) shows the reconstructed results using phase retrieval, each of which is the best result selected from performing 20 times of phase retrieval using random initial phases. It can be found that the quality of the results deteriorates sharply when targets become complex or have separated subsections. Also, because of the randomness of initial phases, the positions and shapes of the reconstructed results change a lot for each test.
Figure 4(c,d) show the reconstructed results generated by deconvoluting estimated speckle pattern in Fig. 3(f) and that in Fig. 3(g), respectively. It is obvious to see that compared with the results generated by phase retrieval, the proposed method improves the reconstruction performance for all the targets significantly, which is also robust to the structure of the targets. Although the reconstructed results derived by deconvoluting speckle pattern estimated from two training targets lose some highfrequency details, as shown in Fig. 4(c), their reconstruction qualities are still much higher than that of phase retrieval. It demonstrates that an acceptable reconstruction quality can be achieved by applying less training targets. For the targets with separated subsections, like those on the last two rows with spatial extensions more than 150 microns, the proposed method always provides reconstruction results superior to those of phase retrieval. It demonstrates a big improvement in the FoV for our method.
Figure 4(e–g) show the reconstruction results generated by deconvolution using 4, 16, 36 times lower sampling rate in collecting the IIMs of the imaging targets, as defined in Eq. (22). The speckle pattern estimated using four training targets, as shown in Fig. 3(g), is also downsampled to be \(\overline{\ddot{S}}\). The reconstructed results are upsampled using bilinear interpolation and SSIM is calculated between the upsampled results and the original images. It can be found that acquiring only 1/4 × 1/4 of the data, the reconstruction quality of the proposed method is still acceptable. Even if reducing the data acquired to be only 1/6 × 1/6 of those used by phase retrieval, the proposed method can still provide reconstruction quality better than that of phase retrieval. It shows high potential in significantly reducing the complexity in speckle scanning and data acquisition process for collecting IIMs, which greatly benefits the real applications.
Conclusions
To noninvasively image complex targets through strongly scattering media with resolution beyond the limitation of illumination speckle grain lighted on the targets, speckle pattern estimation and deconvolution method is proposed. Crosscorrelation matching is proposed to correct the orientations of training targets and align their corresponding IIMs, which are applied as the inputs of the speckle pattern estimation model. Both theoretical derivation and experimental results have demonstrated that the proposed speckle pattern estimation model can accurately estimate the distribution of the pointspread speckle pattern. The proposed deconvolution method is also robust even with much less data acquired. In conclusion, our imaging method relieves the complexity limitation for imaging targets, improves the reconstruction performance and reduces the datacollection complexity for the tobeobserved targets.
The principal limitation of our proposed work is the dependency on the reconstruction accuracy of training targets by phase retrieval. While, the reconstruction accuracy of phase retrieval is constrained by the relative spectral bandwidths between the imaging targets and the speckle pattern^{17}. So, we put this as one of our future works to seek the way, such as introducing low rank feature of S, to improve the prior information for the speckle pattern estimation model, and to further analyze the solution space. Evaluating structural complexity of target from its IIM can be quite valuable in real scenario for selecting suitable training targets without phase retrieval. A preliminary experiment shows a kind of correlation between the entropy of the IIM and the phaseretrieval performance, which will be investigated further with other possible features. Another problem faced in the data collection process is the movements of imaging targets when collecting their IIMs, and it causes the structural deformation of IIMs and reconstructed results. A possible solution can be used is to track targets in realtime by analyzing the linear relationships between the variance and decorrelation time of the integrated intensity with target displacement along the axial and transversal directions, respectively^{56}. Thus, the incidentangle offset of laser beam can be calculated. Beyond these, deconvoluting tobeobserved targets from different depths and introducing the deformation of memory effect can be future works as well.
Change history
05 July 2018
A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has not been fixed in the paper.
References
Ntziachristos, V. Going deeper than microscopy: the optical imaging frontier in biology. Nat. Methods 7, 603–614 (2010).
Abramson, N. Lightinflight recording by holography. Opt. Lett. 3, 121–123 (1978).
Huang, D. et al. Optical coherence tomography. Science 254, 1178–1181 (1991).
Nasr, M. B., Saleh, B. E. A., Sergienko, A. V. & Teich, M. C. Demonstration of dispersioncanceled quantumoptical coherence tomography. Phys. Rev. Lett. 91, 083601 (2003).
Denk, W., Strickler, J. H. & Webb, W. W. Twophoton laser scanning fluorescence microscopy. Science 248, 73–76 (1990).
Helmchen, F. & Denk, W. Deep tissue twophoton microscopy. Nat. Methods. 2, 932–940 (2005).
Tyson, R. K. & Wizinowich, P. L. Principles of Adaptive Optics (Academic Press, 2010).
Ishimaru, A., Jaruwatanadilok, S. & Kuga, Y. Imaging through random multiple scattering media using integration of propagation and array signal processing. Waves in Random & Complex Media 22, 24–39 (2012).
Gibson, A., Hebden, J. & Arridge, S. Recent advances in diffuse optical imaging. Phys. Med. Biol. 50, R1–R43 (2005).
Culver, J. P., Ntziachristos, V., Holboke, M. J. & Yodh, A. G. Optimization of optode arrangements for diffuse optical tomography: a singularvalue analysis. Opt. Lett. 26, 701–703 (2001).
Vellekoop, I. M. & Mosk, A. P. Focusing coherent light through opaque strongly scattering media. Opt. Lett. 32, 2309–2311 (2007).
Katz, O., Small, E., Bromberg, Y. & Silberberg, Y. Focusing and compression of ultrashort pulses through scattering media. Nat. Photon. 5, 372–377 (2011).
Mosk, A. P., Lagendijk, A., Lerosey, G. & Fink, M. Controlling waves in space and time for imaging and focusing in complex media. Nat. Photon. 6, 283–292 (2012).
Katz, O., Small, E. & Silberberg, Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light. Nat. Photon. 6, 549–553 (2012).
Bertolotti, J. et al. Noninvasive imaging through opaque scattering layers. Nature 491, 232–234 (2012).
Yang, X., Pu, Y. & Psaltis, D. Imaging blood cells through scattering biological tissue using speckle scanning microscopy. Opt. Express 22, 3405–3413 (2014).
Katz, O., Heidmann, P., Fink, M. & Gigan, S. Noninvasive singleshot imaging through scattering layers and around corners via speckle correlations. Nat. Photon. 8, 784–790 (2014).
Wu, T., Katz, O., Shao, X. & Gigan, S. Singleshot diffractionlimited imaging through scattering layers via bispectrum analysis. Opt. Lett. 41, 5003–5006 (2016).
Zhuang, H., He, H., Xie, X. & Zhou, J. High speed color imaging through scattering media with a large field of view. Sci. Rep. 6, 32696 (2016).
Edrei, E. & Scarcelli, G. Memoryeffect based deconvolution microscopy for superresolution imaging through scattering media. Sci. Rep. 6, 33558 (2016).
Feng, S., Kane, C., Lee, P. A. & Stone, A. D. Correlations and fluctuations of coherent wave transmission through disordered media. Phys. Rev. Lett. 61, 834 (1988).
Freund, I., Rosenbluh, M. & Feng, S. Memory effects in propagation of optical waves through disordered media. Phys. Rev. Lett. 61, 2328 (1988).
Freund, I. Looking through walls and around corners. Physica A: Statistical Mechanics and Its Applications 168, 49–65 (1990).
Yaqoob, Z., Psaltis, D., Feld, M. S. & Yang, C. Optical phase conjugation for turbidity suppression in biological samples. Nat. Photon. 2, 110–115 (2008).
Popoff, S. M. et al. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. Phys. Rev. Lett. 104, 100601 (2010).
Cizmar, T., Mazilu, M. & Dholakia, K. In situ wavefront correction and its application to micromanipulation. Nat. Photon. 4, 388–394 (2010).
Aulbach, J., Gjonaj, B., Johnson, P. M., Mosk, A. P. & Lagendijk, A. Control of light transmission through opaque scattering media in space and time. Phys. Rev. Lett. 106, 103901 (2011).
Conkey, D. B., CaravacaAguirre, A. M. & Piestun, R. Highspeed scattering medium characterization with application to focusing light through turbid media. Opt. Express 20, 1733–1740 (2012).
Vellekoop, I. M., Lagendijk, A. & Mosk, A. P. Exploiting disorder for perfect focusing. Nat. Photon. 4, 320–322 (2010).
Vellekoop, I. M. & Aegerter, C. M. Scattered light fluorescence microscopy: imaging through turbid layers. Opt. Lett. 35, 1245–1247 (2010).
Hsieh, C.L., Pu, Y., Grange, R., Laporte, G. & Psaltis, D. Imaging through turbid layers by scanning the phase conjugated second harmonic radiation from a nanoparticle. Opt. Express 18, 20723–20731 (2010).
Judkewitz, B., Wang, Y. M., Horstmeyer, R., Mathy, A. & Yang, C. Specklescale focusing in the diffusive regime with time reversal of varianceencoded light (TROVE). Nat. Photon. 7, 300–305 (2013).
Park, J. H. et al. Subwavelength light focusing using random nanoparticles. Nat. Photon. 7, 455–459 (2013).
Hsieh, C.L., Pu, Y., Grange, R. & Psaltis, D. Digital phase conjugation of second harmonic radiation emitted by nanoparticles in turbid media. Opt. Express 18, 12283–12290 (2010).
He, H., Guan, Y. & Zhou, J. Image restoration through thin turbid layers by correlation with a known object. Opt. Express 21, 12539–12545 (2013).
Kong, F. et al. Photoacousticguided convergence of light through optically diffusive media. Opt. Lett. 36, 2053–2055 (2011).
Chaigne, T. et al. Controlling light in scattering media noninvasively using the photoacoustic transmission matrix. Nat. Photon. 8, 58–64 (2014).
Xu, X., Liu, H. & Wang, L. V. Timereversed ultrasonically encoded optical focusing into scattering media. Nat. Photon. 5, 154–157 (2011).
Si, K., Fiolka, R. & Cui, M. Fluorescence imaging beyond the ballistic regime by ultrasoundpulseguided digital phase conjugation. Nat. Photon. 6, 657–661 (2012).
Wang, Y. M., Judkewitz, B., DiMarzio, C. A. & Yang, C. Deeptissue focal fluorescence imaging with digitally timereversed ultrasoundencoded light. Nat. Commun. 3, 928 (2012).
Lai, P., Wang, L., Tay, J. W. & Wang, L. V. Photoacoustically guided wavefront shaping for enhanced optical focusing in scattering media. Nat. Photon. 9, 126–132 (2015).
Horstmeyer, R., Ruan, H. & Yang, C. Guidestarassisted wavefrontshaping methods for focusing light into biological tissue. Nat. Photon. 9, 563 (2015).
Goodman, J. W. & Cox, M. E. Introduction to Fourier optics (Roberts and Company Publishers, 2005).
Dainty, J. C. I. The statistics of speckle patterns. Progress in Optics 14, 1–46 (1977).
Fienup, J. R. Phase retrieval algorithms: a comparison. Appl. Opt. 21, 2758–2769 (1982).
Miao, J., Charalambous, P., Kirz, J. & Sayre, D. Extending the methodology of Xray crystallography to allow imaging of micrometresized noncrystalline specimens. Nature 400, 342–344 (1999).
Wang, Z., Jin, X., Hu, Y. & Dai, Q. Noninvasive imaging based on speckle pattern estimation and deconvolution. IEEE Visual Communications and Image Processing Conference (2017).
Goodman, J. W. Statistical Properties of Laser Speckle Patterns. Topics in Applied Physics 9, 9–75 (1963).
Boyd, S. & Vandenberghe, L. Convex Optimization (Cambridge Univ. Press, 2004).
Grant, M. & Boyd, S. CVX: Matlab software for disciplined convex programming, version 2.0 beta. http://cvxr.com/cvx (2013).
Grant, M. & Boyd, S. Graph implementations for nonsmooth convex programs. Recent Advances in Learning and Control 371, 95–110 (2008).
Akkermans, E. & Montambaux, G. Mesoscopic Physics of Electrons and Photons 427–439 (Cambridge Univ. Press, 2007).
Richardson, W. H. Bayesianbased iterative method of image restoration. Journal of the Optical Society of America 62, 55–59 (1972).
Lucy, L. B. An iterative technique for the rectification of observed distributions. Astronomical Journal 79, 745 (1974).
Wang, Z., Bovik, A. C., Sheikh, H. R. & Simoncelli, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 600–612 (2004).
Dogariu, A. & Akhlaghi, M. I. Tracking hidden objects using stochastic probing. Optica 4, 447–453 (2017).
Acknowledgements
The work is supported by the National Natural Science Foundation of China (Grant No. 61771275) and Shenzhen Project JCYJ20170817162658573, China.
Author information
Authors and Affiliations
Contributions
Z. Wang and X. Jin proposed the scheme and designed the experiments; Z. Wang conducted the experiments and analyzed data; X. Jin discussed the results; X. Jin and Z. Wang wrote the manuscript; X. Jin, Z. Wang and Q. Dai revised the manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Wang, Z., Jin, X. & Dai, Q. Noninvasive imaging through strongly scattering media based on speckle pattern estimation and deconvolution. Sci Rep 8, 9088 (2018). https://doi.org/10.1038/s41598018274671
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598018274671
Further reading

Spatial light modulator aided noninvasive imaging through scattering layers
Scientific Reports (2019)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.