Non-invasive super-resolution imaging through dynamic scattering media

Super-resolution imaging has been revolutionizing technical analysis in various fields from biological to physical sciences. However, many objects are hidden by strongly scattering media such as biological tissues that scramble light paths, create speckle patterns and hinder object’s visualization, let alone super-resolution imaging. Here, we demonstrate non-invasive super-resolution imaging through scattering media based on a stochastic optical scattering localization imaging (SOSLI) technique. After capturing multiple speckle patterns of photo-switchable point sources, our computational approach utilizes the speckle correlation property of scattering media to retrieve an image with a 100-nm resolution, an eight-fold enhancement compared to the diffraction limit. More importantly, we demonstrate our SOSLI to do non-invasive super-resolution imaging through not only static scattering media, but also dynamic scattering media with strong decorrelation such as biological tissues. Our approach paves the way to non-invasively visualize various samples behind scattering media at nanometer levels of detail.


Introduction
Optical imaging beyond the diffraction-limit resolution has enabled incredible tools to advance science and technology from non-invasive investigation of the interior of biological cells 1,2 to chemical reactions at single molecule levels 3 .Super-resolution stimulated emission depletion (STED) microscopy 4 has been progressed rapidly to achieve three-dimensional (3D) imaging with super-high spatiotemporal precision 5,6 .
For single-molecule detection and localization approaches 7,8 , such as stochastic optical reconstruction microscopy (STORM) or photo-activated localization microscopy (PALM), positions of photo-switchable probes are determined as centers of diffractionlimited spots.Repeating multiple imaging processes, each with a stochastically different subset of active fluorophores, allows positioning a number of emitters at fewnanometer resolution so that a high-resolution image is reconstructed 9 .After these pioneering techniques, the field of super-resolution microscopy has developed rapidly with various other techniques [10][11][12] to bring the optical microscopy within the resolution of electron microscopy.However, the requirement of sample transparency makes the super-resolution microscopy techniques impossible to access objects, which are hidden by strongly scattering media (Fig. 1a and supplementary Fig. S1, S2) such as biological tissues, frosted glass or around rough wall corners.These media do not absorb light significantly; however, they create noise-like speckle patterns 13 and challenge even our low-resolution visualization of samples.
Many approaches have been demonstrated to overcome the scattering effects and enabled imaging or focusing capability through scattering media.The most straightforward approaches utilize ballistic photons 14 , such as optical coherence tomography 15 or multi-photon microscopy 16 .However, strongly scattering media significantly reduce number of ballistic photons and lower the signal tremendously 17 .Some techniques require a guide star or an access on the other side of scattering media to characterize or reverse their scattering effects before imaging such as wave-front shaping techniques [18][19][20][21] , transfer matrix measurement 22,23 .The memory effect of light through scattering media 24,25 implying a shift-invariant point spreading function (PSF), allows imaging by deconvolution [26][27][28] of a speckle pattern with the PSF, which is measured invasively.A scattering medium with a known PSF is a scattering lens that, in turn, is a low pass filter like any conventional lens.Deconvolution recovers and enhances the transmission of high frequency components (i.e. a sharper cut-off lowpass filter) therefore, resolution is slightly higher than the diffraction limit (Fig. 1b and supplementary Fig. S2d-f).Deconvolution imaging currently provides the bestresolution images from speckle patterns.Each measured PSF is only valid for one scattering medium, therefore, cannot be used for dynamic scattering media.Noninvasive imaging through scattering media where the image is retrieved without characterizing scattering media is desired for real applications.Diffuse optical tomography 29,30 and time-of-flight imaging 14,29,31 are capable of seeing though scattering layers and around corners non-invasively; however, the spatial resolution is much lower than the optical diffraction limit.Thanks to the shift-invariance speckletype PSF of thin scattering media, the 2D image and even the 3D image of a sample can be revealed non-invasively from the speckle patterns by a phase retrieval algorithm [32][33][34] .The limited performance of the algorithm and the camera, together with presence of noise and sample's complexity, usually makes the image retrieval process fail or converge with some artifacts and slightly lower resolution compared to the diffraction limit (Fig. 1c and supplementary Fig. S2g-i).
Here, we present our stochastic optical scattering localization imaging (SOSLI) technique to do non-invasive super-resolution imaging through scattering media.The technique only requires an imaging sensor to capture speckle patterns created by blinking emitters behind a scattering medium; no other optics or complicated alignment are needed (Fig. 1a, Fig. 2a, and supplementary Fig. S1).The positions of emitters in each stochastic frame are first determined at very high precision from the corresponding speckle pattern, and then a super-resolution image of the full sample is reconstructed by superposing a series of emitter position images (Fig. 1d).We demonstrate the image reconstruction with resolution beyond the diffraction limit by a factor of five as a proof of concept, while there is no fundamental limit for SOSLI on resolution, similar to current super-resolution microscopy techniques.More interestingly, the localization algorithm is based on a single-shot speckle pattern with minimum correlation among adjacent patterns; therefore, we develop adaptive SOSLI to do super-resolution imaging through dynamic scattering media such as fresh chicken eggshell membrane with decorrelation of up to 80%.Our SOSLI demonstrates a desired technique to see through translucent media such as biological tissues or frosted glass with unprecedented clarity.

Stochastic Optical Scattering Localization Imaging (SOSLI)
An object  consists of stochastically blinking emitters: , where   is the i th blinking pattern (a subset of  ) and N is the total number of the blinking patterns.
After light propagating through scattering media, each   produces a speckle pattern  , captured by a camera (Fig. 2a).If object size is within the memory effect of the scattering media, the PSF is shift-invariant and speckle-type, therefore the speckle pattern   (Fig. 2b) of object   preserves the object's autocorrelation (Fig. 2c).The image of   can be retrieved from its autocorrelation by an iterative phase retrieval algorithm 33 (Fig. 2d).The limitation in the camera's bit depth, photon budget and performance of phase retrieval algorithms in presence of image acquisition noise degrade the diffraction-limit resolution of this non-invasive retrieval image.However, a standard localization algorithm 35,36 is employed to find the position of emitters at very high resolution (Fig. 2e) and remove algorithm artifacts.Similar to other localization microscopy techniques, the precision is higher for spatially sparse emitter samples where only one emitter is temporally active in a diffraction-limited region.The sharp and clear image   , presents the precise relative emitter positions of pattern   , while losing their exact positions because   , is only retrieved from autocorrelation of   through autocorrelation of   .The estimated PSF of the scattering medium can be retrieved by deconvolution (Fig. 2f): ′ = (  ,   , ), which is also shifted in comparison to the actual PSF because of losing exact emitter positions in   , .
Next, a series of clean super-resolution images   ′ with emitter positions is reconstructed for a corresponding series of stochastic patterns   by deconvolution of its corresponding speckle pattern   with the estimated ' and localization as presented in Fig. 2g.A super-resolution image of the full sample (Fig. 2h) is now reconstructed by superposing all individual images as: , which represents object  with an arbitrary position.This principle is valid provided that PSF does not change among the group of   .For comparison, we present the typical simulation image (Fig. 2i) retrieved from autocorrelation of a single speckle pattern, in which simulation parameters are similar with the exception that all the emitters are on.The simulated diffraction limit is about 3.2 pixels.This current state-of-the-art technique for noninvasive imaging through scattering media shows a blurry image where the low spatial frequency presents the diffraction limit of the optical system together with some artifact from the phase retrieval algorithm.In contrast, the image reconstructed by SOSLI is much sharper (Fig. 2h).

Super-resolution imaging through a ground glass diffuser
To prove our concept, we first demonstrate SOSLI for non-invasive super-resolution imaging through a ground glass diffuser.Microscopic objects comprising multiple stochastic blinking emitters are created by de-magnifying projector images through a microscope objective.The de-magnifying image of each pixel in a digital micro-mirror device (DMD) is an intermittent emitter with a size of about 1.34 µm (Supplementary Fig. S1a).The microscopic object is placed 10 mm behind the ground glass diffuser, which is kept unknown in all demonstrations.The incoherent light from the object propagating through the optical diffuser is recorded by a monochromatic camera, which is 100 mm in front of the diffuser.An iris with diameter of 1 mm is placed immediately after the optical diffuser to act as the aperture of the imaging system.A larger iris size enhances the diffraction limit of the imaging system and achieve a sharper image (supplementary Fig. S2); however, it reduces the speckle contrast that is vital for phase retrieval approach.
Figure 3 shows the experimental results for three different imaging approaches.
Single-shot non-invasive imaging through scattering media is performed in Fig. 3a, where all emitters are on.The result is recovered from the autocorrelation of a single speckle pattern by the phase retrieval algorithm.The image is very blurred, and we cannot distinguish 2 lines with a gap of 4 µm between them (Fig. 3b).We can calculate the diffraction limit of our system as 0.61 /NA = 6.7 , where  = 550 nm and NA = 0.05.Beside the diffraction limit, the performance of the phase retrieval algorithm in the presence of experimental noise also degrades the image quality and limits the resolution.With the DMD projector, we can measure the PSF by turning on a single pixel at the center only and capture its speckle pattern.Such an "invasive guiding star" for the PSF measurement allows us to calculate the image by deconvolution and significantly enhances the resolution (Fig. 3c).The invasive deconvolution approach is more deterministic, robust to the noise and enhances the high spatial frequency components of the image.This allows us to distinguish the 2 lines with a 4-µm gap between them (Fig. 3d).However, we still cannot see the gap of 2.68 µm between 2 lines.Most strikingly, our super-resolution image reconstructed non-invasively by SOSLI is remarkably clear as presented in Fig. 3e.We can resolve very well all the smallest features of our sample, i.e. 2 thin lines (1.34 µm width) with a gap of 1.34 µm in between (Fig. 3f).The smallest sample feature is smaller than the diffraction limit by a factor of 5. Figure 3e-f also clearly illustrates that the capability of SOSLI is far beyond our sample's smallest features (pixel size), which are currently limited by the projector and optics of the sample creating system.
It is worth to highlight some important factors in our SOSLI performance.Supplementary Fig. S3 presents more detail for the reconstruction process in which the localization is important to remove all the background noise and artifacts in the emitter image resulting from the phase retrieval algorithm.This leads to a better estimation of PSF for a series of deconvolution calculation after that.In addition, the localization process also allows SOSLI to tolerate more errors in deconvolution process due to imperfect PSF estimation and noise in image acquisition (Supplementary Fig. S4-5).
Our approach relies on stochastic emitter patterns to reconstruct a full object; therefore, the image quality is improved with more stochastic patterns (supplementary Fig. S6).
Figure 4 presents some images of more complex objects for performance comparison among the three techniques.Similar to Fig. 3, the complex objects are best resolved with our SOSLI approach (Fig. 4a-c), while the retrieval image from autocorrelation of a single speckle pattern shows the poorest performance that also has some artifacts (Fig. 4d-f).The invasive imaging approach by deconvolution shows moderate performance in Fig. 4g-i.Obviously, our SOSLI for non-invasive imaging through scattering media goes far beyond the diffraction limit and surpasses all the current state-of-the-art imaging through scattering media, including both invasive and non-invasive techniques.

Super-resolution imaging through a biological tissue
Our SOSLI demonstrations in Fig. 3 and Fig. 4 rely on a fixed PSF for reconstruction of multiple stochastic emitter patterns; and therefore, we cannot directly use for dynamic scattering media such as biological tissues.Figure 5a shows the decorrelation behaviors of PSFs for two different scattering media.For static scattering media such as ground glass diffusers, the PSF is a constant pattern and the correlation of 1 is achieved for any measured PSFs at any time.On the other hand, dynamic scattering media such as fresh chicken eggshell membrane, the PSF is gradually changed and the correlation with the initial one decreases with time.In our experiment for fresh chicken eggshell membrane, the correlation reduces from 1.0 to 0.2 after 300 measurements, with the fastest decay rate in the first 70 measurements (correlation decreases to 0.54).
The reconstruction by SOSLI with a single estimated PSF shows a noisy and blurred image due to this decorrelation of the membrane (Fig. 5b).Supplementary Fig. S7 presents the deconvolution images from stochastic speckle patterns with an estimated PSF from the first speckle pattern.Obviously, the assumption of a static PSF in SOSLI does not hold in this case.For very large PSF decorrelations, the localization process cannot distinguish the emitters from strong background and artifacts in the deconvolution images, leading to the result in Fig. 5b.
We introduce an adaptive approach to demonstrate our SOSLI for super-resolution imaging through dynamic scattering media.We now utilize SOSLI to localize and then superpose emitters in 50 stochastic patterns, in which the fresh chicken eggshell membrane still can retain its PSF correlation of more than 60%.With a total collection of 300 stochastic patterns, we divide this into 11 sections, each containing 50 speckle patterns in which the first 25 patterns are overlapped with the previous section; the other 25 patterns are then overlapped with the next section (Supplementary Fig. S8).We can directly apply SOSLI with single estimated PSFs to reconstruct 11 super-resolution images, each one representing a part of the object.However, we cannot directly superpose these 11 images to reconstruct the full image because their absolute positions are lost in each SOSLI procedure.In fact, the relative positions among the images are the relative positions among their respective retrieved PSFs, which are achieved independently (similar to the discussion on estimated PSFs from different speckle patterns in supplementary Fig. S4, S5).The correlation between two retrieved PSFs is a bright spot with background; the intensity of the bright spot presents the correlation between two PSFs; and the position of the bright spot (relative to the center) presents the relative position between two PSFs.Because PSF correlation only reduces to about 65% between two adjacent sections in our experiment, the brightest spot in correlation between two estimated PSFs is very clear and its center is very easy to locate (similar to emitter localization).With this procedure, the relative positions of 11 images are identified (Supplementary Fig. S9).We only need to shift our individual superresolution images to make the relative position zero before superposing them to receive a full image.Figure 5c presents the superposing result after alignment of their individual images.The final image with adaptive SOSLI is super-resolution, much clearer and less noise compared to SOSLI with a static PSF (Fig. 5b).For comparison, the low-resolution images obtained by the phase retrieval algorithm and invasive deconvolution of this object through chicken eggshell membrane are similar to Fig. 1bc, respectively.By doing adaptive SOSLI, we reconstruct a super-resolution image noninvasively through dynamic scattering media with effective correlation of more than 0.6 (shading area in Fig. 5a) while the actual correlation reduces to 0.2 during image acquisition.The procedure can continue with more stochastic pattern acquisition and the membrane is even completely decorrelated but the effective correlation for adaptive SOSLI still can maintain at more than 60%.Our SOSLI with an adaptive PSF shows a very practical approach to do non-invasive super-resolution imaging through dynamic scattering media.

Discussion and Conclusion
Our SOSLI relies on a shift-invariant speckle-type PSF of scattering media, which is valid if the hidden object behind the scattering media is within the memory effect region of the scattering media.Therefore, we can image a larger object with thinner scattering media.However, there is no fundamental physics to limit our SOSLI's resolution, similar to conventional super-resolution microscopy.In fact, we have practical challenges for SOSLI to reach high resolution.The most challenging requirement is the photon budget for each intermittent emitter, i.e. the number of photons per emitter per blink.For the localization approach in super-resolution microscopy [7][8][9] where all the photons go into the diffraction limit spots, the resolution can be enhanced by √ where  is the number of captured photons.With scattering media, we need to capture many more photons, because the photons are now scattered everywhere, form speckles, and we need multiple speckles to retrieve an image.The noise and bit depth of camera, as well as the sparsity of active emitters are also important factors that affect our computational approach, limiting the resolution.
It is important for any imaging technology to beat the dynamics of both the object and the environment.Like any other localization approach [7][8][9] , our SOSLI requires a static object during the whole image acquisition process.However, the adaptive SOSLI can significantly mitigate the environment dynamics (i.e. the decorrelation of scattering media).It is worth to note that our method for localizing emitters in a stochastic speckle pattern relies on a single-shot image, and then the final super-resolution image relies on alignment of the localized-emitter patterns.The former implies that a stochastic speckle pattern can be captured sufficiently fast to beat the dynamics of the scattering media.
The latter is more important for SOSLI.In extremely dynamic scattering media, the deconvolution might fail even when using the estimated PSF for immediately next speckle pattern because of fast decorrelation.We can then apply the most adaptive SOSLI by retrieving every emitter pattern from its speckle autocorrelation by phase retrieval algorithm then localization.The only requirement for the most adaptive approach is that the scattering media do not decorrelate completely between two consecutive shots.In our demonstration (supplementary Fig. S10), only 20% correlation is sufficient to determine the relative positions between two super-resolution images.Superposing then can be carried out after proper alignment to achieve a superresolution image.
In summary, we have presented our simulation and proof-of-concept demonstration of SOSLI for non-invasive super-resolution imaging through both static and dynamic scattering media.We only need a camera to capture multiple images of scattered light from stochastic emitters behind scattering media, and then our computational approach will localize these emitters non-invasively to reconstruct a super-resolution image.Our experimental results show that SOSLI enhances resolution by a factor of 5 compared to the diffraction limit, showing features with considerably more detail compared to both state-of-the-art invasive and non-invasive imaging through scattering media.The demonstrated resolution enhancement is currently limited by our sample preparation while the SOSLI technique presents no fundamental limit to achieve higher resolution.
The adaptive SOSLI allows super-resolution imaging non-invasively through highly dynamic scattering media with decorrelation of up to 80% while capturing two consecutive speckle patterns.Our SOSLI demonstration shows a promising approach for optical imaging through dynamic turbid media, such as biological tissue, with unprecedented clarity.

Method
Scale bar: All the experimental results of recovered images show a scale bar of 10 camera pixels that is equivalent to 65 µm in the imaging plane.This is corresponding to 6.5 µm on the object plane because the magnification is 10.However, we do not know the scale bar on the object plane or magnification of the imaging system in non-invasive approach because the distance from the object to scattering media is unknown.We can only resolve the sample by angle resolution.The scale bar of 65 µm in the imaging plane is equivalent to the angle of 0.65 mrad.

Data processing:
In all experiments, the resolution of the raw camera images is 2560×2160 pixels.We crop them into a resolution of 2048×2048 pixels for implementations of all the mentioned techniques in this work.The final reconstructed images are cropped to a square window with dimensions ranging between 75×75 pixels and 150×150 pixels (depending on the imaged object dimensions).Algorithms are developed in Matlab and run on a normal PC (Intel Core i7, 16 GB memory).A typical procedure for SOSLI with 300 speckle patterns takes 2-3 minutes.

Funding
The

The state-of-the-art non-invasive and invasive imaging
We conduct experiments for the demonstration of the current state-of-the-art noninvasive and invasive imaging through a 120-grit ground glass diffuser using the experimental setup shown in supplementary Fig. S1.A non-invasive image is retrieved from the autocorrelation of a single-shot speckle pattern by applying the phase retrieval algorithm.An invasive image is the deconvolution of a single-shot speckle pattern with an invasively measured PSF.The diameters of the imaging iris are set as 1 mm, 2 mm and 3 mm that correspond to NAs of 0.05, 0.1 and 0.15, respectively, and the diffraction limit of 6.7 µm, 3.4 µm and 2.3 µm respectively.We can easily see the effects of NA on resolution from the results given in supplementary Fig. S2.An imaging system is a low pass filter, where higher NA (higher cut-off frequency) provides higher resolution (i.e.sharper) images than lower NA does.Because of the effects of noise, camera's dynamic range and dark counts on the performance of the phase retrieval algorithm, the single-shot non-invasive images have a slightly lower resolution than the diffraction limit.If these effects are too high, the algorithm may not even converge.On the other hand, the deconvolution images have a slightly higher resolution than the diffraction limit because deconvolution recovers and enhances the high frequency components of images, i.e. sharper cut-off low pass filter.retrieval/localization results (Fig. S3d-e).Figure S3i-j present the emitter positions localized after deconvolution of another speckle pattern (Fig. S3h) with two estimated PSFs in Fig. S3f-g.The emitter positions in Fig. S3i-j automatically align very well with those in Fig. S3d-e

Reconstruction with PSFs estimated from different speckle patterns
During our implementation of SOSLI, we randomly choose one pattern out of multiple collected speckle patterns for estimation of the PSF.Interestingly, different speckle patterns give us slightly different estimated PSFs.The differences are not only arbitrary shift from each other but also the pattern itself (Supplementary Fig. S4a-c and S5b-c).However, the reconstructed images from these estimated PSFs are very much similar (Supplementary Fig. S4 d-f) with different shifts.It seems that we only estimate the main features of the PSF, which is sufficient for reconstruction.We test the similarity of these three estimated PSFs by calculating the correlation among them.The autocorrelation of PSF1 shows its random speckle nature with a bright spot (Gaussian profile) at the center (Supplementary Fig. S5a).The correlation of PSF1 with the other PSFs indicates an off-center bright spot with some background (Supplementary Fig. S5b-c), implying that these PSFs share the main features and shift from each other.The clear bright spot allows us to locate its center then obtain the relative shift between PSFs.The obtained relative shifts (Supplementary Fig. S5d) are exactly equal to the relative shifts between the retrieved objects (Supplementary Fig. S4 d-f).The observation is interesting, important and useful.Because of the localization, we can easily remove the background artifact and noise in the phase retrieval images and deconvolution images.Therefore, the SOSLI approach can tolerate more error in PSF estimation.This not only explains how and why the SOSLI should work very well in static scattering media, but also inspires us to conceive a successful solution for dynamic scattering media in the section 7.
Scale bar: 200 camera pixels for a-e, and 10 camera pixels for f-h (see method).

Adaptive approach for SOSLI to do super-resolution imaging through a biological tissue
Our adaptive approach for SOSLI need to define the number of stochastic frames for each section depending on the decorrelation time and image acquisition time.The requirement is that every two adjacent sections have some correlation.In our approach, we even divide the set of patterns into overlapping sections (supplementary Fig. S8b) to guarantee correlation between adjacent sections.By doing sectional reconstruction, we effectively operate SOSLI with high speckle correlation for dynamic scattering media (shading area in Fig. 5a and supplementary Fig. S8a).Individual super-resolution images are reconstructed for each section independently.Before superposing these images on top of each other, we need to align them.
As presented in the previous section, the relative shifts between the reconstructed images are equal to the relative shifts between the estimated PSFs when the scattering media is not completely decorrelated.We calculate the correlation pattern of the estimated PSFs for every two adjacent sections to identify the relative shifts between them.The vectors indicated the relative shifts between section  and section  as follows.
, ⃗⃗⃗⃗⃗ = {  ⋆   } − {  ⋆   } where {} position gives 2D coordinate of the brightest spot center of an image, and ⋆ indicates the correlation calculation.{  ⋆   } is simply the center of the image because of speckle-type PSF.Supplementary Fig. S9a-j show the 10 vectors  , ⃗⃗⃗⃗⃗ for the frame set from 2 to 11 with respect to the first frame set.There relative positions are presented in supplementary Fig. S9k-l.With this, all the individual superresolution images can be inversely shifted before superposing (supplementary Fig. S8c) to achieve the full image as presented in Fig. 5c.

Figure 1 .
Figure 1.Super-resolution imaging through scattering media with SOSLI in

Figure 2 .
Figure 2. Principle and simulation results of SOSLI.(a) Object constitutes many

Figure 3 .
Figure 3. Experimental results of imaging through a ground glass diffuser with

Figure 4 .
Figure 4. Experimental demonstration of three techniques for imaging several

Figure 5 .
Figure 5. Experimental demonstration of non-invasive super-resolution imaging , respectively because they are reconstructed from the same estimated PSFs.The absolute positions of emitters and PSF are not important in our SOSLI.They do not affect our results and usual imaging techniques do not concern about absolute position.Supplementary Fig. S3.Reconstructed and individual stochastic emitter pattern and estimated PSF for deconvolution.(a) A typical stochastic speckle pattern.(b-c) Typical images retrieved from autocorrelation by phase retrieval algorithm.(d-e) Localized-emitter images.(f-g) Estimated PSFs.(h) Another stochastic speckle pattern.(i-k) Emitter positions calculated from a single speckle pattern and two estimated PSFs.Scale bar: 200 camera pixels for a, h, g, h and 10 camera pixels for b, c, d, e, i, j (see method).
Supplementary Fig. S4.Retrieved results of an object with the PSFs estimated from different speckle patterns arbitrarily picked.(a-c) The estimated PSFs, (d-f) the corresponding reconstructed images at super-resolution.Scale bar: 200 camera pixels for a-c, and 10 camera pixels for d-f (see method).