Introduction

Optical microscopy is an indispensable tool in biomedical investigations and clinical diagnostics. However, its penetration depth is limited to approximately a millimeter inside complex samples such as biological tissue, due to light scattering. While non-optical imaging techniques, such as those based on ultrasound sonography or magnetic resonance imaging (MRI), allow deeper investigations, their resolution is typically orders of magnitude inferior to that of optical microscopes. As a result, it is currently impossible to conduct microscopic investigations at depths, an important goal that is at the focus of many recent works1.

The state-of-the-art approaches for deep-tissue high-resolution optical-contrast imaging are based on the combination of light and sound1. These techniques combine the advantages of optical contrast with those of the near scatter-free propagation of ultrasound in soft tissues. The leading deep-tissue imaging techniques can be divided to two: acousto-optic imaging (AOI)2,3,4, and photo-acoustic tomography (PAT)3,5. PAT relies on the generation of ultrasonic waves by absorption of light in a target structure under pulsed optical illumination. In PAT, images of absorbing structures are reconstructed by recording the propagated ultrasonic waves with detectors placed outside the sample. In contrast to PAT, AOI does not require optical absorption but is based on the acousto-optic effect: in AOI a focused ultrasound spot is used to locally modulate light at chosen positions inside the sample. The ultrasound spot is generated and scanned inside the sample by an external ultrasound transducer. The modulated frequency-shifted light is detected outside the sample using one out of a variety of interferometry-based approaches2,4. This enables the reconstruction of the light intensity traversing through the localized acoustic focus inside the sample. AOI and PAT thus both provide images of optical contrast with a spatial resolution limited by the dimensions of the ultrasound focus, which is dictated by acoustic diffraction. Ultimately, the ultrasound focus size in soft tissue is limited by the attenuation of high-frequency ultrasonic waves. As a result, the practical attainable resolution deteriorates with imaging depth, typically providing a depth to resolution ratio of approximately 1005,6.

Since the acoustic-scale resolution of AOI and PAT does not allow microscopic imaging many efforts have been devoted to develop approaches that can surpass the acoustic resolution limit7. In PAT, nonlinear effects8, dynamic speckle illumination9,10, temporal fluctuations from flowing absorbers11, or localization of isolated absorbers12,13 were exploited to provide sub-acoustic resolution. However, these photo-acoustic based approaches still require optical absorption and relative intense laser pulses. In AOI, surpassing the acoustic resolution limit was first demonstrated using nonlinear acousto-optic effects14, requiring high acoustic-pressures. Recent approaches that rely on optical wavefront-shaping to focus light into the acoustic focus, can surpass the acoustic diffraction limit via either digital phase-conjugation15,16, or by measurement of the acousto-optic transmission matrix17. Unfortunately, these novel wavefront-shaping based approaches require a very large number of measurements (up to several thousands16,17) and digital wavefront-shaped illumination to take place within the speckle decorrelation time of the sample.

As the speckle decorrelation time in tissue can reach sub-millisecond timescales18, these approaches for super-resolution AOI have so far been limited to static samples. Here, we present an approach for super-resolution AOI, which, unlike previous works, exploits the naturally short speckle decorrelation time of dynamic samples rather than trying to overcome it. Our approach is made possible by adapting the principles of super-resolution optical fluctuations imaging (SOFI)19,20, a super-resolution technique originally developed for microscopic imaging of blinking fluorophores, to AOI. Specifically, we show that naturally dynamic fluctuating optical speckle grains in AOI can be considered as the analogues of blinking fluorescence molecules of SOFI, allowing super-resolved AOI. We show that a super-resolved AOI image can be obtained by statistical analysis of temporal fluctuations of ultrasonically modulated light, recorded using a conventional AOI setup. Our work extends the application of the fundamental principles of SOFI from microscopy19, ultrasound sonography21, and photoacoustics9,11, to AOI. Most importantly, since in our approach super-resolution originates from the natural temporal fluctuations of speckles, only a single measurement needs to be performed within each speckle decorrelation time, in contrast to the thousands of measurements (and wavefront-shaping processes) required by the state of the art approaches16,17. Our approach thus allows to perform super-resolved AOI with speckle decorrelation times orders of magnitude shorter than current approaches.

Results

Principle

The principle of our approach is presented in Fig. 1. In our approach, a set of measurements performed using a conventional AOI setup is used to obtain super-resolution. Figure 1a shows a schematic description of a typical AOI experiment, where the goal is to image a target object hidden inside a scattering sample. In AOI, a laser beam at a frequency \({f}_{{\mathrm{opt}}}\) illuminates the sample, and an ultrasound wave at a central frequency \({f}_{{\mathrm{US}}}\) is focused at a position, \({{\bf{r}}}_{{\bf{us}}}\), which is scanned across the volume of interest. The acousto-optic signal for each ultrasound focus position, \({P}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}})\), is the total detected power of the light that has been frequency-shifted to \({f}_{{\mathrm{AO}}}={f}_{{\mathrm{opt}}}+{f}_{{\mathrm{US}}}\), by the acousto-optical interaction at the focus. Considering linear ultrasound modulation and an ultrasound focus pressure distribution given by \(h({\bf{r}}-{{\bf{r}}}_{{\bf{us}}})\), the acoustically-modulated power, as measured by an ideal detector placed outside the sample (see Discussion) is given by:

$${P}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}})=\iint h({\bf{r}}-{{\bf{r}}}_{{\bf{us}}}){I}_{{\mathrm{obj}}}({\bf{r}})dv=[h* {I}_{{\mathrm{obj}}}]({{\bf{r}}}_{{\bf{us}}})$$
(1)

where the integration is performed over the entire sample volume. Equation (1) shows that the AOI image in the ideal case, \({P}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}})\), is the convolution of the optical intensity distribution inside the medium, \({I}_{{\mathrm{obj}}}({\bf{r}})\), with the ultrasound focus pressure distribution, \(h({\bf{r}})\). Thus, the imaging resolution in AOI is dictated by the ultrasound focus size, which serves as the effective point spread function (PSF) of the AOI system. Importantly, although the acousto-optic modulation is a coherent phase modulation of the optical fields, the convolution relation of Eq. (1) is not affected by the phases of the detected-field, which are crucial for wavefront-shaping based acousto-optic approaches16,17,18. As a result, Eq. (1) and our approach are appropriate to any type of acousto-optic detection system with a sufficiently large etendue (see Discussion).

Fig. 1: Principle of super-resolution acousto-optic imaging (AOI) via speckle fluctuations and numerical results.
figure 1

a Schematic of the experimental setup: An AOI setup is equipped with a rotating diffuser for producing controlled speckle fluctuations. An object hidden inside a scattering sample is imaged by scanning a focused ultrasound beam (in orange) over the object, and recording the ultrasound-modulated light power at each position. b At each position of the ultrasound focus, the fluctuations of the ultrasonically modulated light power due to the diffuser rotation (speckle decorrelation) are recorded. These are statistically analyzed to provide super-resolved AOI images (c–f). c Schematic depiction of the source of AOI fluctuations: at each position of the ultrasound focus (orange), different speckle realizations lead to different ultrasound-modulated power. d A simulated image produced with the average detected signal at each position provides a conventional AOI image resolution. e A simulated image produced from the second-order cumulant (variance) of the fluctuations at each position provides a \(\sqrt{2}\) improved resolution without deconvolution. f The simulated third-order cumulant image provides a \(\sqrt{3}\) improved resolution without deconvolution.

Full size image

As the common laser illumination in AOI is spatially and temporally coherent, the light intensity distribution inside the medium, \({I}_{{\mathrm{obj}}}({\bf{r}})\), is a speckle intensity pattern that is given by the product of the target object transmission, \({T}_{obj}({\bf{r}})\), with an illuminating speckle intensity pattern, \(S({\bf{r}})\): \({I}_{{\mathrm{obj}}}({\bf{r}})={T}_{{\mathrm{obj}}}({\bf{r}})S({\bf{r}})\). Due to the natural dynamics of the sample, the speckle pattern illuminating the object \(S({\bf{r}})\) dynamically fluctuates at timescales given by the speckle decorrelation-time, \({\tau }_{corr}\). The speckle decorrelation time can reach sub-millisecond timescales in living tissue18, forming one of the fundamental challenges for wavefront-shaping based imaging techniques7. However, as we show below, the temporally fluctuating speckles can be utilized rather than struggled with for improved resolution.

Defining the speckle pattern illuminating the object at time \({t}_{i}\) by \({S}_{i}({\bf{r}})\), the measured ultrasonically-tagged intensity at the time \({t}_{i}\) is given by:

$${P}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}},{t}_{i})\equiv {{\bf{P}}}_{i}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}})=[h* {I}_{i}]({{\bf{r}}}_{{\bf{us}}})=[h* ({T}_{{\mathrm{obj}}}\times {S}_{i})]({{\bf{r}}}_{{\bf{us}}})$$
(2)

Equation (2) shows that the conventional AOI intensity measurements, \({P}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}},{t}_{i})\), are expected to temporally fluctuate at timescales given by \({\tau }_{corr}\), as shown in Fig. 1b. Most importantly, Eq. (2) shows that the measured intensity fluctuations in AOI are mathematically equivalent to the measured intensity fluctuations in fluorescence microscopy with blinking fluorophores, utilized in SOFI19: the fluctuating speckle grains, \({S}_{i}\), effectively serve as blinking fluorophores on an object having a spatial labeling distribution \({T}_{{\mathrm{obj}}}({\bf{r}})\), which is imaged by a microscope with a PSF given by \(h({\bf{r}})\).

Since the temporal fluctuations of individual speckle grains are uncorrelated22, the principles of SOFI can be directly applied to AOI. Thus, all that is required for super-resolved AOI is to acquire a set of \(i=1:m\) conventional AOI measurements, temporally separated by more than \({\tau }_{corr}\), at each position of the ultrasound focus. In case that \({\tau }_{corr}\) is too long, a rotating diffuser can be introduced at the illumination path for generating rapid fluctuations at controlled timescales, as shown in Fig. 1a, b. To achieve super-resolution via SOFI, for each probed position \({{\bf{r}}}_{{\bf{us}}}\) (each AOI image ‘pixel’), the \({n}^{th}\)-order statistical cumulant, \({C}_{n}({{\bf{r}}}_{{\bf{us}}})\), of the recorded temporal intensity fluctuations is calculated and taken as the reconstructed pixel intensity for \({{\bf{r}}}_{{\bf{us}}}\). The \({n}^{th}\)-order cumulant provides a \(\sqrt{n}\)-times resolution increase without deconvolution (see Supplementary Note 2), and up to \(n\)-times resolution increase with deconvolution19. For example, the second-order cumulant:

$${C}_{2}({{\bf{r}}}_{{\bf{us}}})={\left\langle {\left[{P}_{i}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}})-{\left\langle {P}_{i}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}})\right\rangle }_{i}\right]}^{2}\right\rangle }_{i}$$
(3)

which is simply the variance, provides a \(\sqrt{2}\) resolution increase before deconvolution, and a factor of 2 resolution increase with deconvolution (See detailed derivations in Supplementary Note 2). For SOFI to work, all that is required is the presence of uncorrelated temporal fluctuations of sub-acoustic-diffraction sized sources of signals. This condition is naturally fulfilled by randomly fluctuating optical speckles.

Figure 1c depicts the proposed data acquisition scheme utilizing a conventional AOI system, along with numerically simulated results using \(m=2000\) speckle realizations: the acoustic focus (in orange) scans the object (a negative USAF resolution target). For each acoustic focus position the simulated frequency-shifted light intensity is recorded repeatedly at times \({t}_{1}..{t}_{m}\), with a different speckle pattern illuminating the object at each time. While the conventional AOI image, or the average of the simulated intensities at each point, provide an image blurred by the acoustic PSF, as depicted in Fig. 1d, the higher-order cumulants images shown in Fig. 1e, f allow to resolve the target features beyond the resolution of the ultrasound focus.

Experimental results for transverse scanning

To experimentally demonstrate our approach we constructed a proof-of-principle experiment using the setup schematically shown in Fig. 1a and explained in detail in Supplementary Fig. 1. The setup is a conventional pulsed AOI setup, with the only addition of a controlled rotating diffuser before the sample for generating random speckle patterns with controlled decorrelation time. To allow direct optical inspection of the target object and ultrasound focus we used a sample composed of a target object (A portion of a USAF 1963A resolution target) placed in a transparent water tank between two scattering diffusers. A direct optical image of the object illuminated by one speckle realization, as captured by removing the second diffuser and mounting an imaging lens, is shown in Fig. 2a. We used the same configuration to characterize the ultrasound focus (see Supplementary Fig. 2), and the speckle grain size. The dimensions of the ultrasound focus in this experiment were \(\Delta {\rm{X}}=350\ \upmu {\rm{m}},\Delta {\rm{Y}}=440\ \upmu {\rm{m}}\) full-width at half max (FWHM) in the horizontal (transverse) and vertical (axial) directions, correspondingly (Supplementary Fig. 2a, b), where axial refers to the axis along the ultrasound propagation direction (see Fig. 1a).

Fig. 2: Experimental demonstration of super-resolution acousto-optic imaging (AOI) along the transverse (x) axis using 160 speckle realizations.
figure 2

a–e results with large speckle grain size: a target object, \(I(x,y)\) (a section of USAF1963A resolution target), illuminated by a single speckle realization, \({S}_{i}(x,y)\), as imaged directly without the scattering layer. b ultrasonically-tagged optical intensity distribution at the object plane, for one position (\({x}_{us},{y}_{us}\)) of the ultrasound focus, \(h(x,y)\), visualizing the ultrasound focus size. c Detected ultrasound-modulated light pattern for (b), as recorded at the camera plane through the second diffuser. d AOI image profile obtained by scanning the ultrasound focus along the dashed line in a: blue—conventional AOI trace, obtained from the ensemble average of the recorded power at each position; red—square-root of the second-order cumulant of the acousto-optic fluctuations, showing improved resolution; orange—third-order cumulant; black, dashed: cross-section of the target object, along the dashed line in a. e The results from d after deconvolution. f–i as in a, b, d, e, for the small speckle grain size. Acoustic focus width: \(350\ \upmu {\rm{m}}\) FWHM on scanning axis. Scanning direction: from left to right. scale-bars: \(200\ \upmu {\rm{m}}\) (a, b, f–i), \(330\ \upmu {\rm{m}}\) (c).

Full size image

As a first demonstration, we performed conventional AOI with the second diffuser in place by scanning the ultrasound focus along the transverse (x) axis, over the path marked by the dashed orange line in Fig. 2a, with a step size of \(10\ \upmu {\rm{m}}\). Different from conventional AOI, for each ultrasound focus position, \({{\bf{r}}}_{{\bf{us}}}\), we recorded m = 160 different ultrasound-modulated light intensities, \({P}_{i}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}})\), each with a different (unknown) speckle realization, \({S}_{i}({\bf{r}})\) (i = 1..m), obtained by rotating the controlled diffuser placed before the sample. Figure 2b shows the ultrasonically-tagged optical intensity distribution at the object plane.

The experimental results obtained with this setup for two different speckle grain size are presented in Fig. 2. An example for the raw detected acoustically-modulated light pattern at the camera plane, for one ultrasound position \({{\bf{r}}}_{{\bf{us}}}\) and speckle realization \(i\), which is a random speckle pattern, is shown in Fig. 2c. For each measurement, the total acoustically-modulated power, \({P}_{i}^{{\mathrm{AO}}}({{\bf{r}}}_{{\bf{us}}})\), is calculated from this pattern by integration over all camera pixels. To demonstrate super-resolution AOI, the first three statistical cumulants of \({P}_{i}^{AO}({{\bf{r}}}_{{\bf{us}}})\) were calculated for each ultrasound focus position, \({{\bf{r}}}_{{\bf{us}}}\). Figure 2d shows the one-dimensional AOI traces obtained using our proposed approach. While the first cumulant (realizations average), having the conventional AOI resolution, does not resolve the target features, the higher-order cumulants clearly resolve the two target lines. To display a fair comparison between the different cumulants orders, each of the plotted traces is the \({N}^{th}\)-root of the \({N}^{th}\)-order cumulant. As in conventional SOFI, additional resolution improvement, up to a factor of \(N\) for the \({N}^{th}\)-order cumulant, can be obtained by deconvolving the \({N}^{th}\)-order cumulant trace with the \({N}^{th}\) power of the measured acoustic PSF, \(h({\bf{t}})\)23. Figure 2e presents the results of a such a deconvolution, performed using a Richardson-Lucy deconvolution on the experimentally measured cumulant traces. The resolution increase in the high-order cumulants is enhanced in the deconvolved traces, as expected.

Different from conventional SOFI in fluorescence microscopy19, where the labelling concentration of the fluorescent molecules is controlled by sample preparation, in AOI the number of fluctuating speckle grains contained inside the acoustic focus, \({N}_{{\mathrm{speckles}}}\), is determined by the ratio between the ultrasound focus size, \({D}_{{\mathrm{US}}}\), and the size of the optical speckle grains, \({D}_{{\mathrm{speckle}}}\): \({N}_{{\mathrm{speckles}}}\approx {({D}_{{\mathrm{US}}}/{D}_{{\mathrm{speckle}}})}^{2}\) for a completely resolution cell. The larger is the number of uncorrelated fluctuating speckles contributing to the AOI signal, the smaller is the relative amplitude of fluctuations compared to the mean measured AOI signal. Quantitatively, the AOI signal standard-deviation is \(\sqrt{{N}_{{\mathrm{speckles}}}}\)-times smaller than the mean AOI signal. Thus, for given experimental signal-to-noise conditions, applying SOFI to AOI is expected to be more challenging the larger is \({N}_{{\mathrm{speckles}}}\), as shown in Supplementary Fig. 5. However, more importantly, \({N}_{{\mathrm{speckles}}}\) affects also the statistical estimation accuracy of the cumulants, even in the absence of measurement noise, as we explain below.

The results of Fig. 2a-e were obtained with an optical speckle grain size of \(\sim \!12.3\ \upmu {\rm{m}}\) at the target plane (FWHM, see Methods), yielding \({N}_{{\mathrm{speckles}}}\approx 500\). The speckle grain size in this first experiment was set by controlling the beam diameter on the rotating diffuser and removing the first diffuser in the sample. While the speckle grains in this experiment are considerably smaller than the ultrasound focus, they are still considerably larger than the speckle grain size expected deep inside volumetric scattering samples such as tissue, where the speckle grain size is of the optical wavelength scale. Considering high frequency (>50 MHz) ultrasound, and infrared laser illumination, \({N}_{{\mathrm{speckles}}} \, > \sim \!2500\) speckle grains are expected to be contained within the ultrasound focus in deep-tissue AOI experiments.

To experimentally demonstrate our approach with smaller speckle grains, we have repeated the experiments of Fig. 2a-e with a speckle grain size of \(\sim 3.4\ \upmu {\rm{m}}\), by mounting all three diffusers in place. This speckle grain size results in \({N}_{{\mathrm{speckles}}}\approx 6600\) over the entire area of the ultrasound focus. The results of this experiment are shown in Fig. 2f-i. To minimize the effective number of contributing speckle grains we have chosen to use laser pulses having a temporal duration that is significantly shorter than \(1/{f}_{us}\). Such a choice results in an acousto-optic PSF that is axially-modulated by the ultrasound wavelength, as can be seen in Fig. 2b, g and Supplementary Fig. 2a, having an effective area that is two times smaller than that of the acoustic PSF using long laser pulses16,17. The results of the second-order cumulant display a clear resolution increase both for the raw trace and its deconvolution. In order to validate that the experimental resolution improvements are in-line with the theoretical prediction of SOFI, we have compared the experimental results to a numerical convolution of the object with the measured PSF and the measured PSF squared. These results, shown in Supplementary Fig. 3, confirm that the second-order cumulant indeed allow for the theoretical resolution improvement. We have focused our validation on the second-order cumulant as it is less sensitive to speckle grain size and number of realizations as the higher-order cumulants (see the following discussion and Fig. 3).

Fig. 3: Numerical investigation of cumulants estimation error for noise-free measurements.
figure 3

a Relative estimation error of the \({N}^{th}\) root of the first three cumulants as a function of the number of speckles, \({N}_{{\mathrm{speckles}}}\), contained inside the acoustic focus, for 200 realizations (continuous lines) and 2,000 realizations (dashed lines). blue: first cumulant, red: second cumulant, orange: third cumulant. b, c Simulated acousto-optic imaging (AOI) reconstructions for noiseless measurements with 400 realizations, for two different values of \({N}_{{\mathrm{speckles}}}\), marked by the dashed green lines in a: b \({N}_{{\mathrm{speckles}}}=6\), and c \({N}_{{\mathrm{speckles}}}=4000\). The larger estimation errors of higher-order cumulants in c are in-line with the experimental results along the transverse (x) axis for the small speckle grain size.

Full size image

However, while the second cumulant trace resolves the two object lines with improved contrast, the third-order cumulant for the small speckles case is corrupted by strong artifacts. Interestingly, and of high practical importance, the increased noise in the third (and higher-order) cumulants is not a result of experimental SNR conditions, but represents a fundamental limitation of SOFI, originating from the very nature of the cumulant statistical analysis24. While SOFI promises a potential infinite resolution increase for increasing cumulants order, such an increase is only possible for an infinite number of speckle realizations (ensemble size), even in the absence of measurement noise. For a finite number of realizations (finite sample size), the inherent statistical estimation errors of higher-order cumulants24 present the dominant source of image artifacts in SOFI.

High-order cumulants estimation limitations

To quantitatively study the theoretical and practical limitations of high-order cumulants estimation on our approach as a function of \({N}_{{\mathrm{speckles}}}\), and number of realizations, we have performed a set of numerical simulations in the noise-free case. The results of this study are presented in Fig. 3 and Supplementary Fig. 4. Additional results that include the effect of measurement noise are given in Supplementary Fig. 5. Details on the simulations are given in Supplementary Notes 3 and 4.

Figure 3a presents the relative estimation errors of the \(N\)-th root of the first three cumulants as a function of \({N}_{{\mathrm{speckles}}}\), for m = 200 and m = 2000 realizations. The estimation error of the third-order cumulant grows as a function of \({N}_{{\mathrm{speckles}}}\), while the estimation errors of the second cumulant (the standard deviation) is largely insensitive to \({N}_{{\mathrm{speckles}}}\). The effect of the estimation errors on simulated AOI traces, in the absence of measurement noise, is presented in Fig. 3b, c. While the effect is small for a small number of speckles (\({N}_{{\mathrm{speckles}}}=6\), Fig. 3b), it is dominant for a large number of speckle (\({N}_{{\mathrm{speckles}}}=4000\), Fig. 3c). Thus, obtaining high quality AOI images using high-order cumulants can only be achieved in practical AOI conditions by averaging a large number of measurements, or when a small number of speckles are transmitted through the target objects, i.e., for sparsely transmitting (mostly absorbing) objects.

The inherent difficulty in accurately estimating the high-order statistical cumulants for large \({N}_{{\mathrm{speckles}}}\) can be intuitively understood from the central-limit theorem: since the measured AOI signal is the sum of \({N}_{{\mathrm{speckles}}}\) independent random variables, the larger is \({N}_{{\mathrm{speckles}}}\), the closer is the distribution of the AOI fluctuations in a given spatial position to a Gaussian distribution, whose cumulants of orders three and above are equal to zero.

Another, more conventional, factor that affects the estimation accuracy is measurement noise. For the approach to work, the fluctuations amplitude has to be larger than the measurement noise. We study the effect of measurement noise numerically in Supplementary Fig. 5. A larger number of realizations, \(m\), can be used to mitigate the effects of both estimation errors and measurement noise, sacrificing acquisition time.

Experimental results for axial scanning

To demonstrate the resolution increase also along the ultrasound propagation direction, i.e. along the vertical (y) axis in Fig. 1a, we have performed an experiment where the ultrasound focus was scanned along the ultrasound propagation axis. The experiment is similar to the one presented in Fig. 2 but with the ultrasound position varying by adjusting the relative delay between the ultrasound pulses and the optical laser pulses, and rotating the target by 90 degrees. In addition, shorter, 170 ns-long ultrasound pulses were used to obtain a smaller initial ultrasound focus in the axial direction. The ultrasound focus dimensions in this experiment were \(\Delta {\rm{Y}}\ =\ 260\ \upmu {\rm{m}}\), \(\Delta {\rm{X}}\ =\ 350\ \upmu {\rm{m}}\) full-width at half max (FWHM), at the axial and transverse directions, respectively (Supplementary Fig. 2c, d).

The experimental results of this experiment for an axial step size of \(10 \, \upmu {\rm{m}}\), and an optical speckle grain size of \(\sim \!\! 11\ \upmu {\rm{m}}\) at the target plane are presented in Fig. 4. The target object shown in Fig 4a is the same as in Fig. 2a, f. The ultrasonically-tagged optical intensity distribution at the object plane for one position (xus, yus) of the ultrasound focus is shown in Fig 4b. Figure 4c shows the ultrasound tagged intensity as imaged directly without the target object, averaged over 40 speckle realizations. As expected, similar to the results of transverse scanning (Fig. 2), while the first cumulant has a difficulty to resolve the dip between the two lines, the higher-order cumulants provide an improved resolution where the dip is visible. Interestingly, comparing the results of Fig 2d, h to the results of Fig. 4d, one can observe that the scanned profile in the axial direction is less smooth. This is due to the cosine modulation of the acoustic focus in the vertical axis (Supplementary Fig. 2d), resulting from the sub-nanosecond optical pulses used in our experiments, which effectively ’freeze’ the ultrasound propagating pulse.

Fig. 4: Experimental demonstration of super-resolution acousto-optic imaging (AOI) along the ultrasound propagation direction (y-axis).
figure 4

a target object, \(I(x,y)\), illuminated by a single speckle realization, \({S}_{i}(x,y)\), as imaged directly without the second scattering layer. b ultrasonically-tagged optical intensity distribution at the object plane, for one position (\({x}_{us},{y}_{us}\)) of the ultrasound focus, \(h(x,y)\), visualizing the ultrasound focus size. c ultrasound focus intensity as imaged directly without the target object, averaged over 40 speckle realizations. d AOI profile obtained by scanning the ultrasound focus along the dashed line in a from top to bottom: blue: conventional AOI (first cumulant); red: square-root of the second-order cumulant of the acousto-optic fluctuations, showing improved resolution; orange: third-order cumulant; black, dashed: cross-section of the target object, along the dashed line in a. Acoustic focus is \(260\ \upmu {\rm{m}}\) FWHM wide along scanning axis. scale-bars: \(280\ \upmu {\rm{m}}\).

Full size image

Discussion

We have presented proof of principle results for super-resolved AOI using dynamic speckle fluctuations, and have analyzed the sensitivity of the approach to measurement noise and the inherent estimation errors of finite statistics.

Another important factor that affects the effective measurement accuracy in AOI is the number of optical modes (speckles) that are detected by the AOI system. Equation (1) implicitly assumes ideal acousto-optic detection, i.e., an AOI detector that detects all of the acoustically-modulated light power (collecting all of the scattered optical modes). However, in practice, any optical detector has a limited etendue, which can be characterized by the number of optical modes (speckle grains), \({N}_{{\mathrm{det}}}\), that can be detected by it. For the digital-camera based system used in our experiments, the number of camera pixels poses the limit on the number of speckle grains that can be detected with a high signal to noise (SNR). In deep-tissue imaging, the number of total speckles in the scattered field outside the sample may be considerably larger than \({N}_{{\mathrm{det}}}\). In such conditions, the relative error of the total measured intensity, \({P}^{{\mathrm{AO}}}\), only due to speckle statistics is: \(\epsilon =\Delta {P}^{{\mathrm{AO}}}/{P}^{{\mathrm{AO}}}\approx 1/\sqrt{{N}_{{\mathrm{det}}}}\). This error is only due to speckle statistics (detecting \({N}_{{\mathrm{det}}}\) speckles from a field containing a much larger number of speckles), and is added to any other noise sources. In order to be able to accurately measure small fluctuations, a large number of speckles needs to be detected at the camera plane. For example, for the second SOFI cumulant (i.e., the variance), the ratio of the square root of the second cumulant to mean measured intensity is: \(\sqrt{{C}_{2}}/{C}_{1}=\sqrt{{C}_{2}}/{P}^{AO}\approx 1/\sqrt{{N}_{{\mathrm{speckles}}}}\). Thus the SNR for measuring the square root of the second cumulant in the case of no measurement noise is: \({\mathrm{SNR}}=\sqrt{{C}_{2}}/\Delta {P}^{{\mathrm{AO}}}=\sqrt{{N}_{{\mathrm{det}}}/{N}_{{\mathrm{speckles}}}}\). To allow super-resolved AOI using our approach, the experimental AOI system should thus provide \({N}_{{\mathrm{det}}}\gg {N}_{{\mathrm{speckles}}}\), i.e., to detect more optical modes than the number of modes transmitted through the ultrasound focus. This condition can be met with modern megapixel-count cameras, or nonlinear-crystals based detection25.

Our approach is general and can be applied to any conventional AOI system. It does not necessitates wavefront-measurements, spatial light modulatorS (SLM), complex computations, nonlinear effects, or memory-effect speckle correlations26, as required by alternative approaches. Most importantly, our approach is designed to work with short speckle decorrelation times, which is one of the major limiting factors of current approaches15,16,17.

While our approach is general and relieves the requirements on the speckle decorrelation time, it relies on multiple measurements at each position of the ultrasound focus, assuming the imaged structure remains static within the acquisition time. While this requirement for fast speckle decorrelation and static structure is not fulfilled by all natural samples, it may be naturally fulfilled by fast flowing blood in relatively static blood vessels11, or alternatively in ex-vivo samples or in industrial applications. In the case that the natural speckle decorrelation time is not short enough, fast MEMS-based modulation can be used to achieve a sufficiently short speckle decorrelation time27, assuming a fast enough acousto-optic acquisition system is used.

The requirement for multiple speckle realization significantly increase the acquisition time, a fundamental aspect shared with most super-resolution imaging approaches. In our proof-of-principle experiments, we have performed 160 measurements at each spatial position to study the potential of higher cumulants analysis, using a setup that was not optimized for speed. Thus, the current acquisition time, composed of 24 phase-shifting images per measurement, was 2 seconds per realization, excluding diffuser rotation and data processing. The number of measurements can be reduced by a factor of two if one is only interested in estimating the second-order cumulant, as we study in Supplementary Note 5 and in Supplementary Fig. 6 using the experimental data of Figs. 2 and 4. Moreover, the acquisition time can be shortened by orders of magnitude using faster detection approaches, such as lock-in camera detection28, and fast cameras29. Another approach for improved acquisition time is using an ultra-fast plane-wave AOI approach25, based on nonlinear crystals.

We have demonstrated the use of basic cumulants for super-resolution AOI. Additional statistical analysis techniques, such as cross-cumulants analysis23, and balanced SOFI30, could further improve the resolution and image quality. Other, more advanced algorithms, such as compressed-sensing sparsity-based reconstruction31, should improve the resolution, the reconstruction fidelity, and decrease the number of required measurements, i.e. the acquisition time, even further, as was recently demonstrated in PAT10,32.

We used a controllable diffuser to produce controlled speckle decorrelation. This may not be required in in vivo imaging, where the natural speckle decorrelation caused by blood flow or tissue decorrelation may provide the source for fluctuations11, turning the natural sample dynamics into a positive effect in AOI.

Methods

Acousto optic system components

The experimental setup used in this work is detailed and shown schematically in Supplementary Note 1. The ultrasound focus was provided by an ultrasound transducer (V315, Olympus, F\(\#=1.33\)), driven by 300 ns long sinusoidal pulses at \({f}_{us}=10\ {\rm{MHz}}\) central frequency, with a \(150 \, {V}_{{\mathrm{pp}}}\) amplitude for the experiments of Fig. 2, and 170 ns long pulses with \(180 \, {\rm{V}}_{\rm{pp}}\) for the experiment of Fig. 4.

Scanning process

The ultrasound focus position was horizontally scanned using a motorized translation stage (Thorlabs), and its vertically position was controlled by adjusting the relative delay between the ultrasound pulses and the optical pulses from the q-switched laser (Standa STA-01, providing 1 ns long pulses, at 25 KHz repetition rate, 532 nm wavelength). For a field of view of 1 mm, measurements at 100 positions were taken. In each measurement position, 160 different speckle realizations were acquired, resulting in 160 \(\times\) 100 total number of measurements, i.e., 16,000 measurements. The ultrasound-modulated light was detected via off-axis phase-shifting digital holography33, using a high-resolution sCMOS camera (Zyla 4.2 Plus, Andor), and a frequency-shifted reference arm.

Estimation of speckle grain size

The optical speckle grain size was calculated from a direct image of the speckle at the target object plane as imaged using an imaging lens by a camera. The image of the optical speckle was autocorrelated, and the autocorrelation full width at half max (FWHM) was measured and divided by \(\sqrt{2}\) to estimate the speckle grain FWHM, assuming Gaussian autocorrelation.