Abstract
Studying the internal structure of complex samples with light is an important task but a difficult challenge due to light scattering. While the complex optical distortions induced by scattering can be effectively undone if the medium’s scatteringmatrix is known, this matrix generally cannot be retrieved without the presence of an invasive detector or guidestar at the target points of interest. To overcome this limitation, the current stateoftheart approaches utilize focused ultrasound for generating acoustooptic guidestars, in a variety of different techniques. Here, we introduce the acoustooptic transmission matrix (AOTM), which is an ultrasonicallyencoded, spatiallyresolved, optical scatteringmatrix. The AOTM provides both a generalized framework to describe any acoustooptic based technique, and a tool for light control and focusing beyond the acoustic diffractionlimit inside complex samples. We experimentally demonstrate complex light control using the AOTM singular vectors, and utilize the AOTM framework to analyze the resolution limitation of acoustooptic guided focusing approaches.
Similar content being viewed by others
Introduction
Conventional optical focusing and imaging techniques fail in strongly scattering media because of the multiplescattering events that any incident optical beam undergoes in its propagation inside them. However, the complex wavefront distortions, even deep inside diffusive samples, can be effectively reversed by highresolution shaping of the input optical wavefront^{1}, in a fashion analogous to timereversal experiments in ultrasound^{2}. The ability to digitally control optical interference in multiplyscattering media using spatial light modulators (SLMs) has recently given rise to new focusing and imaging techniques^{3,4}. Following the pioneering work of Vellekoop and Mosk^{5}, wavefront optimization was used for correcting spatial^{6,7,8,9}, temporal^{10,11}, spectral^{12}, and polarization^{13} distortions, and to optimize transmission^{14} through multiplyscattering media. A generalized theoretical framework underlying all experiments involving light propagation in complex media is the reflection/transmission matrix (TM) formalism^{6,15,16,17}. The TM essentially contains the medium’s response at every output spatial mode to excitation by any input spatial mode, i.e., the medium’s set of Green functions. Experimental access to the optical TM was first made possible by measuring the response from each pixel of an SLM placed at a chosen input plane to each pixel of a camera placed at a desired output plane^{6}. However, while such experimentally measured TM allowed focusing and imaging, this could only be done at the camera output plane, located outside the scattering sample. Thus, severely limiting its practical use for noninvasive imaging inside complex samples, where the goal is to perform imaging without direct access to the target plane. To control scattered light inside scattering samples a “guide star” providing feedback on the optical intensity at the target point is required^{4}. While the singular value decomposition (SVD) of an alloptical TM was used to focus light on isolated reflective targets^{18,19,20}, such alloptical approaches are not capable of focusing light unless an isolated, strongly reflective target is present^{16,18}. Moreover, even in the presence of such a reflective target, the obtained focus is of the target dimensions, and not the desired optical diffraction limit^{20}. The requirement for an isolated reflective target can be overcome using advanced mechanisms for guidestars, based on nonlinear particles^{21,22,23}, photoacoustic feedback^{24,25,26}, and acoustooptic tagging^{27,28,29,30,31,32,33}. The last two techniques combine light and sound to benefit from the near scatteringfree propagation of ultrasound in optically scattering samples. In photoacoustics, the interaction of light with absorbing parts of the sample generates ultrasound waves, which allows ultrasonic imaging and optical focusing on optical absorbers. In acoustooptics, the requirement for optical absorption is removed by locally modulating the diffused light inside the sample using a focused ultrasound beam^{34}. Detection of the acoustically modulated (frequency shifted) light enables measuring only the light that has traveled through the ultrasonic focal spot, with a spatial resolution given by the ultrasound focus dimensions. Light can also be focused back into the ultrasound focus via optical phaseconjugation of the tagged light in “timereversed ultrasonically encoded” (TRUE) optical focusing^{27,28,30}, or via iterative optimization^{33}. Acoustooptic tagging holds many advantages, being a noninvasive, threedimensional (3D)positionable, labelfree, nonionizing guidestar. Its main drawback, however, is that the spatial dimensions of the ultrasound focus are given by the ultrasonic wavelength, yielding a resolution that is orders of magnitude inferior than the optical diffraction limit^{4}.
To date, two approaches to overcome the acoustic diffraction limit in acoustooptics have been put forward: iterative TRUE (iTRUE)^{31,32}, and time reversal of varianceencoded (TROVE) optical focusing^{29}. In TROVE the intensity fluctuations of ultrasonically tagged light for different random inputs are analyzed, and an optical wavefront that focuses to the location with increased fluctuations is computed, allowing, in principle, optical speckle size focusing^{29}. In iTRUE, multiple iterations of phaseconjugation are used to improve the focusing resolution^{31,32}. At each iteration the optical beam is refocused to the ultrasonically tagged region, modulated again by the focused ultrasound, and thus shrinks progressively. N iterations of iTRUE are required for a √N resolution increase beyond the acoustic diffraction limit. Both iTRUE and TROVE rely on a digital optical phaseconjugation (DOPC) system^{35}, a rather complex apparatus, which conjugates a highresolution SLM and a camera. DOPC systems requires a very precise pixeltopixel alignment that can be experimentally challenging^{35}. Nonetheless, DOPC is until now the only tool that allows to overcome the acoustic diffraction limit in acoustooptic guided optical focusing.
Here, we introduce a novel generalized concept for light control using acoustooptic guidance: the acoustooptic transmission matrix (AOTM). Our concept is based on the understanding that any experiment that utilizes linear ultrasound tagging can be described by a single linear operator, and that this operator can be described by a single matrix. We show that a single AOTM provides a general, concise, and full description of light propagation in any acoustooptic experiment. We experimentally demonstrate how the AOTM can be measured using a single or multiple ultrasonic focused beams, and how its SVD allows for subacoustic optical focusing and light control inside a complex medium, without a DOPC system. We use the AOTM to show how all present acoustooptic techniques, namely TRUE, iTRUE, and TROVE, can be described under the same framework, and how the AOTM enables performing TRUE, iTRUE, and TROVE experiments without a DOPC system, i.e., using any arbitrary positioning of an SLM and a camera. We also utilize the AOTM framework to analyze the resolution limitation of SVDbased acoustooptic guided focusing approaches.
Results
Principle of the AOTM with a single ultrasonic beam
Consider a general acoustooptical experiment, such as the one schematically depicted in Fig. 1a, where diffused, quasimonochromatic light, is ultrasonically tagged by a focused ultrasound beam, and subsequently measured by a camera placed outside the medium. Assuming linear light propagation and linear acoustooptic interaction, the relationship between any input optical field \(E^{{\mathrm{in}}}(f_{\mathrm{o}})\) at the optical frequency f_{o}, and the ultrasonically tagged output field at the camera plane, \(E^{{\mathrm{out}}}(f_{\mathrm{o}} + f_{{\mathrm{US}}})\), which is frequency shifted to a frequency f_{o}+f_{US} by the ultrasound beam at frequency f_{US}, is given by a linear operator T_{u}:
where u is an index describing the ultrasound focus position. We define the matrix describing this operator in the spatial domain as the AOTM. Specifically, each element of the AOTM, t_{u,mn}, gives the complex amplitude of the acoustically tagged optical field at the output spatial position r_{m}, \(E_m^{{\mathrm{out}}}\), as result of an input field at position r_{n}, \(E_n^{{\mathrm{in}}}\), and an ultrasound modulating focus positioned at r_{u} (an acoustooptic “Green function”). Importantly, r_{m}, is located at the camera plane, outside of the scattering medium, and r_{u} is located inside the medium. As a result of linearity, the output acoustically tagged field measured at r_{m} for a general input field is:
where the summation is done over all spatial input modes, n. Fig. 1a depicts the basic setup required for acquiring the AOTM and subsequently exploiting it for light control. The setup is based on the wellestablished approach for measuring the optical transmission matrix (TM)^{6}, with the addition of an ultrasound transducer that generates an ultrasound pulsed focus inside the medium, and a pulsed optical reference arm for offaxis holography. The setup is composed of an illuminating laser beam that passes through a computercontrolled SLM to provide injection of controlled optical modes into the medium. The output acoustically tagged scattered fields at a frequency f_{o} + f_{US} are measured outside the medium via offaxis, phaseshifting holography^{36}, using the pulsed reference beam that is synchronized with the ultrasound pulses (see Methods).
The AOTM is acquired by sequentially injecting each of the n = 1..N_{SLM} input modes, \(E_n^{{\mathrm{in}}}(f_{\mathrm{o}})\), having the laser frequency f_{o} into the medium (where N_{SLM} is the number of SLM pixels used), and measuring the frequencyshifted output field \(E_m^{{\mathrm{out}}}\left( {f_{\mathrm{o}} + f_{{\mathrm{US}}}} \right)\) at each of the m = 1..M camera pixels simultaneously. Following Equation (2), in each of the n = 1..N_{SLM} measurements steps, the n^{th} column of T_{u} is acquired, giving a matrix of dimensions M × N_{SLM} (Fig. 1b). The measured AOTM thus describes the propagation of light from the input plane of the SLM, through the small ultrasound focus inside the medium, to the camera plane. While an alloptical TM that is measured without the ultrasound focus^{6}, reflects the combined interference of all optical paths inside the scattering medium, using the AOTM, the spatially localized ultrasound focus allows spatially resolved probing and focusing in a specific target volume inside the medium, as we show below.
Once the AOTM is acquired (as is simulated in Fig. 1 using a random independent and identically distributed (i.i.d.) scattering matrix model for a multiplyscattering medium, see Methods), it can be used for optical focusing inside the medium by several different approaches (Fig. 1d–p). The most straightforward but least powerful approach is via direct phaseconjugation: displaying the phase conjugate of the m^{th} row of the AOTM on the SLM (dashed orange box in Fig. 1b, Fig. 1e), leads to light focusing at the m^{th} camera pixel (Fig. 1g), and as result of the ultrasound tagging, also to light concentrating inside the acoustic focus (Fig. 1f). The dimensions of the intensityenhanced volume inside the sample are the dimensions of the ultrasound focus, as would be obtained in TRUE focusing experiments^{27,28,30}. This is not a coincidence but rather a deeper result obtained from optical reciprocity: phaseconjugating an AOTM row mathematically describes a TRUE experiment with a virtual initial illumination at the m^{th} camera pixel (see Supplementary Note 2). The peak to background ratio (PBR) of the formed focus at the ultrasound focus plane inside the scattering medium is also the same as theoretically expected from TRUE focusing, i.e., PBR ≈ N_{SLM}/N_{s}, where N_{s} is the number of speckle grains contained in the ultrasound focus. The PBRs obtained in the simulations of Fig. 1 with a phaseonly SLM having N_{SLM} = 4096 pixels are in good accordance with these expected values.
A considerably more powerful optical focusing and control approach that allows surpassing the acoustic diffraction limit, is via SVD of the AOTM. SVD is a powerful tool in matrix analysis, which was recently used to identify transmission eigenchannels^{37,38,39,40}, and for selective focusing^{18,25}. The main interest in the SVD of the AOTM is that the first singular vector of the AOTM gives the optimal wavefront for the tightest optical focusing, as would be obtained after infinite iterations of iTRUE^{31,32}. This important result is proved in detail in Supplementary Note 3. A short intuitive explanation is that since the AOTM, T_{u}, describes a single pass through the medium and ultrasound focus, the matrix \(T_u^{\mathrm{H}}T_u\) (where ^{H} is the Hermitian conjugate) describes two iterations of phaseconjugation of iTRUE going back and forth through the ultrasound focus. Performing 2k additional iterations of iTRUE is then mathematically described as taking the matrix \(T_u^{\mathrm{H}}T_u\) to kth power. Thus, iTRUE iterations are in fact power iterations of the matrix \(T_u^{\mathrm{H}}T_u\). Since performing an infinite number of power iterations is the mathematical approach for finding the largest eigenvalued eigenvector of \(T_u^{\mathrm{H}}T_u\). This eigenvector is, by definition, the first singular vector of the AOTM, T_{u}, providing the sharpest optical focus. The fact that the SVD of the AOTM provides sharp focusing is due to the gaussianlike shape of the ultrasound tagging focus. This is in contrast to the SVD of the alloptical TM (Fig. 1c, gray dots), which yields transmission eigenchannels whose injection into a scattering medium does not lead to spatial focusing^{20,38}.
Numerical results for injecting the first singular vector of the AOTM show the formation of a tightly focused spot at the center of the acoustic focus (Fig. 1h–j). The focus size reaches the optical diffraction limit (i.e., a single optical speckle grain) if a sufficiently large number of input modes, N_{SLM}, are measured, where the required N_{SLM} is larger for larger ultrasound focus size, or smaller speckle grain size (see Supplementary Notes 5–6). Importantly, not only the first singular vector of the AOTM is of interest: singular vectors with decreasing singular values will form concentric rings with increasing diameter around the center of the ultrasound focus (Fig. 1c insets, Fig. 1k–p). Using low singular values leads to concentration of energy outside the acoustic focus, in a fashion resembling open channels in systems containing localized absorption^{40}. This resemblance to open channels may be understood from the fact that the singular values of the AOTM represent the energy transmission through the acoustic focus. The distribution of singular values of the AOTM (Fig. 1c—blue dots) is dictated by the shape and size of the acoustic focus, and is very different than the general Marčenko–Pastur distribution of the alloptical TM (Fig. 1c—gray dots). The number of significant singular values of the AOTM is approximately the number of optical modes (speckles) contained inside the acoustic focus, since it is the number of optical modes that decompose the effective virtual “aperture” that is formed by the ultrasound focus. For the Gaussianshaped ultrasound focus considered in Fig. 1 the singular values decrease gradually. Different shaped ultrasound foci would result in different distributions (see a numerical example in Supplementary Note 4).
AOTM using a single ultrasonic beam
To experimentally demonstrate our approach we used the setup schematically described in Fig. 2a, (see Methods and Supplementary Note 1). As was used in several recent works^{29,41,42}, in our proofofconcept experiments the sample was made of two optical diffusers separated by a distance to allow access to the focusing plane. A pulsed focused ultrasound transducer with a central frequency of 15 MHz placed perpendicular to the light propagation direction was used for acoustooptic tagging. Similar to recent works^{29,42}, a relatively large speckle grain size (∼ 30 μm) compared to the optical wavelength was chosen as a convenient proof of principle for singlespeckle grain focusing using the few thousands of input modes, N_{SLM}, that can be experimentally measured with our setup within the samples decorrelation time. Achieving singlespeckle grain focusing with smaller sized speckles, requires larger N_{SLM} (see Supplementary Notes 5–6). However, a too small N_{SLM} would still provide a focus size that is smaller than the ultrasound diffraction limit and a higher PBR than TRUE (see Supplementary Figures 6–9). Fig. 2 presents the experimental results of optical control using the SVD of a measured AOTM. Supplementary Figure 10 displays the AOTM measured in this experiment and the distribution of its singular values. As expected, injecting the first singular vectors generates a sharp optical focus with dimensions smaller than the acoustic focus (Fig. 2c, d). Injecting singular vectors with lower singular values results in the formation of concentric rings of increased optical intensity around the center of the ultrasound focus (Fig. 2e, f). Using low singular values leads to concentration of energy outside the acoustic focus (Fig. 2g–i). The fullwidth at halfmax (FWHM) transverse horizontal dimensions of the ultrasound focus (Fig. 2b) and the sharpest focus formed via SVD of the AOTM (Fig. 2c) are 170 μm ± 10 μm and 35 μm ± 5 μm, respectively. The vertical dimensions of these foci (along the ultrasound axial dimension) are 175 μm ± 20 μm and 35 μm ± 5 μm, respectively. Thus, the SVD of the AOTM provides here a resolution increase beyond the acoustic diffraction limit of approximately 4.8. We note that the AOTM approach is general and is not limited to thin scattering layers. This is proved numerically in Fig. 1, where light propagation in the multiplescattering diffusive regime was simulated, by considering random i.i.d. matrices for modeling the light propagation in a random multiplyscattering sample (see Methods).
AOTM using two ultrasound beams
The first singular vector of a single AOTM (Figs. 1–2) allows focusing only at a single point located at the center of the ultrasonic focus. Given the large number of measurements required for such focusing, this forms a limitation for the use of this approach for applications such as imaging. However, as was recently shown by Judkewitz et al. in their TROVE work^{29}, a joint analysis of several matrices of acoustooptically modulated output modes, measured for different ultrasound foci positions, can allow focus scanning. Here, we apply the same mathematical approach to jointly decompose multiple AOTMs measured for two different ultrasound focus position. Specifically, we exploit the joint analysis of two AOTMs to perform a scan of a tight optical focus over multiple positions. For this purpose, we consider the separate measurement of two AOTMs, T_{1} and T_{2}, using two different ultrasound focused beam that are spatially shifted such that the two ultrasound focal spots, P_{1}(x,y) and P_{2}(x,y), partially overlap (Fig. 3a–d). Such ultrasound focal spots are easily obtained in the same experimental setup presented in Fig. 2 by changing the timedelay, Δτ = z/v_{US}, between the ultrasound pulse and the optical pulses (Fig. 3a), where z denotes the axial distance of the acoustic focal spot from the ultrasound transducer, and v_{US} is the speed of sound in the medium. Injecting the first singular vector of T_{1} or T_{2} would only form a tight optical focus at the center of each of the two ultrasound foci. However, the joint information in the two matrices can be exploited to scan even a sharper focus along the axis connecting the centers of the two acoustic foci (Fig 3i–k).
Noting that the SVD of a single AOTM, T, is obtained by diagonalizing the operator T^{H}T, referred in ultrasound experiments as the timereversal operator (TRO)^{43,44,45}, and following the multifocus analysis of TROVE^{29}, we used the first eigenvectors or singular vectors of the following generalized TRO, A_{α}, to form a focus at different controlled positions:
where α is a positive weighting parameter controlling the focus position. α = 1 yields focusing at the middle of the line connecting the two ultrasound foci centers (Fig. 3i). The matrix A_{α} is the multiplication between the TRO of the weighted sum of T_{1} and T_{2}: \(T_{1 + \alpha 2} = \left( {T_1 + \alpha T_2} \right)\) (see Fig. 3d), by the inverse of their weighted difference: \(T_{1  \alpha 2} = \left( {T_1  \alpha T_2} \right)\). Scanning the tight focus is made possible because, as result of linearity, the difference matrix: \(T_{1  2} = \left( {T_1  T_2} \right)\) describes an AOTM of a virtual ultrasound focus that is the subtraction of the first ultrasound focus pressure field from the other (Fig 3e). This difference acoustic pressure field is equal to zero at a specific distance between the two ultrasound foci centers (Fig. 3e). Dividing the sum of the two ultrasound foci (Fig. 3d) by their difference (Fig. 3e) results in a sharp peak at this distance (Fig. 3f). The position of the sharp peak is controlled by the parameter α. In practice, to take into account measurement noise, the matrix inversion in the calculation of A_{α} is performed via a pseudoinverse (^{+}) with a proper regularization parameter^{46}.
Figure 3i–k present numerical results obtained with this approach in a multiplyscattering medium, along with a comparison to TRUE focusing (Fig. 3g, h). Fig. 4 presents experimental results of a proofofprinciple experiment with two scattering layers. It can be noticed that focusing using the joint information of the two matrices (Fig. 4e–h) yields a sharper focus than that obtained via SVD of each matrix separately (Fig. 4d). This is made possible since the virtual ultrasound focus obtained by dividing by pressure difference T_{1−α2} (Fig. 3f) is smaller than each of the single ultrasound foci (Fig. 3b, c). This result can be extended to allow scanning in two or three dimensions using more ultrasound beams, as was demonstrated by Judkewitz et al. using four ultrasound foci^{29}.
Interestingly, the analogy between the results obtained by SVD of the AOTM and those obtained with TROVE is not coincidental. The underlying reason is that the variance maximization approach of TROVE is, in fact, based on performing an SVD to a measured matrix of acoustically modulated output modes that is very similar to the AOTM: in TROVE, the largest variance mode is found by diagonalizing a spatial covariance matrix in the form of C^{H}C, where C is a matrix of output fields measured for random input fields. Such diagonalization is mathematically equivalent to performing an SVD of the matrix C. The largest variance mode of TROVE is thus identical to the highest singular valued vector of the AOTM. The only mathematical difference between TROVE and the AOTM is that the matrices used in AOTM are measured using known orthogonal input modes, while in TROVE the input modes are random and unknown. As a result, the two approaches provide comparable focusing performance. While mathematically similar, the experimental implementation of the AOTM and TROVE is quite different: a practical advantage of the AOTM is that it is not based on a DOPC system, and thus it allows any arbitrary positioning of the SLM and the camera, and any arbitrary pixelcount in each, removing the strict alignment constraints of the DOPC systems required in TROVE and TRUE^{35}.
In the experimental results of Fig. 4, the FWHM transverse dimensions of the ultrasound foci (Fig. 4a, b) and the sharpest focus obtained by SVD of A_{α} (Fig. 2f) are 180 μm ± 10 μm, and 50 μm ± 5 μm, respectively. The vertical dimensions of these foci (along the ultrasound axial dimension) are 200 μm ± 20 μm, and 60 μm ± 5 μm, respectively, thus providing here a resolution increase of > ×3.3. The difference between the obtained foci dimensions are attributed to the stability of the experimental system and the limited number of input modes measured and controlled (see Supplementary Notes 5–6).
Discussion
The AOTM provides a generalized mathematical framework and a useful approach for noninvasive highresolution optical investigation and light control in complex media. As such, the stateoftheart acoustooptic guidestar techniques can be readily revisited using the AOTM, including TRUE, iTRUE, and TROVE. While all current acoustooptic based approaches have so far been limited to use the first singular vector of the AOTM, we have shown that lower singular vectors can be exploited to concentrate light around and outside the ultrasound focus. Such minimization of the light intensity inside the acoustic focus may prove useful in e.g., laser therapy applications, when a specific small volume needs to be protected from laser irradiation. This result can also be linked to the recent use of wavefrontshaping for glare reduction^{47}, but with the AOTM, in a noninvasive fashion. The decomposition of the large ultrasound focus to singular vectors can be linked to the results of allacoustical DORT experiments on extended reflective targets^{48,49}, where it has been shown that the singular vectors in freespace correspond to the prolate spheroidal wavefunctions^{49}.
In our proofofconcept experiments, the speckle grain size was chosen to be considerably larger than the optical wavelength, to allow demonstrating singlespeckle scale focusing with the experimentally measured number of input modes N_{SLM}, signal to noise ratio and system stability. While the optical speckle grain size constitutes an ultimate limit to the attainable focus size, the actual size of the focus obtained by SVD of the AOTM (and by iTRUE and TROVE) is expected to be larger than the speckle grain when a too small number of input or output modes is measured (i.e., if the measured AOTM is of too small size). The required matrix size for different experimental parameters of speckle grain size and SNR is studied analytically and numerically in Supplementary Notes 5–6 (see Supplementary Figures 6–8). Our analysis suggests that the minimum number of probed modes (both input and output) that is required for singlespeckle scale focusing using a single AOTM grows cubically with the ratio between the ultrasound focus diameter and speckle grain size. Thus, the number of required measurements will be considerably larger in experiments in thick diffusive samples where the speckles are of diffractionlimited dimensions. Our theoretical and numerical analyses suggest that for the case of diffractionlimited speckles, even when high frequency ultrasound is used, the number of probed input modes that are required to achieve speckle scale focusing with a single AOTM is of the order of N_{SLM} ≈ 10^{6}. With our experimental setup acquisition speed (see below) and a sequential measurements scheme, such a large number of measurements would be impractically large, and beyond the sample decorrelation time. Given these practical limitations of our setup, and similar to previous works^{29}, we have chosen to use large speckle grain size, to prove the ability of singlespeckle grain focusing.
Concerning the practical timescales for speckle decorrelation in tissue, iterative phaseconjugation (iTRUE) seems more suitable than TROVE or AOTM for single point focusing, as it requires only a few tens of iterations to reach the resolution increase obtained by thousands of TROVE or AOTM measurements. While the total number of measurements (and thus the acquisition time) for focusing at a single position is smaller for iTRUE, it requires repeated iterations for focusing at any different position, and does not allow focus scanning like AOTM or TROVE (see Supplementary Table 1).
The AOTM approach, in its current embodiment, thus presents a novel framework for analysing the performance limits of acoustooptic based light control experiments, but does not allow imaging or focusing in thick scattering biological samples. This fundamental limitation posed by the small speckle grain size, while shared with TROVE and iterative optimization approaches, was not analyzed in previous works, and is importantly highlighted here. Extending acoustooptic approaches to allow optical diffractionlimited imaging in thick scattering samples is thus still a challnge. Potential solutions could involve the joint decomposition of several AOTMs, which effectively form a virtual ultrasound focus that is sharper than the ultrasound beam itself (Figs. 3–4), multiplexed measurements^{50}, and faster SLMs (see below). The significantly smaller speckle grain dimensions expected in experiments in thick tissue can be alleviated by using longer optical excitation wavelengths^{51}, and a higher frequency (> 50 MHz) and smaller Fnumber ultrasound transducer (e.g., a ∼ 20 μm diameter 90 MHz ultrasound focus is expected to contain less than 1000 diffractionlimited speckle grains at an illumination wavelength of 1700 nm).
Beyond its conceptual significance, an important practical advantage of the AOTM compared to TRUE, iTRUE, and TROVE, is that the positions of the SLM and camera are arbitrary and no DOPC system nor any careful alignment is required. This also allows an asymmetry in the pixel numbers: measuring on a much larger number of output camera pixels than SLM pixels, as permitted by stateoftheart cameras. However, a practical disadvantage of the presented acquisition of the AOTM compared to iTRUE is in the slow sequential measurement of input modes. We have used orthogonal phaseonly input modes as the basis for our AOTM measurements, however, as with the TM, any set of spanning modes, including amplitudeonly inputs can be used to retrieve the AOTM^{52}.
The experimental system in our experiments was based on a continuouswave (cw) laser, a liquid crystal SLM with a refresh rate of ∼10 Hz, and an ultrasound transducer with 15 MHz central frequency. This choice of available equipment is not optimal for pulsed AOTM measurements in thick tissue, as the laser peakpower and average power are three orders of magnitude lower compared to pulsed lasers^{28,29,30}, and the liquid crystal SLM is three to four orders of magnitude slower compared to ferroelectic or MEMS based devices^{53,54} or galvanometric scanners^{50,55}, used for rapid measurement of the TM. These technical limitations restricted our experimental demonstrations to thin static diffusers, which allowed long decorrelation times and sufficiently large scattered light intensity. Since our experiments did not take advantage of any other specific attributes of the thin scatterers, extension to thick multiplyscattering samples may be made possible by using a pulsed laser source and a MEMS based SLMs. However, even with these advancements, given the number of modes required for probing when diffractionlimited speckles are considered (Supplementary Note 5), the use of this approach is expected to be technically very challenging, and limited to samples with very long decorrelation times, such as exvivo or non bloodperfused samples.
The AOTM could be a valuable tool for studying various models of optical propagation in diffusive media, such as the intensity distribution of transmission eigenchannels^{40}, Anderson localized modes, or the memoryeffect inside samples^{56,57}. Since the AOTM defines the information content of any acoustooptic experiment, new analysis approaches may be put forward to retrieve more information from within the scattering medium, or generate new focusing modes, e.g., using the generalized Wigner–Smith operator to maximize momentum transfer to a target^{58}.
The approach can be scaled up to multiple acoustic beams, allowing a large field of view. However, when large volumes are considered, the computational resources for storing the corresponding AOTMs are expected to be very high, and the acquisition procedure very long. This is because the dimension of the AOTM with L ultrasound foci would result in a matrix with L × M × N_{SLM} elements. The size of the matrix, and the acquisition time, may be lowered by replacing the focused ultrasound beams with ultrasonic plane waves, or other parallelized measurements, which can be digitally coherently combined to synthetize various ultrasonic foci in 3D^{59,60}. The AOTM could also be extended to nonmonochromatic optical illumination, providing spectral, and temporal information on the medium’s response, This could be realized by using ultrashort optical pulses^{16}, or spectrally tunable sources^{61}, but would results in even higher dimensionality of the AOTM.
Methods
Simulations
For the simulation of an AOTM measurement, two random matrices with Gaussian i.i.d. random amplitude entries and random i.i.d. phase distribution from zero to 2π were generated to simulate the alloptical transmission matrices between: (1) the SLM plane and the ultrasound focal plane, T_{SLMUS}, and (2) the ultrasound focal plane and the camera plane, T_{USCAM}. The propagation from the SLM to the camera was simulated by multiplying T_{SLMUS} by the inputfield E_{in}. The ultrasound tagging was simulated by multiplying the field T_{USCAM}E_{in} pixelbypixel by a Gaussianshaped ultrasound pressure amplitude P_{US}. The result was propagated to the camera by multiplying with T_{USCAM}. The number of simulated input and output modes were 4096 and 2048, respectively (Fig. 1b). The simulated random i.i.d. transmission matrices provide a representative model for the diffusive light propagation regime, deep beyond the scattering and transport mean free paths in a multiplyscattering medium, without any correlations (memory effect, ballistic component, etc.). This i.i.d. matrices model is justified since the number of simulated (and experimentally measured) degrees of freedom in such a medium with the small ultrasound tagging volume would be considerably smaller than the total number of optical modes at such depths. This will yield cropped (“filtered”) matrices, which are essentially i.i.d.^{6}, and will not allow observing transport related phenomena such as the bimodal distribution of transmission eigenchannels^{62}.
Experiments
The full experimental setup is described in detail in Supplementary Note 1: a longcoherence continuouswave (cw) laser at a center wavelength of 810 nm was used as the light source. The laser is a single longitudinal mode, tunable, extended cavity, semiconductor laser, producing up to 1.5 Watt. The maximum power used in our experiments was 300 mW. The laser was provided by DTU Fotonik, Denmark (see acknowledgements).
To measure the ultrasonically modulated light, the laser beam was split to two arms of an interferometer. At the first arm, the laser beam illuminates an SLM (Holoeye Pluto), which is imaged on the first diffuser (Light shaping diffuser, Newport). A spherically focused ultrasonic transducer (V319SUF0.75INPTF, Olympus; 15 MHz central frequency, 0.75” focal length, 0.5” element diameter), emits 133 ns long pulses at a center frequency of 15 MHz inside a water filled glass tank. At the second, reference, arm, two acoustooptic modulators (MT80, AAOptoelectronics) are used to frequency shift and time gate the reference arm signal, to produce a 133 ns pulse with a central frequency that is shifted by 15 MHz + (5KHz/4) from the original laser frequency. The pulsed reference beam is focused on a mirror placed next to the output plane of the medium (the second diffuser) and is combined with the first beam on a fast camera (Photron Fastcam SA4) operating at f_{cam} = 5000 frames per second, with a 1 microsecond exposure time. To measure the weak ultrasonically tagged field in the presence of the strong untagged background, a doubleheterodyne holographic technique was employed^{36}, combining offaxis and phaseshifting interferometry. To maximize the measurements’ signal to noise, the SLM was imaged on the first diffuser surface, and a Hadamard input basis was used to measure the AOTM^{6}. To compensate for any slow phase drifts of the reference arm, a flatphase mask was displayed on the SLM before each input mode was injected, and the phase of the output field measured with a flatphase input was subtracted from the phase of each measured output field. The total acquisition time for a single AOTM with 3072 input modes (6144 displayed phase patterns) was limited by the SLM refresh rate (~ 6 Hz), and was 18 min. An iris placed in front of the first diffuser was used to control the speckle size at the target plane. Triggering of all instruments is detailed in Supplementary Figure 2. To minimize the fraction of the unshaped light from the SLM limited fillfactor, a blazed grating phase pattern was displayed on the SLM and the zero diffraction order was blocked. The displayed phase patterns on the SLM were composed of macro pixels of 20×20 physical pixels, displaying the phase sum of the blazed grating and the desired input phase. For direct inspection of the optical fields at the ultrasound focal plane, the second diffuser was removed, and an imaging lens was placed between the camera and the scattering medium. The diffusers used in the doubleslice experimental demonstrations were light shaping diffusers (Newport) providing no measurable ballistic components. In the experiments of Fig. 2 the diffuser 1 was composed of a stack of two diffusers, one with a scattering angle of 0.5° and the other 1°. The second slice (between acoustic focus and camera) was a 0.5° light shaping diffuser. In the experiments presented in Fig. 4, both slices were 0.5° light shaping diffusers. The acoustic focus was located at a distance of 10 cm from each of the scattering layers.
Data analysis
The experimentally measured AOTMs had 3072 input modes. For each input mode a camera image of the output field with a resolution of 512 × 352 pixels (Fig. 2) or 320 × 240 (Fig. 4) was acquired. To minimize the computational memory requirements, considering the speckle grain size on the camera, one out of three camera pixel in each dimension was taken as an output mode (×3 undersampling). For the results of Figs. 1–2, SVD of T was performed in Matlab (Mathworks). For the results of Figs. 3–4 SVD was performed on the matrix \(A_\alpha = \left( {T_{1  \alpha 2}^HT_{1  \alpha 2}} \right)^ + \left( {T_{1 + \alpha 2}^HT_{1 + \alpha 2}} \right)\), where ^{+} is a Tikhonov regularized pseudo inverse^{46}. Since the number of columns and rows of T can be substantially different, the SVD of \(B_\alpha = \left( {T_{1  \alpha 2}T_{1  \alpha 2}^H} \right)^ + \left( {T_{1 + \alpha 2}T_{1 + \alpha 2}^H} \right)\) can be used as well to calculate the optimal focusing input vectors, reducing the size of the analyzed matrix^{29}.
Data availability
All relevant data are available from the authors upon request.
References
Yaqoob, Z., Psaltis, D., Feld, M. S. & Yang, C. Optical phase conjugation for turbidity suppression in biological samples. Nat. Photon 2, 110–115 (2008).
Fink, M. Time reversal of ultrasonic fields. I. Basic principles. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 39, 555–566 (1992).
Mosk, A. P., Lagendijk, A., Lerosey, G. & Fink, M. Controlling waves in space and time for imaging and focusing in complex media. Nat. Photon 6, 283–292 (2012).
Horstmeyer, R., Ruan, H. & Yang, C. Guidestarassisted wavefrontshaping methods for focusing light into biological tissue. Nat. Photonics 9, 563–571 (2015).
Vellekoop, I. M. & Mosk, A. P. Focusing coherent light through opaque strongly scattering media. Opt. Lett. 32, 2309–2311 (2007).
Popoff, S. M. et al. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. Phys. Rev. Lett. 104, 100601 (2010).
Vellekoop, I. M., Lagendijk, A. & Mosk, A. P. Exploiting disorder for perfect focusing. Nat. Photon 4, 320–322 (2010).
Vellekoop, I. M. & Aegerter, C. M. Scattered light fluorescence microscopy: imaging through turbid layers. Opt. Lett. 35, 1245–1247 (2010).
Katz, O., Small, E. & Silberberg, Y. Looking around corners and through thin turbid layers in real time with scattered incoherent light. Nat. Photonics 6, 549–553 (2012).
Aulbach, J., Gjonaj, B., Johnson, P. M., Mosk, A. P. & Lagendijk, A. Control of light transmission through opaque scattering media in space and time. Phys. Rev. Lett. 106, 103901 (2011).
Katz, O., Small, E., Bromberg, Y. & Silberberg, Y. Focusing and compression of ultrashort pulses through scattering media. Nat. Photon 5, 372–377 (2011).
Small, E., Katz, O., Guan, Y. & Silberberg, Y. Spectral control of broadband light through random media by wavefront shaping. Opt. Lett. 37, 3429–3431 (2012).
Guan, Y., Katz, O., Small, E., Zhou, J. & Silberberg, Y. Polarization control of multiply scattered light through random media by wavefront shaping. Opt. Lett. 37, 4663–4665 (2012).
Popoff, S., Goetschy, A., Liew, S., Stone, A. D. & Cao, H. Coherent control of total transmission of light through disordered media. Phys. Rev. Lett. 112, 133903 (2014).
Rotter, S. & Gigan, S. Light fields in complex media: Mesoscopic scattering meets wave control. Rev. Mod. Phys. 89, 015005 (2017).
Choi, Y. et al. Measurement of the timeresolved reflection matrix for enhancing light energy delivery into a scattering medium. Phys. Rev. Lett. 111, 243901 (2013).
Choi, Y. et al. Overcoming the diffraction limit using multiple light scattering in a highly disordered medium. Phys. Rev. Lett. 107, 023902 (2011).
Popoff, S. M. et al. Exploiting the timereversal operator for adaptive optics, selective focusing, and scattering pattern analysis. Phys. Rev. Lett. 107, 263901 (2011).
Badon A. et al. Smart optical coherence tomography for ultradeep imaging through highly scattering media. Sci. Adv. 2, e1600370 (2016).
Jeong, S. et al. Focusing of light energy inside a scattering medium by controlling the timegated multiple light scattering. Nat. Photonics 12, 277–283 (2018).
Hsieh, C.L., Pu, Y., Grange, R. & Psaltis, D. Digital phase conjugation of second harmonic radiation emitted by nanoparticles in turbid media. Opt. Express 18, 12283–12290 (2010).
Tang, J., Germain, R. N. & Cui, M. Superpenetration optical microscopy by iterative multiphoton adaptive compensation technique.Proc. Natl Acad. Sci. USA 109, 8434–8439 (2012).
Katz, O., Small, E., Guan, Y. & Silberberg, Y. Noninvasive nonlinear focusing and imaging through strongly scattering turbid layers. Optica 1, 170–174 (2014).
Kong, F. et al. Photoacousticguided convergence of light through optically diffusive media. Opt. Lett. 36, 2053–2055 (2011).
Chaigne, T. et al. Controlling light in scattering media noninvasively using the photoacoustic transmission matrix. Nat. Photon 8, 58–64 (2014).
Conkey D. B. et al. Superresolution photoacoustic imaging through a scattering wall. Nat. Commun. 6, 7902 (2015).
Xu, X., Liu, H. & Wang, L. V. Timereversed ultrasonically encoded optical focusing into scattering media. Nat. Photon 5, 154–157 (2011).
Wang, Y. M., Judkewitz, B., DiMarzio, C. A. & Yang, C. Deeptissue focal fluorescence imaging with digitally timereversed ultrasoundencoded light. Nat. Commun. 3, 928 (2012).
Judkewitz, B., Wang, Y. M., Horstmeyer, R., Mathy, A. & Yang, C. Specklescale focusing in the diffusive regime with time reversal of varianceencoded light (TROVE). Nat. Photon 7, 300–305 (2013).
Si, K., Fiolka, R. & Cui, M. Fluorescence imaging beyond the ballistic regime by ultrasoundpulseguided digital phase conjugation. Nat. Photon 6, 657–661 (2012).
Si K., Fiolka R., Cui M. Breaking the spatial resolution barrier via iterative soundlight interaction in deep tissue microscopy. Sci. Rep. 2, 748 (2012).
Ruan H., Jang M., Judkewitz B., Yang C. Iterative timereversed ultrasonically encoded light focusing in backscattering mode. Sci. Rep. 4, 7156 (2014).
Tay, J. W., Lai, P., Suzuki, Y. & Wang, L. V. Ultrasonically encoded wavefront shaping for focusing into random media. Sci. Rep. 4, 3918 (2014).
Elson, D. S., Li, R., Dunsby, C., Eckersley, R. & Tang, M.X. Ultrasoundmediated optical tomography: a review of current methods. Interface Focus 1, 632–648 (2011).
Cui, M. & Yang, C. Implementation of a digital optical phase conjugation system and its application to study the robustness of turbidity suppression by phase conjugation. Opt. Express 18, 3444–3455 (2010).
Atlan, M., Forget, B. C., Ramaz, F., Boccara, A. C. & Gross, M. Pulsed acoustooptic imaging in dynamic scattering media with heterodyne parallel speckle detection. Opt. Lett. 30, 1360–1362 (2005).
Vellekoop, I. M. & Mosk, A. P. Universal optimal transmission of light through disordered materials. Phys. Rev. Lett. 101, 120601 (2008).
Kim, M. et al. Maximal energy transport through disordered media with the implementation of transmission eigenchannels. Nat. Photon 6, 581–585 (2012).
Sarma, R., Yamilov, A. G., Petrenko, S., Bromberg, Y. & Cao, H. Control of energy density inside a disordered medium by coupling to open or closed channels. Phys. Rev. Lett. 117, 086803 (2016).
Liew, S. F., Popoff, S. M., Mosk, A. P., Vos, W. L. & Cao, H. Transmission channels for light in absorbing random media: from diffusive to ballisticlike transport. Phys. Rev. B 89, 224202 (2014).
Liu Y. et al. Optical focusing deep inside dynamic scattering media with nearinfrared timereversed ultrasonically encoded (TRUE) light. Nat. Commun. 6, 5904 (2015).
Ma, C., Xu, X. & Wang, L. V. Analog timereversed ultrasonically encoded light focusing inside scattering media with a 33,000 × optical power gain. Sci. Rep. 5, 8896 (2015).
Prada, C. & Fink, M. Eigenmodes of the time reversal operator: solution to selective focusing in multipletarget media. Wave Motion 20, 151–163 (1994).
Prada, C., Manneville, S., Spoliansky, D. & Fink, M. Decomposition of the time reversal operator: detection and selective focusing on two scatterers. J. Acoust. Soc. Am. 99, 2067–2076 (1996).
Prada, C. & Thomas, J.L. Experimental subwavelength localization of scatterers by decomposition of the time reversal operator interpreted as a covariance matrix. J. Acoust. Soc. Am. 114, 235–243 (2003).
Popoff S., Lerosey G., Fink M., Boccara A. C., Gigan S. Image transmission through an opaque material. Nat. Commun. 1, 81 (2010).
Daniel, A., Liberman, L. & Silberberg, Y. Wavefront shaping for glare reduction. Optica 3, 1104–1106 (2016).
Komilikis S., Prada C., Fink M. Characterization of extended objects with the DORT method. In Proc. Ultrasonics Symposium 1996, 14011404, IEEE (1996).
Robert, J.L. & Fink, M. The prolate spheroidal wave functions as invariants of the time reversal operator for an extended scatterer in the Fraunhofer approximation. J. Acoust. Soc. Am. 125, 218–226 (2009).
Cui, M. A high speed wavefront determination method based on spatial frequency modulations for focusing light through random scattering media. Opt. Express 19, 2989–2995 (2011).
Horton, N. G. et al. In vivo threephoton microscopy of subcortical structures within an intact mouse brain. Nat. Photon 7, 205–209 (2013).
Drémeau, A. et al. Referenceless measurement of the transmission matrix of a highly scattering material using a DMD and phase retrieval techniques. Opt. Express 23, 11898–11911 (2015).
Conkey, D. B., CaravacaAguirre, A. M. & Piestun, R. Highspeed scattering medium characterization with application to focusing light through turbid media. Opt. Express 20, 1733–1740 (2012).
Liu, Y., Ma, C., Shen, Y., Shi, J. & Wang, L. V. Focusing light inside dynamic scattering media with millisecond digital optical phase conjugation. Optica 4, 280–288 (2017).
Papadopoulos, I. N., Jouhanneau, J.S., Poulet, J. F. & Judkewitz, B. Scattering compensation by focus scanning holographic aberration probing (FSHARP). Nat. Photonics 11, 116–123 (2017).
Freund, I., Rosenbluh, M. & Feng, S. Memory effects in propagation of optical waves through disordered media. Phys. Rev. Lett. 61, 2328–2331 (1988).
Judkewitz, B., Horstmeyer, R., Vellekoop, I. M., Papadopoulos, I. N. & Yang, C. Translation correlations in anisotropically scattering media. Nat. Phys. 11, 684–689 (2015).
Ambichl, P. et al. Focusing inside disordered media with the generalized Wigner–Smith operator. Phys. Rev. Lett. 119, 033903 (2017).
Laudereau, J.B., Grabar, A. A., Tanter, M., Gennisson, J.L. & Ramaz, F. Ultrafast acoustooptic imaging with ultrasonic plane waves. Opt. Express 24, 3774–3789 (2016).
Montaldo, G., Tanter, M., Bercoff, J., Benech, N. & Fink, M. Coherent planewave compounding for very high frame rate ultrasonography and transient elastography. IEEE Trans. Ultrason. Ferroelectr. Freq. Control 56, 489–506 (2009).
Andreoli D. et al. Deterministic control of broadband light through a multiply scattering medium via the multispectral transmission matrix. Sci. Rep. 5, 10347 (2015).
Goetschy, A. & Stone, A. Filtering random matrices: the effect of incomplete channel control in multiple scattering. Phys. Rev. Lett. 111, 063901 (2013).
Acknowledgements
The authors thank Mingjun Chu and Paul Michael Petersen, Technical University of Denmark, for providing the longcoherence laser source, and Benjamin Judkewitz and Sebastien Popoff for helpful discussions. This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grants no. 278025, 677909), and LABEX WIFI (Laboratory of Excellence within the French Program “Investments for the Future”) under references ANR10LABX24. O.K. was supported by a Marie Curie intraEuropean fellowship (IEF) and an Azrieli Faculty Fellowship. S.G. acknowledge support from the Institut Universitaire de France (IUF).
Author information
Authors and Affiliations
Contributions
M.F. conceived the idea. O.K. developed the idea, performed modeling and numerical simulations, built the setup, collected data, and performed data analysis. O.K., S.G., F.R., and M.F. contributed to the design of the experiments and to the writing of the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Journal peer review information: Nature Communications thanks the anonymous reviewers for their contribution to the peer review of this work. Peer reviewer reports are available.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Katz, O., Ramaz, F., Gigan, S. et al. Controlling light in complex media beyond the acoustic diffractionlimit using the acoustooptic transmission matrix. Nat Commun 10, 717 (2019). https://doi.org/10.1038/s41467019085836
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467019085836
This article is cited by

Delivering broadband light deep inside diffusive media
Nature Photonics (2024)

Roadmap on computational methods in optical imaging and holography [invited]
Applied Physics B (2024)

Directly and instantly seeing through random diffusers by selfimaging in scattering speckles
PhotoniX (2023)

Enhanced light focusing inside scattering media with shaped ultrasound
Scientific Reports (2023)

Tracing multiple scattering trajectories for deep optical imaging in scattering media
Nature Communications (2023)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.