Abstract
When light propagates through opaque material, the spatial information it holds becomes scrambled, but not necessarily lost. Two classes of techniques have emerged to recover this information: methods relying on optical memory effects, and transmission matrix (TM) approaches. Here we develop a general framework describing the nature of memory effects in structures of arbitrary geometry. We show how this framework, when combined with wavefront shaping driven by feedback from a guidestar, enables estimation of the TM of any such system. This highlights that guidestar assisted imaging is possible regardless of the type of memory effect a scatterer exhibits. We apply this concept to multimode fibres (MMFs) and identify a ‘quasiradial’ memory effect. This allows the TM of an MMF to be approximated from only one end  an important step for microendoscopy. Our work broadens the applications of memory effects to a range of novel imaging and optical communication scenarios.
Similar content being viewed by others
Introduction
Coherent images are encoded in spatial light modes: patterns in the intensity and phase of light. When these patterns propagate through an opaque scattering material, such as frosted glass, a multimode optical fibre (MMF), or biological tissue, they distort and fragment, and the spatial information they carry becomes scrambled. This corrupts the formation of images of objects hidden behind or inside turbid media. In the last decade or so, a series of pioneering studies have demonstrated how digital light shaping technology can be used to measure and reverse scattering effects  unscrambling the light back to the state it was in before it entered the medium^{1,2,3,4}. These techniques take advantage of the linear (in electric field) and deterministic nature of the scattering, and have enabled focussing and imaging through scattering systems^{5,6,7,8,9}. At the heart of this capability lies the transmission matrix (TM) concept, which describes the scattering as a linear operation relating a set of input spatial light modes incident on one side of the scatterer, to a new set of output modes leaving on the opposite side^{3}. Once the TM of a scatterer has been measured, it tells us what input field is required to create an arbitrary output field, and thus permits the transmission of images through the scatterer. However, measurement of the TM typically requires full optical access to both sides of the scatterer  impossible when the image plane is, for example, embedded in living tissue.
To overcome this limitation and look inside scattering environments, a useful suite of tools has emerged in the form of optical memory effects: the presence of underlying correlations between the incident and transmitted fields^{10,11,12,13}. These hidden correlations have been extensively studied in thin randomly scattering layers which have direct applications to imaging through biological tissue. In this case tilting^{10}, shifting^{12}, spectral^{14} and temporal^{15} correlations have been revealed. The tilt and shift memory effects are related to correlations in the Fourierspace or realspace TM of a scatterer  and have enabled extraction of subsets of the TM which can be used to image over small areas inside scattering systems.
In parallel with these advances, TM approaches have also spurred the development of alternative methods of seeing into tissue much more deeply, albeit invasively, by guiding light along narrow waveguides. These techniques use MMFs to achieve highresolution imaging at the tip of a needle  acting as ultralow footprint endoscopes. Modal dispersion scrambles coherent optical signals transmitted through MMFs, and so before they can be deployed in scanning imaging systems, their TM must also first be measured. MMF based microendoscopy has great potential for deep tissue imaging, as indicated by a swathe of recent successes^{16,17,18,19}, yet a major challenge holding back broader uptake of this technique is the fragility of the TM used to control the optical field at the distal (far) facet. After TM calibration, which as described above conventionally requires access to both ends of the fibre, the MMF must be held completely static, as even small perturbations in fibre configuration (e.g., bends or twists) or temperature dephase the propagating fibre modes, severely reducing the contrast of focussed spots at the distal facet, and the fidelity of the reconstructed images. Flexible operation of microendoscopic imaging systems based on current optical fibre technology require a TM characterisation method that can be rapidly performed on the fibre insitu, with access only at the proximal (near) end.
Here we develop a general framework describing how monochromatic memory effects arise in samples of arbitrary geometry, and apply it to the MMF case. We show that deployment of a guidestar located on the distal facet of an MMF (and capable of reporting its local field intensity to the proximal end), combined with an estimate of the basis in which the TM is close to diagonal, provides a way to approximate the TM of, and thus image through, optical fibres. Crucially, this approach only requires access to the proximal end of the MMF: offering a route to insitu TM calibration of flexible microendoscopic imaging systems. More generally, the concept we describe here enables guidestar based scanning imaging to be performed through scattering systems of any geometry, without invoking conventional shift or tilt memory effects, as long as we have an estimate of the basis in which the TM is quasidiagonal. Our work broadens the applications of memory effects beyond thin randomly scattering layers to a range of novel imaging and optical communication scenarios.
Results
The memory effect in an arbitrary basis
We start by describing the emergence of memory effects in optical systems of arbitrary geometry. These systems may encompass, for example, the wellunderstood case of thin scattering layers, but also networks of waveguides, or any other linear optical system through which light is transmitted. We assume that we know the geometry of the object before us  for example a thin scattering layer, or an optical fibre  but we do not have detailed knowledge of its scattering properties  for example we do not know the spatial function of the refractive index throughout a layer, or the length or bend configuration of a fibre. Given this level of prior information, we describe when and how optical memory effects may be used to achieve imaging.
In the general case, our aim is to image through a scattering system that has an unknown monochromatic TM, T. As a first step, let’s assume that the geometry of the system reveals enough information for us to estimate a basis in which T is approximately diagonal, i.e., T = UDU^{−1}, where we know matrix U^{−1} which allows us to transform to the quasidiagonal basis from knowledge of the field in realspace. We assume that D is quasidiagonal (i.e., it has a significant proportion of its power concentrated on the diagonal), but we have no knowledge of the complex elements of D. We consider the following questions: is advance knowledge of a basis in which the TM of a scatterer is quasidiagonal enough information to (i) predict a memory effect in, and (ii) image through, the system?
The memory effect occurs when a given modulation (i.e., transformation) of the field incident onto a scatterer modifies the output field in a deterministic way  for example, creating a lateral displacement of the output field. To explore this idea in a general basis, we express the input field as a column vector u, which holds the complex coefficients of the input field in realspace. Incident field u is transformed into an output field v = Tu via propagation through a scatterer. We consider a modulation of the incident field in the form \({\bf{u}}^{\prime} ={\bf{O}}{\bf{u}}={\bf{U}}{\bf{M}}{{\bf{U}}}^{1}{\bf{u}}\), where M is a diagonal matrix. After such a modulation, the output field v is changed to \({\bf{v}}^{\prime}\), which can be written as
Given that M is exactly diagonal, and D is close to diagonal, the two matrices will commute with an error [D, M] that is due to the offdiagonal elements of D, i.e., DM = MD + [D, M]. As D becomes closer to diagonal this error will reduce, becoming zero in the limit of an exactly diagonal matrix. We therefore rewrite Eq. (1) as
which shows that for input transformations of the form O = UMU^{−1}, the output field undergoes the same transformation with an error η = U[D, M]U^{−1}u. In more condensed notation, Eq. (2) is
Equation (3) describes a general form of the memory effect. In answer to our first question (i), Eq. (3) tells us that without knowing anything about the elements of the quasidiagonal matrix D, we can deterministically modify the field transmitted through a scattering system. The basis in which the TM is diagonal is linked to the form of the field modification, i.e., memory effect, that is possible. This highlights the existence of an infinite family of memory effects operating in scattering systems of different geometries, each requiring its own unique but predictable set of transformations on input fields, with a deterministic transformation of the output that is not, in general, a translation of the field. We now focus on some special cases, and highlight when imaging is possible using these memory effects.
Memory effect based imaging through scattering systems
We first consider the wellknown tilt–tilt memory effect^{10,11}. In a thin randomly scattering layer, multiple scattering is governed at a statistical level by a diffusion process, meaning that light focussed to a given lateral position on the input will only diffuse locally to nearby lateral positions at the output. In this case our assumption is that U = I, the identity matrix, i.e., the monochromatic TM is quasidiagonal in the position basis (and equivalently exhibits diagonal correlations in the Fourier basis^{12}). Our transformation O is thus also diagonal in the position basis: O = IMI = M. This means that any spatially varying phase change applied to the input field, is also applied to the output field. More specifically, a tilt of the input beam results in a tilt of the output beam, in which case the diagonal elements of M correspond to a phase ramp modulation given by \({\bf{M}}=\,\text{diag}\,[{e}^{{\rm{i}}(\delta {k}_{x}{\bf{x}}+\delta {k}_{y}{\bf{y}})}]\), where diag[d] places vector d on the main diagonal of a square zero matrix, x and y are vectors specifying the x and y coordinates of each pixel on the input plane, and δk_{x} and δk_{y} specify the desired change in x and ycomponents of the wavevector normal to the tilted wavefront.
Tilting of the output beam is not in itself useful to image the output plane of the sample. However, in the farfield of the output, this tilt corresponds to a lateral shift, and so unknown speckle patterns can be controllably translated in the farfield. Bertolotti et al.^{20} showed that imaging is then possible from the input side by measuring the fluorescent intensity excited by these unknown patterns as they are scanned in two dimensions. This procedure yields the amplitudes of the Fourier components of the image, but not the phases of these Fourier components. Despite this missing information, a diffraction limited image of an object in the farfield can be estimated using a phase retrieval algorithm with appropriate constraints^{21}.
Next we consider how the more recently discovered shift–shift memory effect appears in our general framework^{12}. The shift–shift memory effect occurs for some anisotropic disordered materials, assuming that the scatterer has a thickness less than the transport meanfreepath (TMFP) of the system. The TMFP is the average length over which scattering randomises the photon direction (wavevector). This means that we expect only small changes in the angular deviation of rays from input to output, and so assume the TM of the scatterer to be quasidiagonal when represented in the two dimensional Fourier basis U = F^{−1}. In this case a transformation O = F^{−1}MF of the input beam leads to approximately the same transformation of the output beam. Again choosing a phase ramp along the diagonal of M, O becomes a spatial shift operator, and so lateral displacement of the input beam results in a corresponding shift of the output beam, i.e., \({\bf{M}}=\,\text{diag}\,[{e}^{{\rm{i}}({{\bf{k}}}_{x}\delta x+{{\bf{k}}}_{y}\delta y)}]\), where k_{x} and k_{y} are vectors specifying the x and ycomponents of the wavevectors arriving at the input plane, and δx and δy specify the desired lateral shift of the input . Equivalently the TM exhibits diagonal correlations in realspace^{12}. In this case, imaging in the nearfield of the output is theoretically possible by once again scanning unknown speckle patterns in 2D and performing phase retrieval^{20}. We note that the shift–shift and tilt–tilt memory effects are Fourier duals of each other, one moving the field, the other redirecting it. As a Fourier transform and its inverse contain factors of exp(−ikx) and exp(+ikx) respectively, then the applied phases in the two memory effects must similarly differ by a minus sign.
The addition of a ‘guidestar’ embedded in the scatterer provides more information, and so improves the memory effect based imaging that is possible: yielding images with higher a signaltonoise ratio (SNR), and without the need for phaseretrieval. First implemented in astronomy to correct for atmospheric turbulence^{22}, a guidestar is a point within the scatterer capable of signalling the intensity of its local field to the outside world. For use in microscopy, guidestars have been created from highly reflective embedded particles^{23}, or via a range of alternative methods exploiting, for example, fluorescent^{24}, magnetic^{25}, movementbased^{26}, photoacoustic^{27}, or acoustooptic^{28} properties. The input field required to focus on a guidestar, u^{gs}, can be calculated by phase conjugation of a field emanating from the guidestar^{2}, or by phase stepping holography^{4}. u^{gs} constitutes a single column of the inverse of the TM of the scatterer, T^{−1} (with the output discretised in the realspace ‘pixel’ basis). In cases where the TM is unitary (or at least approximately so), T^{−1} = T^{†}, and knowledge of u^{gs} is equivalent to knowing one row of the TM^{29}.
Memory effects that laterally shift the output field (such as the tilt and shift memory effects) can be understood to arise from the existence of correlations in the realspace TM of the scatterer (or its Fourier transform), so that measuring one row of the TM also provides information about other rows^{12}. Therefore once u^{gs} is found and a focus formed on the guidestar, these shift memory effects can be exploited to controllably scan the focus over a local area around the guidestar, known as the isoplanatic patch. The size of the isoplanatic patch is governed by the distance over which the output field can be translated before it decorrelates (i.e., the degree to which the approximation in Eq. (3) holds). This strategy has been used to create scanning imaging systems inside randomly scattering samples by a suitable combination of tilting and/or lateral translation of the input wavefront^{30,31}. Therefore, in an initial answer to our second question (ii), so far it has been shown that diffraction limited imaging is possible using the shift–shift and tilt–tilt memory effects, which are capable of laterally translating the output field in two dimensions.
Rotational and radial memory effects in multimode fibres
We now consider the emergence of memory effects in other geometries, and examine an MMF as a key example. The near cylindrical symmetry of MMFs leads to the prediction of a basis in which the TM is approximately diagonal. For example, solving the wave equation in an idealised straight section of step index fibre at a single wavelength results in a set of orthogonal circularly polarised eigenmodes that maintain a constant spatial profile and polarisation on propagation^{32}. Here, following work by Plöschner et al.^{33}, we refer to these modes as propagation invariant modes (PIMs). Even though real optical fibres differ from this idealised case, it was recently shown in ref. ^{33} that the TM of a short length of step index MMF is relatively diagonal when represented in the PIM basis.
Following our general framework, in this case U = P, the matrix transforming from the PIM basis to realspace, and so the TM of an MMF can be factorised according to T = PDP^{†}, where as before D is quasidiagonal, and here we have used the conjugate transpose P^{†} in place of the inverse P^{−1} under the assumption that P is sufficiently oversampled in realspace such that P^{†}P ∝ I. (This is because the fibre modes are continuous functions of position, and orthonormal with respect to an integral over all space. Therefore as the sampling in real space is increased, the matrix multiplication P^{†}P becomes an increasingly better approximation to the set of mode overlap integrals, and therefore ever closer to the form of the identity matrix). Figure 1a shows a simulated lowdimensional example of these matrices for an MMF that supports 42 modes per polarisation at a wavelength of 633 nm. Circularly polarised PIMs are specified by an azimuthal and radial index, ℓ and p respectively. For a step index fibre of radius a, the transverse field of ψ_{ℓ,p}, in cylindrical coordinates (denoted by radial position r and azimuthal position θ), is given by
where N_{ℓ,p} is a normalisation constant ensuring each mode has a total intensity equal to 1, \({{\mathcal{J}}}_{\ell }\) is a Bessel function of the first kind of order ℓ, and \({{\mathcal{K}}}_{\ell }\) is a modified Bessel function of the second kind of order ℓ. u_{ℓ,p} and ω_{ℓ,p} are normalised transverse wavenumbers of each PIM in the core and cladding respectively, which may be found from the roots of the characteristic equation describing the parameters of the fibre (see ref. ^{32} and Methods for more details). ℓ, the vortex charge, is proportional to the amount of orbital angular momentum (OAM) carried by the PIM, which is characterised by a helical wavefront that accrues a phase change of 2πℓ around one circuit of the fibre axis^{34}.
Equation (3) leads to an input modulation of O = PMP^{†}, which tells us that phase changes imparted to individual PIMs at the input of the fibre are preserved at the output. Applying a phase ramp modulation linearly proportional to azimuthal mode index ℓ, i.e., M = diag[e^{−iℓδϕ}], is equivalent to rotating the input field around the fibre axis by an angle δϕ. Here ℓ is a vector specifying the vortex charge of each PIM. As these ℓdependent phase changes are preserved through the fibre, the output field is also rotated by δϕ, as shown in Fig. 1b–e. This effect is already wellknown, and coined the rotational memory effect^{35}. Our general framework tells us that it is a rotational analogue of the shiftshift memory effect: the angular dependence of the MMF TM is quasidiagonal in the 1D cylindrical Fourier basis (i.e., the OAM basis), and therefore an angular shift of the input field leads to an angular shift of the output field. Figure 1a shows the corresponding diagonal correlations in the realspace TM, when the realspace pixels are ordered along, for example, an Archimedes spiral, winding out from the central axis of the fibre (see spiral inset in Fig. 1).
We might wonder whether there is a corresponding effect where we can displace the output field from the fibre in a radial direction. Equation (3) highlights that the form of any memory effect based field transformation is intrinsically linked to the basis in which the TM is diagonal. As shown in all cases above, a displacement operation at the output is diagonalized in the Fourier basis. Therefore, without knowledge of the TM, any lateral displacement of the unknown output field is only possible in systems where the TM is quasidiagonal in the Fourier basis. The radial dependence of the MMF TM is not quasidiagonal in the Fourier basis. This means that a ‘radially–shifting’ memory effect, that serves to translate the entire output field of an MMF radially, does not occur—and in any case, such translations would be illdefined at the fibre axis and corecladding interface. In Supplementary Note 1, we explore the disruption of shifting memory effects in MMFs in more detail. We show how the presence of the fibre edges—the corecladding interface—breaks the translational symmetry thus disrupting the equivalent of the 2D shiftshift memory effect in MMFs.
Despite this, modulations that operate on the field in the radial direction do exist. For example, consider the effect of a modulation based on the radial mode index p on the input beam: M = diag[e^{ipδρ}], i.e., a phase ramp linearly proportional to p, where vector p holds the radial index of each PIM, and where δρ = πδr/a, and δr is radial distance across the fibre facet. Such a modulation may be viewed as a ‘quasiradial’ memory effect: although as described above, it can’t enable a simple translation of the distal field.
Figure 2 shows the behaviour of the quasi–radial memory effect. Intriguingly, if the output field is concentrated to a diffraction limited spot centred on the fibre axis, then the above modulation on the input field does indeed correspond to a radial shift of this point—i.e., it is transformed into a ring of intensity of radius δr (see Fig. 2a). However, output fields corresponding to diffraction limited points at other radii on the distal facet show a variety of different transformations, depending on the radial coordinate of the point (see Fig. 2b–d). We emphasise that all of these output transforms can be achieved using the same input modulation, regardless of the length of the fibre. This tells us that if the field is localised to a diffraction limited spot anywhere on the distal facet, the intensity pattern can be deterministically transformed—with a fidelity governed once again by how well the approximation in Eq. (3) holds.
In Supplementary Note 2 we investigate two other quasiradial memory effects: Firstly we show that if the position of the initial focussed spot is known, then this information enables a phaseonly modulation linearly proportional to the radial component of the wavevectors composing the modes, k_{r}, to be applied  which is capable of moving the spot with less distortion than shown in Fig. 2 (although still not perfectly). Secondly we show that a modulation of the form \({\bf{M}}=\,\text{diag}\,[\cos ({{\bf{u}}}_{\ell ,p}r/a)]\) enables a point to be radially expanded about its initial position, wherever that may be on the distal facet. Yet for an arbitrary speckled output beam, these radially varying transformations all interfere in a way that cannot be predicted without knowledge of the amplitudes and phases of the original distal speckle field  which itself requires knowledge of the entire TM.
Guidestar assisted imaging through MMFs
The lack of a true ‘radially–shifting’ memory effect has implications for imaging through uncalibrated MMFs. As scanning of unknown speckle patterns is only permitted in 1D (azimuthally), then there is no way to perform imaging based on 2D scanning of these speckle patterns as in ref. ^{20}. However, provided we know how to focus the field at the output to a point, the existence of the quasiradial memory effects suggests a form of ghost imaging might be possible. For example, a sequence of known intensity patterns could be projected to the distal end of the fibre using the rotational and quasiradial memory effects, and their level of overlap with the distal scene recorded back at the proximal end by measuring the total intensity of the return signal, allowing an image to be computationally reconstructed^{36,37}. Although this idea holds promise, we now show that it is possible to do even better, and use the information held in matrix P (i.e., knowledge of the basis in which the TM is quasidiagonal), to scan a welldefined focus around a guidestar in both azimuthal and radial directions. This opens up a 2D isoplanatic patch at the output of the fibre, within which scanning imaging is possible.
We imagine placing a single guidestar, such as a fluorescent particle, at the distal facet of the fibre (see the red circle on the fibre schematic in Fig. 1). The proximal field that will focus onto the guidestar, u^{gs}, can be measured using phase stepping holography, with access only to the proximal end, as described in, for example, ref. ^{4}, or in a single shot using phase conjugation^{2}.
So far, guided by Eq. (3), we have restricted ourselves to transforms on the input field of a particular form O = UMU^{−1}. We now relax this and consider more general transformations of the input field to achieve a radial shift in the position of a focussed spot on the distal facet. To accomplish output field modulations beyond what is possible with conventional memory effects alone (such as in this case), knowledge of the full TM of the scatterer is required  see Supplementary Note 1 for more discussion. Fortunately, it is indeed possible to obtain an estimate of the TM using the information at our disposal. In the case of an MMF, we can make the assumption that the TM is unitary (i.e., power loss is minimal). We also assume the real fibre’s TM to be perfectly diagonal in the PIM basis. Under these assumptions, we can now find \({\bf{D}}^{\prime}\), a diagonal approximation to the true quasidiagonal TM D. To proceed, we show that the measurements on the guidestar reveal the phase delay of each PIM due to propagation through the MMF. This phase information can then be used to populate the complex elements along the main diagonal of \({\bf{D}}^{\prime}\), while setting all offdiagonal elements to zero.
More specifically, the defining equation for the input field u^{gs} that focuses onto a guidestar is
where we denote the output field focussed onto the guidestar as \({{\bf{v}}}^{{m}_{0}}\), which has all elements equal to zero except for one at the spatial point corresponding to the location of the guidestar, indexed by m_{0}. Substituting the perfectly diagonal approximation \({\bf{D}}^{\prime}\) that we wish to find in place of the real quasidiagonal TM D, and multiplying both sides of Eq. (5) by the adjoint of P then gives \({\bf{D}}^{\prime} {{\bf{P}}}^{\dagger }{{\bf{u}}}^{{\rm{gs}}}={{\bf{P}}}^{\dagger }{{\bf{v}}}^{{m}_{0}}\). Our assumption of the unitarity of the fibre TM means that the elements of the diagonal matrix \({\bf{D}}^{\prime}\) only vary by their phase term, i.e., the n^{th} element \({D}_{nn}^{\prime}=\exp ({\rm{i}}{\phi }_{n})\). Therefore we can also rewrite the defining equation for u^{gs} as
If we can estimate the PIM basis P and we know the input field u^{gs} that focuses onto the guidestar at m_{0}, we can estimate the phases on the diagonal of \({\bf{D}}^{\prime}\) by rearranging Eq. (6) for \({{\rm{e}}}^{{\rm{i}}{\phi }_{n}}\), yielding
where the two contributions to the phase are given by
Here γ_{n} is the phase delay mode n picks up on propagation through the fibre, and σ_{n} is a mode dependent phase correction term, equal to the relative phase required for each PIM to constructively interfere at the lateral location of the guidestar if it were positioned at the proximal end of the MMF. The intuition behind σ_{n} is to ensure that we incorporate only the phase delay due to the scatterer into the estimated TM, and remove any apparent phase delay associated purely due to the choice of diagonal basis. The Approximate Transmission Matrix (ATM) of the MMF in the realspace basis is then given by \({\bf{T}}^{\prime} ={\bf{P}}{\bf{D}}^{\prime} {{\bf{P}}}^{\dagger }\). The ATM can now be used to predict the proximal field modulations required to move the focus both azimuthally and radially (see Fig. 1f, g). Evidently, as will be shown later, knowledge of the ATM gives more general control over the output field than simply shifting the position of a focus.
The fidelity of the ATM will clearly depend upon how well we can estimate the basis in which the TM is diagonal (i.e., the validity of the approximation in Eq. (3)). Figure 3 shows simulations of the imaging performance through an MMF as our ability to accurately estimate the diagonal basis is compromised. We simulate the measurement of u^{gs} using a guidestar placed at the corecladding boundary (marked by a red circle in Fig. 3), and construct the ATM. See Methods for a detailed description of these simulations, which capture how power is spread away from the diagonal of the TM in a realistic manner. Figure 3a–f shows the powerratio p_{r}  the ratio of power focussed into a target spot using the ATM compared to total power transmitted to the distal facet, as p_{d}, the percentage of power on the main diagonal of the real TM D, decreases. Each panel maps how the powerratio varies over the distal facet  capturing the size and shape of the isoplanatic patch. We see that the error in the ATM \({\bf{T}}^{\prime}\) is nonuniformly distributed over its columns—meaning that columns representing points close to the guidestar are well captured, while the errors in columns representing points further away grow with distance from the guidestar. This results in an isoplanatic patch that gradually collapses around the location of the guidestar as p_{d} reduces. Figure 3g–l shows simulations of the guidestar based imaging performance, indicative of the spatial variation in contrast and resolution in each case. Here imaging across the entire facet appears to be disrupted once p_{d} falls below ~25%.
We now consider the factors that govern the area of the isoplanatic patch at the distal facet of the MMF. The angular extent of the tilt–tilt memory effect, α, is related to a single parameter describing the sample: the thickness of the scattering layer L_{s}, so that α ~ λ/(2πL_{s}), where λ is the incident wavelength^{13}. This raises the question of whether it is also possible to derive a similar equation describing the limits of the memory effects in MMFs. We start by noting that, as shown in Fig. 3a, an ideal perfectly straight MMF supports a rotational memory effect of 2π, and a quasiradial memory effect of r, regardless of its numerical aperture, core diameter or length. However, the size and shape of the isoplanatic patch exhibited by real MMFs is highly sensitive to a large number of parameters. These include misalignments of the optical system at the input and output of the fibre (at least 6degrees of freedom at each end), the departure of the crosssectional refractive index profile of the fibre from the ideal (or assumed) case, how this profile varies along the fibre’s length, and the bend configuration of the fibre^{33}. This renders finding an analytical expression for the extent of the memory effect in MMFs a challenging task. Nevertheless, the combined effect of these parameters is to spread power into the offdiagonal elements of the real TM D, and we find that the approximate area of the isoplanatic patch, A_{iso}, is related to the fraction of power remaining on the diagonal of the actual TM, p_{d}, according to: A_{iso} ~ πr^{2}p_{d}. See Methods for a derivation of this expression and the approximations used. This trend is qualitatively apparent in Fig. 3.
Proof of principle experiments
In order to test our concept in practice, we design an experiment capable of measuring the TM of a step index MMF at a single circular input and output polarisation. We study an MMF of NA = 0.22, core diameter = 50 μm, and L ~ 30 cm in length, supporting 754 spatial modes per polarisation at a wavelength of 633 nm. These dimensions are chosen as a typical example of an MMF that may find microendoscopic imaging applications. Since the input circular polarisation is maintained over propagation through short lengths of fibre, we sample an orthogonal subspace of the full TM (i.e., a single polarisation), which is sufficient for highcontrast imaging behind the distal end^{33}. The experimental setup is similar to that described in ref. ^{38} and is shown in Fig. 4a. The spatial mode of input light is controlled using a digital micromirror device (DMD) in the Fourier plane of the proximal fibre facet, and the distal fibre facet is imaged onto a highspeed camera, along with a coherent reference beam enabling extraction of the complex field via phase stepping holography (See Methods and Supplementary Notes 3 and 4 for more details. Supplementary Note 9 details the loss throughout the experimental system).
We first fully sample the TM of the MMF at a single polarisation. Supplementary Movie 1 shows the input probe fields, and holographically reconstructed output fields during this measurement. After correcting for coarse misalignments in the input and output beams following the strategy outlined in ref. ^{33} (we note this does not require an optimisation procedure), and transforming to the PIM basis representation, the fully sampled TM possesses p_{d} ~ 15% of its power on the main diagonal (TM shown in Fig. 4b). The degree of mode coupling in the TM can also be quantified by the ratio of L/ℓ_{f}, where ℓ_{f} is the TMFP in the fibre mode (PIM) basis, i.e., the estimated length of fibre beyond which the TM can be considered fully coupled. In this case we find our experimentally measured TM has a level of mode coupling consistent with L/ℓ_{f} ~ 0.02, a value we estimate by following the methods given in ref. ^{39}. Therefore, in this experiment we anticipate a guidestar assisted imaging performance equivalent to that predicted in Fig. 3f—i.e., an isoplanatic patch extending around the guidestar, but not reaching across the entire output facet of the fibre.
Figure 4c and d shows the experimentally measured power ratio maps of foci generated at the distal facet of the fibre using the fully sampled TM 4(c), and a guidestar based ATM (Fig. 4d). Supplementary Movie 2 shows the focus as it is scanned in 2D across the isoplanatic patch at the distal facet of the MMF. The ATM was constructed from data recorded by a single pixel of the camera imaging the output facet, mimicking the action of a guidestar. The extent of the isoplanatic patch in Fig. 4d matches well with Fig. 3f as predicted. Figure 4e–g shows transmission images of a resolution target using the fully sampled TM. Figure 4h–j shows the same scenes imaged using the ATM (more detail of how these images was recorded is given in Methods). As expected, we see that imaging is possible through a small isoplanatic patch around the guidestar, and objects outside this patch are not discernible as we are unable to create a high contrast focus in these regions. The gradual reduction in resolution and contrast away from the guidestar is analogous to the situation found in memory effect based imaging through thin scattering layers.
We note that in these experiments, the object (resolution target) is positioned ~40 μm away from the distal facet of the fibre. Therefore, in order to reconstruct an image of the object in focus, the scanned foci had to be axially refocussed from the distal facet to the object plane. Due to the cylindrical symmetry of the fibre, the radial kvector \(\left({k}_{r}=\sqrt{{k}_{x}^{2}+{k}_{y}^{2}}\right)\) of input light is approximately preserved at the output—which is a consequence of the quasidiagonal nature of the fibre TM in the PIM basis. As previously demonstrated in refs. ^{17,40}, refocussing can be achieved by taking advantage of this symmetry and adding a quadratic ‘lensing’ phase term directly to the field that is generated by the DMD. We note this input transformation is also related to the recently reported ‘chromatoaxial’ memory effect studied in other forward scattering media^{41}. Therefore we actually create fields corresponding to defocussed spots at the distal facet of the fibre, which then focus onto the resolution target. Despite this increase in complexity, Fig. 4h–j demonstrates that this refocussing can be successfully achieved using the ATM, which affords a high level of control over the field within the isoplanatic patch. Supplementary Note 5 discusses the refocussing capability in more detail.
In addition to imaging, it is also possible to project patterns through the MMF into the isoplanatic patch, examples of which are described in Supplementary Note 6. We also studied the effect of fibre deformation on guidestar assisted focusing, and found only a minor reduction in the powerratio of foci was observed when the fibre was bent through 90^{∘} with a radius of curvature of ~10 cm, as described in Supplementary Note 7. This result shows there is potential for applying singleended guidestar based TM calibration to image through flexible MMFs undergoing time varying deformations. More generally, these refocussing and pattern projection results show that pointspreadfunction engineering should be possible within the isoplanatic patch^{42}.
The location of the guidestar on the distal facet also plays a key role in determining the accuracy of the ATM, and thus the size and shape of the isoplanatic patch. Each focus is formed as a superposition of PIMs which interfere constructively. We can only measure the phase change of PIMs that have an intensity profile that overlaps with the guidestar position. Therefore, it is preferable to choose a guidestar location that overlaps with the highest number of PIMs. Inspecting the rows of matrix P, which hold this overlap information, reveals that guidestar locations at the corecladding boundary overlap with all of the PIMs  although we note that some PIMs of index ℓ = 0 will have very low intensity at the edge of the core as their fields are concentrated mainly on the fibre axis. Therefore we have chosen a guidestar positioned at the corecladding boundary.
It is straight forward to test alternative guidestar positions in our proofofprinciple experiments—moving the guidestar simply means choosing a different camera pixel to use for ATM construction. Figure 5 shows the experimental power ratio maps for three different choices of guidestar location, along with the theoretical level of overlap with the PIMs in each case. As expected, the area of the isoplanatic patch is maximised when the guidestar is placed on the corecladding boundary (Fig. 5a), enabling recovery of most of the diagonal elements of the ATM in the PIM basis. As the guidestar location is moved radially inwards towards the centre of the core, the number of PIMs sampled by the guidestar progressively decreases (Fig. 5a–c). When the guidestar is placed at a radius of a/2, we observe a strong arc in the isoplanatic patch, a signature of the rotational memory effect (Fig. 5e). This isoplanatic arc is formed as projection of foci to the same radius as the guidestar require the same combination of PIMs, the phase delays of which have been accurately sampled in the ATM. However we also observe a sharp reduction in the powerratio when the focus is moved radially in Fig. 5e as these points require the constructive interference of different combinations of PIMs, some of which are not wellsampled by the guidestar. When the guidestar is located on the fibre axis, it exclusively samples only the PIMs with a vortex charge of ℓ = 0 (Fig. 5c), and the isoplanatic patch contracts to a single spot as shown in Fig. 5f.
There is one point on the distal facet that we know how to focus on perfectly: the position of the guidestar itself, which has been directly measured. However, as can be seen in both simulations and experiments in Figs. 3–5, as p_{d} decreases, then using the ATM does not yield a perfect focus even onto the guidestar itself. This is a consequence of our assumptions that the fibre TM is perfectly diagonal and unitary  assumptions that become less accurate with lower values of p_{d}. To mitigate this, when scanning over the guidestar location, the proximal field returned by the ATM can be replaced with u^{gs}. Points at the same radius as the guidestar may also be created with a higher power ratio than achievable with the ATM, by rotating the field used to generate the guidestar focus around the fibre axis  i.e., directly employing the rotational memory effect^{35}. This observation also points to a more sophisticated approach to reconstruct the ATM: instead of forcing \({\bf{D}}^{\prime}\) to be diagonal, we could try to iteratively search for a nondiagonal and potentially nonunitary \({\bf{D}}^{\prime}\), allowing the minimum power in offdiagonal elements^{43}, so that \({\bf{P}}{\bf{D}}^{\prime} {{\bf{P}}}^{\dagger }\) also perfectly focusses onto the guidestar when operating on input field u^{gs}. This is a severely underconstrained problem that we plan to investigate in more detail in the future.
Equation (7) stipulates that we must know the location of the guidestar on the distal facet in order to apply the phase correction term σ. However, if the location of the guidestar is unknown and σ is omitted (i.e., σ = 0), \({\bf{D}}^{\prime}\) still provides an estimate of the TM of the MMF, but with an unknown rotation between the proximal and distal planes. We note that in this case, the approximate radial location of a guidestar at an unknown position can be retrieved, as it is uniquely encoded in the relative amplitudes of the PIMs overlapping with the guidestar, which is extracted by decomposing u^{gs} in the PIM basis. This implies that the methods presented here may be possible even without specifically engineering a guidestar on the distal facet of the fibre. For example, a guidestar may be formed at a random location by optimising a nonlinear feedback signal at the proximal end of the fibre^{44}. Once the proximal field required to generate a focus, u^{gs}, is found, the focus can then be scanned in 2D using the methods we have presented here.
Discussion
In summary, we have demonstrated the possibility of guidestar assisted imaging through multimode fibres, which can be calibrated with access only to the proximal end of the fibre. We believe this technique to hold promise for the development of flexible multimode fibre based imaging systems. The reconstruction of the ATM, and of the images through the fibre, requires no iterative phase retrieval, and invokes no assumptions about the statistical properties of the scene at the end of the fibre—instead relying on assumptions about the properties of the fibre itself. Using a highspeed DMD capable of modulating input light at ~ 20 kHz, measurement of the proximal field that focuses on a fluorescent guidestar, u^{gs} (from which the ATM is calculated) consisting of, for example, ~1000 modes can be performed in ~200 ms using phase stepping holography^{38,45}. Using a highly reflective guidestar, the same calibration could potentially be achieved in a single shot—albeit with a lower SNR^{2,23} (Supplementary Note 8 contrasts these two methods). Therefore there is potential for both calibration and imaging to be performed in realtime.
In our proof of principle experiments, the isoplanatic patch does not extend over the full area of the distal facet. However our simulations show that the area of this isoplanatic patch can be significantly increased by making a better estimate of the basis in which the TM of the MMF is diagonal (see Fig. 3). Such an estimate can be achieved by optimising the fibre parameters with access to the full TM, before subsequent deployment of the system as a flexible microendoscope^{33,46}, which will be a focus of our efforts in future work. To image through longer MMFs which possess higher levels of mode coupling (and therefore lower values of p_{d} even after fibre parameter optimisation), we envisage that multiple guidestars, distinguished by, for example, emission frequency or speckle pattern contrast^{47,48}, could be used to increase the total area of the distal facet through which it is possible to image. If necessary, accurate guidestar placement could potentially be achieved using, for example, laserablationassisted attachment of fluorescent sensors^{49}.
We highlight that recently there have been several innovative proposals to achieve single ended TM characterisation of an MMF. For example, Gu et al. proposed recovery of the forward TM by measurement of the round trip reflection matrix of a fibre in which the distal facet has been engineered with a partial reflector of spatially varying reflection coefficient, and moveable shutter^{50}. The same group more recently suggested a simpler approach with a uniform partial reflector that can go some way to improving spot formation^{51}. Gordon et al. proposed the placement of a stack of patterned spectral filters at the distal facet and recovering the forward TM by measuring the reflection matrix at multiple frequencies^{52}. Chen et al. proposed placing a compact spatial multiplexer at the distal facet capable of imprinting a mode dependent time delay or frequency modulation^{53}. The engineering requirements of these proposals are technically challenging, and so they have yet to be experimentally realised in MMFs of high enough resolution to be useful for imaging. The reimaging properties of graded index fibres have also been demonstrated to enable a spot to be focussed at a chosen location within up to half of a distal facet using a nonlinear feedback signal from twophoton fluorescence^{54}. However, in this case, an optimisation process taking on the order of several minutes was required to achieve a focus at each new location. The guidestar assisted imaging concept we present here is arguably simpler than these recent proposals, although it may be more limited in terms of the accuracy of the forward TM that can be recovered, and the length of the fibre that may be calibrated. In addition to spatial correlations in weakly coupled MMFs, we also note that spatiotemporal correlations have been identified in MMFs with strong mode coupling that may prove useful for pulse delivery^{55,56}. Ultimately, we hope that some combination of our imaging strategy, in conjunction with the alternative concepts above, may provide the most robust method to enable imaging through flexible MMFs.
Beyond applications to step index optical fibres, and in answer to our questions posed earlier, we have shown that guidestar assisted imaging need not invoke tilt or shift memory effects, and can be extended to any scattering system in which we have an estimate of the basis in which the TM is quasidiagonal. Prior knowledge of this sort can be understood as a generalisation of the concept of the TMFP to spaces other than the momentum basis^{39}. i.e., advance knowledge that a scattering system possesses an optical path length that is much lower than a ‘generalised’ TMFP in a known (arbitrary) basis. The diagonal nature of the TM indicates that spatial modes input in the diagonal basis are not randomised into the entire vector space describing the allowed modes within the structure. Our work provides a guide on how to efficiently hunt for and make use of memory effects in optical systems of arbitrary geometry. This concept may be applied to a range of other media including gradedindex^{57,58} and photonic crystal fibres^{59}, fewmode fibre bundles (which have the constraint that the guidestar is ideally placed in the farfield of the output^{60}), photonic lanterns^{61}, opaque walls (i.e., imaging around corners)^{62}, and artificially engineered photonic networks and scattering systems.
Methods
Construction of matrix P
To perform guidestar assisted imaging through MMFs, we must estimate matrix P, that transforms from the propagation invariant mode (PIM) basis to realspace. The PIMs are found by solving the wave equation in a cylindrical geometry. This reduces to finding the roots of the scalar characteristic equations describing the modal dispersion^{32}.
where \({{\mathcal{J}}}_{\ell }\) is a Bessel function of the first kind of order ℓ, and \({{\mathcal{K}}}_{\ell }\) is a modified Bessel function of the second kind of order ℓ. u and ω are normalised transverse wavenumbers which are related to the fibre parameters through
Here a is the radius of the core, NA is the numerical aperture of the fibre and \({n}_{{\rm{core}}}\) and n_{clad} are the refractive indices of the core and cladding respectively. In this work we use the manufacturer’s values of a = 25 μm and NA=0.22. k is the vacuum wavenumber: k = 2π/λ, where λ is the vacuum wavelength. β is the propagation constant describing the phase velocity of each mode. Equation (10) may have multiple roots u_{ℓ,p}, indexed by the vortex charge ℓ specified in the order of the Bessel functions, and radial index p which counts the roots for a particular choice of ℓ. Once the roots have been found, the function ψ_{ℓ,p}(r, θ) represents the complex 2D field profile of the mode indexed by ℓ, p according to Equation (4). Here ω_{ℓ,p} are related to u_{ℓ,p} through Equation (13), and r and θ are the radial and azimuthal coordinates respectively across the core. Roots with a real phase velocity β_{ℓ,p} are propagating modes within the core. K_{ℓ} takes argument ω_{ℓ,p}, which is imaginary when \({\beta }_{\ell ,p}\;<\;{k}^{2}{n}_{core}^{2}\). This then describes the evanescent field in the cladding of the fibre (i.e., for the region r ≥ a). The PIMs, indexed by ℓ, p are ordered into a 1D list, indexed by n. Once the 2D complex functions describing each PIM have been found, column n of matrix P is constructed by representing the complex field of the n^{th} PIM in Cartesian coordinates on a 2D grid, and reshaping it into a column vector.
To account for some of the experimental misalignments, misalignment operators are measured and applied at each end of the fibre that remove course position and tilt misalignments of the input and output. These operators are found following the coarse alignment methods in ref. ^{33}. In brief, we make use of the fact that both ends of the MMF are accessible before it is employed as an endoscope, and measure the full realspace TM of the fibre (at a single polarisation). This real space TM can be processed without optimisation to reveal estimates of the position and tilt misalignments of the inputs and outputs in the following ways: (i) The central position of the core at output can be found by measuring the centreofmass of the sum of the intensity of all output measurements. (ii) The tilt of the output can be found by measuring the centreofmass of the sum of the intensity of the Fourier transform of all output measurements. Both of these methods can be understood as operations involving summing over the absolute square of the columns of the measured TM. Under the assumption that the TM is unitary, the transpose of this TM is equivalent to the TM if it were measured in the opposite direction through the fibre. Therefore the position of the core at the input, and the tilt of the input light can be found by performing the equivalent to (i) and (ii), but summing over the rows rather than the columns of the TM. Once these misalignments are found, they can be represented as misalignment matrices at the proximal end (R_{pr}) and at the distal end (R_{dl}), as described in more detail in Supplementary Note 4.
Simulations
The optical transformation properties of fibres in realspace are modelled following the methods in ref. ^{33}, by decomposing real space input fields into the PIMs, multiplying by a TM represented in the PIM ↦ PIM basis (D), and transforming back to real space (see Fig. 1a): T = PDP^{†}.
An ideal fibre (as simulated in Figs. 1 and 2), of length L is modelled by a diagonal matrix D, where the n^{th} diagonal element is given by \({e}^{iL{\beta }_{n}}\). To model nonideal fibres with quasidiagonal TMs in the PIM basis, as shown in Fig. 3, simulated misalignment matrices R are introduced that transform the realspace field at each end of the fibre into a new realspace basis that is laterally (dx and dy) and axially (dz) shifted, and tilted (about x and y), and defocussed a small randomly chosen amount. A quasidiagonal D_{q} with realistic offdiagonal coupling is created by absorbing these misalignment matrices into the TM in the PIM ↦ PIM basis, i.e., \({{\bf{D}}}_{{\rm{q}}}={{\bf{R}}}_{{\rm{dl}}}^{^{\prime} \dagger }{\bf{D}}{{\bf{R}}}_{{\rm{pr}}}^{\prime}\), where \({{\bf{R}}}_{{\rm{pr}}}^{\prime}={{\bf{P}}}^{\dagger }{{\bf{R}}}_{{\rm{pr}}}{\bf{P}}\) and \({{\bf{R}}}_{{\rm{dl}}}^{\prime}={{\bf{P}}}^{\dagger }{{\bf{R}}}_{{\rm{dl}}}{\bf{P}}\) are the two different misalignment operators at the proximal and distal end of the fibre respectively here represented in the PIM basis. Therefore to reduce the power on the diagonal (p_{d}) of D_{q}, and spread it off the diagonal in a realistic manner, the magnitudes of the randomly chosen misalignment in each dimension are increased.
Figure 3 shows modelling of the reconstruction of the ATM, and the performance of using it to focus and image through the fibre p_{d} is decreased. To simulate this, we use T = PD_{q}P^{†} to calculate the field required to focus on the guidestar, u^{gs}, where \({{\bf{u}}}^{{\rm{gs}}}={{\bf{T}}}^{\dagger }{{\bf{v}}}^{{m}_{0}}\), and column vector \({{\bf{v}}}^{{m}_{0}}\) represents the desired field in vectorised form (i.e., zeros everywhere except for 1 at the element m_{0} corresponding to the location of the guidestar). u^{gs} is then used to reconstruct the ATM \({\bf{T}}^{\prime}\) according to Eqs. (7)–(9). The ATM \({\bf{T}}^{\prime}\) is then used to calculate the estimated input fields required to raster scan a focus across the distal facet of the fibre, which are then propagated through the real TM T, enabling the power ratio maps and reconstructed images to be modelled.
Binary amplitude hologram design
A DMD is used to shape the light into the MMF during TM measurement and for imaging. To generate an arbitrary (bandwidth limited) spatially varying complex field \({\mathcal{A}}=A(x,y){e}^{i\alpha (x,y)}\) in the first diffraction order in the Fourier plane of the DMD (x and y being Cartesian coordinates of realspace in the optical Fourier plane of the DMD), we encode the inverse Fourier transform of \({\mathcal{A}}\), i.e., \({\mathcal{B}}={{\mathcal{F}}}^{1}({\mathcal{A}})=B(x^{\prime} ,y^{\prime} ){e}^{i\gamma (x^{\prime} ,y^{\prime} )}\), into a binary amplitude hologram to be displayed on the DMD, \({\bf{H}}(x^{\prime} ,y^{\prime} )\). Here \(x^{\prime}\) and \(y^{\prime}\) are Cartesian coordinates on the DMD chip. Following^{63,64}, \({\bf{H}}(x^{\prime} ,y^{\prime} )\) is given by
where
and where B_{max} is the maximum amplitude of B, and \({\phi }_{tilt}(x^{\prime} ,y^{\prime} )={k}_{x}x^{\prime} +{k}_{y}y^{\prime}\) is a phase gradient directing the desired beam into the first diffraction order at an angle defined by wavevectors k_{x} and k_{y}. This hologram H can be intuitively understood as one in which the local phase of the grating (i.e., local lateral position of the grating cycle) varies in proportion with the desired phase of the target complex field, and the local duty cycle of the grating varies as a function of the desired amplitude. H also results in light transmitted into other diffraction orders, which can be spatially filtered in the Fourier plane of the DMD, leaving only the desired optical field transmitted into the rest of the optical system. The wavevectors k_{x} and k_{y} also place a bandwidth limit on the complex field that can be generated and still successfully separated from other diffraction orders in the Fourier plane of the DMD.
Imaging through guidestar calibrated MMFs
The experimental setup, along with TM and ATM measurement are described in detail in Supplementary Notes 3 and 4. Once the TM and ATM are measured, they are used to capture the scanning imaging results are presented in Fig. 4. A transmissive resolution target mounted on a manually operated 3D translation stage was manoeuvred to ~ 40 μm from the distal facet of the fibre. The scanning spot was refocussed from the distal fibre facet to the plane of the resolution target by adding a quadratic Fresnel lens phase function to the hologram displayed on the DMD. The fully sampled TM or the ATM was used to calculate the proximal fields required to attempt to raster scan the focus across the resolution target. At each spot position x, y, the total transmitted intensity arriving at camera C1 was recorded (I(x, y)), which directly represented pixel x, y of the reconstructed image. We also attempted reflection based imaging by using a reflective resolution target and recording the total intensity of light reflected back through the fibre to camera C3. However we found that reflection images were very noisy due to the low laser power (1 mW) of the laser available for the experiment. Reflection imaging through MMFs has been successfully demonstrated in the past, but due to the low numerical aperture of MMFs, signals are typically very low when imaging beyond the end of the fibre facet, as we were doing here, and so laser powers on the order of at least several hundred mW are necessary for the return signal to be accurately measured. Reflection imaging using the ATM should be possible in future work using higher power lasers.
The extent of the isoplanatic patch in MMFs
The approximate area of the isoplanatic patch, A_{iso}, is related to the fraction of power remaining on the diagonal of the actual TM, p_{d}, according to: A_{iso} ~ πr^{2}p_{d}. This can be shown as follows:
Here Tr[⋅] refers to the trace of a matrix, and N_{1} and N_{2} are normalisation constants given by N_{1} = ∑_{n}∑_{m}∣D_{nm}∣^{2} and N_{2} = ∑_{n}∑_{m}∣S_{nm}∣^{2}. We have introduced matrix \({\bf{S}}={\bf{L}}{\bf{T}}{{\bf{T}}}^{^{\prime} \dagger }\), i.e., the real TM of the fibre represented with a change of input and output basis. The basis of S is chosen such that the input modes are the set of input fields predicted by the ATM to focus to each point across the distal facet of the fibre, and the output modes are the actual fields generated at the output of the fibre, which will be well formed foci only at positions within the isoplanatic patch. Here matrix L reduces the resolution of the output pixel basis to that commensurate with the size of the diffraction limited focus. Thus S natively describes the action of focussing using the ATM, and tends to the identity matrix as D tends to a perfectly diagonal matrix. The approximation that the normalised traces are equivalent in the first line of Eq. (19) is made under the assumption that the offdiagonal terms of D are small. An alternate way of expressing the fraction of power on the diagonal of S is given on the second line of Eq. (19), which corresponds to the mean powerratio of foci created at the output of the fibre. This mean powerratio also provides an estimate of the area of the isoplanatic patch A_{iso} as a fraction of the total area of the core (third line), by considering the most compact form function p_{r}(r, θ) can take: as we expect p_{r} to be highest near the guidestar, and the maximum value of p_{r} is limited to 1, then the area over which this maximum region could extend gives an approximation to A_{iso}.
Data availability
The experimental measurements shown in Figs. 4 and 5, and Supplementary Figs. 4–7, are available at https://doi.org/10.24378/exe.3203. All other data are available from the corresponding authors on request.
References
Vellekoop, I. M. & Mosk, A. P. Focusing coherent light through opaque strongly scattering media. Opt. Lett. 32, 2309–2311 (2007).
Yaqoob, Z., Psaltis, D., Feld, M. S. & Yang, C. Optical phase conjugation for turbidity suppression in biological samples. Nat. Photonics 2, 110 (2008).
Popoff, S. M. et al. Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media. Phys. Rev. Lett. 104, 100601 (2010).
Čižmár, T., Mazilu, M. & Dholakia, K. In situ wavefront correction and its application to micromanipulation. Nat. Photonics 4, 388 (2010).
Vellekoop, I. M., Lagendijk, A. & Mosk, A. P. Exploiting disorder for perfect focusing. Nat. Photonics 4, 320 (2010).
Popoff, S., Lerosey, G., Fink, M., Boccara, A. C. & Gigan, S. Image transmission through an opaque material. Nat. Commun. 1, 81 (2010).
DiLeonardo, R. & Bianchi, S. Hologram transmission through multimode optical fibers. Opt. Express 19, 247–254 (2011).
Mosk, A. P., Lagendijk, A., Lerosey, G. & Fink, M. Controlling waves in space and time for imaging and focusing in complex media. Nat. Photonics 6, 283 (2012).
Rotter, S. & Gigan, S. Light fields in complex media: Mesoscopic scattering meets wave control. Rev. Mod. Phys. 89, 015005 (2017).
Feng, S., Kane, C., Lee, P. A. & Stone, A. D. Correlations and fluctuations of coherent wave transmission through disordered media. Phys. Rev. Lett. 61, 834 (1988).
Freund, I., Rosenbluh, M. & Feng, S. Memory effects in propagation of optical waves through disordered media. Phys. Rev. Lett. 61, 2328 (1988).
Judkewitz, B., Horstmeyer, R., Vellekoop, I. M., Papadopoulos, I. N. & Yang, C. Translation correlations in anisotropically scattering media. Nat. Phys. 11, 684–689 (2015).
Yílmaz, H. et al. Angular memory effect of transmission eigenchannels. Phys. Rev. Lett. 123, 203901 (2019).
de Boer, J. F., van Albada, M. P. & Lagendijk, A. Transmission and intensity correlations in wave propagation through random media. Phys. Rev. B 45, 658 (1992).
Kadobianskyi, M., Papadopoulos, I. N., Chaigne, T., Horstmeyer, R. & Judkewitz, B. Scattering correlations of timegated light. Optica 5, 389–394 (2018).
Choi, Y. et al. Scannerfree and widefield endoscopic imaging by using a single multimode optical fiber. Phys. Rev. Lett. 109, 203901 (2012).
Čižmár, T. & Dholakia, K. Exploiting multimode waveguides for pure fibrebased imaging. Nat. Commun. 3, 1027 (2012).
Ohayon, S., CaravacaAguirre, A., Piestun, R. & DiCarlo, J. J. Minimally invasive multimode optical fiber microendoscope for deep brain fluorescence imaging. Biomed. Opt. express 9, 1492–1509 (2018).
Turtaev, S. et al. Highfidelity multimode fibrebased endoscopy for deep brain in vivo imaging. Light.: Sci. Appl. 7, 92 (2018).
Bertolotti, J. et al. Noninvasive imaging through opaque scattering layers. Nature 491, 232 (2012).
Fienup, J. R. Reconstruction of an object from the modulus of its fourier transform. Opt. Lett. 3, 27–29 (1978).
Fugate, R. Q. et al. Measurement of atmospheric wavefront distortion using scattered light from a laser guidestar. Nature 353, 144–146 (1991).
Yoon, S. et al. Deep optical imaging within complex scattering media. Nat. Rev. Phys. 2, 1–18, 2020.
Vellekoop, I. M., Cui, M. & Yang, C. Digital optical phase conjugation of fluorescence in turbid tissue. Appl. Phys. Lett. 101, 081108 (2012).
Ruan, H. et al. Focusing light inside scattering media with magneticparticleguided wavefront shaping. Optica 4, 1337–1343 (2017).
Zhou, E. H., Ruan, H., Yang, C. & Judkewitz, B. Focusing on moving targets through scattering samples. Optica 1, 227–232 (2014).
Kong, F. et al. Photoacousticguided convergence of light through optically diffusive media. Opt. Lett. 36, 2053–2055 (2011).
Judkewitz, B., Wang, Y. M., Horstmeyer, R., Mathy, A. & Yang, C. Specklescale focusing in the diffusive regime with time reversal of varianceencoded light (trove). Nat. photonics 7, 300 (2013).
Horstmeyer, R., Ruan, H. & Yang, C. Guidestarassisted wavefrontshaping methods for focusing light into biological tissue. Nat. Photonics 9, 563 (2015).
Osnabrugge, G., Horstmeyer, R., Papadopoulos, I. N., Judkewitz, B. & Vellekoop, I. M. Generalized optical memory effect. Optica 4, 886–892 (2017).
Papadopoulos, I. N., Jouhanneau, J.S., Poulet, J. F. A. & Judkewitz, B. Scattering compensation by focus scanning holographic aberration probing (fsharp). Nat. Photonics 11, 116–123 (2016).
Okamoto, K. Fundamentals of Optical Waveguides. (Academic press, 2006).
Plöschner, M., Tyc, T. & Čižmár, T. Seeing through chaos in multimode fibres. Nat. Photonics 9, 529 (2015).
Leach, J., Padgett, M. J., Barnett, S. M., FrankeArnold, S. & Courtial, J. Measuring the orbital angular momentum of a single photon. Phys. Rev. Lett. 88, 257901 (2002).
Amitonova, L. V., Mosk, A. P. & Pinkse, P. W. H. Rotational memory effect of a multimode fiber. Opt. express 23, 20569–20575 (2015).
Shapiro, J. H. Computational ghost imaging. Phys. Rev. A 78, 061802 (2008).
Amitonova, L. V. & De Boer, J. F. Compressive imaging through a multimode fiber. Opt. Lett. 43, 5427–5430 (2018).
Turtaev, S. et al. Comparison of nematic liquidcrystal and dmd based spatial light modulation in complex photonics. Opt. Express 25, 29874–29884 (2017).
Xiong, W. et al. Principal modes in multimode fibers: exploring the crossover from weak to strong mode coupling. Opt. Express 25, 2709–2724 (2017).
Leite, I. T. et al. Threedimensional holographic optical manipulation through a highnumericalaperture softglass multimode fibre. Nat. Photonics 12, 33–39 (2018).
Zhu, L. et al. Chromatoaxial memory effect through a forwardscattering slab. Optica 7, 338–345 (2020).
Boniface, A., Mounaix, M., Blochet, B., Piestun, R. & Gigan, S. Transmissionmatrixbased pointspreadfunction engineering through a complex medium. Optica 4, 54–59 (2017).
Li, S. et al. Compressively sampling the optical transmission matrix of a multimode fibre. Light.: Sci. Appl. 10, 1–15 (2021).
Sivankutty, S. et al. Ultrathin rigid endoscope: twophoton imaging through a gradedindex multimode fiber. Opt. express 24, 825–841 (2016).
Conkey, D. B., CaravacaAguirre, A. M. & Piestun, R. Highspeed scattering medium characterization with application to focusing light through turbid media. Opt. express 20, 1733–1740 (2012).
Matthès, M. W., Bromberg, Y., de Rosny, J., & Popoff, S. M. Learning and avoiding disorder in multimode fibers. Preprint at: https://arxiv.org/abs/2010.14813 (2020).
Boniface, A., Blochet, B., Dong, J. & Gigan, S. Noninvasive light focusing in scattering media using speckle variance optimization. Optica 6, 1381–1385 (2019).
Boniface, A., Dong, J. & Gigan, S. Noninvasive focusing and imaging in scattering media with a fluorescencebased transmission matrix. Nat. Commun. 11, 6154 (2020).
Kamaljith, V. et al. Ultrafastlaserablationassisted spatially selective attachment of fluorescent sensors onto optical fibers. Opt. Lett. 45, 2716–2719 (2020).
Gu, R. Y., Mahalati, R. N. & Kahn, J. M. Design of flexible multimode fiber endoscope. Opt. express 23, 26905–26918 (2015).
Gu, R. Y., Chou, E., Rewcastle, C., Levi, O., and Kahn, J. M. Improved spot formation for flexible multimode fiber endoscope using partial reflector. Preprint at: https://arxiv.org/abs/1805.07553 (2018).
Gordon, G. S. D. et al. Characterizing optical fiber transmission matrices using metasurface reflector stacks for lensless imaging without distal access. Phys. Rev. X 9, 041050 (2019).
Chen, H., Fontaine, N. K., Ryf, R., Neilson, D. T. & Winzer, P. Remote spatiotemporal focusing over multimode fiber enabled by singleended channel estimation. IEEE J. Sel. Top. Quantum Electron. 26, 1–9 (2020).
Rosen, S., Gilboa, D., Katz, O. & Silberberg, Y. Focusing and scanning through flexible multimode fibers without access to the distal end. Preprint at: https://arxiv.org/abs/1506.08586 (2015).
Xiong, W., Hsu, C. W. & Cao, H. Longrange spatiotemporal correlations in multimode fibers for pulse delivery. Nat. Commun. 10, 1–7 (2019).
Mounaix, M. & Carpenter, J. Control of the temporal and polarization response of a multimode fiber. Nat. Commun. 10, 1–8 (2019).
Carpenter, J., Eggleton, B. J. & Schröder, J. 110 × 110 optical mode transfer matrix inversion. Opt. express 22, 96–101 (2014).
Flaes, D. E. B. et al. Robustness of lighttransport processes to bending deformations in gradedindex multimode waveguides. Phys. Rev. Lett. 120, 233901 (2018).
Mekis, A. et al. High transmission through sharp bends in photonic crystal waveguides. Phys. Rev. Lett. 77, 3787 (1996).
Kuschmierz, R., Scharf, E., Koukourakis, N. & Czarske, J. W. Selfcalibration of lensless holographic endoscope using programmable guide stars. Opt. Lett. 43, 2997–3000 (2018).
Choudhury, D. et al. Computational optical imaging with a photonic lantern. Nat. Commun. 11, 1–9 (2020).
Katz, O., Heidmann, P., Fink, M. & Gigan, S. Noninvasive singleshot imaging through scattering layers and around corners via speckle correlations. Nat. photonics 8, 784 (2014).
Brown, B. R. & Lohmann, A. W. Complex spatial filtering with binary masks. Appl. Opt. 5, 967–969 (1966).
Lee, W.H. Binary computergenerated holograms. Appl. Opt. 18, 3661–3669 (1979).
Acknowledgements
D.B.P. thanks Dr. Ben Sherlock for help with early parts of the experiment, Prof. Jacopo Bertolotti for useful discussions and careful reading of the manuscript, and Emily Beth WaiSee for encouragement. D.B.P. and S.L. thank Dr. Sergey Turtaev and Dr. Ivo Leite for advice on setting up the experiment. S.L. acknowledges support from the National Natural Science Foundation of China under Grant no. 61705073. S.A.R.H. acknowledges financial support from a Royal Society TATA University Research Fellowship (RPG2016186). T.C. thanks the European Regional Development Fund (CZ.02.1.01/0.0/0.0/15_003/0000476) and the European Research Council (724530) for financial support. D.B.P. acknowledges financial support from the Royal Academy of Engineering and the European Research Council (804626).
Author information
Authors and Affiliations
Contributions
D.B.P., T.T. and T.C. conceived the idea for the project. D.B.P. supervised the project. S.A.R.H. and D.B.P. formulated the general memory effect theory. S.A.R.H., T.T. and D.B.P. investigated the quasiradial memory effect. D.B.P., S.A.R.H., T.T. and S.L. performed the simulations. S.L. built the optical setup and performed all experiments and analysis, with support from D.B.P. and T.C. D.B.P., S.A.R.H. and S.L. wrote the manuscript with editorial input from all other authors.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Communications thanks Antoine Boniface and Mickael Mounaix for their contribution to the peer review of this work. Peer reviewer reports are available.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Li, S., Horsley, S.A.R., Tyc, T. et al. Memory effect assisted imaging through multimode optical fibres. Nat Commun 12, 3751 (2021). https://doi.org/10.1038/s41467021237291
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467021237291
This article is cited by

Flexible ultrathin superresolution endoscopy
Science China Physics, Mechanics & Astronomy (2024)

Directly and instantly seeing through random diffusers by selfimaging in scattering speckles
PhotoniX (2023)

Singleended recovery of optical fiber transmission matrices using neural networks
Communications Physics (2023)

Highgain and highspeed wavefront shaping through scattering media
Nature Photonics (2023)

Single multimode fibre for in vivo lightfieldencoded endoscopic imaging
Nature Photonics (2023)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.