Light-field microscopy with correlated beams for high-resolution volumetric imaging

Light-field microscopy represents a promising solution for microscopic volumetric imaging, thanks to its capability to encode information on multiple planes in a single acquisition. This is achieved through its peculiar simultaneous capture of information on light spatial distribution and propagation direction. However, state-of-the-art light-field microscopes suffer from a detrimental loss of spatial resolution compared to standard microscopes. In this article, we experimentally demonstrate the working principle of a new scheme, called Correlation Light-field Microscopy (CLM), where the correlation between two light beams is exploited to achieve volumetric imaging with a resolution that is only limited by diffraction. In CLM, a correlation image is obtained by measuring intensity correlations between a large number of pairs of ultra-short frames; each pair of frames is illuminated by the two correlated beams, and is exposed for a time comparable with the source coherence time. We experimentally show the capability of CLM to recover the information contained in out-of-focus planes within three-dimensional test targets and biomedical phantoms. In particular, we demonstrate the improvement of the depth of field enabled by CLM with respect to a conventional microscope characterized by the same resolution. Moreover, the multiple perspectives contained in a single correlation image enable reconstructing over 50 distinguishable transverse planes within a 1 mm3 sample.

. Comparison of resolution and DOF limits of three microscopy techniques: standard microscopy (with no directional resolution), standard light-field microscopy, and CLM. Here, is the light wavelength and �x 0 = 0.61 /NA 0 the diffraction-limited resolution cell, with NA 0 the numerical aperture of the microscope. N u is the number of directional resolution cells per side in standard light-field imaging, and a is the size of the smallest details within the sample. The first and second columns represent the resolution limit in the focal plane and the maximum DOF achievable for objects with details at the resolution limit, respectively. The third column indicates the maximum DOF achievable for objects with detail size a > �x 0 larger than the resolution limit. The fourth column represents the axial resolution for objects of detail size a, which, as shown in the "Materials and methods" section, does not depend on the specific technique. The last column reports the number of viewpoints per direction. Properties of the light-field microscope are derived by the general features of light-field imaging devices (see Ref. 46 for a detailed discussion), while the resolution and DOF limits in CLM are derived from the theoretical analysis reported in Ref. 44 , as well as in the "Materials and methods" section. Notice that all the quantities are evaluated in the limit of small displacements with respect to the objective focal plane.

Microscopy method
Resolution limit DOF at resolution limit DOF extension at object detail size a Axial resolution at object detail size a Viewpoint multiplicity www.nature.com/scientificreports/ of the details of interest. In sharp contrast with conventional light-field microscopy, the viewpoint multiplicity of CLM can be even one order of magnitude larger than in conventional light-field imaging, without the diffractionlimited resolution to be affected; this is especially true for imaging systems with a large numerical aperture and for refocusing far away from the focused plane (i.e., for large values of object details a). Most important, the DOF extension capability of CLM is independent on the numerical aperture of the imaging systems (see "Materials and methods" for details); a large numerical aperture can thus be chosen for maximizing the volumetric resolution without affecting the DOF. This is very different from conventional light-field microscopy, where the numerical aperture and N u need to be properly chosen to achieve an acceptable compromise of resolution and DOF. The paper is organized as follows. In the "Results" section, we outline the theoretical basis of the method and show the experimental results. In the "Discussion" section, we discuss the results, their impact on state of the art, and the possibilities of further improvement. In the "Materials and methods" section, we provide a detailed description of the experimental setup and on the methods to extract relevant information from correlation measurements.

Results
Concept. The correlation light-field microscope, schematically represented in Fig. 1, is based on a conventional microscope made of an objective lens (O) and a tube lens (T) to reproduce the image of the sample on a high resolution sensor array (detector D a ); this microscope can properly reconstruct only the slice of the three-dimensional object falling within its DOF. The capability of CLM to refocus out-of-focus parts of the three-dimensional sample comes from its ability to also gain directional information about light coming from the sample. In our architecture, this is done by means of the beam splitter (BS) that reflects a fraction of light emerging from the objective lens toward an additional lens (L), which images the objective lens on a second high resolution sensor array (detector D b ). Further details on the experimental setup are reported in the "Materials and methods" section.
The three-dimensional sample is a chaotic light emitter or, alternatively, a diffusive, transmissive, or reflective sample illuminated by an external chaotic light source. The chaotic nature of light enables light-field imaging thanks to the rich information encoded in correlations between intensity fluctuations. In fact, the intensity Figure 1. Schematic representation of light-field microscopy with correlated beams. Light from the sample (green pyramid and yellow parallelepiped) is split in two optical paths by a beam splitter (BS), placed after the objective lens (O). The microscope can image only the part of the sample which is at focus (green), while the part outside the DOF is blurred (yellow). Along the transmitted path, the tube lens (T) focuses on detector D a (blue) the 3D sample, with a DOF defined by the numerical aperture of the objective. Along the reflected path of the BS, the objective lens is imaged on detector D b (magenta) by means of an additional lens (L). The intensity patterns registered by the two array detectors, in a collection of N frames, are processed by a computer to reconstruct the correlation function (Eq. (1)) encoding three-dimensional plenoptic information on the sample. All experimental results presented throughout the paper have been obtained by employing this setup. where . . . is the average over the source statistics, I a (ρ a ) and I b (ρ b ) are the intensities at the transverse positions ρ a and ρ b on detectors D a and D b , within the same frame, respectively, and �I j ( The statistical reconstruction of the correlation function requires, under the hypotheses of a stationary and ergodic source, to collect a set of N independent frames. In the best-case scenario, the exposure time of each frame is matched with the coherence time of the source. The light-field imaging capability of CLM explicitly emerges when considering the geometrical optics limit of the above correlation function, which reads 44,45 where F(ρ s ) is the intensity profile of light from the sample, P(ρ O ) is the intensity transmission function of the objective, f is the distance from the objective of the generic plane within the three-dimensional object, f T is the focal length of the tube lens, and M L is the magnification of the image of the objective lens retrieved by D b . When the plane of interest is on focus (i.e., f = f O , with f O the focal length of the objective), the correlation simply gives a focused image identical to the one retrieved by detector D a . However, as shown in Fig. 2, points of the three-dimensional samples that are out-of-focus (i.e., they lie in planes at a distance f = f O from the objective) are seen as shifted, and their displacement depends on the specific pixel ρ b chosen on sensor D b , corresponding to the point ρ O = −ρ b /M L on the objective. In other words, for three-dimensional samples that are thicker than the natural DOF of the microscope, different values of ρ b correspond to different choices of the point of view on the sample: The correlation function in Eq. (1) has the form of a four-dimensional array, characterized by both detector coordinates (x a , y a , x b , y b ) , encoding all the spatial and angular information needed for refocusing and multi-perspective image. By fixing the coordinates (x b , y b ) of the 4D array, one makes a "slice" of the correlation function, which corresponds to selecting an image of the sample from a chosen viewpoint on the objective lens. This property enables to detect the position of sample details, in three dimensions, and to highlight hidden parts of the sample. The refocused image of a sample plane placed at an arbitrary distance f from the objective can be obtained by properly stacking and summing such different perspectives 44,45 : with M = f T /f O being the natural microscope magnification. The refocusing procedure increases significantly the signal-to-noise ratio (SNR) with respect to the one characterizing the single perspective associated with ρ b . Notice that the technique can be generalized to the case of samples with axially varying diffraction index, upon replacing physical distances with optical distances. Moreover, reconstruction of the correlation function was proved to be robust against the presence of turbulence and scattering surrounding the sample, within a limit distance defined by the emitted light wavelength and transverse coherence 44 .
Before moving on to the experimental demonstration of CLM, let us spend a few words on the advantages offered by the capability of CLM to collect a large number of viewpoints, as enabled by the large objective plane on which perspectives can be selected, and by the consequent wide angle from which the sample can be observed. www.nature.com/scientificreports/ First, refocusing shares with 3D imaging the close connection between maximum observation angles and achievable DOF. Second, the higher the number of points of view that are superimposed to obtain the refocused image (see the integration over ρ b in Eq. (3)), the more effective will be both the enhancement of features on the plane of interest and the suppression of contributions from neighboring planes, as well known in 3D imaging. Moreover, superimposing a large number of perspectives to form the final image is advantageous in terms of noise reduction. Indeed, when points of view with statistically independent noise are summed, the process results in a pointwise increase of the signal-to-noise ratio that is proportional to the square root of the number of contributions.
Volumetric imaging by CLM. The refocusing and depth mapping capability of CLM has preliminarly been tested with a simple and controllable three-dimensional object, made of two planar resolution targets (named 3D 1 and 3D 2 ) placed at two different distances from the objective lens, well outside its natural DOF. Data have been acquired by focusing the correlation light-field microscope onto a plane that did not contain neither one of the two test targets. Correlation measurements, combined with Eq. (3), has been employed to refocus the two test targets, separately, starting from this dataset. The illuminated parts of both targets contain triple slits with center-to-center distance d = 49.6 µm and slit width a = d/2 ; the overall linear field of view (FOV) is 0.54 mm . The test targets are placed at a distance of 2.5 mm from each other (i.e. where f O is the focal length of the objective lens), which is 6 times larger than the natural DOF of the microscope at the given size of the sample details (i.e., the circle of confusion, see Table 1). The reported results have been obtained by evaluating the correlation function over N = 5 × 10 3 acquired frames; we report in the Supplementary Information further details on SNR variation with the number of collected frames. The improvement of CLM over standard microscopy can be observed in Fig. 3a, where we report the resolution-versus-DOF compromise in the two microscopes. In particular, the curves indicate the resolution limits of CLM and a standard microscope (SM) with the same numerical aperture, as a function of the distance from the objective focal plane. The resolution limit is defined in both cases as the value d of the center-to-center distance between two slits of width a = d/2 , such that their images can be discriminated at 10% visibility; such a definition generalizes the Rayleigh criterion for resolution to out-of-focus images. For a fixed slit separation d (vertical axis), one can identify the longitudinal range f − f O (horizontal axis) where the images of the two slits can be discriminated, based on our theoretical results. Points labeled from A to E and from A' to D' in Fig. 3a were experimentally investigated to demonstrate the agreement of the experimental results with the theoretical prediction of the resolution and DOF limits. We explored the range between −1 mm and +1 mm along the optical axis, in steps of 250 µ m, by employing different triple-slit masks of a planar resolution test target, characterized by centerto-center distances ranging from 44 µm (A and A') to 4 µm (E). In particular, point E is close to the diffraction limit of a standard microscope and shows that CLM is capable of the same resolution at focus. The experimental validation of the CLM refocusing capability for the cases A and D' is reported in Fig. 3b; the refocused images obtained in all the other cases are reported for completeness in the Supplementary Information. The red points in Fig. 3a identify the parameters of the resolution targets 3D 1 and 3D 2 that compose our three-dimensional test object. The successful experimental refocusing of both targets, reported in Fig. 3c, demonstrates that CLM enables achieving a 6 times larger DOF than standard microscopy, at the given resolution, or, alternatively, a 6 time better resolution, at the given DOF. The leftmost panel reports for comparison the standard microscope image of the three-dimensional test object, in which both target planes are clearly out of focus and none of the triple-slit groups can be recognized. Remarkably, the sets of triple slits placed at f 3D 2 − f O = 1250 µm (i.e., the object placed farthest from the objective) are still perfectly resolved. This means that the resolution achieved on farther planes is not heavily influenced by the presence of other details placed along the optical path, despite the substantial spatial filtering that they perform.
The results in Fig. 3c also demonstrate that CLM improves by over one order of magnitude the acquisition speed with respect to previous correlation-based light-field imaging protocols 37,38 , where 5 × 10 4 frames (to be compared with the current 5 × 10 3 ) and additional low-pass Gaussian filtering were employed in post-processing to achieve a comparable SNR. This improvement directly comes from the elimination of ghost imaging from the CLM architecture, and its replacement by conventional imaging at both sensor arrays. Actually, correlation between direct images has been shown to enable a significant improvement of the SNR with respect to ghost imaging 38 .
After the three-dimensional target, we tested the effectiveness of CLM on a thick phantom reproducing features of interest in biomedical applications; the sample is made of birefringent starch granules, suspended at random positions in a transparent non-birefringent gel. The focused plane inside the sample was arbitrarily chosen at approximately half of its thickness. In Fig. 4a, we show the standard image of the focused plane, while Fig. 4b reports the images of four different planes refocused by CLM, located at an optical distance from the focused plane of −10 µm , −130 µm , −310 µm , and +200 µm , respectively. It is evident that some aggregates appear focused in only one of the four images, which provide a tool to identify their longitudinal optical distance from the focal plane. The volumetric resolution of CLM enabled us to refocus 54 planes over a 1 mm thick volume, with a transverse resolution smaller than 20 µm and a longitudinal resolution smaller than 90 µm , within a FOV of about 1 mm 2 (see video in the Supplementary Information).
Interestingly, in the current CLM architecture, the SNR is high enough for images from different viewpoints to be effectively observable (hence, available for further data analysis, such as three-dimensional reconstruction). In    www.nature.com/scientificreports/ viewpoints images in which the details of the object can be clearly distinguished, a feature which is particularly relevant in view of implementing 3D reconstruction algorithms based on viewpoint multiplicity.

Discussion
The refocusing capability of CLM has brought to a 6 times larger DOF than in a conventional microscope having the same numerical aperture, and the same (diffraction-limited) resolution in the focused plane. These results are in excellent agreement with the expected refocusing range of CLM 44 at the given resolution ( d = 50 µm ), thus showing the reliability of the proposed CLM architecture. The volumetric resolution achieved by CLM to image a complex thick sample ( 1 × 1 × 1 mm 3 ) is a further very interesting result, considering the scanningfree nature of CLM. In the employed CLM setup, we obtained 54 independent axial sections of the biological phantom. Considering how the axial and transverse resolution change along the optical axis, this corresponds to a total 7.21 × 10 6 voxels within the volume of interest. Notice that the device considered in this work is meant as a proof-of-principle demonstrator, and its parameters are not optimized. However, the properties reported in Table 1 provide a guideline to scale the setup towards smaller resolutions in view of real applications. All these results are unattainable by standard light-field microscopy, due to its hyperbolic tradeoff between spatial resolution and multi-perspective views (hence, maximum achievable DOF). CLM shares with traditional light-field microscopy the capability of performing three-dimensional imaging without moving the sample nor any part of the optical apparatus; resolution depends on the distance from the focal plane, as shown in Fig. 3, but is uniform in the transverse directions, as far as light propagation can be considered paraxial 44 . These features can be compared to the properties of a consolidated technique such as confocal microscopy, which requires both longitudinal and transverse scanning to perform imaging with uniform volumetric resolution, and of a much less time-consuming technique such as light-sheet microscopy with non-diffracting beams, which requires only longitudinal scanning, trading this interesting feature with inhomogeneous illumination and resolution in the transverse direction. The main drawback of CLM lies in its operational definition: while in incoherent first-order imaging the SNR can be increased by just exposing the sensor for a time much larger than the source coherence time, a reliable reconstruction of the correlation function (1) requires to collect a large number of distinct frames, whose duration should be preferably matched with the source coherence time.
Increasing the acquisition speed of CLM is the principal challenge that needs to be addressed to guarantee its competitiveness with state-of-the art light-field microscopes 25 . Such speed-up is in fact of paramount importance both for avoiding radiation damage of biomedical samples, for in vivo imaging, and for studying dynamic processes. The large SNR of CLM with respect to the original CPI scheme represents a first significant step in this direction, as it enabled to increase the acquisition speed by one order of magnitude, still guaranteeing an even higher SNR (see Ref. 33 ). Similar to the approach for noise mitigation implemented here (Figs. 4, 5), and outlined in the "Materials and methods" section, a further step toward acquisition speed-up is compressive sensing and deep learning techniques, as increasingly applied to imaging tasks [47][48][49][50] . From the hardware viewpoint, the acquisition speed of our microscope has ample room for improving, both by investigating possible optimizations in our current acquisition routine and by employing cameras with better time performance. The most immediate way to start boosting the time performance of the current CLM, for example, is to employ the camera (see "Materials and methods" section) in rolling-shutter mode, rather than global shutter, which we have been using to guarantee that the retrieved intensity patterns I a and I b (which are then correlated pixel by pixel) are simultaneous statistical sampling of the chaotic source. This condition is consistent with the theoretical model (i.e., Eq. (1)), but it is certainly interesting to search for a regime in which moving slightly away from the theory introduces small enough artifacts to justify the gain in speed. With our camera, this could mean even doubling the frame rate, reducing the current acquisition time to about 20 seconds (from the present 43). Also the chaotic source can be significantly improved by replacing the ground-glass disk now in use (see the "Materials and methods" section) with a digital micromirror device (DMD), which adds versatility and available statistics, while significantly decreasing the source coherence time due to its typical frame rate of about 30 kHz. Also, since the DMD patterns are completely user-controllable, their features can be customized to achieve the desired SNR with the lowest number of frames possible, even experimenting with structured illumination. In this scenario, the acquisition speed will essentially be limited by the maximum frame rate of the sensor and, eventually, by the data transferring speed. This issue can be addressed by replacing our current sCMOS with faster cameras, capable of reaching 6.6 kfps at full resolution 51 , or with ultra-fast high-resolution SPAD arrays, enabling acquisition rates as high as 10 5 binary frames per second, in a 512 × 512 array 52,53 . When choosing alternative cameras, speed should not be favored at the expenses of readout noise, dynamic range, detection efficiency, or minimum exposure time, all which are relevant parameters in correlation-based imaging. In this respect, SPAD arrays are of particular interest due to their much shorter minimum exposure time, ranging from a few hundreds of ps to 10 ns 52-55 , although their binary nature may pose challenges. The minimum exposure time of the camera also regulates the possibility of extending CLM to uncontrolled thermal sources, including the fluorescence samples at the core of the related microscopy technique. CLM and fluorescence microscopy are certainly compatible due to the chaotic nature of fluorescent light, but an experimental challenge needs to be addressed in such a context: matching the low coherence time of fluorescent light with the minimum exposure time of the sensor. An analogous problem was successfully faced by Hanbury-Brown and Twiss in the Narrabri stellar interferometer 56 and in more recent correlation imaging experiments performed with sunlight 57,58 . In the context of CLM, we shall address this challenge in future works.
We finally remark that the need for many short exposure times to create a single image is common to other established microscopy techniques, such as STORM 59 . This result is encouraging in view of avoiding photobleaching and photo-damage, also considering that the SNR requirements on each single frame of CLM are www.nature.com/scientificreports/ much weaker than in STORM, where the signal should be clear enough to allow centroid estimation. Exposurerelated problem could also be reduced by modulating illumination in time, with a frequency matched with the acquisition frame rate.

Materials and methods
Experimental setup. The experimental setup employed to demonstrate CLM is shown in Fig. 6. The controllable chaotic light source is a single-mode laser with wavelength = 532 nm (CNI MLL-III-532-300 mW) illuminating a rotating ground glass disk (GGD), with diffusion angle θ d ≃ 14 • , whose speed defines the source coherence time ( ≈ 90µs). The laser spot size on the disk is enlarged to a diameter of 8 mm by a 6× beam expander, and the sample is placed at a distance of 10 mm after the GGD; the effective numerical aperture of our systems is thus NA = 0.23 which defines our expected diffraction-limited resolution δ = 1. To fully exploit the dynamic range of the camera and maximize the SNR, we balance the intensities of the beams on the two halves of the sCMOS camera by means of a half-wave plate placed before in the laser beam, before the GGD. The camera sensor is characterized by 2560 × 2160 pixels of size δ p = 6.5 µm and can work at up to 50 fps in full-frame mode (in global shutter mode, 100 fps with rolling shutter). Since the resolution δ = 1.6 µm on the object corresponds to a magnified resolution cell Mδ = 6.7 µm on the sensor, data reported in Figs. 4, 5 were generally acquired with a 2 × 2 hardware binning; no binning was applied when acquiring data corresponding to points C, D, their primed counterparts, and E, in Fig. 3a.
The test targets employed to acquire data reported in Fig. 3 are Thorlabs R3L3S1N and R1DS1N. The exposure time was set at τ = 92.3 µs to match the coherence time fo the source, and the acquisition rate of the camera to R = 120 Hz , the maximum speed possible at our FOV in global shutter mode.
Noise mitigation in the CLM correlation function. All the refocused images reported in the article are obtained by applying the refocusing formula in Eq. (3) to the experimental four-dimensional correlation function. We also applied a correction to eliminate edge-effects due to the size of the detectors, as described in Ref. 45 . The problem of the noisy background occurring in the refocused images of the biomedical phantom (Figs. 4,5) was tackled by pre-processing the correlation function. The statistical noise, which is quantified by the variance of the quantity measured in Eq. (1), can be reduced by optimizing the correlation function with the introduction of an additional term; this approach is consistent with so called differential ghost imaging 60 , where each pixel of www.nature.com/scientificreports/ D b is considered as a bucket detector. The mentioned correction consists in subtracting from the correlation function a spurious self-correlation between intensity fluctuations on each single pixel of the spatial detector D a and intensity fluctuations on the whole detector, thus obtaining the modified correlation with I TOT a the total intensity impinging on the detector D a . The free parameter K can be fixed by the condition of minimizing the variance of the modified correlation function. Derivation of F (ρ a , ρ b ) with respect to K immediately shows that the minimum is reached for The different outcome of the analysis when considering the standard correlation function of Eq. (1) and the modified one of Eq. (4) can be found in the Supplementary Information. Viewpoint multiplicity. The viewpoint multiplicity is defined as the effective number of viewpoints per transverse direction. In the case of CLM, the viewpoint multiplicity is estimated as the number of resolution cells falling within the diameter D of the objective lens. The resolution cell must be evaluated by considering that, in the correlation function, the sample acts as an aperture, and thus determines the resolution on the objective lens (see Ref. 44  Axial resolution. The DOF of each single refocus image provides information on the CLM axial resolution. As reported in the second column of Table 1, CLM is characterizd by the same DOF as a standard microscope in the objective focal plane, determined by the numerical aperture and wavelength of illumination. When the object is moved out of focus, the DOF of the refocused image can be determined by a geometrical-optics approach. If two point sources, placed on opposite sides of the optical axis and separated by a distance a along the transverse direction x, are located on a transverse plane placed at a distance z from the objective, the correlation function they generate reads as given by Eq. (2), with δ (2) (ρ) the two-dimensional Dirac delta and û x the unit vector along the x-axis. By refocusing the image described by Eq. (9) through the refocusing algorithm of Eq. (3), we see that, as soon as refocusing is implemented on an axial coordinate different from the one identifying the object position ( f = z ), each point-like source generates a "circle of confusion" analogous to the one of conventional imaging. Unlike other CLM features, the circle of confusion depends on the numerical aperture of the CLM device. The refocused image at a generic distance f = z reads Assuming the objective lens transmission function P is a circular iris of radius R, Eq. (10) represents two circles of radius MR|f − z|/z and centered in ±Mf /2z a . Therefore, there will exist a range for which the refocusing parameter f generates two separate circles; outside of that range, the two circles begin to overlap and, ultimately, the two sources can no longer be resolved. Particularly, there will be two refocusing positions f ′ and f ′′ for which Eq. (10) describes two tangent circles. We thus define the DOF of the refocused images as  www.nature.com/scientificreports/ where the approximation holds for |f ′ − f ′′ | ≪ f O . Hence, depending on the size of the object details of interest, the images refocused by CLM have a DOF that depends on the numerical aperture in the same fashion as for a standard microscope. However, the ratio between the extended DOF available to CLM and the DOF of the refocused image gives the number of independent planes accessible through refocusing, which is: This result shows that, as is the case of conventional light-field imaging, the number of longitudinal planes that can be refocused is proportional to the viewpoint multiplicity on the lens plane. The advantage of CLM over conventional light-field imaging is the larger accessible viewpint multiplicity, as already discussed in the "Introduction".

Data availability
The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.