Opposite-view digital holographic microscopy with autofocusing capability

Digital holographic microscopy (DHM) has its intrinsic ability to refocusing a sample by numerically propagating an object wave from its hologram plane to its image plane. In this paper opposite-view digital holographic microscopy (OV-DHM) is demonstrated for autofocusing, namely, digitally determining the location of the image plane, and refocusing the object wave without human intervention. In OV-DHM, a specimen is illuminated from two sides in a 4π-alike configuration, and two holograms are generated and recorded by a CCD camera along two orthogonal polarization orientations. The image plane of the sample is determined by finding the minimal variation between the two object waves, and consequently refocusing is performed by propagating the waves to the image plane. Furthermore, the field of view (FOV) of OV-DHM can be extended by combining the two object waves which have an angle in-between. The proposed technique also has the potential to reduce speckle noise and out-of-focus background.

Digital holographic microscopy (DHM) is a non-invasive, high-resolution, whole-field technique for measuring microscopic specimens, especially translucent sample. [1][2][3][4][5] In microcopy, samples are often observed in in-focus scene through manual or mechanical focusing. However, this focusing manner becomes nearly impossible when measuring a moving sample or dynamic processes. In DHM, the difficulty in mechanical focusing is circumvented by a refocusing process: propagating an object wave from hologram plane to image plane. Notably, the DHM enables to refocus laterally-separated regions of a hologram to different focal planes and consequently, it can provide 3D information of the sample. 6 A key issue in reconstructing a refocused image from the out-of-focus hologram is the image plane determination, i.e., to find the distance between the hologram plane and the image plane. Hitherto, there have been many reports on image plane detection, which are based on amplitude analysis, [7][8][9] intensity gradient, 10 self-entropy, 11 local intensity variance, 12 spectral norms, 13 wavelet theory, 14 and so on. [15][16][17][18][19][20] Recently, we also reported non-conventional illumination based image plane determination approaches, which are based on two-wavelength illumination, 21 off-axis illuminations 22 or structured illumination. 23 The image plane was determined by finding the minimal difference between the reconstructed object waves which are aligned with two wavelength illuminations, two off-axis illuminations, or two diffraction orders of structured illumination.
In aside of the image plane determination, the non-conventional illumination schemes [21][22][23][24][25][26] enable an increased data acquisition along the designed illuminations. Normally, DHM uses a plane wave for illumination and consequently, its resolution and axial sectioning ability of DHM is worse than that of the conventional microscope, which employs Koehler illumination with a boarder spectrum. Off-axis illumination, 22, 24 structured illumination 23 and speckle illumination 25,26 can improve the lateral resolution of DHM, and in the meantime contribute to improving the axial sectioning ability of DHM. Recently, opposed-view dark-field digital holographic microscopy was proposed, which collects the scattered light concurrently from both opposite views, and therefore improves the contrast of internal structures and as well the signal-to-noise ratio. 27,28 In this paper, we present an opposite-view digital holographic microscopy (OV-DHM) for autofocusing and field of view (FOV) extension. The OV-DHM enables to determine the image plane automatically and refocus a sample digitally, providing the possibility to image moving samples or dynamic processes. Compared with conventional autofocusing methods, the presented technique can be used for more general samples. Furthermore, OV-DHM can extend the field of view (FOV) of imaging by combining the two reconstructed object waves, which have an angle in-between. Furthermore, OV-DHM can collect more frequency spectrum (from two sides), and thus it has the potential to suppress out-of-focus background.

Results
Configuration of OV-DHM. The schematic diagram of our home-built opposite-view digital holographic microscopy (OV-DHM) is shown in Fig. 1. The experiment setup is based on a common-path Sagnac configuration, which is comprised of a polarization-maintaining beamsplitter PBS and two mirrors M 1 and M 2 . A laser beam from the fiber end 1 is split by the PBS into two copies, of which the polarizations are along the horizontal and vertical directions, respectively. The copy which has horizontal polarization goes through the Sagnac configuration clockwise, while the other one goes through the Sagnac configuration anti-clockwise. Two telescope systems MO 1 -L 3 and L 4 -MO 2 are placed between M 1 and M 2 , and are used to image a sample with a magnification of 20X. A sample is located between the objectives MO 1 and MO 2 . After the illumination beams transmit the sample in opposite directions, the output object waves (namely O 1 and O 2 ) are magnified by the two telescope systems, and superimposed with a common reference wave R via a non-polarizing beamsplitter BS. The reference wave is linearly polarized with its polarization azimuth 45° with respect to the polarizations of O 1 and O 2 . Two hologram I 1 = |O 1 + R| 2 and I 2 = |O 2 + R| 2 were obtained separately by rotating the polarizer P to horizontal and vertical directions, respectively. We note that the reference wave R was adjusted to have an angle of 10 ± 0.1 mrad with respect to the two in-line object waves. It is worthy to mention that the OV-DHM configuration can be further upgraded by employing two CCD cameras to record the two opposite-view holograms at the same time (see Supplementary Fig. 1).
Autofocusing principle of the OV-DHM. As seen in Fig. 2(a), a CCD is placed to record the in-focus images of the middle plane P mid of the two objectives clockwise and anti-clockwise in OV-DHM. If a sample is located to have a distance Δz from P mid , the hologram I 1 will have a defocusing distance M 2 Δz along clockwise direction, while the hologram I 2 will have an opposite defocusing distance −M 2 Δz along the anti-clockwise direction. Here M is the magnification of the OV-DHM system. For reconstruction, the two object waves, O r1 and O r2 , which have an arbitrary distance −Δd from I 1 (and Δd from I 2 ) can be reconstructed by using angular spectrum method: 29 Here k = 2π/λ denotes the wave vector; FT{·} and FT −1 {·} denote Fourier-transformation and inverse Fourier-transformation operators, ξ and η are the spatial coordinates in the frequency domain. R D is a digitalized reference waves, which has a linear phase term (corresponding to the angle between object wave and reference wave) used to shift the spectrum of the real image to the center of the frequency domain. W filter (ξ, η) is the window function used to select the real images of I 1 R D and I 2 R D in the frequency domain, blocking their dc terms and twin images (see Supplementary Fig. S2). The cut-off frequency of W filter was chosen to maintain the highest frequency of the object wave (defined with ω 0 ). The angle between the object and reference waves should be designed to yield an off-axis hologram which has a carrier frequency ≥3ω 0 in order to a separation between the real image, twin image and dc term. 30 A CCD camera which has a sampling frequency ≥8ω 0 is required to record the hologram.
OV-DHM has an intrinsic property that, the holograms I 1 and I 2 have opposite defocusing distances. Thus, the difference between |O r1 (Δd)| 2 and |O r2 (−Δd)| 2 reaches the minimum when the reconstruction distance Δd is correct (match M 2 Δz). Otherwise, the larger |Δd − M 2 Δz| is, the larger is the variation of |O r1 (Δd)| 2 − |O r2 (− Δd)| 2 . Thus, this property can be used as focus criterion to determine image plane of OV-DHM. For this purpose, the focus criterion can be defined as: In Equation (2), RMS denotes the operation of root-mean-square operation. In implementation, an in-focus plane can be determined by finding the minimum of the criterion function in Equation (2).
Autofocusing of lensless OV-DHM. Lensless OV-DHM was firstly performed on the configuration sketched in Fig. 1 omitting the imaging units MO 1 -L 3 and MO 2 -L 4 . A structured glass plate which has both amplitude and phase modulations was used as a sample. The sample was located in the plane having 7.5 ± 0.1 cm distance from the plane P mid , which has an equal distance of 40 cm to the CCD plane clockwise and anti-clockwise. Figure 3(a) shows the two holograms I 1 and I 2 obtained by rotating the polarizer P to horizontal and vertical directions. The two object waves (O r1 and O r2 ) were reconstructed and propagated for a distance of −40 cm ± Δd with Δd varying between −20 cm and 20 cm. For each reconstruction distance, the focus criterion defined with Equation (2) was shown in Fig. 3(b). The proposed criterion shows a minimum at Δd = −7.5 ± 0.2 cm, which is in good agreement with the prior-set value. Here the error 0.2 cm is the calculation step for digital re-focusing. In contrast, the conventional criterion, e.g., intensity analysis based (IAB) criterion, failed to find the correct image plane. This is because the phase distribution of the object wave introduces additional intensity variation in out-of-focus planes, which balances surpass the intensity variation of the object wave in the image plane. In Fig. 3(c), we show the reconstructed |O r1 | and |O r1 | − |O r2 | at different Δd. It is distinct that the variation between |O r1 | and |O r2 | becomes minimal at the image plane. In contrast, the intensity modulation of a single object wave, e.g., O 1 , does not appear a monistic change with Δd. This, in turn, explains why the conventional IAB criterion can not find the correct image plane. For a sample with both amplitude and phase modulation, the advantage of the proposed method over the conventional autofocusing methods, i.e., integrated amplitude modulus (IAM), Laplace Filtering based differential method (LAP), Variance of intensity distribution (VAR) based methods, has also been verified (see Supplementary Text1 and Supplementary Fig. 3). Eventually, by using the obtained Δd = −7.5 ± 0.2 cm, the focused amplitude and phase image of the sample were reconstructed and shown in Fig. 3(d) and (e). A clear area on |O r1 |, |O r2 | and (|O r1 | + |O r2 |)/2, indicated with the green rectangle in Fig. 3(d), were selected for comparison on speckle noise. The standard deviations on their intensity fluctuation 0.30, 0.36 and 0.21 were obtained, implies that speckle noise can be efficiently reduced by averaging the two reconstructed object waves.

Autofocusing of lens-based OV-DHM.
Lens-based OV-DHM was firstly carried out on a rectangular-grid target (R155731-09100, Edmund Optics, Barrington, NJ, USA) with sharp absorbing structures, as is shown in Fig. 4(a). After the two opposite-viewed holograms were recorded, the focus criterion of the proposed method was calculated and compared with the focus criterion (IAM) 7 in Fig. 4(b). The IAM method determined the image plane at Δz = 95 μm with which the amplitude modulus is minimized for an amplitude object. The proposed criterion tells the focus plane at Δz = 89 μm. The deviation between the two methods 6 μm is within the range of the depth of field (DOF) of the imaging system (2λ/NA 2 =7μm). To follow up, the lens-based OV-DHM was carried out for microscopic biological sample. Human HeLa cells (LGC Standards GmbH, Wesel, Germany) sat on a coverslip surface and covered with another coverslip was used as a microscopic sample. Figure 5(a) shows the two opposite-view holograms I 1 and I 2 , and the zoomed areas of I 1 and I 2 highlight the dense fringes due to the angle between the object wave and the reference wave. After calculated with Equation (2), the focus criterion in Fig. 5(b) shows a minimum at Δz = −180 ± 5 μm, which was further verified by Fig. 5(c) where the amplitude difference O r1 and O r2 reaches its minimum at Δz = −180 μm. By using the obtained Δz, the amplitude and phase images of the HeLa cells were reconstructed and shown in Fig. 5(d) and (e), respectively. The comparison between Fig. 5(d) and (e) reveals that phase image of these HeLa cells has higher contrast compared with their amplitude/ intensity image, which is available with a conventional microscope. We note that the refractive index of the cell can be further determined by using the method proposed in refs 31 and 32.

Field Of View (FOV) extension by OV-DHM.
The OV-DHM has also the ability to extend the field of view (FOV) of phase imaging. To achieve this purpose, the two object waves were aligned to have an angle α in-between (after magnified by the objectives). By doing this, the CCD camera recorded different parts of the sample image/diffraction pattern along two object waves (see Fig. 6(a) and (b)). In this case, image plane determination can still be performed by replacing |O r1 (x,y,Δd)| 2 − |O r2 (x, y, −Δd)| 2 in Equation (2) with |O r1 (x,   Fig. 6(d) and (e), and the counterparts after phase unwrapping operation 33 were in Fig. 6(f) and (g). It can be seen that the two phase images exhibit different parts of the tested sample. After the two images were combined together, the whole field of view is extended from 1.2 × 10 5 μm 2 to 1.6 × 10 5 μm 2 in Fig. 6(h); that said, 30% FOV extension was achieved by the OV-DHM. The combined image shows Cos7 cells proliferation status: one to two after 24 hours relocation on a glass plate. The phase image also confirms that the nucleus is much denser in refractive index than that of cytosol in a cell.

Discussions
In this paper, opposite-view digital holographic microscopy (OV-DHM) was presented for autofocusing and field of view (FOV) extension. The out-of-focusing distance (a distance between the image plane and CCD plane) is determined by searching the minimal difference between two object waves reconstructed from two opposite-view holograms. Based on this distance, refocusing of the sample has been performed by propagating the reconstructed object waves to their common image plane. Compared with the conventional image plane determination methods, this method does not rely on the type of a sample (i.e., can be used for the sample with both amplitude and phase modulations). This advantage is due to the intrinsic illumination scheme of OV-DHM and is valid for both lensless OV-DHM and lens-based OV-DHM. The autofocusing can be performed on different regions of the sample: by refocusing and resorting different laterally-separated regions of a sample into their correct axial planes, a 3D image of the sample can be obtained (Supplementary text 4 and Supplementary Figs 4-6). Furthermore, averaging the two object waves can also contribute to suppressing the out-of-focusing background, since only the in-focus components (e.g., cells) appear the same in the two images, wherever the out-of-focus components have different defocusing distance in the two images (see Supplementary Figs 5 and 6).
In case there is an angle between the two opposite-view object waves, extension of the field of view (FOV) can be performed by combining the two reconstructed images. Since the lateral shift of the two reconstructed object waves has a linear relation with the defocusing distance, 22 out-of-focus information and speckle noise can be suppressed by averaging the two object waves ( Supplementary Figs 5 and 6). Furthermore, the angle between the two illuminations has also the potential to improve the lateral resolution of the OV-DHM, considering a larger spectrum can be synthesized by combining the spectra of the two off-axis propagated object waves [22][23][24] . It is worthy to notice that there is a conflict between FOV extension and the improvement on sectioning and resolution improvement, since they rely on different experimental setting. To extend the FOV, the sample should be placed has a certain distance Δz from the middle plane of the two objectives (which corresponds to the in-focus plane). This defocus distance Δz will provide a lateral shift of 2tan(α/2)M 2 Δz between two opposite-view images in CCD/CMOS plane, which is used for FOV extension. Whereas, to improve the resolution and sectioning ability, a sample (or a section of it) should be located in the middle plane of the two objectives, where the two object waves will have the same image on CCD/CMOS plane. The potential of OV-DHM on improving section ability could be further explored by using two coherent opposite-view illuminations to generate a structured illumination with in the axial direction, similar to I 5 M microscope. 34 Furthermore, the two object waves with orthogonal polarizations have also the potential to provide polarization-resolved information for the anisotropic sample 35 .
The OV-DHM has also the following disadvantages: firstly, the configuration of the OV-DHM is far more complex than that of the convention DHM, and the requirement of two opposite-view object waves makes it not compatible with a common microscope frame. Secondly, the autofocusing of OV-DHM requires symmetry on two opposite-view object waves. Thus, the two waves should be aligned carefully, and collimated well to avoid defocusing and other aberrations.

Methods
OV-DHM Setup and alignment. The OV-DHM setup was constructed according to the sketch in Fig. 1.
For the lensless OV-DHM mode, the imaging units MO 1 -L 3 and MO 2 -L 4 was omitted. The middle plane of the mirror M 1 and M 2 has the same distance 40 cm to the CCD plane along clockwise and anti-clockwise directions. For the lens-based OV-DHM mode, two telescope systems MO 1 -L 3 , and MO 2 -L 4 , were used to image samples with a magnification of 20x. MO 1 and MO 2 are two identical, plan-field microscopic objectives (Plan 25X/0.4, Nanjing Yingxing Optical Instrument Company, Nan Jing, China). L 3 and L 4 are the achromatic lens (AC254-200-A-ML, Thorlabs GmbH, Munich, Germany) with focal length 200 mm and diameter 1 inch. An additional telescope system with a magnification M = 1.5 was placed before the CMOS camera to further magnify the object waves.
The following procedure can be performed to make the two opposite-view object waves along the same line: A circular aperture (diameter: 2 mm) was located in the center of the input beam before the polarization beamsplitter PBS (PBS251, Thorlabs GmbH, Munich, Germany). Then, adjust the mirror M 2 to make sure the two circular patterns (of the clockwise-propagated and anti-clockwise-propagated beams) on the mirror M 1 are overlapped. Similarly, adjust the mirror M 1 to make sure the two circular patterns on the mirror M 2 are overlapped. In addition, the angle between the in-line object waves (O 1 and O 2 ) and the reference wave was set to 10 ± 0.3 mrad, which was evaluated by Fourier analysis on the off-axis hologram 36 . The generated off-axis hologram was recorded by a Complementary metal-oxide-semiconductor (CMOS) camera (1920 ×1200pixels, 5.86 μm/pixel, 54 fps, DMK 23UX174, Imaging Source, Bremen, Germany). In the reconstruction of the hologram, an asymmetric window function W filter (ξ,η) was used to collect more high-frequency spectrum in the directions other than the carrier-frequency direction (see Supplementary Fig. 2).
Compared to an in-focus recording hologram, the out-of-focus recording hologram requires increased spatial bandwidth product (SBP), and the SBP consumption increases with the out-of-focus recording distance. 37,38 We note that this requirement can be released by using a larger magnification in the OV-DHM system or utilizing a Slightly-off-axis 4, 30 and on-axis 39, 40 recording scheme. In our experiment, a CCD with pixel size 1/4.4 AU (airy unit, the diameter of the first-order diffraction-limited Airy disc in CCD plane) was used, with which a slight resolution reduction still happen in the case of a large out-of-focus distance. Thus, in order to avoid the SBP deficit and high-frequency cutting by the objective, we limited the out-of-focusing distances in a range of [−wd/10, wd/10], with the wd being the working distance of the used objective.
Numerical compensation for axial misalignment of OV-DHM. In the lens-based OV-DHM, The two telescope systems were aligned such that the clockwise and anti-clockwise images of the middle plane P mid (of the two objectives MO 1 and MO 2 ) have a defocusing distance of 6 cm and 0 cm on the CCD. For numerical compensation, the reconstruction distance Δd − 0.06 m and −Δd were used (instead of Δd and −Δd) in Equation (2) for image plane determination of O r1 and O r2 .