Super-resolution three-dimensional fluorescence and optical diffraction tomography of live cells using structured illumination generated by a digital micromirror device

We present a multimodal approach for measuring the three-dimensional (3D) refractive index (RI) and fluorescence distributions of live cells by combining optical diffraction tomography (ODT) and 3D structured illumination microscopy (SIM). A digital micromirror device is utilized to generate structured illumination patterns for both ODT and SIM, which enables fast and stable measurements. To verify its feasibility and applicability, the proposed method is used to measure the 3D RI distribution and 3D fluorescence image of various samples, including a cluster of fluorescent beads, and the time-lapse 3D RI dynamics of fluorescent beads inside a HeLa cell, from which the trajectory of the beads in the HeLa cell is analyzed using spatiotemporal correlations.

Because both fluorescence imaging and QPI provide complementary information about cell pathophysiology, efforts have been made to combine and utilize both modalities for correlative bioimaging [36][37][38][39][40][41][42][43][44][45] . Several techniques for performing both 3D wide-field fluorescence imaging and 3D RI tomography of cells have been reported 33,39,40 . More importantly, for super-resolution 3D fluorescence imaging, Chowdhury et al. performed 3D SIM and ODT simultaneously utilizing a liquid crystal spatial light modulator (SLM) for structured illuminations 41 , and Descloux et al. combined 3D phase imaging with super-resolution optical fluctuation imaging by spatial filtering of stacked images measured at various axial positions 42 . However, use of an SLM prevents fast imaging, one of the advantages of ODT. Furthermore, photobleaching and phototoxicity inevitably follow the use of an SLM.
In this paper, we propose and demonstrate with experiments a multimodal approach for measuring both the 3D RI and fluorescence distributions of live cells that combines ODT and 3D SIM using a digital micromirror device (DMD). Time-multiplexed structured illumination patterns displayed by the DMD are used to perform 3D super-resolution fluorescence imaging and obtain 3D RI tomograms of samples simultaneously and effectively. The fast dynamic response of the DMD enables imaging of live cells with high spatiotemporal resolution, molecular specificity, and reduced damage from phototoxicity. Because the same optical imaging system with the DMD is used for both modes, the correlation between 3D fluorescence and 3D RI data can be easily analyzed. The present method will open a new avenue into the study of cell biology, medical research, and diagnosis.

Results and Discussion
Optical setup. Figure 1 shows the schematic of the optical setup, equipped with a DMD (DLP6500EVM, Texas Instruments, USA), used in our experiments. This setup was able to operate in both the ODT mode and the 3D SIM mode [ Fig. 1(a-c)] by modifying a commercial ODT setup (HT-1H, Tomocube Inc., Republic of Korea) to incorporate fluorescence excitation.
To minimize photobleaching and phototoxicity that occur in fluorescence imaging of live cells, the two operating modes are separate utilizing dispersion property of a DMD and two lasers with different wavelengths (λ ODT = 473 nm, MSL-FN-473, CNI Laser, P.R. China) and λ SIM = 532 nm (LSS-0532, Laserglow Technologies, Canada)] for coherent imaging and fluorescence excitation, respectively. To minimize phototoxicity 46 , the laser power for ODT was attenuated to 0.11 mW and the exposure time was shorter than 0.1 s. To simplify the setup, the two laser beams were coupled into a single-mode fiber using an RGB combiner (RGB30HF, Thorlabs Inc., USA). By selectively blocking one of the laser beams, the present setup alternated between the ODT mode and the 3D SIM mode. In both modes, the illumination part that included the DMD, lens L1 (f 1 = 250 mm), and objective lens 1 [UPLSAPO 60XW, numerical aperture (NA) 1.2, Olympus Inc., Japan] was the same, but the modes used different measurement schemes [ Fig. 1(b,c)].
ODT mode. In the ODT mode, a sample is illuminated with various illumination patterns and the optical fields diffracted from the sample are measured using Mach-Zehnder interferometry 22 [ Fig. 1(b)]. A 1 × 2 single-mode fiber optic coupler (SMFC) (TW560R3F1, Thorlabs) splits the laser beam (λ ODT = 473 nm) into a reference beam and a sample beam. To control the angle of the beam impinging on a sample, the DMD displays time-multiplexed patterns that are projected onto the sample 47 . Using objective lens 2 (UPLSAPO 60XW, NA 1.2) and lens L2, (f 2 = 180 mm), the beam diffracted from the sample is collected and projected onto the image plane, where it interferes with the slightly tilted reference beam to generate an off-axis hologram. Multiple 2D holograms with various illumination patterns are recorded using an image sensor (Cam1; FL3-U3-13Y3M-C, FLIR Systems, USA).
The DMD displays a time-multiplexed pattern that systematically controls the illumination angles. The superposition of the two plane waves from the DMD results in the formation of structured illumination that impinges on the sample [ Fig. 2(a,b)]. The two plane waves diffracted by the displayed pattern in the DMD are mutually conjugated optical fields. Thus, the phase shift Δφ between the plane waves and the incident angles of the plane waves on the sample can be precisely controlled by the pattern displayed by the DMD. In addition, the time-multiplexing method significantly reduces unwanted diffraction noise that usually occurs when binary control is used in a DMD. By exploiting temporal averaging unwanted diffraction from a DMD, the time-multiplexing method enables to reduce unwanted diffraction from the DMD 47 . For efficiency, the limited number of time-multiplexed patterns stored on the DMD control board consists of a three-binary-pattern sequence.
In addition to the normal angle, we used 28 time-multiplexed patterns (14 pairs) to generate 28 plane waves at different azimuthal angles so that the illumination beams scanned a circular pattern within the NA of the condenser lens [ Fig. 2(b)]. To decompose the time-multiplex pattern into two plane waves, a pair of structured patterns with different relative phase shifts, 0 and π, respectively, was needed. Therefore, 28 time-multiplexed patterns were constructed and displayed to produce the 28 plane waves used for circular scanning 48 .
The optical fields diffracted from a sample are retrieved from the measured holograms corresponding to the time-multiplexed structured illuminations [ Fig. 2(c)] using a field retrieval algorithm based on Fourier transform [insets in Fig. 2 49 . Then, using the known phase shifts in the structured illuminations, the retrieved optical fields under the structured illuminations are decomposed into the scattered fields from the sample with respect to plane-wave illumination components as follows: where E 1,2 , Δφ 1,2 , and S 1,2 are the retrieved fields, the known phase shifts, and the decomposed scattered fields for each plane-wave component 3D SIM mode. In the 3D SIM mode, the same DMD generates structured illumination patterns that excite fluorescent molecules in the sample plane [ Fig. 1(c)]. An excitation laser beam (λ SIM = 532 nm) is reflected from To obtain a 3D fluorescence image with subdiffraction resolution, we used the principle of SIM (Fig. 3) 7,13 . SIM images were reconstructed from fluorescence images obtained using structured excitation patterns. To project a structured excitation pattern onto a sample, the DMD is used to rapidly generate a time-multiplexed pattern comprising three plane waves [ Fig. 3(a,b)]. The two oblique plane waves are mutually conjugated fields that are diffracted by the displayed pattern, while the third plane wave has a normal incident angle. The various patterns displayed on the DMD systematically control the phase shift and the incident angles of the two oblique plane waves relative to the residual plane wave. To reconstruct the SIM images, 15 time-multiplexed patterns are displayed as a combination of three azimuthal angles and five phase shifts [ Fig. 3(b) where n is the phase-shift index, F n is a recorded fluorescence image, C m is an indexed component image, and Δφ n is the relative phase shift of the three plane waves that result in structured illumination. Using the known phase shifts controlled by the DMD, the component images can be decomposed as follows: Next, the Fourier spectra of the decomposed component images are shifted in the Fourier space to reconstruct a SIM image using the principle of SIM [ Fig. 3(d,e)]. Furthermore, a 3D SIM image can be obtained by stacking 100 reconstructed SIM images using a 150-nm interval along the axial direction [ Fig. 3(f)] 13,53 . To compare the spatial resolution of the 3D SIM images with the 3D wide-field fluorescence (WF) images, 3D WF images are reconstructed by stacking the 2D WF images along the axial direction.

Measurements of a monolayer of fluorescent beads.
To demonstrate the validity of the proposed method, we measured the 3D RI distribution and 3D fluorescence image of a cluster of red fluorescent 500-nm-diameter PS beads (Fig. 4). In Fig. 4(a,c,e) for the whole field of view (FOV) and in Fig. 4(b,d,f) for the magnified FOV, a monolayer of fluorescent beads is clearly seen in both the 3D RI distribution and the 3D fluorescence image. In addition, the reconstructed 3D RI distribution matches the reconstructed 3D SIM and WF images well. This clearly shows the feasibility of our method.
The lateral and axial resolutions of ODT in the proposed method were 172 nm and 665 nm, respectively, as calculated from the spatial bandwidth of the reconstructed 3D RI along the lateral and axial directions 50 . Because the axial resolution of ODT in the proposed method was larger than the diameter of the PS beads, the reconstructed RI distribution of the beads showed elongated images in the axial direction as well as RI values smaller than the known RI value of PS (n = 1.608 at λ = 473 nm) 15,17 .
The cross-sectional slices of the reconstructed 3D SIM images [ Fig. 4(c,d)] show the ability to perform optical sectioning and enhanced lateral and axial resolutions, 127 nm and 408 nm, respectively, of 3D SIM compared to 3D WF, 254 nm and 816 nm, respectively [ Fig. 4(e,f)]. The lateral and axial resolutions are calculated from the spatial bandwidth of the reconstructed 3D image along the lateral and axial directions 50 . Moreover, the extended lateral and axial bandwidths are clearly seen in the 3D Fourier spectrum of the 3D SIM image in Fig. 4(h) compared to those seen in the spectrum of the 3D WF image [ Fig. 4(g)]. In particular, the cross-sectional axial slice of the 3D Fourier spectrum of the 3D SIM does not exhibit the missing cone, which causes severe deterioration of the 3D WF image along the axial direction 7 .
4D Time-lapse measurements of fluorescent beads inside a HeLa cell. The proposed multimodal method combines the advantages of ODT and 3D SIM by providing label-free 3D RI distribution with superior spatiotemporal resolution along with high molecular specificity, which can be effectively demonstrated by measuring movements of a small object inside a complex 3D RI distribution 54 . To demonstrate the synergistic advantages and the applicability of the proposed method, we measured the dynamics of the 3D RI distribution of 500-nm-diameter fluorescent PS beads ingested by a HeLa cell (Fig. 5).
First we measured the 3D RI distribution and the 3D fluorescence image of PS beads inside a HeLa cell using the ODT and 3D SIM modes, respectively [ Fig. 5(a,b)]. The 3D isosurface of the cell [Fig. 5(c)] rendered from the reconstructed 3D RI distribution [ Fig. 5(a)] shows that the fluorescent beads inside the cell are difficult to distinguish from the surrounding cytoplasm because the axial resolution of ODT is larger than the diameter of After identifying the fluorescent beads in the reconstructed 3D RI distribution, we obtained time-lapse 3D RI distribution measurements of the HeLa cell every 30 s over 1 h using ODT [ Fig. 5(e)]. Because the subcellular dynamics of the HeLa cell was slow compared to the tomographic acquisition speed, the reconstructed 3D RI distributions of the beads between consecutive measurements were highly correlated, which enabled the 3D dynamics of the fluorescent beads inside the HeLa cell to be tracked without additional 3D SIM measurements [ Fig. 5(e)].

Conclusion
In this paper, we proposed multimodal approach to combine ODT and 3D SIM using a DMD and experimentally demonstrated its feasibility and applicability. Each mode can be operated separately by using dispersion property of a DMD and different illumination wavelengths. As a result, the combination of the respective advantages of ODT and 3D SIM yields the synergistic effects of our method. To demonstrate the feasibility of the proposed method, we reconstructed 3D RI and fluorescence distributions of a planar cluster of fluorescent PS beads using the ODT and 3D SIM modes, respectively. Furthermore, to demonstrate the applicability of our method, we reconstructed 3D fluorescence and time-lapse 3D RI distributions of beads inside a HeLa cell. By exploiting the spatiotemporal correlations of 3D fluorescence and time-lapse 3D RI distributions, the beads were tracked as the cell changed morphologically.
Our proposed method provides 3D structural and biochemical information on fluorescent-labeled live cells with high molecular specificity. The advantages of our method, such as its high spatiotemporal resolution and minimization of the photobleaching problem, provide it the potential for use in a wide range of applications, including the study of subcellular dynamics over a long time.

Materials and Methods
Microsphere preparation. The red fluorescent PS beads (L3280, Sigma-Aldrich, St. Louis, MO, USA) were diluted in distilled water, placed on a coverslip, and dried overnight so that the beads adhered to the surface of the coverslip. Before measurements were taken, phosphate-buffered saline (PBS) solution (pH 7.4, 50 mM, Welgene, Gyeongsangbuk-do, South Korea) was gently dropped on the coverslip, which was then covered by another coverslip.
HeLa cell preparation. HeLa cells, a human cervical cancer cell line, were cultured in Dulbecco's modified Eagle's medium (DMEM) (Welgene) with 10% fetal bovine serum (FBS) (Welgene) and 1% penicillin  /v), Welgene] inside a humidified incubator (37 °C, 5% CO 2 , 95% air). After 12 h of incubation, trypsin EDTA (Welgene) was applied to detach the cells from the culture flask. The detached cells were then collected via centrifugation (1000 rpm for 3 min) and incubated on a coverslip using the same previous culture conditions for 3 h. The diluted red fluorescent PS beads (100× diluted from stock solution) were added to the culture medium and became inserted as fluorescent probes inside the HeLa cells. Cells on the coverslip were maintained with the bead solution for 12 h and then washed three times with PBS solution.