High speed, complex wavefront shaping using the digital micro-mirror device

Digital micro-mirror devices (DMDs) have been deployed in many optical applications. As compared to spatial light modulators (SLMs), they are characterized by their much faster refresh rates (full-frame refresh rates up to 32 kHz for binary patterns) compared to 120 Hz for most liquid crystal SLMs. DMDs however can only display binary, unipolar patterns and utilize temporal modulation to represent with excellent accuracy multiple gray-levels in display applications. We used the built-in time domain dynamic range representation of the DMD to project 8-bit complex-fields. With this method, we demonstrated 8-bit complex field modulation with a frame time of 38.4 ms (around 0.15 s for the entire complex-field). We performed phase conjugation by compensating the distortions incurred due to propagation through free-space and a scattering medium. For faster modulation speed, an electro-optic modulator was used in synchronization with the DMD in an amplitude modulation mode to create grayscale patterns with frame rate ~ 833 Hz with display time of only 1.2 ms instead of 38.4 ms for time multiplexing gaining a speed up by a factor of 32.

In what follows, we first introduce the basic concept we use in the paper. After that, in the next sub-section in the methods section, we show how we used this technique for phase conjugation applications. We refer to this method as Opto-Electronic Phase Conjugation (OEPC) to distinguish it from conventional Digital Phase Conjugation. The results section was divided into 2 main sub-sections. The first subsection includes two experiments that demonstrate phase conjugation conducted using our time-multiplexed technique. The second sub-section shows a new modality for using the DMD which is amplitude modulation scheme.

Methods
Complex wavefront shaping with the DMD. We consider a complex field f (x, y) = A(x, y)e jϕ (x,y) where A(x, y) is the 2D amplitude and ϕ(x, y) is the 2D phase pattern. We represent a complex pattern on the DMD by presenting positive and negative portions of real and imaginary parts of the complex field, sequentially. In order to represent the grayscale of the real and imaginary parts of the complex field, we take advantage of the time modulation feature of the DMD. f (x, y) can be written as follows: where BR α and BI α are the α bit-planes for the real and imaginary parts, respectively. In our optical system, the field is interferometrically measured at the detector. This allows us to reconstitute the complete complex field by temporal integration on the detector which aggregates the contributions of each of the bit planes of both real and imaginary parts. Alternatively, the summation can be done post-detection on the computer. We used the experimental setup shown in Fig. 1. A continuous wave (CW) laser at 725 nm was used in all our experiments. The laser beam is spatially filtered and collimated with a pinhole and a pair of lenses. The signal beam is reflected from the DMD device (DLP-4500NIR 0.45 WXGA near-Infrared, resolution: 912 × 1140, pixel pitch: 10.8 µm, Ajile Light Industries). A 4f. system was used to image the DMD plane onto the detector plane to retrieve in-focus images. For holographic detection, the signal and reference beams are combined using a non-polarizing beam splitter and projected onto a complementary metal-oxide-semiconductor (CMOS) camera (Edmund optics, pixel size = 3.45 µm, resolution = 2056 × 1542 pixels). Although in the current experiment there is no object to be imaged, objective lenses were used to allow the DMD to be used for imaging applications.
The grayscale image was translated into 8 binary bit-planes (corresponding to the 8-bit image) and the 8 binary images were displayed onto the DMD sequentially to be captured onto the detector within a limited exposure time in a time-multiplexing manner. Since time-multiplexing is used, this means that the least significant bit-plane will be displayed for 1 time unit while the most significant bit-plane will be displayed for 128 time units (i.e. 1 time unit = 150 µs). This was repeated for both the real and the imaginary parts of the image. Figure 2 demonstrates successful phase modulation as a result of this technique. Each of the real and the imaginary parts were projected, one at a time, onto the DMD in a time multiplexing fashion. After capturing the two grayscale mages of the real and the imaginary parts, they were added digitally onto the computer which is equivalent to phase modulation by the DMD as seen in Fig. 2.
For phase maps whose phase exceeds π/2 radians, we split the real and imaginary parts into positive and negative values and displayed each one separately onto the DMD. This is expressed by Eq. (2): Opto-electronic phase conjugation (OEPC) using DMD. Assuming an initial complex field ( U i ) incident on a scattering medium, we can write the output as follows: where T io is the transmission matrix describing the propagation of U i through the scattering medium. After taking the complex conjugate of this function and propagate through the same medium, we get: Assuming a perfect time-symmetric system (i.e. T −1 oi = T * io ), Eq. 4 simplifies to: Equation (5) shows how OEPC removes the aberrations associated with the medium (i.e. free space propagation or mouse liver tissue in our experiments) and the input field is retrieved. In order to demonstrate OEPC using the DMD, a simplified interferometric setup was used as shown in Fig. 3. The detector used in this experiment is a CMOS camera (Photonfocus Inc., resolution: 1312 × 1280, pixel size: 8 µm). First, the setup was calibrated to ensure pixel-to-pixel alignment between the DMD and the camera. The incident angle is chosen to be at θ degrees with respect to the direction normal to the DMD surface. Given that the DMD has an "on" and "off " state beam at ± 12° with respect to the normal direction, θ was chosen to be around 12° relative to the normal direction as shown in Fig. 3b. Owing to the specific distribution of the pixels in the DMD, (Fig. 3a), Figure 2. Phase modulation using DMD as expressed in equation (1) by splitting the complex field into real and imaginary parts. www.nature.com/scientificreports/ interpolation was used in order to successfully align the pixels of the DMD and SLM. As seen in Fig. 3a, the center-to-center distance between two successive columns (i.e. C n−1 and C n ) is 10.8 µm while the distance between two successive rows is equal to 5.4 µm (i.e. 10.8/2 µm) as given by the DMD specification. This difference was accommodated by the use of interpolation to ensure successful mapping of the detector image onto the DMD screen. After calibrating the system, the recording step ( Fig. 3b) was performed to by displaying phase and/or amplitude maps as expressed by Eq. (2). After acquiring the image on the CMOS detector, the hologram was processed and the complex field was extracted on a personal computer and its complex conjugate was displayed back onto the DMD (Fig. 3c) and finally the original non-distorted image is reconstructed by adding the real and the imaginary parts digitally.

Results
OEPC for diffracted pattern after passing through scattering media. An experiment was performed with a scattering medium in between the DMD and the detector as shown in Fig. 4. The scattering medium was a 10 µm-thick fixed mouse liver-tissue sandwiched between a standard cover-slide of 1 mm thickness and a coverslip of 170 µm thickness. In the OEPC experiment, a simple object (a binary amplitude circular mask of ~ 430 µm diameter) was displayed on the DMD and the reflected light was captured by the detector. By extracting the complex field of the acquired hologram, taking its complex conjugate and feeding it back to the DMD with the scattering medium in place, we were able to retrieve the original pattern. Figure 5 shows the diffracted and the refocused spot after OEPC. Figure 5b shows how we get a full-width at half-maximum (FWHM) of 250 µm whereas the defocused spot (Fig. 5a) has a FWHM of 400 µm.  www.nature.com/scientificreports/ A similar experiment was performed with the USAF target being the input amplitude-only field passing through the same scattering medium (i.e. liver tissue). The image on the left hand-side of Fig. 6 shows the diffracted intensity pattern through the sample while the right hand-side image shows the refocused USAF target after compensating the effect of defocusing and scattering through the sample.
Grayscale modulation using amplitude modulator synchronized with the DMD. In order to boost further the speed of grayscale modulation using the DMD, amplitude modulation was used instead of time modulation. As shown in Fig. 7, an electro-optic modulator (Thorlabs, EO-AM-NR-C1) was used to modulate the light intensity for each bit-plane. In this case we used light intensity instead of light amplitude since holographic recording was not used on the camera. Therefore the calibration was done for intensity recording. The NI PCIe-6321 data acquisition tool was used for fast synchronization permitting speeds up to 90 kHz which is much faster than the DMD speed (6.6 kHz). Figure 7 shows the calibration process of the amplitude modulator (AM). A photodetector (Thorlabs, PDA36A-EC) was used to measure the voltage signal from the AM as a function of the applied voltage. As the characteristic half-wave voltage (V π ) at 725 nm for the AM was around 240 V, high voltage was needed to get the full modulation range which was generated using a high voltage amplifier (HVA200, V in = − 10:10, V out = − 200:200, Thorlabs). A quarter wave plate (QWP) was used to get the full range since V π is higher than the maximum possible voltage generated by the voltage amplifier.
A look-up-table (LUT) is extracted from Fig. 7c to modulate the 8 bit-planes of the 8-bit grayscale image. After calibrating the AM, the amplitude modulator was synchronized with the DMD and images were recorded using the camera as shown in Fig. 8. With such modulation scheme a maximum achievable frame rate of 833 Hz could be reached (i.e. 1 150 µs×8 ). Using this amplitude-modulation scheme, we showed efficient grayscale modulation for sinusoidal grating using the DMD (see supplementary video). Figure 9 shows an example of amplitude modulation for a sinusoidal grating with a grating period of 15 pixels displayed onto the DMD and imaged by the camera where it shows the effectiveness of our proposed scheme  www.nature.com/scientificreports/ as compared to binary grating where the higher diffraction orders are visible as shown in Fig. 9(b1). For the amplitude modulation case, higher orders are highly suppressed while the 0th, + 1, − 1 orders are dominant. The DMD consists of individually-controlled mirrors, and this creates many diffraction orders 29 . This phenomenon is shown in Fig. 10 where the pattern recorded on the DMD (a square) is repeated multiple times. The photograph in Fig. 10 was taken by placing the camera close to the DMD in order to capture the multiple patterns and demonstrate the effect.
During the performance of the experiment it was noticed that using a lens (L3 in Fig. 8) with higher focal length showed better reconstructed images. This is due to the fact that as the lens gets far from the DMD, additional diffracted orders from the DMD miss the lens and are not captured by the camera while only the zero-diffracted beam is able to pass through the lens and be captured by the camera. Notice that in practice, this approach is not light efficient, and a lens design such as the one used in display applications 30,31 that captures all the diffracted orders and superimposes them on the camera is preferable.
To further validate our results, phase modulation was performed as described earlier in which the complex field is divided into real and imaginary components and then each component was presented separately using the electro-optic amplitude modulation scheme. We recorded directly intensity on the camera where the signals corresponding to the different bit planes were integrated on the camera. The real and imaginary parts were readout and were added digitally on the pc to form the complex field. A blazed grating and a binary phase grating were synthesized on the DMD. As shown in Fig. 11, the blazed grating shows all the power focused into the + 1 order. On the other hand, the binary phase grating resulted in multiple orders as expected.

Discussion
In the optical OEPC experiments, the curvature of the DMD surface was corrected digitally by acquiring another hologram with all the DMD pixels in the "on" state (i.e. background hologram), then all the collected holograms for the real and the imaginary parts were corrected by dividing the extracted fields over the background field, this resulted in a clean image without phase-artifacts as shown in Figs. 2, 3, 4, 5, and 6. For the time-multiplexing scheme, the 8-bit frame rate is rather low (~ 26 frames per second). This speed can be further boosted using a faster commercially available DMD 32 (refresh rates ~ 32 kHz instead of 6.6 kHz) and also reducing the region of interest (ROI) on the DMD if the application allows a smaller number of pixels. For instance, using the same DMD we used in our study, by decreasing the ROI to 16 × 1140 instead of 912 × 1140, refresh rates can be boosted to 100 kHz (corresponding to a complex-field display rate ~ 110 frames per second). The speed of time modulation technique can be further enhanced by using 4-bit images instead of 8-bit images which will yield a speed enhancement by a factor of 16. Figure 12 shows the effect of changing the number of bits used in the DMD when carrying out OEPC on the focusing quality after passing through the scattering medium (the liver tissue). It is observed that both 4-bit (Fig. 12b) and 8-bit (Fig. 12c)    www.nature.com/scientificreports/ Table 1 shows a comparison between amplitude and time modulation schemes in terms of the frame rate for the used DMD whose refresh rate is 6.66 kHz.
Current DMD technologies have refresh rate as high as 32 kHz which would scale up the modulation speed as shown in Table 2.

Conclusion
High speed, complex wavefront shaping using the digital micro-mirror device was demonstrated. The DMD was used in a time-multiplexing mode which allows for 8-bit grayscale image representation on the DMD within 38.4 ms allowing for focusing in live samples or for other applications without compromising the accuracy. For higher modulation rates, amplitude modulation was demonstrated with maximum achievable frame rate for 8-bit images of 833 Hz yielding a speed up by a factor of 32 as compared to time multiplexing schemes.
The method we demonstrated relies on time integrating holographic recordings of the fields or intensity patterns corresponding to multiple binary patterns on the DMD. The method is particularly useful for optical systems that use digital holography. The method might also be useful in applications where the projected pattern is integrated in time by a light sensitive material such as a fluorescent molecule in imaging or a photo-initiator molecule in 3D printing.