Introduction

SLMs have been widely used in optical applications including wave-front shaping and light focusing1,2,3,4,5,6,7,8. A main limitation of liquid crystal SLMs is their low refresh rates, typically less than 120 Hz. This limitation can make it impossible to achieve high performance in applications that require high speed such as beam focusing in live samples. This has opened the door for another technology, DMDs, to be used for similar applications9,10,11,12,13,14. As opposed to SLMs, DMDs are characterized by their much faster refresh rates. DMD devices have a maximum refresh rate of 32 kHz15. However, as an “on–off” device, DMD allows only binary amplitude-modulation as compared to the grayscale and phase modulation that is possible with SLMs. Despite its binary amplitude-modulation, it was shown that DMDs can outperform SLMs in beam-shaping applications16. In17, Conkey et al. demonstrated the use of Lee computer generated holograms18 to achieve beam focusing in scattering media with signal to background ratio up to ~ 160 in ~ 34 ms. An improved optical performance was achieved by projecting Lee holograms on the DMD along with spatial filtering and misaligned optical lenses for phase modulation using “Super-pixel” technique where each super-pixel is composed of multiple pixels of the DMD19. In20, Drémeau et.al used the DMD as a binary input in a reference-less optical system for beam focusing. A Bayesian phase retrieval algorithm21 was used to precisely estimate the transmission matrix as well as for beam focusing through a white paint layer which acted as the highly scattering medium. In recent years22, Wang et.al implemented a binary-based digital optical phase conjugation (DOPC) system based on the use of DMD instead of SLMs. They used phase retrieval methods to design a binary computer generated hologram and demonstrated speed enhancement by approximately 2 orders of magnitude as compared to a liquid crystal SLM with a low playback latency of ~ 5 ms.

In display application, the DMD is used to modulate the time integrated light intensity through sequential pulse width modulation23,24,25,26. In the context of optical diffraction tomography27, Lee et.al demonstrated a time-multiplexing structured illumination control scheme using DMDs28. In their paper, the authors showed an improved performance by using time-multiplexed 8-bit amplitude modulation as compared to binary amplitude modulation (i.e. Lee holograms) in which the unwanted artifacts were diminished.

In this paper, we use a DMD to perform complex (amplitude and phase) modulation. We used the built-in dynamic range feature of the DMD device and to control the frame time of the DMD, to project an 8-bit image in 38.4 ms (256 × 150 µs). We show that by splitting the complex field into real and imaginary parts and using the time modulation scheme of the DMD we were able to synthesize a complex signal. We demonstrated this method for wavefront shaping by phase conjugating a measured field in free space and/or passing through a scattering medium. This paves the way for imaging and optical phase conjugation systems that are fast, digital accuracy without compromising light efficiency. Finally, to increase further the frame rate for grayscale modulation using the DMD, we modulated the intensity of the illuminating beam for each bit-plane while fixing the time for each bit-plane. Using this modulation scheme, the frame time for each grayscale image was reduced to 1.2 ms (8 × 150 µs).

In what follows, we first introduce the basic concept we use in the paper. After that, in the next sub-section in the methods section, we show how we used this technique for phase conjugation applications. We refer to this method as Opto-Electronic Phase Conjugation (OEPC) to distinguish it from conventional Digital Phase Conjugation. The results section was divided into 2 main sub-sections. The first subsection includes two experiments that demonstrate phase conjugation conducted using our time-multiplexed technique. The second sub-section shows a new modality for using the DMD which is amplitude modulation scheme.

Methods

Complex wavefront shaping with the DMD

We consider a complex field \(f(x,y) = A(x,y)e^{j\varphi (x,y)}\) where \(A(x,y)\) is the 2D amplitude and \(\varphi (x,y)\) is the 2D phase pattern. We represent a complex pattern on the DMD by presenting positive and negative portions of real and imaginary parts of the complex field, sequentially. In order to represent the grayscale of the real and imaginary parts of the complex field, we take advantage of the time modulation feature of the DMD. \(f(x,y)\) can be written as follows:

$$ \begin{aligned} f(x,y) & = {\text{Re}} \{ f(x,y)\} + i*{\text{Im}} \{ f(x,y)\} \\ & = \sum\limits_{\alpha = 1}^{8} {BR_{\alpha } 2^{\alpha - 1} + } i*\sum\limits_{\alpha = 1}^{8} {BI_{\alpha } 2^{\alpha - 1} } \\ \end{aligned} $$
(1)

where \(BR_{\alpha }\) and \(BI_{\alpha }\) are the \(\alpha\) bit-planes for the real and imaginary parts, respectively. In our optical system, the field is interferometrically measured at the detector. This allows us to reconstitute the complete complex field by temporal integration on the detector which aggregates the contributions of each of the bit planes of both real and imaginary parts. Alternatively, the summation can be done post-detection on the computer.

We used the experimental setup shown in Fig. 1.

Figure 1
figure 1

Optical setup. (M mirror, L lens, OBJ objective lens, BS beam splitter). 4f systems are used to image DMD pattern onto the detector with a spatial magnification defined by the 4f systems.

Full size image

A continuous wave (CW) laser at 725 nm was used in all our experiments. The laser beam is spatially filtered and collimated with a pinhole and a pair of lenses. The signal beam is reflected from the DMD device (DLP4500NIR 0.45 WXGA near-Infrared, resolution: 912 × 1140, pixel pitch: 10.8 µm, Ajile Light Industries). A 4f. system was used to image the DMD plane onto the detector plane to retrieve in-focus images. For holographic detection, the signal and reference beams are combined using a non-polarizing beam splitter and projected onto a complementary metal–oxide–semiconductor (CMOS) camera (Edmund optics, pixel size = 3.45 µm, resolution = 2056 × 1542 pixels). Although in the current experiment there is no object to be imaged, objective lenses were used to allow the DMD to be used for imaging applications.

The grayscale image was translated into 8 binary bit-planes (corresponding to the 8-bit image) and the 8 binary images were displayed onto the DMD sequentially to be captured onto the detector within a limited exposure time in a time-multiplexing manner. Since time-multiplexing is used, this means that the least significant bit-plane will be displayed for 1 time unit while the most significant bit-plane will be displayed for 128 time units (i.e. 1 time unit = 150 µs). This was repeated for both the real and the imaginary parts of the image. Figure 2 demonstrates successful phase modulation as a result of this technique. Each of the real and the imaginary parts were projected, one at a time, onto the DMD in a time multiplexing fashion. After capturing the two grayscale mages of the real and the imaginary parts, they were added digitally onto the computer which is equivalent to phase modulation by the DMD as seen in Fig. 2.

Figure 2
figure 2

Phase modulation using DMD as expressed in equation (1) by splitting the complex field into real and imaginary parts.

Full size image

For phase maps whose phase exceeds π/2 radians, we split the real and imaginary parts into positive and negative values and displayed each one separately onto the DMD. This is expressed by Eq. (2):

$$ \begin{aligned} f(x,y) & = {\text{Re}} \{ f(x,y)\} + i*{\text{Im}} \{ f(x,y)\} \\ & = \left[ {\sum\limits_{\alpha = 1}^{8} {BR_{\alpha }^{p} 2^{\alpha - 1} } - \sum\limits_{\alpha = 1}^{8} {BR_{\alpha }^{n} 2^{\alpha - 1} } } \right] \\ & \quad + i*\left[ {\sum\limits_{\alpha = 1}^{8} {BI_{\alpha }^{p} 2^{\alpha - 1} } - \sum\limits_{\alpha = 1}^{8} {BI_{\alpha }^{n} 2^{\alpha - 1} } } \right] \\ \end{aligned} $$
(2)

where \(BR_{\alpha }^{p}\) and \(BR_{\alpha }^{n}\) are the \(\alpha\) bit-planes for the positive and negative parts of the real component, respectively, while \(BI_{\alpha }^{p}\) and \(BI_{\alpha }^{n}\) are the \(\alpha\) bit-planes for the positive and negative parts of the imaginary component, respectively.

Opto-electronic phase conjugation (OEPC) using DMD

Assuming an initial complex field (\(U_{i}\)) incident on a scattering medium, we can write the output as follows:

$$ U_{o} = T_{io} U_{i} $$
(3)

where \(T_{io}\) is the transmission matrix describing the propagation of \(U_{i}\) through the scattering medium. After taking the complex conjugate of this function and propagate through the same medium, we get:

$$ \begin{aligned} U_{final} & = T_{oi} U_{o}^{*} \\ & = T_{oi} (T_{io} U_{i} {)}^{*} = T_{oi} T_{io}^{*} U_{i}^{*} \\ \end{aligned} $$
(4)

Assuming a perfect time-symmetric system (i.e. \(T_{oi}^{ - 1} = T_{io}^{*}\)), Eq. 4 simplifies to:

$$ U_{final} = T_{oi} T_{io}^{*} U_{i}^{*} { = }T_{oi} T_{oi}^{ - 1} U_{i}^{*} = \, U_{i}^{*} $$
(5)

Equation (5) shows how OEPC removes the aberrations associated with the medium (i.e. free space propagation or mouse liver tissue in our experiments) and the input field is retrieved. In order to demonstrate OEPC using the DMD, a simplified interferometric setup was used as shown in Fig. 3. The detector used in this experiment is a CMOS camera (Photonfocus Inc., resolution: 1312 × 1280, pixel size: 8 µm). First, the setup was calibrated to ensure pixel-to-pixel alignment between the DMD and the camera. The incident angle is chosen to be at θ degrees with respect to the direction normal to the DMD surface. Given that the DMD has an “on” and “off” state beam at ± 12° with respect to the normal direction, θ was chosen to be around 12° relative to the normal direction as shown in Fig. 3b. Owing to the specific distribution of the pixels in the DMD, (Fig. 3a), interpolation was used in order to successfully align the pixels of the DMD and SLM. As seen in Fig. 3a, the center-to-center distance between two successive columns (i.e. Cn−1 and Cn) is 10.8 µm while the distance between two successive rows is equal to 5.4 µm (i.e. 10.8/2 µm) as given by the DMD specification. This difference was accommodated by the use of interpolation to ensure successful mapping of the detector image onto the DMD screen. After calibrating the system, the recording step (Fig. 3b) was performed to by displaying phase and/or amplitude maps as expressed by Eq. (2). After acquiring the image on the CMOS detector, the hologram was processed and the complex field was extracted on a personal computer and its complex conjugate was displayed back onto the DMD (Fig. 3c) and finally the original non-distorted image is reconstructed by adding the real and the imaginary parts digitally.

Figure 3
figure 3

OEPC optical setup. (a) DMD projector grid showing the pixel orientation. (b) Recording step is performed first to record the hologram after free-space propagation. The second step is the (c) playback where the complex conjugate of the detected field is displayed onto the DMD and the original field is retrieved. M mirror, BS beamsplitter.

Full size image

Results

OEPC for diffracted pattern after passing through scattering media

An experiment was performed with a scattering medium in between the DMD and the detector as shown in Fig. 4. The scattering medium was a 10 µm-thick fixed mouse liver-tissue sandwiched between a standard cover-slide of 1 mm thickness and a coverslip of 170 µm thickness.

Figure 4
figure 4

Experimental setup with the scattering medium inserted in between the DMD and the camera (M mirror, L1&L2 lenses, d = 28 cm).

Full size image

In the OEPC experiment, a simple object (a binary amplitude circular mask of ~ 430 µm diameter) was displayed on the DMD and the reflected light was captured by the detector. By extracting the complex field of the acquired hologram, taking its complex conjugate and feeding it back to the DMD with the scattering medium in place, we were able to retrieve the original pattern. Figure 5 shows the diffracted and the refocused spot after OEPC. Figure 5b shows how we get a full-width at half-maximum (FWHM) of 250 µm whereas the defocused spot (Fig. 5a) has a FWHM of 400 µm.

Figure 5
figure 5

Intensity pattern before and after refocusing using OEPC technique. (a) Diffracted intensity pattern after propagating through the scattering medium path while (b) shows the intensity pattern after OEPC.

Full size image

A similar experiment was performed with the USAF target being the input amplitude-only field passing through the same scattering medium (i.e. liver tissue). The image on the left hand-side of Fig. 6 shows the diffracted intensity pattern through the sample while the right hand-side image shows the refocused USAF target after compensating the effect of defocusing and scattering through the sample.

Figure 6
figure 6

Intensity pattern before and after refocusing using OEPC. Left: diffracted intensity pattern after propagating through the scattering medium. Right: shows the intensity pattern after OEPC.

Full size image

Grayscale modulation using amplitude modulator synchronized with the DMD

In order to boost further the speed of grayscale modulation using the DMD, amplitude modulation was used instead of time modulation. As shown in Fig. 7, an electro-optic modulator (Thorlabs, EO-AM-NR-C1) was used to modulate the light intensity for each bit-plane. In this case we used light intensity instead of light amplitude since holographic recording was not used on the camera. Therefore the calibration was done for intensity recording. The NI PCIe-6321 data acquisition tool was used for fast synchronization permitting speeds up to 90 kHz which is much faster than the DMD speed (6.6 kHz). Figure 7 shows the calibration process of the amplitude modulator (AM). A photodetector (Thorlabs, PDA36A-EC) was used to measure the voltage signal from the AM as a function of the applied voltage. As the characteristic half-wave voltage (Vπ) at 725 nm for the AM was around 240 V, high voltage was needed to get the full modulation range which was generated using a high voltage amplifier (HVA200, Vin = − 10:10, Vout = − 200:200, Thorlabs). A quarter wave plate (QWP) was used to get the full range since Vπ is higher than the maximum possible voltage generated by the voltage amplifier.

Figure 7
figure 7

Amplitude modulation. (a) Calibration system for the amplitude modulator. (b) Amplitude modulation (orange) as a function of the applied voltage (blue) without the QWP. (c) Amplitude modulation (orange) as a function of the applied voltage (blue) with the QWP. Comparing (b) and (c), adding the QWP resulted in creating an input circular polarization which resulted in getting the complete modulation range.

Full size image

A look-up-table (LUT) is extracted from Fig. 7c to modulate the 8 bit-planes of the 8-bit grayscale image. After calibrating the AM, the amplitude modulator was synchronized with the DMD and images were recorded using the camera as shown in Fig. 8. With such modulation scheme a maximum achievable frame rate of 833 Hz could be reached (i.e.\(\frac{1}{{150\;\upmu {\text{s}} \times 8}}\)). Using this amplitude-modulation scheme, we showed efficient grayscale modulation for sinusoidal grating using the DMD (see supplementary video).

Figure 8
figure 8

Amplitude modulation using AM synchronized with the DMD.

Full size image

Figure 9 shows an example of amplitude modulation for a sinusoidal grating with a grating period of 15 pixels displayed onto the DMD and imaged by the camera where it shows the effectiveness of our proposed scheme as compared to binary grating where the higher diffraction orders are visible as shown in Fig. 9(b1). For the amplitude modulation case, higher orders are highly suppressed while the 0th, + 1, − 1 orders are dominant.

Figure 9
figure 9

Sinusoidal grating projected from the DMD using amplitude modulation scheme. (a1, a2) The recorded gratings. (b1, b2) Fourier transform of recorded gratings. (a1, b1) show the binary grating and the associated Fourier transform where the diffraction noise (artifacts) are dominant. (a2, b2) show the sinusoidal grating and the associated Fourier transform in which the 3 main orders are visible with no visible diffraction artifacts.

Full size image

The DMD consists of individually-controlled mirrors, and this creates many diffraction orders29. This phenomenon is shown in Fig. 10 where the pattern recorded on the DMD (a square) is repeated multiple times. The photograph in Fig. 10 was taken by placing the camera close to the DMD in order to capture the multiple patterns and demonstrate the effect.

Figure 10
figure 10

Diffraction patterns from the DMD as a result of mirror deflections.

Full size image

During the performance of the experiment it was noticed that using a lens (L3 in Fig. 8) with higher focal length showed better reconstructed images. This is due to the fact that as the lens gets far from the DMD, additional diffracted orders from the DMD miss the lens and are not captured by the camera while only the zero-diffracted beam is able to pass through the lens and be captured by the camera. Notice that in practice, this approach is not light efficient, and a lens design such as the one used in display applications30,31 that captures all the diffracted orders and superimposes them on the camera is preferable.

To further validate our results, phase modulation was performed as described earlier in which the complex field is divided into real and imaginary components and then each component was presented separately using the electro-optic amplitude modulation scheme. We recorded directly intensity on the camera where the signals corresponding to the different bit planes were integrated on the camera. The real and imaginary parts were read-out and were added digitally on the pc to form the complex field. A blazed grating and a binary phase grating were synthesized on the DMD. As shown in Fig. 11, the blazed grating shows all the power focused into the + 1 order. On the other hand, the binary phase grating resulted in multiple orders as expected.

Figure 11
figure 11

Phase gratings projected from the DMD using amplitude modulation scheme. (a1, a2) The recorded gratings. (b1, b2) Fourier transform of recorded gratings. (a1, b1) show the blazed grating and the associated Fourier transform where the diffraction noise (artifacts) are dominant. (a2, b2) show the binary phase grating and the associated Fourier transform in which the multiple orders are present.

Full size image

Discussion

In the optical OEPC experiments, the curvature of the DMD surface was corrected digitally by acquiring another hologram with all the DMD pixels in the “on” state (i.e. background hologram), then all the collected holograms for the real and the imaginary parts were corrected by dividing the extracted fields over the background field, this resulted in a clean image without phase-artifacts as shown in Figs. 2, 3, 4, 5, and 6. For the time-multiplexing scheme, the 8-bit frame rate is rather low (~ 26 frames per second). This speed can be further boosted using a faster commercially available DMD32 (refresh rates ~ 32 kHz instead of 6.6 kHz) and also reducing the region of interest (ROI) on the DMD if the application allows a smaller number of pixels. For instance, using the same DMD we used in our study, by decreasing the ROI to 16 × 1140 instead of 912 × 1140, refresh rates can be boosted to 100 kHz (corresponding to a complex-field display rate ~ 110 frames per second). The speed of time modulation technique can be further enhanced by using 4-bit images instead of 8-bit images which will yield a speed enhancement by a factor of 16. Figure 12 shows the effect of changing the number of bits used in the DMD when carrying out OEPC on the focusing quality after passing through the scattering medium (the liver tissue). It is observed that both 4-bit (Fig. 12b) and 8-bit (Fig. 12c) modulation depth resulted in a well preserved focused spot however with a speed enhancement factor of 16 with a display time of 2.4 ms (2.4 ms = 150 µs × 16) with an effective frame rate of 415 frames per second.

Figure 12
figure 12

Effect of changing the modulation depth on the focusing quality. (a) 2-bit, (b) 4-bit, (c) 8-bit. (b), and (c) show well preserved focused spot while (a) shows stronger artifacts.

Full size image

Table 1 shows a comparison between amplitude and time modulation schemes in terms of the frame rate for the used DMD whose refresh rate is 6.66 kHz.

Table 1 Comparison between different modulation schemes in terms of frame rate/display time.
Full size table

Current DMD technologies have refresh rate as high as 32 kHz which would scale up the modulation speed as shown in Table 2.

Table 2 Comparison between different modulation schemes in terms of frame rate/display time for DMD with refresh rate of 32 kHz.
Full size table

Conclusion

High speed, complex wavefront shaping using the digital micro-mirror device was demonstrated. The DMD was used in a time-multiplexing mode which allows for 8-bit grayscale image representation on the DMD within 38.4 ms allowing for focusing in live samples or for other applications without compromising the accuracy. For higher modulation rates, amplitude modulation was demonstrated with maximum achievable frame rate for 8-bit images of 833 Hz yielding a speed up by a factor of 32 as compared to time multiplexing schemes.

The method we demonstrated relies on time integrating holographic recordings of the fields or intensity patterns corresponding to multiple binary patterns on the DMD. The method is particularly useful for optical systems that use digital holography. The method might also be useful in applications where the projected pattern is integrated in time by a light sensitive material such as a fluorescent molecule in imaging or a photo-initiator molecule in 3D printing.