Glasses-free three-dimensional (3D) displays are one of the game-changing technologies that will redefine the display industry in portable electronic devices. However, because of the limited resolution in state-of-the-art display panels, current 3D displays suffer from a critical trade-off among the spatial resolution, angular resolution, and viewing angle. Inspired by the so-called spatially variant resolution imaging found in vertebrate eyes, we propose 3D display with spatially variant information density. Stereoscopic experiences with smooth motion parallax are maintained at the central view, while the viewing angle is enlarged at the periphery view. It is enabled by a large-scale 2D-metagrating complex to manipulate dot/linear/rectangular hybrid shaped views. Furthermore, a video rate full-color 3D display with an unprecedented 160° horizontal viewing angle is demonstrated. With thin and light form factors, the proposed 3D system can be integrated with off-the-shelf purchased flat panels, making it promising for applications in portable electronics.
Glasses-free three-dimensional (3D) displays are regarded as one of the most promising technologies that will redefine portable electronics1,2,3,4,5,6,7, yet little progress has been made in the physical foundation of 3D displays. Multiview 3D displays discretize the light field into spaced views to provide a 3D stereoscopic experience for many observers simultaneously8,9,10,11,12,13,14,15,16. To obtain ideal 3D display utility, high-resolution 3D images should be projected to spaced views with smooth motion parallax over a large viewing angle17,18. However, limited by state-of-the-art display information, the spatial resolution, angular resolution, and viewing angle have become critical trade-offs in 3D displays1,19. For example, to build a reasonably good 3D display with a 1 K spatial resolution and a 3° angular resolution along the vertical and horizontal directions, a display panel with a 50 K resolution is required for a viewing angle of 150°, which is an order of magnitude greater than that of commercial products.
One solution is to exploit the temporal redundancy to increase the amount of display information20,21,22,23,24,25,26. For example, a multi-projection time-multiplexed 3D display has been proposed by using a steering screen to improve the angular resolution23. Most recently, a slim-panel holographic video display with an angular steering backlight has been introduced to increase the effective space-bandwidth product at the expense of a reduced refresh rate26. On the other hand, nature provides us with another strategy for addressing the critical trade-off among the spatial resolution, angular resolution, and viewing angle when display information is limited27,28,29,30. Recently, a foveated vision strategy was introduced in a single-pixel computational imaging system, thereby increasing the frame rate by reducing the number of pixels in the peripheral region27. In a vertebrate eye, the spatially variant visual density provides a high angular resolution of 1 min of arc at the central region of the retina (the fovea centralis), as well as a wide viewing angle of 160° without significantly increasing the total information that needs to be processed.
Inspired by the vertebrate eyes, we propose a general approach of 3D display, through which spatially variant information is projected based on the frequency of observation. Densely packaged views are arranged at the center, while sparsely arranged views are distributed at the periphery. In fact, package views in a gradient density are straightforward, but nontrivial. First, the angular separation of the views needs to be varied. Second, the irradiance pattern of each view has to be tailored so as to eliminate overlap between views to avoid crosstalk. Third, one should avoid gaps between views to ensure smooth transition within the field of view (FOV). As a result, views with hybrid dots, lines, or rectangle distributions are desirable to achieve gradient density. However, 3D display based on geometric optics, such as lenticular lens, microlens arrays, or pinhole arrays, can neither manipulate gradient view distribution nor expand the FOV31,32,33. The progress in planar diffractive optics based on photonics opened an opportunity for precise light field manipulation34,35,36,37. For example, a silicon-based metasurface diffractive optical element was reported for structured light projection over 120° FOV38. The full-color and dynamic 3D holography using metasurfaces made of subwavelength nanostructures with spatially varying orientations or sizes were also reported39,40,41,42. Metamaterials can provide superior light manipulation for 3D display, but the design and fabrication of nanostructures over a large scale for display (5–100 inch) application is a big challenge43,44.
To manipulate view distribution over a large scale, we design and propose a feasible strategy based on the two-dimensional (2D)-metagrating complex (2DMC). The 2DMCs are proposed to individually control both the propagation direction and the irradiance distribution of the emergent light from each 2D metagrating. As a result, the 3D display system provides a high spatial and angular resolution at the central viewing zone, i.e., the most comfortable observing region. Since the periphery viewing zone is less used in most occasions, we suppress the redundant depth information and broaden the FOV to a range comparable to that of a 2D display panel. Furthermore, a homemade flexible interference lithography (IL) system is developed to enable the fabrication of the view modulator with >1,000,000 2D metagratings over a size >9 inch. With total display information <4 K, a static or video rate full-color 3D display with an unprecedented FOV of 160° is demonstrated. The proposed 3D display system has a thin form factor for potential applications in portable electronic devices.
The foveated glasses-free 3D display
Generally, the spatial resolution (multiview display pixels Nmul) and the angular resolution (angular separation ∆θ) determine the visual experience provided by a multiview 3D display3. Therefore, we adopt the information density (pixels per degree (PPD)) to evaluate the performance of the 3D display:
where ID represents the information density. A higher information density provides a higher spatial resolution with more fluidic motion parallax. In prior studies, constant information density was provided within the viewing angle by views with the same distribution pattern (Fig. 1a). In contrast, we propose 3D display with spatially variant information density by precisely manipulating the view distribution into hybrid dot/line/rectangle shape (Fig. 1b).
Figure 1c illustrates the schematic of the view modulator with 2DMCs. To generate a horizontally variant display information density, we define 9 irradiance patterns with variant widths. Pixelated 2D metagratings (3 × 3), which are grouped into a voxel, are designed to provide the predefined view distribution. We reserve detailed calculation of the 2D metagratings in the view modulator pixel by pixel to the Supplementary Information (Section 1). As a result, the information density distribution will be modulated as in the foveated vision.
Design and fabrication of 2D metagratings
The cornerstone of the proposed display architecture is a large-scale 2DMC on the view modulator. With a size up to 9 inch, the data volume of 2DMCs is >1.8 Tb. Due to the large data volume, both the design and fabrication of 2DMCs is nontrivial.
Figure 2a illustrates the schematic of the design process of 2DMCs. We first designed the phase hologram of the nanostructures according to the target view distribution by the Gerchberg–Saxton algorithm45. Four typical phase holograms responsible for typical view distributions are summarized in Table 1, and they can be applied in different scenarios. For example, dot-shaped views provide the highest information density. The vertically oriented and horizontally oriented line-shaped views reduce the information density in one direction while maintaining the information density in the other direction. The rectangular-shaped views reduce the information density along both directions, which are typically adopted for peripheral observing region. Although the diffractive pattern for each voxel is the same, the position of each pixel and the diffraction angle for the emergent beam varied. As a result, a unique nanostructure is donated to each pixel over the entire view modulator. Furthermore, with negligible tolerance, it has been proven that the 2D metagratings corresponding to the same view have similar shapes but with different scaling factors of periods and orientations (for additional information, see Section 2 in the Supplementary Information).
The fabrication of a view modulator with complex nanostructures remains a challenge. On the one hand, electron-beam lithography (EBL) is a typical nanopatterning tool in the laboratory46,47,48,49, but it suffers from limited throughput and size. On the other hand, laser direct writing (LDW) technology can fabricate patterns over several inches50, yet the minimum resolvable line width is diffraction limited to a submicron scale. Herein we developed a versatile IL system, as shown in Fig. 2b. A collimated and expanded laser beam illuminates a phase-modulated system, which consists of two Fourier transform lenses and a binary optical element (BOE) inserted in between. Then an interference pattern is formed by the multiple diffractive beams of the BOE at the back focal plane of the second Fourier transform lens. Finally, the interference pattern light field is minified by an objective lens and projected on the photoresist. The patterned structures on the photoresist are a minified multibeam interference pattern of the BOE. Details about the principles of our versatile IL system can be found in the Supplementary Information (Sections 2 and 3). Therefore, we enabled the fabrication of 2DMCs on the view modulator to form dot, linear, and rectangular hybrid shaped view distribution shown in Table 1.
Furthermore, the axial movement and axial rotation of the BOEs between two Fourier transform lenses lead to variations in the scaling factor of periods and orientation of the patterned 2D metagratings51, respectively (Fig. 2c). A pixelated 2D metagrating can be fabricated by pulse exposure. On the one hand, the 2DMCs for one view can be patterned by precisely controlling the scaling factor of periods and the orientation. On the other hand, the 2DMCs for views with different irradiance shapes can be achieved by inserting the corresponding BOEs in the homemade IL system. Furthermore, it is worth noting that the periodic tuning accuracy of fabricated 2D-metagrating can reach within 1 nm. The processing efficiency of the IL system can reach 20 mm2 mins−1, 500 times faster than the speed of EBL.
Performance of the space-variant information density 3D display
Enabled by the homemade IL system, we fabricated several view modulators with different screen sizes and complexity (for details about the fabrication, see “Materials and methods”). The typical parameters of the three prototypes are summarized in Table 2. To prove the concept, we made a 6-inch view modulator with horizontal-variant information density. Figure 3a shows the variation of the scaling factor for periods of the 2D metagratings on the view modulator. The proposed 6-inch view modulator contains a total of 800 × 600 voxels, and each voxel is composed of 3 × 3 pixels for 9 views. That is to say, a total of 4,320,000 2D metagratings need to be patterned on the view modulator. A microscopic image of the 2DMCs on the view modulator is shown in Fig. 3b. Figures 3c and S4 presents the measured and simulated radiation pattern of the 9-view modulator prototype (for details about the simulation, see “Materials and methods”), respectively. Seven views (Views 2–8) are uniformly distributed in the central region with an angular separation of 10°, while the peripheral views (Views 1 and 9) cover 40° on each side of the central views. The crosstalk is measured as 14.88% (for detailed measurement, see Section 4 in the Supplementary Information). Compared with the theoretical value of 8%, a slight increment in experimental value is observed. Besides, the diffraction efficiency of 2DMC for red/green/blue (R/G/B) color is measured as 8.89, 7.72, and 11.92%, respectively. In contrast, the theoretical diffraction efficiency of 500 nm deep 2DMC is 20%. The experimental deviations for both crosstalk and diffraction efficiency are induced by nanofabrication inaccuracy and the misalignment in system assembly.
A shadow mask with hybrid images of numbers is adopted to match the 9-view modulator pixel by pixel. When the light from a collimated light-emitting diode (LED) illuminates the prototype, we record the “1–9” numbers projected to each view, as shown in Fig. 3e. The horizontal FOV is 160°, and the vertical FOV is 50° (Visualization 1). The information density is modulated to 80 PPD at the central region and 26.7 PPD at the periphery (Fig. 3d).
For video rate full-color 3D displays, we successively stack a liquid crystal display (LCD) panel, color filter, and view modulator together to keep the system thin and compatible (Fig. 4a). Since most LCD panels have already been integrated with a color filter, the system integration can be simply achieved by pixel to pixel alignment of the 2D-metagrating film with the LCD panel via one-step bonding assembly. The layout of 2DMCs on the view modulator is designed according to the off-the-shelf purchased LCD panel (P9, HUAWEI) (Fig. 4b). To minimize the thickness of the prototype, 2D metagratings are nanoimprinted on a flexible polyethylene terephthalate (PET) film with a thickness of 200 µm (Fig. 4c), resulting in a total thickness of <2 mm for the whole system (Fig. 4d).
A white LED light illuminates the 2D metagratings from the back with a filtered wavelength and modulated intensity. The emergent beams from R/G/B 2DMCs are combined for full-color display (Fig. 5a, b, Visualization 2 and 3). The FOV reaches a record of 160°. The information density is modulated to 200 PPD at the central region and 70.6 PPD at the periphery.
Here we achieved full-color 3D display with significantly suppressed color dispersion by several ways. First, since 2D-metagratings are wavelength sensitive, the structure of 2DMC is designed pixel by pixel according to the wavelength. Second, from the system point of view, the introduction of color filter significantly filters out the influence of color dispersion. Third, we pre-calibrate the white balance of the prototype. The displayed images can be pre-processed to further reduce the color dispersion.
In the experiment, we changed the information density by varying the angular resolution. In another strategy, we can assign more pixels to the central views, thereby the spatial resolution is increased as in the foveated vision.
Facilitated by the rapid advancement of nano-optics, we presented a general design strategy of glasses-free 3D display from view modulation aspect. The view modulator for multiple view projection is no longer a simple conjugate relation between image and object. We proved that 2DMC can be designed to precisely tailor the view distribution for gradient view arrangement and enlarged viewing angle. The view modulator with 2DMCs can be further designed to eliminate crosstalk or increase viewing depth. Moreover, by combining views with a fan-shaped irradiance pattern, a tabletop 3D display system with variant information density can be realized.
In summary, we propose a facile and robust approach for spatially variant information density 3D display with a large-scale 2DMC served as a view modulator. A homemade flexible IL system is developed to enable the nanopatterning of view modulator with increased complexity for portal electronic devices. As a result, high angular resolution is preserved in the central region, while a wide viewing angle is maintained. The display information is arranged nonuniformly based on the observing habit of human beings. Hence, we demonstrate a full-color, video rate 3D display with a thin form factor. The viewing angle sets a record of 160° for the glasses-free 3D display.
The demonstrated spatially variant information density 3D display opens a new avenue for glasses-free 3D displays by tackling the critical trade-off among the spatial resolution, angular resolution, and viewing angle. We anticipate the ultrawide-FOV foveated 3D display to be used in commercial applications, such as consumer electronic devices.
Materials and methods
(1) For the inserted BOE: First, a 2.5-inch quartz plate was precleaned and spin coated with hexamethyldisilazane (DisChem, SurPass 3000) and positive photoresist (MicroChemicals, AZ® P4620) at a total thickness of 1 μm. Then the quartz plate was micropatterned with various binary phase holograms using a homemade LDW system (SVG Optronics, MiScan200). After photolithography, the phase holograms were developed in a NaOH solution and blown dry. Finally, the BOE structures were etched to a depth of 700 nm, and the minimum period was approximately 7.5 μm. The BOEs were finally inserted in the self-developed IL system for the fabrication of view modulator.
(2) For the view modulators: First, a glass substrate was cut and precleaned. The glass substrate was then coated with positive photoresist (RUIHONG Electronics Chemicals, RJZ-390) at a thickness of 1 μm. Then the 2DMCs were successively patterned by a self-developed IL system. The pixel size of each unit can be adjusted according to the aperture size. It took 27 h to prepare a 6-inch 2DMC with 2400 × 1800 pixels. After IL, the nanopatterns were developed in a NaOH solution and blown dry, and then it was electroplated with a layer of nickel (NI) to make a master mold. The NI-plated mold was then used to imprint the 2D metagratings onto the polyurethane (PUA) resin, adopting roll-to-plate nanoimprint lithography. The PUA resin was then subsequently cured by ultraviolet light for 3 min. Finally, those 2DMCs were effectively mass transferred on a flexible PET membrane to form a view modulator.
3D simulations were performed using the finite-difference time-domain (FDTD) method, and FDTD simulations were conducted using Lumerical’s FDTD solver. The refractive index of the photoresist was set as 1.476. We used a plane wave source with an incident angle of 30°, and the wavelength was 540 nm. We used Bloch and perfectly matching layer boundary conditions for the transverse and longitudinal directions, respectively. The practical 2DMCs were replaced with spatial-multiplexing gratings with multiple periods. The periods ranged from 600 to 1400 nm. The mesh accuracy was chosen as a compromise among the accuracy, memory requirements, and simulation time.
Nam, D. et al. Flat panel light-field 3-D display: concept, design, rendering, and calibration. Proc. IEEE 105, 876–891 (2017).
Wetzstein, G. et al. Compressive light field displays. IEEE Computer Graph. Appl. 32, 6–11 (2012).
Fattal, D. et al. A multi-directional backlight for a wide-angle, glasses-free three-dimensional display. Nature 495, 348–351 (2013).
Yoon, H. et al. Arrays of Lucius microprisms for directional allocation of light and autostereoscopic three-dimensional displays. Nat. Commun. 2, 455 (2011).
Hong, K. et al. Full-color lens-array holographic optical element for three-dimensional optical see-through augmented reality. Opt. Lett. 39, 127–130 (2014).
Li, K. et al. Full resolution auto-stereoscopic mobile display based on large scale uniform switchable liquid crystal micro-lens array. Opt. Express 25, 9654–9675 (2017).
Teng, D. D. et al. Multiview three-dimensional display with continuous motion parallax through planar aligned OLED microdisplays. Opt. Express 23, 6007–6019 (2015).
Ni, L. X. et al. 360-degree large-scale multiprojection light-field 3D display system. Appl. Opt. 57, 1817–1823 (2018).
Lee, J. H. et al. Optimal projector configuration design for 300-Mpixel multi-projection 3D display. Opt. Express 21, 26820–26835 (2013).
Takaki, Y. & Nago, N. Multi-projection of lenticular displays to construct a 256-view super multi-view display. Opt. Express 18, 8824–8835 (2010).
Yang, S. W. et al. 162-inch 3D light field display based on aspheric lens array and holographic functional screen. Opt. Express 26, 33013–33021 (2018).
Okaichi, N. et al. Integral 3D display using multiple LCD panels and multi-image combining optical system. Opt. Express 25, 2805–2817 (2017).
Liao, H. E. Super long viewing distance light homogeneous emitting three-dimensional display. Sci. Rep. 5, 9532 (2015).
Krebs, P. et al. Homogeneous free-form directional backlight for 3D display. Opt. Commun. 397, 112–117 (2017).
Wan, W. Q. et al. Multiview holographic 3D dynamic display by combining a nano-grating patterned phase plate and LCD. Opt. Express 25, 1114–1122 (2017).
Zhou, F. et al. Pixelated blazed gratings for high brightness multiview holographic 3D display. IEEE Photonics Technol. Lett. 32, 283–286 (2020).
Watanabe, H. et al. Pixel-density and viewing-angle enhanced integral 3D display with parallel projection of multiple UHD elemental images. Opt. Express 28, 24731–24746 (2020).
Zhao, Z. F. et al. Bionic-compound-eye structure for realizing a compact integral imaging 3D display in a cell phone with enhanced performance. Opt. Lett. 45, 1491–1494 (2020).
Lee, S. et al. Tomographic near-eye displays. Nat. Commun. 10, 2497 (2019).
Ting, C. H. et al. Multi-user 3D film on a time-multiplexed side-emission backlight system. Appl. Opt. 55, 7922–7928 (2016).
Hwang, Y. S. et al. Time-sequential autostereoscopic 3-D display with a novel directional backlight system based on volume-holographic optical elements. Opt. Express 22, 9820–9838 (2014).
Yang, L. et al. Viewing-angle and viewing-resolution enhanced integral imaging based on time-multiplexed lens stitching. Opt. Express 27, 15679–15692 (2019).
Xia, X. X. et al. Time-multiplexed multi-view three-dimensional display with projector array and steering screen. Opt. Express 26, 15528–15538 (2018).
Fan, H. et al. Full resolution, low crosstalk, and wide viewing angle auto-stereoscopic display with a hybrid spatial-temporal control using free-form surface backlight unit. J. Disp. Technol. 11, 620–624 (2015).
Liu, B. Y. et al. Time-multiplexed light field display with 120-degree wide viewing angle. Opt. Express 27, 35728–35739 (2019).
An, J. et al. Slim-panel holographic video display. Nat. Commun. 11, 5568 (2020).
Phillips, D. B. et al. Adaptive foveated single-pixel imaging with dynamic supersampling. Sci. Adv. 3, e1601782 (2017).
Cem, A. et al. Foveated near-eye display using computational holography. Sci. Rep. 10, 14905 (2020).
Yoo, C. et al. Foveated display system based on a doublet geometric phase lens. Opt. Express 28, 23690–23702 (2020).
Chang, C. L., Cui, W. & Gao, L. Foveated holographic near-eye 3D display. Opt. Express 28, 1345–1356 (2020).
Lv, G. J. et al. Autostereoscopic 3D display with high brightness and low crosstalk. Appl. Opt. 56, 2792–2795 (2017).
Song, M. H. et al. Integral imaging system using an adaptive lens array. Appl. Opt. 55, 6399–6403 (2016).
Yang, L. et al. Demonstration of a large-size horizontal light-field display based on the LED panel and the micro-pinhole unit array. Opt. Commun. 414, 140–145 (2018).
Chen, Z. G. & Segev, M. Highlighting photonics: looking into the next decade. eLight 1, 2 (2021).
Xiong, J. H. & Wu, S. T. Planar liquid crystal polarization optics for augmented reality and virtual reality: from fundamentals to applications. eLight 1, 3 (2021).
Huang, K. et al. Planar diffractive lenses: fundamentals, functionalities, and applications. Adv. Mater. 30, 1704556 (2018).
Banerji, S. et al. Imaging with flat optics: metalenses or diffractive lenses? Optica 6, 805–810 (2019).
Ni, Y. B. et al. Metasurface for structured light projection over 120° field of view. Nano Lett. 20, 6719–6724 (2020).
Huang, L. L. et al. Three-dimensional optical holography using a plasmonic metasurface. Nat. Commun. 4, 2808 (2013).
Hu, Y. Q. et al. 3D-Integrated metasurfaces for full-colour holography. Light. Sci. Appl. 8, 86 (2019).
Hu, Y. Q. et al. Trichromatic and tripolarization-channel holography with noninterleaved dielectric metasurface. Nano Lett. 20, 994–1002 (2020).
Li, J. X. et al. Magnesium-based metasurfaces for dual-function switching between dynamic holography and dynamic color display. ACS Nano 14, 7892–7898 (2020).
Wan, W. Q. et al. Holographic sampling display based on metagratings. iScience 23, 100773 (2020).
Shi, J. C. et al. Spatial multiplexing holographic combiner for glasses-free augmented reality. Nanophotonics 9, 3003–3010 (2020).
Gerchberg, R. W. & Saxton, W. O. A practical algorithm for the determination of phase from image and diffraction plane pictures. Optik 35, 237–246 (1972).
Chen, Y. F. Nanofabrication by electron beam lithography and its applications: a review. Microelectron. Eng. 135, 57–72 (2015).
Zeitner, U. D. et al. High performance diffraction gratings made by e-beam lithography. Appl. Phys. A 109, 789–796 (2012).
Manfrinato, V. R. et al. Resolution limits of electron-beam lithography toward the atomic scale. Nano Lett. 13, 1555–1558 (2013).
Manfrinato, V. R. et al. Determining the resolution limits of electron-beam lithography: direct measurement of the point-spread function. Nano Lett. 14, 4406–4412 (2014).
Qiao, W. et al. Toward scalable flexible nanomanufacturing for photonic structures and devices. Adv. Mater. 28, 10353–10380 (2016).
Wan, W. Q. et al. Efficient fabrication method of nano-grating for 3D holographic display with full parallax views. Opt. Express 24, 6203–6212 (2016).
This work was financially supported by the Natural Science Foundation of China (NSFC) (Nos. 61975140 and 62075145), Leading Technology of Jiangsu Basic Research Plan (BK20192003), and Suzhou Natural Science Foundation of China (SYG201930).
Conflict of interest
The authors declare the following competing financial interest: W.Q., L.C., J.H., and E.H. are co-inventors on a related pending patent application.
About this article
Cite this article
Hua, J., Hua, E., Zhou, F. et al. Foveated glasses-free 3D display with ultrawide field of view via a large-scale 2D-metagrating complex. Light Sci Appl 10, 213 (2021). https://doi.org/10.1038/s41377-021-00651-1