Devices known as volumetric displays allow 3D images to be generated in a transparent enclosure. Because these images occupy three dimensions, they exhibit the spatial characteristics that we associate with real-world scenes. The images can be viewed without the need for glasses by many simultaneous observers, and changes in vantage point allow content to be seen from different orientations. In a paper in Nature, Smalley et al.1describe an innovative approach to volumetric-display implementation that allows 3D images to be formed in the air, removing the need for a transparent enclosure.
For more than 100 years, volumetric displays have been the subject of extensive research2. Although it is relatively easy to make a small (tabletop) display that works fairly well, it is extremely difficult to develop a larger display that works very well. There are two overarching (but often conflicting) problems. The first relates to the techniques that are currently used to produce dynamic images of relatively high visual quality. The second concerns the optical characteristics of the imaging volume, which must allow light emanating from the image to propagate, and emerge from the volume, without distortion — think of the distortion that occurs when light emerges from a tropical-fish tank.
With respect to the first problem, in most volumetric displays, the imaging volume is formed by the cyclic motion of a transparent surface (Fig. 1a). To produce a 3D image, a sequence of image slices is depicted on the surface as it moves through the volume. Given the need to refresh images at least 30 times per second to avoid perceptible flicker3, the surface must move rapidly.
The motion of the surface can be either translational (along a straight line) or rotational. When translational motion is used, the dimensions of the imaging volume are limited by mechanical issues arising from the surface’s mass and acceleration. In the case of rotational motion, the surface’s linear speed increases with distance from the axis of rotation. This impinges on image quality and so can ultimately restrict the diameter of the imaging volume. There is also a ‘dead’ region in the vicinity of the rotational axis, in which image points cannot be formed4.
A further limitation of these displays is that the surface’s movement precludes the insertion of haptic probes — tools that recreate the sense of touch by applying forces, motion or vibrations to the user. Such probes can simulate the solidity associated with physical versions of images, so that, for example, virtual clay could be moulded and would feel like real clay.
Smalley et al. sought to overcome all of these difficulties using the photophoretic effect5, whereby laser light is used to trap and move small particles (with diameters of 5–100 micrometres). To create a point of light at a given location in 3D space, the authors used non-visible laser radiation to move a particle, and as the particle passed through the required position, it was illuminated with red, green or blue light (Fig. 1b). The authors suggest that complex, high-fidelity, dynamic images could be formed by introducing parallelism — the simultaneous movement of many particles.
There are at least three key advantages of Smalley and colleagues’ approach. First, it does not require the cyclic motion of a surface — movement is restricted to that of low-mass particles. Second, the presence of these particles will have minimal impact on the propagation of light through the imaging volume. And third, because the image is formed in the air, image components can coexist with haptic probes and other interaction tools.
The authors provide several photographs of image content produced using their technique (see Figure 2 of the paper1). However, these photographs required long exposure times — of the order of tens of seconds. For implementing a viable display, there is therefore a pressing need to explore ways of increasing the speed of particle motion and of introducing parallelism such that many image points can be created simultaneously.
The introduction of a high degree of parallelism poses a further challenge, relating to the fact that each point in the imaging volume must be individually accessible. This is reminiscent of an equivalent problem that was encountered in the late 1960s, in connection with a type of 3D display called a photochromic-based volumetric display6,7. Another concern is that the insertion of haptic probes into the image volume will probably give rise to shadow regions that will interfere with the propagation of light used for particle motion and illumination. However, the judicious design of such probes would ameliorate this potential problem.
In terms of photorealism, it is unlikely that these devices will ever directly compete with high-end stereoscopic 3D displays. However, despite more than a century of research into volumetric displays, there has been relatively little work on exploring ways of capitalizing on key image characteristics. In particular, volumetric displays provide considerable freedom in viewing position, and support both vertical and horizontal motion parallax, which means that observers can move and change their view of an image in a wholly natural way.
Consequently, these devices offer exciting, and largely unexplored, opportunities to advance spatial imaging (in areas such as neurosurgery) and dynamic imaging (in fields including fluid dynamics, robotics and sports training). With regard to the latter, there is a need to better support the visualization of complex forms of 3D motion8. Moreover, creating volumetric images in the air enables direct interaction, thereby allowing, for example, 3D design tasks to be carried out in a natural way in 3D space.
Smalley and colleagues’ approach could provide the foundation for the next generation of volumetric displays. Such devices will not only enhance our understanding of complex spatial and geometric dynamics, but also support innovative user interaction.
Nature 553, 408-409 (2018)