Earlier this month, the company Apple introduced its latest product: the Apple Vision Pro. The device is an augmented-reality and virtual-reality headset that will be released in the United States in early 2024 and has a price tag of US$3,499. Cost alone will narrow the initial market of the Vision Pro. And whether there is a place for such a device in everyday electronics remains to be seen. But there is undoubtedly some interesting technology at play here.

Photograph of the transparent organic photodetector array developed by Kamijo, van Breemen and colleagues. The array can be placed over a commercial display, such as a laptop screen, to create a touchless user interface. Credit: Reproduced under a Creative Commons licence CC BY 4.0

Back in March, we discussed the potential of micro-light-emitting diodes (microLEDs), and the role they could play in the creation of displays for near-to-eye technologies — such as augmented-reality devices — and beyond1. But the Vision Pro showcases the still-evolving capabilities of organic light-emitting diodes (OLEDs). The headset incorporates two screens that are made from micro-OLED technology and offer 23 million pixels in total. This, the company states, is more than a 4K TV for each eye.

The Vision Pro also does not have any controller accessories, which is a departure from other related headsets (such as the Meta Quest Pro). The system is instead controlled via voice input, eye tracking and hand gesture recognition.

The development of gesture recognition technology has been a topic of some interest in the electronics research community over the last few years. This exploratory work has often focused on using wearable sensors — together with machine learning — to recognize and track gestures, with increasingly sophisticated results2. The Vision Pro takes a camera-based approach and thus has similarities to systems such as the Leap Motion Controller and the (discontinued) Microsoft Kinect, which use near-infrared cameras to form touchless user interfaces. Researchers are, however, also exploring other ways to build such (wearable-free) interfaces.

In an Article in this issue of Nature Electronics, Takeshi Kamijo, Albert van Breemen and colleagues report a touchless user interface that is created by simply placing an optical imager over a commercial display such as a laptop screen. The imager — which is visually transparent — is based on an organic photodetector array that can detect near-infrared light reflected off fingers and hands.

To achieve optical transparency, the researchers use a printed copper grid with a line width of 1 µm as a bottom conductive electrode. This is combined with an organic photodetector array in which each square pixel in a 16 × 16 array is itself made up of an array of 14 × 14 square subpixels that have a side length of 50 µm. As a result of the set-up, the imager has a visible-light transmittance of around 70%.

The team — who are based at Asahi Kasei Corporation in Japan, the TNO at Holst Centre in the Netherlands, Eindhoven University of Technology, and imec in Belgium — show that the system can be used as a penlight-controlled as well as a gesture-controlled touchless interface. The approach also minimizes calibration, requiring it only when the imager is first attached to a display.

Augmented- and virtual-reality headsets are not the intended application here. Instead, the authors highlight the potential of the technology in automated teller machines (ATMs), electric signage and interactive whiteboards. Other potential roles for the technology can be envisaged, and the possibilities of this and other gesture-controlled touchless interfaces — including that of the Apple Vision Pro — are intriguing. But the path to widespread adoption remains uncertain.