Main

Touch screens currently dominate the way we interact with machines, but there is an increasing demand for touchless user interfaces for applications in which hygiene is a concern, such as automated teller machines (ATMs), ticket vending machines and kiosks. Various types of touchless technology are in development including voice recognition1, eye tracking2, near-field communication3, radio-frequency signal-based hand gesture4 and gesture or motion detection5,6. Commercial touchless user interfaces—such as the Microsoft Kinect7 and Leap Motion Controller8—use near-infrared (NIR) cameras5,6. Up to 70% of 850 nm NIR light reflects off the skin9,10 and is invisible to the human eye; thus, it is commonly used in touchless user interfaces11,12.

The Microsoft Kinect uses a time-of-flight technology that offers a working range of tens of centimetres to a few metres. A calibration process is required11 to ensure sufficient positional accuracy and depends on light power and reflection from an object6. The Leap Motion Controller is a hand-gesture-controlled user interface with submillimetre accuracy and its interaction space is limited to an 60 × 60 × 60 cm3 inverted pyramid space above the controller12. ShadowSense touch is based on the detection of an object’s shadow as it enters, hovers within or transitions out of a sensor’s field of view. The object’s position in the touch plane is then calculated based on the ratio of a fully illuminated condition to the shadowed state for multiple light sources and multiple sensors.

In this Article, we report a touchless user interface that is based on a large-area solution-processed NIR-sensitive organic photodetector (OPD) array. The interface is visually transparent and can be placed in front of a conventional display, eliminating issues related to the field of view and providing high positional accuracy. NIR light reflected off fingers and hands illuminates the screen and acts as a spot input signal, which is detected by the OPD array and used to control the screen (Fig. 1). Each individually readable pixel in the 16 × 16 OPD array consists of an array of 14 × 14 OPD subpixels electrically connected in parallel. Electro-optical modelling is used to design NIR-sensitive OPDs with an external quantum efficiency (EQE) of 36% at 850 nm. Our OPDs exhibit a detectivity of 7.2 × 1012 Jones at 850 nm, together with a low dark current and a linear behaviour over a wide range of NIR-light intensities.

Fig. 1: Touchless user interface demos.
figure 1

a, Schematic of a large-area, 16 × 16 visually transparent NIR-sensitive OPD array (imager) that is placed in front of a laptop display. b, Schematic of the touchless user interface demo using NIR-emitting penlight. c, Photograph of the touchless user interface demo using NIR-emitting penlight. d, Schematic of a large-area, 16 × 16 visually transparent NIR-sensitive OPD array (imager) with integrated NIR LEDs that is placed in front of a laptop display. e, Schematic of the touchless user interface demo using gesture recognition of reflected NIR light. f, Photograph of the touchless user interface demo using gesture recognition of reflected NIR light.

Full size image

To ensure good transparency, we use a scalable sub-3-µm printed copper (Cu) grid conducting electrode with a visible-light transmittance (VLT) of over 70%, as needed for ATM display applications13. Such transparent conductive electrode (TCE) metal grids14,15,16,17,18,19,20,21 provide good electro-optical performance, imperceptibility to human eyes22 and flexibility23 when the metal line width, pitch and thickness are suitably tuned24. By using high-resolution additive printing25,26,27,28,29,30,31, we have simplified fabrication compared with conventional vacuum-based methods32,33,34 and enabled scalability and high productivity by roll-to-roll manufacturing (Supplementary Table 1 provides a comparison between our visually transparent NIR-sensitive OPD array and previously reported approaches).

Visually transparent NIR-sensitive OPD

We use an inverted OPD stack (Fig. 2a) based on a 300 nm active film consisting of a blend of poly([2,6′-4,8-di(5-ethylhexylthienyl)benzo[1,2-b;3,3-b]dithiophene]{3-fluoro-2[(2-ethylhexyl)carbonyl]thieno[3,4-b]thiophenediyl}) (PCE-10, also called PTB7-Th)35 as a donor polymer and 2,2′-((2Z,2′Z)-(((4,4,9,9-tetrakis(4-hexylphenyl)-4,9-dihydro-indaceno[1,2-b:5,6-b′]dithiophene-2,7-diyl)bis(4-((2-ethylhexyl)oxy)thiophene-5,2-diyl))bis(methanylylidene))bis(5,6-difluoro-3-oxo-2,3-dihydro-1H-indene-2,1-diylidene))dimalononitrile (IEICO-4F) (ref. 36) as a non-fullerene acceptor, slot-die coated on top of a printed Cu grid TCE (70–120 nm) electrode with an amorphous indium gallium zinc oxide (a-IGZO) electron transport layer (16 nm) and SU8 edge cover layer (1.8 µm). The OPD layer was photolithographically patterned to achieve a high optical transparency37,38. This was followed by a thermally evaporated MoOx hole transport layer (60 nm) and indium tin oxide (ITO) thin transparent electrode (100 nm), and finished with an optically transparent laminated barrier film (160 µm). Details of the step-by-step fabrication are provided in the Methods section.

Fig. 2: Visually transparent NIR-sensitive OPD.
figure 2

a, Device layout of a discrete NIR-sensitive OPD. b, Chemical structures of PCE-10 (also called PTB7-Th) and IEICO-4F. c, Band diagram of the NIR-sensitive OPD. d, Top-view schematic of a printed Cu grid TCE, showing line width (WCu), pitch (PCu), gap and unit cell (red dashed enclosure). e, Microscopic image of the printed Cu grid TCE with WCu and PCu of 1 and 20 μm, respectively. f, Schematic of a part of the patterned parallel OPD subpixel array design that forms an OPD main pixel: OPD subpixel width (WOPD), OPD subpixel pitch (POPD), OPD subpixel overlap with edge cover layer (ROPD). The grey grid lines represent the printed Cu grid shown in e. g, Micrograph of patterned OPD subpixels with WOPD, POPD and ROPD of 50, 240 and 10 μm, respectively.

Full size image

The chemical structures of both donor and acceptor materials as well as the band energy diagram of the NIR-sensitive OPD are displayed in Fig. 2b,c, respectively. Figure 2d,e shows a top-view schematic and microscopic image of the printed Cu grid TCE used in this work. The final line width (WCu), pitch (PCu) and line thickness chosen for the Cu grid were 1 μm, 20 μm and ~100 nm, respectively. Supplementary Table 2 lists all the geometric values and characteristics of the Cu grids. To realize a high VLT across the device area facing towards the user, an electrically connected parallel OPD subpixel array, processed by photolithographic patterning37,38, was implemented (Fig. 2f,g). The effect of both Cu grid and patterned OPD design on the optical transmittance and its design optimization are discussed below.

Electro-optical simulations and design rules for NIR OPDs

The effect of the metal grid design on the performance of organic photovoltaics was studied experimentally16,17,19,20,21 and simulated39,40,41. These studies, however, only provide the optimal metal grid’s design for specific device architectures and for metal grids with much wider line width (≥10 µm) than employed in this work39,40. To optimize the performance of our NIR-sensitive OPDs with a printed Cu grid TCE, EQE simulations were performed by a combination of numerical electro-optical and two-dimensional (2D) finite element modelling (FEM) measurements. Details of the EQE simulations are provided in the Methods section.

To verify the EQE simulation, a comparison of the simulated and experimental EQE of discrete OPDs for a limited number of different Cu line width and pitch values was performed (Fig. 3a). A bottom electrode receiving light intensity (Ip) of 1.05 mW cm−2 at wavelength (λ) of 850 nm and applied voltage (V) of −2 V was used. The EQE simulations generally reproduced the experimental results very well, with a maximum difference of 1%. Figure 3a presents all the simulation results, varying the Cu line pitch between 2 and 80 µm for line widths of 1 and 2 µm. An optimal pitch can be derived for each line width. A pitch of 10–20 µm in combination with a line width of 1 µm has been extracted as a design rule. Supplementary Fig. 1 provides more details about the EQE simulations.

Fig. 3: Optimization of discrete OPD and OPD array design for touchless user interfaces by electro-optical modelling.
figure 3

a, EQE simulations of discrete OPDs (filled circles) and experimental results (open squares) versus Cu line pitch for a Cu line width of 1 µm (blue symbols) and 2 µm (grey symbols). b, Optical transmittance versus wavelength for the patterned OPD (OPD with patterned subpixels, green), non-patterned OPD (blue) and simulated optical transmittance for the patterned OPD (black line). c, VLT versus OPD subpixel pitch for a Cu grid with a pitch of 10 µm (green circles) and 20 µm (red circles), and the Cu line width is 1 µm in both cases; the total active device area versus OPD subpixel pitch (blue circles) is also shown. To increase the reliability of the large-area OPD patterning process, the overlap (ROPD) with the edge cover layer for the alignment tolerance was enlarged to 10 µm, resulting in a total OPD subpixel size of 70 × 70 µm2 and keeping the same 50 × 50 µm2 OPD subpixel active area in this calculation.

Full size image

An optical microscopy image of part of the parallel OPD subpixel array is shown in Fig. 2g. The fill factor of the OPD subpixel array is a trade-off between the VLT and pixel response. To design an optimal parallel OPD subpixel array of the individually readable pixels in the 16 × 16 OPD array, we built an optical transmittance prediction model. The parallel OPD subpixel array was modelled by dividing it into three simplified components: (1) a photoactive stack without the Cu lines; (2) a non-photoactive stack, that is, the open area of the parallel array without the Cu lines; and (3) the printed Cu line. Supplementary Fig. 2 provides details of the geometry. The VLT of the parallel OPD subpixel array was calculated by the summation of the simulated optical transmittance for each component using Setfos 5.2 (Supplementary Fig. 3a), weighted by their fill factors. Details of the optical transmittance prediction model are provided in the Methods section. The experimental and calculated overall optical transmittance of the parallel OPD subpixel array as well as a non-patterned (solid photoactive layer) OPD are shown in Fig. 3b. Their VLT values were calculated from the overall optical transmittance in accordance with ISO 9050:2003 (ref. 42). As input parameters, an OPD subpixel active area (WOPD) of 50 × 50 µm2, pitch (POPD) of 240 µm and a Cu line (WCu, 1 µm; PCu, 10 µm; thickness, 100 nm) were used. Each subpixel has a 5 µm overlap (ROPD) with the edge cover layer, resulting in a total OPD subpixel size of 60 × 60 µm2. The VLT of the patterned parallel OPD subpixel array was 64%, a substantial increase compared with the non-patterned OPD for which a value of 17% was calculated. Overall, the calculated optical transmittance appropriately reproduced the experimental spectrum and its VLT of 64% (Fig. 3b). Figure 3c presents all the simulation results of the VLT and total active device area, varying the POPD between 70 and 400 µm for two optimal Cu grid designs (WCu = 1 µm and PCu = 10 µm (green circles); WCu = 1 µm and PCu = 20 µm (red circles)). The VLT reaches a plateau for both Cu grid designs at an OPD subpixel pitch of 240 µm with values of 64% (WCu = 1 µm and PCu = 10 µm) and 70% (WCu = 1 µm and PCu = 20 µm). The total active device area decreases inverse quadratically with increasing POPD, whereas the photocurrent scales linearly with the total active device area (Supplementary Fig. 4). Based on these results, we selected a Cu grid with WCu = 1 µm and PCu = 20 µm and an OPD subpixel pitch of 240 µm, thereby maximizing the VLT, EQE and total active device area. The resulting VLT of ~70% is suitable for a touchless user interface integrated on top of low-end displays.

Discrete NIR-sensitive OPD

We used discrete NIR-sensitive OPDs with the optimal Cu grid and OPD subpixel design to measure the OPD performance. These patterned NIR-sensitive OPDs consisted of a 9 × 9 OPD subpixel array with a 50 × 50 µm2 subpixel active area and a subpixel pitch of 240 µm in a 4.0000 mm2 OPD main pixel, resulting in a total active device area of 0.2025 mm2. The printed Cu grid TCE used in the discrete OPDs had WCu = 1 μm and PCu = 20 μm (Supplementary Table 2).

The current density–voltage (J–V) characteristics, recorded in dark and under illumination at a wavelength of 850 nm, are shown in Fig. 4a. Similar results were obtained for non-patterned OPDs, indicating that the performance is not influenced by the OPD patterning process. Even at a very low scan rate of 5 mV s−1, displacement currents are non-negligible at low device currents, as indicated by a non-zero Jdark at 0 V and hysteretic effects. These phenomena occur in the non-patterned OPD, too (Supplementary Fig. 5). Figure 4a (inset) presents a J–V curve that has been reconstructed from static Jdark measurements at different discrete biases, a more reliable way of determining Jdark. Clearly, the hysteretic effects are absent in this case. The reverse-bias dark current (Jdark) at –2 V is 1.8 × 10−6 mA cm−2, which compares favourably with previously reported NIR-sensitive OPDs43,44,45,46,47,48,49. The photocurrent density (Jphoto) at –2 V under illumination with a light intensity of 0.28 mW cm−2 at a wavelength of 850 nm is 8 × 10−2 mA cm−2, which is more than four orders of magnitude larger than Jdark.

Fig. 4: Characteristics of discrete NIR-sensitive OPDs.
figure 4

a, J–V sweep measured at a scan rate of 5 mV s−1 in dark (blue) and under light (850 nm with a light intensity of 0.28 mW cm−2; grey). The scan direction is indicated with the arrowheads. The inset shows the J–V sweep reconstructed from static measurements at discrete biases. b, Linearity plot measured at −2 V, showing the photocurrent density Jphoto minus the dark current density Jdark as a function of light intensity from 200 nW cm−2 to 0.8 mW cm−2. c, EQE as a function of wavelength measured at −2 V. d, Spectral responsivity (SR; closed circles) and detectivity D* (open circles) measured at −2 V.

Full size image

Figure 4b shows a linearity plot of the photoresponse of the OPD to NIR-light illumination (λ = 850 nm). The close-to-linear intensity dependence (Jphoto ≈ Ipα, where α = 0.93) allows the incident light intensity to be directly represented by the photocurrent within the measurement range of 200 nW cm−2 to 0.8 mW cm−2. The linearity is beneficial for a touchless user interface application because the photocurrent can be used for both 2D control (that is, in-plane direction of a display) and depth control (that is, out-of-plane direction of a display).

The EQE and spectral responsivity (SR) as a function of wavelength at a reverse bias of –2 V are depicted in Fig. 4c,d, respectively. The EQE of the OPD in the NIR region is between 36% (at 850 nm) and 25% (at 940 nm), and then decreases towards the onset at 1,050 nm. This corresponds to SR values of 0.25 and 0.19 A W−1 at 850 and 940 nm, respectively. The EQE value at 850 nm—the wavelength showing the highest degree of reflectance of human skin in the NIR region (30–70%) (refs. 9,10)—is amongst the highest reported in the literature for NIR-sensitive OPDs43,44,45,46,47,48,49. Detectivity D* (Fig. 4d) is calculated using D* = SR × A1/2 × IN−1, where A is the total active device area (0.2025 mm2) and IN is the noise current density50. Here IN was measured at –2 V at frequencies (f) between 1 and 50 Hz (Supplementary Fig. 6). Below approximately 10 Hz, a 1/f behaviour is observed. Above 10 Hz, the noise current was constant at ~10 fA Hz1/2. Taking this value together with the SR of 0.25 A W−1, the specific detectivity at 850 nm and –2 V is 1.1 × 1012 Jones (1 Jones = 1 cm Hz1/2 W−1). We also calculated the detectivity using D* = SR × (2qJdark)−1/2, as this allows an easy comparison with the detectivity values reported in the literature. We obtained a value of 7.2 × 1012 Jones (Supplementary Fig. 7). This value is a factor of seven higher than the noise-based detectivity value, illustrating that the dark-current-based methodology typically overestimates the device detectivity and should preferably not be used. It ranks the detectivity of our NIR-sensitive OPD amongst the highest-ever reported in the literature in the 800–900 nm range43,44,45,46,47,48,49.

The NIR-sensitive OPD is protected from the ambient conditions using a laminated barrier film to ensure a long lifetime. Supplementary Fig. 8 shows the J–V sweeps of the NIR-sensitive OPD, after fabrication and after six months of storage under normal (indoor) conditions. The dark current density values measured at −2 V remain constant, even after six-month storage.

NIR-sensitive OPD array fabrication and characterization

To realize a visually transparent NIR-sensitive 16 × 16 OPD array, we implemented a 14 × 14 parallel OPD subpixel array for every OPD main pixel in the 16 × 16 array. The active area of each OPD subpixel is 50 × 50 µm2 and the OPD subpixels have a pitch of 240 µm. The total photoactive area for each main pixel is 0.49 mm2 (Supplementary Fig. 9). The pixels have a pitch of 6.24 mm, that is, 4.2 pixels per inch (ppi). This is a comparable resolution to commercial projected capacitive touch panels51, leading to a total sensor area of 9.7 × 9.7 cm2. A readout integrated circuit is placed on the bonding pads to electrically connect each pixel individually. An optically transparent barrier film was laminated on top of the sensor area to avoid degradation of the device by moisture. Details of the step-by-step fabrication are provided in the Methods section. Figure 5a displays a photograph of the OPD array placed in front of a laptop display, clearly showing the high VLT (~70%) that hardly impairs the visibility of the display.

Fig. 5: Visually transparent NIR-sensitive OPD array fabrication and characterization.
figure 5

a, Photograph of the visually transparent NIR-sensitive OPD array held in front of a laptop display. b, Pixel current density as a function of light intensity (850 nm) of the visually transparent NIR-sensitive OPD array at –2 V. c, Histogram of the pixel current density of the visually transparent NIR-sensitive OPD array in the dark (grey) and under an 850 nm NIR light with an intensity of ~45 µW cm−2 (blue) at –2 V. The dark response was offset corrected. d, Transient photocurrent response of the OPD array at –2 V on 850 nm light pulses of 50 ms duration; the inset shows the transient photocurrent response of the OPD array at –2 V on 850 nm light pulses of 1 ms duration (1 kHz). Here 37% of the maximum signal is indicated in green.

Full size image

The image sensor was characterized by measuring the linearity of the photoresponse as well as the uniformity of the dark response and photoresponse over the whole pixel array. First, the current density was measured as a function of light intensity of 850 nm NIR light at –2 V (Fig. 5b). The photocurrent density scales linearly with the light intensities up to 90 µW cm−2 (R2 = 0.99), with current densities ranging from 0.3 × 10−2 to 2.7 × 10−2 mA cm−2 for light intensities ranging from 10 to 90 µW cm−2, respectively, after which the current density starts to saturate due to the limitations of the readout speed and maximum charge that can be stored in the pixel. The threshold of this saturation can be increased by increasing the bias voltage and changing the readout setting of the peripheral electronics. Next, the pixel current density of the full OPD array (16 × 16 pixels) was measured both in dark and under NIR illumination conditions in the linear range (~45 µW cm−2, 850 nm) at –2 V (Fig. 5c). Both dark and light responses show a narrow Gaussian distribution with current densities of 3.7 ± 2.2 × 10−6 and 1.5 ± 0.3 × 10−2 mA cm−2, respectively.

Next, we measured the transient photocurrent response of the OPD array at −2 V on 850 nm light pulses at a light intensity of 46 µW cm−2 of 50 ms duration. The rise and fall times, defined as the times it takes to reach 90% and to drop to 10% of the steady-state values, are 2.4 and 2.8 ms, respectively (Fig. 5d).

Touchless user interface demo using NIR-emitting penlight

As a first demonstration of our technology, we show a touchless user interface demo using an NIR-emitting penlight (Fig. 6a–d). The transparent imager was put in front of a 160 ppi laptop display (Fig. 6a). Given the high VLT of the imager (~70%), the visibility of the display is barely harmed.

Fig. 6: Still images taken from videos of touchless user interface demos.
figure 6

a–d, Touchless user interface demo using NIR-emitting penlight. e–h, Touchless user interface demo using gesture recognition. a, Imager in front of a 160 ppi laptop display. b, Panning the screen by moving the penlight. c, Zooming in by moving the penlight towards the screen. d, Zooming out by moving the penlight away from the screen. e, Imager with an NIR LED array in front of a 160 ppi laptop display. f, Entering a pin code by a click gesture. g, Choosing the amount of cash by a click gesture. h, Collecting cash.

Full size image

A battery-driven NIR-emitting penlight containing two control buttons was custom designed. The NIR-emitting penlight generates three different frequencies of 800, 960 and 1,200 Hz corresponding to a right click, left click and idle state, respectively. The left click is used to pan a screen. The right click is used to initiate zooming so that moving the penlight closer towards or away from the screen results in a zooming-in or zooming-out action. The photoresponse signals from all the pixels are bandpass filtered to suppress the background noise resulting from ambient conditions and display light. An x signal is determined by the summation of all the pixel signals on that column. A y signal is determined by the summation of all the pixel signals on that row. The resulting x and y coordinates are computed by determining the centre of a Gaussian fit performed on the respective signals (Supplementary Fig. 10). The standard deviation, that is, a measure for the spot size, has a linear relationship with the distance between the penlight and imager when panning the screen.

We analysed the image sensor response in the frequency range as used in the touchless user interface demo, which allows (1) narrow bandpass filtering to sufficiently suppress the background noise from ambient conditions and display light and (2) a sufficiently high pointer position update rate that allows a smooth touchless user interaction. A frequency of 1 kHz was selected as a compromise between filtering performance and image sensor dynamic response. Figure 5d (inset) shows the image sensor response at a frequency of 1 kHz, resulting in 37% of the maximum signal (indicated in green) and hence a loss of 8.6 dB. However, the mean value of the bandpass-filtered signals of all the 256 pixels still gives a decent signal-to-noise ratio (SNR) of 54 dB (Supplementary Fig. 11a). The remaining 54 dB available still provides sufficient SNR for a position accuracy of 20 µm, that is, less than a quarter of a pixel in the commercial 160 ppi laptop display used in this work (Supplementary Fig. 11b).

Supplementary Fig. 13 shows the NIR power density (in microwatts per square centimetre) measured at the image sensor versus distance of the NIR-emitting penlight. Supplementary Video 1 shows a video of the use of the NIR-emitting penlight together with the transparent image sensor in front of a laptop display as a touchless user interface for Google Earth.

Touchless user interface demo using gesture recognition

Next, we demonstrate a touchless user interface using gesture recognition (Fig. 6e–h). In this case, NIR-emitting light-emitting diodes (LEDs) are integrated around the circumference of the imager. This assembly is again put in front of a 160 ppi laptop display (Fig. 6e). Gestures are imaged by capturing the reflected NIR light of a finger that is illuminated by NIR-emitting LEDs. The concept is schematically visualized in Fig. 1e. The position information of the finger is extracted similar to the penlight demo. Position detection of a complete hand would require more advanced algorithms and most probably more advanced illumination and optics, but we consider this to be outside the scope of the current work. The variation in light intensity resulting from a change in distance from a finger to the imager is used to detect a click gesture. SNR and position accuracy of the gesture demo were determined in a similar way as described for the penlight demo. In this case, an SNR of 56 dB is obtained (Supplementary Fig. 12a), resulting in a position accuracy of 0.65 mm, that is, four pixels in the commercial 160 ppi laptop display used in this work (Supplementary Fig. 12b). Supplementary Fig. 14 shows the NIR power density (in microwatts per square centimetre) measured at the image sensor versus the distance of the finger.

Supplementary Video 2 shows a demonstration of a gesture-controlled ATM screen using the transparent image sensor in front of a laptop display.

Conclusions

We have reported a solution-processed, large-area, visually transparent NIR-sensitive OPD array that can be used on top of commercial displays for touchless user interface applications. Good transparency was achieved by using a printed Cu grid TCE with a line width of 1 µm and a patterned NIR-sensitive OPD based on a parallel array of 196 patterned 50-µm-sized OPD subpixels. EQE simulations were used to determine the design of our Cu grid, resulting in an EQE value of 36% at 850 nm. Our discrete subpixels have a low dark current density of approximately 10−6 mA cm−2 and detectivity of approximately 1012 Jones at 850 nm. The 16 × 16 image sensor design was optimized using optical modelling of the parallel OPD subpixel array to achieve a VLT of around 70% and a maximized total active device area and thus a high signal output. We showed that our transparent NIR-sensitive image sensor can be used in both penlight-controlled and gesture-controlled touchless user interface when integrated in front of a commercial display. The approach is based on scalable flat-panel-display-compatible fabrication processes, which will help facilitate adoption. Our technology should be applicable to a range of display applications—including ATMs, electric signage and interactive whiteboards—without size limitation and calibration requirements.

Methods

Fabrication of NIR-sensitive OPDs

Discrete OPDs and OPD arrays were made using the same fabrication method. Printed Cu grids were deposited on glass substrates (1.1 mm, EAGLE XG) by Asahi Kasei’s high-resolution printing technology52 to form the pixelated transparent bottom electrode. Next, a thin film (16 nm) of a-IGZO was sputtered and patterned by wet etching on the bottom electrode as an electron transport layer as well as a hole blocking layer. An edge cover layer of an SU8 resist was deposited and photolithographically structured, preventing shorts between the bottom and top electrodes as well as defining an OPD subpixel active area. Supplementary Fig. 15 shows the three-dimensional topology of the OPD subpixel active area by cross-section scanning electron microscopy photographs of the SU8 edge cover layer and IGZO electron transport layer on the Cu grid bottom electrode. The 300-nm-thick photoactive layer is based on a bulk heterojunction structure consisting of a p-type donor polymer, PCE-10 (PTB7-Th, purchased from 1-Material), and n-type non-fullerene acceptor small molecule, namely, IEICO-4F (purchased from 1-Material), in a 2:3 weight ratio. The optical bandgap, highest occupied molecular orbital and lowest unoccupied molecular orbital values of these materials are 1.60, −5.20 and −3.59 eV for PCE-10 (ref. 35) and 1.24, −5.44 and −4.19 eV for IEICO-4F (ref. 36), respectively. The effective bandgap, that is, the difference between the highest occupied molecular orbital of the donor and the lowest unoccupied molecular orbital of the acceptor, is 1.01 eV. PCE-10 and IEICO-4F were dissolved in chlorobenzene at 20 mg ml−1 with 40 µl chloronaphthalene, added to optimize the nanomorphology of the photoactive layer53. The PCE-10:IEICO-4F OPD blend was slot-die coated under ambient conditions and subsequently annealed at 60 °C for 5 min under ambient conditions. The photoactive layer was patterned using photolithography in a similar way as that in other works37,38 to make the parallel OPD subpixel array structure. As a hole transport layer, a 60-nm-thick MoOx layer was deposited by thermal evaporation, followed by a sputtered ITO thin transparent electrode (~100 nm thickness; sheet resistance, ~40 Ω ▫−1). The low Jdark value of the OPDs is attributed to the charge-blocking properties of a-IGZO (refs. 54,55) and MoOx under reverse bias, whereas the low effective bandgap (1.01 eV) enhances bulk thermal charge generation and charge injection56. The OPD devices were protected from oxygen and moisture by an optically transparent laminated barrier film. The laminated barrier film was a multilayer stack that has a low-temperature plasma-deposited amorphous hydrogenated silicon nitride (a-SiNx:H) layer and an organic layer between a polyethylene terephthalate substrate (125 µm, Melinex ST504) and a barrier adhesive layer. The total thickness of the laminated barrier film was 160 µm.

Characterization of discrete OPDs

Discrete OPDs were characterized in a glovebox under a N2 atmosphere and at ambient conditions. The J–V characteristics in dark and under NIR-light conditions were measured using a semiconductor parameter analyser (Agilent 4155C) with a manual probe station and in-house OPD measurement setup. The OPDs were illuminated from the bottom through the Cu grid TCE with a NIR LED light source (850 nm wavelength; light intensity of ~0.28 mW cm−2; 15414185BA210 from Würth Elektronik). The voltage was swept from –3 to 2 V using a scan speed of 5 mV s−1. Static J–V characteristics were measured using the same setup by setting a fixed voltage and measuring the current in time. The EQE was measured using a custom-made setup consisting of the following: a tungsten–halogen lamp, a chopper, a monochromator (Oriel, Cornerstone 130), a pre-amplifier (Stanford Research Systems SR570) and a lock-in amplifier (Stanford Research Systems SR830 DSP). Although the setup was in ambient air, the devices were constantly kept sealed in a N2-filled box equipped with a quartz window. For this measurement, a circular aperture with a diameter of 1 mm was used to define the active area. To convert the current signal from the device into an EQE value, a comparison was made with a reference calibrated silicon solar cell. In the range of wavelengths from 350 to 1,050 nm, the standard deviation of this setup is less than 0.005 electrons per photon. Noise measurements were performed at room temperature and in dark conditions, exploiting a battery-powered current-to-voltage conversion readout circuit developed with off-the-shelf components. The OPD is first connected by means of two probes and triaxial cables to a transimpedance amplifier (TIA) implemented with the operation amplifier (Analog Devices ADA4530). An adjustable d.c. voltage source is applied to the non-inverting terminal of the TIA to modify the biasing of the device under test. The feedback network of the TIA is designed with a 1 GΩ resistor and a 1 pF compensation capacitor, required for the stability of the circuit. The input-referred noise of the TIA is dominated by the Johnson–Nyquist noise component associated with the feedback resistor. Next, the output of the TIA is fed to an active bandpass amplifier, realized with an operational amplifier (Analog Devices AD8065) in the closed-loop configuration, which exhibits an in-band voltage gain of 100 V V–1. The 3 dB bandwidth of the conditioning chain is approximately limited to the frequency range of 0.1–100.0 Hz. Finally, the output of the readout chain is connected to an HP35670A dynamic signal analyser to extract the noise spectral density of the OPD. The optical characteristics of the discrete OPDs were measured by ultraviolet–visible–NIR spectroscopy (Agilent Cary 5000) in the wavelength range from 300 to 1,200 nm with a step of 1 nm.

EQE simulations for NIR-sensitive OPDs

EQE simulations for NIR-sensitive OPDs with a printed Cu grid TCE were carried out by coupling the input of the simulated J–V curves of the NIR-sensitive OPD stacks on the Cu line and in a gap area between the Cu lines based on a numerical electro-optical simulation and the output of the surface potential and current density distributions derived from the printed Cu grid geometrical structure solved by 2D FEM.

First, the numerical electro-optical simulations for the photogenerated J–V curves of our NIR-sensitive OPDs were performed by using Setfos 5.2 (FLUXiM). The numerical electro-optical simulations are performed by coupling between the input of optical properties based on the transfer matrix method57 and the output of the electric characteristics calculated by the numerical drift–diffusion simulation58. Specifically, a simulated photon absorption profile in the OPD stack is used to determine charge generation (electron–hole pair) in absorbing layers and the generated charges are distributed across the OPD stack, leading to electric current between both electrodes. For the numerical electro-optical simulations of the NIR-sensitive OPDs, we modelled two types of OPD stack, one is for the OPD stack on the printed Cu line and the other is for the OPD stack in the gap area (Supplementary Fig. 16). For the optical simulations of OPD stacks, wavelength-dependent refractive index (n) and extinction coefficient (k) of each layer and their thicknesses were used as the input parameters. The n–k profiles as a function of wavelength of each layer were determined by spectroscopic ellipsometry (Supplementary Fig. 17), except for the printed Cu lines and the polyethylene terephthalate substrate of the transparent laminated barrier film. For the Cu lines, the n–k profile was derived from the layer structure of the printed Cu line. For the polyethylene terephthalate substrate, the n–k profile was retrieved from an available database59. The thicknesses of all the layers were identical to those used during the fabrication of the NIR-sensitive OPD. For numerical drift–diffusion simulations of the OPD stacks, we modelled them with four elements: the printed Cu grid TCF for the OPD stack on the printed Cu line or a quasi-transparent electrode for the OPD stack in the gap area (Supplementary Fig. 16), a-IGZO, the photoactive layer and MoOx/ITO thin transparent electrode. The simulations were carried out using input parameters and boundary conditions of both electrodes (Supplementary Table 3). The work function of the quasi-transparent electrode was set to the same value as the printed Cu grid TCE. In the numerical electro-optical simulations, the OPD stacks were modelled upside down compared with the experimental inverted NIR-sensitive OPDs, and the polarity of an applied voltage is also inverted for implementing the simulated J–V curves for each OPD stack in subsequent 2D FEM simulations. The applied voltage was swept from –1.0 to 2.5 V with a step size of 5 mV, and 850 nm NIR light with a light intensity of 1.05 mW cm–2, which was matched with the light intensity at 850 nm in the EQE measurement, was illuminated from the printed Cu grid TCE in the simulations. The resulting simulated J–V curves that are coupled with the simulated absorption rate profiles (Supplementary Fig. 18a,b) for each OPD stack are shown in Supplementary Fig. 18c,d.

Next, the 2D FEM simulations for the surface potential and the current density distributions derived from the printed Cu grid structure for our NIR-sensitive OPDs were performed by Laoss 4.0 (FLUXiM). The NIR-sensitive OPDs were approximated as a two-dimensional:one-dimensional:two-dimensional system, which combined 2D FEM electrodes (‘top electrode in the simulation software’, a-IGZO/printed Cu grid TCE; ‘bottom electrode in the simulation software’, ITO thin transparent electrode) with the J–V curves simulated by Setfos for the aforementioned one-dimensional OPD stack models. These simulated J–V curves were set on the corresponding area on the 2D FEM electrodes. The 2D FEM solved Ohm’s law for the local current density in the electrodes, which were coupled with a given small-area wavelength-dependent J–V curve at a given point for charge conservation. Supplementary Table 4 lists the input parameters for the 2D FEM simulations. The simulations were done for 3 × 3 unit cells of the printed Cu grid TCE. Meshing conditions were set so that the mesh edge size was half or less than half of the Cu line width. Boundary conditions were set so that the voltage on the four edges of the top electrode (a-IGZO/printed Cu grid TCE) was at the applied voltage of 2 V (corresponding to –2 V for the experimental inverted OPD stack) and the voltage on the bottom electrode (ITO thin transparent electrode) was 0 V. The solving parameters were set using a Newton solver with convergence parameters of the relative residual convergence type, L2 norm, a tolerance of 1 × 10–7 and a maximum iteration count of 10. These were kept constant in all the simulations in this work. An example of the simulated surface potential and current density distributions of the system are shown in Supplementary Fig. 1a,b. Based on the calculated current density distribution, the EQE value is calculated with the following formula:

$${\mathrm{EQE}}\left(\lambda \right)=\frac{{J}_{\mathrm{s}}(\lambda )}{{I}_{\mathrm{P}}(\lambda )}\times \frac{{hc}}{\lambda q},$$

where Js(λ) is the global current density derived from the current density distribution across the whole studied system and IP(λ) is the illuminated light intensity that was set to the same value used in the J–V curve simulations performed in Setfos. Here h, c, λ and q are the Plank constant, speed of light, wavelength (850 nm) and elementary charge, respectively.

Optical transmittance prediction model for patterned OPDs

A parallel OPD subpixel array was modelled by dividing into three simplified components: (1) a photoactive stack without the Cu lines; (2) a non-photoactive stack (that is, an open area of the parallel array) without the Cu lines; and (3) the printed Cu line (Supplementary Fig. 2). The overall optical transmittance (T(λ)) of the parallel OPD subpixel array was calculated by the following equation:

$$\begin{array}{ll}T\left(\lambda \right)&=\left\{\frac{{\left({W}_{{\mathrm{OPD}}}+2{R}_{{\mathrm{OPD}}}\right)}^{2}}{{P}_{{\mathrm{OPD}}}^{2}}\times {T}_{{\mathrm{OPD}}}\left(\lambda \right)+\left(1-\frac{{\left({W}_{{\mathrm{OPD}}}+2{R}_{{\mathrm{OPD}}}\right)}^{2}}{{P}_{{\mathrm{OPD}}}^{2}}\right)\times {T}_{{\mathrm{open}}}\left(\lambda \right)\right\}\\&\times \left\{\left(1-\frac{{\left({P}_{{\mathrm{Cu}}}-{W}_{{\mathrm{Cu}}}\right)}^{2}}{{P}_{{\mathrm{Cu}}}^{2}}\right)\times {T}_{{\mathrm{Cu}}}\left(\lambda \right)+\frac{{\left({P}_{{\mathrm{Cu}}}-{W}_{{\mathrm{Cu}}}\right)}^{2}}{{P}_{{\mathrm{Cu}}}^{2}}\right\},\end{array}$$

where WOPD, ROPD and POPD represent the OPD subpixel width, OPD subpixel overlap with the edge cover layer for alignment tolerance and OPD subpixel pitch, respectively (Supplementary Fig. 2b). Here WCu and PCu are the line width and pitch of the printed Cu grid TCE, respectively. These geometric variables are used for calculating the fill factors of the photoactive stack and printed Cu lines. Also, TOPD(λ), Topen(λ) and TCu(λ) are the simulated optical transmittance of the photoactive stack without Cu lines, non-photoactive stack without Cu lines and printed Cu line, respectively (Supplementary Fig. 3a). These optical simulations were performed by Setfos 5.2 (FLUXiM) in the same manner as that described in the ‘EQE simulations for NIR-sensitive OPDs’ section. For the SU8 edge cover layer, the n–k profile was retrieved from the datasheet60.

Touchless user interface demo characterization

Photocurrent measurements as a function of light intensity were performed and the 16 × 16 OPD array was illuminated with a 10 × 10 cm2 LED tile emitting at a wavelength of 850 ± 15 nm (Phlox). The LED tile was driven using a voltage source (TTi EL302R power supply). The light intensity was measured using a calibrated photodiode (FDS1010-CAL, Thorlabs). The OPD response was measured using a custom-made electronic system and software (LabVIEW based). A silicon readout integrated circuit (Analog Devices AD71124) is routed to and collects all the data from the 256 pixels. The image sensor is biased at −2 V using a custom-made board and connected to a field-programmable gate array digital interface that reads the data. The field-programmable gate array interface is connected to a personal computer through a USB connection. To use the prototype in typical indoor office settings, that is, in the presence of visible light, we have employed a bandpass filter. Consequently, we can distinguish between the static background and dynamic signal that we are interested in by electrically filtering out the static background.

For the NIR-emitting penlight demo, a custom-made battery-driven NIR-emitting penlight was built using an LED with a peak emission at 850 nm wavelength (TSHG6200, Vishay). The NIR-emitting penlight has two buttons using which periodic pulse train signals at two different frequencies of 800 and 960 Hz are generated when the respective buttons are pressed to activate a right and left click, respectively. An idle frequency of 1,200 Hz is generated when no buttons are active. The left click is used to pan a screen. The right click is used to initiate zooming in so that moving the penlight closer towards or away from the screen during the right click results in a zooming-in or zooming-out action.

The SNR and position accuracy of the penlight touchless user interface were determined by illuminating the centre of the image sensor with the penlight at a distance of 60 mm inside a light-tight cabinet. The bandpass-filtered pixel signals have been averaged and 1,024 samples were saved. A subset of 100 pixel signals (Supplementary Fig. 11a), representing 2.5 s, was used to determine the standard deviation of the amplitude, resulting in the SNR of the image sensor.

The x position resulting from the Gaussian fit used to calculate the position in the software is plotted versus time. The bandpass-filtered pixel signals have been averaged and 1,024 samples were saved. A subset of 100 pixel signals (Supplementary Fig. 11b), representing 2.5 s, was used to calculate the peak-to-peak error and position accuracy. The range of the position signal is from 0 to 1 and corresponds to a distance of 100 mm.

For the gesture recognition demo, a printed circuit board was made containing a total of 40 LEDs with a peak emission of 850 nm (15411085A4570, Würth Elektronik) and a total power of 0.35 W that are equally arranged at the four sides of the printed circuit board. An LED modulation frequency of 1 kHz was used to filter the background noise. The SNR and position accuracy of the gesture-based touchless user interface were determined in a similar way as described for the penlight demo (Supplementary Fig. 12).