Introduction

Interest in virtual interactions is rapidly increasing in a variety of fields, including education, healthcare, retail, robotics and manufacturing1,2,3,4,5,6,7,8,9, as virtual environments can provide users with indirect but realistic experiences through low-cost, high accessibility and contactless interactions6,10. Existing virtual/augmented reality (VR/AR) systems mainly rely on visual and auditory stimuli to enhance human interaction. Providing tactile feedback to this system can further improve the immersion and accuracy of virtual interactions11,12,13,14. However, the tactile feedback system is at a preliminary stage compared to audio and video technologies in terms of a high-quality reproduction of original sensation. Further improvements in device performance such as spatiotemporal resolution, multimodality, and real-time communication are required to enable high fidelity and sophisticated tangible interactions in remote and/or virtual locations. Moreover, the tactile feedback platform satisfied by the above characteristics is referred to as a telehaptic system.

The ultimate goal of the telehaptic system is to transmit sophisticated and near-realistic sense of touch from one user to another. The core capability of the telehaptic transducer depends on precisely monitoring the spatiotemporal and physical deformations occurring on soft skin surfaces and reproducing them from a distance. Tactile sensor technology developed to date is in a fairly mature stage and satisfies high sensitivity, spatial resolution, fast response speed, wide range of detection for low-frequency pressure, and high-frequency vibration, even thermal stimulation15,16,17,18,19,20,21,22, which are required to detect sophisticated tactile sensation. On the other hand, the actuator technology required to reproduce the sense of touch has been mainly developed in a limited way to improve output performance such as high acceleration and displacement, and few studies have been reported so far to reproduce a sophisticated tactile sensation based on high spatial resolution and wide frequency range characteristics. In order to enable sophisticated tactile communication, actuators should be also developed to meet all these performances, like sensors. In addition, these telehaptic systems must satisfy not only the collection/reproduction performance of sophisticated tactile information as mentioned above, but also the functional needs to minimize feeling of irritation and discomfort when they are worn or attached. In other words, sensors and actuators should be designed to have mechanical flexibility and skin conformability so that tactile stimuli can be detected or reproduced without interfering with the function of natural hands, and tactile signals can be transmitted and received remotely with untethered design.

Until now, tactile actuators have been developed using various materials and principles to satisfy advanced performance, high mechanical flexibility and high-resolution array structures. For example, electromagnetic type actuators, soft actuators using electroactive polymers (EAP)23,24,25,26,27, shape memory polymers(SMP)28, and pneumatic/hydraulic pressure28,29,30, and piezoelectric type actuators have been utilized for achieving the actuating performance and mechanical flexibility (Supplementary Table 1).

Electromagnetic type actuators are advantageous in generating large displacement, but it is difficult to achieve high spatial resolution of several millimeters because they require permanent magnets or coils that occupy a volume. Recently, miniaturized coil and magnet-based 32 actuators have been embedded in silicone elastomer and wirelessly operated by near-field communication (NFC), enabling untethered and skin-conformable designs3. However, since the pixel size of these devices is of the order of centimeters, it is difficult to apply them to narrow and sensitive areas such as fingers and fingertips.

Soft actuators using EAP, SMP materials and pneumatic/hydraulic pressure system have advantage of intrinsic mechanical flexibility, and output performance related to high displacement and force. Moreover, a millimeter-level high resolution has been achieved by microfabrication technology. However, these soft actuators have disadvantages of relatively slow response speed and inability to express high-frequency vibrations, and require high operating voltage or large-scale air compression equipment, making it difficult to implement flexible, compact, and untethered telehaptic systems. For example, flexible hydraulically amplified electrostatic actuators for wearable haptics have been reported27, which enabled soft actuator with a 6 mm pitch resolution and out-of-plane displacements of 500 μm. However, the response time is around 5 ms, which make difficult to express high-frequency vibrations larger than 200 Hz. A 4 × 4 pneumatic tactile display with extremely high spatial resolution (2.5 mm) using a bistable electroactive polymer (BSEP) thin film has also been reported29. However, the device is operated using joule heating to control the stiffness of membrane and pneumatic pressure to generate displacements, which shows a limit to its actuating speed due to the slow hydraulic, pneumatic flow and cooling time for heat dissipation.

Piezoelectric actuators are an attractive candidate technology that can generate high spatiotemporal tactile patterns with low operational burden. Piezoelectric actuators are easy to miniaturize and have a wide frequency range of up to 10 kHz, which are advantageous in generating high-resolution, clear tactile responses required for emulated buttons and textures30. However, the most commonly used piezoceramic actuators have limitations in their flat and rigid form factors, which are disadvantageous for on-skin applications. To overcome these limitations, ultra-thin Pb[ZrxTi1-x]O3 (PZT) membranes were used as mechanical sensors and actuators with high spatial resolution31. These thin and flexible PZT membranes were effective in detecting modulus in medical applications, but has a critical drawback of insufficient output performance for abundant tactile reproduction (Supplementary Table 2). Therefore, developing an actuator that simultaneously satisfies performance such as output force/displacement sufficient for tactile reproduction, fast response speed and wide frequency range playback, and high spatial resolution and high mechanical flexibility comparable to tactile sensors is a key task in implementing a telehaptic system that immersively reproduces sophisticated tactile information.

Here, we demonstrate a skin-attachable, untethered telehaptic system that consists of an ultra-flexible bimodal (UFB) tactile sensor and a flexible tactile actuator array that can acquire and reproduce tactile stimuli patterns with 1.8 mm pitch high spatial resolution. The ultra-flexible bimodal tactile sensor consists of a fast adaptive (FA) and slow adaptive (SA) mechanoreceptor-like pressure sensor array that can simultaneously monitor static and dynamic pressures with high spatial resolution. The sensor and actuators are both mechanically flexible, skin-conformable and wirelessly connected to each other for providing sophisticated and highly immersive telehaptic interface. The flexible piezoelectric actuator array can generate sinusoidal vibration patterns as well as near-realistic high-resolution tactile stimuli patterns in real-time corresponding to the signals acquired from each sensor pixel at a remote site. Finally, the real-time remote transmission of high-definition tactile stimuli has been demonstrated using the developed 32-channel telehaptic system. Various types of spatiotemporal tactile patterns, including textures of fabrics, and pressing, clicking, and flapping of butterfly wings generated by an anvil and a commercial haptic vibrator device (PowerHapTM, TDK Corp.), were acquired using the high-resolution bimodal sensor array. They were transmitted in real-time to the high-resolution flexible actuator array located 3 m away and then regenerated with high fidelity, as proven by short-time Fourier-transform spectral analysis.

Results and discussion

Design of skin-attachable tactile sensor and actuator array

The telehaptic system is designed to transmit and regenerate tactile stimuli using a wirelessly coupled on-skin tactile sensor and actuator, as shown in Fig. 1. The basic principle of the on-skin telehaptic interface is shown in Fig. 1a. The skin-attached bimodal tactile sensor array measures various tactile stimuli such as spatially distributed normal pressure, impacts, and friction-induced vibrations by using dual mechanism of piezoelectric and piezoresistive effect. Moreover, the surface properties of a target object such as shape and texture can be recognized based on the collected tactile information. The dual mechanism of a bimodal tactile sensor is advantageous for increasing the detection range over a unimodal sensor. A single piezoelectric sensor has difficulty in detecting sustained small static pressure because there is a drift of charge output due to their principle of operation. On the other hand, piezoresistive sensor has delayed response time and hysteresis, which can affect the accuracy of high-frequency vibration detection32,33,34,35. In this work, the spatially distributed complementary bimodal sensor array is integrated in-plane on ultra-thin substrate for collecting various tactile stimuli, including low-frequency pressure and high-frequency vibration from several hertz to order of kilohertz, maintaining conformal contact with the human skin.

Fig. 1: Ultra-flexible bimodal (UFB) tactile sensor and sub-mm tactile actuator array for telehaptic system.
figure 1

a Schematic of telehaptic system composed of tactile acquisition (left) and rendering (right). Physical surface properties including texture and shape are recorded by on-skin tactile sensor array and reproduced by tactile actuator array. b 3D schematic of ultra-flexible bimodal (UFB) tactile sensor array. c Optical microscopy image of the UFB tactile sensor array, scale bar is 2 mm. d Image of the UFB tactile sensor laminated on the fingertip, scale bar is 2 mm. e UFB tactile sensor with a wireless system attached to the hand for tactile acquisition, scale bar is 2 cm. f 3D schematic of flexible sub-mm tactile actuator array. g Image of the ultra-flexible tactile actuator array, scale bar is 2 mm. h an ultra-flexible sub-mm tactile actuator array laminated on the fingertip, scale bar is 5 mm. i tactile actuator array with a wireless system attached to the hand for tactile rendering, scale bar is 2 cm.

Full size image

The bimodal tactile sensor array is designed to have piezoresistive type static pressure sensors and piezoelectric type dynamic pressure sensors arranged in a chessboard pattern densely packed into an area of a centimeter square. There are 32 channels of static pressure sensor with 1.8 mm pitch and 4 channels of dynamic pressure sensor with 4.8 mm pitch on a fingertip area, of which spatial resolution is inspired by the density of human mechanoreceptors, Merkel disks (1–2 mm) and Pacinian corpuscle (5–7 mm)36, respectively. Moreover, these dual monitoring systems play an important role for recording complex tactile signals such as shape and texture by monitoring the spatially distributed pressure and vibrations (Fig. 1b, c and Supplementary Fig. 1).

For the static pressure sensing, Polypyrrole (Ppy) was chemically grafted on elastomeric micropyramids (made with PDMS, polydimethylsiloxane) because of its reproducibility and high sensitivity based on our previous work37,38. Unlike many other flexible conductive materials such as poly(3,4-ethylenedioxythiophene) polystyrene sulfonate (PEDOT:PSS) and indium tin oxide (ITO), Ppy can be covalently grafted by chemically reacting with the hydroxyl groups on the PDMS surface; therefore, durability and reliability of sensor can be effectively enhanced by using Ppy-grafted PDMS37. Also, micro-scale periodic pyramid structures are suitable for miniaturization of sensor pixels, which resulted in individual pixel area of 1 mm2.

The piezoelectric polymer PVDF-TrFE thin film was exploited to monitor the dynamic pressure. 1.2 × 1.2 mm2 piezoelectric pixels were fabricated by spin-coating and reactive ion etching process (Supplementary Fig. 2). Moreover, the optimal annealing temperature has been investigated to improve β-phase crystallinity and piezoelectric characteristics of PVDF-TrFE thin film. A 3 μm-thick PVDF-TrFE thin film was annealed in a vacuum oven at optimal temperature of 130 °C, which was determined based on the Fourier-transform infrared spectroscopy (FT-IR) and X-ray diffraction (XRD) analysis results (Supplementary Fig. 3).

The bimodal sensor pixels are in-plane integrated on an ultra-flexible PI layer (<5 μm) to achieve a higher bending compliance, which is beneficial for making a conformal contact with the skin surface39. Owing to the ultra-flexibility of the sensor array, it can be conformally laminated on a curvilinear fingertip surface, as shown in Fig. 1d, which is beneficial for acquiring reliable tactile information without mechanical distortion at the interface between the skin and the object. The collected tactile information is amplified by the signal processing module located at the back of the hand and then wirelessly transmitted to the actuator array using Bluetooth communication, realizing the untethered operation of the telehaptic system (Fig. 1e).

The actuator array generates vibration patterns for rendering virtual surface properties originated from the delivered tactile information composed of various frequencies, amplitudes, and spatial distributions. The high-resolution tactile actuator array is fabricated by using miniaturized sub-millimeter scale piezoelectric ceramics with high piezoelectric coefficient (475 ± 25 pC N−1 in average) and small pixel size (<1 mm2), and these actuators are assembled into a 32-channel-array on an ultra-flexible polyimide substrate (Fig. 1f, g). The actuators are mounted on an ultra-flexible PI substrate by thermo-compression bonding process using anisotropic conductive film (ACF) (Supplementary Fig. 4), enabling a flexible and skin-conformable actuator array. Therefore, piezoceramic actuators can generate spatially distributed tactile stimuli patterns with high resolution (1.8 mm pitch) over a wide dynamic frequency range (1 Hz–1 kHz). In this way, the tactile actuator array is designed to have mechanical flexibility to directly adhere onto a fingertip and deliver spatially segmented actuation on the curved surface (Fig. 1h). The tactile signals transmitted from the bimodal sensor array are reproduced individually for each pixel of the actuator array via the drive module located on the back of the hand (Fig. 1i), which supplies up to 60 Vpp for each pixel.

Detection of static/dynamic pressure for tactile recording

Figure 2a–c shows the structural schematics and the image of the in-plane integrated tactile sensor array. For the dynamic pressure sensors, the PVDF-TrFE film was pixelated and it detects the dynamic signals generated from piezoelectric effect. The lateral electrode design makes dipole alignment in horizontal direction during the polarization process. As the bimodal tactile sensor is attached to the skin, the external dynamic pressure perpendicular to the dipole direction generates electric charges by the d31 mode piezoelectric effect. The sensor provides a fast response and a wide range of frequency detection when dynamic vibration is applied. It was shown that the rising time of the patterned dynamic pressure sensor is 250 µs as shown in Fig. 2d. The human tactile response time is 385 µs40, which implies that the sensor has a sufficiently fast response and is capable of precisely capturing tactile stimuli. Furthermore, it can detect dynamic vibrations with a sufficient average SNR of 15.4 dB (=10logS N−1) in the frequency range from 1 Hz to 1 kHz, as demonstrated in Supplementary Fig. 5 using a commercial actuator (PowerHapTM, TDK Corp., Japan, peak acceleration 2.5 G @ 100 g mass). The decrease in the output current in the high-frequency range is attributed to the inherent frequency dependence of the vibrator, showing a reduced force with increasing frequency.

Fig. 2: Ultra-flexible bimodal tactile sensor array for tactile acquisition.
figure 2

a schematic of PVDF-TrFE based dynamic pressure sensor structure and SEM surface (top), scale bar is 150 nm and cross-section image (bottom), scale bar is 3 μm, b optical microscopic image of ultra-flexible bimodal tactile sensor array. Scale bar is 500 μm c schematic of static pressure sensor structure. Polypyrrole coated micropyramids come in contact with interdigit electrodes under pressure. d Response time of piezoelectric dynamic pressure sensor, the black line is the input square wave of the vibrator, the blue line is the output current of the dynamic sensor. e optical microscopic images of deformed micropyramid as the applied pressure: 0, 4, and 40 kPa, scale bar is 100 μm. f Characteristics of piezoresistive static pressure sensor showing the change in current and resistance in the pressure range from 3 to 80 kPa. Pressure sensitivity is plotted with dotted line. g schematic of static and dynamic pressure sensing demo. h result of simultaneous monitoring of static (blue) and dynamic (red) pressure while five different areas were stimulated (left bottom: LB, left top: LT, right top: RT, right bottom: RB, center: C). Spatial distribution of static pressure (blue) and dynamic pressure (red) are shown in heatmap. Red shaded circles are the areas stimulated by a 1 x 1 cm commercial vibrator.

Full size image

For the static pressure sensing, the resistance change was monitored by interdigitated electrodes underneath the conductive ppy-grafted micorpyramids. The soft micropyramid structures allow a sensitive change in contact area when pressure is applied to the sensor as shown in Fig. 2e. The contact area between the electrode and the conductive pyramid increases under pressure loading, resulting in a decrease in the contact resistance (Rc). As applied pressure increases from 3 to 80 kPa, the current change (I/I0) increased nearly 10 times from 23.39 to 222.17. The entire sensing range of static pressure sensor is 3–80 kPa, which is the recognizable pressure range of human touch41. The highest value of sensitivity was 13.93 kPa−1 in the low-pressure range of 3–10 kPa, and the sensitivity decreased to 4.65 kPa−1 (10–30 kPa) and 0.32 kPa−1 (30–80 kPa) in the higher-pressure range, as shown in Fig. 2f.

To demonstrate the bimodal sensing and spatial resolution of the tactile sensor array, static (~0.25 Hz) and dynamic pressure (100 Hz) were simultaneously applied to 5 different locations (left bottom, left top, right top, right bottom, center) of the sensor, as shown in Fig. 2g and Supplementary Fig. 6. Static pressure changes were measured by 32-channel static pressure sensor pixels and the dynamic vibrations were detected by the 4 channel dynamic sensor pixels. Figure 2h shows spatial distribution of static and dynamic pressure plotted by red (dynamic pressure) and blue (static pressure) heatmaps, when stimulated using a 1 cm × 1 cm commercial vibrator (PowerHapTM, TDK Corp.) such as circles shaded in red. The bimodal sensor can monitor static pressure and dynamic pressure simultaneously and selectively, which is advantageous to acquire complex tactile information including surface shape and texture.

The mechanical compliance of the ultra-flexible and ultra-thin bimodal sensor enables conformable attachment onto the soft skin surface. The ultra-thin sensor is also advantageous for enabling natural deformation of skin upon application of small area pressure (i.e., thick sensors have higher stiffness preventing natural deformation of skin). To verify the mechanical compliance of the ultra-flexible bimodal sensor, the compressive strain-stress curve of a flexible and an ultra-flexible substrate laminated on a silicone elastomer, which implies stiffness of each film was measured. Stiffness of each film. The flexible substrate is a 125 μm-thick PI film typically used for FPCB, and the ultra-flexible substrate is 4 μm-thick PI film used for the bimodal sensor of this study (Supplementary Fig. 7). The strain-stress characteristics of the silicone elastomer (Eco-flex 00–30, modulus: 210 kPa (measured) and 100 kPa (ref. 42), which has a Young’s modulus similar to that of human skin (epidermis modulus: 140 to 600 kPa43), was measured for reference, and each substrate laminated on the elastomer was pressed by a load cell. Compared to the stress of the silicone elastomer was 21 kPa at 10% strain, the stresses of the flexible and ultra-flexible substrates on the reference elastomer were, respectively, 44 and 24 kPa under the same conditions. This result means that the bimodal sensor integrated on the ultra-flexible substrate can greatly improve mechanical conformability when attached to the skin and therefore precisely collect tactile information along natural skin deformation.

Sub-millimeter scale actuator for vibrotactile stimuli

An ultra-flexible high-resolution actuator array is fabricated by integrating sub-mm piezoelectric ceramics on an ultra-thin PI substrate. The sub-mm piezoceramic actuators were miniaturized to individually stimulate an area of less than 1 mm2 on the fingertip, as shown in Fig. 3a, b. Reducing the area of actuator causes the degradation of output performance, which means that there is a trade-off relationship between miniaturization and the output performance. To overcome this issue, we increase the layers of ceramics by manufacturing a multi-layered actuator, which has 64 layers of 26 μm-thick PZT ceramics as shown in Fig. 3c, d. Moreover, an improvement of output performance by stacked layers has been analyzed by finite element analysis (FEA) as shown in Supplementary Fig. 8.

Fig. 3: Flexible sub-mm tactile actuator array for tactile rendering.
figure 3

a Image of a single actuator unit on a fingertip, scale bar is 2 mm. b Optical microscope image of sub-mm tactile actuator. Multi-stack piezoceramic block is miniaturized for high-resolution tactile rendering, scale bar is 200 μm. c magnified optical microscopic image of multi-layered piezoceramic actuator, scale bar is 100 μm. d Schematic of multi-layered piezoelectric actuator and its operation mechanism. e FEM simulation of stress distribution in flexible actuator arrays that have 3 different substrate thickness during bending. f side view of FEM simulation. g Sum of stress at the interface between actuator and substrate during bending. Stress is relieved by chessboard/diagonal arrangement (solid line) compared to square arrangement (dashed line). h output performance of a single actuator under increased pressure preloads while input voltage is applied from 5 to 60 Vpp. i Frequency dependence of single actuator under pressure loading. Red line show output displacement under pressure loading while the frequency is changed from 100 to 1000 Hz. Blue line shows displacement under pressure loading while input voltage is changed from 5 to 60 Vpp. Inset is a magnified plot in frequency range 200 to 500 Hz. j Amplitude modulation of sub-mm piezo actuator for expression of static pressure feeling. Low frequency of 1–10 Hz has been demonstrated with a carrier frequency of 300 Hz.

Full size image

Furthermore, to achieve high mechanical flexibility and high spatial resolution simultaneously, the thickness of substrate is reduced and the actuators are arranged in a chessboard (diagonal) pattern. Figure 3e–g shows the simulation results of stress distribution while the actuator array is bent (Supplementary Fig. 9). When bending occurs, the stress is concentrated on the interface between the ceramics and the substrate as shown in red color in Fig. 3e, f. This is due to the actuator array structure in which rigid ceramics are regularly arranged on a flexible substrate, and the simulation results show that it can be significantly relieved as the thickness of the flexible substrate decreases. Figure 3g shows the change in interfacial stress as the thickness of the flexible substrate decreases while the bending radius of the actuator array is changed. When the bending radius is 13 mm, the interfacial stress decreases to about 1/100 as the substrate thickness decreases from 100 to 5 μm. Furthermore, when the substrate thickness is 5 μm, the actuator array arranged in the chessboard pattern proposed in this paper showed ~21% lower stress than that of the grid pattern. Thus, the actuator array integrated in a chessboard pattern on a 5 μm-thick ultra-flexible PI substrate can have high mechanical flexibility and skin conformability, even though the ceramics are very rigid, like the tactile sensor shown in the previous section. Moreover, after bending test of ultra-flexible actuator array, mechanical failure was not observed such as crack and rupture even at 3 mm bending radius as shown in Supplementary Fig. 9.

In practical applications, tactile actuators can experience the contact force of a human finger and their output performance can be affected by this external force. To verify the performance stability of the developed actuator according to the external load, the characteristics of the ceramic pixel was analyzed under increased pressure preloads. Figure 3h shows the output performance of a ceramic actuator pixel under the operating voltage range of 5 to 60 Vpp when an external pressure from 0 kPa to 592 kPa is applied. The pressure applied to the actuator pixel was calculated by measuring the force gauge value while uniformly pressing the entire upper surface of the pixel using an anvil of 1 mm2, as shown in the inset schematic of Fig. 3h. The acceleration generated in the ceramic pixel increases as the pressure preload increases, which can be understood by the effective output force equation Fmax eff ≈ kT \(\Delta L_0\left( {1 - \frac{{k_{{{\mathrm{T}}}}}}{{k_{{{\mathrm{T}}}} + k_{{{\mathrm{s}}}}}}} \right)\) of the piezo actuator model with an external preload by spring44, where ∆L0 [m] is nominal displacement without external force, kT, ks [N m−1] is stiffness of piezo actuator and external spring, respectively. Effective generating force of piezo actuator increases as the ks is increased by the preload. The acceleration value of the pixel corresponding to this output force showed a maximum of 2.1 m s−2, which is more than 21 times larger than the human vibrotactile threshold (Pacinian threshold of 0.1 m s−2)45,46,47,48, under the 60 Vpp driving voltage and 592 kPa preload. Here, since the output force generated in the pixel is coupled with reduction in displacement, the displacement value of the ceramic pixel is expected to be the lowest under the maximum pressure preload, and eventually was measured at 1.3 μm when a driving voltage of 60 Vpp and a pressure of 592 kPa were applied. The displacement results (blue dots) shown in Fig. 3i were measured by LSV on the surface of the pixel using 200 Hz sinusoidal signal. Considering that the generated displacement at an external pressure of 592 kPa is about 8.5 times larger than the human Pacinian threshold of 150 nm45,46,47,48, it is confirmed that the miniaturized multi-layered ceramic pixels integrated on the ultra-flexible substrate proposed in this study show sufficient performance to provide haptic feedback. In addition, the miniaturized actuator pixel can generate notable vibrations in a wide frequency range of 100 to 1000 Hz even under the pressure loading. When the input voltage of 50 Vpp is applied during the frequency sweep, its displacement values are sufficiently larger than the Pacinian threshold in the entire frequency range as shown in Fig. 3i. The displacement is slightly decreased as the frequency increases but it is negligibly small change, which is from 1009 nm at 200 Hz to 998 nm at 500 Hz. These results indicate that the miniaturized ceramic-based piezoelectric actuator is robust under an external load or contact force up to 592 kPa, which is a challenging issue for other soft or flexible actuators45.

The piezo actuator provides vibrotactile feedbacks that mainly presented by vibrations. To demonstrate a feeling of low-frequency static pressure, amplitude modulation can be utilized in the form of envelops as depicted in Fig. 3j. Here, envelope frequency ranging from 1 Hz to 10 Hz was generated by using 300 Hz carrier frequency. The carrier frequency and envelop frequency were confirmed by FFT analysis. Along with the carrier frequency peak (300 Hz), low-frequency peaks at 1, 2, 3, 5, 7, and 10 Hz, were clearly visible in the FFT result (Supplementary Fig. 10).

Wireless tactile acquisition and rendering system

For realizing immersive telehaptic communication, not only sensor and actuator, but also proper designing of signal processing units and wireless connections are crucial for transmitting the complex tactile stimuli with high similarity and low delay. In the proposed telehaptic system, the tactile sensor and actuator were wirelessly coupled by signal processing modules, as shown in Fig. 4. The module for processing and transmitting the sensor signals is composed of preamplifiers, a microcontroller, and a Bluetooth low-energy (BLE), as shown in Fig. 4a, b. Pre-amplifiers for bimodal pressure sensor array have been designed as two different circuits for each sensing type. The static pressure was measured by a voltage divider and a voltage follower circuit. The dynamic pressure was measured by using a high gain voltage-mode piezoelectric amplifier, which can amplify pA-order charge generation (Supplementary Fig. 11b, c). The static/dynamic pressure signals acquired from the bimodal tactile sensor have been converted into digital signals using 12-bit Analog-to-Digital Converter (ADC), and then wirelessly transmitted to the actuator drive module using BLE. The module for the tactile actuator array is composed of an actuator driver, a microcontroller, and a BLE for generating vibrations in accordance with the transmitted sensor signals. The actuator driver distributed voltage inputs to each actuator pixel with peak-to-peak voltage in the range of 40 V to 60 V, and it can generate maximum voltage up to 90 V. A LED driver has been added to the actuator drive module to visualize the tactile patterns regenerated via telehaptic communication (Fig. 4c, d).

Fig. 4: Wireless tactile acquisition and rendering system for telehaptic communication.
figure 4

a Image of wireless tactile acquisition module. It illustrates the integrated circuit components of the UFB sensor array: (i) boost/inverting converter, (ii) BLE module, (iii) Microcontroller, (iv) 4 ch D-sensor array, and (v) 32 ch S-sensor array. b System block diagram of tactile acquisition from the UFB sensor array. c Image of wireless tactile rendering circuit board: (vi) Buck converter, (vii) Microcontroller, (viii) BLE, and (ix) 32 ch actuator drivers. d System block diagram of tactile rendering from a tactile actuator array. e Image of the UFB tactile sensor (left) applying tactile stimuli patterns with a vibrator. The sub-mm actuator array (right) measures the vibration amplitude and the acceleration using a LSV, scale bar is 5 mm. f Wireless transmission of tactile stimuli. The sensor output (red) and actuator velocity (blue) was measured while 50 Hz was applied to the sensor using a vibrator. g The delay time between sensor output (red) and actuator output (blue). Sensor signal was wirelessly transmitted, and the vibration was measured by LSV. h the tactile signals (blue) recorded by UFB tactile sensor, while tactile stimuli were applied by the vibrator. i Vibrational displacement (red) of the actuator rendered by transmitted sensor signals. j Correlation matrix correlogram of signal patterns describing the correlation coefficient between the signals acquired by the sensor (S1, S2) and the rendered actuator signals (A1, A2). The S1(e), S2(e) and A1(e), A2(e) were envelope of sensor and actuator signals, respectively.

Full size image

To measure the delay time of the wireless telehaptic system, a 50 Hz sine wave was applied to a dynamic sensor pixel using a vibrator. Moreover, then the vibration signal transmitted to an actuator pixel 3 m away was evaluated using a 3D laser scanning vibrometer (LSV) as shown in Fig. 4e. The delay time was 1.55 ms, which was measured by comparing the signal applied to the sensor with the signal regenerated by the actuator (Fig. 4f, g). Moreover, the on-skin telehaptic system showed high signal similarity (>80%) in detecting and regenerating arbitrary tactile stimuli, meaning that it can provide realistic tactile communication by accurately reproducing the tactile information collected from the sensor array beyond the limits of predetermined conventional haptic systems.

The similarity between the acquired complex tactile signals and regenerated tactile stimuli was verified in detail by tactile patterns measured at each pixel of the sensor/actuator array. A Vibrational signal patterns similar to motorcycle vibration were transmitted through the telehaptic system, as shown in the blue plot depicting the sensor signal (Fig. 4h) and the red plot depicting the actuator’s vibrational displacement (Fig. 4i). The correlation between the sensor signals and the actuator vibration signals (orange windows of Fig. 4h, i, with the area name of S1, A1, S2, and A2) was calculated and shown in the correlation matrix correlogram from Spearman’s correlation which assesses monotonic relationships between sensor signal and actuator signal (Fig. 4j). If correlation coefficient is close to 1, detected sensor signal and regenerated actuator signal are exactly same, which is very challenging. The correlation coefficients between S1-A1 and S2-A2 were both 0.8 and 0.81, respectively, and the correlation coefficients of signal envelop for S1-A1 and S2-A2 were 0.97 and 0.91, respectively. In other words, even if both the amplitude and frequency of the tactile patterns are considered, the wireless telehaptic system exhibits a signal similarity of 80% and 81% between the acquired and generated tactile stimuli. Furthermore, in terms of similarity between signal envelops, our system shows signal similarity of 97% and 91% between collected and regenerated tactile signals (S1(e)-A1(e) and S2(e)-A2(e)), respectively, as shown in Fig. 4j. This result confirms that the on-skin telehaptic system wirelessly interconnecting high-resolution sensor/actuator array can transmit and share tactile information with very low latency and high signal similarity, and therefore are advantageous for real-time haptic rendering in virtual reality.

Demonstration of a wireless telehaptic system

Figure 5a shows a skin-attachable wireless telehaptic system demonstrating the transmission of spatial pressure distribution. Volunteer 1 (left) attaches the ultra-flexible bimodal (UFB) tactile sensor on the fingertip of the index finger and rolls a plastic stick on it. The UFB tactile sensor acquires the spatial distribution of the static pressure changes and transmits the signals to the actuator processing module. The tactile information is reproduced by a tactile actuator attached on the fingertip of volunteer 2 (right) and visualized by a light-emitting diode (LED) display placed on the back side of the hand of volunteer 2 (Supplementary Movie 1). Figure 5b shows magnified images of the pressed positions of the UFB sensor on the fingertip and the LEDs illuminated on the actuator drive module at the same time (inset of Fig. 5b). The UFB sensor can detect more complex shapes due to its highly pixelated static pressure sensor array, such as small alphabet letters (I, V, U), which are mountable on fingertip, as shown in Fig. 5c, and the actuator array can regenerate the acquired tactile signals, as shown in the LSV scanning image (Fig. 5d and Supplementary Movie 2). The red color in Fig. 5d indicates a larger vibrational amplitude.

Fig. 5: Demonstration of a wireless telehaptic system.
figure 5

a Spatial distribution of tactile stimuli (static pressure) was acquired by UFB tactile sensor array and the signals were wirelessly transmitted to the actuator by the telehaptic system. Pressed position and pressure scale were visualized by the LED display pixels and their brightness, scale bar is 5 cm. b Magnified image of pressed positions (top, center, bottom) on the UFB tactile sensor by a stick, scale bar is 1 cm. Insets are illuminated LED pixels indicating sensor signal for each position, scale bar is 3 mm. c Pressure in the form of letter shapes (I, V, U) was applied to the UFB tactile sensor and displayed by the graphical use interface (GUI). d 3D scan image of vibrational displacement while acquiring sensor signals rendered by the actuator array. e Texture information (dynamic pressure) was acquired by UFB sensor while rubbing the different fabrics. f 3 different types of fabric with various roughness. Fabric 1 is rough, fabric 2 is smooth, and fabric 3 is bumpy because of the height differences in the fiber bundles, scale bar is 1 mm. g STFT spectrogram of sensor signal corresponding to fabric 1 (left), fabric 2 (center), and fabric 3 (right). h STFT spectrogram of the measured vibration velocity of the actuator for each fabric.

Full size image

The UFB tactile sensor can also detect dynamic vibrations that are relevant to texture and motion. The volunteer with the UFB tactile sensor attached to the fingertip rubbed the surface of three different fabrics (Fig. 5e, f and Supplementary Movies 3 and 4). To amplify the surface vibration, the PDMS grating pattern was laminated on the surface of the sensor (Supplementary Fig. 12). The roughness of the fabric increases with fabric number; for example, fabric 3 is the roughest among the fabrics used in the study (Supplementary Fig. 13). To further investigate the correlation between the sensor and actuator signals after transmitting the texture information of the fabrics to the actuator, short-term Fourier-transform (STFT) analysis was conducted. Figure 5g, h show the STFT results of the sensor signal and the actuator vibration. The main frequencies (fmain) of the sensor signals were 403, 326, and 229 Hz for fabrics 1, 2, and 3, respectively. The main frequency decreased when the sensor rubbed the rougher surface49. Similar to the tendency of sensor signals, the main frequencies of the actuator signals were 400 Hz, 326 Hz, and 229 Hz (Supplementary Fig. 14). The STFT patterns of the sensor and actuator signals show strong similarities, which indicates that the vibration feature has been successfully transmitted. Other complicated tactile stimuli such as press, click, and butterfly wing flap have also been successfully transmitted by the telehaptic system (Supplementary Figs. 15 and 16). Here, the pattern of press was generated by a pressure applying system with an anvil, while tactile patterns of click and butterfly wing flap were generated by using sample waveforms of the vibration actuator (PowerHapTM, TDK Corp., and BOS1901, Boreas technologies). The correlation coefficients of the envelop signals of the sensor and actuator were, respectively, 0.89, 0.98, and 0.93 for pressing, clicking, and butterfly wing flapping patterns. We have also tested the similarity of sine waves with various frequencies (100, 200, 300, 400, and 500 Hz). The correlation coefficient was 0.99, 0.96, 0.99, 0.92, and 0.82 at 100, 200, 300, 400, and 500 Hz, respectively. In other words, the signals reproduced by the actuator shows similarities of >90% to the sensor signals, except for 500 Hz. At 500 Hz, the actuator signals included a beat wave with a beat frequency of 7 Hz, which may have originated from the noise in the optical measurement system, i.e., the doppler signal and speckle noise of the laser vibrometer50.

In this study, we demonstrated telehaptic communication with an on-skin platform. Ultra-flexible sensor array and miniaturized actuator array enable real-time tangible interactions with various tactile stimuli. The piezoresistive and piezoelectric hybridized pressure sensor array was used to detect the spatial distribution of static pressure levels and frequency information of vibrotactile stimuli at high resolution. A miniaturized 32-channel actuator array successfully regenerated the detected tactile stimuli signals. Although the miniaturized piezoelectric actuator can sufficiently generate high spatiotemporal vibration patterns above the detection limit of the skin, the displacement should be further improved for greater enhancement of tactile feelings, through designing displacement amplification mechanisms. The proposed sensor and actuator system can transmit and share tactile sensations remotely, which may open a way for tactile-based human-machine interface. The telehaptic platform can enable assisting the users with manual tasks, making people feel physically connected even over long distances, and enriching the feeling of presence in virtual environments like augmented/virtual reality (AR/VR).

Methods

Fabrication of ultra-flexible bimodal (UFB) tactile sensor array

The polyimide precursor was spin-coated onto a carrier glass and cured to form a 2.5 µm-thick base layer. Moreover, Ti/Au (10 nm/200 nm) electrode and transmission line were fabricated on the base layer by photolithography, with an interdigitated (IDE) pattern with minimum line and space width of 50 and 100 μm, respectively. Moreover, 2.5 µm-thick PI was fabricated on the device for encapsulation layer and windows of contact electrode are opened by PI wet etching. After the fabrication of ultra-flexible backplane, it was peeled off from the carrier glass by laser lift off process with energy density of 180 mJ/cm2 (KORONATM LLO, AP systems). The 32-channel sensor pixel array for static pressure sensor is arranged in 1.8 mm pitch and 4 additional pixels for dynamic pressure sensor are added between the pixels. The width and length of the IDE electrode pattern for the static pressure sensor are 0.25 and 1.1 mm, respectively. The gap between the two counter electrodes is 0.45 mm. The electrode size and gap were determined by considering pixel size, sensor sensitivity, and processability. Resistance change is proportional to area of the electrode pad, such that larger width and length are desirable for sensitivity, but the area is limited for high-resolution device fabrication. The gap was chosen to be 0.45 mm for sharing the backplane with actuator array, which is the suitable size for the mounting process. For the dynamic pressure sensor, the gap between the counter electrode was set at 0.1 mm. The electrode pattern for the piezoelectric sensor has a narrower gap than that of the piezoresistive sensor because smaller gap is advantageous for detecting changes in electric field originated from piezoelectric material under pressure.

A microstructured PDMS film was fabricated using a replica molding process for static pressure sensor, i.e., soft photolithographic molding. A Si wafer was etched using potassium hydroxide (KOH) solution to form micropyramid structures with a side wall angle of 54.7°. The PDMS pyramid soft mold was prepared by mechanically mixing the PDMS (Sylgard 184, Dow Corning) base solution and the curing agent (10:1 wt.%). It was cured at 80 °C for 3 h and lifted off the replica. The width and the gap of the micropyramid structure are both 100 μm. The pyramid sheet was prepared by spin-coating on a soft replica at 800 rpm for 30 s and heating in a convection oven at 80 °C. Polypyrrole (Pyrrole (98%), Sigma Aldrich) was coated on the cured 100-µm-thick micropyramid film by depositing pyrrole monomer silane (Tridecafluoro-1,1,2,2,-tetrahydrooctyl trichlorosilane, Sigma Aldrich). The pyrrole was then polymerized in a pyrrole and oxidant solution (iron (III) chloride hexahydrate, Sigma Aldrich) at 25 °C.

The PVDF-TrFE solutions (20 wt.%) were prepared for dynamic pressure sensor by dissolving PVDF-TrFE copolymer pellets (75:25 (mol%), Piezotech) in 2-butanone (MEK, 99 +%, Sigma Aldrich) and acetone (1:1 (wt.%)). The PVDF-TrFE solution was spin-coated at 4000 rpm for 40 s and onto the IDE metal electrode substrate. The solution was sequentially degassed overnight under vacuum to eliminate the bubbles. The PVDF-TrFE films were well crystallized, following annealing at 130 °C for 3 h. The β-phase crystallinity can be optimally improved above the Tcrystal and below melting temperature shown in the DSC result, which corresponds to XRD and FT-IR results (Supplementary Fig. 3). After preparation of the PVDF-TrFE film, a 10-nm/200-nm-thick Ti/Au film was deposited by E-beam evaporation (KVET-C500200, Korea Vacuum) under a DC field of 10 kV at 9 × 10−7 Torr as an etch stop mask for reactive ion etching (RIE). The PVDF-TrFE film was etched under 10 sccm O2 and 20 sccm CF4 gas environments for 30 min with 20 W RF power. It was polarized under a DC field of 50 kV mm−1 at 25 °C for 3 h. Finally, the micro-pattern for enhanced texture perception was prepared by the PDMS micromolding process with line width, height, and spacing, each equal to 1 mm.

Fabrication of ultra-flexible sub-mm piezoelectric (UFP) tactile actuator array

The sub-mm tactile actuator is a piezoelectric ceramic multilayer, which is composed of 64 layers of lead zirconate titanate (PZT) ceramic with a thickness of 26 µm. Fabrication process of sub-mm tactile actuator is industrially standardized process of piezoelectric ceramic actuator. Firstly, green sheet of PZT slurry with top Ag electrode is laminated into 64-layer-stack and sintered in high temperature (~1200 ˚C). After the sintering process, the piezoelectric ceramic multilayer is cut into a die of dimensions 0.98 × 0.98 × 0.8 mm. Finally, contact electrodes (Ag) were sputtered on both sides of the ceramic block for connecting inner electrodes and on bottom for contact pad (designed by ETRI and manufactured by WiSOL Co., Ltd). The tactile actuator pixel was bonded together for double stack actuator and electrically connected by silver epoxy. Then, 32 actuator pixels are firmly anchored above the flexible printed circuit board (FPCB) by reflow soldering process for flexible actuator array. For the ultra-flexible actuator array, actuators are mounted on 5 µm-thick PI substrate with carrier glass by using adhesive conductive film (ACF) transfer process and peeled off from the carrier glass by using above mentioned laser lift off process (LLO).

Measurement of UFB tactile sensor array

The static sensitivities of the bimodal pressure sensor were measured by a load cell pressure measurement station, which has a power supply (E3631A, Keysight) with a bias voltage of 2 V, a low-noise current preamplifier (SR570, Stanford Research Systems), and a force gauge (PI V-275.431, force range of 0.01 N to 10 N and resolution of 0.001 N (0.01 gf)).

The characteristics of the dynamic pressure sensors were acquired using a low-noise current preamplifier (SR570, Stanford Research Systems) and oscilloscope (DPO5054, Tektronics). We used a function generator (33220A, Agilent) and a piezo amplifier (E663, PI) to apply the dynamic frequency.

The strain-stress measurement setup consisted of a load cell pressure station with a force gauge and the signals were captured using a data acquisition system (DAQ).

PVDF-TrFE has been characterized by X-ray Diffraction (D/MAX-2500 with R-AXIS IV++ diffractometer, Rigaku, Japan), Fourier-Transform Infrared Spectroscopy (Nicolet iS50 FT-IR, Thermo Fisher Scientific, USA), and Differential Scanning Calorimeter (TGA/DSC1, Mettler-Toledo, USA).

Measurement of ultra-flexible tactile actuator array

The characteristics of the ultra-flexible actuator array were investigated using a laser scanning vibrometer (LSV) system (PSV-400, PolyTech GmbH, Germany) with a laser scanning head (PSV-400), a scanning vibrometer controller (OFV-5000), and a junction box (PSV-400).

Wireless tactile acquisition and rendering system

The tactile sensor and actuator array were operated by a wireless tactile acquisition circuit board (Transmit Data, TX) and a tactile rendering circuit board (Receive Data, RX), respectively. The tactile TX circuit comprises of a preamplifier circuit, a microcontroller with Bluetooth communication system, a power management circuit to provide constant voltage, and a battery.

The sensor signal acquired by the bimodal tactile sensor was amplified using a preamplifier circuit. A voltage-mode amplifier (AD8643, Analog Devices) circuit was used for the dynamic pressure sensor, and a voltage divider and follower circuit were used for the static pressure sensor.

A microcontroller (nRF528321, Nordic semi) was mounted on a PCB board for data sampling (12-bit built-in ADC). The sampled data was transmitted to the RX board using a 2.4 GHz RF built-in Bluetooth module. The actuator driver (BOS1901, Boreas Technologies) for driving piezoelectric actuators is controlled by elevating the voltage from 3.3 V to a maximum voltage of 95 Vpp. The LED driver and the LED matrix (PIM442, PIMRONI Ltd) were used for the visualization of tactile feedback. The brightness of the LED was controlled based on the spatial distribution and pressure level data received from the static pressure sensor array.

The TX board was powered by a Li-polymer battery (TW401215, 15 × 12 × 4 mm2, 40 mAh), and the power was regulated by a boost and inverting converter. The RX board is composed of a 300 mAh Li-polymer battery, and a buck converter is used for power regulation.

The tactile acquisition signals are collected by ultra-flexible tactile sensor on vibration generator. Vibration generator generates waveforms such as motorcycle, butterfly wing flap, click, and sine waves. Moreover, the acquired tactile signals are wirelessly transmitted to ultra-flexible tactile actuator and measured by laser scanning vibrometer. Spearman’s correlation that assesses monotonic relationships between sensor signal and actuator signal has been used to calculate signal correlation coefficient.

Demonstration of wireless telehaptic system

To attach ultra-flexible sensor and actuator on skin, supporting adhesive layer, which can prevent large strain by the high pressure or shear stress has been used. This supporting adhesive layer is double sided medical adhesive tape (3 M™ 1509), which is biocompatible and highly flexible.

Haptic communication was demonstrated with an ultra-flexible static/dynamic pressure sensor, a customized 32-channel amplifier, and a multi-morph piezoelectric actuator. The bimodal pressure sensor was connected to an amplifier. The noise of the signal is filtered, and the gain of the signal is controlled. The tactile stimuli information (sensor output) was recorded simultaneously using an oscilloscope (DPO-5054, Tektronics). The recognized information was transmitted to the actuator. The vibration of the actuator was monitored using the LSV system. The monitored vibration information included vibration displacement for each point within the scanning area.

To quantitatively analyze the average roughness of the three fabrics, the average maximum height of the 1.6 mm profile was measured using a confocal microscope (DCM-8, Leica).

The signals from the ultra-flexible tactile sensor attached to the fingertip were acquired while rubbing the three different fabrics in the same direction and transmitted to the tactile actuator.