Traditional technologies for virtual reality (VR) and augmented reality (AR) create human experiences through visual and auditory stimuli that replicate sensations associated with the physical world. The most widespread VR and AR systems use head-mounted displays, accelerometers and loudspeakers as the basis for three-dimensional, computer-generated environments that can exist in isolation or as overlays on actual scenery. In comparison to the eyes and the ears, the skin is a relatively underexplored sensory interface for VR and AR technology that could, nevertheless, greatly enhance experiences at a qualitative level, with direct relevance in areas such as communications, entertainment and medicine1,2. Here we present a wireless, battery-free platform of electronic systems and haptic (that is, touch-based) interfaces capable of softly laminating onto the curved surfaces of the skin to communicate information via spatio-temporally programmable patterns of localized mechanical vibrations. We describe the materials, device structures, power delivery strategies and communication schemes that serve as the foundations for such platforms. The resulting technology creates many opportunities for use where the skin provides an electronically programmable communication and sensory input channel to the body, as demonstrated through applications in social media and personal engagement, prosthetic control and feedback, and gaming and entertainment.
Subscribe to Journal
Get full journal access for 1 year
only $3.90 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
All data are contained within the manuscript. Raw data are available from the corresponding authors upon reasonable request.
Custom code used in this study is available from the corresponding authors upon reasonable request.
Kapur, P., Jensen, M., Buxbaum, L. J., Jax, S. A. & Kuchenbecker, K. J. Spatially distributed tactile feedback for kinesthetic motion guidance. In 2010 IEEE Haptics Symp. (ed. Basodogan, C.) 519–526 (IEEE, 2010).
Sardini, E., Serpelloni, M. & Pasqui, V. Wireless wearable T-shirt for posture monitoring during rehabilitation exercises. IEEE Trans. Instrum. Meas. 64, 439–448 (2015).
Kim, Y. et al. A bioinspired flexible organic artificial afferent nerve. Science 360, 998–1003 (2018).
Novich, S. D. & Eagleman, D. M. Using space and time to encode vibrotactile information: toward an estimate of the skin’s achievable throughput. Exp. Brain Res. 233, 2777–2788 (2015).
Sundaram, S. et al. Learning the signatures of the human grasp using a scalable tactile glove. Nature 569, 698–702 (2019).
Withana, A., Groeger, D. & Steimle, R. Tacttoo: a thin and feel-through tattoo for on-skin tactile output. In Proc. 31st Ann. ACM Symp. on User Interface Software and Technology (ed. Wilson, A. D.) 365–378 (ACM, 2018).
Huang, C.-T., Shen, C.-L., Tang, C.-F. & Chang, S.-H. A wearable yarn-based piezo-resistive sensor. Sens. Actuators A 141, 396–403 (2008).
Cogan, S. F. Neural stimulation and recording electrodes. Annu. Rev. Biomed. Eng. 10, 275–309 (2008).
Song, Y. M. et al. Digital cameras with designs inspired by the arthropod eye. Nature 497, 95–99 (2013).
Rogers, J. A., Someya, T. & Huang, Y. Materials and mechanics for stretchable electronics. Science 327, 1603–1607 (2010).
Wilska, A. On the vibrational sensitivity in different regions of the body surface. Acta Physiol. Scand. 31, 285–289 (1954).
Mahns, D. A., Perkins, N. M., Sahai, V., Robinson, L. & Rowe, M. J. Vibrotactile frequency discrimination in human hairy skin. J. Neurophysiol. 95, 1442–1450 (2006).
Han, S. et al. Battery-free, wireless sensors for full-body pressure and temperature mapping. Sci. Transl. Med. 10, eaan4950 (2018).
Kim, D.-H. et al. Epidermal electronics. Science 333, 838–843 (2011).
Kim, J. et al. Battery-free, stretchable optoelectronic systems for wireless optical characterization of the skin. Sci. Adv. 2, e1600418 (2016).
Zhang, Y. et al. Experimental and theoretical studies of serpentine microstructures bonded to prestrained elastomers for stretchable electronics. Adv. Funct. Mater. 24, 2028–2037 (2014).
Huang, X. et al. Epidermal radio frequency electronics for wireless power transfer. Microsyst. Nanoeng. 2, 16052 (2016).
Chung, H. U. et al. Binodal, wireless epidermal electronic systems with in-sensor analytics for neonatal intensive care. Science 363, eaau0780 (2019).
Beh, T. C., Kato, M., Imura, T., Oh, S. & Hori, Y. Automated impedance matching system for robust wireless power transfer via magnetic resonance coupling. IEEE Trans. Ind. Electron. 60, 3689–3698 (2013).
Kim, J., Son, H., Kim, K. & Park, Y. Efficiency analysis of magnetic resonance wireless power transfer with intermediate resonant coil. IEEE Antennas Wirel. Propag. Lett. 10, 389–392 (2011).
The Electronic Code of Federal Regulations (e-CFR) Title 47, 1.1310. Radiofrequency radiation exposure limits. https://www.ecfr.gov/cgi-bin/retrieveECFR?gp=&SID=81a1c5a13f8d02f165a06af36937d612&mc=true&n=pt47.1.1&r=PART&ty=HTML#se47.1.1_11310 (e-CFR, 2019).
Akamatsu, M., Mackenzie, I. S. & Hasbroucq, T. A comparison of tactile, auditory, and visual feedback in a pointing task using a mouse-type device. Ergonomics 38, 816–827 (1995).
Lee, Y. RFID Coil Design (Microchip Technology Inc., 1998).
We gratefully acknolwedge C. J. Su, T. Banks, J. H. Kim, Y. G. Xue and J. K. Chang for their efforts in constructing and testing the optimized systems. This work was supported by the Center for Bio-Integrated Electronics at Northwestern University. Z.X. and X.F. acknowledge support from the National Basic Research Program of China (grant number 2015CB351900) and the National Natural Science Foundation of China (grant numbers 11320101001 and 11402134). X.Y., Z.X., D.L. and Y.L. acknowledge support from the City University of Hong Kong (grant number 9610423). R.S. acknowledges support from the Engineering and Physical Sciences Research Council (grant number EP/L016028/1) and the China Scholarship Council. Y.H. acknowledges support from the NSF (grant number 1635443).
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Peer review information Nature thanks Sriram Subramanian, Xiaoming Tao and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Extended data figures and tables
a, Distribution of the normalized magnetic field strength (H) around the Cu coil (300 turns) used in the actuator, where d is the distance between the Cu coil and the magnet. The dashed white circles correspond to the holes in the Cu coil. b, Normalized magnetic flux (Φ/Φd=0) through the haptic actuator versus the distance to the Cu coil (300 turns) of the actuator. c, Schematic illustration of the distance between a haptic actuator and a Cu coil in the actuator. d, The normalized magnetic flux (Φ/ΦMax) through the magnet versus turn number and outer diameter (Douter) of the Cu coil.
a, Top view and cross-sectional view of the actuator design. The parameters presented here are optimized for tuning the resonance frequency to the skin-sensitive range. b–d, Optical images and normalized amplitude–frequency curves of three actuators with different central angles (θ) of 150° (b), 186° (c) and 217° (d), working without any contact. The actuators shown in b and c are 18 mm in diameter and 2.5 mm thick.
a, Cross-sectional schematic illustration of an actuator in contact with skin. b, Theoretical results of the resonance frequency of actuators shown in a versus the central angle θ of the PI handling layer. The dashed lines indicate the resonance frequency of 200 Hz at θ = 217°. c, Comparison of experimental (E; data symbols) and simulation (theory, T; lines) results. d, Experimental results of the normalized amplitude–frequency curves of the actuator (θ = 217°) in contact with skin, for different values of skin elastic modulus: 60 kPa, 130 kPa and 200 kPa. e, Travel amplitude of the magnet as a function of the input power (data points) for an actuator in contact with artificial skin samples with elastic moduli of 60 kPa, 130 kPa and 200 kPa. In c, e, error bars correspond to the calculated standard deviation.
a, b, Cross-sectional schematic illustration (a) and optical image (b) of an actuator in contact with artificial skin. Here PDMS, serving as artificial skin, had three different values of elastic modulus, and glass was used for artificial bone. c, Comparison of the experimental (E) and simulation (theory, T) results of the resonance frequency of the actuator in contact with different moduli and thicknesses of artificial skin. The error bars correspond to the calculated standard deviation. d, Optical images captured using a high-speed camera and a working actuator travelling up and down, when in contact with 130-kPa artificial skin. e, FEA results for the amplitude of an actuator: left, when separated from skin; right, when in contact with skin. f, Schematic illustration and FEA results (colour-coded amplitude) for mechanical coupling between an array of haptic actuators, with activation in an ‘N’ pattern.
a, Mutual interference of two actuators at different relative angles α at 200 Hz. Two representative cases were studied, with one actuator (no. 1) positioned along (left) or perpendicular to (right) the bisector of the other actuator (no. 2). Here, only actuator 1 was actuated. The amplitude ratio, that is, the amplitude of actuator 2 over the amplitude (induced by mutual interference) of actuator 1, shown in the table demonstrates that α = 45°, 90° and 270° result in relatively small mutual interference for both representative cases simultaneously. b, Mutual interference of two small actuators at different relative angles α at their resonant frequency of 200 Hz. The size of the actuators and the distance between them were scaled to 1/10 of the original design as shown in a, that is, the distance between two actuators was 2.1 mm rather than 21 mm. Here the thickness of the PI disk was set at 1.8 μm to enable the resonant frequency for the small actuator to be 200 Hz. Two representative cases were studied, with one actuator (no. 1) positioned along (left) or perpendicular to (right) the bisector of the other actuator (no. 2). Here only actuator 1 was actuated. The amplitude ratios (actuator 2 to actuator 1) due to mutual interference are shown in the table. c, Optimization of the actuators’ arrangement. The mutual interference among actuators was studied for α = 0°, α = 45° and a combination of 90°/270° referring to the simulation results in a, and for representative cases when actuators A (around the centre), B (near the boundary) and C (near the corner) are actuated separately. The results show that α = 90°/270° yields the smallest mutual interference (see tables under). The number gives the amplitude ratio due to the mutual interference among the actuators—that is, the amplitude of all actuators over the amplitude of the activated actuator—where 1 represents the activated actuator.
a, Interference between the primary coil and NFC antennas. Shown are schematic illustrations of a primary coil with (left) or without (right) four NFC antennas along the Z direction of the transmission antenna. b, Comparison of experimental (E) and theoretical (T) results of voltage induced by a single primary coil versus a primary coil with four NFC antennas. c, d, Transmission antennas used for operating the VR devices: c, small size, 318 mm × 338 mm; and d, large size, 620 mm × 852 mm. e, An transmission antenna placed in the X–Y plane. f, The magnetic field strength (H) in the Z–X plane (the middle plane of the coil) for the small (left) and large (right) transmission antennas (RF readers). g, Theoretical results show that the small and large transmission antennas are suitable for short (<24 cm) and long (>24 cm) working distances, respectively. h, i, Output power of the serpentine primary coil of the epidermal VR device as a function of distance to the small transmission antenna (h), and the large transmission antenna (i). The error bars correspond to the calculated standard deviation.
a, Configuration of an intermediate coil (20 cm × 20 cm, wound with Cu wire with a diameter of 0.2 mm) oriented parallel to the X–Y plane, at a distance Z from the transmission antenna. b, Computational results for the magnetic field distribution induced by an RF transmission antenna tuned to resonance with a receiver antenna with and without an intermediate coil. c, Comparison of the magnetic field strength along the Z direction of the transmission antenna with (W) and without (W/O) the intermediate coil. d, Amplification factor η along the Z direction of the transmission antenna with and without the intermediate coil. e–g, Simulation and experimental results for the inductance (e), resistance (f) and Q factor (g) as a function of frequency.
a–d, Output power from the primary coil of an epidermal VR device as a function of tilt angle χ (a, geometry; b, data), bending radius, R (c) and bending cycles to an R of 2.8 cm (d). The distance between the device and the antenna was fixed at 20 cm for all measurements. e, Measured phase responses of the antennas used for wireless control over the SoCs as a function of radius of curvature. The resonance frequency is 13.56 MHz before bending. Bending induces only very slight shifts in these curves.
a, Schematic illustration of a custom-built laser array system for real-time visualization of the operation of a complete epidermal VR system. b–d, The array of lasers (b), the corresponding array of beams reflecting from the haptic actuators, each mounted with a reflective disk (diameter of 8 mm), across an entire system (c), and their arrival at a monitoring screen (d). e, Representative frames extracted from video recorded using a high-speed camera to capture oscillatory motions of each of the laser spots. These motions directly determine the motions of the cantilever-based actuators. f, Schematic illustration of a laser spot produced by projection of a reflected beam onto a screen during the operation of the actuator. g, Representative frames extracted from video recorded using a high-speed camera showing the oscillatory motion of laser spot 1. h, Pictures of a laser spot on a reflector that mounts on a haptic actuator (left) and on the monitoring screen (right). The diameter of the laser spot is ~3 mm. i, Calculated displacements of four actuators determined from the measurement setup geometry and the amplitude of motion of the laser spots in g, e and Supplementary Video 4. The traces are offset in the y direction to facilitate visual inspection. The calculated displacements are somewhat smaller than those measured directly from individual actuators using high-speed cameras owing to slight misalignments of the lasers and to shifts in the resonance frequencies due to absence of the PDMS encapsulation layer for the devices measured using the laser technique. The results allow direct visualization and measurement of the vibration amplitudes, direction and frequency of the cantilever beams associated with each actuator across the full array.
Extended Data Fig. 10 Pictures of the operation of an epidermal VR system, visualized with a reflected array of 32 laser beams.
Activation of a given haptic actuator causes the corresponding reflected spot to transform from a circular to an elliptical shape, owing to the vibratory motions (top row). The results in the lower three rows show representative spatial patterns of actuation, including numbers 0 to 9 and letters ‘N’, ‘A’, ‘T’, ‘U’, ‘R’ and ‘E’. We note that the detailed shapes of the laser spots on the screen depend critically on the positioning of each of the beams across the corresponding reflectors mounted on the cantilevered actuator structures.
This file contains Supplementary Figures S1-S12 with full legends.
High-speed-camera recordings of an actuator during operation at a power of 1.75 mW.
FEA of the key vibrational mode for an actuator mounted on the skin, where U represents the displacement (unit: mm) and U3 represents the displacement (unit: mm) in the normal direction of the skin.
Real-time control and visualization of the operation of an epidermal VR system, enabled by the reflection of an array of 32 laser beams off of the device and onto a screen.
Visualization of the vibratory motions of the haptic actuators in an epidermal VR system recorded by using reflected laser beams and a high-speed digital camera.
About this article
Cite this article
Yu, X., Xie, Z., Yu, Y. et al. Skin-integrated wireless haptic interfaces for virtual and augmented reality. Nature 575, 473–479 (2019). https://doi.org/10.1038/s41586-019-1687-0
Nature Materials (2021)
Low cost exoskeleton manipulator using bidirectional triboelectric sensors enhanced multiple degree of freedom sensory system
Nature Communications (2021)
Nature Materials (2021)
Nature Biomedical Engineering (2021)
Nature Biomedical Engineering (2021)