Introduction

Wearable electronics have found many applications in fitness tracking, mobile health monitoring, medical diagnosis, human–machine interfaces (HMIs), assistive technology, internet of things (IoT), etc.1,2,3,4,5,6,7,8 Nevertheless, despite the growing needs, developing wearable devices that offer both functionality and wearability has remained a major challenge. In recent years, stretchable thin-film electronics and wearables, including epidermal sensors or electronic tattoos (e-tattoos), have achieved mechanical properties similar to human skin and demonstrated a wide range of functionalities, including physiological sensing,9,10,11,12,13,14 on-skin display,15 UV detection,16 transdermal therapeutics,17 HMIs,4,18 and prosthetic electronic skin.19 Compared with conventional wearable devices, a unique advantage of e-tattoos is their thinness and softness, which enables noninvasive but intimate coupling with microscopically rough human skin.20,21 Such conformability is confirmed to be responsible for the high signal-to-noise ratio (SNR) and low-motion artifacts of e-tattoo sensors.4,22,23 Although the concept of imperceptible electronics has been introduced for ultra-lightweight plastic electronics,24,25 none of them are optically transparent, which limits their wearability on areas of esthetic prominence such as the human face.

The development of optically transparent e-tattoos requires transparent conductive thin films26,27 such as 2D atomic sheets,28 nanocrystals-based composites,29,30 and intrinsically conductive transparent polymers.31 However, transparent conductive materials for wearable e-tattoos have to meet some additional criteria. For skin integration, these materials must be biocompatible and electrochemically inert. Furthermore, the thickness of conductors should be scalable to microns or even sub-microns to ensure conformability to skin.9,32 As a result, graphene and MoS2 nanomaterials are of significant relevance for e-tattoos due to their atomic thinness, optical transparency, electrochemical inertness, and biocompatibility.33,34,35,36,37,38 Recently, graphene-based wearable strain gauge, temperature sensor, heater, and interconnects have been reported.39,40 However, the combined thickness of the patch is few micrometers39 or hundreds of micrometers,40 which is too thick to fully conform to the micromorphology of human skin according to theoretical analysis and experimental validation.32 Monolayer graphene-based kirigami for stretchable electronics has been realized by other researcher. However, the application of such devices for on-skin electrophysiological measurements could be challenging since such devices need to be suspended in liquid or air to have 3D deformability. Furthermore, the use of laser beam to unfold and refold such structure is not realistic in living systems.41 Bilayer MoS2-based strain gauge and pressure sensor with graphene interconnects has been demonstrated with thickness below 75 nm.42 However, this device is flexible but not stretchable and it only measures mechanical strain. Also it is important to notice that since sub-micrometer-thick e-tattoos are difficult to reuse, low-cost, rapid prototyping and manufacturing process is desirable. The micro-transfer printing technology43 applied to fabricate the 2D materials-based skin patches39,40 is known to be cost and labor intensive owing to photolithography, chemical etching, and meticulous integration processes. To overcome such limitations, we recently developed a “wet transfer, dry patterning” method, which enable time and cost-effective freeform fabrication of 500-nm-thick graphene electronic tattoo (GET) sensors out of graphene/polymer bilayer.44 GET sensors were successfully applied for electrophysiology, skin hydration, and skin temperature measurements.44 However, since the previous GET sensors does not consist of any encapsulation over graphene, wherever it touches the skin, the average signal will be picked up. This induces limitation in application of GET for EOG in which the non-localized measurement is not desirable. We therefore have to modify the “wet transfer, dry patterning” process to achieve GET EOG sensors.

EOG has various applications including brain and sleep studies, assistive technology, sleep and mental disorder diagnosis, and HMIs.45,46,47,48,49 EOG along with electroencephalogram (EEG) and electromyogram (EMG) are used for patients suffering from neural system disorders such as progressive neuro-motor degenerative diseases. However, compared with EEG and EMG, EOG has two major advantages. First, analysis of the EOG signals is less complex because the relationship between EOG and eye movement is linear within a certain range, and the waveform is easy to detect. Second, EOG signal is more stable.50,51 Conventionally, silver/silver chloride (Ag/AgCl) gel electrodes are used for EOG measurement and even currently, most EOG measurements are measured using commercially available dry or wet silver/silver chloride electrodes.45,49,52,53,54 For example, Ubeda et al. measured EOG using dry silver–silver chloride electrodes and used the signal for controlling a robotic arm.55 In other works, EOG sensors were mounted on goggles for wearability.47,56,57,58 Vehkaoja et al. engineered silver-coated fibers as flexible EOG sensors, but they should be moisturized with conductive saline solution to enable recording.59 Although these electrodes are readily available and affordable, they are thick, stiff, uncomfortable to delicate eye skin, and noticeable on face. Wet electrodes are known for low electrode–skin interface impedance and low susceptibility to motion but gels are irritating to skin and dry out by time resulting in degrading the quality of the recorded signal. Dry electrodes are better option for long-term measurements but they suffer from high electrode–skin interface impedance due to non-conformable contact to skin and they are susceptible to motion artifact.60 Ultrathin and stretchable tattoo-like EOG sensors based on filamentary serpentine gold ribbons on polymer substrate have been reported.22 But gold-based electrodes and interconnects are very noticeable on prominent area such as human face. In other reported work, EOG sensor combined with other sensors and circuit components were integrated in a patch with the thickness of several millimeter.61 Such a patch is not only obvious, but also obstructive to facial expression. The GET EOG sensor system presented here has a thickness of 350 nm, optical transparency of 85% in the visible regime (wavelength from 400 to 800 nm), and stretchability up to 50%, which is truly mechanically and optically imperceptible EOG sensor. We found that the GET EOG sensors are capable of detecting eye movement with a resolution of about 4°. Connecting the imperceptible GET EOG sensors to an OpenBCI Cyton board capable of wireless communication, we demonstrate real-time wireless quadcopter control through eye movements.

Results

GET EOG sensors

The GET EOG sensor was fabricated out of large area graphene (Gr) grown on copper foil (Fig. 1a, b). Poly(methyl methacrylate) (PMMA) precursor was first coated and cured over the graphene to form a 350 nm film of PMMA (Fig. 1c). After etching away the copper (Fig. 1d), the Gr/PMMA bilayer was rinsed by deionized (DI) water and transferred onto a piece of commercial tattoo paper (Silhouette) with graphene facing up (Fig. 1e). To record EOG signals and interpret eye movement in the up, down, left, and right directions, a minimum of four EOG sensors are needed around the eyes. Each GET EOG sensor consists of three parts, (1) a disk-shaped electrode where graphene is exposed to make direct contact with the skin to capture the EOG signals, (2) an exposed rectangular graphene terminal pad to connect to flexible gold connector, and (3) an encapsulated serpentine-shaped interconnect linking parts (1) and (2). To encapsulate the serpentine interconnects, we applied a shadow mask made by a desktop mechanical cutter plotter (Silhouette) to cover the electrode and the terminal pad (Fig. 1f), followed by spray coating liquid bandage (Nexcare) to cover the exposed interconnects (Fig. 1g). Within 10–15 s, the liquid bandage solidified into an insulating polymeric layer of roughly 2 µm thickness (Fig. S2). Encapsulating interconnects allow for localized EOG measurement from the exposed disk-shaped electrodes right next to the eyes. After the encapsulation, we removed the shadow mask and used the same mechanical cutter plotter (Silhouette) to carve Gr/PMMA with pre-designed patterns (Fig. 1h). To show the necessity of encapsulation layer on interconnect for localized EOG measurements, we placed two electrodes on the top of an eyebrow of a human subject. One was placed vertically above an eyebrow (location B in Fig. S1a) and the other one at the inclined top, where GET interconnect normally passes through (location A in Fig. S1a). These electrodes work as two separate positive terminals of two channels of our recording system. We then placed another electrode at the bottom of the same eye (Fig. S1a), which acts as a shared electrode connected to the negative terminal of both channels. Then, we measured EOG signals simultaneously using both channels during eyes moving up and down. If EOG signal is naturally localized, there should be no recorded signal using the electrode placed in location A. However, clearly there is the nonzero EOG signal measured at location A as it is shown in Fig. S1b. The EOG recorded at location A is in phase with but has smaller amplitude than that measured at location B. Therefore, covering interconnects with encapsulation layer is necessary for localized EOG measurements. The GET EOG sensors were designed in the serpentine-shaped ribbons to enable stretchability. The stretchability of the GET was measured in our previous to be about 50%.44 In the last step of EOG GET fabrication, extraneous Gr/PMMA was carefully peeled off, leaving GET EOG sensors on the tattoo paper (Fig. 1i). A layer of water-soluble adhesive comes with the tattoo paper can facilitate the release of GET EOG. To laminate GET EOG around the eyes, the tattoo paper was placed on skin with GET EOG facing the skin, followed by wetting the backside of the tattoo paper and then carefully peeling the paper off the skin, leaving open-mesh GET EOG sensors on the skin (Fig. 1j and Supplementary Video VS1).

Fig. 1
figure 1

Schematics of the fabrication process for imperceptible GET EOG sensor system. a Growing graphene on copper foil using atmospheric pressure chemical vapor deposition (APCVD). b As-grown graphene on copper foil. c Spin coating and curing 350-nm-thick PMMA on graphene. d Copper etching and rinsing graphene/PMMA (Gr/PMMA) with DI water. e Transferring Gr/PMMA bilayer on a tattoo paper with graphene facing up. f Covering Gr/PMMA by a shadow mask. g Spray coating liquid bandage over the exposed interconnect area. h Removing shadow mask and carving Gr/PMMA into serpentine-shaped patterns using a mechanical cutter plotter. i Pealing off extraneous Gr/PMMA. j Transferring GET EOG sensor on skin surface just like wearing a temporary tattoo

Full size image

The GET EOG sensors formed steady contact with human skin without any tape or adhesive (Fig. S3), as predicted by our previous analytical models.32 Photos of GET EOG applied around eyes under different facial expressions are displayed in Fig. 2a–d. It is evident that the GET is hardly visible in all cases. Under extreme facial expressions, neither delamination nor cracking of the GET EOG was observed and the subject did not perceive the GET sensors at all. To study the effect of extreme facial expression on sensor electrical characteristics, the end-to-end electrical resistance of one laminated electrode around the eye was measured before and after extreme face expressions. We found negligible change (<0.5%) after a short period (2 min) of extreme facial expressions. However, after an hour of speaking and abundant facial expressions, including laughing, frowning, sour face, and surprise, the electrical resistance increased by 4%.

Fig. 2
figure 2

Imperceptible GET EOG sensors worn around human eyes. a Without facial expression. b Sour face. c Laughing. d Frowning. No delamination nor crack was observed in GET after extreme facial expressions. e A schematic for the locations of GET EOG sensors laminated on face. A Ag/AgCl gel electrode was placed behind the right ear as the reference electrode. f A schematic of representative EOG signals corresponding to different types of eye movement and blinking

Full size image

EOG measurement

EOG reflects the potential difference between the retina and the cornea of human eyes, which can be modeled as a dipole with a positive potential on the cornea and a negative potential on the retina. EOG signals can be measured by placing electrodes around the eyes as illustrated by Fig. 2e. Two GET sensors were placed next to the outer corner of each eye for the measurement of the lateral motion of the eyes. Another pair of GET sensors were placed at the top (above eyebrow) and bottom of one of the eyes to measure the vertical eye movement. Reference electrode was a gel electrode placed over the mastoid bone behind the right ear. A GET sensor located at the corner of the left eye was connected to the positive terminal of one channel of our recording system (OpenBCI) and the other one located at the corner of right eye was connected to the negative terminal of the same channel (Fig. 2e). This channel was used to record horizontal (i.e., left and right) eye movements. The other pair of electrodes located above the eyebrow and below the eye (Fig. 2e) was connected to another channel of OpenBCI system for recording vertical (i.e., up and down) eye movements. Figure 2f is a schematic drawing of representative EOG signals for eye movement in different directions. When the eyes move from the center toward one of the electrodes, a positive potential from the cornea is captured by the electrode closest to it and a negative potential of the retina is captured by the opposite electrode. This means if the eyes move to the left, the pair of horizontal electrodes would be able to capture a positive potential, and a negative potential appears by moving the eyes toward the right. Eyes moving upward produces a positive potential and looking downward results in a negative potential; blinks would be recorded as sharp positive peaks. The amplitude of EOG signals varies from case to case and is reported to be between 5 and 20 µV per degree of eye movement and in the low-frequency range up to 30 Hz.56 The cause of the blink signal is different from the EOG signal. EOG originates from potential difference between retina and cornea but blinks are due to activity of the levator palpe-brae superioris and the orbicularis oculi muscles located around the eyes. Blinks that start with upper eyelid moving downward followed by the orbicularis oculi muscle activity has the duration of 300–550 ms.52,62

To acquire EOG, GET sensors were connected to an OpenBCI Cyton Biosensing board, which can perform data acquisition (DAQ) at a sampling rate of 250 Hz and then wirelessly transfers EOG data to a laptop for signal processing and quadcopter control. Figure 3a shows a schematic of the complete HRI system. Flexible gold/polyethylene terephthalate (Au/PET) ribbons were fabricated by mechanically cutting and patterning commercially available Au/PET sheets (Rotex Tech.) to connect GET EOG sensors to thin lead wires fed into the OpenBCI (Figs. S4a–S4e). To reduce mechanical motion, the lead wires were fixed around ears using two earbud hooks (Fig. S4f). Figure 3b–d shows pictures of the wireless EOG sensor system worn by a subject. To guide the eye movement, we placed a signaling board with five individually controllable light-emitting diodes (LEDs) in front of the subject (Fig. S5). The distance from the board was adjusted to provide 45° of eyeball movement from center toward every direction (up, down, left, and right). To measure the EOG of a specific eye movement, the subject was asked to move his eyes toward the only LED that was on at a given time. The EOG signals recorded by GET EOG sensors during eye movement are shown in Fig. 3e, f. The amplitude of EOG signal recorded using GET was 17.7 ± 1.5 µV per degree in the horizontal direction and 12.8 ± 0.5 µV in the vertical direction, which are consistent with those reported in literature.56 We have observed clear voltage steps for successive eye movement of 4° in both horizontal and vertical directions as shown in supplementary Fig. S6.

Fig. 3
figure 3

Illustrations of GET EOG system worn by a subject and actual EOG signals recorded by GET EOG sensors. a A schematic of the GET EOG sensors connected to an OpenBCI board, which performs DAQ and wirelessly transmits EOG data to the laptop. The laptop performs data classification and wirelessly sends commands to the quadcopter. Photos of b front view, c side view and, d back view of the GET EOG sensor system worn on a subject. EOG signals recorded by GET sensors during the following repeated eye movement: e looking straight, right, and left each for few seconds, f looking straight, down, and up each for few seconds, g repeated intentional blinks

Full size image

Human–robot interface demonstration

The EOG signal during eye movement was utilized for human–robot interface. The flowchart of MATLAB coding used for EOG signal processing and classification of EOG signals for quadcopter control is shown in Fig. 4a. Prior to classification, baseline drift removal and data averaging were implemented. The baseline drift removal process involves averaging a sliding window of 250 samples to predict the drift, then removing that average value from the next data point of the window.63 To smooth out the signal, every ten samples were averaged. Rises and falls in the electric potential representing the right/left and up/down movement of the eyes can then be easily detected. Five separate thresholds were used in order to discretely detect blink (500 µV), right (400 µV), left (−400 µV), up (400 µV), and down (−400 µV) movement. The blink signals with amplitude higher than 500 µV were registered as Stay command. EOG signals due to up, down, left, and right eye movement were registered as commands to be sent to the quadcopter (Qualcomm Snapdragon) if their magnitudes were between 400 and 500 µV. During real-time quadcopter control, the maximum angle of eye movements in the vertical direction was about 70° and in the horizontal direction about 50°, as convenient for the human subject. The classification was performed by first computing the number of samples passing each unique threshold. The threshold met most frequently was given a unique label corresponding to that threshold. The label that occurred most often in a set of 1000 labels was chosen and transmitted to the robot-operating system (ROS). The label was converted to a command, and the command prompts the corresponding change in the position of the quadcopter. Figure 4b–f displays photos of a subject controlling a quadcopter by looking up, down, left, and right. The quadcopter followed the eye movement to fly up, down, left, and right. The flying distance of the quadcopter was pre-set in the ROS to be independent from the angle of eye movements. We only used the EOG signal to control the direction of the flight. Supplementary Video VS2 offers a real-time example of how the GET EOG sensor is used for HRI. We found the accuracy of our classifier to be 92 ± 2%.

Fig. 4
figure 4

Flowchart of the human–robot interface (HRI) used to control the quadcopter via eyeball movement. The EOG signals captured by sensors are sampled and transmitted to a laptop by the OpenBCI board. The raw data are first averaged and then passed onto the threshold detector to determine the eyeball movement direction and issues a command accordingly. From the command and the current location obtained from the motion capture system, the path planner calculates and sends a sequence of waypoints that the quadcopter needs to follow to the on-board position controller, which in turn sends attitude commands to the attitude controller (both controllers are part of the PX4 stack). The attitude controller then tilts the quadcopter so that it moves to the target waypoint which is then detected by Vicon motion capture system movement. b Quadcopter stayed stationary if subject did not move his eyes or if he blinked, c flew upward by looking up, d down by looking down, e right by looking toward the right and, f left by looking to the left. The insets in Fig. 4b to 4f show magnified photos of the human subjects eyes looking in different directions to control the flying of the quadcopter.

Full size image

Discussion

The results presented here is the proof of concept for ultrathin, ultrasoft, imperceptible GET sensors for recording EOG from the face and their potential application in HRI. The wearability of the GET sensor is considerably higher, and the manufacturing cost is significantly lower than existing e-tattoos with similar functionality. The primary consequence of the low thickness of GET is the conformal contact with skin texture and as a result, the improved SNR and decreased susceptibility to motion. Our previous work reported that the SNR of the signal measured recorded by the GET (15.22 dB) is slightly higher than the SNR of the signal recorded by commercial gel electrodes (11 dB).44 Flexible connectors and DAQ systems used in this work could be further improved. For example, using transparent conductive materials instead of gold/PET as the flexible connectors can further reduce the visibility of the system. Replacing commercially available OpenBCI board with customized, miniature-printed circuit board (PCB) that can directly communicate with the quadcopter will make the system even more mobile. Other 2D materials can potentially be integrated into the current GET platform for multimodal sensing, and on-tattoo signal amplification/processing.

Methods

Graphene growth and fabrication: Graphene was grown using atmospheric pressure chemical vapor deposition system (APCVD) on copper foil (OAK-MITSUI, 99.4%). The growth was done at ~1030 °C by flowing hydrogen at 10 sccm for 15 min, followed by flowing methane at 2 sccm and argon with 300 sccm for 10 min. Poly(methyl methacrylate) (PMMA 950 A4, MICRO-CHEM) was spin-coated on the graphene at 3000 rpm and cured at 180 °C for 2 min. After etching the copper using copper etchant (Transene Company INC.), Gr/PMMA was transferred on Silhouette tattoo paper. Using shadow mask, some area of Gr/PMMA was covered and then a Nexcard liquid bandage was sprayed on graphene (interconnect were formed from the area of graphene covered with liquid bandage). After liquid bandage was dried out shadow masks were removed. Then EOG GET was cut out of Gr/PMMA using Silhouette Cameo mechanical cutter plotter into pre-designed stretchable patterns. The extraneous area was then peeled off manually, leaving GET EOG sensor on the tattoo paper.

All experiments were conducted under approval from the Institutional Review Board at the University of Texas at Austin (protocol number: 2015-05-0024)

Data availability

All relevant data are available upon request.

Data acquisition system: OpenBCI’s Cyton Biosensing Board consists of an 8-channel neural interface, a 32 bit-processor, and a Bluetooth (RFduino) module. The board houses a PIC32MX250F128B microcontroller at the core, and the ADS1299 24-bit analog-to-digital converter with eight differential input channels. OpenBCI Cyton is actually one Python node in a larger collection of Python nodes, which work together in an ROS-based system.