Imperceptible electrooculography graphene sensor system for human–robot interface

Abstract

Electrooculography (EOG) is a method to record the electrical potential between the cornea and the retina of human eyes. Despite many applications of EOG in both research and medical diagnosis for many decades, state-of-the-art EOG sensors are still bulky, stiff, and uncomfortable to wear. Since EOG has to be measured around the eye, a prominent area for appearance with delicate skin, mechanically and optically imperceptible EOG sensors are highly desirable. Here, we report an imperceptible EOG sensor system based on noninvasive graphene electronic tattoos (GET), which are ultrathin, ultrasoft, transparent, and breathable. The GET EOG sensors can be easily laminated around the eyes without using any adhesives and they impose no constraint on blinking or facial expressions. High-precision EOG with an angular resolution of 4° of eye movement can be recorded by the GET EOG and eye movement can be accurately interpreted. Imperceptible GET EOG sensors have been successfully applied for human–robot interface (HRI). To demonstrate the functionality of GET EOG sensors for HRI, we connected GET EOG sensors to a wireless transmitter attached to the collar such that we can use eyeball movements to wirelessly control a quadcopter in real time.

Introduction

Wearable electronics have found many applications in fitness tracking, mobile health monitoring, medical diagnosis, human–machine interfaces (HMIs), assistive technology, internet of things (IoT), etc.1,2,3,4,5,6,7,8 Nevertheless, despite the growing needs, developing wearable devices that offer both functionality and wearability has remained a major challenge. In recent years, stretchable thin-film electronics and wearables, including epidermal sensors or electronic tattoos (e-tattoos), have achieved mechanical properties similar to human skin and demonstrated a wide range of functionalities, including physiological sensing,9,10,11,12,13,14 on-skin display,15 UV detection,16 transdermal therapeutics,17 HMIs,4,18 and prosthetic electronic skin.19 Compared with conventional wearable devices, a unique advantage of e-tattoos is their thinness and softness, which enables noninvasive but intimate coupling with microscopically rough human skin.20,21 Such conformability is confirmed to be responsible for the high signal-to-noise ratio (SNR) and low-motion artifacts of e-tattoo sensors.4,22,23 Although the concept of imperceptible electronics has been introduced for ultra-lightweight plastic electronics,24,25 none of them are optically transparent, which limits their wearability on areas of esthetic prominence such as the human face.

The development of optically transparent e-tattoos requires transparent conductive thin films26,27 such as 2D atomic sheets,28 nanocrystals-based composites,29,30 and intrinsically conductive transparent polymers.31 However, transparent conductive materials for wearable e-tattoos have to meet some additional criteria. For skin integration, these materials must be biocompatible and electrochemically inert. Furthermore, the thickness of conductors should be scalable to microns or even sub-microns to ensure conformability to skin.9,32 As a result, graphene and MoS2 nanomaterials are of significant relevance for e-tattoos due to their atomic thinness, optical transparency, electrochemical inertness, and biocompatibility.33,34,35,36,37,38 Recently, graphene-based wearable strain gauge, temperature sensor, heater, and interconnects have been reported.39,40 However, the combined thickness of the patch is few micrometers39 or hundreds of micrometers,40 which is too thick to fully conform to the micromorphology of human skin according to theoretical analysis and experimental validation.32 Monolayer graphene-based kirigami for stretchable electronics has been realized by other researcher. However, the application of such devices for on-skin electrophysiological measurements could be challenging since such devices need to be suspended in liquid or air to have 3D deformability. Furthermore, the use of laser beam to unfold and refold such structure is not realistic in living systems.41 Bilayer MoS2-based strain gauge and pressure sensor with graphene interconnects has been demonstrated with thickness below 75 nm.42 However, this device is flexible but not stretchable and it only measures mechanical strain. Also it is important to notice that since sub-micrometer-thick e-tattoos are difficult to reuse, low-cost, rapid prototyping and manufacturing process is desirable. The micro-transfer printing technology43 applied to fabricate the 2D materials-based skin patches39,40 is known to be cost and labor intensive owing to photolithography, chemical etching, and meticulous integration processes. To overcome such limitations, we recently developed a “wet transfer, dry patterning” method, which enable time and cost-effective freeform fabrication of 500-nm-thick graphene electronic tattoo (GET) sensors out of graphene/polymer bilayer.44 GET sensors were successfully applied for electrophysiology, skin hydration, and skin temperature measurements.44 However, since the previous GET sensors does not consist of any encapsulation over graphene, wherever it touches the skin, the average signal will be picked up. This induces limitation in application of GET for EOG in which the non-localized measurement is not desirable. We therefore have to modify the “wet transfer, dry patterning” process to achieve GET EOG sensors.

EOG has various applications including brain and sleep studies, assistive technology, sleep and mental disorder diagnosis, and HMIs.45,46,47,48,49 EOG along with electroencephalogram (EEG) and electromyogram (EMG) are used for patients suffering from neural system disorders such as progressive neuro-motor degenerative diseases. However, compared with EEG and EMG, EOG has two major advantages. First, analysis of the EOG signals is less complex because the relationship between EOG and eye movement is linear within a certain range, and the waveform is easy to detect. Second, EOG signal is more stable.50,51 Conventionally, silver/silver chloride (Ag/AgCl) gel electrodes are used for EOG measurement and even currently, most EOG measurements are measured using commercially available dry or wet silver/silver chloride electrodes.45,49,52,53,54 For example, Ubeda et al. measured EOG using dry silver–silver chloride electrodes and used the signal for controlling a robotic arm.55 In other works, EOG sensors were mounted on goggles for wearability.47,56,57,58 Vehkaoja et al. engineered silver-coated fibers as flexible EOG sensors, but they should be moisturized with conductive saline solution to enable recording.59 Although these electrodes are readily available and affordable, they are thick, stiff, uncomfortable to delicate eye skin, and noticeable on face. Wet electrodes are known for low electrode–skin interface impedance and low susceptibility to motion but gels are irritating to skin and dry out by time resulting in degrading the quality of the recorded signal. Dry electrodes are better option for long-term measurements but they suffer from high electrode–skin interface impedance due to non-conformable contact to skin and they are susceptible to motion artifact.60 Ultrathin and stretchable tattoo-like EOG sensors based on filamentary serpentine gold ribbons on polymer substrate have been reported.22 But gold-based electrodes and interconnects are very noticeable on prominent area such as human face. In other reported work, EOG sensor combined with other sensors and circuit components were integrated in a patch with the thickness of several millimeter.61 Such a patch is not only obvious, but also obstructive to facial expression. The GET EOG sensor system presented here has a thickness of 350 nm, optical transparency of 85% in the visible regime (wavelength from 400 to 800 nm), and stretchability up to 50%, which is truly mechanically and optically imperceptible EOG sensor. We found that the GET EOG sensors are capable of detecting eye movement with a resolution of about 4°. Connecting the imperceptible GET EOG sensors to an OpenBCI Cyton board capable of wireless communication, we demonstrate real-time wireless quadcopter control through eye movements.

Results

GET EOG sensors

The GET EOG sensor was fabricated out of large area graphene (Gr) grown on copper foil (Fig. 1a, b). Poly(methyl methacrylate) (PMMA) precursor was first coated and cured over the graphene to form a 350 nm film of PMMA (Fig. 1c). After etching away the copper (Fig. 1d), the Gr/PMMA bilayer was rinsed by deionized (DI) water and transferred onto a piece of commercial tattoo paper (Silhouette) with graphene facing up (Fig. 1e). To record EOG signals and interpret eye movement in the up, down, left, and right directions, a minimum of four EOG sensors are needed around the eyes. Each GET EOG sensor consists of three parts, (1) a disk-shaped electrode where graphene is exposed to make direct contact with the skin to capture the EOG signals, (2) an exposed rectangular graphene terminal pad to connect to flexible gold connector, and (3) an encapsulated serpentine-shaped interconnect linking parts (1) and (2). To encapsulate the serpentine interconnects, we applied a shadow mask made by a desktop mechanical cutter plotter (Silhouette) to cover the electrode and the terminal pad (Fig. 1f), followed by spray coating liquid bandage (Nexcare) to cover the exposed interconnects (Fig. 1g). Within 10–15 s, the liquid bandage solidified into an insulating polymeric layer of roughly 2 µm thickness (Fig. S2). Encapsulating interconnects allow for localized EOG measurement from the exposed disk-shaped electrodes right next to the eyes. After the encapsulation, we removed the shadow mask and used the same mechanical cutter plotter (Silhouette) to carve Gr/PMMA with pre-designed patterns (Fig. 1h). To show the necessity of encapsulation layer on interconnect for localized EOG measurements, we placed two electrodes on the top of an eyebrow of a human subject. One was placed vertically above an eyebrow (location B in Fig. S1a) and the other one at the inclined top, where GET interconnect normally passes through (location A in Fig. S1a). These electrodes work as two separate positive terminals of two channels of our recording system. We then placed another electrode at the bottom of the same eye (Fig. S1a), which acts as a shared electrode connected to the negative terminal of both channels. Then, we measured EOG signals simultaneously using both channels during eyes moving up and down. If EOG signal is naturally localized, there should be no recorded signal using the electrode placed in location A. However, clearly there is the nonzero EOG signal measured at location A as it is shown in Fig. S1b. The EOG recorded at location A is in phase with but has smaller amplitude than that measured at location B. Therefore, covering interconnects with encapsulation layer is necessary for localized EOG measurements. The GET EOG sensors were designed in the serpentine-shaped ribbons to enable stretchability. The stretchability of the GET was measured in our previous to be about 50%.44 In the last step of EOG GET fabrication, extraneous Gr/PMMA was carefully peeled off, leaving GET EOG sensors on the tattoo paper (Fig. 1i). A layer of water-soluble adhesive comes with the tattoo paper can facilitate the release of GET EOG. To laminate GET EOG around the eyes, the tattoo paper was placed on skin with GET EOG facing the skin, followed by wetting the backside of the tattoo paper and then carefully peeling the paper off the skin, leaving open-mesh GET EOG sensors on the skin (Fig. 1j and Supplementary Video VS1).

Fig. 1
figure1

Schematics of the fabrication process for imperceptible GET EOG sensor system. a Growing graphene on copper foil using atmospheric pressure chemical vapor deposition (APCVD). b As-grown graphene on copper foil. c Spin coating and curing 350-nm-thick PMMA on graphene. d Copper etching and rinsing graphene/PMMA (Gr/PMMA) with DI water. e Transferring Gr/PMMA bilayer on a tattoo paper with graphene facing up. f Covering Gr/PMMA by a shadow mask. g Spray coating liquid bandage over the exposed interconnect area. h Removing shadow mask and carving Gr/PMMA into serpentine-shaped patterns using a mechanical cutter plotter. i Pealing off extraneous Gr/PMMA. j Transferring GET EOG sensor on skin surface just like wearing a temporary tattoo

The GET EOG sensors formed steady contact with human skin without any tape or adhesive (Fig. S3), as predicted by our previous analytical models.32 Photos of GET EOG applied around eyes under different facial expressions are displayed in Fig. 2a–d. It is evident that the GET is hardly visible in all cases. Under extreme facial expressions, neither delamination nor cracking of the GET EOG was observed and the subject did not perceive the GET sensors at all. To study the effect of extreme facial expression on sensor electrical characteristics, the end-to-end electrical resistance of one laminated electrode around the eye was measured before and after extreme face expressions. We found negligible change (<0.5%) after a short period (2 min) of extreme facial expressions. However, after an hour of speaking and abundant facial expressions, including laughing, frowning, sour face, and surprise, the electrical resistance increased by 4%.

Fig. 2
figure2

Imperceptible GET EOG sensors worn around human eyes. a Without facial expression. b Sour face. c Laughing. d Frowning. No delamination nor crack was observed in GET after extreme facial expressions. e A schematic for the locations of GET EOG sensors laminated on face. A Ag/AgCl gel electrode was placed behind the right ear as the reference electrode. f A schematic of representative EOG signals corresponding to different types of eye movement and blinking

EOG measurement

EOG reflects the potential difference between the retina and the cornea of human eyes, which can be modeled as a dipole with a positive potential on the cornea and a negative potential on the retina. EOG signals can be measured by placing electrodes around the eyes as illustrated by Fig. 2e. Two GET sensors were placed next to the outer corner of each eye for the measurement of the lateral motion of the eyes. Another pair of GET sensors were placed at the top (above eyebrow) and bottom of one of the eyes to measure the vertical eye movement. Reference electrode was a gel electrode placed over the mastoid bone behind the right ear. A GET sensor located at the corner of the left eye was connected to the positive terminal of one channel of our recording system (OpenBCI) and the other one located at the corner of right eye was connected to the negative terminal of the same channel (Fig. 2e). This channel was used to record horizontal (i.e., left and right) eye movements. The other pair of electrodes located above the eyebrow and below the eye (Fig. 2e) was connected to another channel of OpenBCI system for recording vertical (i.e., up and down) eye movements. Figure 2f is a schematic drawing of representative EOG signals for eye movement in different directions. When the eyes move from the center toward one of the electrodes, a positive potential from the cornea is captured by the electrode closest to it and a negative potential of the retina is captured by the opposite electrode. This means if the eyes move to the left, the pair of horizontal electrodes would be able to capture a positive potential, and a negative potential appears by moving the eyes toward the right. Eyes moving upward produces a positive potential and looking downward results in a negative potential; blinks would be recorded as sharp positive peaks. The amplitude of EOG signals varies from case to case and is reported to be between 5 and 20 µV per degree of eye movement and in the low-frequency range up to 30 Hz.56 The cause of the blink signal is different from the EOG signal. EOG originates from potential difference between retina and cornea but blinks are due to activity of the levator palpe-brae superioris and the orbicularis oculi muscles located around the eyes. Blinks that start with upper eyelid moving downward followed by the orbicularis oculi muscle activity has the duration of 300–550 ms.52,62

To acquire EOG, GET sensors were connected to an OpenBCI Cyton Biosensing board, which can perform data acquisition (DAQ) at a sampling rate of 250 Hz and then wirelessly transfers EOG data to a laptop for signal processing and quadcopter control. Figure 3a shows a schematic of the complete HRI system. Flexible gold/polyethylene terephthalate (Au/PET) ribbons were fabricated by mechanically cutting and patterning commercially available Au/PET sheets (Rotex Tech.) to connect GET EOG sensors to thin lead wires fed into the OpenBCI (Figs. S4a–S4e). To reduce mechanical motion, the lead wires were fixed around ears using two earbud hooks (Fig. S4f). Figure 3b–d shows pictures of the wireless EOG sensor system worn by a subject. To guide the eye movement, we placed a signaling board with five individually controllable light-emitting diodes (LEDs) in front of the subject (Fig. S5). The distance from the board was adjusted to provide 45° of eyeball movement from center toward every direction (up, down, left, and right). To measure the EOG of a specific eye movement, the subject was asked to move his eyes toward the only LED that was on at a given time. The EOG signals recorded by GET EOG sensors during eye movement are shown in Fig. 3e, f. The amplitude of EOG signal recorded using GET was 17.7 ± 1.5 µV per degree in the horizontal direction and 12.8 ± 0.5 µV in the vertical direction, which are consistent with those reported in literature.56 We have observed clear voltage steps for successive eye movement of 4° in both horizontal and vertical directions as shown in supplementary Fig. S6.

Fig. 3
figure3

Illustrations of GET EOG system worn by a subject and actual EOG signals recorded by GET EOG sensors. a A schematic of the GET EOG sensors connected to an OpenBCI board, which performs DAQ and wirelessly transmits EOG data to the laptop. The laptop performs data classification and wirelessly sends commands to the quadcopter. Photos of b front view, c side view and, d back view of the GET EOG sensor system worn on a subject. EOG signals recorded by GET sensors during the following repeated eye movement: e looking straight, right, and left each for few seconds, f looking straight, down, and up each for few seconds, g repeated intentional blinks

Human–robot interface demonstration

The EOG signal during eye movement was utilized for human–robot interface. The flowchart of MATLAB coding used for EOG signal processing and classification of EOG signals for quadcopter control is shown in Fig. 4a. Prior to classification, baseline drift removal and data averaging were implemented. The baseline drift removal process involves averaging a sliding window of 250 samples to predict the drift, then removing that average value from the next data point of the window.63 To smooth out the signal, every ten samples were averaged. Rises and falls in the electric potential representing the right/left and up/down movement of the eyes can then be easily detected. Five separate thresholds were used in order to discretely detect blink (500 µV), right (400 µV), left (−400 µV), up (400 µV), and down (−400 µV) movement. The blink signals with amplitude higher than 500 µV were registered as Stay command. EOG signals due to up, down, left, and right eye movement were registered as commands to be sent to the quadcopter (Qualcomm Snapdragon) if their magnitudes were between 400 and 500 µV. During real-time quadcopter control, the maximum angle of eye movements in the vertical direction was about 70° and in the horizontal direction about 50°, as convenient for the human subject. The classification was performed by first computing the number of samples passing each unique threshold. The threshold met most frequently was given a unique label corresponding to that threshold. The label that occurred most often in a set of 1000 labels was chosen and transmitted to the robot-operating system (ROS). The label was converted to a command, and the command prompts the corresponding change in the position of the quadcopter. Figure 4b–f displays photos of a subject controlling a quadcopter by looking up, down, left, and right. The quadcopter followed the eye movement to fly up, down, left, and right. The flying distance of the quadcopter was pre-set in the ROS to be independent from the angle of eye movements. We only used the EOG signal to control the direction of the flight. Supplementary Video VS2 offers a real-time example of how the GET EOG sensor is used for HRI. We found the accuracy of our classifier to be 92 ± 2%.

Fig. 4
figure4

Flowchart of the human–robot interface (HRI) used to control the quadcopter via eyeball movement. The EOG signals captured by sensors are sampled and transmitted to a laptop by the OpenBCI board. The raw data are first averaged and then passed onto the threshold detector to determine the eyeball movement direction and issues a command accordingly. From the command and the current location obtained from the motion capture system, the path planner calculates and sends a sequence of waypoints that the quadcopter needs to follow to the on-board position controller, which in turn sends attitude commands to the attitude controller (both controllers are part of the PX4 stack). The attitude controller then tilts the quadcopter so that it moves to the target waypoint which is then detected by Vicon motion capture system movement. b Quadcopter stayed stationary if subject did not move his eyes or if he blinked, c flew upward by looking up, d down by looking down, e right by looking toward the right and, f left by looking to the left. The insets in Fig. 4b to 4f show magnified photos of the human subjects eyes looking in different directions to control the flying of the quadcopter.

Discussion

The results presented here is the proof of concept for ultrathin, ultrasoft, imperceptible GET sensors for recording EOG from the face and their potential application in HRI. The wearability of the GET sensor is considerably higher, and the manufacturing cost is significantly lower than existing e-tattoos with similar functionality. The primary consequence of the low thickness of GET is the conformal contact with skin texture and as a result, the improved SNR and decreased susceptibility to motion. Our previous work reported that the SNR of the signal measured recorded by the GET (15.22 dB) is slightly higher than the SNR of the signal recorded by commercial gel electrodes (11 dB).44 Flexible connectors and DAQ systems used in this work could be further improved. For example, using transparent conductive materials instead of gold/PET as the flexible connectors can further reduce the visibility of the system. Replacing commercially available OpenBCI board with customized, miniature-printed circuit board (PCB) that can directly communicate with the quadcopter will make the system even more mobile. Other 2D materials can potentially be integrated into the current GET platform for multimodal sensing, and on-tattoo signal amplification/processing.

Methods

Graphene growth and fabrication: Graphene was grown using atmospheric pressure chemical vapor deposition system (APCVD) on copper foil (OAK-MITSUI, 99.4%). The growth was done at ~1030 °C by flowing hydrogen at 10 sccm for 15 min, followed by flowing methane at 2 sccm and argon with 300 sccm for 10 min. Poly(methyl methacrylate) (PMMA 950 A4, MICRO-CHEM) was spin-coated on the graphene at 3000 rpm and cured at 180 °C for 2 min. After etching the copper using copper etchant (Transene Company INC.), Gr/PMMA was transferred on Silhouette tattoo paper. Using shadow mask, some area of Gr/PMMA was covered and then a Nexcard liquid bandage was sprayed on graphene (interconnect were formed from the area of graphene covered with liquid bandage). After liquid bandage was dried out shadow masks were removed. Then EOG GET was cut out of Gr/PMMA using Silhouette Cameo mechanical cutter plotter into pre-designed stretchable patterns. The extraneous area was then peeled off manually, leaving GET EOG sensor on the tattoo paper.

All experiments were conducted under approval from the Institutional Review Board at the University of Texas at Austin (protocol number: 2015-05-0024)

Data availability

All relevant data are available upon request.

Data acquisition system: OpenBCI’s Cyton Biosensing Board consists of an 8-channel neural interface, a 32 bit-processor, and a Bluetooth (RFduino) module. The board houses a PIC32MX250F128B microcontroller at the core, and the ADS1299 24-bit analog-to-digital converter with eight differential input channels. OpenBCI Cyton is actually one Python node in a larger collection of Python nodes, which work together in an ROS-based system.

References

  1. 1.

    Bareket, L. et al. Temporary-tattoo for long-term high fidelity biopotential recordings. Sci. Rep. 6, 25727 (2016).

    Article  Google Scholar 

  2. 2.

    Huang, X., Yeo, W. H., Liu, Y. & Rogers, J. A. Epidermal differential impedance sensor for conformal skin hydration monitoring. Biointerphases 7, 52 (2012).

    Article  Google Scholar 

  3. 3.

    Kenry, Yeo, J. C. & Lim, C. T. Emerging flexible and wearable physical sensing platforms for healthcare and biomedical applications. Microsyst. Nanoeng. 2, 16043 (2016).

  4. 4.

    Jeong, J. W. et al. Materials and optimized designs for human-machine interfaces via epidermal electronics. Adv. Mater. 25, 6839–6846 (2013).

    Article  Google Scholar 

  5. 5.

    Gao, W. et al. Fully integrated wearable sensor arrays for multiplexed in situ perspiration analysis. Nature 529, 509–514 (2016).

    Article  Google Scholar 

  6. 6.

    Nyein, H. Y. et al. A wearable electrochemical platform for noninvasive simultaneous monitoring of Ca(2+) and pH. ACS Nano 10, 7216–7224 (2016).

    Article  Google Scholar 

  7. 7.

    Metcalf, D., Milliard, S. T., Gomez, M. & Schwartz, M. Wearables and the internet of things for health: wearable, interconnected devices promise more efficient and comprehensive health care. IEEE Pulse 7, 35–39 (2016).

    Article  Google Scholar 

  8. 8.

    Lo, B. P. L., Ip, H. & Yang, G. Z. Transforming health care: body sensor networks, wearables, and the internet of things. IEEE Pulse 7, 4–8 (2016).

    Article  Google Scholar 

  9. 9.

    Kim, D.-H. et al. Epidermal electronics. Science 333, 838–843 (2011).

    Article  Google Scholar 

  10. 10.

    Bandodkar, A. J., Jia, W. & Wang, J. Tattoo-based wearable electrochemical devices: a review. Electroanal 27, 11 (2015).

    Article  Google Scholar 

  11. 11.

    Webb, R. C. B. et al. Ultrathin conformal devices for precise and continuous thermal characterization of human skin. Nat. Mater. 12, 938–944 (2013).

    Article  Google Scholar 

  12. 12.

    Kim, J. et al. Battery-free, stretchable optoelectronic systems for wireless optical characterization of the skin. Sci. Adv. 2, e1600418 (2016).

    Article  Google Scholar 

  13. 13.

    Jang, K. I. et al. Rugged and breathable forms of stretchable electronics with adherent composite substrates for transcutaneous monitoring. Nat. Commun. 5, 4779 (2014).

    Article  Google Scholar 

  14. 14.

    Xu, S. et al. Soft microfluidic assemblies of sensors, circuits, and radios for the skin. Science 344, 70–74 (2014).

    Article  Google Scholar 

  15. 15.

    Koo, J. H. et al. Wearable electrocardiogram monitor using carbon nanotube electronics and color-tunable organic light-emitting diodes. ACS Nano 11, 10032–10041 (2017).

    Article  Google Scholar 

  16. 16.

    Araki, H. et al. Materials and device designs for an epidermal UV colorimetric dosimeter with near field communication capabilities. Adv. Funct. Mater. 27, 1604465 (2017).

    Article  Google Scholar 

  17. 17.

    Son, D. et al. Multifunctional wearable devices for diagnosis and therapy of movement disorders. Nat. Nanotechnol. 9, 397–404 (2014).

    Article  Google Scholar 

  18. 18.

    Song, J.-K. et al. Wearable force touch sensor array using a flexible and transparent electrode. Adv. Funct. Mater. 27, 1605286 (2017).

    Article  Google Scholar 

  19. 19.

    Chortos, A., Liu, J. & Bao, Z. Pursuing prosthetic electronic skin. Nat. Mater. 15, 937–950 (2016).

    Article  Google Scholar 

  20. 20.

    Wang, S. et al. Mechanics of epidermal electronics. J. Appl. Mech. 79, 031022 (2012).

    Article  Google Scholar 

  21. 21.

    Wang, L. & Lu, N. Conformability of a thin elastic membrane laminated on a soft substrate with slightly wavy surface. J. Appl. Mech. 83, 041007 (2016).

    Article  Google Scholar 

  22. 22.

    Jeong, J. W. et al. Capacitive epidermal electronics for electrically safe, long-term electrophysiological measurements. Adv. Healthc. Mater. 3, 642–648 (2014).

    Article  Google Scholar 

  23. 23.

    Wang, Y. et al. Low-cost, μm-thick, tape-free electronic tattoo sensors with minimized motion and sweatartifacts. NPJ Flexible Electronics 6, 1–6 (2018).

    Google Scholar 

  24. 24.

    Kaltenbrunner, M. et al. An ultra-lightweight design for imperceptible plastic electronics. Nature 499, 458–463 (2013).

    Article  Google Scholar 

  25. 25.

    Melzer, M. et al. Imperceptible magnetoelectronics. Nat. Commun. 6, 6080 (2015).

    Article  Google Scholar 

  26. 26.

    Hecht, D. S., Hu, L. & Irvin, G. Emerging transparent electrodes based on thin films of carbon nanotubes, graphene, and metallic nanostructures. Adv. Mater. 23, 1482–1513 (2011).

    Article  Google Scholar 

  27. 27.

    Ellmer, K. Past achievements and future challenges in the development of optically transparent electrodes. Nat. Photonics 6, 809 (2012).

    Article  Google Scholar 

  28. 28.

    Akinwande, D. et al. A review on mechanics and mechanical properties of 2D materials—graphene and beyond. Extrem. Mech. Lett. 13, 42–77 (2017).

    Article  Google Scholar 

  29. 29.

    Xue, J. et al. Nanowire-based transparent conductors for flexible electronics and optoelectronics. Sci. Bull. 62, 143–156 (2017).

    Article  Google Scholar 

  30. 30.

    Song, J. & Zeng, H. Transparent electrodes printed with nanocrystal inks for flexible smart devices. Angew. Chem. 54, 9760–9774 (2015).

    Article  Google Scholar 

  31. 31.

    Wang, Y. et al. A highly stretchable, transparent, and conductive polymer. Sci. Adv. 3, e1602076 (2017).

    Article  Google Scholar 

  32. 32.

    Wang, L., Qiao, S., Ameri, S. K., Jeong, H. & Lu, N. A thin elastic membrane conformed to a soft and rough substrate subjected to stretching/compression. J. Appl. Mech. 84, 111003 (2017).

    Article  Google Scholar 

  33. 33.

    Shah, P., Narayanan, T. N., Li, C. Z. & Alwarappan, S. Probing the biocompatibility of MoS2 nanosheets by cytotoxicity assay and electrical impedance spectroscopy. Nanotechnology 26, 315102 (2015).

    Article  Google Scholar 

  34. 34.

    Song, I., Park, C. & Choi, H. C. Synthesis and properties of molybdenum disulphide: from bulk to atomic layers. RSC Adv. 5, 7495–7514 (2015).

    Article  Google Scholar 

  35. 35.

    Ameri, S. K. et al. Three dimensional graphene scaffold for cardiac tissue engineering and in-situ electrical recording. IEEE EMBC 3, 4201–4203 (2016).

    Google Scholar 

  36. 36.

    Ameri, S. K., Singh, P. K. & Sonkusale, S. Utilization of graphene electrode in transparent microwell arrays for high throughput cell trapping and lysis. Biosens. Bioelectron. 61, 625–630 (2014).

    Article  Google Scholar 

  37. 37.

    Na, S. R. et al. Clean graphene interfaces by selective dry transfer for large area silicon integration. Nanoscale 8, 7523–7533 (2016).

    Article  Google Scholar 

  38. 38.

    Pinto, A. M., Goncalves, I. C. & Magalhaes, F. D. Graphene-based materials biocompatibility: a review. Colloids Surf. B Biointerfaces 111, 188–202 (2013).

    Article  Google Scholar 

  39. 39.

    Choi, M. K. et al. Thermally controlled, patterned graphene transfer printing for transparent and wearable electronic/optoelectronic system. Adv. Funct. Mater. 25, 7109–7118 (2015).

    Article  Google Scholar 

  40. 40.

    Lee, H. et al. A graphene-based electrochemical device with thermoresponsive microneedles for diabetes monitoring and therapy. Nat. Nanotechnol. 11, 566–572 (2016).

    Article  Google Scholar 

  41. 41.

    Blees, M. K. et al. Graphene kirigami. Nature 524, 204–207 (2015).

    Article  Google Scholar 

  42. 42.

    Park, M. et al. MoS2-based tactile sensor for electronic skin applications. Adv. Mater. 28, 2556–2562 (2016).

    Article  Google Scholar 

  43. 43.

    Meitl, M. A. et al. Transfer printing by kinetic control of adhesion to an elastomeric stamp. Nat. Mater. 5, 33–38 (2006).

    Article  Google Scholar 

  44. 44.

    Ameri, S. K. et al. Graphene electronic tattoo sensors. ACS nano 11, 7634–7641 (2017).

    Article  Google Scholar 

  45. 45.

    Boukadoum, A. M. & Ktonas, P. Y. EOG-based recording and automated detection of sleep rapid eye movements: a critical review, and some recommendations. Psychophysiology 23, 598–611 (1986).

    Article  Google Scholar 

  46. 46.

    Ding, Q., Tong, K. & Li, G. Development of an EOG (electro-oculography) based human-computer interface. Int. Conf. IEEE Eng. Med. Biol. 7, 6829–6831 (2005).

    Google Scholar 

  47. 47.

    Nam, Y., Koo, B., Cichocki, A. & Choi, S. GOM-Face: GKP, EOG, and EMG-based multimodal interface with application to humanoid robot control. IEEE Trans. Bio Med. Eng. 61, 453–462 (2014).

    Article  Google Scholar 

  48. 48.

    Rac-Lubashevsky, R., Slagter, H. A. & Kessler, Y. Tracking real-time changes in working memory updating and gating with the event-based eye-blink rate. Sci. Rep. 7, 2547 (2017).

    Article  Google Scholar 

  49. 49.

    Lam, R. W., Beattie, C. W., Buchanan, A., Remick, R. A. & Zis, A. P. Low electrooculographic ratios in patients with seasonal affective disorder. Am. J. Psychiatry 148, 1526–1529 (1991).

    Article  Google Scholar 

  50. 50.

    Siddiqui, U. & Shaikh, A. N. An overview of “electrooculography”. Int. J. Adv. Res. Comput. Commun. Eng. 2, 3 (2013).

    Google Scholar 

  51. 51.

    Dey, N. Classification and Clustering In Biomedical Signal Processing (IGI Global, Hershey, PA, 2016).

  52. 52.

    Bour, L., Visser, O. B., Aramideh, M. & Speelman, J. Origin of eye and eyelid movements during blinking. Mov. Disord. 17, S30–S32 (2002).

    Article  Google Scholar 

  53. 53.

    Yamagishi, K., Hori, J. & Miyakawa, M. Development of EOG-based communication system controlled by eight-directional eye movements. Int. Conf. IEEE Eng. Med. Biol. 1, 2574–2577 (2006).

    Google Scholar 

  54. 54.

    Magosso, E., Ursino, M., Zaniboni, A., Provini, F. & Montagna, P. Visual and computer-based detection of slow eye movements in overnight and 24-h EOG recordings. Clin. Neurophysiol. 118, 1122–1133 (2007).

    Article  Google Scholar 

  55. 55.

    Ubeda, A., Ianez, E. & Azorın, M. J. Wireless and portable EOG-based interface for assisting disabled people. IEEE/ASME Trans. Mechatron. 6, 4 (2011).

    Google Scholar 

  56. 56.

    Bulling, A., Ward, J. A., Gellersen, H. & Troster, G. Eye movement analysis for activity recognition using electrooculography. IEEE Trans. Pattern Anal. Mach. Intell. 33, 741–753 (2011).

    Article  Google Scholar 

  57. 57.

    Mélodie, J. T. V., Bulling, A. & Gellersen, H. Wearable eye tracking for mental health monitoring. Comput. Commun. 35, 6 (2012).

    Google Scholar 

  58. 58.

    Barea, R., Boquete, L., Rodriguez-Ascariz, J. M., Ortega, S. & Lopez, E. Sensory system for implementing a human-computer interface based on electrooculography. Sensors 11, 310–328 (2011).

    Article  Google Scholar 

  59. 59.

    Vehkaoja, A. T. et al. Wireless head cap for EOG and facial EMG measurements. Int. Conf. IEEE Eng. Med. Biol. 6, 5865–5868 (2005).

    Google Scholar 

  60. 60.

    Searle, A. & Kirkup, L. A direct comparison of wet, dry and insulating bioelectric recording electrodes. Physiol. Meas. 21, 271–283 (2000).

    Article  Google Scholar 

  61. 61.

    Jang, K. I. et al. Self-assembled three dimensional network designs for soft electronics. Nat. Commun. 8, 15894 (2017).

    Article  Google Scholar 

  62. 62.

    Merino, M., Gomez, I. M. & Molina, A. J. Envelope filter sequence to delete blinks and overshoots. BioMed. Eng. 14, 48 (2015).

    Google Scholar 

  63. 63.

    Baek, C., Goo, Y. & Seo, J.-M. Real-time baseline wander removal from electrooculography using probabilistic baseline prediction. MBEC 45, 78–81 (2014).

    Google Scholar 

Download references

Acknowledgements

This work is supported by the US National Science Foundation (NSF) under Grant Nos. ECCS-1541684 and CNS-1738293. N.L. is very thankful to the support from the Young Investigator Program (YIP) of Office of Naval Research (ONR) under Grant No. N00014-16-1-2044. D.A. acknowledges the support of David & Doris Lybarger Endowed Faculty Fellowship and ONR.

Author information

Affiliations

Authors

Contributions

S.K.A. designed and fabricated the EOG sensor and system, the EOG recording and HRI experiments, and contributed in data processing. M.K. and I.K. contributed in fabricating devices, I.K. contributed in signal processing and performing EOG recording. W.P. and M.A. contributed in the writing codes for HRI and ROS. H.J. contributed in HRI experiment. U.T. supervised HRI experiment and N. L. and D.A. supervised the research.

Corresponding authors

Correspondence to Deji Akinwande or Nanshu Lu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Ameri, S.K., Kim, M., Kuang, I.A. et al. Imperceptible electrooculography graphene sensor system for human–robot interface . npj 2D Mater Appl 2, 19 (2018). https://doi.org/10.1038/s41699-018-0064-4

Download citation

Further reading