An extremely simple macroscale electronic skin realized by deep machine learning

Complicated structures consisting of multi-layers with a multi-modal array of device components, i.e., so-called patterned multi-layers, and their corresponding circuit designs for signal readout and addressing are used to achieve a macroscale electronic skin (e-skin). In contrast to this common approach, we realized an extremely simple macroscale e-skin only by employing a single-layered piezoresistive MWCNT-PDMS composite film with neither nano-, micro-, nor macro-patterns. It is the deep machine learning that made it possible to let such a simple bulky material play the role of a smart sensory device. A deep neural network (DNN) enabled us to process electrical resistance change induced by applied pressure and thereby to instantaneously evaluate the pressure level and the exact position under pressure. The great potential of this revolutionary concept for the attainment of pressure-distribution sensing on a macroscale area could expand its use to not only e-skin applications but to other high-end applications such as touch panels, portable flexible keyboard, sign language interpreting globes, safety diagnosis of social infrastructures, and the diagnosis of motility and peristalsis disorders in the gastrointestinal tract.


2
Table of Contents S1. DNN architectures along with the test result for 6 X 6-lattice sheets S1-1. DNN for position recognition S1-2. DNN for pressure prediction S1-3. Regression result for pressure sensing (16 features     S3. DNN with 16 features and 100 labels for 10 X 10-lattice sheets S3-1 DNN for position recognition Figure S4. The DNN architecture with 16 features and 100 labels for position recognition in 10 X 10 lattice e-skin sheet with a size of 40 mm X 40 mm.  representative sectors in 10 X 10 lattice e-skin sheet with a size of 40 mm X 40 mm. Table S4. Selected sectors only were selected for use in the pressure evaluation for brevity. The sectors were selected, such that a certain symmetry pattern can be generated in the original lattice matrix. Also, the pressure evaluation on the other sectors should be definitely successful.   Aside from the conventional classification DNN models dealt with in the manuscript, we developed a regression-typed DNN model to predict exact touch points rather than the identification of a sector number out of the pre-designed matrix. In this case the output is two real values designating the real coordinates (x and y) on the e-skin sheet area. Figure S9 shows the DNN architecture.
When compared to EIT-driven sensor, the resolution error much smaller. Also it should be noted that that the amount of data used for the estimation of the resolution and error values are much greater in comparison to the EIT case wherein only a touch per each position was presented. Figure S6.

S6. The comparison between DNN-and EIT-driven sensors
Lee et al.* has very recently reported another type of pattern free pressure sensor based on the electrical impedance tomography (EIT). The EIT approach is a model-based numerical process and has been used for many materials for years. Although the material and the training specimen for use in their approach was similar to ours, the performance of our DNN-driven e-skin has proven to be tremendously superior to the EIT-driven e-skin in terms of the pressure sensitivity, spatial resolution, response time, pressure sensing range, and more importantly the similarity to human skin. Such superiorities, in particular, the spatial resolution and pressure sensitivity, were reasonably validated here.
We adopted a pick-block type touch position recognition to evaluate the performance of DNN-driven sensors, while the EIT-driven sensor was evaluated by the Euclidian distance between the predicted and real position of touches. Both the evaluation systems cannot be compared reasonably due to the difference in operation system. Therefore, the accuracy (or error) estimation manner was transformed to one another, such that a reasonable comparison could be available.
Also, we introduced a regression-typed DNN and an independent experiment was performed for training and test for this regression-typed DNN. As a result, we achieved a real number regression of spatial coordinates (x and y) based on the Euclidian distance between the predicted and real position of touches. The test accuracy in the form of actual distance was estimated to be about 0.78 ± 0.44 mm, which is far smaller than reported values (please see subsection S6-2)  Figure S7. The group-data-based test accuracy for DNN-driven sensor was consistently 100% for every lattice design (4 X 4, 6 X 6, and 10 X 10 sheets with size of 40 X 40 mm). In contrast, the accuracy for EIT-driven sensor sheet with a 7 X 7 lattice for almost the same size was reported to be 89.89. In addition, when this 7 X 7 lattice result is converted to secure an equivalent comparison condition, the accuracy was estimated to be 91.84 and 59.18 for conversions into 6 X 6 and 10 X 10 lattice cases, respectively. Slightly bigger and smaller squares than the background lattice represent the converted spatial resolution, and outliers located out of these converted resolution were counted for the accuracy calculations. Note that the DNN-driven sensor has proven to outperform the EIT-driven one in terms of the spatial resolution. Figures re-used from ref. [15].

S6-2. Evaluation of test accuracy for the regression-typed DNN-driven sensor
The test accuracy for the regression-typed DNN was estimated from the Euclidian distance between the predicted and real position of touches. The details on the regression-typed DNN architecture was well described in the section S4.