Introduction

Electronic skin is an important environmental detector in artificial robots, which needs to be sensitive to a variety of tactile signals1,2,3,4,5,6,7, including temperature8, humidity9, and especially stress forces10,11,12,13,14. A traditional approach is to use a combination of different types of sensor array to directly measure multiple physical information15,16,17,18,19, which ensures that the signals do not interfere with each other. In order to obtain more accurate physical information, it needs to rely on a greater density and a wider variety of sensors20,21. However, the complex manufacturing process and restricted sensor types limit its application22. Another type of solution is creating and detecting fields of tactile relevant physical quantities, including stress fields23,24,25,26,27, temperature fields28, magnetic fields29, etc. The continuity of the field in time and space allows us to fit and reproduce the whole field using limited data, thus making more accurate predictions with small cost. Previous studies have used optical method to detect changes in stress field, which can obtain the shape and hardness of objects in contact with the sensor, however, limited by the size and rigid property of camera, the optical method is difficult applied in complex curved and stretchable environments23. Another study cleverly used the sensing material itself as a carrier of the stress and temperature fields. By analyzing the spatial distribution of these two fields, they can identify various tactile motions28. Still this approach can only give rough classification, not accurate physical results, and it also was not in accordance with human habits.

As a delicate and powerful tactile organ, skin allows human to respond quickly and interact precisely with surroundings. Due to the soft and stretchable nature of skin, it transforms complex external contact forces into its own deformation, constituting an internal stress field (Fig. 1) and precisely perceives the field through various tactile receptors30,31. This approach to field perception places high demands on tactile receptors inside skin to work stably under large deformation and stress. For example, Meissner corpuscle, one of the tactile receptors in the epidermis of skin, consist of nerve endings, Schwann cells, and connective tissue32. The coiled nerve endings together with the flattened Schwann cells form sensitive mechanoreceptors and the surrounding connective tissue wrapped them very tightly, ensuring that the tactile corpuscle work stably even under large deformation and stress in the skin.

Fig. 1: Concept of stress field sensing and bio-mimicking of Meissner corpuscle structures.
figure 1

Left side of the figure shows the construction of human skin. A large number of Meissner corpuscles, Merkel discs, and tactile nerve endings are distributed in the epidermis of the skin, while Ruffini corpuscles and Pacinian corpuscles are distributed in the dermis. When external force is present, the skin undergoes corresponding deformations that change the surrounding environment of the tactile corpuscles and nerve endings. The nerve signals characterize the stress field of the soft skin tissue, which reflects the external force. Based on this stress field sensing concept, we propose the BMRs idea as shown in right side of the figure. An extremely soft Ecoflex Gel is used as a deformation layer in the middle to act as the carrier of the stress field. The ‘epidermal’ sensing array and ‘dermal’ sensing array are used to sample the pressure distribution on the top and bottom surface of the deformation layer, respectively. The microstructure of the Meissner corpuscles is shown in the lower left block diagram. Tissues wrapping around the Meissner corpuscles help them work stably at large deformation. Inspired from this special structure, the sensing units are isolated from each other with silicone structure as shown in lower right block diagram. They are suspended and protected in separate cells.

Full size image

Inspired by how human skin sensing the mechanical stimuli and how the structures of skin support the stress field sensing principle, we propose in this work a design of electronic skin, called biomimetic mechanoreceptors (BMRs), with bio-mimicking, simple structure, stable operation and multi-parameter perception based on the stress field reconstruction. The BMRs consist of a skin-like deformation layer sandwiched by an ‘epidermal’ sensing array and a ‘dermal’ sensing array, mimicking the structures of human skin. The deformation layer is made of an elastomer. As the carrier for the stress field, it is easy to undergo deformation once external pressure is applied. The sensing arrays contain pressure detecting units that imitate the structure of Meissner corpuscles. They sample the stress field distribution on the top and bottom of the deformation layer. By reconstructing and analyzing the stress field, the BMRs can measure three-dimensional forces precisely and distinguish objects with different hardness, nullifying the needs for integrating multiple sensors to directly measure the corresponding tactile signals. Because the change of stress field is only affected by external force and do not rely on sampling location, this electronic skin can be fabricated with facile methods and easily integrated on curved surfaces for practical tactile tasks. More importantly, the principle of stress field sensing allows for modular design, i.e., each part of the BMRs can be replaced for specific sensing needs, which makes our BMRs a versatile and reliable solution for general tactile sensing.

Results

Structure of BMRs

As mentioned above, BMRs are made in the form of a sandwich structure with epidermal and dermal sensing arrays wrapped around mechanical deformation layer. The epidermal and dermal sensing arrays have similar structure. To ensure stable sensing performance, Thermoplastic polyurethanes/Carbon black (TPU/CB) foam33,34 prepared by the template method was cut into 100 um thickness, only the central sheet was used as sensing material which shows high sensitivity, about 35 ms response time and stable performance (more details are described in Supplementary Fig. 1). Stretchable serpentine interconnects (150 um in width) and island-bridge structure (1.5 × 1.5 mm island, 5 mm spacing) are etched out of the laminated sensing layer and Cu/PI layer by laser direct writing35,36. Obtained a layer with 8 × 8 sensing units (Supplementary Fig. 2).

A special feature worth of mentioning is the patterned silicone layer, which is dispensed around sensing units. It used to bond top and bottom cross-aligned devices (Fig. 2). After curing, the cross-aligned silicone structure forms a natural wall, similar to the tissue around the Meissner corpuscle in human skin (Fig. 1), which can protect the sensing unit wrapped in it and make the sensor work stably under large deformation. At the same time, silicone can also play a supporting role. In absence of external force, the double-layer resistive material is separated, avoiding the stress generated during packaging to affect measurement results. More details of the structural modulation of imitation Meissner corpuscle are described in Supplementary Fig. 3. The as-fabricated sensing units have uniform performance, so they can accurately map the pressure distribution of a large variation of mechanical stimuli (Supplementary Figs. 4, 5).

Fig. 2: Fabrication procedures of BMRs and the assembled device.
figure 2

a Schematic diagram of the fabrication procedures for BMRs. b Exploded view of the layered structure of BMRs. A deformation layer made of Ecoflex gel is sandwiched between an ‘epidermal’ sensing array and a ‘dermal’ sensing array. Both ‘epidermal’ and ‘dermal’ sensing arrays contain five functional layers, i.e., PDMS substrate, upper Cu/PI serpentine electrode, pressure sensor array, protective silicone, and lower Cu/PI serpentine electrode. c Optical images of a BMRs device. Scale bar: 1 cm (c).

Full size image

The deformation layer, as the carrier of the stress field, is the most important part of the BMRs. Softness and thickness are two key parameters that affect measurement results. Here, we use 7 mm Ecoflex Gel (Shore hardness 00035) as the deformation layer material, which can respond sensitively to external force at smaller thickness (Supplementary Fig. 6). It also has high adhesion. Combining the mechanical deformation layer with the epidermal and dermal sensing arrays completes the BMRs.

3D force detection by BMRs

It is this bio-inspired, epidermal sensing array-deformation layer-dermal sensing array sandwich structure that allows our BMRs to act like human skin by perceive the change of stress field37. For example, when finger pressed against skin at an angle, the most intuitive human sensation is skin deformation. The surface skin moves along with finger, while the closer to interior the less skin moves. Here, we simulate a 45° angle of external force applied to deformation layer (Fig. 3a). As force grow, the pressure field gradually increase with horizontal displacement. If we focus on the pressure field without displacement, the pressure distribution at top surface is almost constant while one at bottom surface gradually moves in the direction of shear force as force becomes larger. This is due to the fact that although there is horizontal displacement of pressure contact point, it is relatively stationary with respect to the top surface of deformation layer. In this way, we can use epidermal and dermal sensing arrays to measure the pressure distribution between top and bottom surface of the mechanical deformation layer and analysis 3D force (Fig. 3b).

Fig. 3: Stress field sensing-based 3D force detection.
figure 3

a Simulation results of the horizontal displacement field, pressure field and displacement-free pressure field inside the elastomer when 3D forces of different magnitude at 45° angle are applied. b Illustration of the 3D force detection based on the stress field sensing. Under normal pressure, the center of gravity of the top surface pressure distribution (epidermal centroid) and the center of gravity of the bottom surface pressure distribution (dermal centroid) are located in the direction of normal force; With tilted force, the elastomer surface is displaced in the direction of shear force. The dermal centroid is shift from the original location. c, d Raw and combined field data of the epidermal and dermal sensing arrays at different polar and azimuth angles. The red color represents the pressure distribution on the top surface and the green color represents the pressure distribution on the bottom surface. e, f Variation of epidermal and dermal centroid coordinates versus polar angle. g Variation of centroid deviation length versus polar angle. h Variation of centroid deviation length and azimuth angle versus azimuth ground truth. i, j The relationship between the standard deviation of deviation length and angle measurement with ground truth of polar and azimuth angles. Scale bar: 1 cm (c, d).

Full size image

In order to quantify the relationship between stress field and 3D force, the physical quantity, center of gravity of epidermal and dermal pressure distribution, is represented as:

$$\begin{array}{*{20}{c}} {x = \frac{{\mathop {\sum }\nolimits_i \mathop {\sum }\nolimits_j x_if_{ij}}}{{\mathop {\sum }\nolimits_i \mathop {\sum }\nolimits_j f_{ij}}} \qquad y = \frac{{\mathop {\sum }\nolimits_i \mathop {\sum }\nolimits_j y_jf_{ij}}}{{\mathop {\sum }\nolimits_i \mathop {\sum }\nolimits_j f_{ij}}}} \end{array}$$
(1)

Where xi, yj are the sensor locations of the corresponding layers and fij is magnitude of pressure. The deviation vector composed of the center of gravity points of epidermal and dermal pressure distribution can be used to represent 3D force, where the deviation length or the vector module is related to force magnitude and polar angle, the vector direction is related to azimuth angle.

Using a homemade 3D force test equipment (Supplementary Fig. 7) and 16 × 16 readout circuit (Supplementary Fig. 8), we systematically investigated the response of BMRs to 3D forces. The magnitude of the external force was kept at 5 N. The polar angle range was between 40° and 140° (90° was set as the vertical direction for marking convenience) and the azimuth angle range was between 0° and 90° (fourfold symmetry). Position of the indenter was corrected for each test to press at the same position on the MAE. The results of top and bottom surface pressure distribution are shown in Fig. 3c, d. The raw 8 × 8 pressure distribution data (Supplementary Fig. 9) is interpolated by the method of cubic spline, where the pressure distribution on the top surface is shown in red and bottom one is shown in green. Due to the diffusion effect of force in deformation layer, the pressure distribution on the bottom surface is more dispersed than the top surface. As the direction of force changes, the bottom pressure distribution is shifted accordingly. Through the change of the center of gravity, we can see this phenomenon more intuitively. In Fig. 3e and f, As the polar angle increases, the center of gravity of the top surface remains almost unchanged, while the center of gravity of the bottom surface moves toward the shear force. So, we can obtain the relationship between polar angle and deviation length (Fig. 3g). For azimuthal angle, the direction of the deviation vector is the same as that of the shear force, as shown in Fig. 3h. The vector angle is the same as the azimuth angle.

With the continuity of stress field, 3D force can be measured accurately. The deviation length standard error of polar angle and azimuth angle can be controlled within 0.16 mm. For vertical force case, it can best reach 0.07 mm, which indicates that the accuracy is improved at least 31 times compared to sensor spacing. The standard error of tested polar angle increased with ground truth. The minimum is about 1.8° at 40° and the maximum is about 5° for vertical force case (Fig. 3i), which is due to the fact that the prediction of the polar angle is very sensitive to deviation length near 90°. For azimuth angle, the standard error is basically around 3.5° (Fig. 3j). This is because magnitude of azimuth angle is only related to the angle of deviation vector.

Hardness sensation by BMRs

As mentioned above, the external force on the stress field is not only spatially dependent, but also temporally dependent. For some physical quantities such as hardness, the spatial distribution of the stress field alone cannot provide comprehensive information. For example, humans can identify hardness by some simple finger-pressing processes on an object, especially for objects near skin hardness. The dynamic relative deformation of the skin during finger-pressing is significant, but humans are not very good at recognizing the hardness of objects with similar pressure distributions at static state.

Here we have selected four different hardness indenters, The materials used and corresponding hardness values for the tests are shown in the Supplementary Table 1. By pressing BMRs at same speed until the pressure reached 5 N (More setup details were shown in Supplementary Figs. 10, 11), The results are shown in Fig. 4a. As the hardness of the indenter increases, the relative deformation of indenter becomes smaller and Pressure distribution is more concentrated. Here, using the diffusion ring, the pressure distribution of different hardness indenters under the same pressure can be clearly distinguished.

Fig. 4: Measurement of the hardness of an object.
figure 4

a Dynamic response of the BMRs when pressing on an object. The 1.5 cm radius hemispherical objects of different hardness are pressed at the same speed by the BMRs up to 5 N. The red circles indicate the diffusion length of each sensing layer. b, c Total pressure measured at top and bottom sensing layers for indenters of different hardness during the pressing process. d, e After pressure normalization, the relationship between total pressure value and diffusion length. Diffusion length at top and bottom sensing layers for indenters of different hardness during the pressing process. Scale bar: 1 cm (a).

Full size image

To quantitatively study the response of BMRs to hardness, we first investigated the relationship between pressure sum and hardness. It should be emphasized that pressure sum is the total value of the grayscale of pixel points in the pressure distribution image. In Fig. 4b and c, the pressure sum response curves are clearly separated for different hardness indenters. It is an obvious result that the harder the object, the faster it reaches 5 N for the same pressing speed as mentioned above.

Not only the pressure sum is related to hardness, but there is also a strong correlation between the distribution of pressure and hardness. Here, to quantify the stress field deformation, we define a diffusion length to represent the concentration of pressure distribution as follows:

$$\begin{array}{*{20}{c}} {{\mathrm {Diffusion}}\;{\mathrm {length}} = \frac{{\mathop {\sum }\nolimits_i \mathop {\sum }\nolimits_j d_{ij}f_{ij}}}{{\mathop {\sum }\nolimits_i \mathop {\sum }\nolimits_j f_{ij}}}} \end{array}$$
(2)

dij represents the distance of pixel from the center of gravity.

In Fig. 4d and e, it is clearly found that the diffusion length increased with the pressing force. Because of the difference between top and bottom layer stress field distribution, the diffusion length response curves for epidermal sensing array are relatively lower than dermal ones. More importantly, for different hardness indenters, this phenomenon is significant changing. The harder indenter curves have larger diffusion length and the diffusion length ratio between epidermal and dermal sensing array is smaller. It is reasonable that the relative deformation between the hard indenter and the BMRs is larger than that between the soft indenter and the BMRs. So, using MASs can easily identify hardness by simple pressing like human skin.

Combining the above two physical quantities. It shows that a systematic analysis of stress field can lead to a high degree of redundancy of hardness-related information, ensuring fast and accurate identification under complex situations.

Modularity of BMRs

Benefitting from the principle of stress field perception, our sensors possess another feature that we call ‘modularity’. Since the external forces are reflected as strains in the deformation layer of BMRs, the tactile sensing is only related to the mechanical properties of the deformation layer. Meanwhile, epidermal and dermal sensing layers are responsible for sampling the pressure distribution and the sampling locations shall not affect the sensing result. This separation of roles and non-interference between epidermal and dermal sensing layers allow BMRs to be used in scenarios where there is a misalignment between different layers. For example, as shown in Fig. 5a and b, the epidermal sensor array is intentionally misaligned with the dermal sensor array during the assembly. This misalignment can be decomposed into a ‘rotation’ and ‘translation’, which is a typical affine transformation38. Since the module of ‘rotation’ is 1, the affine transformation can be calibrated by corresponding two test points on the epidermal and dermal sensor arrays (More details are described in supplementary material). As shown in Fig. 5c, two sets of test data were acquired by normal force pressed on the misaligned BMRs. Although the pressure point on top and bottom surface deformation layer is same, the epidermal and dermal sensor arrays data do not coincide. After calibration, the affine transformation was corrected and the centers of gravity of two test points overlap again (Fig. 5d). To validate this, we demonstrate a 3D force with φ = 40°, θ = 90° results in the same measurement after calibration as the previous device that was perfectly aligned (Fig. 3d). This feature demonstrates that our design can tolerate the random assembly error of the device as along as we can accurately measure the error.

Fig. 5: Demonstration of the modularity of BMRs and its application on curved surface.
figure 5

a Illustration of random orientation of epidermal and dermal sensor arrays. In a real usage case, the epidermal and dermal sensing array may be misaligned due to assembly error or fatigue. b Schematic diagram of the misaligned model and the correction points. Point 1 and point 2 are arbitrary test points. c Before and after calibration of the force data for test points 1, 2 and an applied 3D force at φ = 40°, θ = 90°. d, e 3D force measurement from the BMRs attached on the curved surfaces of robotic arm and elbow. Scale bar: 1 cm (d, e).

Full size image

This feature also allows for the assembly of our device onto curved surface, which may introduce local displacement between the sensing units on the epidermal and dermal layers. For instance, we affixed the BMRs to both the arms and joints of UR5 robot. The two surfaces are differently curved. By applying pressure from different directions, the BMRs responded accordingly on both surfaces (Fig. 5e, f).

Furthermore, we don’t even have to use the same design or the same sensing principle of the pressure sensor for epidermal layer as that for the dermal layer, which means that the design of each layer can be modular for different sensing tasks. For example, for sensitive tactile perception of robot fingertips, a low modulus gel can be used as deformation layer with a highly sensitive sensing array;39,40 while for robot joints where high stress fatigue resistance is the primary need, a high strength rubber can be used as the deformation layer with a sensing array for wide measurement range41,42.

Discussion

Inspired by how tactile receptors in human skin percept mechanical stimuli, we developed an artificial tactile receptor-based electronic skin, composed of a skin-like deformation layer as the carrier for stress field sandwiched by two layers of bionic tactile corpuscles as the field detector. By analyzing the stress fields, complicate tactile information such as 3D force and hardness of the object in contact with the electronic skin can be resolved. As such, the field sensing principle significantly simplifies the structure of multi-parameter electronic skin. In addition, because of the continuous differentiability of the stress field, our approach achieved up to 71 folds of improvement in spatial resolution from the physical spacing of the sensing units, i.e., a simple electronic skin, made of only 2 × 8 × 8 piezoresistive units with 5 mm pitch, can detect 3D forces with location accuracy of 0.07 mm, polar angle resolution of 1.8° and azimuthal angle resolution of 3.5°. Moreover, we demonstrate flexible and stable pressure distribution information allows BMRS to distinguish between objects of different hardness. Furthermore, our electronic skin can be applied on curved surfaces such as robotic hands, arms and so on. The modular design makes each part of the electronic skin exchangeable to adapt to specific sensing task. Therefore, our approach provides a versatile and reliable solution for general-purpose tactile sensing in robotics. To summarize, our stress field sensing approach represents a strategy capable of obtaining multi-parameter of tactile information from simple construction of the electronic skin. It is noteworthy that this simple approach is also powerful, reliable, and universally applicable.

Methods

Fabrication of porous TPU/CB film

In order to prepare porous conducting polymer, several precursors are needed. TPU (Elastollan 35 A, BASF) was dissolved in N,N-dimethylformamide (DMF, Aladdin) solvent at 1:2 mass ratio. Carbon black (SUPER P Li, TIMCAL) was added to DMF at 60 mg/ml and sonication (JY92-IIDN, SCIENTZ) was used to disperse CB at 180 W for 30 min. The grain size of NaCl was reduced to below 100 μm using a planetary ball mill (F-P400, FOCUCY) at 600 rpm for 15 min. The conductive polymer mixture was obtained by mixing the prepared precursors together uniformly in a certain ratio. Then it was put into a mold and kept in oven at 80° for 4 h to let the DFM solvent evaporate. The cured film was immersed in water for 12 h to dissolve the NaCl grains completely. The water was changed for every 2 h. Finally, the porous TPU/CB sheet was dried in oven and cut into 200μm think films.

Fabrication of BMRs

Three basic films were used in the BMRs fabrication step: TPU/CB porous film as described above, Cu/PI film (20 um thickness, purchased on Taobao) and PDMS film (10:1 ratio, 100 um in thickness, Sylgard 184, Dow Corning). The TPU/CB porous film was sticked on Cu/PI by screen printing silver paste (purchased on Taobao), then the PI side was laminated on PDMS film. Using laser marking machine (EP-20-SHG-S, HAN SLASER), a well-controlled pre-designed serpentine wire bridge and square island patterns were etched on the film, which kept TPU/CB and Cu/PI film was totally pierced and PDMS film unchanged. Removed redundant parts, a single structure body was obtained. Silicone (V-704, purchased on Taobao) was dispensed on the structure body by dispensing machine (E2, Nordson EFD) at 5 mm/s. Waiting 30 s, until the silicone was semi cured, then taping two of glued structure bodies together crosswise to make sure each sensing unit was aligned. Waiting another 12 h, until whole sensing array was cured. The deformation layer was obtained by pour Ecoflex Gel (1:1 weight ration of Parts A and B, Smooth-On) into 3D-printed mold and cured for 2 h. Finally, assembling epidermal and dermal sensing arrays with deformation layer, the MAR was obtained.

Preparation of indenters

Indenters with three different hardness were made by Ecoflex 0020, Ecoflex 0030, Ecoflex 0030 + Dragon Skin 30 (1:1 weight ratio) and Dragon Skin 30. The materials were all purchased on Smooth-On. Using a 1.5 cm radius and 2 cm high cylinder with hemisphere head 3D-printed mold, the indenters were made by standard curing process for 2 h.

Customized 3D force testing platform

The 3D force and hardness tests were used a homemade 3D force testing platform (Supplementary Fig. 4). A customer designed XYZ gantry liner stage (purchased on Taobao) was used as the main frame. Repeat accuracy can be up to 0.01 mm. In the Z liner module, a pressure sensor (SBT671-50N, SIMBATOUCH) was installed by 3D-printed connector. Combined with a pressure digital signal transmitter (SBT904, SIMBATOUCH), it can achieve a sensitivity of 0.05 N. A 2 DOF servo motor (LD20MG, Hiwonder) was put on the bottom of the XYZ stage to change polar angle and a R axis manual rotation stage (RS60-L, HUIKE) on the motor was used to change azimuth angle. On the rotation stage, a 2 × 80 × 80 mm acrylic was used as placing platform to support our BMRs. All digital lines were connected to a Arduino Mega 2560 board to control motion and record singles.

Data acquisition circuit

To simplify the readout circuit for multiple sensors, a standard 16 × 16 passive matrix was design for data acquisition from the BMRs. Two 74HC595 shift registers with 16 channels parallel output were used for column selector and a CD74HC4067 analog switch with 16 channels was used for row selector. Four LM324 operational amplifiers was set in negative feedback to achieve virtually equal potential using virtual ground, which can overcome the crosstalk issue of the sensor arrays43. At last, the same Arduino Mega 2560 board in 3D force testing platform was used to control readout channel and collect data.

Finite element analysis

Finite element analysis was performed using COMSOL 5.4. The 50 × 50 × 7 mm size deformation layer was modeled as Yeoh hyperelastic material. The C1, C2, and C3 parameters was set as 1000, 2670, and 6.3944. External force was loaded at a rigid cubic which combined at top of deformation layer to simplify the static friction in pressing process.

Experiment setup and test procedures

For basic measurements, the TPU/CB porous sensing units was loaded on a universal material testing instrument (E1000, Instron) and lined with a digital multimeter (6500 A, Keithley) for synchronous resistance measurement. A standard 10 mm length square indenter was glued on the testing instrument. Press speed was kept at 0.01 mm/s in sensitivity test and press frequency kept at 0.4 Hz in fatigue testing.

For silicone structural modulation measurement, different shape of silicones was dispensed at varied speed from 3 mm/s to 6 mm/s. The cross-section parameter was measured by optical microscope and sensing threshold hold was measured at same setup as sensitivity test.

For pressure distribution, 3D forces and hardness measurement, the BMRs were put on 3D force testing platform and lined with data acquisition board. The external force, press speed, polar angle and azimuth angle were set according to each experimental requirement. 8 bit voltage digital signals were acquired at fixed 14 frames per second.