Introduction

In nature, primates (humans and monkeys) extend their fingers to explore unknown spaces or perceive novelties, reflecting the importance of fingertip tactile sensing. Similarly, tactile sensing is an irreplaceable source of information for robots to perceive their surroundings1,2, which has incomparable advantages to visual and auditory senses in tasks such as object positioning and recognition3,4. Therefore, the research and development of tactile sensors for robotic fingers have received extensive attention, especially for tactile arrays similar to human fingertips5, including optical-fiber-based tactile sensors6,7, the Biotac multimodal tactile sensors based on conductive liquids8 and the bimodal soft electronic skin equipped with a compliant magnetic microelectromechanical system9. However, these tactile sensors of robotic fingers can only detect external forces in a single direction, indicating the poor ability to achieve multidirectional tactile information for complex tasks10.

Therefore, some studies are dedicated to achieving multidirectional tactile sensing. For example, the vision-based tactile sensors represented by Insight11, OmniTact12, and Geltip13 can achieve multidirectional perception by converting tactile information into visual images and then analyzing the image data with their computer vision algorithms. Notably, these strategies require complex structures, special optical lenses, as well as supporting analysis algorithms and software. The uSkin10,14,15,16 based on the Hall element arrays, can detect the force-driven magnetic field changes around the sensitive element arrays, realizing the sensitive detection of multidirectional forces. However, the whole setup is rigid, and the signal should be maintained by an external source, which loses flexibility and requires more space for the battery17. Table 1 provides a detailed comparison of representative state-of-the-art finger-like sensors for multidirectional tactile sensing.

Table 1 A comparison between state-of-the-art finger-like tactile sensors and this study.
Full size table

Flexible tactile sensors based on Faraday’s law of electromagnetic induction are a great solution for multidirectional tactile sensing, whose advantages include self-generated signals, large output currents, simple structures, and high sensitivity. Some self-generated-signal sensors have been proposed18,19,20 based on this. Li et al.18 reported a strain sensor that can be used for motion monitoring, but it can only detect strain changes due to joint deformation and is not suitable for multidirectional sensing of fingertips. Ma et al.19 reported a Merkel’s disks bioinspired magnetoelectric tactile sensor, which has been integrated into a robotic arm to help the robot feel and distinguish varied objects, yet it can only realize unidirectional tactile sensing. Yue et al.20 demonstrated the self-generated-signal stretchable tentacle sensor arrays, which can recognize surrounding objects located in non-visual conditions. However, it is difficult to integrate into a robot to give it multidirectional tactile sensing. Besides, the rigid copper coils chosen are neither flexible nor soft for the sensors. Alternatively, the Ga-In alloy, as a non-toxic liquid metal (LM) at room temperature, might serve as soft conductors for developing types of magnetoelectric tactile sensors21,22,23,24,25,26.

Here, we demonstrate a soft magnetoelectric finger (SMF) based on an LM coil array that enables self-generated-signal & multidirectional tactile sensing. The SMF consists of a ‘finger’ skin-like flexible sheath containing five LM coils and a designed ‘phalangeal bone’ with a magnet inside. Notably, its sensing electrical signal comes from the mutual movement between well-dispersed LM coils and the magnet, indicating its self-generated-signal feature, which has been analyzed by Abaqus and Maxwell software. The multidirectional sensing performance of five LM coils enables the SMF to resolve forces in different directions based on the analysis of five signals. In addition, benefitting from machine learning, the SMF can distinguish six varied common objects with different Young’s modulus with an accuracy of 97.46%, verifying the potential of the SMF in object recognition. Finally, the SMF has been integrated into the robotic arm to explore a designed black box, including identifying obstacles on the wall/ground and accurately judging the type of objects. It is believed that the self-generated-signal soft SMF provides a concept for the development of self-generated-signal and multidirectional tactile sensors for the robots’ exploration of complex unknown space, especially in non-visual recognition environments.

Result

Design, preparation, and characterization of the SMF

For the human who has been touched, the nervous impulses generated by the tactile receptors in the fingertip skin are transmitted to the brain through synapses for further processing, as illustrated in Fig. 1a. Inspired by finger perception, we designed an SMF that will enable robots to explore unknown space. Figure 1b shows a conceptual diagram of the SMF, which consists of two parts: an external electrical part that mimics the tactile receptors of a human finger and an internal magnetic part that collaborates with the outside sheath. The former is an Ecoflex polymeric sheath embedded with five LM coils, and the latter consists of a rigid core prepared by a 3D-printed part in the shape of a phalangeal bone with a magnet inside. Pressing the soft sheath in different directions can induce different voltage signals in the five LM coils due to the changes in the magnetic flux through them. Thus, the SMF is able to achieve multidirectional sensing by analyzing the signals, which enables object recognition and black box exploration (Fig. 1c).

Fig. 1: Schematic illustration of the SMF.
figure 1

a The schematic diagram of the tactile perception mechanism of a human finger. b The schematic diagram of the SMF, which generated five induced voltage signals through the embedded LM coils (Ch1–Ch5). c The SMF enables tactile exploration in an unknown black box.

Full size image

The whole fabrication process of the SMF is shown in Fig. 2. First, the required inner/outer molds and the designed phalangeal bone were prepared using 3D printing. Then, the Ecoflex was filled into the assembled molds (Fig. 2a, b) to get the soft sheath. Five microchannels were molded on the sheath (Fig. 2c, optical picture shown in Supplementary Fig. 1a), which then were injected with LM (Fig. 2d) and connected with wires (Fig. 2e and Supplementary Fig. 1b). Finally, after encapsulating it with Ecoflex (Fig. 2f and Supplementary Fig. 1c), a sheath with a thickness of ~3 mm was obtained (Fig. 2g and Supplementary Fig. 1d). Here, the resistance of the LM coils (including the LM coil and copper wires of ~ 3 cm in length for connection) were measured, as shown in Supplementary Table 1. A ~0.35 T (The superficial magnetic field in the center of the N-pole plane measured by the Gauss meter) NdFeB magnet (side length of 5 mm) was placed in the 3D-printed phalangeal bone, shown in Fig. 2h. It is worth noting that there is a gap of ~5 mm between the sheath and the phalangeal bone, which will make the sheath easy to deform by external forces, thereby improving the sensitivity of the SMF.

Fig. 2: Schematic illustration of SMF fabrication.
figure 2

a, b As-prepared 3D printed molds were assembled and then used to pour Ecoflex liquid. c After solidification and demolding, a thin sheath with five microchannels was obtained. d LM were injected into the microchannels. e Copper wires were embedded to facilitate the electrical connections. f The secondary encapsulating of Ecoflex mixture was undertaken before the g electrical part was prepared finally. h A NdFeB magnet was embedded on an as-prepared 3D-printed bone to get the magnetic part. i The sheath was mounted on the bone to obtain the SMF.

Full size image

Then, the morphology of LM inside the sheath was investigated. Supplementary Fig. 2a shows the optical photograph of the LM coils distributed on the outside sheath. Supplementary Fig. 2b illustrates the scanning electron microscope (SEM) image of an LM line embedded in the Ecoflex microchannel, with LM in the middle and Ecoflex polymer on both sides. Further, elemental analysis was performed along the black line in Supplementary Fig. 2b using energy dispersive spectroscopy (EDS), and the result (Supplementary Fig. 2f) indicates that the middle part is gallium indium alloy and two sides are mainly polymer dominated by C, O, and Si elements. In addition, a zoomed-in view of the Ecoflex and LM interface (Supplementary Fig. 2e) illustrates the adhesion between the LM and the polymeric parts, which can also be seen in the cross-sectional SEM image (Supplementary Fig. 2c). This is due to the presence of the gallium oxide skin on the surface of the LM23,25, which allows it to adhere to the Ecoflex polymer. Supplementary Fig. 2d shows the gallium oxide skin with some wrinkles, indicating that the LM can adhere well to the microchannels.

The sensing mechanism and performance of the SMF

The multidirectional tactile sensing capability of the SMF comes from the spatial arrays of five LM coils surrounding the magnet. The five LM coils, named Ch1–Ch5, are arranged toward five directions (Fig. 1b) of the inside magnet to detect forces from diverse directions. Ch1–Ch5 were connected to voltage acquisition equipment (LinkZill) via connective wires to monitor and collect voltage signals in real time. As shown in Fig. 3a, we vertically placed the SMF on the dynamic press setup for testing, and Ch1, Ch2, Ch3, and Ch5 that could be seen were marked (Ch4 was not visible in this image). Pressing the Ch1 side three times with a displacement of 3.5 mm and a compression speed of 10 mm s−1 produced the signals shown in Fig. 3b, where its average peak values of Ch1–Ch5 are around −9.51, 1.33, 0.55, 1.52, and 0.82 μV, respectively.

Fig. 3: Sensing mechanism of the SMF when experiencing a vertical force.
figure 3

a Optical images before and after the Ch1 side of the SMF was pressed (scale bar is 1 cm). b Voltage signals were collected at a compression speed of 10 mm s−1 and deformed of 3.5 mm 3 times. c 3D numerical calculated displacement distribution diagrams of the shape deformation of the SMF when the Ch1 side was pressed at a deformation of 3.5 mm. d 3D simulated distribution of the magnetic intensity inside the coils before and after compression, and e the simulated result of the magnetic induction lines through Ch1 before and after compression (Scale bars in d and e are 1 cm). f Distinguishable average peak values of the induced voltage signals of Ch1 in the SMF obtained under different parameters (n = 3, data presented as mean ± s.d.). g The dependence of voltage on the force for Ch1 of the SMF at several different compression speeds (n = 3, data presented as mean ± s.d.).

Full size image

Obviously, the Ch1 signal is much higher than those of Ch2–5 when the finger is subjected to a vertical force. Thus, numerical analysis via Abaqus and Maxwell was performed to explain this difference in the sensing mechanism. Figure 3c illustrates the finite element mechanical simulation results of the SMF before/after compression on the Ch1 side. The color bars in Fig. 3c, from blue to red, represent the gradually increasing amount of deformation. It is obvious that the deformation is mainly concentrated on Ch1 (The details of simulations can be found in Supplementary Note 1). The pre- and post-deformation models were then imported into Ansys Maxwell software for magnetic simulation. The position of each equivalent coil on the sheath surface can be calculated before and after deformation. Therefore, the magnetic intensity distribution through each coil was achieved, as shown in Fig. 3d from the main view. The color bars from red to blue correspond to the gradient of magnetic intensity from strong to weak. Meanwhile, the 3D stray magnetic field generated by the magnet around the SMF was measured by a multidimensional magnetic field testing system, as depicted in Supplementary Fig. 3a, b, which displays the actual magnetic field on two 40 × 40 mm profiles parallel to the XZ and YZ planes, respectively, past the center of the magnet (Gray square). This result matches the simulated results on the same profiles (Supplementary Fig. 3c, d), confirming the repeatability of the simulation outcomes. The total magnetic flux of each coil before/after compression can be calculated according to the following Eq. (1)27:

$$E\left({\rm{V}}\right)=-\frac{\Delta \varPhi }{\Delta t}=-\frac{{\varPhi }_{\left({\rm{after}}\right)}-{\varPhi }_{\left({\rm{before}}\right)}}{\Delta t}$$
(1)

where E(V) is the output voltage, ΔΦ is the total magnetic flux change, and Δt is the response time of the sheath.

Taking Ch1 as an example, the magnetic flux through Ch1 before and after compression increases from 7.12 × 10−6 to 10.26 × 10−6 Wb, so the change of magnetic flux is 3.14 × 10−6 Wb. Considering that the compression time is about 0.35 s, according to Eq. (1), a theoretically calculated voltage value of 8.97 μV is yielded, which is close to the experimental result of 9.51 μV. In contrast, the magnetic flux passing through the side equivalent coils changes slightly (ΔΦ are ranging from 0.02 to 0.56 × 10−6 Wb), as shown in Supplementary Fig. 4 and Supplementary Table 2. The difference between the experimental result and the theoretical one is mainly caused by the position differences of LM coils between calculated and actual conditions. In addition, the directions of the magnetic induction lines through Ch1 were all positive (pointing outward), and after compression, the total magnetic flux increased as the density of the magnetic induction lines increased (shown in Fig. 3e), resulting in a negative voltage output based on Eq. (1). Conversely, a positive voltage was collected when recovering. Thus, in one compression-recovery cycle, a voltage signal with a downward–upward peak shape was generated on Ch1, while voltage signals with upward-downward peak shapes were found on Ch2–5. (The wiring method is mentioned in the Methods). Notably, all signals were self-generated by the force-driven related motion between the magnetic and electrical parts. Therefore, as a self-generating-signal tactile sensor, like TENG or other similar tactile sensors28,29, the SMF is capable of detecting contact events but not static forces.

Figure 3f shows the average peak values of the induced voltage of Ch1 in the SMF under different experimental parameters, illustrating that the peak values of the obtained signals increased with the compression speed and displacement. The corresponding peak signals of forces are shown in Supplementary Fig. 5. The detection range for force is 0.04 (equal to 8.00 kPa) to 15.00 N (equal to 98.68 kPa), and the minimum detection for force was 0.04 N (Supplementary Fig. 6a, d), which is roughly the gravitational force generated by a piece of A4 paper. Since the peak of the induced voltage is related to the compression speed, the dependence of voltage generated by the Ch1 of the SMF on the compression speed was investigated (Fig. 3g). Taking a compression speed of 10 mm s−1 as an example, showing a sensitivity of 3.87 μV N−1 in the low-force range (less than 2.50 N) and 0.83 μV N−1 in the high-force range. Notably, since the compression speed affects the electrical signal, it needs to be given when the SMF is to be used to measure the stimulation force. Furthermore, the long-term stability of SMF was tested, and it remained reliable after 10,000 pressing-recovery cycles (Supplementary Fig. 7). The response of Ch1 to different vibration frequencies was tested. As shown in Supplementary Fig. 8a, the higher the vibration frequency, the higher the peak signal generated at the same displacement. It is worth noting that the mechanical vibration stimuli applied here to Ch1 are sinusoidal, with amplitudes of displacement. (As an example, Supplementary Fig. 8b illustrates the sinusoidal mechanical vibration stimulus at a frequency of 6 Hz and a displacement of 3 mm). The mechanical stimuli applied in other tests are all pulses. In particular, the signals generated by Ch1 at different vibration frequencies at a displacement of 3 mm are shown in Supplementary Fig. 9, indicating that Ch1 of the SMF responded well to vibrations with frequencies in the range of 1–6 Hz. As the vibration frequency increases, the thin sheath composed of soft Ecoflex silicone and LM coils exhibited different signal peaks due to its slow recovery speed.

In addition, the impact of the number of turns of the LM coils in the SMF on the induced voltage signal was investigated. As shown in Supplementary Fig. 10a, the SMF with three turns of LM coils was prepared (The resistances of individual three-turn coils were measured and are presented in Supplementary Table 1), and the electric signals generated by it were collected under a pulse stimulus applied to Ch1 at a displacement of 3.5 mm and a compression speed of 10 mm s−1. Comparing the peak values with those generated by the SMF with a single turn of LM coils under the same parameters (Supplementary Fig. 10b), it is evident that as the number of turns of LM coils increases, the electrical signals generated by each LM coil exhibited a corresponding increase by about 70%. In this regard, we conducted theoretical simulations using MAXWELL to calculate the magnetic flux passing through each of the three-turn coils before and after deformation. Considering that the compression time is about 0.35 s, according to Eq. (1), the theoretical induced voltage is yield (shown in Supplementary Table 3). It can be observed that the theoretical results align closely with the actual experimental results, and the discrepancy may stem from differences between the actual coil positions and the simulated ones.

Besides Ch1 of the SMF, other side coils, including Ch2–5, have been carefully investigated experimentally to demonstrate the sensing performance of the four sides of the SMF. It is worth noting that Ch2/Ch4 and Ch3/Ch5 are located in opposite positions and have a certain symmetry. To simplify the experiments, Ch4 and Ch5 were chosen as representatives for testing, as shown in Supplementary Figs. 11a, b, 12a, b, respectively. Here, we used a spherical object to apply forces to the SMF, ensuring that the pressing position was in the center of the Ch4/Ch5 coils. The signal groups and average peak voltage values of Ch4 and Ch5 are shown in Supplementary Figs. 11c, d, 12c, d, respectively. Compared with Ch1, the minimum detections of Ch4 and Ch5 are ~0.18 N and ~0.43 N, respectively (Supplementary Fig. 6b, e/c, f). This result can be attributed to the magnetic field distribution of the magnetic polar towards Ch1 in the SMF.

Multidirectional tactile sensing

Besides normal (perpendicular) pressures (Fig. 3; Supplementary Figs. 11 and 12), the spatial arrangement of the five LM coils enables the SMF to distinguish shear forces in diverse directions. Taking Ch4 as an example (as shown in Fig. 4a), in addition to the forces on the center of the Ch4 side (referred to as n4, see Supplementary Fig. 11), the SMF can also distinguish shear forces from the Ch4 side to the Ch3 side (in short 4 → 3, see Fig. 4b), 4 → 1, and 4 → 5.

Fig. 4: Schematic diagram of the shear forces and the signal groups generated under those forces.
figure 4

a Partly magnified image of the SMF in order to illustrate the directions of the shear forces, which begin with the Ch4 side (4 → 5, 4 → 1, and 4 → 3, the points of application are P45, P14, and P34, respectively). b Schematic illustration of 4 → 3, the orthogonal reference frame is given. c Schematic illustration of the plane expansion of SMF centered on Ch1, where each arrow represents a sort of shear force, is divided into five categories depicted in varied colors. d–h Voltage signals are collected when the SMF suffers different shear forces, where the characteristic signals are marked by dotted boxes.

Full size image

To quantify these forces, an orthogonal reference frame has been established with the center of the SMF’s base as the origin (illustrated in Supplementary Fig. 13a). Also, the positions and naming conventions for the primary force application points are specified, as shown in Supplementary Fig. 13b. Here, P2 and P3 represent the force application points located at the centers of the Ch2 and Ch3 coils, respectively, while P23 represents the force application point positioned between the Ch2 and Ch3 coils. In this case, based on the established orthogonal reference frame, the force applied to the center of the Ch2 side, n2, has its application point at P2, and its direction is [−,0,0] (where the ‘+’ represents the component vector of force on the axis follows the positive direction of the axis; the ‘−’ represents the component vector of force on the axis follows the negative direction of the axis. ‘0’ means that the component vector of force on the axis remained), indicating that the force n4 acts in the positive X-direction. As for the shear force 4 → 3 (Fig. 4b), its application point is P34, and its direction is [+,+,0]. Based on this, we can differentiate some forces based on their corresponding application points and directions, which are listed in Supplementary Table 4. Figure 4c illustrates the categorization of all shear forces in Supplementary Table 4 based on their starting positions, classified into five categories and described using varied colors. Figure 4d–h displays the signals generated by the five LM coils when the SMF experiences shear forces from different directions.

The signals collected by each LM coil can be roughly classified into 2 types: a “downward–upward” signal representing an increase in magnetic flux passing through the coil, indicating that the coil moves closer to the magnet, and an “upward-downward” signal showing that the LM coil moves away from the magnet. Taking 4 → 3 as an example (Fig. 4g), the “downward-upward” signal of Ch4 represents Ch4 moving closer to the magnet, while the “upward–downward” signal of Ch3 represents Ch3 moving away from the magnet. This indicates the applied shear force as 4 → 3, and these two signals are considered to be characteristic signals for determining the shear. Similarly, the characteristic signals for other shear forces have been analyzed and summarized in Table 2.

Table 2 The characteristic voltage peaks of signal groups generated by diverse shear forces.
Full size table

In summary, the SMF is able to distinguish forces from different directions, exhibiting its multidirectional tactile sensing capability.

Object recognition

The SMF was used to recognize different objects with diverse Young’s moduli. Six common objects were selected as test subjects, including sponge cubes, porous plastics, paperboard, foamed plastics, Ecoflex rubber, and cured resin, as shown in Fig. 5a. It is noteworthy that all six objects are of equal size. The stress-strain correlation curves for each object were measured and are depicted in Supplementary Fig. 14a, while Young’s modulus estimated by fitting the linear portion of the stress-strain curve within the range of 0% to 20% strain for each object is presented in Supplementary Fig. 14b. It can be seen that Young’s modulus varies for different objects. The SMF was integrated into the compression platform to touch these objects at a speed of 10 mm s−1 and a displacement of 3.5 mm 360 times (pulse-like mechanical stimulation). Accordingly, induced voltage signals of Ch1 were collected and shown in Supplementary Fig. 15. Since the six groups of objects have different Young’s moduli, varied degrees of shape deformation occurred when they were touched by the SMF at the same deformation, resulting in different induced voltage signals of Ch1. Machine learning algorithms were introduced accordingly to achieve accurate identification of these objects.

Fig. 5: The SMF enables object recognition via machine learning.
figure 5

a Optical images of six objects: sponge cube (SC), porous plastics (PP), paperboard (PB), foamed plastic (FP), ecoflex rubber (ER), and cured resin (CR), scale bars are 1 cm. b The flowchart of the machine learning process. c Clustering patterns among the six materials based on the t-SNE algorithm. d Confusion map of six objects.

Full size image

The flowchart of machine learning is shown in Fig. 5b. Since the compressions occurred coherently during the experiment, the raw data of each acquired object was divided into 120 strips, that is, about 3 times of compressions in a group to ensure the integrity of the information and labeled them according to categories. The collected voltage data underwent feature engineering and was transformed into a series of features, including maximum (max), minimum (min), skewness (sk), kurtosis (kurt), root mean square (rms), standard deviation (std), variance (var), peak-to-peak amplitude (pp), and more. These features will be input into the machine learning model for training. For each object, 80% (96 groups) of the data was selected as the training set, and the remaining 20% (24 groups) was known as the test set. In the end, the training set had a total of 576 pieces of data, and the test set had a total of 144 pieces of data.

For this multiclass problem, tests were conducted on five different machine learning algorithms using the same dataset. Among them, the LightGBM classifier achieved the highest accuracy rate of 97.46% in classifying six different objects (as depicted in Supplementary Fig. 16). The test results of the confusion matrix for the LightGBM classifier are shown in Fig. 5d.

To facilitate a more intuitive representation of the signal disparities among the six materials, the T-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm was applied for data visualization. T-SNE is a robust nonlinear technique widely employed for reducing dimensionality and visualizing high-dimensional datasets. The resulting plot in Fig. 5c demonstrates notable clustering patterns among the six materials, indicating promising prospects for utilizing machine learning in material classification endeavors.

Black box exploration

As a multidirectional tactile finger capable of recognizing different objects, the SMF is suitable for exploring unknown areas in dark environments. Thus, after being integrated with a robotic arm, the SMF can be used to explore a designed black box, as shown in Fig. 6a. The designed black box (Fig. 6b, c) includes two parts: One is square-looped walls (45 mm in length) with randomly dispersed hemispherical obstacles (named W1–W4). The other is the bottom with four square regions (named R1–R4), on which there are pits, planes, and obstacles with different heights.

Fig. 6: Black box exploration of the SMF: Illustration and Step 1.
figure 6

a Optical image of the self-built test system, where the SMF was integrated into a robotic arm. b Schematic illustration of the black box design, with unknown areas marked; an optical image of b is also shown in (c) (Scale bars are 2 cm). Step 1: d The SMF rotated counterclockwise around the walls from W1 to W4 at a speed of 10 mm s−1, and the collected signal groups are shown in (e). Taking W1 as an example, the shear force is represented by the orange arrow in (f), and the corresponding orthogonal reference frame is given. g Final result of Step 1, where red regions represent obstacles.

Full size image

The procedure of exploring the black box in a dark environment consists of three steps. The first step is to touch the W1–W4. The SMF (Ch3 close to the W1 wall) started at the top left corner of the black box with an entry depth of ~22 mm. As shown in Fig. 6d, the SMF moved one turn counterclockwise along the inside walls of the black box at a speed of 10 mm s−1 (Supplementary Video 1). The collected signals were processed and shown in Fig. 6e (the useful portion was intercepted here). Taking W1 as an example, the direction of the shear force can be known as 2 → 3 (the point of application is P23, and the direction is [−,+.0]) according to the characteristic peaks of the signal group, represented by the orange arrow in Fig. 6f. And the starting point of the signal peak of 1.4 s proves that there is an obstacle on W1 with a distance of ~14 mm away from the starting point. Each wall was divided into three regions with an area of 15 × 15 mm2. In this case, the obstacle can be recognized in the first region, which can be seen in Fig. 6g (the red part). Similarly, W2 has an obstacle in the second region, W3 has an obstacle in the first region, and W4 does not have any obstacle.

In the second step, the SMF was raised with an entry depth of 20 mm and moved to the middle part of the black box (Fig. 7a). Considering the detection range of Ch1, the SMF returned to the reference level after probing downward by 5 mm (compression speed of 10 mm s−1). If no contact signal was detected, the next descent distance could be increased by 5 mm (return after a descent of 10 mm) and repeated in the next probing (Supplementary Video 2). As shown in Fig. 7b, c, no touching response was detected within 15 mm (1.5 s) for R1, which means R1 is a pit. In contrast, diverse voltage signals were detected at 2 mm (0.2 s), 2 mm (0.2 s), and 14 mm (1.4 s) for R2–R4 respectively, indicating that R2 and R3 contain obstacles, while R4 is a plane with a height of 1 mm (Fig. 7f).

Fig. 7: Black box exploration of the SMF: Step 2 and Step 3.
figure 7

Step 2: a The SMF probed the four regions R1 to R4 from a reference level at a compression speed of 10 mm s−1, and the collected Ch1 signals are shown in (b). c Result of Step 2 shows that R1 is a pit, R2 and R3 have raised obstacles, and R4 is a plane. Step 3: Object types of the obstacles were judged. d Ch1 signals were obtained by pressing R2 and R3 with a compression speed of 10 mm s−1 and a displacement of 3.5 mm for three times and inputted to the LightGBM classifier for prediction. e Prediction result indicates that R2 is made of porous plastics and R3 is made of cured resin. f Result of Step 2 and Step 3, which is consistent with the actual situation.

Full size image

The third step attempted to judge the type of obstacles in R2 and R3. The SMF was moved above the R2/R3 objects with an entry depth of 22 mm. Then, the SMF contacted the objects with a compression speed of 10 mm s−1 and a displacement of 3.5 mm three times (Supplementary Video 3). The collected data (Fig. 7d) was processed and inputted into the LightGBM classifier for prediction. Finally, the resulting probability matrix (Fig. 7e) indicates that the probability of R2 being a porous plastic was 0.609, which meant that the obstacle of R2 was porous plastic. Similarly, the probability of R3 being a cured resin was 0.958, so the obstacle of R3 was concluded as cured resin (Fig. 7f). Overall, the black box exploration experiment proved that the SMF combined with machine learning can be feasibly used to explore unknown spaces in a dark environment.

Discussion

This study demonstrates a self-generated-signal multidirectional magnetoelectric tactile finger, which consists of an elastic sheath with five LM coils and a designed ‘bone’ containing a magnet, endowing it with considerably deformable ability under various external forces. The spatial array of five LM coils enables the SMF to distinguish forces in different directions, including normal forces and shear forces. Both the experiments and simulations have proven that external forces can cause the shape deformation of the elastic sheath, leading to self-generated electrical signals. With the assistance of machine learning, the SMF can recognize 6 varied objects and identify objects accurately and spatially arranged in a ‘black box’. This work will promote the development of the self-generated-signal & multidirectional tactile finger for future robots.

The soft tactile finger reported here is promising for robotics because the signal is self-generated and senses along multidirections using soft, magnetoelectric principles. However, despite the demonstrated functionality and advantages, further research and development are still needed to address the challenges, as follows.

Currently, the SMF can accurately determine the stimulation force only under a known compression speed. Although in practical scenarios where it is integrated into robots or robotic arms, the movement speed can be easily fixed, it remains a potential drawback. One possible solution is to conduct a more in-depth analysis of the sensing signals to extract additional useful information to help determine force.

The prosthetic finger is designed with five sensing units to achieve multi-directional tactile perception. However, due to the limited number of sensing units, achieving more directions of tactile perception is challenging at the current stage. Further research efforts may focus on increasing the number of LM coils to enhance the capabilities in this regard.

Furthermore, additional research efforts may need to address challenges related to precision, stability, and cost-effectiveness. Improving the fitting of multi-turn LM coils on irregular, three-dimensional surfaces can help enhance the sensitivity and precision of the SMF. In addition, based on the principle of SMF identifying different objects, it can currently only distinguish between objects with different Young’s modulus, for which combination with other technologies such as visual recognition may be a solution. Optimizing the preparation and design of the LM coil array to enhance stability and reduce costs is also crucial.

Additionally, a focus on data processing and machine learning algorithms is necessary. Through interdisciplinary collaboration and continuous innovation, this SMF could play a significant role in robotics, human-machine interaction, virtual reality, and other fields.

Methods

Fabrication of the SMF

The SMF consists of a finger skin-like flexible sheath containing five LM coils and a designed phalangeal bone with a magnet inside; for the flexible sheath, designed inner and outer molds were prepared via a photocuring 3D printer (ANYCUBIC Photon). After assembling the outer mold, the Ecoflex liquid, obtained from mixing two commercial liquid silicone (A&B, T605) according to the mass ratio of 1:1, was filled, and then the inner mold was assembled. The Ecoflex liquid was solidified at 50 °C for about 30 min to get the thin sheath with five microchannels; the optical picture is shown in Supplementary Fig. 1a. After that, the LM (green, EGaIn, ≥99.99%, Sigma-Aldrich) was injected into the microchannels respectively. The LM coils were led out through the copper wires to facilitate the electrical connections. Notably, the conductive silver glue was added at the contact between the copper wires and the LM, and heated at 100 °C for 60 min to ensure a stable connection. Finally, the LM coils were coated with the Ecoflex liquid to encapsulate the LM and heated at 50 °C for about 30 min. As for the magnetic part, the cubic NdFeB magnet (side length of 5 mm) was placed in the 3D-printed bone structure. Then, the flexible sheath was mounted on the bone to obtain the SMF. The wiring method needs to be adjusted to ensure that the induced voltage is negative when the corresponding coil is pressed. For example, as shown in Supplementary Fig. 17a, the induced voltage signal of Ch1 was negative when Ch1 was pressed and positive when the finger was released to recover deformation (the mechanical stimulus applied includes the displacement and the corresponding force as in Supplementary Fig. 17b).

Characterization

The optically digital photos and videos were recorded by a commercial camera (α6300, Sony). The morphology characterization of LM integrated into Ecoflex polymer was observed by a field emission scanning electron microscope (FESEM, JSM-7600F, JEOL), and the element analysis was conducted in combination with energy disperse spectroscopy (EDS, Oxford). The superficial magnetic field of an NdFeB magnet plate was measured by WT10A Teslameter (WEITE Magnetic Technology Co., Ltd., China). Electrical responses were measured in real-time by LinkZill (Wuhan LinkZill Technology Company, Ltd., Wuhan, China). To test the mechanical performance of the SMF, the Instron All-ElectroPuls Electric Dynamic Test Instrument, which is non-magnetic (E3000, Instron) was used in this study. The robot arm (KUKA KR30) was purchased from Midea Group (China). The 3D magnetic stray fields were measured by a multidimensional magnetic field test system (F-30, CH-Magnetoelectricity Technology, China).

Numerical simulation

The Abaqus 6.14-4 was used to calculate the shape deformations of the SMF when suffering a vertical force. Subsequently, the original and deformed models were imported to the Ansys Maxwell finite element analysis software, which was used to conduct 3D modeling of an NdFeB magnetic cube integrated with LM rings. The permanent magnet was set as a cube with a side length of 5 mm, which was magnetized in the direction of the positive z-axis according to NdFeB parameters. In this study, the magnetic coercivity (Hc) was set as −954929.6 A·m−1, as well as residual flux density (Br) of ~1.2 T. More details can be found in Supplementary Notes 1 and 2.