A tactile sensor system with sensory neurons and a perceptual synaptic network based on semivolatile carbon nanotube transistors

The human sensory system has a fascinating stimulus-detection capability attributed to the fact that the feature (pattern) of an input stimulus can be extracted through perceptual learning. Therefore, sensory information can be organized and identified efficiently based on iterative experiences, whereby the sensing ability is improved. Specifically, the distributed network of receptors, neurons, and synapses in the somatosensory system efficiently processes complex tactile information. Herein, we demonstrate an artificial tactile sensor system with a sensory neuron and a perceptual synaptic network composed of a single device: a semivolatile carbon nanotube transistor. The system can differentiate the temporal features of tactile patterns, and its recognition accuracy can be improved by an iterative learning process. Furthermore, the developed circuit model of the system provides quantitative analytical and product-level feasibility. This work is a step toward the design and use of a neuromorphic sensory system with a learning capability for potential applications in robotics and prosthetics. An array of electronic components that can mimic information processing in the brain has been developed by scientists in South Korea. Digital electronic devices operate using only ones and zeros, but neurons in the brain and the synapses that connect them work on an analog basis. Neuromorphic devices attempt to mimic this analog behavior to achieve low-power information processing. Previous neuromorphic networks have been created using different types of structures to mimic neurons and synapses. Sung-Jin Choi from Kookmin University in Seoul and co-workers have now demonstrated a device that can simulate both synaptic and neuronal functions simultaneously. The researchers used an array of carbon nanotube transistors to create a neuromorphic system that can recognize different tactile patterns and learn in the process. The system has potential applications in robotics and prosthetics. We demonstrate an artificial tactile sensor system with a sensory neuron and a perceptual synaptic network that is composed of a single device type, i.e., a semivolatile carbon nanotube transistor. The system can differentiate the temporal features of tactile patterns, and its recognition accuracy can be improved by an iterative learning process.


Introduction
Over the past half-century, device electronics have been successfully advancing the Information Age thanks to consistent performance improvements based on the downscaling of digital devices that can provide reliable logic-gate operations, thus benefiting from their robustness to high levels of noise. Recently, research paradigms have shifted the focus to conventional devices with analog characteristics, which were previously considered a drawback. With an innovative computing architecture used to overcome the energy inefficiency of the conventional von Neumann architecture-typically referred to as a neuromorphic system 1 -the analog behaviors of these devices have begun to be rehighlighted as essential features for the implementation of neuromorphic systems 2 . Given that the numberless internal states of an analog device can be adjusted with minimal energy consumption and given that they can be maintained over the long term (i.e., they are nonvolatile) 3 , analog devices can store more data in a single device with better energy efficiency than digital devices. Specifically, recent advances in these analog devices allow them to emulate the functionality of biological synapses and neurons, while the ability of their crossbar arrays to solve cognitive tasks related to learning and recognition has been demonstrated experimentally 4-8 .
A neuromorphic system will provide energy-efficient computing power for artificial intelligence that can replicate cognitive function up to the human brain level.
Interestingly, some analog devices exhibit volatile behaviors for input stimuli; i.e., they yield a temporal state enhancement that quickly decays to its initial state, which constitutes one of the essential characteristics of a neuromorphic system. Early research on this volatile behavior focused on the emulation of short-term plasticity (STP) in biological synapses, such as paired-pulse facilitation (PPF) 9,10 . Moreover, recent studies have emulated biological neurons, including their capability to integrate temporal input stimuli 11,12 based on the leaky integrate-and-fire (I&F) neuron model 13 . By exploiting the response of the volatile device depending on the frequency of the input stimuli, the spatial or temporal patterns in the input stimuli can be classified. For example, the integration of sensor devices for various external stimuli (e.g., pressure, tactile, or visual stimuli) with processing components (i.e., a volatile analog device as a sensory neuron) has been proposed to capture the similarities of the human sensory perception system [14][15][16][17][18][19] .
Notably, unlike the volatile or nonvolatile analog devices mentioned above, a specific device can achieve switchable volatile/nonvolatile behaviors, as desired 20,21 . Although this is not a generic term, we will refer to it as a "semivolatile" characteristic (in fact, in the case of a twoterminal-based analog device (memristor), a device with semivolatile behavior has been referred to as a secondorder memristor 22,23 ). Previous studies of semivolatile behavior focused only on the capture of a psychological model related to the human brain's processes of memorizing and forgetting (i.e., the transition from short-term memory to long-term learning) 20,21 . However, more attention should be paid to the potential of a semivolatile device to simulate both synaptic and neuronal functions simultaneously, i.e., the processes of leaky integration (in the neuron) and learning/recognition (in the synaptic network) of input stimuli and their emulation with a single type of device. To date 24 , neuromorphic systems have been implemented with an artificial neural network based on the integration of two different types of devices (for neuronal and synaptic devices). If the entire artificial neural network with neuronal/synaptic devices can be implemented with a single device type, a simpler fabrication process for high-density integration will be achieved. Consequently, the goal of achieving large-scale and on-chip implementation of neuromorphic systems can be realized much sooner than expected.
In this paper, we demonstrate a biorealistic tactile sensor system wherein both the sensory neurons and perceptual synaptic network are implemented by a semivolatile carbon nanotube (CNT) transistor. The semivolatile transistor can switch the operation mode according to the bias condition. In this way, a single device type is allowed to play two different roles (neuronal and synaptic functions) simultaneously. In our tactile sensor system (Fig. 1), the sensing receptor, action potential in the axon, sensory neuron that processes information, and synaptic network for perception learning are emulated by a tactile sensor, a voltage-controlled oscillator (VCO) circuit, one neuronal CNT transistor, and the synaptic CNT transistor array, respectively. The tactile sensor converts pressure stimuli into resistance changes. These are then converted to digital signals and modulated such that their frequencies vary with the pressure intensity based on the use of the VCO. Subsequently, the output of the VCO is delivered to the neuronal CNT transistor that operates in volatile mode. A leaky-integrating output corresponding to the frequency of the VCO output is then generated. Finally, the sampled output of the neuronal CNT transistor is fed to the network of synaptic CNT transistors that operate in a nonvolatile mode, and the learning/recognition processes for distinguishing the input stimuli pattern are conducted in a supervised learning manner.

Fabrication of the tactile sensor
A flexible and transparent polydimethylsiloxane (PDMS) substrate (thickness of 1 mm) was prepared by mixing PDMS prepolymer (Sylgard 184A, Dow Corning) and a curing agent at a ratio of 10:1 by weight. Subsequently, the substrate was cleaned by oxygen plasma treatment, functionalized with a poly-L-lysine solution (0.1% w/v in H 2 O, Sigma-Aldrich), and acted as an effective adhesion layer for the CNTs. The substrate was thoroughly rinsed with deionized (DI) water and dried with flowing nitrogen gas. Subsequently, the 99% metallic CNT network film was directly formed on the PDMS substrate by spray coating the preseparated metallic CNT solution (concentration of 0.01 mg/mL) on a hot plate (at 100°C) followed by thorough rinsing with isopropanol and DI water. Two Cu electrodes were then formed at both ends of the CNT network film with silver paste for reading and inducing electrical signals. Subsequently, a top PDMS layer was cast with liquid PDMS with a thermal curing agent onto the CNT network film to fabricate a sandwich-like structure. In the final step, all the layers were annealed at 100°C for 1 h. Fig. 1 Conceptual design of our tactile sensor system compared with a biological sensor system. In biological systems, pressure stimuli applied to mechanoreceptors change the receptor potential of each mechanoreceptor. The receptor potentials initiate action potentials. Action potentials from multiple nerve fibers combine through neurons and contribute to information processing. Finally, the synaptic network in the brain recognizes the input pressure pattern. Our artificial tactile sensory system is composed of a tactile sensor device, voltage-controlled oscillator (VCO), neuronal device, and synaptic network.

Fabrication of the semivolatile CNT transistor array
CNT transistors were fabricated on p-doped rigid silicon substrates with a thermally grown SiO 2 layer with a thickness of 50 nm. We used the local back-gate structure for the modulation of the channels in the CNT transistors. To form the local back-gate, a Ti layer with a thickness of 20 nm was deposited by e-beam evaporation and patterned by a subsequent lift-off process. Subsequently, an Al 2 O 3 layer (thickness of 40 nm) and a SiO 2 layer (thickness of 10 nm) were deposited sequentially to form a gate insulator by atomic layer deposition. The top surface of the SiO 2 layer was then functionalized with a 0.1 g/mL poly-L-lysine solution for 20 min to form an amineterminated layer, which acted as an effective adhesion layer for the deposition of the CNTs. The CNT network channel was then formed by immersing the chip into a 0.01 mg/mL 99% semiconducting CNT solution (NanoIntegris, Inc.) for 8 min at an elevated temperature of 100°C. The source/drain electrodes that consisted of Ti and Pd layers (each 2 nm and 30 nm, respectively) were then deposited and patterned using conventional thermal evaporation and a lift-off process, respectively. Finally, additional photolithography and oxygen plasma etching steps were conducted to remove unnecessary CNTs other than those in the channel area, thus isolating the devices from one another.
In the case of the crossbar array, Cu (thickness of 80 nm) and SiO x (thickness of 150 nm) were sequentially deposited and patterned for the metal line and interlayer dielectric layer (ILD), respectively.

Results and discussion
The operational principle of our semivolatile CNT transistor is based on different hole-movement mechanisms in the traps (i.e., interface and surface traps). It has been shown in other studies 25 that typical CNT b Schematics of the trapping/detrapping of holes at the interface/surface traps. c Temporal I D enhancement and decaying behavior when a single V G pulse is applied. The use of V high = 0.5 V is not sufficient to allow the ejection of all the trapped holes from the interface/surface traps. d The summarized short-term I D change (ΔG ST ) and long-term I D change (ΔG LT ) according to the V high level. When V high > 2.5 V is applied to the gate of the CNT transistor, the CNT transistor operates in a nonvolatile mode. Moreover, when V high < 2.5 V is applied, the CNT transistor operates in volatile mode. e Cumulative I D enhancement behavior based on the application of a train of pulses. Because the CNT transistor operates in volatile mode (V high = 0.5 V), the final current level (I end ) decays to the initial level (I start ) again (these data are not shown in this graph). f Variation in ΔG ST as a function of f pre enables the differentiation of the pressure intensity.
transistors have a drain current (I D ) hysteresis that is related to the gate voltage (V G ) sweep (Fig. 2a). The hysteresis is attributed to the trapping/detrapping of holes at (1) the interface trap through the tunneling process and (2) the surface trap through the diffusion process in the lateral direction (see Fig. 2b, Fig. S1, and Supplementary Information Note 1 for a more detailed discussion). Upon the application of negative V G values, traps are filled by holes. The positively charged traps bend the energy band of the CNT downward, thereby resulting in the suppression of I D owing to the enlarged Schottky barrier width at the drain/CNT junction (left inset of Fig. 2a). A positive V G ejects the trapped holes, and the consequent upward band bending leads to the increase in the I D owing to the narrowing of the Schottky barrier width (right inset of Fig.  2a). Notably, because the interface trap is adjacent to the CNT channel, the tunneling of holes between them is relatively fast. Conversely, the diffusion of holes between surface traps in the lateral direction is relatively slow. These different hole-movement mechanisms lead to a semivolatile behavior in the CNT transistor. Figure 2c shows one example of a I D response following the application of a single V G pulse. The low level of the V G pulse (V low = −3 V) is the undisturbed read voltage for the channel conductance (i.e., −3 V cannot lead to any trapping/detrapping process). Accordingly, the pulse width was minimized to 5 μs to eliminate any time-cumulative effects. Before the onset of the measurements, a negative V G (−6 V, direct current (DC)) was applied for 1 s to ensure that all the traps were filled by holes at the initial state. When a single V G pulse was applied, the trapped holes were ejected to the CNT channel. If the high level of the V G pulse (V high ) was large enough to cause the ejection of all the trapped holes at the interface and surface traps (given that there are no holes to refill the traps after the pulse is removed), the I D change, i.e., the nonvolatile mode operation, would be maintained over the long term. By contrast, if V high is not sufficient to cause the ejection of all the trapped holes (e.g., V high = 0.5 V, as shown in Fig. 2c), I D temporarily increases and decays to its initial state subsequently, i.e., the volatile mode operation (I start , I peak , and I end , denote the initial, peak, and final I D levels, respectively). During volatile mode operation, only the holes at the interface trap are ejected by applying the pulse, but most of the holes at the surface trap remain. After the pulse ends, surface traps act as reservoirs of holes and help refill the empty interface traps, which results in the gradual recovery of I D . Consequently, the level of V high determines the operation mode of the semivolatile CNT transistor. Figure 2d shows the summarized channel conductance change (defined as ΔG) according to the level of V high . Here, the amount of shortterm enhancement in I D is defined as ΔG ST = (I peak − I start )/ I start , and the amount of long-term change in I D is defined as ΔG LT = (I end − I start )/I start . Notably, ΔG LT is zero until V high attains a value of 2.5 V, which indicates that there is only a temporary change in the drain current. Moreover, ΔG LT gradually increases once V high exceeds 2.5 V. Therefore, the semivolatile CNT transistor has two switchable operation modes, which can be controlled by adjusting the V high level.
As mentioned above, the volatile behavior can be exploited to emulate neuronal functions. When a series of pulses (referred to as presynaptic spikes, V pre ) are applied to the volatile CNT transistor, ΔG ST is dependent on the frequency of V pre (f pre ). Figure 2e shows one example of the transient I D behavior when a train of five pulses for which f pre = 10 5 Hz is applied, and Fig. 2f shows the summarized ΔG ST values according to f pre . As the interval of each pulse becomes shorter (f pre increases), larger cumulative I D enhancement leads to higher I peak values. By contrast, as the interval of the pulse train increases, the decaying I D becomes more dominant, and I peak is suppressed. Therefore, the volatile CNT transistor can emulate the neuronal leaky-integration function that corresponds to the frequency of the input stimulus. Additionally, the compact circuit model for the volatile CNT transistor is implemented (see also Fig. S2 and Supplementary Information Note 2). Different temporal responses in the tunneling/diffusion processes can be emulated with two resistor-capacitor circuits that have different time constants. As shown in Fig. 2c, e (yellow dotted curves), the circuit model can capture the measured results with high accuracy. This circuit model can be combined with the circuit model for other components in our tactile sensor system, which allows a quantitative analysis of the entire system through the circuit simulation (this concept will be discussed later).
The operations of each component in our tactile sensor system will be discussed in the order of signal flow, wherein all signal flows are manipulated with custommade software (the detailed experimental setup is presented Fig. S3 in Supplementary Information Note 3). First, the pressure stimulus was detected by a previously demonstrated tactile sensor device (Fig. 3a) 26 that was fabricated with the use of the percolated solution that processed 99% of the metallic CNTs owing to their high bendability and material uniformity. The pressure stimulus leads to resistance changes in the tactile sensor (ΔR sensor ) in the range of 2.5-10 MΩ (Fig. 3b). The value of ΔR sensor is then converted to the frequency of the presynaptic spike (V pre ) by the VCO (Fig. 3c-the detailed design of the VCO is presented Fig. S4 in Supplementary  Information Note 4). As the pressure intensity increases, V pre oscillations are generated at an increasingly higher frequency (Fig. 3d). Consequently, f pre varies proportionally to the pressure intensity (Fig. 3e). Notably, because the VCO is designed based on a conventional digital circuit, the signal conversion (R sensor versus f pre ) can be characterized accurately by circuit simulations (Fig. 3e).
Subsequently, the output of the VCO (V pre ) was delivered to the neuronal device, i.e., to the volatile CNT transistor (Fig. 4a). Among our 10 × 10 CNT transistors 27,28 , one selected CNT transistor served as the neuronal device that operated in volatile mode. The other 10 × 4 CNT transistors acted as a synaptic network that operated in a nonvolatile mode to classify the pattern of input pressure (as discussed in a subsequent part of this section). When the pressure stimulus was detected by the tactile sensor, V pre was applied to the gate electrode of the neuronal device, and a temporal I D enhancement and its subsequent decay were observed, as shown in Fig. 4b. Obviously, this transient I D behavior can be captured accurately through the circuit simulation from the combined VCO model and the volatile CNT transistor model discussed above. Moreover, the neuronal device can integrate temporally correlated tactile stimuli. As a proof of concept, two different tactile patterns in one row (convex or flat) were used as the target of recognition (Fig.  4c), wherein we defined the convex pattern in the pair with the number "1" and the flat pattern with "0". Therefore, the pattern pairs were labeled "'00", "01", "10", and "11". In the experiment, the tactile sensor was attached to a finger, and the finger was brought close to the patterns and was moved from left to right. A complete move/touch action was completed in ≤1 s. When the tactile sensor passes through the "1" pattern, the pressure stimulus at the sensor causes a temporal increase in I D . Moreover, the "0" pattern cannot cause any change in I D . Figure 4d shows the transient I D responses according to the different tactile patterns. Note that the "11" pattern leads to the largest I D change because this pattern provides two successive pressure stimuli. Interestingly, although both the "01" and "10" patterns have only one convex pattern, the timing information of the two patterns are different. The response to the "10" pattern decays earlier than the response to the "01" pattern. Therefore, the I D value attributed to the "01" pattern is higher than that attributed to the "10" pattern. Because the responses to the pattern pairs are distinguishable, these responses can be used as specific features for recognition.
Finally, the biorealistic perceptual learning and recognition processes are demonstrated. In principle, the transient I D responses after the completion of the move/ touch action are sampled. For example, I D is sampled after a period of 1 s at 0.1 s intervals (i.e., I D (t i ) where i = 1 to m, Fig. 5a). The input vector U i is defined as the normalized I D (t i ): U i = I D (t i )/I ref , where 0 ≤ U i ≤ 1. Next, the boundary vectors (W j , where j = 1 to n) are defined as the Fig. 3 The tactile seonsor device co-integrated with the VCO circuit. a Photograph of the tactile sensor device. b Resistance change in the tactile sensor (ΔR sensor ) according to the pressure intensity. c Photograph of the VCO integrated on the printed circuit board. d The output of the VCO (V pre ) according to ΔR sensor . Higher pressure intensity leads to higher oscillation frequency f pre . e Variation in f pre as a function of R sensor that shows the consistency with the simulation results obtained based on the circuit model of the VCO.
references for the classification of U i . In our experiment, four boundary vectors, namely, W 1 , W 2 , W 3 , and W 4 (n = 4), correspond to the tactile patterns of "00", "01", "10", and "11", respectively (the dotted curves in Fig. 5a). Based on the Euclidean distance between U i and W j , the tactile pattern that corresponds to U i can be inferred. Therefore, two processes need to be performed iteratively: (1) the learning process for determining W j with the labeled U i and (2) the recognition process based on the calculation of the Euclidean distance between U i and W j . These learning and recognition processes are similar to the knearest neighbors (KNN) algorithm, and they can be performed efficiently with a resistive crossbar array 6 . W j can be directly mapped as a set of weights in the nonvolatile CNT transistor array in a columnwise fashion, i.e., to the W matrix (W = [W 1 W 2 W 3 W 4 ]; thus, the size of W is m × n, Fig. 5b). The weights are in turn linearly mapped to the device conductance values. Additionally, to calculate the Euclidean distance, the S matrix (of size 1 × n) should be included 6 . Thus, an (m + 1) × n nonvolatile CNT transistor array (referred to as the synaptic network wherein m = 9 and n = 4) was exploited in our experiment.
The details of the learning and recognition processes are presented in Fig. S5 in Supplementary Information Note 5. In brief (Fig. 5c), during the learning process, W and S are iteratively updated according to the designed learning rule. The learning rule was based on a supervised scheme using the label U i . By using the update-verify feedback method 28,29 , the device conductance in the synaptic network was updated and had an increased accuracy as desired. Herein, because the device conductance (weight) should be maintained over a long period, the CNT transistors in the synaptic network should be operated in a nonvolatile mode. Therefore, pulses with sufficiently large V high and V low values were applied to the gate electrode (V high = +8 V, and V low = −9 V) to update the device conductance. Moreover, during the recognition process, only a small DC bias (V high = V low = V G,read = −3 V) was applied to the gate electrodes of all the CNT transistors to read the device conductance. Instead, U i was converted according to the drain voltage magnitude (0 V ≤ V D, Ui ≤ 2 V). The sum of the total source currents for each column represented the sum of the product of V D,Ui and W j (∑I S,Cj = ∑V D,Ui • W j ). As a result, as the ∑I S,Cj values obtained from a particular column in the synaptic network increased, the Euclidean distance between U i and W j -which was recorded in the column-decreased. Figure 5d shows the updating of the weights in the W matrix during the iterative learning process, wherein each column represents W j . All the weights were randomly assigned to an initial state. As the leaning process was repeated, the weights were adjusted to adhere to W 4 (i) > W 3 (i) > W 2 (i) > W 1 (i). Subsequently, when U i (without a label) was input to the learned matrix, the tactile pattern could be inferred through the column that generated the largest output current. As shown in Fig. 5e, a recognition error rate less than 5% can be obtained through 50 or more learning iterations. Therefore, our perceptual sensory system can be improved by learning repeated stimuli and can achieve a performance that is very close to the biological sensory system.

Conclusions
In summary, our tactile sensor system was based on the semivolatile CNT transistor device and was composed of a tactile sensor, VCO, neuronal device, and synaptic network. It captured essential morphological and functional similarities related to the biological sensory system. Our tactile sensor system can distinguish temporally The learning and recognition processes in the CNT transistor array. a Example of boundary vectors (W j ) and the measured data. By calculating the Euclidean distance between the measured data and each boundary vector and by finding the minimum distance, the tactile pattern that generates the measured data can be inferred. b Mapping the process of the Euclidean distance calculation onto the 10×4 CNT transistor array (synaptic network). All the boundary vectors W 1 , W 2 , W 3 , and W 4 can be directly mapped as weights in the 9×4 CNT transistor array (W matrix) in a columnwise fashion. Additionally, the 1×4s matrix is included in the synaptic network. The input vector is coded using pulses with different levels and is then applied to the drain of the CNT transistor in the row direction. The integrated currents at the column outputs allow direct comparisons of the Euclidean distances. The highest column current (∑I S,Cj ) represents the shortest Euclidean distance between U i and W j . c Flowchart of the learning and recognition processes. d The measured evolution of the W matrix weights (i.e., device conductance of each CNT transistor) during the iterative learning process. Each column represents the corresponding boundary vector. e The recognition error rate as a function of the learning iterations. Each data point is the averaged error rate following 30 repetitions. correlated pressure stimuli, and the features of tactile patterns can be extracted for pattern recognition. Note that the recognition accuracy can be improved through an iterative learning process, thus illustrating a similarity to biological perceptual learning processes. In addition to the prior attempts executed to capture the similarities of the human sensory system, the developed circuit models of our tactile sensor system enabled a quantitative analysis of the entire system that achieved product-level feasibility based on its similarity with existing digital circuits. Moreover, the developed learning and recognition processes can be equally applicable to other advanced sensors [30][31][32][33] .