Electronic system with memristive synapses for pattern recognition

Memristive synapses, the most promising passive devices for synaptic interconnections in artificial neural networks, are the driving force behind recent research on hardware neural networks. Despite significant efforts to utilize memristive synapses, progress to date has only shown the possibility of building a neural network system that can classify simple image patterns. In this article, we report a high-density cross-point memristive synapse array with improved synaptic characteristics. The proposed PCMO-based memristive synapse exhibits the necessary gradual and symmetrical conductance changes, and has been successfully adapted to a neural network system. The system learns, and later recognizes, the human thought pattern corresponding to three vowels, i.e. /a /, /i /, and /u/, using electroencephalography signals generated while a subject imagines speaking vowels. Our successful demonstration of a neural network system for EEG pattern recognition is likely to intrigue many researchers and stimulate a new research direction.

Scientific RepoRts | 5:10123 | DOi: 10.1038/srep10123 using a simple crossbar array 13 . In this paper, we present a high-density cross-point memristive synapse array with improved synaptic characteristics, and describe a learning scheme to mitigate the unintended switching problem often encountered with cross-point arrays. The conductance of our memristive synapse changes more gradually and symmetrically in the presence of voltage pulses above a certain threshold voltage; otherwise, the memristive synapse retains its conductance.
We also present experimental results from a memristive HNN system that recognizes the human thought pattern relating to three vowels, /a/, /i/, and /u/, based on electroencephalography (EEG) signals generated while a subject imagines speaking these vowels. We believe that this is the first primitive prototype of an electronic system that utilizes a cross-point memristive synapse array for EEG pattern recognition. Our results suggest a new research direction for memristive HNNs.

Results
System description. As shown in fig. 1, the proposed system can be divided into two functional blocks. The first block, mainly realized in software, captures the EEG signals from a subject, and processes them to extract the distinct features of three vowels. Each feature is converted into a series of 32-bit binary code to be used by the 32 pre-neurons to generate a spike signal. The second block is a single-layer neural network that consists of 32 pre-neurons, 192 memristive synapses, and 6 post-neurons. The pre-neurons are hard-coded into a field-programmable gate array (FPGA), and 192 memristive synapses in a cross-point memristive synapse array are selected for synaptic interconnection. A leaky integrate-and-fire neuron is used as the post-neuron, and decision logic determines which post-neuron fires first based on the output signals from the post-neurons. The system's overall control signals are also generated by the FPGA. Note that, to increase the system's recognition rate, a pairwise comparison is performed. In other words, the post-neurons are paired into three groups, and each group compares two of the three vowels, i.e. /a/ vs /i/, /a/ vs /u/, and /i/ vs /u/.
The system operates in two different modes, learning and testing. In learning mode, the system's learning network adjusts the conductance of memristive synapses using spike signals generated according to the EEG signals of the pre-selected vowel. In testing mode, the subject randomly chooses and imagines saying one of the three vowels. The system analyses the subject's EEG signal, and attempts to determine which of the three vowels was chosen. EEG analysis of imagined speech. The experimental paradigm for this study is described in fig. 2(a).
The acquired EEG data are segmented according to the trial and the stimuli associated with /a/, /i/, and /u/ ( Fig. 2(b), top left). The segmented data are then analysed to identify the distinct features of the three experimental conditions (/a/, /i/, and /u/).
Initially, continuous raw EEG data are segmented into each condition, with artifact rejection used to counter the low signal-to-noise ratio of EEG data. To estimate the activity evoked by the speech imagery data, we averaged the EEG data over all trials for each condition, a process known as time-lock analysis. A time-frequency analysis was then conducted using the Morlet wavelet. As shown in the top right panel of fig. 2(b), the alpha band (8-12 Hz) activities of each vowel are distinct. To identify the current sources of the speech imagery EEG data, we conducted source localization using a Laplacian-weighted current density estimator. As shown on the right of the second row in fig. 2(b), the current sources of speech imagery EEG data were located close to Broca's and Wernicke's areas, as well as the primary and secondary cortex. Broca's area and Wernicke's area are closely related to speech production and perception, respectively.
Based on these results, we extracted the features needed to classify speech imagery EEG data for each vowel. First, we applied an IIR band-pass filter (Butterworth order: 5, bandwidth: 8-30 Hz) and baseline correction using pre-stimulus data to eliminate residual noise. We then used independent component analysis to eliminate artefacts, and decomposed the EEG data into intrinsic mode functions (IMFs) using multivariate empirical mode decomposition (MEMD). After MEMD, the dominant alpha-band IMF was extracted based on the time-frequency analysis. To enhance the classification performance, we applied a common spatial pattern filter that maximizes the variance between groups. Finally, we binarized the extracted features for input to the hardware.

Considerations for implementing a cross-point memristive synapse array.
To simplify the architecture of HNN, memristive synapses with advanced synaptic behavior should use identical pulses and a simple cross-array structure with two terminals. Advanced synaptic behavior implies not only gradual, but also symmetric responses in both potentiation and depression. The asymmetric I-V characteristics (|current (@ + 1 V)|<<|current (@ − 1 V)|) of a previous Al/PCMO memristive synapse 14 are shown in fig. 3(a). The Al/PCMO memristive synapse has an inhomogeneous barrier at the interface between the Al (reactive metal) and PCMO (p-type semiconductor) 15 . Because the conductance of the memristive synapse must be simultaneously updated in the HNN, the asymmetric conductance response ( Fig. 3(b)) from identical pulses limits the overall accuracy of recognition 16 . Advanced synaptic behavior also requires the conductance to vary continuously when a relatively large voltage bias is applied, but the conductance should remain constant when a smaller or no bias is applied. This characteristic is vital for the nondestructive read/write scheme that eliminates the unintended switching issue, as explained in detail below. A simple, two-terminal cross array structure is necessary for the practical implementation of high-density HNN 17 . In this work, we have enhanced the synaptic behavior without a complicated programming scheme by engineering a PCMO-based memristive synapse. We optimized the nitrogen concentration during TiN x deposition (Fig. 3(c)) to minimize the inhomogeneous barrier between the top electrode(TE)-PCMO interface 18 . This allowed us to obtain symmetric responses for the I-V and conductance characteristics ( Fig. 3(d),(e)). We also developed a 200 mm wafer-scale PCMO-based memristive synapse (Fig. 3(f)) that exhibits excellent switching uniformity and analogue memory behaviour (see Supplementary Fig. S1 online). Memristive HNN learning. To classify feature code by using memristive HNN, we require a learning algorithm. We propose a modified learning algorithm based on a conventional and widely used supervised learning algorithm 19 . In conventional supervised learning, feature codes and label data are required. The firing neuron is predetermined by the label data and synaptic weights are updated by feature codes, allowing the predetermined neuron to fire. To apply this algorithm to memristive HNN, the unintended switching problem must be carefully handled.
In the learning mode(rather than testing mode), the unintended switching issue causes non-linear conductance changes of unwanted memristive synapses. Generally, in memory applications 20 , this problem is easily solved using the half-bias scheme. However, it is not so simple in memristive HNN, because multiple memristive synapses must be updated at the same time (see Supplementary Fig. S2 online).
To overcome this problem, the conventional half-bias scheme was modified to enable its application to memristive HNN. The proposed learning requires two operation phases for a given feature code, potentiation and depression. The spike signals applied to the top electrode (TE) and bottom electrode (BE) are determined according to label data and feature codes. Note that when the pulse amplitude between the TE and BE is in the range |V|<V R ≈ 1 V (where V R is the read voltage), the state changes are negligible. To update the memristive synapse conductance, the training voltage (|V T | > V R ) should be supplied to the target cell by applying V H (or V L ) and V L (or V H ) to the TE and BE of the cross-point array, respectively. As shown in fig. 4, the proposed learning algorithm for a single feature code can be easily explained with simple memristive HNN (4 pre-neurons & 2 post-neurons) and detail of the circuit implementation are presented in the bottom-left of fig. 5.
In the potentiation phase, if the label data is '1' , the TE of all synapses in the first row (labelled as Top1) is connected to V L , then the BE of each synapse is either connected to V H for a feature code of '1' , or to V CM for '0' . As V H and V L are 5 V and 3 V spike signals, respectively, the synapse whose BE is connected to V H sees a 2 V potential difference across it and increases its conductance. Whereas the synapses in the first row are trained, those in the second row retain their states by applying V CM to Top2. This guarantees that the potential difference across the synapses in the second row does not exceed V R and minimizes unintended switching effect.
In the depression phase, conductance of the selected synapses reduces by applying V H and V L to the TE and BE of the associated synapses, respectively. This process is repeated until all the memristive synapses are trained for all feature codes.
Notice that the initial conductance was set to a mid-value between minimum (≈1.5 nA/V) and maximum (≈5.5 nA/V) conductance. As shown in fig. 3(e), the change in conductance for each learning pulse depends on the current state of a memristive synapse. When a memristive synapse is in its low-conductance state, pulses will result in a rapid increase in conductance. The change rate is about 0.2 ~ 0.5 nA/V for each pulse. The change rate gradually decreases to about 0.02 ~ 0.05 nA/V as the synapse approaches its high-conductance state. Finally, the conductance becomes saturated.

Memristive HNN testing and classification results.
In testing mode, applied feature codes to the memristive HNN are recognized by the decision logics based on the output signals of the post-neurons. Like the learning mode, the testing mode requires two operation phases, integrating and refractory. The spike signals applied in these periods are shown in the bottom-right of fig. 4(b) and detail of circuit implementation is shown in the bottom-right of fig. 5.
A leaky integrate-and-fire neuron, including a comparator and inverting leaky integrator, is used as the post neuron 21 . Integrator output decreases as the current flowing through the memristive synapse accumulates on the integration capacitor during the integrating phase. If the trained conductance is large, we observe a large amplitude in the input current. Thus, the output voltage quickly decreases. As soon as the integrator output drops below a neuron's threshold voltage (V TH = 3 V), the comparator output generates a high value (logic value of 1), and the neuron is assumed to have fired. Note that the time required for the integrators to reach V TH with a given feature code depends on the trained conductance of the memristive synapse. After the integrating phase has completed, the refractory period starts. During this phase, the charge on the integration capacitor needs to be fully discharged through the leaky path to prepare the neuron for the next feature code.
The output measured at each integrator during the testing mode is shown in fig. 6. The outputs from six integrators for the feature /a/ are shown in the first column. As expected, the integrators' output drops almost linearly to their saturation value which depends on the conductance of the memristive synapse and the RC time constant of the integrator, at different rates after an initial reset (=4 V). Thus, for /a/, two of the six neurons generate a fire signal as the integrators' output reaches V TH and the feature code applied to the system is finally recognized as /a/ through the decision logic. After the integration phase, all integrators' outputs are reset to 4 V during the refractory phase. The output responses for /u/ and /i/ are also shown in fig. 6, which are similar results as for /a/.

Discussion
For the first time, we have developed an electronic system implemented as a memristive HNN for EEG pattern recognition. We engineered cross-point memristive synapse array using a 200 mm wafer scale. Based on the EEG results, we extracted the features for classifying speech imagery EEG data for each vowel. We tested the proposed memristive HNN system using preprocessed EEG feature codes, and were able to achieve impressive classification results (see Supplementary Fig. S3 online). This result provides a pathway for the future design of high density memristive HNN by overcoming the scalability, connectivity, and synaptic density challenges.

EEG experiment. This study was approved by the Institutional Review Board (IRB) of Gwangju
Institute of Science and Technology (GIST) and all subjects provided written informed consent. The experiments were carried out in accordance with the approved guidelines of IRB. Six healthy subjects participated in this study. None of the subjects have experienced any neurological disorders, and all are right-handed. A pre-test familiarization experiment was conducted for all subjects. EEG data were recorded at 250 samples/s using a 64-channel EEG device made by Electrical Geodesics, Inc. Electrodes were placed according to the international 10-20 system, and those corresponding to the sensory motor cortex (based on neurophysiological insight) were used for further analysis. We recorded EEG data in a dimly lit room. All experiments were performed in complete silence to reduce the occurrence of noise. During the EEG recording, subjects were seated in a comfortable armchair wearing a set of earphones (ER-4P, Etymotic Research). The experiments consisted of four parts, and the total length of each trial was 3.5 s. The first part, lasting 1 s, was the syllable cue period, in which an /a/, /i/, or /u/ sound was provided at random to the subjects through the earphones. The EEG data recorded during the next period was used for baseline correction. In the last part, subjects imagined speaking the vowel, /a/, /i/, or /u/, which they had heard during the syllable cue period. A fixation mark was presented on the computer monitor before every syllable cue, and an imagination period lasting 250 ms allowed the subjects to prepare to listen or imagine the syllable.
We conducted 100 trials for each syllable; thus, a total of 300 trials per subject were recorded. The IIR band-pass filter was applied to the raw EEG data to eliminate residual artefacts during the analysis period (Butterworth order: 5, bandwidth: 8-30 Hz). The baseline correction was conducted using EEG data recorded during the baseline correction period. MEMD was used to extract the task-related time-frequency components from the filtered EEG data. To extract more effective features, we used the common spatial pattern that maximized the variance between the groups (/a/, /i/, and /u/). The feature vectors extracted from the common spatial pattern were binarized to 32 bits. The preprocessed 32-bit  Supplementary Fig. S4 online illustrates the cross-point region between the two electrodes. To fabricate this structure, a bottom electrode consisting of a 50 nm-thick Pt layer (deposited by electron beam evaporation) and a 20 nm-thick polycrystalline PCMO film (deposited by RF magnetron sputtering) were applied by conventional lithography and reactive ion etching. During the PCMO deposition, the substrate temperature was maintained at 600 °C. Subsequently, an 80 nm-thick SiN x layer was deposited by plasma enhanced chemical vapour deposition, followed by the formation of holes by conventional lithography and reactive ion etching. For the top electrode, 25 nm-thick TiNx (N 2 ambient), 10 nm-thick AlO x layer (as an internal resistor), and 80 nm-thick Pt were sequentially deposited and patterned.
HNN circuit implementation and measurement. Supplementary Fig. S5 online shows the single-layer HNN implemented on printing circuit board. The circuit includes a switch array, switch control logic, 32 × 6 cross-point memristive synapse array, and six neuron circuits. The cross-point Figure 6. The recognition results of speech imagination. Measured output of each integrator during testing mode is shown. When feature code of /a/ is used as an input of memristive HNN, its results is shown in first column. In the cases of /u/ and /i/, its results are also shown in second and third column, respectively.