Associative memory realized by a reconfigurable memristive Hopfield neural network

Although synaptic behaviours of memristors have been widely demonstrated, implementation of an even simple artificial neural network is still a great challenge. In this work, we demonstrate the associative memory on the basis of a memristive Hopfield network. Different patterns can be stored into the memristive Hopfield network by tuning the resistance of the memristors, and the pre-stored patterns can be successfully retrieved directly or through some associative intermediate states, being analogous to the associative memory behaviour. Both single-associative memory and multi-associative memories can be realized with the memristive Hopfield network. Memristors are passive electrical components that can act like simple memories. Here, the authors use an array of hafnium oxide memristors to create a type of artificial neural network, known as a Hopfield network, that is capable of retrieving data from partial information

T he idea of building a cognitive system that can adapt like the biological brain has existed for a long time 1 . However, building an artificial brain with the conventional digital computer based on von-Neumann paradigm 2 is of great difficulty. Digital computers and biological brains process information in fundamentally different ways. Digital computers process information in sequence and are inflexible, precise and deterministic 3,4 ; in contrast, biological brains process data in parallel and are flexible, not precise, error-prone and good at learning new matters 1,4 . Obviously, to efficiently realize the functionalities of biological brains, new computing architectures are required. In the past few decades, artificial neural networks (ANNs) have received much attention, as they have a natural capability for storing information and making it available for use 5 . In 1980s, Hopfield proposed a dynamic ANN called Hopfield network [6][7][8][9] . The Hopfield network has been proved useful in content-addressable memories 10 , and combinatorial optimization problems, such as the travelling salesman problem and the location allocation problem 8,11 . Previous Hopfield network was realized by constructing complementary metal-oxide-semiconductor circuits as the synapses at a cost of large chip area and power consumption 10 . In 1971, Chua predicted the fourth basic circuit element, namely, memristor 12,13 , which was later demonstrated in the laboratory by Williams et al. in 2008 (ref. 14). Subsequently, many studies demonstrated that a memristor can be used as an electronic synapse with its conductance representing the synaptic weight [15][16][17][18][19][20][21][22][23][24][25] . Although synaptic operation of memristors has been widely demonstrated, implementation of even a simple ANN is challenging. Encouragingly, some significant advances have been reported recently. For example, ANN consisting of neurons and synapses has been constructed to realize the Pavlov's dog model [26][27][28] ; Alibart et al. reported the realization of linear pattern classification using a memristive network 29 ; Park et al. realized neuromorphic speech systems using resistive randomaccess memory -based synapse 30 ; and Eryilmaz et al. reported brain-like associative learning using phase-change synaptic device array 31 . Burr et al. demonstrated a neural network with 165K synapses implemented with phase-change devices 32 .
In this work, we have successfully constructed a Hopfield network using HfO 2 memristors and peripheral devices to realize the associative memory that is capable of retrieving a piece of data upon presentation of partial information from that piece of data. The network can be reconfigured to realize various positive and negative synaptic weights. Both single-associative memory and multi-associative memories can be realized with the memristive Hopfield network (MHN). Associative memories via or not via intermediate states can be used to emulate humans' 'weak' or 'strong' memories, respectively. In addition, the proposed MHN shows good robustness to device variation and variations in threshold voltage of the neurons. This study provides a possible method for hardware implementation of artificial neuromorphic networks to emulate memorization.

Results
Memristor characterization. The memristor used in this work has a metal/oxide/metal structure, as shown in Fig. 1a. After a forming process with high voltage, resistance of the memristor can be increased or decreased by voltage bias depending on the voltage polarity. Figure 1b shows the I-V characteristics of five repeated cycles of voltage sweeping for a typical memristor. Each cycle of sweeping takes B10 s and the period of each applied voltage is 40 ms. For the positive voltage polarity, the current increases very little with voltage at low voltages, but it rises rapidly when the voltage is increased to B1 V; for the negative voltage polarity, the current increases with the voltage magnitude, but it decreases gradually when the voltage reaches B À 1 V.
Besides the voltage sweeping, voltage pulses can also be used to change the conductance of the memristor. Figure 1c shows the conductance change of the memristors with different initial conductance (G 0 ) for the pulse duration of 5 ms. Generally, a positive or negative pulse voltage leads to an increase or decrease in the conductance, respectively. The conductance shows little change for small pulse voltage, but the change is large for a large pulse voltage (for example, þ 2 V for positive pulse voltage, À 2 V for negative pulse voltage). The result indicates that the resistance of the memristor can be adjusted to the desired values with an appropriate voltage-pulse programming scheme (see Supplementary Fig. 1). The conductive filament (CF) model can be used to explain the resistance change in the HfO 2 -based resistive memristors 33,34 . Considering the bipolar nature of the memristor, the formation or rupture of some CF consisting of oxygen vacancies are responsible for the resistance change 35 . In the initialization process (that is, the forming process), CF are formed in the HfO 2 thin film to connect the two electrodes, resulting in a low-resistance state. Subsequently, a negative (positive) voltage can lead to the gradual rupture (recovery) of CF, resulting in an increase (decrease) in the resistance. The memristive switching from high-resistance state to low-resistance state could be attributed to CF formation at the grain boundaries containing a high concentration of oxygen vacancies 33 . On the other hand, electric field could play an important role in switching from low-resistance state to high-resistance state 34 . It was suggested that electrical pulses lead to a progressive narrowing of the CF, and finally a gap is formed and the memristor switches to the high-resistance state 33,34 .
MHN implementation. Basic findings from the biological neuron operation have enabled researchers to model the operations of artificial neurons 36 . A Hopfield network consists of a set of interconnected artificial neurons and synapses. In this work, a Hopfield network is constructed with nine synapses realized with six memristors and three neurons. As shown in Fig. 2a, the artificial neuron has three inputs and each input, N i (i ¼ 1, 2 and 3), is connected to a synapse with synaptic weight of w i . The output of the three-input binary artificial neuron is expressed as where y is the neuron's threshold; and the sign function is defined as: An artificial neuron was constructed, as shown in Fig. 2b. An operational amplifier is used to sum the inputs. The switches, S 1 , S 2 and S 3 , are controlled by external signals to obtain positive or negative synaptic weights. The synaptic weights corresponding to input N 1 , N 2 , and N 3 are respectively (M 1 , M 2 and M 3 are the resistance of the memristors, respectively, and the resistance of R is fixed at 3 MO). In the circuit shown in Fig. 2b, transmission gates B 1 , B 2 and B 3 are used to transfer signals without modifying the polarity of the signals; inverters I 1 , I 2 and I 3 are used to achieve negative synapse weights. Figure 2c shows the architecture of a 3-bit MHN realized with nine synapses. The synaptic weight from neuron i to neuron j is given by w i,j , which can be conveniently adjusted by tuning the resistance of the corresponding memristor M ij . M ij and w ij are represented by the resistance matrix M ¼ where t represents the number of updating cycles and t ¼ 0 represents no update taking place and the corresponding state vector is the initial vector X(0). In one updating cycle, new states of the neurons are asynchronously updated from x 1 , x 2 to x 3 in three stages, which are defined as stages a, b and c, respectively.
Single-associative memory. Associative memory is a function of brain that is capable of recalling a piece of data on the information relevant to that piece of data. In this work, patterns are stored into the MHN by tuning the resistance matrix M to obtain the desired weight matrix W (refs 11,36). An optimized scheme based on the outer-product (Hebbian) rule was employed to determine the weight matrix 6,37 . The relationship between the initial state and the final state is determined by the weight matrix, threshold and the refreshing sequence. The target memory that needs to be associatively recalled was set at '110'. To store the pattern binary '110' into the MHN, the resistance matrix was set as To achieve the targeted matrixes in equations (4) and (5), the resistances of the relevant memristors are tuned by applying step-like voltage pulses with an appropriate scheme of voltage magnitudes and pulse numbers to the memristors, which is called the training process. An offline training scheme for setting the predetermined resistances on the memristors is implemented with a C Language program embedded in the semiconductor characterization system (Keithley 4200). The training process for M 13 /M 31 , M 12 /M 21 and M 23 /M 32 is illustrated in Supplementary  Fig. 3. On the other hand, the targeted resistances of M 11 , M 22 and M 33 can be achieved directly from the low-resistance states that have a resistance of around 0.1 kO; thus no training is needed for these elements.
Once the targeted resistances are achieved, they remain unchanged during the network operation. The description of the circuit operation process is presented in Supplementary Note 1.  ARTICLE The threshold vector of the three neurons was set as

Output Inputs
If the associative memory works, the MHN can converge to '110' automatically from any state in the range from '000' to '111'. Figure 3 shows the waveforms of state vector X(t) in the process to retrieve the pre-stored '110'. Clocks at 5 kHz were used to control the MHN. The network requires seven control signals in total (see Supplementary Fig. 4). Figure 3 shows the waveforms of the states (x 1 , x 2 and x 3 ) for different initial states. In each refreshing cycle, three memristors are selected together in one column in the matrix as shown in Fig. 2c and Supplementary Fig. 2. The MHN starting from any initial state vector can successfully retrieve the pre-stored '110'. In an updating cycle, X(t) was updated in three stages and only 1 bit was updated in one stage. As an example, the updating cycles of the MHN starting from X(0) ¼ (0 0 0), as shown in Fig. 3a, are described subsequently. In the first updating cycle, the element x 1 was first updated according to equation (3), and X(1) a ¼ (1 0 0) (stage a); x 2 was then updated in stage b according to equation (3) also and X(1) b ¼ (1 1 0) Fig. 3a,b, Fig. 3h, the MHN started from X(0) ¼ (1 1 1) and it directly stabilized at X(1) ¼ (1 1 0), and no intermediate states were experienced. Direct memorization emulates a simple associative memory, that is, we can retrieve some strong memories without experiencing associative states. In Fig. 4, the retrievals of the pre-stored '110' from different initial states vectors are schematically summarized using a cube with its each corner representing a state of the MHN 11 . For different initial state vectors, the MHN may experience different intermediate state vectors before 'recalling' the pre-stored pattern. In addition to convergence to '110', other final states can also be realized by modifying the weight matrix and threshold vector.
The single-associative memory for pre-stored binary code '110' can also be illustrated with the presentation of different parts of the image of a rabbit, as shown in Fig. 4. The image of a full rabbit is equivalent to the final state '110', while the images of the rabbit partially covered up by grass are equivalent to the initial states, such as '000', '100' and so on. State '000', '100' or other initial states are associated with the final state '110' through the synaptic weight matrix in equation (5), threshold vector in equation (6)     and refreshing sequence, and these initial states represent information relevant to the final state '110'. The final state can be retrieved from the initial states; this means that the image of full rabbit can be recalled by associative memorization.
Multi-associative memories. In human brain, one can recall a piece of data on the information relevant to that piece of data by experiencing some associative states; if the given data are different, one can recall another piece of data via experiencing some other intermediate states.
In the MHN of this work, more than one pattern can be stored at the same time by reconfiguring the resistances of the memristors. To verify the multi-associative memories, '000' and '101' were pre-stored into the MHN, and the resistance matrix was set as  (7) is presented in Supplementary Fig. 5. By selecting proper switch states for each synapse, the weight matrix was set as The threshold vector of the three neurons was set as T ¼ 1 60 6 6 6 ð Þ ð 9Þ Figure 5 shows the signal waveforms of X(t) ¼ (x 1 x 2 x 3 ). As shown in Fig. 5a-d, the MHN could retrieve the pattern '000' when the initial state vectors were X(0) ¼ (0 0 0), X(0) ¼ (1 0 0), X(0) ¼ (0 1 0) or X(0) ¼ (1 1 0), respectively. In Fig. 5e-h, the MHN successfully 'recalled' the pre-stored '101' with the initial states X(0) ¼ (0 0 1), X(0) ¼ (1 0 1), X(0) ¼ (0 1 1) and X(0) ¼ (1 1 1). Similar to the single-associative memory, the MHN exhibited either strong or weak associative memories. For some initial state vectors, the MHN can directly 'recall' '000' or '101', as they have good associability. Starting from some other initial state vectors, the MHN has to experience associative intermediate state(s) before the success of retrieval, as shown in Fig. 5d,g, due to weak associability. In Fig. 6, we schematically summarize the retrieval of pre-stored '000' and '101' from different initial states vectors in a cube with each corner ...   In addition to convergence to '000' and '101', some other final states can also be realized by modifying the weight matrix and threshold vector. The multi-associative memories for pre-stored binary codes '000' and '101' can be also illustrated with the presentation of different parts of the images of a rabbit and a crane, respectively, as shown in Fig. 6. The image of a full rabbit is equivalent to the final state '000', while the images of the rabbit partially covered up by grass (or other images relevant to a full rabbit) are equivalent to the initial states '010', '100' and '110'. Similarly, '101' represents the image of a full crane, while the images of the crane partially covered up by grass are equivalent to initial states '001', '011' and '111'. With the information associated to the full rabbit (or the crane), the MHN can successfully recall the full image of the rabbit (or the crane).
Power consumption. Figure 7a,b shows the effect of threshold variation on the network for single-and multi-associative memories, respectively. The error rate is defined as Error rate ¼ Number of error states Number of total states Â100% ð10Þ An error state means the convergence is done to a wrong final state caused by the variation of threshold voltage (or resistance).The threshold voltage change, DV TH , is defined as where V TH0 is the initial threshold voltage (V TH0 ¼ 2y) and V TH1 is the threshold voltage after adjustment. As can be observed in Fig. 7a, the DV TH tolerance for single-associative memory can be around 100% in the adjustments of both negative and positive directions. For multi-associative memories, the DV TH tolerance in the negative direction can be around 25%, while it can be B15% in the positive direction as shown in Fig. 7b. The threshold variation tolerance of the single-associative memory is better than that of the multi-associative memory. Figure 8 shows the effect of resistance variation in the matrix on single-associative memory. The resistance variation DR is defined as where R 0 is the initial resistance and R 1 is the resistance after adjustment. As one memristor is used to represent two elements in the symmetric positions in equation (4), elements M 12 and M 21 are adjusted together as shown in Fig. 8a. As can be observed in Fig. 8a, in the negative direction (that is, DRo0), the error rate is zero for DRZ À 40%; however, the error rate jumps up to 50% when DRr À 52%. In the positive direction (that is, DR40), the MHN did not exhibit any error for a resistance adjustment; even when the resistance is increased by 100%, the error rate is still zero. For M 23 and M 32 , the MHN did not exhibit any errors for DR in the range of À 42 to 44%, as shown in Fig. 8b. As shown in  Figure 9 shows the effect of resistance variation in the matrix on multi-associative memories. In equation (8), symmetric elements M 12 and M 21 are adjusted together as shown in Fig. 9a. As can be observed in Fig. 9a, M 12 and M 21 can be adjusted for up to around 166% in the positive direction. In the negative direction, the network still does not exhibit any error when the resistance is adjusted for À 86%. As shown in Fig. 9b, M 23 and M 32 can be adjusted for up to B45.6% in the positive direction; in the negative direction, when the resistance is adjusted for À 75%, the network still does not exhibit any error. M 13 and M 31 show À 34.7% adjustment tolerance in the negative direction as shown in Fig. 9c. In the positive direction, the resistance of M 13 and M 31 is adjusted for 50%, the network still does not exhibit any error. It is also observed that variations in the resistances of M 11 , M 22 and M 33 in the low-resistance states does not lead to any error in the network.
Effect of resistance and threshold variations on power consumption. The power consumption of the network is around 80 mW. The core memristor array consumes only 100-300 nW; while 499% of the power is consumed by the commercial operational amplifiers (Texas Instruments LM324NP). The power consumption can be improved if the operational amplifiers are optimally designed by integration with the memristors on a single chip. Figure 10a shows the effect of the threshold variation on the power consumption of the core memristor array for single-and multi-associative memories. As can be observed in the figure, the threshold variation does not affect the power consumption for both single-and multi-associative memories. Figure 10b,c shows the effect of resistance variation on the power consumption of the core memristor array for single-and multi-associative memories, respectively. With the increase of resistance of the memristors, the power consumption also increases. This is due to the fact that the current in the memristor array does not change much, and thus the power of the core memristor array is determined by the resistance of the array (the power is approximately proportional to the resistance). On the other hand, as the resistances of M 11 , M 22 and M 33 are much smaller than that of other elements in the matrix, the influence of resistance variation of the three elements on power consumption is insignificant.

Discussion
In conclusion, a 3-bit MHN has been constructed. The synaptic weights of the MHN are programmable and can be conveniently programmed to positive or negative by adjusting the conductance of the memristors. Single-and multi-associative memories have been realized with the MHN. The study paves the way for the hardware implementation of artificial neuromorphic networks to emulate memorization via associative states.

Methods
Device fabrication and characterization. The memristor used in the MHN is based on a metal-insulator-metal structure with a thin HfO 2 layer as the insulator. The metal-insulator-metal structure was fabricated onto a SiO 2 film, which had been thermally grown on a p-type silicon wafer. An B70-nm Ni layer was deposited on the SiO 2 film using electron beam evaporation to form the bottom electrode. An HfO 2 thin film of B80 nm thickness was deposited onto the Ni layer by Radio Frequency (  Effect of threshold voltage variation on power consumption of the core memristor array for single-associative memory and multi-associative memories (a). Effect of resistance variation on power consumption of the core memristor array for single-associative memory (b) and multi-associative memories (c).
NATURE COMMUNICATIONS | DOI: 10.1038/ncomms8522 ARTICLE of the memristor was carried out with JSM-7500 F scanning electron microscope (JEOL). Electrical characteristics of the memristor were measured with a Keithley 4200 semiconductor characterization system at room temperature.
MHN fabrication and measurement. The MHN was fabricated on a printed circuit board (PCB) and was connected to the HfO 2 memristors with wires ( Supplementary Fig. 6). The MHN consists of six memristors, four transmission gate chips (Texas Instruments CD4066), seven operational amplifiers (Texas Instruments LM324N),and one comparator chip (Texas Instruments LM339) (see Supplementary Fig. 6). The complete circuit schematic of the MHN is shown in Supplementary Fig. 2. In the measurement of the MHN, a field programming gate array (model no. ALTERA EP2C8Q208C8) was used to generate the clock signals, and the waveforms of the clock signals and outputs were recorded with a RIGOL oscilloscope (model no. DS4024). The waveforms of control signals are presented in Supplementary Fig. 4.