Spiking neurons with spatiotemporal dynamics and gain modulation for monolithically integrated memristive neural networks

As a key building block of biological cortex, neurons are powerful information processing units and can achieve highly complex nonlinear computations even in individual cells. Hardware implementation of artificial neurons with similar capability is of great significance for the construction of intelligent, neuromorphic systems. Here, we demonstrate an artificial neuron based on NbOx volatile memristor that not only realizes traditional all-or-nothing, threshold-driven spiking and spatiotemporal integration, but also enables dynamic logic including XOR function that is not linearly separable and multiplicative gain modulation among different dendritic inputs, therefore surpassing neuronal functions described by a simple point neuron model. A monolithically integrated 4 × 4 fully memristive neural network consisting of volatile NbOx memristor based neurons and nonvolatile TaOx memristor based synapses in a single crossbar array is experimentally demonstrated, showing capability in pattern recognition through online learning using a simplified δ-rule and coincidence detection, which paves the way for bio-inspired intelligent systems.


4
S2 is random relative to S1), the neuron cannot fire (Fig. 7c,d). These results are consistent with the spatiotemporal dynamics of the neuron in Fig. 3k,l and imply the potential of the fully memristive neural network in coincidence detection.
To further extend the applicability of the neuronal spatiotemporal dynamics in large-scale neural computation, we have simulated a spiking neural network using Brian2 simulator 48 , where the neuronal parameters were extracted from electrical measurements (Supplementary Fig. 15,Supplementary Note 8)  Poisson process at 400 kHz, and only 15 excitatory spike trains (<1% in proportion) are randomly picked to simultaneously fire for each event (Fig. 7f). Simulations results show that the firing rate of the neuron can be increased by over 10 times as a result of the synchronous events, as shown in Fig. 7g,h, therefore indicating the potential of the present neuronal dynamics in detecting fine temporal correlations in massive signals and small timescales." on S1 and S2. (c) The neuronal response by two asynchronous input pulse trains (1.1 V in amplitude, 1 μs in width, 1.5 μs in interval, for 50 cycles) applied on S1 and S2, where S2 is behind S1 by 1.2 μs. (d) The neuronal response by two asynchronous input pulse trains (1.1 V in amplitude, 1 μs in width, for 50 cycles) applied on S1 and S2, where S1 has and interval of 1.5 μs and S2 is random relative to S2 in timing.
(e) Input pulse trains of the neuron from independent 4000 excitatory and 1000 inhibitory random spike trains following Poisson statistics. (f) Introduction of synchronous events following Poisson statistics 7 In Fig. 4d, although the shapes of the curves corresponding to modulatory inputs of 0.6 V and 0.7 V appear to be similar due to the similar voltage amplitudes, their shapes especially the slopes are dramatically different from the case without modulatory input (m = 0), thus demonstrating a change in neuronal gain. Since the slopes are increased upon application of higher modulatory inputs, the gain modulation has a multiplicative nature (Ref. 17). But we appreciate the reviewer's valuable advice and fully agree that the data quality originally in Fig. 4d are not perfect, so we have performed additional experiments and revised Fig. 4, in order to examine the nature of the neuronal gain more clearly. The further experimental results unambiguously show that the slope of f-d curve increases as the amplitude of modulatory inputs increases (Fig. 4d), demonstrating that the neuronal gain modulation has a multiplicative nature (Ref. 17). In addition, we have revised the model in Eqs. (2-4) slightly to better fit the new experimental data (Fig. 4e), which has the same input gain modulation property as described in the previous model.
The multiplicative gain modulation of the artificial neuron can be an enabling factor in receptive field remapping based on memristive neural networks, which can significantly enhance the stability of artificial visual systems. We have constructed a neural network in simulation based on the experimentally verified gain modulation behavior in Fig. 4, which clearly showcases the receptive field remapping.
We have included these new results in revised Figs. 3a,along with the following discussions: Page 17: "More specifically, experimentally results have demonstrated that the slope is increased upon application of a higher modulatory input (Fig. 4d), and hence the gain modulation has a multiplicative nature 17 ." Page 27: "The gain modulation of the spiking neuron (Fig. 4)  Based upon the memristive neurons with gain modulation (Fig. 4), a onedimensional network model is constructed, which includes 200 neurons connected in an uni-direction, as shown in Fig. 8a. Denote di as the input to neuron i, whose dynamics is given by:   (a) A one-dimensional network where neurons are connected in an uni-direction. Each neuron receives a modulatory signal (orange arrow). A visual input is applied at the future RF (FRF) of a neuron (the yellow one). (b) The visual input at FRF and the modulatory signal (top, red line) triggers a propagation of neural activity from FRF to CRF of the yellow neuron, as if the RF of the yellow neuron is temporally expanded when the modulatory signal is applied, achieving RF remapping (right panel). The distance of remapping is controlled by the duration of the modulatory signal. In the simulation, the visual input is applied during 0-300 ms, while the modulatory signal m = 0.8 is applied during 300-1000 ms. θthr = 0.41, Wi, i-1 = 0.095, τ = 10 ms, dt =1 ms, Iext, 0 = 0.09. Figure 2), from the reviewer's point of view, should be the synaptic device and not part of the neuron. The reviewer suggests the authors provide a pulse count vs. input frequency plot so that it can be compared with those in Figure 4.

The load resistor (in
Our response: Thanks for the suggestion. We fully agree that the load resistor is indeed equivalent with the synaptic device. As a result, we have revised the schematic in Fig. 2d to reflect the correct implication on this point, as also appended below. The following revised sentences were also added to Page 7 "The TS behavior of the Pt/Ti/NbOx/Pt/Ti device can mimic the dynamics of an ion channel located near the soma of a neuron, while the membrane capacitance is represented by Cm".  Figure 3 in the main text to make it clear about the physical connection. In addition, since the pulse count should be discrete values, while that in the 3D plot in Fig. 3c and 3f seem to be continuous values, could the authors comment on it?

The reviewer suggests using a circuit diagram (or microscopic images, etc.) in
Our response: We would like to thank the reviewer for the helpful suggestion. A circuit diagram is now included as Fig. 3a in the revised manuscript, along with the following sentences in Page 10 "Therefore, we use the circuit diagram in Fig. 3a to emulate this function, where the load resistors (R1 and R2)  shown as 3D surface plots. This is now clearly described in the figure caption, and the 13 revised Fig. 3 is appended below for the reviewer's convenience. were systematically changed from 0.6 V to 0.9 V, and applied simultaneously to the respective nodes.
(e) The input-output characteristics for "AND" logic with two input pulses applied on S1 and S2. (f) Schematic diagram of spatial summation with different input pulse intervals. (g) Neuronal response triggered by two input pulses (0.8 V amplitude, 1 μs width, interval 0.1 μs, 10 pulse cycles) applied on S1 and S2 individually and simultaneously. (h) The spatial summation at varied conditions and corresponding fitting results. The intervals of the two input spikes were systematically changed from 0.05 μs to 0.4 μs. (i) The input-output characteristics for "XOR" logic with two input pulses applied on S1 and S2. (j) Schematic diagram of spatiotemporal summation with different time intervals of input pulses. (k) Neuronal response triggered by two input pulses (0.9 V amplitude, 1 μs width, 0.1 μs interval) applied on S1 and S2 with different time intervals of 0, 2.2 and -2.2 μs. (l) Spatiotemporal summation results when the time interval of two input spikes is changed from -6.6 to 6.6 μs. The solid line is fitting result by Eq. (1).

The authors provide an empirical function for the spatiotemporal summation,
which is not used in the neural network demonstration. Since the summation that is used in the later demos is the spatial summation, could the authors provide an equation for Fig. 3c

The demonstrated Boolean logic (AND, OR) is a neuron connected with two synapses (two load resistors) for a linear neural network, which is not entirely new.
It could be more interesting to demonstrate that the nonlinear dynamic (spatiotemporal input?) of the artificial neuron plays a role in the network, e.g. to implement a logic complete NOR or NAND function.
Our response: We would like to greatly thank the reviewer for the very constructive comment. Drawing inspiration from this suggestion, we have used the spiking neuron in the present study to implement XOR function, which is a typical Boolean logic that is not linearly separable and therefore cannot be realized by a linear neural network Specifically, the input voltages to S1 or S2 serve as the input variables p and q, respectively. 0 V is defined as logic "0" for the inputs, and the logic "1" for p and q are defined to be 0.9 V and -0.9 V, respectively. The firing frequency of the artificial neuron is taken as the logic output, whose threshold is set to 0.3 MHz. Given the fact that the threshold switching in NbOx is independent on the input voltage polarity ( We have added the new results in Fig. 3i and Supplementary Fig. 8, along with further discussion in Page 12 "More importantly, the spiking neuron in the present study can also be utilized to implement XOR function, which is a typical Boolean logic that is not linearly separable and therefore cannot be realized by a linear neural network 40 . To do this, the neuronal threshold is set to 0.3 MHz, and the logic "1" for S1 or S2 is designated to be 0.9 V and -0.9 V, respectively. Given the fact that the threshold switching in NbOx is independent on the input voltage polarity (Fig. 2b,Supplementary Figs. 4 and 5), the firing frequency of the neuron exceeds the threshold when either p or q is "1" but cannot reach the threshold when p = 1 and q = 1 are applied simultaneously, as experimentally demonstrated in Fig. 3i and Supplementary Fig. 8.
The successful implementation of logic functions that are not linearly separable, like XOR, further demonstrates the potential of the artificial neuron in achieving complex computing." were systematically changed from 0.6 V to 0.9 V, and applied simultaneously to the respective nodes.
(e) The input-output characteristics for "AND" logic with two input pulses applied on S1 and S2.  Figure 6a illustrates a 3-layer spiking neural network, which is composed of 784 input neurons, 100 hidden neurons and 10 output neurons, where the 784 inputs and 10 outputs correspond to a MNIST data size of 28×28 and 10 possible classes (from 0 to 9), respectively. We evaluate the network performance by MNIST handwritten digit classification, and detailed simulation process is shown in Fig. 6b Fig. 14c).  Supplementary Fig. 14c)  The seemingly different neuronal responses in spike count vs. voltage amplitude between Supplementary Fig. 9b (now Supplementary Fig. 14b) and Fig. 2f lies in two aspects: circuit parameter setup and device-to-device variation. While the RL used in Fig. 2f is 3.6 kΩ, the RL used in Supplementary Fig. 14b is 2.2 kΩ, and the numbers of applied pulses in Fig. 2f and Supplementary Fig. 14b were 20 and 10, respectively, as noted already in Methods and the caption of Supplementary Fig. 9b (now Supplementary Fig. 14b). The lower load resistance decides that the NbOx neuron can start to fire at a lower amplitude of ~0.95 V, and the spike count under the same voltage amplitude is generally higher in the latter case, especially considering the smaller number of applied pulses ( Supplementary Fig. 14b). The remaining difference can be attributed to device-to-device variations. However, in both cases the spike count shows an overall increasing trend, as the pulse amplitude increases. We have added related discussion in Supplementary Note 7 to address this point.
8. Since the volatile memristor might age very quickly as each neuron fire event will switch it. Could the authors comment on endurance performance, and how does it affect the neural network performance? The other concerns are the cycleto-cycle variation and device-to-device variation, and speed/power advantages over competitors, but they are less important in this study, given it is a proof-ofconcept at the current stage.
Our response: We would like to thank the reviewer for raising these points. We have performed new experiments to measure the endurance of the device, and the results show that the device can still function correctly after >10 9 switching cycles, as shown in Supplementary Fig. 2. Such endurance is promising for applications, but the reviewer is absolutely right that the requirement for switching in every firing event places high demand for the aging property of the devices. Fortunately, recent studies revealed that the threshold switching effects may not be from insulator-metal transition as believed previously but can be interpreted by a trap-assisted conduction mechanism similar to Poole-Frenkel model with moderate Joule heating, which actually suggest an electronic nature and much lower switching temperature (Refs. R1-R2). This may indicate a potential for further optimization on the endurance of NbOx based devices.
In light of the reviewer's advice, we have also examined the cycle-to-cycle and device-to-device variation of the devices. Supplementary Fig. 4a shows the I-V  Table 2). Here, the power consumption refers to the peak power consumed by the threshold switching (or resistive switching) device when the neuron fires. It can be found that the switching speed of NbOx threshold switching device in the present work is <50 ns from off-to on-state and <25 ns from on-to offstate ( Supplementary Fig. 3). The intrinsic switching speed of NbOx was reported to be   [R3-R6] 1.2-480 μW [R7-R9] NbOx <10 ns [R10-R13] 10-1600 μW [R14-R16] VOx <700 ps [R17-R20] 23.75-2400 μW [R21-R23] Ge2Sb2Te5 <20 ns [R24-26] ~4.3 μW [R26] NbOx (This work) <50/25 ns ~392 μW In order to address this question, we have added these new results as Supplementary Table 2. We also added the following discussion into the main text to reflect the correct implication:

Figs. 2-5 and Supplementary
Page 6: "Symmetric hysteresis loops were observed in both bias polarities (Fig. 2b) showing the capability to perform pattern recognition through online learning using a simplified -rule. In my opinion the paper deserves acceptance in its current form. However, in preparation for the camera ready version, the authors should follow the guidelines below.
Our response: We would like to sincerely thank the reviewer for pointing out the novelty and significance of our study, and for the very detailed and constructive suggestions this reviewer kindly made. In this revised manuscript, we have carefully considered all the advices, performed new experiments/simulations (revised Figs. 3,4 & 6,new Figs. 7 & 8,new Supplementary Figs. 2,3,5,6,7,8,14c & 15) and revised the manuscript accordingly. Detailed changes and responses can be found below. In this paper the mechanisms behind threshold switching behaviour and negative differential resistance effects in a NbOx memristor device were elucidated by applying concepts from the theory of local activity. Please give credit to this work in the bibliography.

A seminal paper on the modelling and investigation of the nonlinear dynamics of a locally-active NbOx-based memristor from Ascoli et al. is
Our response: We would like to thank the reviewer for pointing out this important reference, which is now included into the revised manuscript as Ref. 34. It is recommended to give appropriate credit to these two papers in the bibliography of your manuscript.

The
Our response: We would like to sincerely thank the reviewer for pointing out these two important references, which represent latest understandings on the threshold switching and NDR effects in NbOx memristors. We have included these two works as Refs. 30 and 31, and revised the discussion on the threshold switching mechanism in revealed that such effects can be well interpreted by a trap-assisted conduction mechanism similar to Poole-Frenkel model with moderate Joule heating, therefore suggesting an electronic nature and much lower switching temperature than the previous insulator-metal transition model 30,31 ." 3. In Fig. 1 the text "Pest-neuron" seems to be incorrect.
Our response: Thank you very much for pointing out this spelling mistake, which should be "Pre-neuron". This has been corrected in the revised Fig. 1, as appended below.

Check the English throughout the manuscript. There are several typos.
Our response: Thanks for the suggestion. We have thoroughly checked the revised manuscript to remove any spelling mistakes. Fig. 2(e) state that ROFF is the initial resistance of the threshold switching device.

In commenting
Our response: We would like to thank the reviewer for the remark. We have now 28 added definition for ROFF in Page 8 of the revised manuscript to reflect this point: "ROFF > RL, where ROFF is the initial resistance of NbOx threshold switching device". Fig. 2(e) is the pulse voltage applied to the left of RL in Fig. 2(d) right? It would be nice if you could also plot the voltage across the threshold switching device.

The voltage depicted in
Our response: We would like to thank the reviewer for the valuable remark. In light of this advice, we have performed new measurements on the voltage across the threshold switching device, and the results are included as Supplementary Fig. 6 in the revised manuscript, along with the following discussion in Page 8 of the main text: " Supplementary Fig. 6 further shows the input voltage, current response and the voltage across the threshold switching device as functions of time, which illustrate the dynamic switching process clearly".
Supplementary Figure 6. The input voltage (blue curve), output current (orange curve) and the voltage across the threshold switching device (green curve) using the circuit in Fig. 2d. 7. In Fig. 2(d) the symbol used for RL is typically adopted for a memristor. Please use the symbol of a resistor instead.
Our response: Thanks for the kind suggestion. Figure 2d has been re-drawn to use the right symbol for a load resistor. The revised figure is also appended below for the reviewer's convenience. 8. In Fig. 2(e) replace R with RL.
Our response: We would like to thank the reviewer for the careful review. This has been corrected, as can be found in revised Fig. 2 in response to question #7. Fig. 2: replace R with RL.

Caption of
Our response: We would like to thank the reviewer for the careful review. This has been corrected, as can be found in revised Fig. 2 in response to question #7. Fig. 3: "were systematically changed from 0.6 V to 0.9 V" -> "were systematically changed from 0.6 V to 0.9 V, and applied simultaneously to the respective nodes"

Caption of
Our response: We would like to thank the reviewer for the kind suggestion. This sentence has been revised accordingly.

page 14 "significant" -> "significance"
Our response: We would like to thank the reviewer for the careful review. This spelling mistake has been corrected in the revised manuscript.

The difference between input modulation and output modulation should be clarified.
Our response: In light of the advice, we have added the following discussion in Page 15 of the revised manuscript: "There are also two forms of neuronal gain modulation through synaptic input. One is input modulation (orange line in Fig. 4b), where the rate coded input-output (I-O) relationship is shifted along the x-axis upon altering the modulatory input, and the other is output modulation (purple line in Fig. 4b) (Fig. 4b)."

What does m signify? Which property of the modulatory input voltage Vm does it symbolise? Does it represent its amplitude Vm?
Our response: Yes "m" represents the amplitude of the modulatory input. In order to avoid confusions, we have added its definition in Page 16 "… where m is the amplitude of the modulatory input".
14. "d50 is the value of the driving input…" -> "d50 is the frequency of the driving input…" Our response: We would like to thank the reviewer for the careful review. This sentence has been revised accordingly. slightly to better fit the new experimental data (Fig. 4e), which has the same input gain modulation property as described in the previous model. We have included the following discussion in the revised manuscript to address this point:

Why is the gain at
Our response: We would like to thank the reviewer for the careful review. This sentence has been corrected in the revised manuscript.

Page 18 "the neurons have been successfully trained, and the neural network
recognizes "1010", indicating" -> "the neuron has been successfully trained, since it recognizes "1010", indicating" Our response: We would like to thank the reviewer for the careful review. This sentence has been corrected in the revised manuscript.  Figure 6a illustrates a 3-layer spiking neural network, which is composed of 784 input neurons, 100 hidden neurons and 10 output neurons, where the 784 inputs and 10 outputs correspond to a MNIST data size of 28×28 and 10 possible classes (from 0 to 9), respectively. We evaluate the network performance by MNIST handwritten digit classification, and detailed simulation process is shown in Fig. 6b  20. End of page 20. "process, and the statistics of the firing numbers" -> "process, and depicts also the statistics of the firing numbers" Our response: We would like to thank the reviewer for the careful review. This grammar mistake has been corrected in the revised manuscript. Fig. 6(b) could be described in some detail in the text.

The flow chart of the offline training in
Our response: Thanks for the suggestion. We have added the following discussion in Page 21-22 of the revised manuscript to explain the flow chart in Fig. 6 Fig. 14c)." 22. The definition of the confusion matrix could be provided in the text.
Our response: We would like to thank the reviewer for the advice. Our response: The reviewer is absolutely right on this. We have removed the references to "Mott transition" from the Discussion section and other places of the revised manuscript, in order to reflect the correct indication on this point.

document.
Our response: Thanks for all the constructive comments above.