Abstract
Metal-oxide memristors have emerged as promising candidates for hardware implementation of artificial synapses – the key components of high-performance, analog neuromorphic networks - due to their excellent scaling prospects. Since some advanced cognitive tasks require spiking neuromorphic networks, which explicitly model individual neural pulses (“spikes”) in biological neural systems, it is crucial for memristive synapses to support the spike-time-dependent plasticity (STDP). A major challenge for the STDP implementation is that, in contrast to some simplistic models of the plasticity, the elementary change of a synaptic weight in an artificial hardware synapse depends not only on the pre-synaptic and post-synaptic signals, but also on the initial weight (memristor’s conductance) value. Here we experimentally demonstrate, for the first time, an STDP behavior that ensures self-adaptation of the average memristor conductance, making the plasticity stable, i.e. insensitive to the initial state of the devices. The experiments have been carried out with 200-nm Al2O3/TiO2−x memristors integrated into 12 × 12 crossbars. The experimentally observed self-adaptive STDP behavior has been complemented with numerical modeling of weight dynamics in a simple system with a leaky-integrate-and-fire neuron with a random spike-train input, using a compact model of memristor plasticity, fitted for quantitatively correct description of our memristors.
Similar content being viewed by others
Introduction
In biological neural systems, neurons communicate with each other with action potential pulses - “neural spikes1”. While some of the network activity information is encoded in the average spiking rate, many experiments in neurobiology suggest that the timing of individual spikes matters and is essential for coordinated processing of temporal and spatial information2,3. Indeed, encoding information with single spikes or inter-spike intervals provides a higher information capacity than the firing-rate codes which represent only the average spiking activity4. These observations motivated the development of spiking neuromorphic hardware circuits which explicitly model neural spikes1,4. An additional motivation5,6,7 for pursuing spiking neuromorphic networks is their higher energy efficiency, recently demonstrated in a very large system8.
In the simplest spiking neuromorphic networks, each neuron is modeled as a leaky-integrate-and-fire unit, which integrates incoming spikes and fires its own spike when the integrated action potential reaches a certain threshold1. The fired spike, weighed according to the strengths of the corresponding synapses, is applied to the input of other neurons. Additionally, the fired spike is also propagated backwards to the input synapses to provide their weights’ adaptation – for example, according to the spike-timing-dependent-plasticity (STDP) rule9,10,11,12, which ensures Hebb-like learning9. For the most common STDP type, found for example in Layer 5 of the neocortex13, the synaptic weight is increased if the post-synaptic spike follows soon after the pre-synaptic one (implying their causal relation), is decreased if their timing order is opposite (implying a random coincidence of the spikes) and is virtually unaltered if the time interval Δt between the spikes is larger than a few milliseconds.
The STDP-enabling hardware based on traditional integrated circuit technologies, in which synaptic weight values are stored, for example, in digital static-random access memories8, or as analog charges in switched capacitor structures7, can hardly ensure the network density necessary for cortex-scale systems. On the other hand, the values may be stored as conductivities of very compact, two-terminal, nonvolatile devices, “memristors”, which may be scaled down to ensure such density14,15,16. This is why, following several suggestions of STDP implementation in memristors, using various shapes of pre- and post-synaptic pulses17,18,19,20,21,22, there has been a recent surge of experimental demonstrations of the STDP functionality in organic23,24,25,26, complex-oxide27, sulfide28,29, silicon-oxide30, hafnium-oxide31 and phase-change32 devices.
In this work we have shown that in metal-oxide memristors STDP may be self-adaptive, excluding the need in the continuous adjustment of the average conductance of each device.
Results
All experiments were carried out with Pt/Al2O3/TiO2−x/Ti/Pt memristors integrated in 12 × 12 crossbar circuits (Fig. 1a) – see the Methods section for fabrication details. Figure 1b shows a typical switching hysteresis of a crossbar-integrated memristor at a quasi-DC symmetric voltage sweep. The ON/OFF current ratio measured at a non-disturbing bias of 0.2 V is close to 10. The results of a detailed electrical characterization of these crossbar-integrated devices, including their switching endurance of at least 5,000 cycles, projected retention time in excess of 10 years and low variability of forming and switching voltages, were reported earlier33.
In the first set of experiments, we have implemented three different biologically-plausible “STDP windows”, i.e. the dependences of the synaptic weight change on the time interval Δt between the pre- and post-synaptic spikes (Fig. 2). In particular, getting each experimental point shown on the bottom panels (g, h, i) of Fig. 2 involved three steps. First, memristor’s conductance G, which represents its synaptic weight and was measured at 0.2 V, was set to an initial value G0 ≈ 33 μS, using a simple but efficient tuning algorithm34. Pre-synaptic and post-synaptic pulses of the waveforms shown on one of the top panels (a, b, c) of Fig. 2, selected following the recommendations of Ref. 21, were then applied to the top and bottom wires leading to the selected memristor inside the crossbar, with a certain delay Δt between the pulses, while the remaining lines of the crossbar were kept grounded. Finally, after the pulse application, the new value of memristor’s conductance was measured and its change calculated. The experiment was repeated 10 times for each particular Δt, every time resetting the device to the same initial conductance with 10% accuracy. As Fig. 2 shows, these three different spike shapes result in three different representative STDP window shapes found in various biological synapses13. Other window shapes, e.g., those which correspond to Δt sign flip (and hence may be used for the anti-Hebbian rule implementation), may be readily obtained by changing switching polarity and/or modifying pulse timings.
The initial conductance of 33 μS, chosen for the described set of experiments, crudely corresponds to the geometrical mean of two extreme conductance values for the considered memristors. In the second set of tests, the experiment with waveforms corresponding to the first STDP window (Fig. 2g) was repeated for several different initial values G0 of conductance, spanning the whole dynamic range of our memristors. As this Fig. 3a shows, that STDP for G0 = 33 μS is balanced, i.e. the maximum change in conductance is roughly the same for positive and negative Δt values. On the other hand, there is no increase in conductance when its initial value is close to its maximum value and no decrease in conductance in the opposite case, i.e. when G0 is close to device’s minimum conductance. Such saturation in the switching dynamics is typical for many types of memristors15,16,33,34,35.
This strong dependence of memristor’s plasticity on its initial state might cause concerns about the possible need in continuous external tuning of each device, which would make large-scale spiking networks impracticable. To investigate this issue, we have carried out numerical simulation of the STDP adaptation, using an analytical, phenomenological (“compact”) model of the experimentally observed conductance change for the particular STDP window shown in Fig. 2g. It has turned out that the change is well described by the following product:
where a, b, c and d are fitting parameters. (In some publications36,37, such functions are called “multiplicative”; note, however, that though at each of the two intervals of Δt, ΔG is indeed a product of separate functions of G0 and Δt, globally it is not, since according to Eq. (3), function ΛG depends not only on G0, but also on Δt – via its sign. Due to this reason, the plots of ΔG as a function of Δt, shown with the continuous surface in Fig. 3b, are not globally self-similar.) As the dots in Fig. 3 show, this function, with an appropriate choice of the fitting parameters, describes the experimentally observed behavior very reasonably - see also Fig. S1 and its discussion in Supplementary Information for additional details. Moreover, we believe that such G0-dependent STDP behavior may be expected for many types of memristive devices with saturating switching dynamics15,16,33,34,35.
Using the STDP model so verified, we have simulated the time evolution of memristor conductances in a simple, generic neuromorphic network with just one soma, described with the leaky-integrate-and-fire model and 100 input synapses (Fig. 4). The network was fed with similar spikes of the shape shown with black lines in Fig. 2a, with random, independent, Poisson-distributed initiation times with 14 Hz average spiking rate. As Fig. 4d shows, memristor conductances eventually evolve to a stable bell-curve distribution independent of their initial values, with the peak of the distribution centered in the intermediate value of the dynamic range. This is not quite surprising, because our model qualitatively corresponds to the typical STDP behavior observed in biology11,38 and also to phenomenological “multiplicative” models that predict similar self-adaptation36,37, which is deemed necessary for long-term stability of spiking neural networks. (As illustrated by the bottom panel of Fig. 4d, so-called “additive” STDP models, in which ΔG is independent of G036, cannot ensure such self-adaptation.)
Discussion
The demonstrated dependence of the STDP window on the applied voltage waveforms (cf. panels (a–c) and (g–i) of Fig. 2) may be readily explained by taking into account that memristor’s conductance changes mostly when the net voltage applied to the device exceeds certain switching threshold voltage – see the dashed lines in Fig. 1b and panels (d–f) of Fig. 2. As the result, the conductance change ΔG semi-quantitatively follows either the time maximum or the time minimum of the applied waveforms – whichever exceeds the corresponding threshold more. Panels (d–f) show these voltage extremal values for the used waveforms (a–c); their comparison with the corresponding experimental STDP windows shown in panels (g–i) indeed confirms their similarity. Some slight deviations from this correspondence, for example, the time asymmetry of the window shown on panel (i), may be readily explained by the switching dynamics dependence on the conductive state of the memristor.
Another unexpected anomaly of the data is the presence of the (weak and broad) second peak in the distribution of final conductances in the numerical simulation of synaptic self-adaptation – see Fig. 4d. This peak might be suppressed by balancing device’s asymmetry by slightly varying parameters of the STDP – see Fig. S2 and its discussion.
Summary
In conclusion, we have experimentally demonstrated that Al2O3/TiO2-based memristors may be used to implement the spike-time dependent plasticity with STDP window shapes similar to those observed in biological neural systems. By fitting the experimental data with a simple compact model, we have shown that such STDP behavior enables self-adaptation of the synaptic weights to a narrow interval in the intermediate value of their dynamic range, at least in a simple (but very representative) spiking network. These results give every hope for stable operation of future large neuromorphic networks based on such memristors.
Additional Information
How to cite this article: Prezioso, M. et al. Self-Adaptive Spike-Time-Dependent Plasticity of Metal-Oxide Memristors. Sci. Rep. 6, 21331; doi: 10.1038/srep21331 (2016).
References
Gerstner, W. & Kistler, W. Spiking Neuron Models (Cambridge U. Press, New York, NY, 2008).
Bialek, W., Reike, F., Ruyter van Steveninck, R. R. & Warland, D. Reading a neural code. Science 252, 1854– 1857 (1991).
Palm, G., Aertsen, A. M. H. J. & Geirstein, G. L. On the significance of correlations among neural spike trains, Biological Cybernetics 59, 1–11 (1988).
Maas, W. Networks of spiking neurons: The third generation of neural network models. Neural Networks 10, 1659–1671 (1997).
Indiveri, G. et al. Neuromorphic silicon neuron circuits. Frontiers in Neuroscience 5 (2011).
Pfeil, T. et al. Six networks on a universal neuromorphic computing substrate. Frontiers in Neuroscience 7 (2013).
Benjamin, B. V. et al. Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations. Proceedings of the IEEE 102, 699–716 (2014).
Merolla, P. A. et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 8, 668–673 (2014).
Hebb, D. O. The Organization of Behavior (Wiley & Sons, New York, NY, 1949).
Gerstner, W., Ritz, R. & Van Hemmen, J. L. Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biological cybernetics 69, 503–515 (1993).
Bi, G.-Q. & Poo, M.-M. Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength and postsynaptic cell type. Journal of Neuroscience 18, 10464–10472 (1998).
Caporale, N. & Dan, Y. Spike timing–dependent plasticity: A Hebbian learning rule. Annual Review of Neuroscience 31, 25–46 (2008).
Abbott, L. F. & Nelson, S. B. Synaptic plasticity: Taming the beast. Nature Neuroscience 3, 1178–1183 (2000).
Likharev, K. K. CrossNets : Neuromorphic hybrid CMOS/nanoelectronic networks. Science of Advanced Materials 3, 322–331 (2011).
Pickett, M. D. et al. Switching dynamics in titanium dioxide memristive devices. Journal of Applied Physics 106, 074508 (2009).
Yang, J. J., Strukov, D. B. & Stewart, D. R. Memristive devices for computing. Nature Nanotechnology 8, 13–24 (2013).
Snider, G. S. Spike-timing-dependent learning in memristive nanodevices. NanoArch’08, 85–92 (2008).
Likharev, K. K. Hybrid CMOS/nanoelectronic circuits: Opportunities and challenges. J. Nanoel. & Optoel. 3, 203–230 (2008).
Linares-Barranco B. & Serrano-Gotarredona T. Memristance can explain Spike-Time-Dependent Plasticity in neural synapses. Nature Preceedingshttp://precedings.nature.com/documents/3010/ (2009).
Afifi, A., Ayatollahi, A. & Rassi, R. STDP implementation using memristive nanodevice in CMOS-nano neuromorphic networks. IEICE Electronics Express 6, 148–153 (2009).
Zamarreño-Ramos, C. et al. On spike-timing-dependent-plasticity, memristive devices and building a self-learning visual cortex. Frontiers in Neuroscience 5 (2011).
Saighi, S. et al. Plasticity in memristive devices for spiking neural networks. Frontiers in Neuroscience 9 (2015).
Alibart, F. et al. An organic nanoparticle transistor behaving as a biological spiking synapse. Advanced Functional Materials 20, 330–337 (2010).
Li, S. et al. Synaptic plasticity and learning behaviours mimicked through Ag interface movement in an Ag/conducting polymer/Ta memristive system. Journal of Materials Chemistry C 1, 5292–5298 (2013).
Subramaniam, A., Cantley, K. D., Bersuker, G., Gilmer, D. & Vogel, E. M. Spike-timing-dependent plasticity using biologically realistic action potentials and low-temperature materials. IEEE Trans. Nanotechnology 12, 450–459 (2013).
Zeng, F., Li, S., Yang, J., Pan, F. & Guo, D. Learning processes modulated by the interface effects in a Ti/conducting polymer/Ti resistive switching cell. RSC Advances 4, 14822–14828 (2014).
Wang, Z. Q. et al. Synaptic learning and memory functions achieved using oxygen ion migration/diffusion in an amorphous InGaZnO memristor. Advanced Functional Materials 22, 2759–2765 (2012).
Nayak, A. et al. Controlling the synaptic plasticity of a Cu2S gap-type atomic switch. Advanced Functional Materials 22, 3606–3613 (2012).
Ohno, T. et al. Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nature Materials 10, 591–595 (2011).
Jo, S. H. et al. Nanoscale memristor device as synapse in neuromorphic systems. Nano Letters 10, 1297–1301 (2010).
Mandal, S., El-Amin, A., Alexander, K., Rajendran, B. & Jha, R. Novel synaptic memory device for neuromorphic computing. Nature Scientific Reports 4, 5333 (2014).
Kuzum, D., Jeyasingh, R. G., Lee, B. & Wong, H.-S. P. Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Letters 12, 2179–2186 (2011).
Prezioso, M. et al. Training and operation of an integrated neuromorphic network based on metal-oxide memristors. Nature 521, 61–64 (2015).
Alibart, F., Gao, L., Hoskins, B. D. & Strukov, D. B. High precision tuning of state for memristive devices by adaptable variation-tolerant algorithm. Nanotechnology 23, 075201 (2012).
Prodromakis, T., Peh, B. P., Papavassiliou, C. & Toumazou, C. A versatile memristor model with nonlinear dopant kinetics. IEEE Trans. Electron Devices 58, 3099–3105 (2011).
Van Rossum, M. C., Bi, G. Q. & Turrigiano, G. G. Stable Hebbian learning from spike timing-dependent plasticity. Journal of Neuroscience 20, 8812–8821 (2000).
Rubin, J., Lee, D. D. & Sompolinsky, H. Equilibrium properties of temporally asymmetric Hebbian plasticity. Physical Review Letters 86, 364–367 (2001).
Debanne, D., Gähwiler, B. H. & Thompson, S. M. Heterogeneity of synaptic plasticity at unitary CA3–CA1 and CA3–CA3 connections in rat hippocampal slice cultures. Journal of Neuroscience 19, 10664–10671 (1999).
Acknowledgements
This work was supported by the Air Force Office of Scientific Research (AFOSR) under the MURI grant FA9550-12-1-0038, DARPA under Contract No. HR0011-13-C-0051UPSIDE via BAE Systems and Army Research Office under Contract No W15QKN-15-C-0008 via Physical Optics Corporation. Useful discussions with L. Sengupta and G. Adam are gratefully acknowledged.
Author information
Authors and Affiliations
Contributions
M.P. and B.H. fabricated devices. M.P. and M.F-B. performed the experiments. M.P. performed numerical simulations. All discussed the results. M.P., K.L. and D.S. wrote the paper. K.L. and D.S. advised on all parts of the project.
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Electronic supplementary material
Rights and permissions
This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/
About this article
Cite this article
Prezioso, M., Merrikh Bayat, F., Hoskins, B. et al. Self-Adaptive Spike-Time-Dependent Plasticity of Metal-Oxide Memristors. Sci Rep 6, 21331 (2016). https://doi.org/10.1038/srep21331
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/srep21331
This article is cited by
-
One-vs-One, One-vs-Rest, and a novel Outcome-Driven One-vs-One binary classifiers enabled by optoelectronic memristors towards overcoming hardware limitations in multiclass classification
Discover Materials (2024)
-
The FAPbI3 perovskite memristor with a PMMA passivation layer as an artificial synapse
Applied Physics A (2023)
-
Scalable nanocomposite parylene-based memristors: Multifilamentary resistive switching and neuromorphic applications
Nano Research (2023)
-
Spiking neural networks based on two-dimensional materials
npj 2D Materials and Applications (2022)
-
Thickness-dependent monochalcogenide GeSe-based CBRAM for memory and artificial electronic synapses
Nano Research (2022)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.