Memories are stored and retained through complex, coupled processes operating on multiple timescales. To understand the computational principles behind these intricate networks of interactions, we construct a broad class of synaptic models that efficiently harness biological complexity to preserve numerous memories by protecting them against the adverse effects of overwriting. The memory capacity scales almost linearly with the number of synapses, which is a substantial improvement over the square root scaling of previous models. This was achieved by combining multiple dynamical processes that initially store memories in fast variables and then progressively transfer them to slower variables. Notably, the interactions between fast and slow variables are bidirectional. The proposed models are robust to parameter perturbations and can explain several properties of biological memory, including delayed expression of synaptic modifications, metaplasticity, and spacing effects.
Subscribe to Journal
Get full journal access for 1 year
only $18.75 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Kandel, E., Swartz, J., Jessel, T., Siegelbaum, S. & Hudspeth, A.J. Principles of Neural Science (McGraw Hill, 2013).
Bhalla, U.S. Molecular computation in neurons: a modeling perspective. Curr. Opin. Neurobiol. 25, 31–37 (2014).
Amit, D.J. & Fusi, S. Learning in neural networks with material synapses. Neural Comput. 6, 957–982 (1994).
Fusi, S. Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol. Cybern. 87, 459–470 (2002).
Fusi, S. & Abbott, L.F. Limits on the memory storage capacity of bounded synapses. Nat. Neurosci. 10, 485–493 (2007).
McCloskey, M. & Cohen, N.J. Catastrophic interference in connectionist networks: the sequential learning problem. Psychol. Learn. Motiv. 24, 109–164 (1989).
Carpenter, G. & Grossberg, S. Pattern Recognition by Self-Organizing Neural Networks (MIT Press, 1991).
McClelland, J.L., McNaughton, B.L. & O'Reilly, R.C. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102, 419–457 (1995).
Fusi, S., Drew, P.J. & Abbott, L.F. Cascade models of synaptically stored memories. Neuron 45, 599–611 (2005).
Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982).
Roxin, A. & Fusi, S. Efficient partitioning of memory systems and its importance for memory consolidation. PLoS Comput. Biol. 9, e1003146 (2013).
Brady, T.F., Konkle, T., Alvarez, G.A. & Oliva, A. Visual long-term memory has a massive storage capacity for object details. Proc. Natl. Acad. Sci. USA 105, 14325–14329 (2008).
Wixted, J.T. & Ebbesen, E.B. On the form of forgetting. Psychol. Sci. 2, 409–415 (1991).
Wixted, J.T. & Ebbesen, E.B. Genuine power curves in forgetting: a quantitative analysis of individual subject forgetting functions. Mem. Cognit. 25, 731–739 (1997).
Abraham, W.C. Metaplasticity: tuning synapses and networks for plasticity. Nat. Rev. Neurosci. 9, 387 (2008).
Anderson, John R. Learning and Memory (Wiley, 1995).
Smolen, P., Zhang, Y. & Byrne, J.H. The right time to learn: mechanisms and optimization of spaced learning. Nat. Rev. Neurosci. 17, 77–88 (2016).
Wu, X.E. & Mel, B.W. Capacity-enhancing synaptic learning rules in a medial temporal lobe online learning model. Neuron 62, 31–41 (2009).
Parisi, G. A memory which forgets. J. Phys. A Math. Gen. 19, L617 (1986).
Lisman, J.E. A mechanism for memory storage insensitive to molecular turnover: a bistable autophosphorylating kinase. Proc. Natl. Acad. Sci. USA 82, 3055–3057 (1985).
Fusi, S., Annunziato, M., Badoni, D., Salamon, A. & Amit, D.J. Spike-driven synaptic plasticity: theory, simulation, VLSI implementation. Neural Comput. 12, 2227–2258 (2000).
Brader, J.M., Senn, W. & Fusi, S. Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881–2912 (2007).
Graupner, M. & Brunel, N. Calcium-based plasticity model explains sensitivity of synaptic changes to spike pattern, rate, and dendritic location. Proc. Natl. Acad. Sci. USA 109, 3991–3996 (2012).
Clopath, C., Ziegler, L., Vasilaki, E., Büsing, L. & Gerstner, W. Tag-trigger-consolidation: a model of early and late long-term-potentiation and depression. PLoS Comput. Biol. 4, e1000248 (2008).
Ziegler, L., Zenke, F., Kastner, D.B. & Gerstner, W. Synaptic consolidation: from synapses to behavioral modeling. J. Neurosci. 35, 1319–1334 (2015).
Shankar, G.M. et al. Amyloid-β protein dimers isolated directly from Alzheimer's brains impair synaptic plasticity and memory. Nat. Med. 14, 837–842 (2008).
O'Connor, D.H., Wittenberg, G.M. & Wang, S.S. Graded bidirectional synaptic plasticity is composed of switch-like unitary events. Proc. Natl. Acad. Sci. USA 102, 9679–9684 (2005).
Carew, T.J., Pinsker, H.M. & Kandel, E.R. Long-term habituation of a defensive withdrawal reflex in Aplysia. Science 175, 451–454 (1972).
Zhou, Q., Tao, H.W. & Poo, M.M. Reversal and stabilization of synaptic modifications in a developing visual system. Science 300, 1953–1957 (2003).
Emes, R.D. et al. Evolutionary expansion and anatomical specialization of synapse proteome complexity. Nat. Neurosci. 11, 799–806 (2008).
Lahiri, S. & Ganguli, S. A memory frontier for complex synapses. Adv. Neural Inf. Process. Syst. 26, 1034–1042 (2013).
Crick, F. Memory and molecular turnover. Nature 312, 101 (1984).
Miller, P., Zhabotinsky, A.M., Lisman, J.E. & Wang, X.J. The stability of a stochastic CaMKII switch: dependence on the number of enzyme molecules and protein turnover. PLoS Biol. 3, e107 (2005).
Si, K., Lindquist, S. & Kandel, E.R. A neuronal isoform of the Aplysia CPEB has prion-like properties. Cell 115, 879–891 (2003).
Shouval, H.Z. Clusters of interacting receptors can stabilize synaptic efficacies. Proc. Natl. Acad. Sci. USA 102, 14440–14445 (2005).
Ji, D. & Wilson, M.A. Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat. Neurosci. 10, 100–107 (2007).
Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S.I. & Battaglia, F.P. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat. Neurosci. 12, 919–926 (2009).
Reymann, K.G. & Frey, J.U. The late maintenance of hippocampal LTP: requirements, phases, 'synaptic tagging', 'late-associativity' and implications. Neuropharmacology 52, 24–40 (2007).
Redondo, R.L. & Morris, R.G. Making memories last: the synaptic tagging and capture hypothesis. Nat. Rev. Neurosci. 12, 17–30 (2011).
Barrett, A.B., Billings, G.O., Morris, R.G. & van Rossum, M.C. State based model of long-term potentiation and synaptic tagging and capture. PLoS Comput. Biol. 5, e1000259 (2009).
Amit, D. Modeling Brain Function (Cambridge Univ. Press, 1989).
Krauth, W. & Mézard, M. Learning algorithms with optimal stability in neural networks. J. Phys. A Math. Gen. 20, L745 (1987).
Tsodyks, M.V. & Feigel'man, M.V. The enhanced storage capacity in neural networks with low activity level. Europhys. Lett. 6, 101–105 (1988).
Barak, O., Rigotti, M. & Fusi, S. The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-off. J. Neurosci. 33, 3844–3856 (2013).
O'Kane, D. & Treves, A. Why the simplest notion of neocortex as an autoassociative memory would not work. Network 3, 379–384 (1992).
Roudi, Y. & Latham, P.E. A balanced memory network. PLoS Comput. Biol. 3, 1679–1700 (2007).
Savin, C., Dayan, P. & Lengyel, M. Optimal recall from bounded metaplastic synapses: predicting functional adaptations in hippocampal area CA3. PLoS Comput. Biol. 10, e1003489 (2014).
We are grateful to L.F. Abbott and U.S. Bhalla for many comments on the manuscript and for discussions. This work was supported by the Gatsby Charitable Foundation, the Simons Foundation, the Swartz Foundation, the Kavli Foundation, the Grossman Foundation and RISE, the Research Initiatives for Science and Engineering. The illustrations of the beakers were generated using the free ray tracing software POV-Ray.
The authors declare no competing financial interests.
About this article
Cite this article
Benna, M., Fusi, S. Computational principles of synaptic memory consolidation. Nat Neurosci 19, 1697–1706 (2016). https://doi.org/10.1038/nn.4401
Integrating prediction errors at two time scales permits rapid recalibration of speech sound categories
Neural Computation (2020)
Conditioning by subthreshold synaptic input changes the intrinsic firing pattern of CA3 hippocampal neurons
Journal of Neurophysiology (2020)