Computational principles of synaptic memory consolidation

Abstract

Memories are stored and retained through complex, coupled processes operating on multiple timescales. To understand the computational principles behind these intricate networks of interactions, we construct a broad class of synaptic models that efficiently harness biological complexity to preserve numerous memories by protecting them against the adverse effects of overwriting. The memory capacity scales almost linearly with the number of synapses, which is a substantial improvement over the square root scaling of previous models. This was achieved by combining multiple dynamical processes that initially store memories in fast variables and then progressively transfer them to slower variables. Notably, the interactions between fast and slow variables are bidirectional. The proposed models are robust to parameter perturbations and can explain several properties of biological memory, including delayed expression of synaptic modifications, metaplasticity, and spacing effects.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Model schematic.
Figure 2: Model construction.
Figure 3: SNR of the synaptic model.
Figure 4: Scaling properties of the synaptic model.
Figure 5: Effects of different discretization schemes of the dynamical variables on memory performance.
Figure 6: Robustness of the model.
Figure 7: Generalizations and features of the model.
Figure 8: Testing the model in experiments.

References

  1. 1

    Kandel, E., Swartz, J., Jessel, T., Siegelbaum, S. & Hudspeth, A.J. Principles of Neural Science (McGraw Hill, 2013).

  2. 2

    Bhalla, U.S. Molecular computation in neurons: a modeling perspective. Curr. Opin. Neurobiol. 25, 31–37 (2014).

    CAS  PubMed  Article  Google Scholar 

  3. 3

    Amit, D.J. & Fusi, S. Learning in neural networks with material synapses. Neural Comput. 6, 957–982 (1994).

    Article  Google Scholar 

  4. 4

    Fusi, S. Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol. Cybern. 87, 459–470 (2002).

    PubMed  Article  Google Scholar 

  5. 5

    Fusi, S. & Abbott, L.F. Limits on the memory storage capacity of bounded synapses. Nat. Neurosci. 10, 485–493 (2007).

    CAS  PubMed  Article  Google Scholar 

  6. 6

    McCloskey, M. & Cohen, N.J. Catastrophic interference in connectionist networks: the sequential learning problem. Psychol. Learn. Motiv. 24, 109–164 (1989).

    Article  Google Scholar 

  7. 7

    Carpenter, G. & Grossberg, S. Pattern Recognition by Self-Organizing Neural Networks (MIT Press, 1991).

  8. 8

    McClelland, J.L., McNaughton, B.L. & O'Reilly, R.C. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychol. Rev. 102, 419–457 (1995).

    Article  Google Scholar 

  9. 9

    Fusi, S., Drew, P.J. & Abbott, L.F. Cascade models of synaptically stored memories. Neuron 45, 599–611 (2005).

    CAS  PubMed  Article  Google Scholar 

  10. 10

    Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982).

    CAS  PubMed  Article  Google Scholar 

  11. 11

    Roxin, A. & Fusi, S. Efficient partitioning of memory systems and its importance for memory consolidation. PLoS Comput. Biol. 9, e1003146 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  12. 12

    Brady, T.F., Konkle, T., Alvarez, G.A. & Oliva, A. Visual long-term memory has a massive storage capacity for object details. Proc. Natl. Acad. Sci. USA 105, 14325–14329 (2008).

    CAS  PubMed  Article  Google Scholar 

  13. 13

    Wixted, J.T. & Ebbesen, E.B. On the form of forgetting. Psychol. Sci. 2, 409–415 (1991).

    Article  Google Scholar 

  14. 14

    Wixted, J.T. & Ebbesen, E.B. Genuine power curves in forgetting: a quantitative analysis of individual subject forgetting functions. Mem. Cognit. 25, 731–739 (1997).

    CAS  PubMed  Article  Google Scholar 

  15. 15

    Abraham, W.C. Metaplasticity: tuning synapses and networks for plasticity. Nat. Rev. Neurosci. 9, 387 (2008).

    CAS  PubMed  Article  Google Scholar 

  16. 16

    Anderson, John R. Learning and Memory (Wiley, 1995).

  17. 17

    Smolen, P., Zhang, Y. & Byrne, J.H. The right time to learn: mechanisms and optimization of spaced learning. Nat. Rev. Neurosci. 17, 77–88 (2016).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  18. 18

    Wu, X.E. & Mel, B.W. Capacity-enhancing synaptic learning rules in a medial temporal lobe online learning model. Neuron 62, 31–41 (2009).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  19. 19

    Parisi, G. A memory which forgets. J. Phys. A Math. Gen. 19, L617 (1986).

    Article  Google Scholar 

  20. 20

    Lisman, J.E. A mechanism for memory storage insensitive to molecular turnover: a bistable autophosphorylating kinase. Proc. Natl. Acad. Sci. USA 82, 3055–3057 (1985).

    CAS  PubMed  Article  Google Scholar 

  21. 21

    Fusi, S., Annunziato, M., Badoni, D., Salamon, A. & Amit, D.J. Spike-driven synaptic plasticity: theory, simulation, VLSI implementation. Neural Comput. 12, 2227–2258 (2000).

    CAS  PubMed  Article  Google Scholar 

  22. 22

    Brader, J.M., Senn, W. & Fusi, S. Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881–2912 (2007).

    PubMed  Article  Google Scholar 

  23. 23

    Graupner, M. & Brunel, N. Calcium-based plasticity model explains sensitivity of synaptic changes to spike pattern, rate, and dendritic location. Proc. Natl. Acad. Sci. USA 109, 3991–3996 (2012).

    CAS  PubMed  Article  PubMed Central  Google Scholar 

  24. 24

    Clopath, C., Ziegler, L., Vasilaki, E., Büsing, L. & Gerstner, W. Tag-trigger-consolidation: a model of early and late long-term-potentiation and depression. PLoS Comput. Biol. 4, e1000248 (2008).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  25. 25

    Ziegler, L., Zenke, F., Kastner, D.B. & Gerstner, W. Synaptic consolidation: from synapses to behavioral modeling. J. Neurosci. 35, 1319–1334 (2015).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  26. 26

    Shankar, G.M. et al. Amyloid-β protein dimers isolated directly from Alzheimer's brains impair synaptic plasticity and memory. Nat. Med. 14, 837–842 (2008).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  27. 27

    O'Connor, D.H., Wittenberg, G.M. & Wang, S.S. Graded bidirectional synaptic plasticity is composed of switch-like unitary events. Proc. Natl. Acad. Sci. USA 102, 9679–9684 (2005).

    CAS  PubMed  Article  Google Scholar 

  28. 28

    Carew, T.J., Pinsker, H.M. & Kandel, E.R. Long-term habituation of a defensive withdrawal reflex in Aplysia. Science 175, 451–454 (1972).

    CAS  PubMed  Article  Google Scholar 

  29. 29

    Zhou, Q., Tao, H.W. & Poo, M.M. Reversal and stabilization of synaptic modifications in a developing visual system. Science 300, 1953–1957 (2003).

    CAS  PubMed  Article  Google Scholar 

  30. 30

    Emes, R.D. et al. Evolutionary expansion and anatomical specialization of synapse proteome complexity. Nat. Neurosci. 11, 799–806 (2008).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  31. 31

    Lahiri, S. & Ganguli, S. A memory frontier for complex synapses. Adv. Neural Inf. Process. Syst. 26, 1034–1042 (2013).

    Google Scholar 

  32. 32

    Crick, F. Memory and molecular turnover. Nature 312, 101 (1984).

    CAS  Article  Google Scholar 

  33. 33

    Miller, P., Zhabotinsky, A.M., Lisman, J.E. & Wang, X.J. The stability of a stochastic CaMKII switch: dependence on the number of enzyme molecules and protein turnover. PLoS Biol. 3, e107 (2005).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  34. 34

    Si, K., Lindquist, S. & Kandel, E.R. A neuronal isoform of the Aplysia CPEB has prion-like properties. Cell 115, 879–891 (2003).

    CAS  PubMed  Article  Google Scholar 

  35. 35

    Shouval, H.Z. Clusters of interacting receptors can stabilize synaptic efficacies. Proc. Natl. Acad. Sci. USA 102, 14440–14445 (2005).

    CAS  PubMed  Article  Google Scholar 

  36. 36

    Ji, D. & Wilson, M.A. Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat. Neurosci. 10, 100–107 (2007).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  37. 37

    Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S.I. & Battaglia, F.P. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat. Neurosci. 12, 919–926 (2009).

    CAS  PubMed  Article  Google Scholar 

  38. 38

    Reymann, K.G. & Frey, J.U. The late maintenance of hippocampal LTP: requirements, phases, 'synaptic tagging', 'late-associativity' and implications. Neuropharmacology 52, 24–40 (2007).

    CAS  PubMed  Article  Google Scholar 

  39. 39

    Redondo, R.L. & Morris, R.G. Making memories last: the synaptic tagging and capture hypothesis. Nat. Rev. Neurosci. 12, 17–30 (2011).

    CAS  PubMed  Article  Google Scholar 

  40. 40

    Barrett, A.B., Billings, G.O., Morris, R.G. & van Rossum, M.C. State based model of long-term potentiation and synaptic tagging and capture. PLoS Comput. Biol. 5, e1000259 (2009).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

  41. 41

    Amit, D. Modeling Brain Function (Cambridge Univ. Press, 1989).

  42. 42

    Krauth, W. & Mézard, M. Learning algorithms with optimal stability in neural networks. J. Phys. A Math. Gen. 20, L745 (1987).

    Article  Google Scholar 

  43. 43

    Tsodyks, M.V. & Feigel'man, M.V. The enhanced storage capacity in neural networks with low activity level. Europhys. Lett. 6, 101–105 (1988).

    Article  Google Scholar 

  44. 44

    Barak, O., Rigotti, M. & Fusi, S. The sparseness of mixed selectivity neurons controls the generalization-discrimination trade-off. J. Neurosci. 33, 3844–3856 (2013).

    CAS  PubMed  PubMed Central  Article  Google Scholar 

  45. 45

    O'Kane, D. & Treves, A. Why the simplest notion of neocortex as an autoassociative memory would not work. Network 3, 379–384 (1992).

    Article  Google Scholar 

  46. 46

    Roudi, Y. & Latham, P.E. A balanced memory network. PLoS Comput. Biol. 3, 1679–1700 (2007).

    CAS  PubMed  Article  Google Scholar 

  47. 47

    Savin, C., Dayan, P. & Lengyel, M. Optimal recall from bounded metaplastic synapses: predicting functional adaptations in hippocampal area CA3. PLoS Comput. Biol. 10, e1003489 (2014).

    PubMed  PubMed Central  Article  CAS  Google Scholar 

Download references

Acknowledgements

We are grateful to L.F. Abbott and U.S. Bhalla for many comments on the manuscript and for discussions. This work was supported by the Gatsby Charitable Foundation, the Simons Foundation, the Swartz Foundation, the Kavli Foundation, the Grossman Foundation and RISE, the Research Initiatives for Science and Engineering. The illustrations of the beakers were generated using the free ray tracing software POV-Ray.

Author information

Affiliations

Authors

Contributions

M.K.B. conceived the original idea. M.K.B. and S.F. developed and analyzed the model, and wrote the article.

Corresponding author

Correspondence to Stefano Fusi.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Benna, M., Fusi, S. Computational principles of synaptic memory consolidation. Nat Neurosci 19, 1697–1706 (2016). https://doi.org/10.1038/nn.4401

Download citation

Further reading