Limits on the memory storage capacity of bounded synapses

Article metrics

Abstract

Memories maintained in patterns of synaptic connectivity are rapidly overwritten and destroyed by ongoing plasticity related to the storage of new memories. Short memory lifetimes arise from the bounds that must be imposed on synaptic efficacy in any realistic model. We explored whether memory performance can be improved by allowing synapses to traverse a large number of states before reaching their bounds, or by changing the way these bounds are imposed. In the case of hard bounds, memory lifetimes grow proportional to the square of the number of synaptic states, but only if potentiation and depression are precisely balanced. Improved performance can be obtained without fine tuning by imposing soft bounds, but this improvement is only linear with respect to the number of synaptic states. We explored several other possibilities and conclude that improving memory performance requires a more radical modification of the standard model of memory storage.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Distributions F of strength for synapses potentiated by the tracked memory and constrained by hard bounds.
Figure 2: Memory performance with hard bounds.
Figure 3: Soft bounds.
Figure 4: Memory performance with generalized soft boundaries.
Figure 5: Optimal soft-boundaries.

References

  1. 1

    Bliss, T.V. & Collingridge, G.L. A synaptic model of memory: long-term potentiation in the hippocampus. Nature 361, 31–39 (1993).

  2. 2

    Bredt, D.S. & Nicoll, R.A. AMPA receptor trafficking at excitatory synapses. Neuron 40, 361–379 (2003).

  3. 3

    Amit, D.J. Modeling Brain Function (Cambridge University Press, New York, 1989).

  4. 4

    Hertz, J., Krogh, A. & Palmer, R.G. Introduction to the Theory of Neural Computation (Addison Wesley Longman, Boston, 1991).

  5. 5

    Amit, D.J. & Fusi, S. Constraints on learning in dynamic synapses. Network 3, 443–464 (1992).

  6. 6

    Amit, D.J. & Fusi, S. Learning in neural networks with material synapses. Neural Comput. 6, 957–982 (1994).

  7. 7

    Fusi, S. Hebbian spike-driven synaptic plasticity for learning patterns of mean firing rates. Biol. Cybern. 87, 459–470 (2002).

  8. 8

    Staubli, U. & Lynch, G. Stable depression of potentiated synaptic responses in the hippocampus with 1–5 Hz stimulation. Brain Res. 513, 113–118 (1990).

  9. 9

    Larson, J., Xiao, P. & Lynch, G. Reversal of LTP by theta frequency stimulation. Brain Res. 600, 97–102 (1993).

  10. 10

    O'Dell, T.J. & Kandel, E.R. Low-frequency stimulation erases LTP through an NMDA receptor–mediated activation of protein phosphatases. Learn. Mem. 1, 129–139 (1994).

  11. 11

    Xiao, M.Y., Niu, Y.P. & Wigstrom, H. Activity-dependent decay of early LTP revealed by dual EPSP recording in hippocampal slices from young rats. Eur. J. Neurosci. 8, 1916–1923 (1996).

  12. 12

    Zhou, Q., Tao, H.W. & Poo, M-m. Reversal and stabilization of synaptic modifications in a developing visual system. Science 300, 1953–1957 (2003).

  13. 13

    Barnes, C.A. Memory deficits associated with senescence: a neurophysiological and behavioral study in the rat. J. Comp. Physiol. Psychol. 93, 74–104 (1979).

  14. 14

    Ahissar, E. et al. Dependence of cortical plasticity on correlated activity of single neurons and on behavioral context. Science 257, 1412–1415 (1992).

  15. 15

    Manahan-Vaughan, D. & Braunewell, K.H. Novelty acquisition is associated with induction of hippocampal long-term depression. Proc. Natl. Acad. Sci. USA 96, 8739–8744 (1999).

  16. 16

    Fu, Y.-X. et al. Temporal specificity in the cortical plasticity of visual space representation. Science 296, 1999–2003 (2002).

  17. 17

    Xu, L., Anwyl, R. & Rowan, M.J. Spatial exploration induces a persistent reversal of long-term potentiation in rat hippocampus. Nature 394, 891–894 (1998).

  18. 18

    Abraham, W.C., Logan, B., Greenwood, J.M. & Dragunow, M. Induction and experience-dependent consolidation of stable long-term potentiation lasting months in the hippocampus. J. Neurosci. 22, 9626–9634 (2002).

  19. 19

    Villarreal, D.M., Do, V., Haddad, E. & Derrick, B.E. NMDA receptor antagonists sustain LTP and spatial memory: active processes mediate LTP decay. Nat. Neurosci. 5, 48–52 (2002).

  20. 20

    Jenkins, J. & Dallenbach, K. Oblivescence during sleep and waking period. Am. J. Psychol. 35, 605–612 (1924).

  21. 21

    Brown, M.W. & Xiang, J.Z. Recognition memory: neuronal substrates of the judgement of prior occurrence. Prog. Neurobiol. 55, 149–189 (1998).

  22. 22

    Wixted, J.T. & Ebbesen, E.B. Genuine power curves in forgetting: a quantitative analysis of individual subject forgetting functions. Mem. Cognit. 25, 731–739 (1997).

  23. 23

    Bi, G.-Q. & Poo, M.-M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, ad postsynaptic cell type. J. Neurosci. 18, 10464–10472 (1998).

  24. 24

    Parisi, G. A memory which forgets. J. Phys. A. 19, L617–L620 (1986).

  25. 25

    Fusi, S., Drew, P.J. & Abbott, L.F. Cascade models of synaptically stored memories. Neuron 45, 599–611 (2005).

  26. 26

    Brunel, N., Carusi, F. & Fusi, S. Slow stochastic Hebbian learning of classes of stimuli in a recurrent neural network. Network 9, 123–152 (1998).

  27. 27

    Fusi, S. & Senn, W. Eluding oblivion with smart synaptic updates. Chaos 16, 026112 (2006).

  28. 28

    Kahn, P.E., Wong, K.Y.M. & Sherrington, D. A memory model with novel behaviour in sequential learning. Network. Comput. Neural Sys. 6, 415–427 (1995).

  29. 29

    van Rossum, M.C., Bi, G.Q. & Turrigiano, G.G. Stable Hebbian learning from spike timing-dependent plasticity. J. Neurosci. 20, 8812–8821 (2000).

  30. 30

    Rubin, J.E. Steady states in an iterative model for multiplicative spike-timing dependent plasticity. Network 12, 131–140 (2001).

  31. 31

    Gutig, R., Aharonov, R., Rotter, S. & Sompolinsky, H. Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity. J. Neurosci. 23, 3697–3714 (2003).

  32. 32

    Festa, R. & Galleani D'Agliano, E. Diffusion coefficient for a brownian particle in a periodic field of force. Physica A 90A, 229–244 (1978).

  33. 33

    Petersen, C.C., Malenka, R.C., Nicoll, R.A. & Hopfield, J.J. All-or-none potentiation at CA3–CA1 synapses. Proc. Natl. Acad. Sci. USA 95, 4732–4737 (1998).

  34. 34

    O'Connor, D.H., Wittenberg, G.M. & Wang, S.S.-H. Graded bidirectional synaptic plasticity is composed of switch-like unitary events. Proc. Natl. Acad. Sci. USA 102, 9679–9684 (2005).

  35. 35

    O'Connor, D.H., Wittenberg, G.M. & Wang, S.S.-H. Dissection of bidirectional synaptic plasticity into saturable unidirectional processes. J. Neurophysiol. 94, 1565–1573 (2005).

  36. 36

    Montgomery, J.M. & Madison, D.V. Discrete synaptic states define a major mechanism of synapse plasticity. Trends Neurosci. 27, 744–750 (2004).

  37. 37

    Scoville, W.B. & Milner, B. Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21 (1957).

  38. 38

    Chklovskii, D.B., Mel, B.W. & Svoboda, K. Cortical rewiring and information storage. Nature 431, 782–788 (2004).

  39. 39

    Willshaw, D.J., Buneman, O.P. & Longuet-Higgins, H.C. Non-holographic associative memory. Nature 222, 960–962 (1969).

  40. 40

    Grossberg, S. Processing of expected and unexpected events during conditioning and attention: a psychophysiological theory. Psychol. Rev. 89, 529–572 (1982).

  41. 41

    Rosenblatt, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65, 386–408 (1958).

  42. 42

    Block, H. The perceptron: a model for brain functioning. I. Rev. Mod. Phys. 34, 123–135 (1962).

  43. 43

    Minsky, M.L. & Papert, S.A. Perceptrons (MIT Press, Cambridge, Massachusetts, 1969; expanded edition, 1988).

  44. 44

    Tsodyks, M.V. & Feigelman, M.V. The enhanced storage capacity in neural networks with low activity level. Europhys. Lett. 6, 101–105 (1988).

  45. 45

    Tsodyks, M.V. Associative memory in neural networks with binary synapses. Mod. Phys. Lett. B B4, 713–716 (1990).

  46. 46

    Abraham, W.C. & Bear, M.F. Metaplasticity: the plasticity of synaptic plasticity. Trends Neurosci. 19, 126–130 (1996).

  47. 47

    Fischer, T.M., Blazis, D.E., Priver, N.A. & Carew, T.J. Metaplasticity at identified inhibitory synapses in Aplysia. Nature 389, 860–865 (1997).

  48. 48

    Montgomery, J.M. & Madison, D.V. State-dependent heterogeneity in synaptic depression between pyramidal cell pairs. Neuron 33, 765–777 (2002).

Download references

Acknowledgements

We are grateful to M. Mattia for useful discussions about Brownian particles in periodic potentials. This research was supported by US National Institute of Mental Health grant 58754 and by a US National Institutes of Health Director's Pioneer Award, part of the NIH Roadmap for Medical Research, through grant number 5-DP1-OD114-02.

Author information

Correspondence to L F Abbott.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Rights and permissions

Reprints and Permissions

About this article

Further reading