Review Article | Published:

Computational principles of memory

Nature Neuroscience volume 19, pages 394403 (2016) | Download Citation

Abstract

The ability to store and later use information is essential for a variety of adaptive behaviors, including integration, learning, generalization, prediction and inference. In this Review, we survey theoretical principles that can allow the brain to construct persistent states for memory. We identify requirements that a memory system must satisfy and analyze existing models and hypothesized biological substrates in light of these requirements. We also highlight open questions, theoretical puzzles and problems shared with computer science and information theory.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    Conditioned Reflexes (Oxford University Press, London, 1927).

  2. 2.

    , & Classical conditioning in a simple withdrawal reflex in Aplysia californica. J. Neurosci. 1, 1426–1437 (1981).

  3. 3.

    & A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. in Classical Conditioning: Current Research and Theory (Appleton-Century-Crofts, 1972).

  4. 4.

    The Psychology of Animal Learning (Academic Press, Oxford, 1974).

  5. 5.

    A Theory of Learning and Generalization (Springer-Verlag, New York, 2002).

  6. 6.

    , & A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

  7. 7.

    , & Predictability, complexity and learning. Neural Comput. 13, 2409–2463 (2001).

  8. 8.

    Integrating with neurons. Annu. Rev. Neurosci. 12, 33–45 (1989).

  9. 9.

    How the brain keeps the eyes still. Proc. Natl. Acad. Sci. USA 93, 13339–13344 (1996).

  10. 10.

    Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory. J. Neurosci. 16, 2112–2126 (1996).

  11. 11.

    & The ecology of foraging behavior: implications for animal learning and memory. Annu. Rev. Psychol. 36, 141–169 (1985).

  12. 12.

    & The neural basis of decision making. Annu. Rev. Neurosci. 30, 535–574 (2007).

  13. 13.

    The Principles of Psychology Vol. 1 (Holt, New York, 1890).

  14. 14.

    & Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21 (1957).

  15. 15.

    , , & Time scales of memory, learning, and plasticity. Biol. Cybern. 106, 715–726 (2012).

  16. 16.

    , & Structural components of synaptic plasticity and memory consolidation. Cold Spring Harb. Perspect. Biol. 7, a021758 (2015).

  17. 17.

    , & A brief history of time (constants). Cereb. Cortex 6, 93–101 (1996).

  18. 18.

    , & Kinetic models of synaptic transmission. Methods Neuronal Model. 2, 1–25 (1998).

  19. 19.

    & Short-term synaptic plasticity. Annu. Rev. Physiol. 64, 355–405 (2002).

  20. 20.

    et al. Systems-wide proteomic analysis in mammalian cells reveals conserved, functional protein turnover. J. Proteome Res. 10, 5275–5284 (2011).

  21. 21.

    Biophysics of Computation: Information Processing in Single Neurons (Oxford University Press, New York, New York, 1999).

  22. 22.

    & The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci. 13, 334–350 (1993).

  23. 23.

    & The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J. Neurosci. 18, 3870–3896 (1998).

  24. 24.

    & Proteostasis in complex dendrites. Nat. Rev. Neurosci. 14, 638–648 (2013).

  25. 25.

    A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948).

  26. 26.

    Information Theory, Inference and Learning Algorithms (Cambridge University Press, Cambridge, 2004).

  27. 27.

    & Grid cells generate an analog error-correcting code for singularly precise neural computation. Nat. Neurosci. 14, 1330–1337 (2011).

  28. 28.

    & A tradeoff between accuracy and flexibility in a working memory circuit endowed with slow feedback mechanisms. Cereb. Cortex 25, 3586–3601 (2014).

  29. 29.

    & Catastrophic interference in connectionist networks: the sequential learning problem. Psychol. Learn. Motiv. 24, 109–165 (1989).

  30. 30.

    , & Cascade models of synaptically stored memories. Neuron 45, 599–611 (2005).

  31. 31.

    & A memory frontier for complex synapses. Adv. Neural Inf. Process. Syst. 26, 1034–1042 (2013).

  32. 32.

    & Neuron activity related to short-term memory. Science 173, 652–654 (1971).

  33. 33.

    Cellular basis of working memory. Neuron 14, 477–485 (1995).

  34. 34.

    & Directing the mind's eye: prefrontal, inferior and medial temporal mechanisms for visual working memory. Curr. Opin. Neurobiol. 15, 175–182 (2005).

  35. 35.

    & Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J. Physiol. (Lond.) 232, 331–356 (1973).

  36. 36.

    The Croonian Lecture: La fine structure des centres nerveux. Proc. R. Soc. Lond. 55, 444–468 (1894).

  37. 37.

    Conditioned Reflexes and Neuron Organization (Cambridge University Press, Cambridge, 1948).

  38. 38.

    The Organization of Behavior: a Neuropsychological Theory (New York: Wiley & Sons, 1949).

  39. 39.

    , & The synaptic plasticity and memory hypothesis: encoding, storage and persistence. Philos. Trans. R. Soc. Lond. B Biol. Sci. 369, 20130288 (2014).

  40. 40.

    , & The molecular and systems biology of memory. Cell 157, 163–186 (2014).

  41. 41.

    Vestibulo-ocular reflex arc. Arch. Neurol. Psychiatry 30, 245–291 (1933).

  42. 42.

    & Improvements in ionic relays. British patent number: GB 148582 (1918).

  43. 43.

    First Draft of a Report on the EDVAC (University of Pennsylvania, Philadelphia, 1945).

  44. 44.

    , & Memory Systems: Cache, Dram, Disk (Morgan Kaufmann, 2010).

  45. 45.

    A mechanism for memory storage insensitive to molecular turnover: a bistable autophosphorylating kinase. Proc. Natl. Acad. Sci. USA 82, 3055–3057 (1985).

  46. 46.

    Studies on the effects of structure on the behavior of enzyme systems. Bull. Math. Biophys. 16, 203–257 (1954).

  47. 47.

    & Excitatory and inhibitory interactions in localized populations of model neurons. Biophys. J. 12, 1–24 (1972).

  48. 48.

    & Computing with neural circuits: a model. Science 233, 625–633 (1986).

  49. 49.

    The existence of persistent states in the brain. Math. Biosci. 19, 101–120 (1974).

  50. 50.

    , & Theory of orientation tuning in visual cortex. Proc. Natl. Acad. Sci. USA 92, 3844–3848 (1995).

  51. 51.

    & Accurate path integration in continuous attractor network models of grid cells. PLoS Comput. Biol. 5, e1000291 (2009).

  52. 52.

    , & Robust spatial working memory through homeostatic synaptic scaling in heterogeneous cortical networks. Neuron 38, 473–485 (2003).

  53. 53.

    & A model of grid cell development through spatial exploration and spike time-dependent plasticity. Neuron 83, 481–495 (2014).

  54. 54.

    , , , & Robust persistent neural activity in a model integrator with multiple hysteretic dendrites per neuron. Cereb. Cortex 13, 1185–1195 (2003).

  55. 55.

    , , & Model for a robust neural integrator. Nat. Neurosci. 5, 775–782 (2002).

  56. 56.

    , , , & In vivo intracellular recording and perturbation of persistent activity in a neural integrator. Nat. Neurosci. 4, 184–193 (2001).

  57. 57.

    & Persistent neural activity in head direction cells. Cereb. Cortex 13, 1162–1172 (2003).

  58. 58.

    et al. Specific evidence of low-dimensional continuous attractor dynamics in grid cells. Nat. Neurosci. 16, 1077–1084 (2013).

  59. 59.

    , , & Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nat. Neurosci. 17, 431–439 (2014).

  60. 60.

    & Neural dynamics for landmark orientation and angular path integration. Nature 521, 186–191 (2015).

  61. 61.

    Self-perpetuating states in signal transduction: positive feedback, double-negative feedback and bistability. Curr. Opin. Cell Biol. 14, 140–148 (2002).

  62. 62.

    , & A proposed neural network for the integrator of the oculomotor system. Biol. Cybern. 49, 127–136 (1983).

  63. 63.

    & Angular path integration by moving “hill of activity”: a spiking neuron model without recurrent excitation of the head-direction system. J. Neurosci. 25, 1002–1014 (2005).

  64. 64.

    & Teleonomic mechanisms in cellular metabolism, growth, and differentiation. Cold Spring Harb. Symp. Quant. Biol. 26, 389–401 (1961).

  65. 65.

    Memory and molecular turnover. Nature 312, 101 (1984).

  66. 66.

    , & Mechanisms of CaMKII action in long-term potentiation. Nat. Rev. Neurosci. 13, 169–182 (2012).

  67. 67.

    et al. Role of the CaMKII/NMDA receptor complex in the maintenance of synaptic strength. J. Neurosci. 31, 9170–9178 (2011).

  68. 68.

    & Biochemical principles underlying the stable maintenance of LTP by the CaMKII/NMDAR complex. Brain Res. 1621, 51–61 (2015).

  69. 69.

    et al. Interactions between the NR2B receptor and CaMKII modulate synaptic plasticity and spatial learning. J. Neurosci. 27, 13843–13853 (2007).

  70. 70.

    , , & Translational switch for long-term maintenance of synaptic plasticity. Mol. Syst. Biol. 5, 284 (2009).

  71. 71.

    , , , & Is persistent activity of calcium/calmodulin-dependent kinase required for the maintenance of LTP? J. Neurophysiol. 85, 1368–1376 (2001).

  72. 72.

    et al. Protein kinase M zeta synthesis from a brain mRNA encoding an independent protein kinase C zeta catalytic domain. Implications for the molecular mechanism of memory. J. Biol. Chem. 278, 40305–40316 (2003).

  73. 73.

    How does PKMζ maintain long-term memory? Nat. Rev. Neurosci. 12, 9–15 (2011).

  74. 74.

    & Does PKM(zeta) maintain memory? Brain Res. Bull. 105, 36–45 (2014).

  75. 75.

    et al. Storage of spatial information by the maintenance mechanism of LTP. Science 313, 1141–1144 (2006).

  76. 76.

    & The protein kinase Mζ network as a bistable switch to store neuronal memory. BMC Syst. Biol. 4, 181 (2010).

  77. 77.

    , & Atypical PKCs in memory maintenance: the roles of feedback and redundancy. Learn. Mem. 22, 344–353 (2015).

  78. 78.

    , , , & PKM-ζ is not required for hippocampal synaptic plasticity, learning and memory. Nature 493, 420–423 (2013).

  79. 79.

    et al. Prkcz null mice show normal learning and memory. Nature 493, 416–419 (2013).

  80. 80.

    , & Compensation for PKMζ function in spatial long-term memory in mutant mice. Soc. Neurosci. Abstr. 573.01 (2015).

  81. 81.

    & Compensation for PKMζ function in late-LTP in mutant mice. Soc. Neurosci. Abstr. 573.15 (2015).

  82. 82.

    & Coordination of size and number of excitatory and inhibitory synapses results in a balanced structural plasticity along mature hippocampal CA1 dendrites during LTP. Hippocampus 21, 354–373 (2011).

  83. 83.

    , & Balance and stability of synaptic structures during synaptic plasticity. Neuron 82, 430–443 (2014).

  84. 84.

    et al. Nanoconnectomic upper bound on the variability of synaptic plasticity. eLife 4, e10778 (2015).

  85. 85.

    & Balanced cortical microcircuitry for maintaining information in working memory. Nat. Neurosci. 16, 1306–1314 (2013).

  86. 86.

    , & Predictive coding of dynamical variables in balanced spiking networks. PLoS Comput. Biol. 9, e1003258 (2013).

  87. 87.

    Cybernetics or Control and Communication in the Animal and the Machine (MIT Press, 1961).

  88. 88.

    Mathematical Control Theory: Deterministic Finite Dimensional Systems (Springer Science & Business Media, Berlin, 2013).

  89. 89.

    & Reflexes in response to stretch (myotatic reflexes). Proc. R. Soc. Lond. Biol. 96, 212–242 (1924).

  90. 90.

    , , & Robustness in bacterial chemotaxis. Nature 397, 168–171 (1999).

  91. 91.

    & Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004).

  92. 92.

    , & Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002).

  93. 93.

    , & Short-term memory in orthogonal neural networks. Phys. Rev. Lett. 92, 148102 (2004).

  94. 94.

    A survey of digital computer memory systems. Proc. IRE 41, 1393–1406 (1953).

  95. 95.

    et al. Local Cortical Circuits (Springer, Berlin, 1982).

  96. 96.

    , & Support for a synaptic chain model of neuronal sequence generation. Nature 468, 394–399 (2010).

  97. 97.

    , & Memory traces in dynamical systems. Proc. Natl. Acad. Sci. USA 105, 18970–18975 (2008).

  98. 98.

    Memory without feedback in a neural network. Neuron 61, 621–634 (2009).

  99. 99.

    & Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726 (1996).

  100. 100.

    Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci. 8, 183–208 (2000).

  101. 101.

    & Balanced amplification: a new mechanism of selective amplification of neural activity patterns. Neuron 61, 635–648 (2009).

  102. 102.

    , & Chaos in random neural networks. Phys. Rev. Lett. 61, 259–262 (1988).

  103. 103.

    Chaotic dynamics on large networks. Chaos 18, 023135 (2008).

  104. 104.

    Olfactory network dynamics and the coding of multidimensional signals. Nat. Rev. Neurosci. 3, 884–895 (2002).

  105. 105.

    , & An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature 419, 65–70 (2002).

  106. 106.

    Neural integration by short term potentiation. Biol. Cybern. 61, 319–325 (1989).

  107. 107.

    , & Synaptic theory of working memory. Science 319, 1543–1546 (2008).

  108. 108.

    , , , & Graded persistent activity in entorhinal cortex neurons. Nature 420, 173–178 (2002).

  109. 109.

    et al. Functional dissection of circuitry in a neural integrator. Nat. Neurosci. 10, 494–504 (2007).

  110. 110.

    , , & Dendritic generation of mGluR-mediated slow afterdepolarization in layer 5 neurons of prefrontal cortex. J. Neurosci. 33, 13518–13532 (2013).

  111. 111.

    et al. Identification of long-lived proteins reveals exceptional stability of essential cellular structures. Cell 154, 971–982 (2013).

  112. 112.

    , & A neuronal isoform of the aplysia CPEB has prion-like properties. Cell 115, 879–891 (2003).

  113. 113.

    et al. The CPEB3 protein is a functional prion that interacts with the actin cytoskeleton. Cell Rep. 11, 1772–1785 (2015).

  114. 114.

    et al. The persistence of hippocampal-based memory requires protein synthesis mediated by the prion-like protein CPEB3. Neuron 86, 1433–1448 (2015).

  115. 115.

    et al. Cytoplasmic polyadenylation element binding protein 1–mediated mRNA translation in Purkinje neurons is required for cerebellar long-term depression and motor coordination. J. Neurosci. 27, 6400–6411 (2007).

  116. 116.

    & Stability of discrete memory states to stochastic fluctuations in neuronal systems. Chaos 16, 026109 (2006).

  117. 117.

    , & A binary Hopfield network with 1/log(n) information rate and applications to grid cell decoding. Preprint at (2014).

  118. 118.

    Nonlinear Dynamics and Chaos: with Applications to Physics, Biology, Chemistry, and Engineering (Westview Press, 2014).

  119. 119.

    , , & Synaptic mechanisms and network dynamics underlying spatial working memory in a cortical network model. Cereb. Cortex 10, 910–923 (2000).

  120. 120.

    & Fundamental limits on persistent activity in networks of noisy neurons. Proc. Natl. Acad. Sci. USA 109, 17645–17650 (2012).

  121. 121.

    & Noise tolerance of attractor and feedforward memory models. Neural Comput. 24, 332–390 (2012).

  122. 122.

    , & Optimizing working memory with heterogeneity of recurrent cortical excitation. J. Neurosci. 33, 18999–19011 (2013).

  123. 123.

    & Dynamic flux tubes form reservoirs of stability in neuronal circuits. Phys. Rev. X 2, 041007 (2012).

  124. 124.

    How much do people remember? Some estimates of the quantity of learned information in long-term memory. Cogn. Sci. 10, 477–493 (1986).

  125. 125.

    Recognition memory for words, sentences, and pictures. J. Verbal Learn. Verbal Behav. 6, 156–163 (1967).

  126. 126.

    , , & Visual long-term memory has a massive storage capacity for object details. Proc. Natl. Acad. Sci. USA 105, 14325–14329 (2008).

  127. 127.

    Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982).

  128. 128.

    & Information capacity of the Hopfield model. IEEE Trans. Inf. Theory 31, 461–464 (1985).

  129. 129.

    & Optimal storage properties of neural network models. J. Phys. A 21, 271 (1988).

  130. 130.

    , & Storing infinite numbers of patterns in a spin-glass model of neural networks. Phys. Rev. Lett. 55, 1530–1533 (1985).

  131. 131.

    A memory which forgets. J. Phys. A 19, L617 (1986).

  132. 132.

    & Computational principles of biological memory. Preprint at <> (2015).

  133. 133.

    , & Robust exponential binary pattern storage in Little-Hopfield networks. Preprint at <> (2012).

  134. 134.

    The hippocampus as a stable memory allocator for cortex. Neural Comput. 24, 2873–2899 (2012).

  135. 135.

    The magical number seven plus or minus two: some limits on our capacity for processing information. Psychol. Rev. 63, 81–97 (1956).

  136. 136.

    , & Working memory capacity and its relation to general intelligence. Trends Cogn. Sci. 7, 547–552 (2003).

  137. 137.

    , & Time causes forgetting from working memory. Psychon. Bull. Rev. 19, 87–92 (2012).

  138. 138.

    , & No temporal decay in verbal short-term memory. Trends Cogn. Sci. 13, 120–126 (2009).

  139. 139.

    Absence seizures: a review of recent reports with new concepts. Epilepsy Behav. 15, 404–412 (2009).

  140. 140.

    , , & Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013).

  141. 141.

    & Neuronal circuits underlying persistent representations despite time varying activity. Curr. Biol. 22, 2095–2103 (2012).

  142. 142.

    , , & All-optical interrogation of neural circuits. J. Neurosci. 35, 13917–13926 (2015).

  143. 143.

    , & The neural bases of multistable perception. Trends Cogn. Sci. 13, 310–318 (2009).

  144. 144.

    et al. Attractor-map versus autoassociation based attractor dynamics in the hippocampal network. J. Neurophysiol. 104, 35–50 (2010).

  145. 145.

    , & Predicting the dynamics of network connectivity in the neocortex. J. Neurosci. 35, 12535–12544 (2015).

  146. 146.

    & Experience-dependent structural synaptic plasticity in the mammalian brain. Nat. Rev. Neurosci. 10, 647–658 (2009).

  147. 147.

    The restless engram: consolidations never end. Annu. Rev. Neurosci. 35, 227–247 (2012).

  148. 148.

    et al. Engineering a memory with LTD and LTP. Nature 511, 348–352 (2014).

  149. 149.

    The challenge of understanding the brain: where we stand in 2015. Neuron 86, 864–882 (2015).

  150. 150.

    et al. The mind and brain of short-term memory. Annu. Rev. Psychol. 59, 193–224 (2008).

Download references

Acknowledgements

We thank K. Harris and K. Raab-Graham for discussions on subcellular substrates of memory. We are grateful to S. Fusi, J. Lisman, J. Murray, X.-J. Wang, and to A. Das, I. Kanitscheider, B. Kriener and J. Widloski of the Fiete group for comments on the manuscript. This work was funded in part by grants from the Simons Collaboration on the Global Brain, the ONR-Young Investigator Program (N000141310529), and the McKnight Foundation to I.F.

Author information

Affiliations

  1. Center for Learning and Memory and Department of Neuroscience, The University of Texas at Austin, Austin, Texas, USA.

    • Rishidev Chaudhuri
    •  & Ila Fiete

Authors

  1. Search for Rishidev Chaudhuri in:

  2. Search for Ila Fiete in:

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Ila Fiete.

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nn.4237

Further reading