Abstract
The brain's ability to tell time and produce complex spatiotemporal motor patterns is critical for anticipating the next ring of a telephone or playing a musical instrument. One class of models proposes that these abilities emerge from dynamically changing patterns of neural activity generated in recurrent neural networks. However, the relevant dynamic regimes of recurrent networks are highly sensitive to noise; that is, chaotic. We developed a firing rate model that tells time on the order of seconds and generates complex spatiotemporal patterns in the presence of high levels of noise. This is achieved through the tuning of the recurrent connections. The network operates in a dynamic regime that exhibits coexisting chaotic and locally stable trajectories. These stable patterns function as 'dynamic attractors' and provide a feature that is characteristic of biological systems: the ability to 'return' to the pattern being generated in the face of perturbations.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Mauk, M.D. & Buonomano, D.V. The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340 (2004).
Buhusi, C.V. & Meck, W.H. What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6, 755–765 (2005).
Ivry, R.B. & Schlerf, J.E. Dedicated and intrinsic models of time perception. Trends Cogn. Sci. 12, 273–280 (2008).
Church, R.M., Meck, W.H. & Gibbon, J. Application of scalar timing theory to individual trials. J. Exp. Psychol. Anim. Behav. Process. 20, 135–155 (1994).
Durstewitz, D. Self-organizing neural integrator predicts interval times through climbing activity. J. Neurosci. 23, 5342–5353 (2003).
Simen, P., Balci, F., de Souza, L., Cohen, J.D. & Holmes, P. A model of interval timing by neural integration. J. Neurosci. 31, 9238–9253 (2011).
Miall, C. The storage of time intervals using oscillating neurons. Neural Comput. 1, 359–371 (1989).
Matell, M.S., Meck, W.H. & Nicolelis, M.A. Interval timing and the encoding of signal duration by ensembles of cortical and striatal neurons. Behav. Neurosci. 117, 760–773 (2003).
Ahrens, M.B. & Sahani, M. Observers exploit stochastic models of sensory change to help judge the passage of time. Curr. Biol. 21, 200–206 (2011).
Buonomano, D.V. & Laje, R. Population clocks: motor timing with neural dynamics. Trends Cogn. Sci. 14, 520–527 (2010).
Medina, J.F. & Mauk, M.D. Computer simulation of cerebellar information processing. Nat. Neurosci. 3 (suppl.), 1205–1211 (2000).
Buonomano, D.V. & Mauk, M.D. Neural network model of the cerebellum: temporal discrimination and the timing of motor responses. Neural Comput. 6, 38–55 (1994).
Buonomano, D.V. & Merzenich, M.M. Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267, 1028–1030 (1995).
Durstewitz, D. & Deco, G. Computational significance of transient dynamics in cortical networks. Eur. J. Neurosci. 27, 217–227 (2008).
Rabinovich, M., Huerta, R. & Laurent, G. Transient dynamics for neural processing. Science 321, 48–50 (2008).
Buonomano, D.V. & Maass, W. State-dependent computations: spatiotemporal processing in cortical networks. Nat. Rev. Neurosci. 10, 113–125 (2009).
Hahnloser, R.H.R., Kozhevnikov, A.A. & Fee, M.S. An ultra-sparse code underlies the generation of neural sequence in a songbird. Nature 419, 65–70 (2002).
Long, M.A., Jin, D.Z. & Fee, M.S. Support for a synaptic chain model of neuronal sequence generation. Nature 468, 394–399 (2010).
Crowe, D.A., Averbeck, B.B. & Chafee, M.V. Rapid sequences of population activity patterns dynamically encode task-critical spatial information in parietal cortex. J. Neurosci. 30, 11640–11653 (2010).
Li, J.X. & Lisberger, S.G. Learned timing of motor behavior in the smooth eye movement region of the frontal eye fields. Neuron 69, 159–169 (2011).
London, M., Roth, A., Beeren, L., Hausser, M. & Latham, P.E. Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex. Nature 466, 123–127 (2010).
Izhikevich, E.M. & Edelman, G.M. Large-scale model of mammalian thalamocortical systems. Proc. Natl. Acad. Sci. USA 105, 3593–3598 (2008).
van Vreeswijk, C. & Sompolinsky, H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726 (1996).
Brunel, N. Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons. J. Physiol. Paris 94, 445–463 (2000).
Banerjee, A., Series, P. & Pouget, A. Dynamical constraints on using precise spike timing to compute in recurrent cortical networks. Neural Comput. 20, 974–993 (2008).
Sompolinsky, H., Crisanti, A. & Sommers, H.J. Chaos in random neural networks. Phys. Rev. Lett. 61, 259–262 (1988).
Monteforte, M. & Wolf, F. Dynamic flux tubes form reservoirs of stability in neuronal circuits. Phys. Rev. X 2, 041007 (2012).
Jaeger, H. & Haas, H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004).
Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009).
Rajan, K., Abbott, L.F. & Sompolinsky, H. Stimulus-dependent suppression of chaos in recurrent neural networks. Physical Rev. E Stat. Nonlin. Soft Matter Phys. 82, 011903 (2010).
Doya, K. in Proc. IEEE Int. Symp. Circuits and Syst. 2777–2780 (1992).
Jaeger, H., Maass, W. & Principe, J. Special issue on echo state networks and liquid state machines. Neural Netw. 20, 287–289 (2007).
Ganguli, S., Huh, D. & Sompolinsky, H. Memory traces in dynamical systems. Proc. Natl. Acad. Sci. USA 105, 18970–18975 (2008).
Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).
Song, S., Sjostrom, P.J., Reigl, M., Nelson, S. & Chklovskii, D.B. Highly nonrandom feature of synaptic connectivity in local cortical circuits. PLoS Biol. 3, e66 (2005).
Watts, D.J. & Strogatz, S.H. Collective dynamics of 'small-world' networks. Nature 393, 440–442 (1998).
Janssen, P. & Shadlen, M.N. A representation of the hazard rate of elapsed time in the macaque area LIP. Nat. Neurosci. 8, 234–241 (2005).
Bueti, D., Lasaponara, S., Cercignani, M. & Macaluso, E. Learning about time: plastic changes and interindividual brain differences. Neuron 75, 725–737 (2012).
Coull, J. & Nobre, A. Dissociating explicit timing from temporal expectation with fMRI. Curr. Opin. Neurobiol. 18, 137–144 (2008).
Merchant, H., Zarco, W., Pérez, O., Prado, L. & Bartolo, R. Measuring time with different neural chronometers during a synchronization-continuation task. Proc. Natl. Acad. Sci. USA 108, 19784–19789 (2011).
Pastalkova, E., Itskov, V., Amarasingham, A. & Buzsaki, G. Internally generated cell assembly sequences in the rat hippocampus. Science 321, 1322–1327 (2008).
Ivry, R.B., Keele, S.W. & Diener, H.C. Dissociation of the lateral and medial cerebellum in movement timing and movement execution. Exp. Brain Res. 73, 167–180 (1988).
Medina, J.F., Garcia, K.S., Nores, W.L., Taylor, N.M. & Mauk, M.D. Timing mechanisms in the cerebellum: testing predictions of a large-scale computer simulation. J. Neurosci. 20, 5516–5525 (2000).
Buonomano, D.V. Decoding temporal information: a model based on short-term synaptic plasticity. J. Neurosci. 20, 1129–1141 (2000).
Litwin-Kumar, A. & Doiron, B. Slow dynamics and high variability in balanced cortical networks with clustered connections. Nat. Neurosci. 15, 1498–1505 (2012).
Liu, J.K. & Buonomano, D.V. Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J. Neurosci. 29, 13172–13181 (2009).
Goldman, M.S. Memory without feedback in a neural network. Neuron 61, 621–634 (2009).
Hopfield, J.J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982).
Wang, X.J. Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci. 24, 455–463 (2001).
Skarda, C.A. & Freeman, W.J. How brains make chaos in order to make sense of the world. Behav. Brain Sci. 10, 161–173 (1987).
Jaeger, H. The “echo state” approach to analysing and training recurrent neural networks. GMD Report No. 148 (German National Research Center for Computer Science) (2001).
Haykin, S. Adaptive Filter Theory (Prentice Hall, 2002).
Kantz, H. A robust method to estimate the maximal Lyapunov exponent of a time series. Phys. Lett. A 185, 77–87 (1994).
Boffetta, G., Lacorata, G., Radaelli, G. & Vulpiani, A. Detecting barriers to transport: a review of different techniques. Physica D 159, 58–70 (2001).
Fagiolo, G. Clustering in complex directed networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 76, 026107 (2007).
Acknowledgements
We thank A. Garfinkel and R. Huerta for helpful discussions and comments on the manuscript. This work was supported by the US National Institutes of Health (NS077340), the National Science Foundation (II-1114833), the Pew Charitable Trusts, and Consejo Nacional de Investigaciones Científicas y Técnicas (Argentina).
Author information
Authors and Affiliations
Contributions
R.L. and D.V.B. designed the experiments and wrote the code, and R.L. performed most of the simulations and data analysis. R.L. designed and performed the stability and structural experiments. D.V.B. conceived of the approach, and R.L. and D.V.B. wrote the paper.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–8 and Supplementary Modeling (PDF 725 kb)
Supplementary Matlab Routines
Supplementary Matlab Routines (ZIP 3501 kb)
Rights and permissions
About this article
Cite this article
Laje, R., Buonomano, D. Robust timing and motor patterns by taming chaos in recurrent neural networks. Nat Neurosci 16, 925–933 (2013). https://doi.org/10.1038/nn.3405
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.3405
This article is cited by
-
Multiplexing working memory and time in the trajectories of neural networks
Nature Human Behaviour (2023)
-
Residual dynamics resolves recurrent contributions to neural computation
Nature Neuroscience (2023)
-
Reservoir computing using self-sustained oscillations in a locally connected neural network
Scientific Reports (2023)
-
Cerebro-cerebellar networks facilitate learning through feedback decoupling
Nature Communications (2023)
-
Distributing task-related neural activity across a cortical network through task-independent connections
Nature Communications (2023)