Article | Published:

Robust timing and motor patterns by taming chaos in recurrent neural networks

Nature Neuroscience volume 16, pages 925933 (2013) | Download Citation

Abstract

The brain's ability to tell time and produce complex spatiotemporal motor patterns is critical for anticipating the next ring of a telephone or playing a musical instrument. One class of models proposes that these abilities emerge from dynamically changing patterns of neural activity generated in recurrent neural networks. However, the relevant dynamic regimes of recurrent networks are highly sensitive to noise; that is, chaotic. We developed a firing rate model that tells time on the order of seconds and generates complex spatiotemporal patterns in the presence of high levels of noise. This is achieved through the tuning of the recurrent connections. The network operates in a dynamic regime that exhibits coexisting chaotic and locally stable trajectories. These stable patterns function as 'dynamic attractors' and provide a feature that is characteristic of biological systems: the ability to 'return' to the pattern being generated in the face of perturbations.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    & The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340 (2004).

  2. 2.

    & What makes us tick? Functional and neural mechanisms of interval timing. Nat. Rev. Neurosci. 6, 755–765 (2005).

  3. 3.

    & Dedicated and intrinsic models of time perception. Trends Cogn. Sci. 12, 273–280 (2008).

  4. 4.

    , & Application of scalar timing theory to individual trials. J. Exp. Psychol. Anim. Behav. Process. 20, 135–155 (1994).

  5. 5.

    Self-organizing neural integrator predicts interval times through climbing activity. J. Neurosci. 23, 5342–5353 (2003).

  6. 6.

    , , , & A model of interval timing by neural integration. J. Neurosci. 31, 9238–9253 (2011).

  7. 7.

    The storage of time intervals using oscillating neurons. Neural Comput. 1, 359–371 (1989).

  8. 8.

    , & Interval timing and the encoding of signal duration by ensembles of cortical and striatal neurons. Behav. Neurosci. 117, 760–773 (2003).

  9. 9.

    & Observers exploit stochastic models of sensory change to help judge the passage of time. Curr. Biol. 21, 200–206 (2011).

  10. 10.

    & Population clocks: motor timing with neural dynamics. Trends Cogn. Sci. 14, 520–527 (2010).

  11. 11.

    & Computer simulation of cerebellar information processing. Nat. Neurosci. 3 (suppl.), 1205–1211 (2000).

  12. 12.

    & Neural network model of the cerebellum: temporal discrimination and the timing of motor responses. Neural Comput. 6, 38–55 (1994).

  13. 13.

    & Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267, 1028–1030 (1995).

  14. 14.

    & Computational significance of transient dynamics in cortical networks. Eur. J. Neurosci. 27, 217–227 (2008).

  15. 15.

    , & Transient dynamics for neural processing. Science 321, 48–50 (2008).

  16. 16.

    & State-dependent computations: spatiotemporal processing in cortical networks. Nat. Rev. Neurosci. 10, 113–125 (2009).

  17. 17.

    , & An ultra-sparse code underlies the generation of neural sequence in a songbird. Nature 419, 65–70 (2002).

  18. 18.

    , & Support for a synaptic chain model of neuronal sequence generation. Nature 468, 394–399 (2010).

  19. 19.

    , & Rapid sequences of population activity patterns dynamically encode task-critical spatial information in parietal cortex. J. Neurosci. 30, 11640–11653 (2010).

  20. 20.

    & Learned timing of motor behavior in the smooth eye movement region of the frontal eye fields. Neuron 69, 159–169 (2011).

  21. 21.

    , , , & Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex. Nature 466, 123–127 (2010).

  22. 22.

    & Large-scale model of mammalian thalamocortical systems. Proc. Natl. Acad. Sci. USA 105, 3593–3598 (2008).

  23. 23.

    & Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726 (1996).

  24. 24.

    Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons. J. Physiol. Paris 94, 445–463 (2000).

  25. 25.

    , & Dynamical constraints on using precise spike timing to compute in recurrent cortical networks. Neural Comput. 20, 974–993 (2008).

  26. 26.

    , & Chaos in random neural networks. Phys. Rev. Lett. 61, 259–262 (1988).

  27. 27.

    & Dynamic flux tubes form reservoirs of stability in neuronal circuits. Phys. Rev. X 2, 041007 (2012).

  28. 28.

    & Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004).

  29. 29.

    & Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009).

  30. 30.

    , & Stimulus-dependent suppression of chaos in recurrent neural networks. Physical Rev. E Stat. Nonlin. Soft Matter Phys. 82, 011903 (2010).

  31. 31.

    in Proc. IEEE Int. Symp. Circuits and Syst. 2777–2780 (1992).

  32. 32.

    , & Special issue on echo state networks and liquid state machines. Neural Netw. 20, 287–289 (2007).

  33. 33.

    , & Memory traces in dynamical systems. Proc. Natl. Acad. Sci. USA 105, 18970–18975 (2008).

  34. 34.

    et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).

  35. 35.

    , , , & Highly nonrandom feature of synaptic connectivity in local cortical circuits. PLoS Biol. 3, e66 (2005).

  36. 36.

    & Collective dynamics of 'small-world' networks. Nature 393, 440–442 (1998).

  37. 37.

    & A representation of the hazard rate of elapsed time in the macaque area LIP. Nat. Neurosci. 8, 234–241 (2005).

  38. 38.

    , , & Learning about time: plastic changes and interindividual brain differences. Neuron 75, 725–737 (2012).

  39. 39.

    & Dissociating explicit timing from temporal expectation with fMRI. Curr. Opin. Neurobiol. 18, 137–144 (2008).

  40. 40.

    , , , & Measuring time with different neural chronometers during a synchronization-continuation task. Proc. Natl. Acad. Sci. USA 108, 19784–19789 (2011).

  41. 41.

    , , & Internally generated cell assembly sequences in the rat hippocampus. Science 321, 1322–1327 (2008).

  42. 42.

    , & Dissociation of the lateral and medial cerebellum in movement timing and movement execution. Exp. Brain Res. 73, 167–180 (1988).

  43. 43.

    , , , & Timing mechanisms in the cerebellum: testing predictions of a large-scale computer simulation. J. Neurosci. 20, 5516–5525 (2000).

  44. 44.

    Decoding temporal information: a model based on short-term synaptic plasticity. J. Neurosci. 20, 1129–1141 (2000).

  45. 45.

    & Slow dynamics and high variability in balanced cortical networks with clustered connections. Nat. Neurosci. 15, 1498–1505 (2012).

  46. 46.

    & Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J. Neurosci. 29, 13172–13181 (2009).

  47. 47.

    Memory without feedback in a neural network. Neuron 61, 621–634 (2009).

  48. 48.

    Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982).

  49. 49.

    Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci. 24, 455–463 (2001).

  50. 50.

    & How brains make chaos in order to make sense of the world. Behav. Brain Sci. 10, 161–173 (1987).

  51. 51.

    The “echo state” approach to analysing and training recurrent neural networks. GMD Report No. 148 (German National Research Center for Computer Science) (2001).

  52. 52.

    Adaptive Filter Theory (Prentice Hall, 2002).

  53. 53.

    A robust method to estimate the maximal Lyapunov exponent of a time series. Phys. Lett. A 185, 77–87 (1994).

  54. 54.

    , , & Detecting barriers to transport: a review of different techniques. Physica D 159, 58–70 (2001).

  55. 55.

    Clustering in complex directed networks. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 76, 026107 (2007).

Download references

Acknowledgements

We thank A. Garfinkel and R. Huerta for helpful discussions and comments on the manuscript. This work was supported by the US National Institutes of Health (NS077340), the National Science Foundation (II-1114833), the Pew Charitable Trusts, and Consejo Nacional de Investigaciones Científicas y Técnicas (Argentina).

Author information

Author notes

    • Rodrigo Laje

    Present addresses: Departamento de Ciencia y Tecnología, Universidad Nacional de Quilmes, Bernal, Argentina, and Consejo Nacional de Investigaciones Científicas y Técnicas, Buenos Aires, Argentina.

Affiliations

  1. Department of Neurobiology, University of California, Los Angeles, California, USA.

    • Rodrigo Laje
    •  & Dean V Buonomano
  2. Department of Psychology, University of California, Los Angeles, California, USA.

    • Dean V Buonomano
  3. Brain Research Institute, University of California, Los Angeles, California, USA.

    • Dean V Buonomano
  4. Integrative Center for Learning and Memory, University of California, Los Angeles, California, USA.

    • Dean V Buonomano

Authors

  1. Search for Rodrigo Laje in:

  2. Search for Dean V Buonomano in:

Contributions

R.L. and D.V.B. designed the experiments and wrote the code, and R.L. performed most of the simulations and data analysis. R.L. designed and performed the stability and structural experiments. D.V.B. conceived of the approach, and R.L. and D.V.B. wrote the paper.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Dean V Buonomano.

Supplementary information

PDF files

  1. 1.

    Supplementary Text and Figures

    Supplementary Figures 1–8 and Supplementary Modeling

Zip files

  1. 1.

    Supplementary Matlab Routines

    Supplementary Matlab Routines

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nn.3405

Further reading