Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Building functional networks of spiking model neurons

Nature Neuroscience volume 19, pages 350–355 (2016)Cite this article

Subjects

Abstract

Most of the networks used by computer scientists and many of those studied by modelers in neuroscience represent unit activities as continuous variables. Neurons, however, communicate primarily through discontinuous spiking. We review methods for transferring our ability to construct interesting networks that perform relevant tasks from the artificial continuous domain to more realistic spiking network models. These methods raise a number of issues that warrant further theoretical and experimental study.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Structure of autonomous and driven networks.
Figure 2: Driven networks approximating a continuous target output.
Figure 3: Two autonomous networks of spiking neurons constructed to integrate the input fin (top, black traces).
Figure 4: Autonomous networks solving a temporal XOR task.

References

  1. Hansel, D. & Sompolinsky, H. Modeling feature selectivity in local cortical circuits. in Methods in Neuronal Modeling 2nd edn. (eds. Koch, C. & Segev, I.) 499–566 (MIT Press, Cambridge, Massachusetts, USA, 1998).

  2. Seung, H.S., Lee, D.D., Reis, B.Y. & Tank, D.W. Stability of the memory of eye position in a recurrent network of conductance-based model neurons. Neuron 26, 259–271 (2000).

    Article  CAS  PubMed  Google Scholar 

  3. Wang, X.-J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).

    Article  CAS  PubMed  Google Scholar 

  4. Renart, A., Song, P. & Wang, X.-J. Robust spatial working memory through homeostatic synaptic scaling in heterogeneous cortical networks. Neuron 38, 473–485 (2003).

    Article  CAS  PubMed  Google Scholar 

  5. Song, P. & Wang, X.-J. Angular path integration by moving “hill of activity”: a spiking neuron model without recurrent excitation of the head-direction system. J. Neurosci. 25, 1002–1014 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Eliasmith, C. A unified approach to building and controlling spiking attractor networks. Neural Comput. 17, 1276–1314 (2005).

    Article  PubMed  Google Scholar 

  7. Maass, W., Joshi, P. & Sontag, E.D. Computational aspects of feedback in neural circuits. PLoS Comput. Biol. 3, e165 (2007).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  8. Burak, Y. & Fiete, I.R. Accurate path integration in continuous attractor network models of grid cells. PLoS Comput. Biol. 5, e1000291 (2009).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  9. Boerlin, M. & Denève, S. Spike-based population coding and working memory. PLoS Comput. Biol. 7, e1001080 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Boerlin, M., Machens, C.K. & Denève, S. Predictive coding of dynamical variables in balanced spiking networks. PLoS Comput. Biol. 9, e1003258 (2013).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  11. Lim, S. & Goldman, M.S. Balanced cortical microcircuitry for maintaining information in working memory. Nat. Neurosci. 16, 1306–1314 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Schwemmer, M.A., Fairhall, A.L., Denève, S. & Shea-Brown, E.T. Constructing precisely computing networks with biophysical spiking neurons. J. Neurosci. 35, 10112–10134 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Buonomano, D.V. & Merzenich, M.M. Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267, 1028–1030 (1995).

    CAS  PubMed  Google Scholar 

  14. Gütig, R. & Sompolinsky, H. The tempotron: a neuron that learns spike timing-based decisions. Nat. Neurosci. 9, 420–428 (2006).

    Article  PubMed  CAS  Google Scholar 

  15. Pfister, J.-P., Toyoizumi, T., Barber, D. & Gerstner, W. Optimal spike-timing-dependent plasticity for precise action potential firing in supervised learning. Neural Comput. 18, 1318–1348 (2006).

    Article  PubMed  Google Scholar 

  16. Diesmann, M., Gewaltig, M.-O. & Aertsen, A. Stable propagation of synchronous spiking in cortical neural networks. Nature 402, 529–533 (1999).

    Article  CAS  PubMed  Google Scholar 

  17. Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002).

    Article  PubMed  Google Scholar 

  18. Reutimann, J., Yakovlev, V., Fusi, S. & Senn, W. Climbing neuronal activity as an event-based cortical representation of time. J. Neurosci. 24, 3295–3303 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Vogels, T.P. & Abbott, L.F. Signal propagation and logic gating in networks of integrate-and-fire neurons. J. Neurosci. 25, 10786–10795 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Liu, J.K. & Buonomano, D.V. Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J. Neurosci. 29, 13172–13181 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  21. Jahnke, S., Timme, M. & Memmesheimer, R.-M. Guiding synchrony through random networks. Phys. Rev. X 2, 041016 (2012).

    CAS  Google Scholar 

  22. Thalmeier, D., Uhlmann, M., Kappen, H.J. & Memmesheimer, R.-M. Learning universal computations with spikes. Preprint at http://arxiv.org/abs/1505.07866 (2015).

  23. DePasquale, B., Churchland, M. & Abbott, L.F. Using firing-rate dynamics to train recurrent networks of spiking model neurons. Preprint at http://arxiv.org/abs/1601.07620 (2016).

  24. Memmesheimer, R.-M., Rubin, R., Ölveczky, B.P. & Sompolinsky, H. Learning precisely timed spikes. Neuron 82, 925–938 (2014).

    Article  CAS  PubMed  Google Scholar 

  25. Eliasmith, C. et al. A large-scale model of the functioning brain. Science 338, 1202–1205 (2012).

    Article  CAS  PubMed  Google Scholar 

  26. Hennequin, G., Vogels, T.P. & Gerstner, W. Optimal control of transient dynamics in balanced networks supports generation of complex movements. Neuron 82, 1394–1406 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Ponulak, F. & Kasin´ski, A. Supervised learning in spiking neural networks with ReSuMe: sequence learning, classification, and spike shifting. Neural Comput. 22, 467–510 (2010).

    Article  PubMed  Google Scholar 

  28. Florian, R.V. The chronotron: a neuron that learns to fire temporally precise spike patterns. PLoS One 7, e40233 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Brea, J., Senn, W. & Pfister, J.-P. Matching recall and storage in sequence learning with spiking neural networks. J. Neurosci. 33, 9565–9575 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).

    Article  CAS  PubMed  Google Scholar 

  31. Mante, V., Sussillo, D., Shenoy, K.V. & Newsome, W.T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Sussillo, D., Churchland, M.M., Kaufman, M.T. & Shenoy, K.V. A neural network that finds a naturalistic solution for the production of muscle activity. Nat. Neurosci. 18, 1025–1033 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Bohte, S.M., Kok, J.N. & Poutré, H.L. Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48, 17–37 (2002).

    Article  Google Scholar 

  34. Tino, P. & Mills, A.J.S. Learning beyond finite memory in recurrent networks of spiking neurons. Neural Comput. 18, 591–613 (2006).

    Article  PubMed  Google Scholar 

  35. Sporea, I. & Grüning, A. Supervised learning in multilayer spiking neural networks. Neural Comput. 25, 473–509 (2013).

    Article  PubMed  Google Scholar 

  36. van Vreeswijk, C. & Sompolinsky, H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726 (1996).

    Article  CAS  PubMed  Google Scholar 

  37. Brunel, N. Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci. 8, 183–208 (2000).

    Article  CAS  PubMed  Google Scholar 

  38. LeCun, Y. Learning processes in an asymmetric threshold network. in Disordered Systems and Biological Organization (eds. Bienenstock, E., Fogelman, F. & Weisbuch, G.) 233–240 (Springer, Berlin, 1986).

  39. Bengio, Y. How auto-encoders could provide credit assignment in deep networks via target propagation. Preprint at http://arxiv.org/abs/1407.7906 (2014).

  40. Laje, R. & Buonomano, D.V. Robust timing and motor patterns by taming chaos in recurrent neural networks. Nat. Neurosci. 16, 925–933 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Fisher, D., Olasagasti, I., Tank, D.W., Aksay, E.R.F. & Goldman, M.S. A modeling framework for deriving the structural and functional architecture of a short-term memory microcircuit. Neuron 79, 987–1000 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Rajan, K., Harvey, C. & Tank, D. Recurrent network models of sequence generation and memory. Neuron (in the press).

  43. Jaeger, H. & Haas, H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004).

    Article  CAS  PubMed  Google Scholar 

  44. Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544–557 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Sussillo, D. & Abbott, L.F. Transferring learning from external to internal weights in echo-state networks with sparse connectivity. PLoS One 7, e37372 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Lukoševičius, M., Jaeger, H. & Schrauwen, B. Reservoir computing trends. Künstl. Intell. 26, 365–371 (2012).

    Article  Google Scholar 

  47. Sussillo, D. Neural circuits as computational dynamical systems. Curr. Opin. Neurobiol. 25, 156–163 (2014).

    Article  CAS  PubMed  Google Scholar 

  48. Eliasmith, C. & Anderson, C. Neural Engineering: Computation, Representation and Dynamics in Neurobiological Systems (MIT Press, Cambridge, Massachusetts, USA, 2003).

  49. Denève, S. & Machens, C. Efficient codes and balanced networks. Nat. Neurosci. 19, 375–382 (2016).

    Article  PubMed  CAS  Google Scholar 

  50. Softky, W.R. & Koch, C. The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs. J. Neurosci. 13, 334–350 (1993).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. Rivkind, A. & Barak, O. Local dynamics in trained recurrent neural networks. Preprint at http://arxiv.org/abs/1511.05222 (2015).

  52. Hosoya, T., Baccus, S.A. & Meister, M. Dynamic predictive coding by the retina. Nature 436, 71–77 (2005).

    Article  CAS  PubMed  Google Scholar 

  53. Vogels, T.P., Sprekeler, H., Zenke, F., Clopath, C. & Gerstner, W. Inhibitory plasticity balances excitation and inhibition in sensory pathways and memory networks. Science 334, 1569–1573 (2011).

    Article  CAS  PubMed  Google Scholar 

  54. Bourdoukan, R., Barrett, D.G.T., Machens, C.K. & Denève, S. Learning optimal spike-based representations. Adv. Neural Inf. Process. Syst. 25, 2294–2302 (2012).

    Google Scholar 

  55. Kennedy, A. et al. A temporal basis for predicting the sensory consequences of motor commands in an electric fish. Nat. Neurosci. 17, 416–422 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Bourdoukan, R. & Denève, S. Enforcing balance allows local supervised learning in spiking recurrent networks. Adv. Neural Inf. Process. Syst. 28, 982–990 (2015).

    Google Scholar 

  57. Potjans, W., Morrison, A. & Diesmann, M. A spiking neural network model of an actor-critic learning agent. Neural Comput. 21, 301–339 (2009).

    Article  PubMed  Google Scholar 

  58. Hoerzer, G.M., Legenstein, R. & Maass, W. Emergence of complex computational structures from chaotic neural networks through reward-modulated Hebbian learning. Cereb. Cortex 24, 677–690 (2014).

    Article  PubMed  Google Scholar 

  59. Vasilaki, E., Frémaux, N., Urbanczik, R., Senn, W. & Gerstner, W. Spike-based reinforcement learning in continuous state and action space: when policy gradient methods fail. PLoS Comput. Biol. 5, e1000586 (2009).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  60. Friedrich, J. & Senn, W. Spike-based decision learning of Nash equilibria in two-player games. PLoS Comput. Biol. 8, e1002691 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Kleindienst, T., Winnubst, J., Roth-Alpermann, C., Bonhoeffer, T. & Lohmann, C. Activity-dependent clustering of functional synaptic inputs on developing hippocampal dendrites. Neuron 72, 1012–1024 (2011).

    Article  CAS  PubMed  Google Scholar 

  62. Branco, T. & Häusser, M. Synaptic integration gradients in single cortical pyramidal cell dendrites. Neuron 69, 885–892 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  63. Druckmann, S. et al. Structured synaptic connectivity between hippocampal regions. Neuron 81, 629–640 (2014).

    Article  CAS  PubMed  Google Scholar 

  64. London, M. & Häusser, M. Dendritic computation. Annu. Rev. Neurosci. 28, 503–532 (2005).

    Article  CAS  PubMed  Google Scholar 

  65. Major, G., Larkum, M.E. & Schiller, J. Active properties of neocortical pyramidal neuron dendrites. Annu. Rev. Neurosci. 36, 1–24 (2013).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank C. Machens, M. Churchland and D. Thalmeier for helpful discussions. Our research in this area was supported by US National Institutes of Health grant MH093338, the Gatsby Charitable Foundation through the Gatsby Initiative in Brain Circuitry at Columbia University, the Simons Foundation, the Swartz Foundation, the Harold and Leila Y. Mathers Foundation, the Kavli Institute for Brain Science at Columbia University, the Max Kade Foundation and the German Federal Ministry of Education and Research BMBF through the Bernstein Network (Bernstein Award 2014).

Author information

Authors and Affiliations

  1. Department of Neuroscience, Columbia University College of Physicians and Surgeons, New York, New York, USA

    L F Abbott, Brian DePasquale & Raoul-Martin Memmesheimer

  2. Department of Physiology and Cellular Biophysics, Columbia University College of Physicians and Surgeons, New York, New York, USA

    L F Abbott

  3. Department for Neuroinformatics, Donders Institute for Brain Cognition and Behavior, Radboud University, Nijmegen, the Netherlands

    Raoul-Martin Memmesheimer

Authors
  1. L F Abbott
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Brian DePasquale
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Raoul-Martin Memmesheimer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to L F Abbott.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abbott, L., DePasquale, B. & Memmesheimer, RM. Building functional networks of spiking model neurons. Nat Neurosci 19, 350–355 (2016). https://doi.org/10.1038/nn.4241

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4241

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing