Two types of asynchronous activity in networks of excitatory and inhibitory spiking neurons

Journal name:
Nature Neuroscience
Volume:
17,
Pages:
594–600
Year published:
DOI:
doi:10.1038/nn.3658
Received
Accepted
Published online

Abstract

Asynchronous activity in balanced networks of excitatory and inhibitory neurons is believed to constitute the primary medium for the propagation and transformation of information in the neocortex. Here we show that an unstructured, sparsely connected network of model spiking neurons can display two fundamentally different types of asynchronous activity that imply vastly different computational properties. For weak synaptic couplings, the network at rest is in the well-studied asynchronous state, in which individual neurons fire irregularly at constant rates. In this state, an external input leads to a highly redundant response of different neurons that favors information transmission but hinders more complex computations. For strong couplings, we find that the network at rest displays rich internal dynamics, in which the firing rates of individual neurons fluctuate strongly in time and across neurons. In this regime, the internal dynamics interact with incoming stimuli to provide a substrate for complex information processing and learning.

At a glance

Figures

  1. Instability of the asynchronous state in a network of LIF neurons.
    Figure 1: Instability of the asynchronous state in a network of LIF neurons.

    (a) Mean firing rate of the neurons in the network as function of the overall synaptic coupling J. Solid line, analytical prediction for the classical asynchronous state (equation (1)); dots, results of numerical simulations. (b) Linear stability of the classical asynchronous state. Top, eigenvalues of the stability matrix (equation (12)) for J = 0.2 mV (left) and for J = 0.8 mV (right). Bottom, radius λmax of the linear spectrum as function of synaptic coupling J. The value of λmax crosses unity at a critical coupling Jc, above which the asynchronous state is unstable. (c) Network activity in the asynchronous state (J = 0.2 mV; blue square in a). Top, rastergram for 100 neurons in the network; bottom, average population activity in 1-ms bins; inset, autocorrelation of the population activity. (d) Network activity above the instability (J = 0.8 mV; orange square in a). Same quantities as in c. Parameter values: constant input μ0 = 24 mV, relative inhibition strength g = 5, number of connections per neuron C = 1,000, number of neurons N = 10,000.

  2. Instability of the asynchronous state in a network of Poisson neurons.
    Figure 2: Instability of the asynchronous state in a network of Poisson neurons.

    The activity of each neuron is fully specified by a temporally varying firing rate. Action potentials are generated stochastically from the underlying rate but do not influence the dynamics. (a) Mean firing rate of the neurons in the network as function of synaptic coupling J. Solid line, predictions for the firing rate in the equilibrium state (equation (1)); dots, results of numerical simulations. (b) Network activity in the equilibrium state (J = 0.2 mV; blue square in a). Top, rastergram for 100 neurons in the network; bottom, average population activity in 1-ms bins. (c) Network activity above the instability (J = 0.8 mV; orange square in a). Same quantities as in b. Parameter values as in Figure 1.

  3. Strong synaptic couplings lead to highly fluctuating instantaneous firing rates of individual neurons.
    Figure 3: Strong synaptic couplings lead to highly fluctuating instantaneous firing rates of individual neurons.

    Top, Poisson network; bottom, LIF network. (a) Firing rates for three example neurons. Left, low synaptic coupling (J = 0.2 mV); right, strong synaptic coupling (J = 0.8 mV). For LIF neurons, we estimated the instantaneous firing rates by convolving the spike trains with a 50-ms-wide Gaussian filter. (b) Autocorrelation (autocorr) function of instantaneous firing rates, averaged over neurons in the network. (c) Autocorrelation function of spike trains, averaged over neurons in the network. All parameters as in Figures 1 and 2.

  4. Temporal inputs are processed differently in the two types of resting asynchronous activity.
    Figure 4: Temporal inputs are processed differently in the two types of resting asynchronous activity.

    A subset of neurons in the network receives an input current that varies in time. We examine the resulting output of the network as seen by a neuron that reads out the activity of the network through NMDA-like synapses. Left, the network at rest is in the classical asynchronous state; right, the network at rest is in the heterogeneous asynchronous state (same parameters as in Fig. 1c,d respectively). (a) Two different temporally varying inputs given to the network. (b) Firing rates in response to the two inputs, as seen by the readout unit. Colored traces, three example neurons; dashed black line, average response of the network. (c,e) Difference between the mean population response and the average response obtained by pooling a subset of neurons from the network, as function of the size of the pooled subset. (d,f) Dimensionality of the network response, quantified by the percentage of variance (var.) explained by successive dimensions in a principal component analysis (PCA). (g,h) Projection on the first three PCA components of the network response to the two inputs shown in a, for the classical (g) and heterogeneous (h) asynchronous states.

  5. Influence of network parameters on asynchronous activity.
    Figure 5: Influence of network parameters on asynchronous activity.

    (a) Nature of the asynchronous state as a function of overall synaptic coupling J and relative inhibition strength g. (b) Nature of the asynchronous state as function of overall synaptic coupling J and constant input current μ0. The color plots display, on a logarithmic scale, the deviation of the mean firing rates from the firing rates predicted by equation (1) for the classical asynchronous state. Dark shades correspond to strong deviations, which are the signature of the heterogeneous asynchronous state (Fig. 1a). Light shades correspond to weak deviations and therefore to the homogeneous (homog.) asynchronous state. The purple lines display the analytical predictions for the transition.

References

  1. Shadlen, M.N. & Newsome, W.T. Noise, neural codes and cortical organization. Curr. Opin. Neurobiol. 4, 569579 (1994).
  2. van Vreeswijk, C. & Sompolinsky, H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 17241726 (1996).
  3. Amit, D.J. & Brunel, N. Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cereb. Cortex 7, 237252 (1997).
  4. Troyer, T.W. & Miller, K.D. Physiological gain leads to high ISI variability in a simple model of a cortical regular spiking cell. Neural Comput. 9, 971983 (1997).
  5. Shadlen, M.N. & Newsome, W.T. The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J. Neurosci. 18, 38703896 (1998).
  6. Shu, Y., Hasenstaub, A. & McCormick, D.A. Turning on and off recurrent balanced cortical activity. Nature 423, 288293 (2003).
  7. London, M., Roth, A., Beeren, L., Häusser, M. & Latham, P.E. Sensitivity to perturbations in vivo implies high noise and suggests rate coding in cortex. Nature 466, 123127 (2010).
  8. Renart, A. et al. The asynchronous state in cortical circuits. Science 327, 587590 (2010).
  9. Brunel, N. Dynamics of sparsely connected networks of excitatory and inhibitory spiking neurons. J. Comput. Neurosci. 8, 183208 (2000).
  10. Monteforte, M. & Wolf, F. Dynamical entropy production in spiking neuron networks in the balanced state. Phys. Rev. Lett. 105, 268104 (2010).
  11. Monteforte, M. & Wolf, F. Dynamic flux tubes form reservoirs of stability in neuronal circuits. Phys. Rev. X 2, 041007 (2012).
  12. Brunel, N. & Wang, X.-J. Effects of neuromodulation in a cortical network model of object working memory dominated by recurrent inhibition. J. Comput. Neurosci. 11, 6385 (2001).
  13. Wang, X.-J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955968 (2002).
  14. Dayan, P. & Abbott, L.F. Theoretical Neuroscience (MIT Press, 2001).
  15. Sompolinsky, H., Crisanti, A. & Sommers, H. Chaos in random neural networks. Phys. Rev. Lett. 61, 259262 (1988).
  16. Jaeger, H. & Haas, H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 7880 (2004).
  17. Sussillo, D. & Abbott, L.F. Generating coherent patterns of activity from chaotic neural networks. Neuron 63, 544557 (2009).
  18. Toyoizumi, T. & Abbott, L.F. Beyond the edge of chaos: amplification and temporal integration by recurrent networks in the chaotic regime. Phys. Rev. E 84, 051908 (2011).
  19. Laje, R. & Buonomano, D.V. Robust timing and motor patterns by taming chaos in recurrent neural networks. Nat. Neurosci. 16, 925933 (2013).
  20. Buonomano, D.V. & Maass, W. State-dependent computations: spatiotemporal processing in cortical networks. Nat. Rev. Neurosci. 10, 113125 (2009).
  21. Wainrib, G. & Touboul, J. Topological and dynamical complexity of random neural networks. Phys. Rev. Lett. 110, 118101 (2013).
  22. Sussillo, D. & Barak, O. Opening the black box: low-dimensional dynamics in high-dimensional recurrent neural networks. Neural Comput. 25, 626649 (2013).
  23. Wallace, E., Maei, H.R. & Latham, P.E.L. Randomly connected networks have short temporal memory. Neural Comput. 25, 14081439 (2013).
  24. Churchland, M.M. & Abbott, L.F. Two layers of neural variability. Nat. Neurosci. 15, 14721474 (2012).
  25. Brunel, N. & Hakim, V. Fast global oscillations in networks of integrate-and-fire neurons with low firing rates. Neural Comput. 11, 16211671 (1999).
  26. Amit, D.J. & Tsodyks, M.V. Quantitative study of attractor neural network retrieving at low spike rates. 1. Substrate spikes, rates and neuronal gain. Network 2, 259273 (1991).
  27. Rajan, K. & Abbott, L.F. Eigenvalue spectra of random matrices for neural networks. Phys. Rev. Lett. 97, 188104 (2006).
  28. Timme, M., Wolf, F. & Geisel, T. Topological speed limits to network synchronization. Phys. Rev. Lett. 92, 074101 (2004).
  29. Ostojic, S. & Brunel, N. From spiking neuron models to linear-nonlinear models. PLoS Comput. Biol. 7, e1001056 (2011).
  30. Schaffer, E.S., Ostojic, S. & Abbott, L.F.A. Complex-valued firing-rate model that approximates the dynamics of spiking networks. PLoS Comput. Biol. 9, e1003301 (2013).
  31. Rajan, K., Abbott, L.F. & Sompolinsky, H. Stimulus-dependent suppression of chaos in recurrent neural networks. Phys. Rev. E 82, 011903 (2010).
  32. Churchland, M.M., Yu, B.M., Sahani, M. & Shenoy, K.V. Techniques for extracting single-trial activity patterns from large-scale neural recordings. Curr. Opin. Neurobiol. 17, 609618 (2007).
  33. Rabinovich, M., Huerta, R. & Laurent, G. Neuroscience. Transient dynamics for neural processing. Science 321, 4850 (2008).
  34. Rigotti, M., Rubin, D.B.D., Wang, X.-J. & Fusi, S. Internal representation of task rules by recurrent dynamics: the importance of the diversity of neural responses. Front. Comput. Neurosci. 4, 24 (2010).
  35. Rigotti, M. et al. The importance of mixed selectivity in complex cognitive tasks. Nature 497, 585590 (2013).
  36. Mazor, O. & Laurent, G. Transient dynamics versus fixed points in odor representations by locust antennal lobe projection neurons. Neuron 48, 661673 (2005).
  37. Churchland, M.M., Cunningham, J.P., Kaufman, M.T., Ryu, S.I. & Shenoy, K.V. Cortical preparatory activity: representation of movement or first cog in a dynamical machine? Neuron 68, 387400 (2010).
  38. Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369378 (2010).
  39. Ginzburg, I. & Sompolinsky, H. Theory of correlations in stochastic neural networks. Phys. Rev. E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics 50, 31713191 (1994).
  40. Gerstner, W. Time structure of the activity in neural network models. Phys. Rev. E Stat. Phys. Plasmas Fluids Relat. Interdiscip. Topics 51, 738758 (1995).
  41. Gerstner, W. Population dynamics of spiking neurons: fast transients, asynchronous states, and locking. Neural Comput. 12, 4389 (2000).
  42. Wong, K.F. & Wang, X.-J. A recurrent network mechanism of time integration in perceptual decisions. J. Neurosci. 26, 13141328 (2006).
  43. Rangan, A.V. & Young, L.S. Emergent dynamics in a model of visual cortex. J. Comput. Neurosci. 35, 155167 (2013).
  44. Litwin-Kumar, A. & Doiron, B. Slow dynamics and high variability in balanced cortical networks with clustered connections. Nat. Neurosci. 15, 14981505 (2012).
  45. Tetzlaff, T., Helias, M., Einevoll, G.T. & Diesmann, M. Decorrelation of neural-network activity by inhibitory feedback. PLoS Comp. Biol. 8, e1002596 (2012).
  46. Roxin, A., Brunel, N., Hansel, D., Mongillo, G. & van Vreeswijk, C. On the distribution of firing rates in networks of cortical neurons. J. Neurosci. 31, 1621716226 (2011).
  47. Helias, M., Deger, M., Rotter, S. & Diesmann, M. Instantaneous non-linear processing by pulse-coupled threshold units. PLoS Comp. Biol. 6, e1000929 (2010).
  48. Ostojic, S. Interspike interval distributions of spiking neurons driven by fluctuating inputs. J. Neurophysiol. 106, 361373 (2011).
  49. Rajan, K., Abbott, L.F. & Sompolinsky, H. Inferring stimulus selectivity from the spatial structure of neural network dynamics. in Advances in Neural Information Processing Systems 23 (eds. Lafferty, J., Williams, C.K.I., Shawe-Taylor, J., Zemel, R. & Culotta A.) 19751983 (Curran, 2010).

Download references

Author information

Affiliations

  1. Group for Neural Theory, Laboratoire de Neurosciences Cognitives, INSERM U960, École Normale Supérieure, Paris, France.

    • Srdjan Ostojic

Competing financial interests

The author declares no competing financial interests.

Corresponding author

Correspondence to:

Author details

Additional data