Optimal responsiveness and information flow in networks of heterogeneous neurons

Cerebral cortex is characterized by a strong neuron-to-neuron heterogeneity, but it is unclear what consequences this may have for cortical computations, while most computational models consider networks of identical units. Here, we study network models of spiking neurons endowed with heterogeneity, that we treat independently for excitatory and inhibitory neurons. We find that heterogeneous networks are generally more responsive, with an optimal responsiveness occurring for levels of heterogeneity found experimentally in different published datasets, for both excitatory and inhibitory neurons. To investigate the underlying mechanisms, we introduce a mean-field model of heterogeneous networks. This mean-field model captures optimal responsiveness and suggests that it is related to the stability of the spontaneous asynchronous state. The mean-field model also predicts that new dynamical states can emerge from heterogeneity, a prediction which is confirmed by network simulations. Finally we show that heterogeneous networks maximise the information flow in large-scale networks, through recurrent connections. We conclude that neuronal heterogeneity confers different responsiveness to neural networks, which should be taken into account to investigate their information processing capabilities.

Studying the collective behavior of large numbers of units interacting non-linearly is a classical theme in physical sciences.In biology, such studies are complicated by the fact that the units are usually non identical, but rather display considerable heterogeneity.This particularly apparent in cerebral cortex, where neuronal size and properties are highly heterogeneous [1][2][3][4][5][6][7][8][9][10][11].Neuronal heterogeneity is particularly high for inhibitory neurons, for which many cell classes were observed [12][13][14][15].In networks of oscillators, as in neuronal networks, such heterogeneity across units (bare frequencies or neurons' excitabilities) induces typically desynchronization at a population scale [16][17][18][19][20][21][22].As one would expect, the more the neurons are different the less they are able to synchronize and to correlate their reciprocal activity.Nevertheless, despite this heterogeneity, cortical populations are able to respond coherently to external stimuli and generate collective oscillations [23,24].
In this paper, we examined networks of excitatory and inhibitory neurons, with an emphasis on inhibitory heterogeneity to understand its possible impact at the population level.We used networks of N neurons, 80% of excitatory (N E = 0.8N ) and 20% of inhibitory (N I = 0.2N ) neurons.The membrane potential V i of each neuron evolves according to the adaptive exponential integrate and fire model [25]: where C m = 200pF is the membrane capacitance, g L = 15nS the leakage conductance, v th = 50mV the effective threshold and ∆=0.5mV defines the action potential rise.The adaptation current w i increases of an amount b = 40nS at each spike emitted by neuron i at times {t sp i } and has an exponential decay with time scale τ w = 500ms.The parameter a indicates subthreshold adaptation, that we will consider a = 0nS if not otherwise stated.The current I ext an external current and I s the synaptic current from other neurons.We consider a random graph where each couple of neurons is connected with probability p = 0.05.By calling {t sp j } the ensemble of spiking times of neuron j the synaptic current received by neuron i evolves as where E E,I is the reversal for excitatory (E E = 0mV) and inhibitory synapses (E I = −80mV), τ s = 5ms the synaptic decay time and Q E,I the interaction strength of excitatory (Q E = 10µS) and inhibitory (Q I = 50µS) synapses.The sum runs over the presynaptic excitatory or inhibitory neurons.The current I ext is an external input received by all neurons, independent and identically distributed, modeled as an additive excitatory Poissonian spike train at a frequency ν ext .In absence of stimuli ν ext is constant in time and we consider ν ext = 2.5Hz to keep the network active.In order to study the response to external stimuli a time varying ν ext (t) will be considered (see Figure captions for details).
To model heterogenetity, we considered a Gaussian distribution of the leakage reversal E i L of inhibitory (excitatory) population N (E E,I L , σ 2 E,I ).The rescaled standard deviation σ E,I /E E,I L will be the main parameters to quantify heterogeneity.The synchronization of the network and its response to external stimuli are quantified by measuring the total amount of evoked spikes R (responsiveness) and the input/output correlation between the stimuli and excitatory neurons firing rate r E , C = r E ν ext − r E ν ext , where • denotes a time average.
A first effect of inhibitory heterogeneity is to increase population responsiveness, as shown in Fig. 1.In this simulation, the homogeneous network shows a spontaneous asynchronous activity where neurons fire irregularly, a dynamical regime due to the balance between excitation and inhibition typically observed in the cortex of awake animals, called asynchronous irregular [26][27][28].The presence of external excitatory stimuli of short duration (see caption of Fig. 1 for details) produces a transient synchronous event (population burst).The total amount of additional spikes evoked by the stimuli is indicated by R. By increasing the amount of heterogeneity in inhibitory neurons σ I we observe a clear increase of the evoked activity R (see Fig. 1a).Indeed, the same input induces a bell-shaped response R in function of the heterogeneity σ I .As can be seen from Fig. 1c,d,e, there is an optimal heterogeneity level (σ I ∼ 0.2), where the stimuli provokes a strong synchronous population response where almost the whole network activates.Eventually, when σ I is too large the response is very weak.As can be noticed from the bottom panels of Fig. 1, increasing heterogeneity in inhibitory neurons decreases excitatory neurons spontaneous activity.This is due to the presence of a fraction of inhibitory neurons with high excitability that inhibits the excitatory population.In Fig. 1b we report R, as in Fig. 1a, as a function of the excitatory spontaneous activity pre-stimulus r E .We can observe that the responsiveness is maximum, in correspondence of σ I ∼ 0.2, for a relatively low value of excitatory spontaneous activity (around r E ∼ 0.7Hz).Interestingly, if we consider a homogeneous network, i.e. σ I = 0, and decrease spontaneous excitatory activity by increasing the average excitability of inhibitory neurons we do not observe an increase of responsiveness R (see blue dots in Fig. 1b).This result shows that the increase in the size of the synchronous response is due to the presence of heterogeneity and cannot be replaced by a modification of the average excitability in the corresponding homogeneous network.
In contrast, increasing the heterogeneity of excitatory neurons σ E has the opposite effect, i.e. a constant decrease in network response (red squares in Fig. 1a).At first sight, this seems contradictory with previous reports where it was shown that the heterogeneity of excitatory neurons' intrinsic excitability can induce higher population responsiveness [29,30].However, in all these studies, the homogeneous system was set in an excitable state characterised by no spontaneous activity in the absence of stimulus.Such an excitable regime is qualitatively different from the one we considered in Fig. 1, where all neurons are characterised by spontaneous and irregular ongoing activity, like in cerebral cortex when excitation and inhibition balance each other.Could this be the origin of the difference we observed?
To verify this scenario, we decreased the constant excitatory drive ν ext that all neurons receive, in such a way that the homogeneous network is silent (excitable) in the absence of external stimuli.We then considered the correlation C between the external input (that activates the network) and the network output r E .We report in Fig. 2 the correlation C in function of both σ E and σ I .We observe that increasing σ E we recover an increase of the correlation C that decreases for high σ E , just like in previous studies [29,30].Indeed, increasing heterogeneity among excitatory neurons permits the presence of active neurons in response to the stimuli (see the white line in Fig. 2 separating silent and active spontaneous activity pre-stimulus).We report here, on top of an optimal response in the direction of σ E , an optimal responsiveness also in the direction of σ I .Indeed, once the network is active (i.e.high σ E ) the role of σ I is the same of that we reported in Fig. 1a, in which case the network was active due to a higher constant external drive.Eventually, when heterogeneity is too large, neurons are too diverse and cannot correlate in order to yield a coherent synchronous population response to the stimuli.We have verified this scenario to be robust across the specific choice of the stimuli (amplitude, frequency etc..).
This result shows for the first time an optimal amount of heterogeneity in both excitatory and inhibitory neurons for a coherent population response to external stimuli.But is this optimal heterogeneity comparable to what is found experimentally?To answer this question, we analysed experimental data acquired from cells originating from the adult human brain from Allen Brain Atlas [31].In the right panels of Fig. 2 we report the histogram of the reversal potential measured experimentally from 200 excitatory (green, upper panel) and 50 inhibitory neurons (red, lower panel).One can see a heterogeneous distribution of the rescaled resting potential e L = E L / ĒL , with σ I ∼ 0.070 and σ E ∼ 0.056, grey circle in Fig. 2. Note that heterogeneity in inhibitory cells is higher with respect to that in excitatory cells.Importantly, these experimental values fall close to the predicted optimal region of network responsiveness.Thus, the model predicts that the optimal heterogeneity matches that found in real neural networks, suggesting that this a crucial factor to understand their responsiveness.
In order to study the mechanism at the origin of the enhanced synchronous population response for heterogeneous inhibitory neurons we developed a mean field approach explicitly taking into account diversity.We started from a mean-field model previously introduced for homogeneous neural populations [32][33][34].The corresponding mean field equations describe the dynamics of collective variables, namely excitatory and inhibitory population firing rates, by assuming a Markovian dynamics over a time scale T ∼ τ m = 15mS.We extend this approach to heterogeneous systems by employing a technique, called heterogeneous mean field (HMF), succesfully applied previously to model networks with hetero-geneous connectivity [35,36].By performing the thermodynamic limit N → ∞, we consider the network as composed by an infinite amount of classes of neurons, each one characterized by a specific leakage current E I L .By defining the transfer function F E I L (r E , r I ) for the class of neurons with leakage reversal E I L as neurons' firing activity in function of excitatory (inhibitory) input rate r E (r I ), the heterogeneous mean field (HMF) equations read:

FIG. 3.
Upper panels show the analysis of stability of the fixed point from the HMF model.In panel a) we report the stable (unstable) fixed point in continuous (dashed) black (blue) line.Black continuous line shows the maximum and the minimum of oscillations or the firing rate r E .In the lower panel we report, for this fixed point, the maximum of eigenvalues' real parts, λmax.The grey area shows the region where λmax > 0 and collective oscillations appear.In the right panel c) we report in grey the region where λmax > 0 in function of σI and the average E I L .Lower panels show the raster plot from a simulation of N = 10000 neurons for d) σI = 0 , e) σI = 0.18 and f) σI = 0.4.The constant drive has a frequency ν0 = 2.5Hz , E I L = −66mV and a = 4nS.
marked in grey, with λ max > 0 where a limit cycle appears.In Fig. 3c we report the critical values for which λ max > 0 in function of σ I and the average E I L .It is important to notice that the limit cycle is observed only for an heterogeneous system and is not observed in an homogeneous case (σ I = 0), whatever the modification of the the average excitability of inhibitory neurons.In order to verify these predictions we performed numerical simulations in the spiking network and we observed that, as predicted by the HMF, an heterogeneous system shows synchronous collective oscillations that are not observed in the homogeneous (or too heterogeneous) case, see raster plots in Fig. 3.We have performed a size analyses in order to verify that collective oscillations observed in the network do not disappear in the thermodynamic limit (see Supplementary materials).This result shows that an optimal amount of heterogeneity in inhibitory neurons increases population coherence by synchronizing the whole network.Note that here, the firing activity of single neurons remains irregular even during collective oscillations, as it happens for sparsely synchronous dynamics in balanced networks [37,38].
In conclusion, we report here three findings.First, we have found that the heterogeneity of inhibitory neu-rons, which has been well documented experimentally [12][13][14][15], optimises the responsiveness of spontaneously active networks to external stimuli.There appears a resonance peak as a function of the level of heterogeneity.A similar effect of diversity-induced resonance was previously observed in excitable or bistable systems [29,[39][40][41][42], where heterogeneity in excitatory elements creates active clusters which were absent in the quiescent homogeneous system.We showed here that an optimal population response is obtained for heterogeneous excitabilities in both excitatory and inhibitory neurons, via a different mechanism taking place in spontaneously active networks with irregular activity in each neuron.Importantly, we found that the level of heterogeneity measured experimentally corresponds to the resonance peak, which suggests that cortical networks may have naturally evolved towards optimal responsiveness by adjusting their heterogeneity.Moreover, while several studies reported that heterogeneity can enhance coding in uncoupled networks [43,44] and decrease neuronal correlations [45][46][47], we report here that a higher input-output population response is linked to an increased synchronisation observed for an optimal heterogeneity of inhibitory neurons.The coding capabilities of neural networks will therefore be largely affected by neuronal heterogeneity, which opens interesting perspectives for future studies.
Second, we designed a mean-field model that explicitly includes heterogeneity, and which can capture this diversity-induced resonance.This new mean-field formulation keeps track of microscopic complexity, compared to traditional mean-field approaches which implicitly assume homogeneous systems and would not predict the correct responsiveness.
Third, we have shown that neuronal heterogeneity is not only important for responsiveness, but also a transition to a sparsely synchronous collective oscillation regime emerges by increasing heterogeneity.This type of diversity-induced oscillations reminds some aspects found in noise-induced transitions in dynamical systems [48,49].Whether the effects of heterogeneity could be considered as analogous to the effect of noise ("quenched noise") in neural networks is also an interesting direction for future studies.

FIG. 1 .
FIG. 1. Panel a) The evoked response R to an external stimuli is reported in function of the heterogeneity of inhibitory (excitatory) neurons σI (σE), black (red) dots (squares).Error bars are estimated as the standard deviation over 20 different realisations.Continuous lines report the prediction based on the mean field model (see main text).Panel b) Data in a), black dots, are reported in function of the spontaneous excitatory firing rate r E before the input arrival (average over 10s).Blue diamonds report R as measured by varying the average leakage current of inhibitory neurons ĒL for σI = σE = 0. Lower panels show the the raster plot, i.e. spiking times of excitatory (inhibitory) neurons marked with green (red) dots, for c) σI = 0, d) σI = 0.2 and e) σI = 0.4 .Lower inset shows the time course of νext.In this simulation we consider N = 10000 neurons.

FIG. 2 .
FIG. 2. Panel a) shows the correlation C for different values of σE and σI .We consider a sinusoidal function for the external rate νext = ν0 + Asin(f t), with A = 0.5Hz, f = 10Hz and ν0 = 0.5Hz.The correlation C is estimated by averaging over 10s.The white line separates the region where the network is characterized by a relatively high (r E > 0.1Hz) spontaneous activity.Panel b) (panel c)) shows the histograms of the resting potential measured over 200 (50) excitatory (inhibitory) neurons in green (red), experimentally measured from cells originating from the adult human brain (data from Allen Brain Atlas [31] ).The continuous line is a Gaussian distribution with the same standard deviation as measured from data, for which σI ∼ 0.070 and σE ∼ 0.056.