Cortical reliability amid noise and chaos

Typical responses of cortical neurons to identical sensory stimuli appear highly variable. It has thus been proposed that the cortex primarily uses a rate code. However, other studies have argued for spike-time coding under certain conditions. The potential role of spike-time coding is directly limited by the internally generated variability of cortical circuits, which remains largely unexplored. Here, we quantify this internally generated variability using a biophysical model of rat neocortical microcircuitry with biologically realistic noise sources. We find that stochastic neurotransmitter release is a critical component of internally generated variability, causing rapidly diverging, chaotic recurrent network dynamics. Surprisingly, the same nonlinear recurrent network dynamics can transiently overcome the chaos in response to weak feed-forward thalamocortical inputs, and support reliable spike times with millisecond precision. Our model shows that the noisy and chaotic network dynamics of recurrent cortical microcircuitry are compatible with stimulus-evoked, millisecond spike-time reliability, resolving a long-standing debate.


Supplementary Note 1
Divergence is nearly saturated at the scale of the microcircuit It is possible that the amount of internally generated variability in terms of divergence depends not just on the dynamical state of the model circuit but also on its size. We have previously shown that increasing the size of the model beyond the size described above does not alter the observed dynamical states 1 . At this size, dendritic trees and thus the afferent connections of neurons in the lateral center of the microcircuit are fully located within the microcircuit. However, a large fraction of their recurrent connections with neurons in the surrounding tissue are with neurons beyond the periphery of the microcircuit. Since these neurons were not included in the simulations, large portions of synaptic input to peripheral neurons were missing. To quantify the effect of this additional input on variability in the microcircuit, we surrounded the original microcircuit with six additional microcircuits, simulating a much larger mesocircuit, providing missing synaptic input to the neurons at the periphery of the microcircuit (Fig. 2b1, blue and grey). Connectivity in this mesocircuit was homogeneous, both within and between the individual microcircuits.
When we compared the divergence of membrane potentials between micro-and mesocircuit simulations, we found that membrane potentials diverged slightly faster in the mesocircuit, although the time courses of divergence followed similar trends (Fig. 2b2). The mean difference in r V (t) was always below 0.06, and the steady state difference below 0.03. Considering the difference at 10-20 ms (which we found to be a good predictor of the relative order of differences at any time), we found this difference to increase towards the periphery of the microcircuit (Fig. 2b3). This suggests that additional direct synaptic input onto a neuron increases variability, but has only a weak effect on indirectly connected neurons. Thus, at the scale of the microcircuit, the amount of internally generated variability is nearly saturated, albeit underestimated for neurons at the periphery.

Highly connected neurons diverge faster
Next, we explicitly quantified how the time course of divergence depends on the amount of the synaptic input. To this end, we examined the relationship between the similarity s r (t) of a given neuron and the number of connections it receives from within the microcircuit (in-degree). Once more, we found that the time course of divergence was faster, the more synaptic inputs a neuron received, as summarized by s r (t) at 10-20 ms (Fig. 2c). Thus, neurons which are more strongly coupled to the local population 2 diverge more quickly. Additionally, we found that for highly connected neurons, divergence increased with their ratio of excitatory vs. inhibitory inputs ( Supplementary Fig. 3b). Repetition of the analysis using RMSD V (t) instead of r V (t) gave qualitatively similar results.

Supplementary Note 2
Predicting the impact of missing noise sources We studied how the magnitude of a generic white noise depolarizing current affects the time course of divergence. Previously, the variance σ 2 s had been set to 0.001% of the firing threshold for each neuron-a level far lower than other sources of noise. When we increased the variance to values from 0.01% up to 10%, and disabled all other noise sources, we observed that increasing variance led to more rapidly diverging network dynamics ( Supplementary Fig. 8a). However, when other noise sources were also enabled, the noisy current injection only affected network dynamics beyond a certain threshold ( Supplementary Fig. 8b).
To characterize this threshold, we determined the magnitude of white noise required to cause a noticeable change in the network divergence rate. To this end, we used a decoupled replay paradigm with only noisy current injection (as above), for various levels of σ 2 s (Fig. 4d1). As above, we quantified the somatic voltage fluctuations due to this noise source, denoted by RMSD dx ∞,dec (d x : only white noise, with magnitude x). In the corresponding network simulations, the rate of divergence was strongly dependent on RMSD dx ∞,dec , with larger values leading to faster divergence ( Fig. 4d2, dashed line). In contrast, when all noise sources were enabled (Fig. 4d2, solid line), there was only a meaningful influence of the noise injection when it was beyond a threshold in the range 0.1%-0.5%. At this threshold, RMSD d ∞,dec was just above 1 mV, approximately half of the value for synaptic noise sources (RMSD ab ∞,dec , Fig. 4d2, vertical purple line "ab"). When σ 2 s is increased even more, the curves for s r,10−20ms with noisy current alone and with all noise sources eventually converge. Thus, when RMSD d ∞,dec was larger than RMSD ab ∞,dec the noisy current injection dominated other noise sources. This suggests that the strongest source of cellular noise dominates over other sources, unless they are of a comparable magnitude. Taken together, under biological conditions, we predict that synaptic noise is the most important cellular noise source, determining the variability of neuronal responses to presynaptic inputs in vivo. This prediction is consistent with previous findings that cortical neurons respond highly reliably to current injections in vitro, where synaptic noise plays no role 3 .

Quenching of spike-count variability
The predicted mechanism of suppression of chaotic dynamics does not yet have a direct experimental confirmation. However, the effect is related to the often observed quenching of variability-in terms of trial-to-trial spike counts-at the onset of stimuli 4 . In the NMC-model, spike count variance is low both during spontaneous (Fig. 1f) and evoked activity to the same stimulus ( Supplementary  Fig. 12b1). However, in the intact animal, a neocortical microcircuit is integrated with the rest of the brain and constantly receiving input: around 80% of corticocortical synapses are formed with non-local neurons 1 , which are not yet accounted for in the NMC-model. In the behaving brain, most of this external input to the microcircuit will likely contain signals: for example, visual cortex is strongly modulated by movement-related activity 5 . Indeed, when we stimulated the NMC-model with in vivo recordings of thalamic input that was recorded across multiple trials ( Supplementary  Fig. 12a1-3), instead of perfectly identical input, Poisson-like spike count variances sometimes emerged ( Supplementary Fig. 12b2). When we stimulated the NMC-model with variable thalamic input to account for the effect of hidden inputs, Fano factors increased to values observed in rat somatosensory cortex in vivo 6 ( Supplementary Fig. 13a1,a3,b1,b2,c2, t < 0 ms). When we added reliable input on top of the variable input, spike count variability was quenched ( Supplementary Fig.  13a1,a3,b1,b2,c2, t > 0 ms), consistent with previous reports 4 . Importantly, repeating one specific input out of the set of variable inputs once again led to a low spike count variance ( Supplementary  Fig. 13a2,c1). Taken together, these results support the hypothesis that the observed cortical spike count variability in vivo is actually a reliable response to unobserved input, i.e. variable inputs projecting from diverse locations throughout the brain 7,8 . From this point of view, the observed quenching of variability at stimulus onset 4 reflects the statistical impact of knowledge of the stimulus 9 .

Potential effects of missing biological detail
The most important missing detail in the NMC-model is ion-channel noise. Other electrical noise sources such as thermal noise are orders of magnitude smaller 10 . The ion-channel noise in irregular firing neurons in the NMC-model (which is responsible for the irregular initiation of action potentials in vitro 11 ) is overshadowed by synaptic noise under in vivo-like conditions (Fig. 4). But how would additional ion-channel noise in axons and dendrites of all neurons impact variability? In dendrites, ion-channel noise is thought to evoke little to no variability in isolated back-propagating action potentials 12 . Thus, mean ion-channel models are likely sufficient for accurate action potential initiation.
Action potentials reliably permeate axonal arbors of neocortical pyramidal neurons without failures 13 . But as action potentials propagate along axons, their timing becomes increasingly variable. Simulations predict that ion-channel noise affects action potential timing in all axons with a diameter below 0.5 µm, with the standard deviation of action potential variability predicted to increase by 0.6 ms per 2 mm in 0.2 µm diameter axons 14 . In the NMC-model, axons have a mean axonal diameter of around 0.3 µm and are modeled deterministically. Therefore, ion-channel noise in longer axons could increase variability of spike timing by up to several milliseconds.
The missing ion-channel noise might push the circuit towards a more variable state. On the other hand, adding missing detail to the synapse models might increase reliability: The reliability of synaptic transmission increases with the number of readily releasable vesicles 15 . Some studies have found univesicular synaptic transmission at cortical synapses 16 , while others have estimated there may be as many as ten releasable vesicles per synapse 17 .
The current version of the NMC-model assumes one readily releasable vesicle per synapse, and thus potentially underestimates synaptic reliability. To estimate the potential impact of multivesicular release, we repeated the simulation experiments with an increasing number of readily releasable vesicles (n rrp ) at all synapses ( Supplementary Fig. 16). As expected, the time course of divergence slowed with increasing n rrp . Nonetheless, for mean n rrp values which reproduce cortical PSP variability data (n rrp = 2 − 3) 18 , synaptic noise remains the dominant source of noise driving the rapid chaotic divergence. In addition, n rrp may vary between and across synapse types, but a systematic exploration thereof is beyond the scope of the present study.

Supplementary Figures
Supplementary Figure 1 Fig.  4 and Supplementary Fig. 5), with pseudo-deterministic synaptic release by not changing the random seeds for vesicle release (but with a change in 'mini' signals for b). (b) As in a, but with deterministic synaptic release (mean release model), apart from abcd which has the fully stochastic model. Based on 20 saved base states (abcd : 40 saved base states); mean of base states ± 95% confidence interval. (a) Similarity s RMSD when only changing random seeds for noisy depolarization, but with different magnitudes of noise. (b) As in a, but with all noise sources enabled by changing random seeds. Based on 10 saved base states; mean of base states ± 95% confidence interval.

Supplementary
Supplementary Figure 9 -Divergence of evoked activity. (a) The similarity s RMSD defined as the difference between the RMSD V of diverging and independent trials, normalized to lie between 1 (identical) and 0 (fully diverged) (mean ± 95% confidence interval, 20 saved base states), for the thalamic stimulus. (b) Population raster plot and population peristimulus time histogram (PSTH) of all 31'346 neurons in the microcircuit, during evoked activity with a simplified "whisker flick" stimulus (60 VPM neurons are firing at the same time, one spike). (c1) As a, but for the "whisker flick" stimulus. (c2) As c1, but for s r instead of s RMSD . . The other 40 fibers contain a reliable signal (FR = 4 Hz) of randomly distributed spikes, that is highly correlated between the fibers (σ corr = 2 ms) and always arriving at roughly the same time between trials. The variable input lasts from -4000 to 3000 ms, whereas the reliable input only starts after t = 0 ms. (a2) Spike response of example L4 PC across 30 trials to one of the 30 inputs from a1 (top stimulus). (a3) Spike response of same example L4 PC across 30 trials to all 30 inputs from a1. (b1) Variance of spike count vs. mean spike count of central 2024 excitatory neurons in layers 4, 5, and 6 (as before) for the 30 variable input trials, split in bins of 500 ms duration. (b2) Fano factors for the same neurons and time bins as in b1. (c1) Mean Fano factor (blue) and standard deviation (light blue) across trials with the same identical input. (c2) Mean Fano factor (blue) and standard deviation (green) of the neurons in b1 and b2 across trials with variable input.
Supplementary Figure 14 -Input synchrony and reliability. (a1) Response of a layer 4 pyramidal cell to a simple thalamic stimulus consisting of 40 synchronous spikes with increasing jitter (but frozen across trials). (a2) The thalamic stimulus, with increasing normally-distributed jitter with standard deviation σ. The stimulus is kept identical across 30 repetitions to study only intrinsic cortical variability, as before. (b) Mean spike-timing reliability of 2024 pyramidal neurons from layers 4, 5 and 6 (as before) versus jitter standard deviation σ (mean of 30 trials, 95%-CI smaller than marker symbols).