The variable responses of sensory neurons tend to be weakly correlated (spike-count correlation, rsc). This is widely thought to reflect noise in shared afferents, in which case rsc can limit the reliability of sensory coding. However, it could also be due to feedback from higher-order brain regions. Currently, the relative contributions of these sources are unknown. We addressed this by recording from populations of V1 neurons in macaques performing different discrimination tasks involving the same visual input. We found that the structure of rsc (the way rsc varied with neuronal stimulus preference) changed systematically with task instruction. Therefore, even at the earliest stage in the cortical visual hierarchy, rsc structure during task performance primarily reflects feedback dynamics. Consequently, previous proposals for how rsc constrains sensory processing need not apply. Furthermore, we show that correlations between the activity of single neurons and choice depend on feedback engaged by the task.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Nature Communications Open Access 16 November 2021
Nature Communications Open Access 22 July 2021
Nature Communications Open Access 09 October 2020
Subscribe to Nature+
Get immediate online access to Nature and 55 other Nature journal
Subscribe to Journal
Get full journal access for 1 year
only $6.58 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Tomko, G. J. & Crapper, D. R. Neuronal variability: non-stationary responses to identical visual stimuli. Brain Res. 79, 405–418 (1974).
Henry, G. H., Bishop, P. O., Tupper, R. M. & Dreher, B. Orientation specificity and response variability of cells in the striate cortex. Vision Res. 13, 1771–1779 (1973).
Cohen, M. R. & Kohn, A. Measuring and interpreting neuronal correlations. Nat. Neurosci. 14, 811–819 (2011).
Shadlen, M. N. & Newsome, W. T. The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J. Neurosci. 18, 3870–3896 (1998).
Sompolinsky, H., Yoon, H., Kang, K. & Shamir, M. Population coding in neuronal systems with correlated noise. Phys. Rev. E 64, 051904 (2001).
Gu, Y., Angelaki, D. E. & DeAngelis, G. C. Contribution of correlated noise and selective decoding to choice probability measurements in extrastriate visual cortex. eLife 3, 1–19 (2014).
Liu, S., Dickman, J. D., Newlands, S. D., DeAngelis, G. C. & Angelaki, D. E. Reduced choice-related activity and correlated noise accompany perceptual deficits following unilateral vestibular lesion. Proc. Natl. Acad. Sci. USA 110, 17999–18004 (2013).
Zohary, E., Shadlen, M. N. & Newsome, W. T. Correlated neuronal discharge rate and its implications for psychophysical performance. Nature 370, 140–143 (1994).
Abbott, L. F. & Dayan, P. The effect of correlated variability on the accuracy of a population code. Neural Comput. 11, 91–101 (1999).
Snippe, H. P. & Koenderink, J. J. Information in channel-coded systems: correlated receivers. Biol. Cybern. 67, 183–190 (1992).
Averbeck, B. B., Latham, P. E. & Pouget, A. Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358–366 (2006).
Cohen, M. R. & Maunsell, J. H. R. Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, 1594–1600 (2009).
Graf, A. B., Kohn, A., Jazayeri, M. & Movshon, J. A. Decoding the activity of neuronal populations in macaque primary visual cortex. Nat. Neurosci. 14, 239–245 (2011).
Mitchell, J. F., Sundberg, K. A. & Reynolds, J. H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).
Johnson, K. O. Sensory discrimination: decision process. J. Neurophysiol. 43, 1771–1792 (1980).
Moreno-Bote, R. et al. Information-limiting correlations. Nat. Neurosci. 17, 1410–1417 (2014).
Haefner, R. M., Gerwinn, S., Macke, J. H. & Bethge, M. Inferring decoding strategies from choice probabilities in the presence of correlated variability. Nat. Neurosci. 16, 235–242 (2013).
Nienborg, H. & Cumming, B. Correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron’s causality? Curr. Opin. Neurobiol. 20, 376–381 (2010).
Shadlen, M. N., Britten, K. H., Newsome, W. T. & Movshon, J. A. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J. Neurosci. 16, 1486–1510 (1996).
Bair, W., Zohary, E. & Newsome, W. T. Correlated firing in macaque visual area MT: time scales and relationship to behavior. J. Neurosci. 21, 1676–1697 (2001).
Lee, D., Port, N. L., Kruse, W. & Georgopoulos, A. P. Variability and correlated noise in the discharge of neurons in motor and parietal areas of the primate cortex. J. Neurosci. 18, 1161–1170 (1998).
Smith, M. A. & Kohn, A. Spatial and temporal scales of neuronal correlation in primary visual cortex. J. Neurosci. 28, 12591–12603 (2008).
Kohn, A. & Smith, M. A. Stimulus dependence of neuronal correlation in primary visual cortex of the macaque. J. Neurosci. 25, 3661–3673 (2005).
Callaway, E. M. Feedforward, feedback and inhibitory connections in primate visual cortex. Neural Netw. 17, 625–632 (2004).
Sillito, A. M., Cudeiro, J. & Jones, H. E. Always returning: feedback and sensory processing in visual cortex and thalamus. Trends Neurosci. 29, 307–316 (2006).
Ruff, D. A. & Cohen, M. R. Attention can either increase or decrease spike count correlations in visual cortex. Nat. Neurosci. 17, 1591–1597 (2014).
Cohen, M. R. & Newsome, W. T. Context-dependent changes in functional circuitry in visual area MT. Neuron 60, 162–173 (2008).
Lange, R. D. & Haefner, R. M. Inferring the brain’s internal model from sensory responses in a probabilistic inference framework. Preprint at bioRxiv https://doi.org/10.1101/081661 (2016).
Haefner, R. M., Berkes, P. & Fiser, J. Perceptual decision-making as probabilistic inference by neural sampling. Neuron 90, 649–660 (2016).
Cumming, B. G. & Nienborg, H. Feedforward and feedback sources of choice probability in neural population responses. Curr. Opin. Neurobiol. 37, 126–132 (2016).
Nienborg, H. & Cumming, B. G. Decision-related activity in sensory neurons may depend on the columnar architecture of cerebral cortex. J. Neurosci. 34, 3579–3585 (2014).
Nienborg, H. & Cumming, B. G. Psychophysically measured task strategy for disparity discrimination is reflected in V2 neurons. Nat. Neurosci. 10, 1608–1614 (2007).
Ahumada, A. J. Jr. Perceptual classification images from Vernier acuity masked by noise. Perception 25, 2 (1996).
Ecker, A. S., Denfield, G. H., Bethge, M. & Tolias, A. S. On the structure of neuronal population activity under fluctuations in attentional state. J. Neurosci. 36, 1775–1789 (2016).
Britten, K. H., Newsome, W. T., Shadlen, M. N., Celebrini, S. & Movshon, J. A. A relationship between behavioral choice and the visual responses of neurons in macaque MT. Vis. Neurosci. 13, 87–100 (1996).
Crapse, T. B. & Basso, M. A. Insights into decision making using choice probability. J. Neurophysiol. 114, 3039–3049 (2015).
Nienborg, H., Cohen, M. R. & Cumming, B. G. Decision-related activity in sensory neurons: correlations among neurons and with behavior. Annu. Rev. Neurosci. 35, 463–483 (2012).
Moran, J. & Desimone, R. Selective attention gates visual processing in the extrastriate cortex. Science 229, 782–784 (1985).
McAdams, C. J. & Maunsell, J. H. Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4. J. Neurosci. 19, 431–441 (1999).
Goris, R. L. T., Movshon, J. A. & Simoncelli, E. P. Partitioning neuronal variability. Nat. Neurosci. 17, 858–865 (2014).
Nienborg, H. & Cumming, B. G. Decision-related activity in sensory neurons reflects more than a neuron’s causal effect. Nature 459, 89–92 (2009).
Wimmer, K. et al. Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT. Nat. Commun. 6, 6177 (2015).
Kwon, S. E., Yang, H., Minamisawa, G. & O’Connor, D. H. Sensory and decision-related activity propagate in a cortical feedback loop during touch perception. Nat. Neurosci. 19, 1243–1249 (2016).
Treue, S. & Martínez Trujillo, J. C. Feature-based attention influences motion processing gain in macaque visual cortex. Nature 399, 575–579 (1999).
James, W. The Principles of Psychology. (Holt, New York, 1890).
Posner, M. I. & Petersen, S. E. The attention system of the human brain. Annu. Rev. Neurosci. 13, 25–42 (1990).
Tajima, C. I. et al. Population code dynamics in categorical perception. Sci. Rep. 6, 22536 (2016).
Knill, D. C. & Richards, W. Perception as Bayesian Inference. (Cambridge University Press, Cambridge, 1996).
Von Helmholtz, H. Handbuch der physiologischen Optik. 9, (Voss, Leipzig, 1867).
Churchland, M. M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).
Judge, S. J., Richmond, B. J. & Chu, F. C. Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res. 20, 535–538 (1980).
Cumming, B. G. & Parker, A. J. Binocular neurons in V1 of awake monkeys are selective for absolute, not relative, disparity. J. Neurosci. 19, 5602–5618 (1999).
Burgess, A. E. & Colborne, B. Visual signal detection. IV. Observer inconsistency. J. Opt. Soc. Am. A. 5, 617–627 (1988).
Kang, I. & Maunsell, J. H. R. Potential confounds in estimating trial-to-trial correlations between neuronal response and behavior using choice probabilities. J. Neurophysiol. 108, 3403–3415 (2012).
Higham, N. J. Computing a nearest symmetric positive semidefinite matrix. Linear Algebra Appl. 103, 103–118 (1988).
We thank B. Wurtz and J. McFarland for useful discussions; R. Krauzlis, B. Conway and A. Ghazizadeh for comments on an earlier version of the manuscript; and B. Nagy, I. Bunea and D. Parker for veterinary care. This work was supported by the National Eye Institute (Intramural Research Program for A.G.B. and B.G.C. and R01-EY028811 for R.M.H.) .
The authors declare no competing interests.
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Integrated supplementary information
a. To perform psychophysical reverse correlation, we first summarized the orientation energy of each trial (see Methods). Example shown for several 0%-signal trials, with black and gray lines used to indicate the choice elicited (example session, monkey ‘lem’). The dashed vertical lines show the associated task orientations. b. The psychophysical kernel (blue curve) is calculated as the difference between the two choice-conditioned averages (black and gray curves), with an arbitrary choice of sign. Note that the circular mean orientation of the kernel (vertical blue shaded region) is significantly offset (p < 0.001, bootstrap test, one-sided) from the nominally “positive” task orientation (black dashed line), indicating the subject’s choices were primarily based on orientations deviated from those given by task instruction. All shaded regions indicate the mean±1 bootstrap SEM obtained from trial resampling. Axis labels and y-axis units are the same as in (a). c. Average kernel circular mean orientations for two subsets of recording sessions within which similar task contexts were used (same groups as in Fig. 2). Error bars indicate ±1 bootstrap SEM obtained by resampling sessions (n=17 sessions for subset 1 and 24 sessions for subset 2). The average “positive” task orientations for the two groups are shown by the dashed lines. Subjects used distinct task strategies for the two subsets of sessions, as shown by the difference in kernel orientations. Note that the subjects did not completely follow task instruction: the distance between kernel orientations is smaller than the distance between the task orientations.
a,b. Average r sc matrices for each subject, as in Fig. 2e, are similar. Difference in overall average r sc may be due to differences in stimulus size (see Methods). c,d. The eigenvectors corresponding to the largest eigenvalues of the matrices in (a) & (b), as in Fig. 4. Error bars are as in Fig. 4. e,f. Model comparison for individual subject data. In both cases, the “single eigenvector” model outperformed the “diagonal ridge” model (p=0.005 for both animals). (The more complex regression model in Fig. 3 could not be reliably fit using the data from each subject individually due to limited sampling of tasks). As in Figs. 3 and 4, model performance is normalized relative to the explainable variance. This was computed separately for each subject in these panels. Errors bars are 1 SEM around the mean, obtained by resampling pairs (n = 289 for monkey ‘lem’ and 522 for monkey ‘jbe’). Statistical comparison performed using a one-sided bootstrap test obtained in the same way. g,h. Histogram of observed CPs, from the subset of neurons (n=63 and 81 for monkeys ‘lem’ and ‘jbe,’ respectively) significantly preferring one of the two task orientations (d´>0.9 at highest signal level). The means were significantly above chance (p < 0.001 and p=0.004, bootstrap test using cell resampling, one-sided). CPs that were individually significant (p < 0.05, bootstrap test using trial resampling, one-sided) are shown in black. The mean CP was significantly higher in monkey ‘lem’ (p=0.02, bootstrap test using cell resampling, one-sided).
a. Average r sc matrix observed during blocks of trials used to measure neuronal orientation tuning, during which the animal fixated passively for reward (matrix obtained as in Fig. 2e). The lattice-like pattern observed during task performance is absent. Data from only 556 pairs are shown, as not all recording sessions included the interleaved 0% signal trials. b. Eigenspectrum of the matrix in (a), with first five eigenvalues shown. The largest eigenvalue of the r sc matrix in (a) did not exceed the chance distribution (mean +/- 1 SEM in blue), unlike during task performance (Fig. 2). Note that the eigenvalues are smaller than in Fig. 4a because the variance within the matrix in (a) is smaller. Error bars are as in Fig. 4a. c. Goodness-of-fit for the “single eigenvector” and “diagonal ridge” models, calculated as % variance explained relative to our estimate of the explainable variance in the data, as in Fig. 4. Error bars around the mean obtained through repeated 50-fold cross-validation from among the 556 pairs, which also provided the basis for the one-sided statistical comparison (p=0.04). Note these results contrast with those during task performance (Fig. 4e).
a. a. The average, task-aligned V1 r sc matrix for orientation (identical to Fig. 2e). b. The r sc matrix attributable to covariability introduced by random fluctuations in the white noise stimuli, identified using a double-pass trial-shuffling procedure (see Methods). The task-aligned lattice-like structure is abolished, and the mean value of r sc is drastically reduced (note difference in color scales). The remaining structure resembles a weak, diagonal ridge (i.e. “limited range correlations”). Axis labels as in (a). c. A scatter plot of r sc before versus after the trial-shuffling procedure. There was no significant correlation between the two (Pearson’s r, p=0.077, bootstrap test, one-sided) and mean r sc after shuffling was not significantly above zero (mean: −0.0004, p=0.399, bootstrap test, one-sided). Red cross indicates the population average.
a. Average, task-aligned r sc matrix (identical to Fig. 2e). b. Average, task-aligned r sc matrix for orientation after normalizing spike counts to remove the influence of the stimulus on the preceding trial (see Methods). Color scaling and axis labels as in (a). c. Scatter plot of r sc before and after the normalization procedure, showing virtually no difference between the two. (Gray line is the identity line, Pearson’s r shown). This demonstrates that stimulus history does not significantly influence observed r sc .
a,b. The average, task-aligned r sc matrix for orientation for two subsets of trials, obtained from a median split based on the duration of the preceding ITI for each session. The short-ITI subset of trials was preceded by an average ITI of 1.1 seconds, while the long-ITI subset was preceded by an average ITI of 4.8 seconds. The lattice-like pattern is clearly present in both subsets. No attenuation in the rsc structure was observed in the long-ITI subset, inconsistent with stimulus history being a major determinant. Axis labels and color scale in (b) are the same as in (a). c. Slope of (type 2) regression line of r sc values obtained using the two subsets of trials against the r sc values obtained using all trials. Values below 1 indicate an attenuation of the r sc structure and values greater than 1 indicate an amplification. The amplitude of the lattice-like pattern was increased after long-ITI trials relative to short-ITI trials (p < 0.001, bootstrap test, one-sided, n=811 pairs). Error bars represent +/- 1 bootstrap SEM. d. The mean r sc value was also greater after long ITIs (p=0.012, bootstrap test, one-sided, n=811 pairs). Error bars represent +/- 1 bootstrap SEM.
a. The trial-averaged trajectories of mean binocular eye position (example session M225, monkey ‘lem’). Black and gray traces correspond to trials preceding leftward and rightward choice saccades, respectively. Red lines indicate eye position 1.5 s after stimulus onset, around which time the trajectories noticeably diverge. b. Eye position trajectories for a subset of trials from which choice could no longer be predicted, identified using LDA after having removed the last quarter of the trial. c. The Euclidean distance between the trajectories associated with the two choices increased over time. The distances before and after the trial-removal procedure are shown in blue and red, respectively, with error bars indicating +/- 1 SEM obtained from resampling trials (n=1,397 and 1,117 trials before and after trial removal, respectively). The distance expected by chance, obtained by shuffling the choice labels, is shown in black. The distance did not exceed chance levels in the subset of trials left after trial removal until 1.5s after stimulus onset. d,e. Histograms of LDA posterior probabilities, giving the likelihood that a trial was associated with a leftward choice saccade before (d) and after (e) the trial-removal procedure. The gray and black distributions represent trials followed by rightward and leftward choices, respectively. f. The task-aligned V1 r sc matrix for orientation (averaged across all sessions) after applying the trial-removal procedure to each session. This is qualitatively identical to the matrix obtained before removing trials (Fig. 2e). g. Eigenspectrum for the matrix in (f), with first five eigenvalues shown. The largest eigenvalue was significantly larger than chance (p=0.004, permutation test, one-sided), similar to the results in Fig. 4a. Error bars are as in Fig. 4a. h. The eigenvector corresponding to the largest eigenvalue (+/- 1 bootstrap SEM) of the matrix in (f), comparable to Fig. 4b.
a. Pseudocolor plot showing goodness-of-fit for a model describing r sc structure using fixed and task-dependent components, using an alternative approach to the one shown in Fig. 3 (see Methods). Goodness-of-fit is shown for versions of such a model allowing different numbers of basis functions for the two components. The best model includes two basis functions for the task-dependent component only. Model performance is quantified under 50-fold cross-validation, and normalized as described in the Methods. b. Fixed component fits for all versions of the model whose performance is shown in (a). For the best model (circled) there is no fixed component whatsoever (i.e. zero basis functions allowed). In versions of the model in which a fixed component is allowed, there is a consistent structure resembling a weak diagonal ridge with peaks near 30°/30° and 120°/120,° consistent with the model fits using the other modeling approach (Fig. 3). c. Task-dependent component fits for all versions of the model whose performance is shown in (a). The fit from the best model is circled. Again, across versions of the model allowing different numbers of basis functions, the fits are consistent and similar to the results of the alternative model shown in Fig. 3. (Note that, due to the diagonal symmetry of correlation matrices, even fits containing only a single basis function contain two peaks/troughs).
a. Readout weights were unobserved but are required to predict CP based on the observed r sc structure using the feedforward linear pooling framework. We sampled from a large distribution of random readout weight profiles that could support task performance (examples shown) to generate a distribution of CP predictions (see Methods). For illustration, uniform and sinusoidal weight profiles are shown in white and black, respectively. b. The predicted mean CP associated with the average of all readout weights are shown in Fig. 8 in the main text. Here we show the uncertainty associated with the distribution of plausible readout weight profiles. Black bars indicate mean CPs +/- 1 SEM. The predictions associated with the uniform and sinusoidal weight profiles in (a) are shown with white and black circles, respectively. Note that the variability introduced by the readout weights is comparable to the uncertainty in the measurements of r sc (compare error bars to those in Fig. 8c), demonstrating that the results are quite robust to not knowing the true weights. This is not surprising given the observed correlation matrix was nearly rank-1. Since the influence of the readout weights depends on their similarity to the correlation matrix eigenvectors, a low rank matrix has fewer ways to be influenced by different readout weights.
a. Schematic of the task design in Cohen and Newsome (2008)27, after Fig. 1c in that study. For this example neuronal pair with preferred directions indicated by the black, solid arrows, one task was chosen such that the neurons contributed to the same choice (“same pool” condition). The other task was chosen so that the neurons contributed to opposite choices (“different pool” condition). The difference in preferred direction (ΔPD) determined the locations along the purples and green dashed lines in (b) to which this pair contributed data. a. Schematic of the task-aligned r sc matrix for preferred direction, analogous to the matrix we measured (Fig. 2e). The purple and green boxes indicate the regions corresponding to pairs supporting the same and opposite choices, respectively. The dashed lines indicate the regions of the matrix it was possible to sample using their approach. c. Reproduction of Fig. 2d, showing the average, task-aligned r sc matrix for orientation. d. For the sake of comparison, we plot the r sc values from our dataset in a way analogous to the main result in the prior study (i.e. the average data sampled along the “same pool” and “different pool” lines, as given by the dashed lines in (c)). This plot is directly comparable to the main data figure from the prior study (their Fig. 4a) and shows qualitatively similar data, albeit with lower overall values of r sc . We notably failed to replicate the inversion of the main result for pairs with opposing stimulus preference. This feature of the data was explained by the authors as an effect of simultaneous attention to both motion direction and the “motion axis” (i.e. orientation). In our task, an analogous mechanism would require simultaneous attention to a single orientation and an orientation pair, which may not be possible. Therefore, it is unclear to what degree this difference reflects a true discrepancy in the data. Standard error bar and statistical comparison both obtained from a bootstrap obtained by resampling from the set of 811 pairs.
The average, task-aligned V1 covariance matrix, calculated as describe in Methods. This is closely similar to the r sc matrix shown in Fig. 2e, demonstrating that this central observation is not a consequence of using r sc values based on normalized spike counts.
About this article
Cite this article
Bondy, A.G., Haefner, R.M. & Cumming, B.G. Feedback determines the structure of correlated variability in primary visual cortex. Nat Neurosci 21, 598–606 (2018). https://doi.org/10.1038/s41593-018-0089-1
This article is cited by
Nature Communications (2021)
Nature Communications (2021)
Nature Neuroscience (2020)
Nature Communications (2020)