Simultaneous multi-area recordings suggest that attention improves performance by reshaping stimulus representations

Abstract

Visual attention dramatically improves individuals’ ability to see and modulates the responses of neurons in every known visual and oculomotor area, but whether such modulations can account for perceptual improvements is unclear. We measured the relationship between populations of visual neurons, oculomotor neurons and behavior during detection and discrimination tasks. We found that neither of the two prominent hypothesized neuronal mechanisms underlying attention (which concern changes in information coding and the way sensory information is read out) provide a satisfying account of the observed behavioral improvements. Instead, our results are more consistent with the hypothesis that attention reshapes the representation of attended stimuli to more effectively influence behavior. Our results suggest a path toward understanding the neural underpinnings of perception and cognition in health and disease by analyzing neuronal responses in ways that are constrained by behavior and interactions between brain areas.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Hypotheses and methods.
Fig. 2: Schematic of our decoding procedure.
Fig. 3: Effects of attention on the stimulus information that can be decoded from small populations of MT neurons.
Fig. 4: Similar attention-related effects on neuronal populations in two brain areas and two tasks.
Fig. 5: Effects of attention on the stimulus information that can be decoded from small populations of MT neurons is explained by changes in response variability.

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Code availability

The custom MATLAB code is available from the authors upon reasonable request.

References

  1. 1.

    Carrasco, M. Visual attention: the past 25 years. Vision Res. 51, 1484–1525 (2011).

  2. 2.

    Maunsell, J. H. R. Neuronal mechanisms of visual attention. Annu. Rev. Vis. Sci. 1, 373–391 (2015).

  3. 3.

    Reynolds, J. H. & Chelazzi, L. Attentional modulation of visual processing. Annu. Rev. Neurosci. 27, 611–647 (2004).

  4. 4.

    Krauzlis, R. J., Lovejoy, L. P. & Zénon, A. Superior colliculus and visual spatial attention. Annu. Rev. Neurosci. 36, 165–182 (2013).

  5. 5.

    Cohen, M. R. & Maunsell, J. H. R. Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, 1594–1600 (2009).

  6. 6.

    Ruff, D. A. & Cohen, M. R. Attention can either increase or decrease spike count correlations in visual cortex. Nat. Neurosci. 17, 1591–1597 (2014).

  7. 7.

    Mitchell, J. F., Sundberg, K. A. & Reynolds, J. H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).

  8. 8.

    Fries, P. Rhythms for cognition: communication through coherence. Neuron 88, 220–235 (2015).

  9. 9.

    Ruff, D. A. & Cohen, M. R. Attention increases spike count correlations between visual cortical areas. J. Neurosci. 36, 7523–7534 (2016).

  10. 10.

    Ruff, D. A. & Cohen, M. R. A normalization model suggests that attention changes the weighting of inputs between visual areas. Proc. Natl Acad. Sci. USA 114, E4085–E4094 (2017).

  11. 11.

    Gregoriou, G. G., Gotts, S. J., Zhou, H. & Desimone, R. High-frequency, long-range coupling between prefrontal and visual cortex during attention. Science 324, 1207–1210 (2009).

  12. 12.

    Born, R. T. & Bradley, D. C. Structure and function of visual area MT. Annu. Rev. Neurosci. 28, 157–189 (2005).

  13. 13.

    Parker, A. J. & Newsome, W. T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998).

  14. 14.

    Gandhi, N. J. & Katnani, H. A. Motor functions of the superior colliculus. Annu. Rev. Neurosci. 34, 205–231 (2011).

  15. 15.

    Sparks, D. L. Conceptual issues related to the role of the superior colliculus in the control of gaze. Curr. Opin. Neurobiol. 9, 698–707 (1999).

  16. 16.

    Wurtz, R. H. & Albano, J. E. Visual-motor function of the primate superior colliculus. Annu. Rev. Neurosci. 3, 189–226 (1980).

  17. 17.

    Horwitz, G. D. & Newsome, W. T. Separate signals for target selection and movement specification in the superior colliculus. Science 284, 1158–1161 (1999).

  18. 18.

    Crapse, T. B., Lau, H. & Basso, M. A. A role for the superior colliculus in decision criteria. Neuron 97, 181–194.e6 (2018).

  19. 19.

    Horwitz, G. D. & Newsome, W. T. Target selection for saccadic eye movements: prelude activity in the superior colliculus during a direction-discrimination task. J. Neurophysiol. 86, 2543–2558 (2001).

  20. 20.

    Cohen, M. R. & Maunsell, J. H. R. Using neuronal populations to study the mechanisms underlying spatial and feature attention. Neuron 70, 1192–1204 (2011).

  21. 21.

    Ni, A. M., Ruff, D. A., Alberts, J. J., Symmonds, J. & Cohen, M. R. Learning and attention reveal a general relationship between population activity and behavior. Science 359, 463–465 (2018).

  22. 22.

    Mayo, J. P. & Maunsell, J. H. Graded neuronal modulations related to visual spatial attention. J. Neurosci. 36, 5353–5361 (2016).

  23. 23.

    Ruff, D. A., Ni, A. M. & Cohen, M. R. Cognition as a window into neuronal population space. Annu. Rev. Neurosci. 41, 77–97 (2018).

  24. 24.

    Cohen, M. R. & Kohn, A. Measuring and interpreting neuronal correlations. Nat. Neurosci. 14, 811–819 (2011).

  25. 25.

    Pooresmaeili, A., Poort, J. & Roelfsema, P. R. Simultaneous selection by object-based attention in visual and frontal cortex. Proc. Natl Acad. Sci. USA 111, 6467–6472 (2014).

  26. 26.

    Ecker, A. S., Denfield, G. H., Bethge, M. & Tolias, A. S. On the structure of neuronal population activity under fluctuations in attentional state. J. Neurosci. 36, 1775–1789 (2016).

  27. 27.

    Denfield, G. H., Ecker, A. S., Shinn, T. J., Bethge, M. & Tolias, A. S. Attentional fluctuations induce shared variability in macaque primary visual cortex. Nat. Commun. 9, 2654 (2018).

  28. 28.

    Moreno-Bote, R. et al. Information-limiting correlations. Nat. Neurosci. 17, 1410–1417 (2014).

  29. 29.

    Kanitscheider, I., Coen-Cagli, R. & Pouget, A. Origin of information-limiting noise correlations. Proc. Natl Acad. Sci. USA 112, E6973–6982 (2015).

  30. 30.

    Kohn, A., Coen-Cagli, R., Kanitscheider, I. & Pouget, A. Correlations and neuronal population information. Annu. Rev. Neurosci. 39, 237–256 (2016).

  31. 31.

    Semedo, J. D., Zandvakili, A., Machens, C. K., Yu, B. M. & Kohn, A. Cortical areas interact through a communication subspace. Neuron 102, 249–259.e4 (2019).

  32. 32.

    Zénon, A. & Krauzlis, R. J. Attention deficits without cortical neuronal deficits. Nature 489, 434–437 (2012).

  33. 33.

    Herrero, J. L., Gieselmann, M. A., Sanayei, M. & Thiele, A. Attention-induced variance and noise correlation reduction in macaque V1 is mediated by NMDA receptors. Neuron 78, 729–739 (2013).

  34. 34.

    Gregoriou, G. G., Rossi, A. F., Ungerleider, L. G. & Desimone, R. Lesions of prefrontal cortex reduce attentional modulation of neuronal responses and synchrony in V4. Nat. Neurosci. 17, 1003–1011 (2014).

  35. 35.

    Luo, T. Z. & Maunsell, J. H. R. Neuronal modulations in visual cortex are associated with only one of multiple components of attention. Neuron 86, 1182–1188 (2015).

  36. 36.

    Nandy, A. S., Nassi, J. J. & Reynolds, J. H. Laminar organization of attentional modulation in macaque visual area V4. Neuron 93, 235–246 (2017).

  37. 37.

    Snyder, A. C., Morais, M. J. & Smith, M. A. Dynamics of excitatory and inhibitory networks are differentially altered by selective attention. J. Neurophysiol. 116, 1807–1820 (2016).

  38. 38.

    Egeth, H. E. & Yantis, S. Visual attention: control, representation, and time course. Annu. Rev. Psychol. 48, 269–297 (1997).

  39. 39.

    Huang, C. et al. Circuit models of low-dimensional shared variability in cortical networks. Neuron 101, 337–348.e4 (2019).

  40. 40.

    Kanashiro, T., Ocker, G. K., Cohen, M. R. & Doiron, B. Attentional modulation of neuronal variability in circuit models of cortex. eLife 6, e23978 (2017).

  41. 41.

    Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013).

  42. 42.

    Krauzlis, R. J., Bollimunta, A., Arcizet, F. & Wang, L. Attention as an effect not a cause. Trends Cogn. Sci. 18, 457–464 (2014).

  43. 43.

    Baruni, J. K., Lau, B. & Salzman, C. D. Reward expectation differentially modulates attentional behavior and activity in visual area V4. Nat. Neurosci. 18, 1656–1663 (2015).

  44. 44.

    Steinmetz, N. A. & Moore, T. Eye movement preparation modulates neuronal responses in area V4 when dissociated from attentional demands. Neuron 83, 496–506 (2014).

  45. 45.

    Averbeck, B. B., Latham, P. E. & Pouget, A. Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358–366 (2006).

  46. 46.

    Wang, X. J. & Yang, G. R. A disinhibitory circuit motif and flexible information routing in the brain. Curr. Opin. Neurobiol. 49, 75–83 (2018).

  47. 47.

    Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X. J. Task representations in neural networks trained to perform many cognitive tasks. Nat. Neurosci. 22, 297–306 (2019).

  48. 48.

    Buonomano, D. V. & Maass, W. State-dependent computations: spatiotemporal processing in cortical networks. Nat. Rev. Neurosci. 10, 113–125 (2009).

  49. 49.

    Huang, G., Huang, G. B., Song, S. & You, K. Trends in extreme learning machines: a review. Neural Netw. 61, 32–48 (2015).

  50. 50.

    Carandini, M. & Heeger, D. J. Normalization as a canonical neural computation. Nat. Rev. Neurosci. 13, 51–62 (2012).

  51. 51.

    Brainard, D. H. The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).

  52. 52.

    Pelli, D. G. The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vis. 10, 437–442 (1997).

  53. 53.

    Engbert, R. & Kliegl, R. Microsaccades uncover the orientation of covert attention. Vision Res. 43, 1035–1045 (2003).

Download references

Acknowledgements

We are grateful to J. Maunsell for granting permission for the use of the V4 data, to K. McKracken for providing technical assistance and to F. Baqai, M. Getz, J. Hennig, C. Huang, A. Ni and J. Maunsell for comments on an earlier version of this manuscript. M.R.C. is supported by National Institutes of Health grant nos. R00EY020844 and R01EY022930 and by Core Grant P30 EY008098s, and by the McKnight, Whitehall, Sloan and Simons Foundations.

Author information

D.A.R. and M.R.C. conceived and designed the experiments, analyzed the data and wrote the manuscript. D.A.R. collected the data.

Correspondence to Douglas A. Ruff.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Integrated supplementary information

Supplementary Figure 1 Effects of attention on common analyses of individual units and pairs of units.

(A) Attention increases firing rates in MT, quantified as the difference in firing rates in the different attention conditions divided by the sum. Units with significant differences in average responses for the two conditions are specified by black bars (two-sided t-test, p<.05). This distribution (mean = 0.04, median = 0.04) is significantly different from zero (N=306 units, two-sided Wilcoxon signed rank test, p=4.0X10-22). (B) Same as A, for SC data. This distribution (mean = 0.073, median = 0.05) is significantly different from zero (N=345 units, two-sided Wilcoxon signed rank test, p=4.8X10-44). (C) Within and between area noise correlations calculated from spike counts during stimulus presentations that preceded successful maintenance of fixation from trials that ended with either a hit or miss or were a successful catch trial. Attention decreases average correlations within MT (N=3,285 pairs, two-sided Wilcoxon signed rank test, p=6.6X10-15), not in the SC (N=3,948 pairs, two-sided Wilcoxon signed rank test, p=0.38) and increases them between the two areas (N=6,934 pairs, two-sided Wilcoxon signed rank test, p=5.7X10-58). Error bars are standard error of the mean. (D) Within and between area noise correlations calculated from spike counts that immediately preceded different behavioral outcomes during cued trials. Misses and false alarms are associated with higher correlations within MT (two-tailed t-test on hits and correct rejects versus misses and false alarms, p=0.0094) and SC (two-sided t-test on hits and correct rejects versus misses and false alarms, p=0.03) but not between the two areas (two-tailed t-test, p=0.23). Error bars are standard error of the mean.

Supplementary Figure 2 Relationship between SC responses during different task epochs and attention-related correlation changes with MT.

(A) Schematic of task timing depicts the three 100ms epochs used to count spikes in SC units. The baseline period began 100ms before the first stimulus appeared, which is after stable fixation had been acquired. The stimulus period was shifted 30 ms after the appearance of the visual stimulus, to account for the earliest visual latencies observed in the SC. The delay period began 100ms after the first stimulus turned off and always ended prior to the onset of the second stimulus. (B) Attention-related changes in MT-SC rSC plotted against the difference between each SC unit’s response during the stimulus and baseline periods. There are multiple MT-SC correlation differences measured for each SC unit. Correlations between MT and SC were calculated using the same data and methods as Supplementary Fig. 1c (N=6,934 pairs, Pearson correlation, rho=0.088, two-sided t-test, p=2.4X10-13). Isolines depicting the decile boundaries are overlaid over the individual data points. (C) Similar to B, but data are now sorted by the difference between each SC unit’s response during the stimulus and delay periods (N=6,934 pairs, Pearson correlation, rho=0.091, two-sided t-test, p=2.8X10-14). (D) Similar to B, but data are now sorted by the difference between each SC unit’s response during the delay and baseline periods (N=6,934 pairs, Pearson correlation, rho=-0.037, two-sided t-test, p=0.0023).

Supplementary Figure 3 Attention has opposite effects on slow fluctuations in neuronal population responses within and across areas.

(A, B) Autocorrelations between projections onto the first principal component of population responses to repeated presentations of the same visual stimulus in (A) MT, and (B) the SC. The x-axis plots time lag in units of stimulus presentations (400-600 ms). Because these were responses to identical stimuli, the first PC is the dimension in population space that accounts for the greatest shared variability across the population. (C) Cross correlation between projections onto the first PCs in MT and the SC (same data and plotting conventions as in A and B). (D) Attention-related difference in autocorrelation or cross correlations between the projections in the previous plots. Error bars represent standard error of the mean. Attention was associated with a statistically significant decrease in autocorrelation overall (t-tests, p=0.04 in MT and 0.02 in SC) in both areas and in 11/15 individual MT data sets and 9/15 SC data sets (bootstrap tests, p<0.05 with a Bonferroni correction) and a significant increase in cross correlation overall (t-test, p=0.0009) and in 11/15 individual data sets (bootstrap tests, p<0.05 with a Bonferroni correction).

Supplementary information

Supplementary Figs. 1–3.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark