Abstract
Internal states such as arousal, attention and motivation modulate brain-wide neural activity, but how these processes interact with learning is not well understood. During learning, the brain modifies its neural activity to improve behavior. How do internal states affect this process? Using a brain–computer interface learning paradigm in monkeys, we identified large, abrupt fluctuations in neural population activity in motor cortex indicative of arousal-like internal state changes, which we term ‘neural engagement.’ In a brain–computer interface, the causal relationship between neural activity and behavior is known, allowing us to understand how neural engagement impacted behavioral performance for different task goals. We observed stereotyped changes in neural engagement that occurred regardless of how they impacted performance. This allowed us to predict how quickly different task goals were learned. These results suggest that changes in internal states, even those seemingly unrelated to goal-seeking behavior, can systematically influence how behavior improves with learning.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The data that support the findings of this study are available from the authors upon reasonable request.
Code availability
The code used in this study for performing analyses and generating figures can be found at https://github.com/mobeets/neural-engagement/.
References
Aston-Jones, G. & Cohen, J. D. An integrative theory of locus coeruleus–norepinephrine function: adaptive gain and optimal performance. Annu. Rev. Neurosci. 28, 403–450 (2005).
McGinley, M. J. et al. Waking state: rapid variations modulate neural and behavioral responses. Neuron 87, 1143–1161 (2015).
Allen, W. E. et al. Thirst regulates motivated behavior through modulation of brain-wide neural population dynamics. Science 364, 253–253 (2019).
Stringer, C. et al. Spontaneous behaviors drive multidimensional, brain-wide activity. Science 364, 255 (2019).
Steinmetz, N. A., Zatka-Haas, P., Carandini, M. & Harris, K. D. Distributed coding of choice, action and engagement across the mouse brain. Nature 576, 266–273 (2019).
Mitchell, J. F., Sundberg, K. A. & Reynolds, J. H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).
Cohen, M. R. & Maunsell, J. H. Attention improves performance primarily by reducing interneuronal correlations. Nat. Neurosci. 12, 1594 (2009).
Vinck, M., Batista-Brito, R., Knoblich, U. & Cardin, J. A. Arousal and locomotion make distinct contributions to cortical activity patterns and visual encoding. Neuron 86, 740–754 (2015).
Averbeck, B. B., Latham, P. E. & Pouget, A. Neural correlations, population coding and computation. Nat. Rev. Neurosci. 7, 358–366 (2006).
Moreno-Bote, R. et al. Information-limiting correlations. Nat. Neurosci. 17, 1410–1417 (2014).
Ruff, D. A. & Cohen, M. R. Simultaneous multi-area recordings suggest that attention improves performance by reshaping stimulus representations. Nat. Neurosci. 22, 1669–1676 (2019).
Cowley, B. R. et al. Slow drift of neural activity as a signature of impulsivity in macaque visual and prefrontal cortex. Neuron 108, 551–567 (2020).
Sugrue, L. P., Corrado, G. S. & Newsome, W. T. Matching behavior and the representation of value in the parietal cortex. science 304, 1782–1787 (2004).
Mazzoni, P., Hristova, A. & Krakauer, J. W. Why don’t we move faster? Parkinson’s disease, movement vigor and implicit motivation. J. Neurosci. 27, 7105–7116 (2007).
Xu-Wilson, M., Zee, D. S. & Shadmehr, R. The intrinsic value of visual information affects saccade velocities. Exp. Brain Res. 196, 475–481 (2009).
Leathers, M. L. & Olson, C. R. In monkeys making value-based decisions, lip neurons encode cue salience and not action value. Science 338, 132–135 (2012).
Li, C.-S. R., Padoa-Schioppa, C. & Bizzi, E. Neuronal correlates of motor performance and motor learning in the primary motor cortex of monkeys adapting to an external force field. Neuron 30, 593–607 (2001).
Andalman, A. S. & Fee, M. S. A basal ganglia–forebrain circuit in the songbird biases motor output to avoid vocal errors. Proc. Natl Acad. Sci. USA 106, 12518–12523 (2009).
Ganguly, K. & Carmena, J. M. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 7, e1000153 (2009).
Hwang, E. J., Bailey, P. M. & Andersen, R. A. Volitional control of neural activity relies on the natural motor repertoire. Curr. Biol. 23, 353–361 (2013).
Jeanne, J. M., Sharpee, T. O. & Gentner, T. Q. Associative learning enhances population coding by inverting interneuronal correlation patterns. Neuron 78, 352–363 (2013).
Law, A. J., Rivlis, G. & Schieber, M. H. Rapid acquisition of novel interface control by small ensembles of arbitrarily selected primary motor cortex neurons. J. Neurophysiol. 112, 1528–1548 (2014).
Sadtler, P. T. et al. Neural constraints on learning. Nature 512, 423–426 (2014).
Poort, J. et al. Learning enhances sensory and multiple non-sensory representations in primary visual cortex. Neuron 86, 1478–1490 (2015).
Athalye, V. R., Santos, F. J., Carmena, J. M. & Costa, R. M. Evidence for a neural law of effect. Science 359, 1024–1029 (2018).
Golub, M. D. et al. Learning by neural reassociation. Nat. Neurosci. 21, 1546–1726 (2018).
Vyas, S. et al. Neural population dynamics underlying motor learning transfer. Neuron 97, 1177–1186 (2018).
Perich, M. G., Gallego, J. A. & Miller, L. E. A neural population mechanism for rapid learning. Neuron 100, 964–976 (2018).
Oby, E. R. et al. New neural activity patterns emerge with long-term learning. Proc. Natl Acad. Sci. USA 116, 15210–15215 (2019).
Arieli, A., Sterkin, A., Grinvald, A. & Aertsen, A. Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science 273, 1868–1871 (1996).
Churchland, M. M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).
Gu, Y. et al. Perceptual learning reduces interneuronal correlations in macaque visual cortex. Neuron 71, 750–761 (2011).
Ecker, A. S. et al. State dependence of noise correlations in macaque primary visual cortex. Neuron 82, 235–248 (2014).
Lin, I.-C., Okun, M., Carandini, M. & Harris, K. D. The nature of shared cortical variability. Neuron 87, 644–656 (2015).
Shenoy, K. V. & Carmena, J. M. Combining decoder design and neural adaptation in brain–machine interfaces. Neuron 84, 665–680 (2014).
Moxon, K. A. & Foffani, G. Brain–machine interfaces beyond neuroprosthetics. Neuron 86, 55–67 (2015).
Golub, M. D., Chase, S. M., Batista, A. P. & Yu, B. M. Brain-computer interfaces for dissecting cognitive processes underlying sensorimotor control. Curr. Opin. Neurobiol. 37, 53–58 (2016).
Orsborn, A. L. & Pesaran, B. Parsing learning in networks using brain–machine interfaces. Curr. Opin. Neurobiol. 46, 76–83 (2017).
Hennig, J. A. et al. Constraints on neural redundancy. Elife 7, e36774 (2018).
Kaufman, M. T., Churchland, M. M., Ryu, S. I. & Shenoy, K. V. Cortical activity in the null space: permitting preparation without movement. Nat. Neurosci. 17, 440–448 (2014).
Stavisky, S. D., Kao, J. C., Ryu, S. I. & Shenoy, K. V. Motor cortical visuomotor feedback activity is initially isolated from downstream targets in output-null neural state space dimensions. Neuron 95, 195–208 (2017).
Shadmehr, R. & Holcomb, H. H. Neural correlates of motor memory consolidation. Science 277, 821–825 (1997).
Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Constantinidis, C. & Klingberg, T. The neuroscience of working memory capacity and training. Nat. Rev. Neurosci. 17, 438–439 (2016).
Kaufman, M. T. et al. The largest response component in the motor cortex reflects movement timing but not movement type. eNeuro 3, ENEURO.0085-16.2016 (2016).
Russo, A. A. et al. Motor cortex embeds muscle-like commands in an untangled population response. Neuron 97, 953–966 (2018).
Osu, R. et al. Short- and long-term changes in joint co-contraction associated with motor learning as revealed from surface EMG. J. Neurophysiol. 88, 991–1004 (2002).
Athalye, V. R., Ganguly, K., Costa, R. M. & Carmena, J. M. Emergence of coordinated neural dynamics underlies neuroprosthetic learning and skillful control. Neuron 93, 955–970 (2017).
Rabinowitz, N. C., Goris, R. L., Cohen, M. & Simoncelli, E. P. Attention stabilizes the shared gain of V4 populations. Elife 4, e08998 (2015).
Ni, A. M., Ruff, D. A., Alberts, J. J., Symmonds, J. & Cohen, M. R. Learning and attention reveal a general relationship between population activity and behavior. Science 359, 463–465 (2018).
Santhanam, G. et al. Factor-analysis methods for higher-performance neural prostheses. J. Neurophysiol. 102, 1315–1330 (2009).
Harvey, C. D., Coen, P. & Tank, D. W. Choice-specific sequences in parietal cortex during a virtual-navigation decision task. Nature 484, 62–68 (2012).
Williamson, R. C. et al. Scaling properties of dimensionality reduction for neural populations and network models. PLoS Comput. Biol. 12, e1005141 (2016).
Huang, C. et al. Circuit models of low-dimensional shared variability in cortical networks. Neuron 101, 337–348 (2019).
Yu, B. M. et al. Gaussian-process factor analysis for low-dimensional single-trial analysis of neural population activity. J. Neurophysiol. 102, 614–635 (2009).
Acknowledgements
The authors thank J. Graves, B. Cowley, M. Smith and E. Yttri for helpful discussions. This work was supported by the Richard King Mellon Presidential Fellowship (to J.A.H.), the Carnegie Prize Fellowship in Mind and Brain Sciences (to J.A.H.), NIH R01 HD071686 (to A.P.B., B.M.Y. and S.M.C.), NSF NCS BCS1533672 (to S.M.C., B.M.Y. and A.P.B.), NSF CAREER award IOS1553252 (to S.M.C.), NIH CRCNS R01 NS105318 (to B.M.Y. and A.P.B.), NSF NCS BCS1734916 (to B.M.Y.), NIH CRCNS R01 MH118929 (to B.M.Y.), NIH R01 EB026953 (to B.M.Y.) and Simons Foundation 543065 (to B.M.Y.).
Author information
Authors and Affiliations
Contributions
J.A.H. performed the analyses. M.D.G., P.T.S., K.M.Q., A.P.B., S.M.C. and B.M.Y. designed the animal experiments. E.R.O., L.A.B. and P.T.S. performed the animal experiments. E.R.O., S.I.R. and E.C.T.-K. performed the animal surgeries. J.A.H., A.P.B., S.M.C. and B.M.Y. wrote the manuscript. All authors discussed the results and commented on the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Neuroscience thanks Karel Svoboda and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Neural engagement showed stereotyped changes relative to experimental events, in multiple example sessions.
Same conventions as Fig. 2d. Note that in contrast to other figures (for example, Fig. 5c), here neural engagement is shown across trials to all eight instructed targets, where trials to different targets were interleaved. As a result, each time course shown here includes variability due to the target-specific differences in neural engagement during learning (for example, see Fig. 5c). Position along the horizontal axis indicates clock time (see legend indicating ‘5 minutes’), so that pauses in the experiment are more visible. All sessions are plotted with the same time scale, and trial indices are marked for reference.
Extended Data Fig. 2 Changes in neural engagement during BCI control could not be explained by hand movements.
a-c. During the BCI experiments, we recorded the hand speed of two animals (monkey J, shown in panel a; and monkey L, shown in panel b), for the hand contralateral to the recording array (the other hand was restrained). Monkey N’s hand speed was not recorded because his hand was restrained in a tube, and the reflection of the light on the tube made his hand difficult to track. We also recorded the hand speed of monkey G (shown in panel c), who performed a center-out arm reaching task (as shown in Fig. 2i-j). This allowed us to compare hand speeds across both types of experiments. We found that the arm movements during the BCI task (panels a and b) were substantially smaller than during the center-out arm reaching task. Black line indicates median across trials to all sessions, while shading indicates median ± 25th percentile (a, n = 25 sessions; b, n = 10 sessions; c, n = 3 sessions). d-e. Even if animals showed little to no arm movements (as shown in panels a and b), might it be the case that the increase in neural engagement at the start of block 2 (Fig. 4c) can be explained by animals moving their hands more than they did on previous trials? We found no substantial increase in hand speed at the start of Block 2 for either monkey. Black line indicates median across sessions, while shading indicates median ± 25th percentile (d, n = 25 sessions; e, n = 10 sessions). Thus, the increase in neural engagement we observe at the start of Block 2 cannot be explained by animals suddenly moving their hands more than during Block 1.
Extended Data Fig. 3 Trials with elevated levels of neural engagement also showed increased pupil size.
a-c. In Fig. 2g, we related neural engagement and pupil size by first averaging the pupil size across time points within a trial. To further explore this relationship, here we consider the time course of pupil size within a trial. Trial-averaged pupil sizes are shown for three example sessions after grouping trials separately based on whether neural engagement during the control interval of each trial during Block 2 was above- (dark gray) or below- (light gray) the median across trials during Block 2. Vertical dashed line indicates the time within each trial when the cursor was released (300 ms; see Methods), that is, the beginning of the control interval. Shading indicates mean ± SE across trials (a, n = 456 trials; b, n = 296 trials; c, n = 202 trials). Within each example session, the time course of pupil size was similar for trials with above- versus below-average levels of neural engagement, but with a larger overall pupil size on trials with above-average neural engagement. d. Prior to computing the correlations between neural engagement and pupil size shown in Fig. 2g (and in the previous panel), we first smoothed the trial-by-trial time courses of pupil size and neural engagement with a 30-trial boxcar filter, similar to previous work correlating population activity and pupil size12. Here we show that neural engagement and pupil size were typically positively correlated even without smoothing. Without smoothing, the median Pearson’s correlation across sessions was ρ = 0.12 (bootstrapped 95% C.I. [0.09, 0.18], n = 44 sessions). Same conventions as Fig. 2g. e. Although the recording of pupil size began part way into Block 1 due to experimental constraints, we computed the trial-by-trial correlation between pupil size and neural engagement during Block 1 for the 13 sessions with a sufficient number of trials (all from monkey J). The median Pearson’s correlation during these sessions was ρ = 0.67 (bootstrapped 95% C.I. [0.41, 0.79], n = 13 sessions). Thus, a positive correlation between neural engagement and pupil size was also present before learning. Same conventions as Fig. 2g.
Extended Data Fig. 4 Changes in neural engagement corresponded to nearly all neural units increasing or decreasing their activity together.
We wanted to understand how changes in neural engagement were represented by the activity of individual units. For each target, a neural engagement axis was defined in 10-dimensional factor space. We used the q × 10 loading matrix from factor analysis (see Methods) to define the neural engagement axis in the q-dimensional population activity space of the q recorded units. For example, if there were 90 units, the neural engagement axis would have 90 coefficients, describing how changes in neural engagement for a given target would be represented by the activity of each of the 90 units. For each target, we computed the percentage of units whose coefficients had the same sign (for whichever sign was in the majority, so that percentages could never be below 50%). Shown in black is the distribution of these percentages across the neural engagement axes for all targets across all sessions (bootstrapped 95% C.I. [97.6%, 97.7%], n = 368 axes (one per target)). This relationship means that an increase in neural engagement corresponds to an increase in the firing rate of most units (by an amount that is unit- and target-dependent). For reference, in gray, is the distribution after sampling random dimensions in factor space, and computing the corresponding effects on individual neural units (bootstrapped 95% C.I. [59.7%, 62.5%], n = 368 random axes). Triangles depict the medians of the ‘data’ and ‘chance’ distributions, which were significantly different (p < 10−10, paired, two-sided sign test, n = 368 axes).
Extended Data Fig. 5 Increased neural engagement corresponded with increased baseline firing rate, modulation depth, and spiking variance in single units.
To understand the relationship between neural engagement and the firing properties of individual units, for each experiment we grouped trials during Block 1 based on whether they had above- versus below-average levels of neural engagement (similar to Extended Data Fig. 3). a-b. For each individual unit from all sessions, we fit two cosine tuning models to each unit’s z-scored spike counts: one model was fit to the average spike counts on trials with above-average levels of neural engagement (‘high engagement trials’), while the other model fit to the spike counts on trials with below-average levels of neural engagement (‘low engagement trials’). The cosine model was of the form \(y=b+m\cos (\theta -{\theta }_{pref})\), where y is the unit’s expected firing rate on a trial to target θ, b is the unit’s baseline (average) firing rate, m is the unit’s modulation depth, and θpref is the unit’s preferred direction. We estimated b, m, and θpref using linear regression. Each dot corresponds to one unit. For most units, both the baseline firing rate (b; panel a) and the modulation depth (m; panel b) were higher on high engagement trials than on low engagement trials (in both cases: p < 10−10, paired, two-sided sign test, n = 4074 units). c-d. For each session, we fit a factor analysis (FA) model to the z-scored spike counts of all units during low engagement trials, and then fit a separate FA model to the z-scored spike counts during high engagement trials. Each model had the same form as equation (1), resulting in parameter estimates of L and Ψ. The estimated private variance of unit i is given by Ψii, while the shared variance is given by \({(L{L}^{\top })}_{ii}\), where the ii subscript indicates the ith diagonal element. Each dot corresponds to one unit. We found that both the private variance (panel c) and shared variance (panel d) of most units was higher on high engagement trials than on low engagement trials (in both cases: p < 10−10, paired, two-sided sign test, n = 4074 units). This result is expected from Extended Data Fig. 4 because the sum of a unit’s shared and private variances equals its spike count variance. Because a unit’s spike count variance tends to increase with its mean spike count, a higher firing rate will typically correspond with a higher shared and/or private variance.
Extended Data Fig. 6 Increased neural engagement during arm movements predicted faster hand speeds towards most targets.
a. For the experiments involving arm movements (see Methods), we visualized the average neural population activity (circles) and neural engagement axes (orange arrows) during baseline reaches to each of eight targets. Same conventions as Fig. 3d. ‘Target directions’ panel is a legend depicting the color corresponding to each target direction. b. We also visualized the monkey’s average hand velocity during reaches to each target (circles), similar to Fig. 3e. Unlike during BCI control, we do not know the causal relationship between neural population activity and hand velocity. To understand how changes in neural engagement related to hand velocity, we used linear regression to predict the monkey’s hand velocity during baseline reaches at each 50 ms timestep during the movement epoch of every trial, using the neural population activity recorded 100 ms prior. Cross-validated r2 for the x- and y- components of hand velocity were 67% and 77%, respectively. The linear regression model (\(\widehat{M}\)) allowed us to estimate how increases in the neural engagement related to the monkey’s average hand velocity towards each target (orange dashed arrows), and to intermediate target directions (gray dashed lines). In this session, an increase in neural engagement predicted an increase in the monkey’s hand speed towards all but the 135∘ target. This suggests that differences in the neural engagement axes across targets may have behavioral relevance. c. We repeated the procedure in panel b during the other two arm movement sessions. Across sessions, increased neural engagement during arm movements predicted faster hand speeds towards most targets.
Extended Data Fig. 7 New BCI mappings induced a variety of relationships between neural engagement and cursor velocity, across targets and sessions.
Same conventions as Fig. 3f, for multiple example sessions (all with the same scale). ‘Target directions’ panel is a legend depicting the color corresponding to each target direction.
Extended Data Fig. 8 Changes in neural engagement and performance per monkey.
a-b. Changes in neural engagement (a) and performance (b) during Block 2, averaged across T + and T − targets for each monkey separately (J: nT+ = 119, nT− = 81 targets; N: nT+ = 50, nT− = 38; L: nT+ = 51, nT− = 29). Same conventions as Fig. 5c (a) and Fig. 5e (b). c-d. Difference in learning speed between T + and T − targets is robust to amount of smoothing (c) and how the peak performance was determined (d). We found the number of trials at which performance for a given target reached x% of its maximum, after first smoothing the performance for each target with a k-trial boxcar filter (see Methods), where in Fig. 5f, k = 8 and x = 100. Here we sweep the amount of smoothing (k; panel c) while holding x = 100 constant, and sweep the threshold percentage (x; panel d) while holding k = 8 constant. Across all monkeys, the blue line was always below the red line, indicating that our result that T + targets reached peak performance more quickly than T − targets was robust to different parameter settings.
Extended Data Fig. 9 Non-uniform task performance did not predict how quickly different targets reached peak performance.
a-b. In BCI tasks, performance across targets is often non-uniform. Can differences in the animal’s pre-learning performance across targets predict how quickly different targets were learned? We defined pre-learning performance in two ways: using the performance under the intuitive BCI mapping during Block 1 (panel a), and using the predicted initial performance under the new BCI mapping (panel b). For the latter quantity, we projected the trial-averaged neural activity for each target during Block 1 onto the new BCI mapping. For each definition of pre-learning performance, we divided all targets from each monkey into two groups based on whether pre-learning performance was above (green, ‘easier’) or below (gray, ‘harder’) the median performance level across all targets. We then found the number of trials needed for each group of targets to reach peak performance during Block 2, similar to Fig. 5f. The median number of trials needed to reach peak performance was not different for targets that were initially harder (gray triangle) versus easier (green triangle) during Block 1 using the intuitive BCI mapping (p = 0.91, two-sided Wilcoxon rank-sum test, n1 = 184 and n2 = 184 targets; panel a). Nor was there a difference in the median number of trials needed to reach peak performance for the targets that were predicted to be initially harder (gray triangle) versus easier (green triangle) under the new BCI mapping (p = 0.06, two-sided Wilcoxon rank-sum test, n1 = 184 and n2 = 184 targets; panel b).
Extended Data Fig. 10 Neural engagement axes were largely unchanged after learning.
Distribution of the angle (‘data’, in black) between the neural engagement axis identified for each target during Block 1 (‘before learning’) vs. during the last 50 trials of Block 2 (‘after learning’). To identify neural engagement axes during the last 50 trials of Block 2, we used the same procedure as used during Block 1 (that is, the procedure used in the main text; see Methods), but applied to the last 50 trials of Block 2. ‘Chance’ (in gray) indicates the distribution of the angle between random directions in ten-dimensional space. Triangles depict the medians of the ‘data’ and ‘chance’ distributions, which were significantly different (p < 10−10, two-sided Wilcoxon rank-sum test, n1 = 368 (data) and n2 = 50, 000 (chance) axes).
Supplementary information
Rights and permissions
About this article
Cite this article
Hennig, J.A., Oby, E.R., Golub, M.D. et al. Learning is shaped by abrupt changes in neural engagement. Nat Neurosci 24, 727–736 (2021). https://doi.org/10.1038/s41593-021-00822-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41593-021-00822-8
This article is cited by
-
Preparatory activity and the expansive null-space
Nature Reviews Neuroscience (2024)
-
Motor cortex retains and reorients neural dynamics during motor imagery
Nature Human Behaviour (2024)
-
Effects of transcranial direct current stimulation over human motor cortex on cognitive-motor and sensory-motor functions
Scientific Reports (2023)
-
Selective modulation of cortical population dynamics during neuroprosthetic skill learning
Scientific Reports (2022)
-
Mice alternate between discrete strategies during perceptual decision-making
Nature Neuroscience (2022)
