A fundamental goal of motor learning is to establish the neural patterns that produce a desired behavioral outcome. It remains unclear how and when the nervous system solves this 'credit assignment' problem. Using neuroprosthetic learning, in which we could control the causal relationship between neurons and behavior, we found that sleep-dependent processing was required for credit assignment and the establishment of task-related functional connectivity reflecting the casual neuron–behavior relationship. Notably, we observed a strong link between the microstructure of sleep reactivations and credit assignment, with downscaling of non-causal activity. Decoupling of spiking to slow oscillations using optogenetic methods eliminated rescaling. Thus, our results suggest that coordinated firing during sleep is essential for establishing sparse activation patterns that reflect the causal neuron-behavior relationship.
Subscribe to Journal
Get full journal access for 1 year
only $4.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Yin, H.H. et al. Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill. Nat. Neurosci. 12, 333–341 (2009).
Dayan, E. & Cohen, L.G. Neuroplasticity subserving motor skill learning. Neuron 72, 443–454 (2011).
Tumer, E.C. & Brainard, M.S. Performance variability enables adaptive plasticity of 'crystallized' adult birdsong. Nature 450, 1240–1244 (2007).
Shmuelof, L. & Krakauer, J.W. Are we ready for a natural history of motor learning? Neuron 72, 469–476 (2011).
Peters, A.J., Chen, S.X. & Komiyama, T. Emergence of reproducible spatiotemporal activity during motor learning. Nature 510, 263–267 (2014).
Ganguly, K. & Carmena, J.M. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 7, e1000153 (2009).
Huber, D. et al. Multiple dynamic representations in the motor cortex during sensorimotor learning. Nature 484, 473–478 (2012).
Ganguly, K., Dimitrov, D.F., Wallis, J.D. & Carmena, J.M. Reversible large-scale modification of cortical networks during neuroprosthetic control. Nat. Neurosci. 14, 662–667 (2011).
Abbott, L.F., DePasquale, B. & Memmesheimer, R.M. Building functional networks of spiking model neurons. Nat. Neurosci. 19, 350–355 (2016).
Lee, D., Seo, H. & Jung, M.W. Neural basis of reinforcement learning and decision making. Annu. Rev. Neurosci. 35, 287–308 (2012).
Clancy, K.B., Koralek, A.C., Costa, R.M., Feldman, D.E. & Carmena, J.M. Volitional modulation of optically recorded calcium signals during neuroprosthetic learning. Nat. Neurosci. 17, 807–809 (2014).
Tononi, G. & Cirelli, C. Sleep and the price of plasticity: from synaptic and cellular homeostasis to memory consolidation and integration. Neuron 81, 12–34 (2014).
Diekelmann, S. & Born, J. The memory function of sleep. Nat. Rev. Neurosci. 11, 114–126 (2010).
Genzel, L., Kroes, M.C., Dresler, M. & Battaglia, F.P. Light sleep versus slow wave sleep in memory consolidation: a question of global versus local processes? Trends Neurosci. 37, 10–19 (2014).
Cramer, S.C. et al. Motor cortex activation is preserved in patients with chronic hemiplegic stroke. Ann. Neurol. 52, 607–616 (2002).
Marshall, L. & Born, J. The contribution of sleep to hippocampus-dependent memory consolidation. Trends Cogn. Sci. 11, 442–450 (2007).
Wilson, M.A. & McNaughton, B.L. Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679 (1994).
Nere, A., Hashmi, A., Cirelli, C. & Tononi, G. Sleep-dependent synaptic down-selection (I): modeling the benefits of sleep on memory consolidation and integration. Front. Neurol. 4, 143 (2013).
Jarosiewicz, B. et al. Functional network reorganization during learning in a brain-computer interface paradigm. Proc. Natl. Acad. Sci. USA 105, 19486–19491 (2008).
Koralek, A.C., Jin, X., Long, J.D. II, Costa, R.M. & Carmena, J.M. Corticostriatal plasticity is necessary for learning intentional neuroprosthetic skills. Nature 483, 331–335 (2012).
Taylor, D.M., Tillery, S.I. & Schwartz, A.B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002).
Moritz, C.T., Perlmutter, S.I. & Fetz, E.E. Direct control of paralyzed muscles by cortical neurons. Nature 456, 639–642 (2008).
Gulati, T., Ramanathan, D.S., Wong, C.C. & Ganguly, K. Reactivation of emergent task-related ensembles during slow-wave sleep after neuroprosthetic learning. Nat. Neurosci. 17, 1107–1113 (2014).
Gulati, T. et al. Robust neuroprosthetic control from the stroke perilesional cortex. J. Neurosci. 35, 8653–8661 (2015).
Fetz, E.E. Volitional control of neural activity: implications for brain-computer interfaces. J. Physiol. (Lond.) 579, 571–579 (2007).
Koralek, A.C., Costa, R.M. & Carmena, J.M. Temporally precise cell-specific coherence develops in corticostriatal networks during learning. Neuron 79, 865–872 (2013).
Orsborn, A.L. et al. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron 82, 1380–1393 (2014).
Mitchell, J.F., Sundberg, K.A. & Reynolds, J.H. Spatial attention decorrelates intrinsic activity fluctuations in macaque area V4. Neuron 63, 879–888 (2009).
Watson, B.O., Levenstein, D., Greene, J.P., Gelinas, J.N. & Buzsáki, G. Network homeostasis and state dynamics of neocortical sleep. Neuron 90, 839–852 (2016).
Peyrache, A., Khamassi, M., Benchenane, K., Wiener, S.I. & Battaglia, F.P. Replay of rule-learning related neural patterns in the prefrontal cortex during sleep. Nat. Neurosci. 12, 919–926 (2009).
Ramanathan, D.S., Gulati, T. & Ganguly, K. Sleep-dependent reactivation of ensembles in motor cortex promotes skill consolidation. PLoS Biol. 13, e1002263 (2015).
Lansink, C.S., Goltstein, P.M., Lankelma, J.V., McNaughton, B.L. & Pennartz, C.M. Hippocampus leads ventral striatum in replay of place-reward information. PLoS Biol. 7, e1000173 (2009).
de Lavilléon, G., Lacroix, M.M., Rondi-Reig, L. & Benchenane, K. Explicit memory creation during sleep demonstrates a causal role of place cells in navigation. Nat. Neurosci. 18, 493–495 (2015).
Singer, A.C. & Frank, L.M. Rewarded outcomes enhance reactivation of experience in the hippocampus. Neuron 64, 910–921 (2009).
Churchland, M.M. et al. Stimulus onset quenches neural variability: a widespread cortical phenomenon. Nat. Neurosci. 13, 369–378 (2010).
Song, W. & Giszter, S.F. Adaptation to a cortex-controlled robot attached at the pelvis and engaged during locomotion in rats. J. Neurosci. 31, 3110–3128 (2011).
Miyamoto, D. et al. Top-down cortical input during NREM sleep consolidates perceptual memory. Science 352, 1315–1318 (2016).
Chuong, A.S. et al. Noninvasive optical inhibition with a red-shifted microbial rhodopsin. Nat. Neurosci. 17, 1123–1129 (2014).
Steriade, M., Nuñez, A. & Amzica, F. A novel slow (<1 Hz) oscillation of neocortical neurons in vivo: depolarizing and hyperpolarizing components. J. Neurosci. 13, 3252–3265 (1993).
Yang, G. et al. Sleep promotes branch-specific formation of dendritic spines after learning. Science 344, 1173–1178 (2014).
de Vivo, L. et al. Ultrastructural evidence for synaptic scaling across the wake/sleep cycle. Science 355, 507–510 (2017).
Maret, S., Faraguna, U., Nelson, A.B., Cirelli, C. & Tononi, G. Sleep and waking modulate spine turnover in the adolescent mouse cortex. Nat. Neurosci. 14, 1418–1420 (2011).
Gupta, A.S., van der Meer, M.A., Touretzky, D.S. & Redish, A.D. Hippocampal replay is not a simple function of experience. Neuron 65, 695–705 (2010).
O'Doherty, J.P., Cockburn, J. & Pauli, W.M. Learning, reward, and decision making. Annu. Rev. Psychol. 68, 73–100 (2017).
Schultz, W. Behavioral theories and the neurophysiology of reward. Annu. Rev. Psychol. 57, 87–115 (2006).
Ishii, S., Yoshida, W. & Yoshimoto, J. Control of exploitation-exploration meta-parameter in reinforcement learning. Neural Netw. 15, 665–687 (2002).
Wallstrom, G., Liebner, J. & Kass, R.E. An implementation of Bayesian adaptive regression splines (BARS) in C with S and R Wrappers. J. Stat. Softw. 26, 1–21 (2008).
Mitra, P. & Bokil, H. Observed Brain Dynamics (Oxford University Press, 2008).
This work was supported by awards from the Department of Veterans Affairs, Veterans Health Administration (VA Merit: 1I01RX001640 to K.G., VA CDA 1IK2BX003308 to D.S.R.); the National Institute of Neurological Disorders and Stroke (1K99NS097620 to T.G. and 5K02NS093014 to K.G.); the American Heart/Stroke Association (15POST25510020 to T.G.); the Burroughs Wellcome Fund (1009855 to K.G.); and start-up funds from the SFVAMC, NCIRE and UCSF Department of Neurology (to K.G.).
The authors declare no competing financial interests.
Integrated supplementary information
Supplementary Figure 1 Modulations of TRD neurons are not significantly high during random time points in sleep.
a, Mean modulation of all TRD and TRI neurons at randomly picked times during Sleeppost. For this analysis, snippets of activity (50 ms window of spiking activity that was then binned at 5 ms resolution) were randomly sampled from all the reactivation times. b, Scatter plots showing the modulation depth for all the individual TRD and TRI (mean in solid line ± s.e.m. in box; unpaired t test t121 = 0.69, P = 0.49).
Supplementary Figure 2 Depth modulation during sleep reactivations predicts rescaling of task–related firing rate during BMI2.
MDreactivation from Sleeppost (in Fig 2c) are compared to changes in modulation from BMI1 to BMI2 for both TRD and TRI neurons (linear regression R2 = 0.17, P < 10-5).
a, Comparison of the total number of 100 ms optogenetic stimulation periods for OPTOUP and OPTODOWN experiments (mean in solid line ± s.e.m. in box; color convention same as Fig 5; unpaired t test t17 = 0.99, P = 0.33). b, Comparison of the proportion of total time that the LED was turned on for the respective Sleeppost during the OPTOUP and OPTODOWN experiments (mean in solid line ± s.e.m. in box; color convention same as Fig 5; unpaired t test t17 = 2.07, P = 0.054).
Comparison of the total sleep durations for Sleeppre and Sleeppost during the OPTOUP, OPTODOWN and OPTOOFF experiments (mean in solid line ± s.e.m. in box, one-way ANOVA, F5,48 = 0.92, P = 0.47; post hoc t test shows no significant difference between any pairwise comparison).
Examples of the first principal components from two separate experiments. Notably, TRD and TRI neurons both had non-zero weights (TRD enclosed in red box).
About this article
Cite this article
Gulati, T., Guo, L., Ramanathan, D. et al. Neural reactivations during sleep determine network credit assignment. Nat Neurosci 20, 1277–1284 (2017). https://doi.org/10.1038/nn.4601
Epidural cerebellar stimulation drives widespread neural synchrony in the intact and stroke perilesional cortex
Journal of NeuroEngineering and Rehabilitation (2021)
Net decrease in spine-surface GluA1-containing AMPA receptors after post-learning sleep in the adult mouse cortex
Nature Communications (2021)
Nature Biotechnology (2021)
Nature Neuroscience (2020)
Large-scale changes in cortical dynamics triggered by repetitive somatosensory electrical stimulation
Journal of NeuroEngineering and Rehabilitation (2019)