Abstract
Learning to perform a behavioural procedure as a well-ingrained habit requires extensive repetition of the behavioural sequence, and learning not to perform such behaviours is notoriously difficult. Yet regaining a habit can occur quickly, with even one or a few exposures to cues previously triggering the behaviour1,2,3. To identify neural mechanisms that might underlie such learning dynamics, we made long-term recordings from multiple neurons in the sensorimotor striatum, a basal ganglia structure implicated in habit formation4,5,6,7,8, in rats successively trained on a reward-based procedural task, given extinction training and then given reacquisition training. The spike activity of striatal output neurons, nodal points in cortico-basal ganglia circuits, changed markedly across multiple dimensions during each of these phases of learning. First, new patterns of task-related ensemble firing successively formed, reversed and then re-emerged. Second, task-irrelevant firing was suppressed, then rebounded, and then was suppressed again. These changing spike activity patterns were highly correlated with changes in behavioural performance. We propose that these changes in task representation in cortico-basal ganglia circuits represent neural equivalents of the explore–exploit behaviour characteristic of habit learning.
This is a preview of subscription content, access via your institution
Relevant articles
Open Access articles citing this article.
-
Computational insights on asymmetrical $$D_{1}$$ and $$D_{2}$$ receptor-mediated chunking: implications for OCD and Schizophrenia
Cognitive Neurodynamics Open Access 12 January 2023
-
Sensory representations in the striatum provide a temporal reference for learning and executing motor habits
Nature Communications Open Access 09 September 2019
Access options
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout
References
James, W. The Principles of Psychology 104–127 (Dover, New York, 1890)
Dickinson, A. Actions and habits: the development of behavioural autonomy. Phil. Trans. R. Soc. Lond. B 308, 67–78 (1985)
Pavlov, I. P. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex (ed. and transl. Anrep, G. V.) (Oxford Univ. Press, London, 1927)
Packard, M. G. & Knowlton, B. J. Learning and memory functions of the basal ganglia. Annu. Rev. Neurosci. 25, 563–593 (2002)
Graybiel, A. M. The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 70, 119–136 (1998)
Poldrack, R. A. et al. Interactive memory systems in the human brain. Nature 414, 546–550 (2001)
Jog, M., Kubota, Y., Connolly, C. I., Hillegaart, V. & Graybiel, A. M. Building neural representations of habits. Science 286, 1745–1749 (1999)
Yin, H. H., Knowlton, B. J. & Balleine, B. W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004)
Olveczky, B. P., Andalman, A. S. & Fee, M. S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3, e153 (2005)
Kao, M. H., Doupe, A. J. & Brainard, M. S. Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 433, 638–643 (2005)
Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, Cambridge, Massachusetts, 1998)
Wilson, C. J. in The Synaptic Organization of the Brain (ed. Shepherd, G. M.) 361–413 (Oxford Univ. Press, New York, 2004)
Doya, K. & Sejnowski, T. J. in Advances in Neural Information Processing Systems Vol. 7 (eds Tesauro, G., Touretzky, D. S. & Leen, T. K.) 101–108 (MIT Press, Cambridge, Massachusetts, 1995)
Doya, K. & Sejnowski, T. in The New Cognitive Neurosciences (ed. Gazzaniga, M. S.) 469–482 (MIT Press, Cambridge, Massachusetts, 2000)
Bouton, M. E. Context and behavioural processes in extinction. Learn. Mem. 11, 485–494 (2004)
Routtenberg, A. & Kim, H.-J. in Cholinergic–Monoaminergic Interactions in the Brain (ed. Butcher, L. L.) 305–331 (Academic, New York, 1978)
Myers, K. M. & Davis, M. Behavioral and neural analysis of extinction. Neuron 36, 567–584 (2002)
Gurney, K., Prescott, T. J. & Redgrave, P. A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol. Cybern. 84, 401–410 (2001)
Graybiel, A. M., Aosaki, T., Flaherty, A. W. & Kimura, M. The basal ganglia and adaptive motor control. Science 265, 1826–1831 (1994)
Djurfeldt, M., Ekeberg, Ö. & Graybiel, A. M. Cortex-basal ganglia interaction and attractor states. Neurocomputing 38–40, 573–579 (2001)
Frank, M. J., Loughry, B. & O'Reilly, R. C. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn. Affect. Behav. Neurosci. 1, 137–160 (2001)
Houk, J. C. & Wise, S. P. Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb. Cortex 5, 95–110 (1995)
Doya, K. Metalearning and neuromodulation. Neural Netw. 15, 495–506 (2002)
Montague, P. R., Hyman, S. E. & Cohen, J. D. Computational roles for dopamine in behavioural control. Nature 431, 760–767 (2004)
Tanaka, S. C. et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neurosci. 7, 887–893 (2004)
Reynolds, J. N. J., Hyland, B. I. & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001)
Mink, J. W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381–425 (1996)
McClure, S. M., Berns, G. S. & Montague, P. R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339–346 (2003)
Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J., Davis, J. & Beiser, D.) 215–232 (MIT Press, Cambridge, Massachusetts, 1995)
Dickinson, A. & Balleine, B. W. Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1–18 (1994)
Acknowledgements
We thank H. F. Hall, P. A. Harlan and C. Thorn for their help. This work was funded by the National Institutes of Health and the Office of Naval Research.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.
Supplementary information
Supplementary Methods
Includes references for Supplementary Figure Legends, Supplementary Table 1, (Summary of training schedules and recording yield for the seven rats included in the study) and Supplementary Table 2 (Training sessions included in each learning stage). (DOC 185 kb)
Supplementary Figure 1
Recording sites and unit classification methods. (PDF 1021 kb)
Supplementary Figure 2
Averaged per-neuron spike frequency plots constructed as in Fig. 2. (PDF 2384 kb)
Supplementary Figure 3
Population firing patterns of striatal neurons of the projection neuron class exhibiting task-related activity near the start of the trial runs and those exhibiting task-related activity near the end of the runs. (PDF 854 kb)
Supplementary Figure 4
Extinction-induced reversal of reconfigured striatal activity is not due to increase in running times during extinction. (PDF 1146 kb)
Supplementary Figure 5
Partial and full extinction training procedures yielded similar behavioral and neural changes. (PDF 1947 kb)
Supplementary Figure 6
Multiple changes in striatal projection neuron activity during successive acquisition, extinction and reacquisition training. (PDF 759 kb)
Supplementary Figure 7
Prediction of behavioral accuracy by a composite neural score combining normalized per-neuron firing rate, proportion of different task-responsive sub-populations, and per-phasic response spike proportion. (PDF 640 kb)
Supplementary Figure 8
Late development of restructuring of striatal spike patterns in rats that learned the task at slow rates. (PDF 1065 kb)
Supplementary Figure 9
Averaged per-neuron spike frequency plots for trials with correct and incorrect behavioral responses. (PDF 2309 kb)
Rights and permissions
About this article
Cite this article
Barnes, T., Kubota, Y., Hu, D. et al. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161 (2005). https://doi.org/10.1038/nature04053
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1038/nature04053
This article is cited by
-
Knowledge generalization and the costs of multitasking
Nature Reviews Neuroscience (2023)
-
Computational insights on asymmetrical $$D_{1}$$ and $$D_{2}$$ receptor-mediated chunking: implications for OCD and Schizophrenia
Cognitive Neurodynamics (2023)
-
Optogenetic inhibition of indirect pathway neurons in the dorsomedial striatum reduces excessive grooming in Sapap3-knockout mice
Neuropsychopharmacology (2022)
-
Opponent control of behavior by dorsomedial striatal pathways depends on task demands and internal state
Nature Neuroscience (2022)
-
Formalizing planning and information search in naturalistic decision-making
Nature Neuroscience (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
