Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories

Abstract

Learning to perform a behavioural procedure as a well-ingrained habit requires extensive repetition of the behavioural sequence, and learning not to perform such behaviours is notoriously difficult. Yet regaining a habit can occur quickly, with even one or a few exposures to cues previously triggering the behaviour1,2,3. To identify neural mechanisms that might underlie such learning dynamics, we made long-term recordings from multiple neurons in the sensorimotor striatum, a basal ganglia structure implicated in habit formation4,5,6,7,8, in rats successively trained on a reward-based procedural task, given extinction training and then given reacquisition training. The spike activity of striatal output neurons, nodal points in cortico-basal ganglia circuits, changed markedly across multiple dimensions during each of these phases of learning. First, new patterns of task-related ensemble firing successively formed, reversed and then re-emerged. Second, task-irrelevant firing was suppressed, then rebounded, and then was suppressed again. These changing spike activity patterns were highly correlated with changes in behavioural performance. We propose that these changes in task representation in cortico-basal ganglia circuits represent neural equivalents of the explore–exploit behaviour characteristic of habit learning.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: T-maze task and behavioural learning.
Figure 2: Plasticity in spike activity patterns of striatal projection neurons.
Figure 3: Multiple changes in projection neuron activity in the sensorimotor striatum during acquisition, extinction and reacquisition training.
Figure 4: Striatal neural activity predictive of behavioural performance.

References

  1. 1

    James, W. The Principles of Psychology 104–127 (Dover, New York, 1890)

    Google Scholar 

  2. 2

    Dickinson, A. Actions and habits: the development of behavioural autonomy. Phil. Trans. R. Soc. Lond. B 308, 67–78 (1985)

    ADS  Article  Google Scholar 

  3. 3

    Pavlov, I. P. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex (ed. and transl. Anrep, G. V.) (Oxford Univ. Press, London, 1927)

    Google Scholar 

  4. 4

    Packard, M. G. & Knowlton, B. J. Learning and memory functions of the basal ganglia. Annu. Rev. Neurosci. 25, 563–593 (2002)

    CAS  Article  Google Scholar 

  5. 5

    Graybiel, A. M. The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 70, 119–136 (1998)

    CAS  Article  Google Scholar 

  6. 6

    Poldrack, R. A. et al. Interactive memory systems in the human brain. Nature 414, 546–550 (2001)

    CAS  Article  Google Scholar 

  7. 7

    Jog, M., Kubota, Y., Connolly, C. I., Hillegaart, V. & Graybiel, A. M. Building neural representations of habits. Science 286, 1745–1749 (1999)

    CAS  Article  Google Scholar 

  8. 8

    Yin, H. H., Knowlton, B. J. & Balleine, B. W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004)

    Article  Google Scholar 

  9. 9

    Olveczky, B. P., Andalman, A. S. & Fee, M. S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3, e153 (2005)

    Article  Google Scholar 

  10. 10

    Kao, M. H., Doupe, A. J. & Brainard, M. S. Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 433, 638–643 (2005)

    ADS  CAS  Article  Google Scholar 

  11. 11

    Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, Cambridge, Massachusetts, 1998)

    Google Scholar 

  12. 12

    Wilson, C. J. in The Synaptic Organization of the Brain (ed. Shepherd, G. M.) 361–413 (Oxford Univ. Press, New York, 2004)

    Google Scholar 

  13. 13

    Doya, K. & Sejnowski, T. J. in Advances in Neural Information Processing Systems Vol. 7 (eds Tesauro, G., Touretzky, D. S. & Leen, T. K.) 101–108 (MIT Press, Cambridge, Massachusetts, 1995)

    Google Scholar 

  14. 14

    Doya, K. & Sejnowski, T. in The New Cognitive Neurosciences (ed. Gazzaniga, M. S.) 469–482 (MIT Press, Cambridge, Massachusetts, 2000)

    Google Scholar 

  15. 15

    Bouton, M. E. Context and behavioural processes in extinction. Learn. Mem. 11, 485–494 (2004)

    Article  Google Scholar 

  16. 16

    Routtenberg, A. & Kim, H.-J. in Cholinergic–Monoaminergic Interactions in the Brain (ed. Butcher, L. L.) 305–331 (Academic, New York, 1978)

    Google Scholar 

  17. 17

    Myers, K. M. & Davis, M. Behavioral and neural analysis of extinction. Neuron 36, 567–584 (2002)

    CAS  Article  Google Scholar 

  18. 18

    Gurney, K., Prescott, T. J. & Redgrave, P. A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol. Cybern. 84, 401–410 (2001)

    CAS  Article  Google Scholar 

  19. 19

    Graybiel, A. M., Aosaki, T., Flaherty, A. W. & Kimura, M. The basal ganglia and adaptive motor control. Science 265, 1826–1831 (1994)

    ADS  CAS  Article  Google Scholar 

  20. 20

    Djurfeldt, M., Ekeberg, Ö. & Graybiel, A. M. Cortex-basal ganglia interaction and attractor states. Neurocomputing 38–40, 573–579 (2001)

    Article  Google Scholar 

  21. 21

    Frank, M. J., Loughry, B. & O'Reilly, R. C. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn. Affect. Behav. Neurosci. 1, 137–160 (2001)

    CAS  Article  Google Scholar 

  22. 22

    Houk, J. C. & Wise, S. P. Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb. Cortex 5, 95–110 (1995)

    CAS  Article  Google Scholar 

  23. 23

    Doya, K. Metalearning and neuromodulation. Neural Netw. 15, 495–506 (2002)

    Article  Google Scholar 

  24. 24

    Montague, P. R., Hyman, S. E. & Cohen, J. D. Computational roles for dopamine in behavioural control. Nature 431, 760–767 (2004)

    ADS  CAS  Article  Google Scholar 

  25. 25

    Tanaka, S. C. et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neurosci. 7, 887–893 (2004)

    CAS  Article  Google Scholar 

  26. 26

    Reynolds, J. N. J., Hyland, B. I. & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001)

    ADS  CAS  Article  Google Scholar 

  27. 27

    Mink, J. W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381–425 (1996)

    CAS  Article  Google Scholar 

  28. 28

    McClure, S. M., Berns, G. S. & Montague, P. R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339–346 (2003)

    CAS  Article  Google Scholar 

  29. 29

    Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J., Davis, J. & Beiser, D.) 215–232 (MIT Press, Cambridge, Massachusetts, 1995)

    Google Scholar 

  30. 30

    Dickinson, A. & Balleine, B. W. Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1–18 (1994)

    Article  Google Scholar 

Download references

Acknowledgements

We thank H. F. Hall, P. A. Harlan and C. Thorn for their help. This work was funded by the National Institutes of Health and the Office of Naval Research.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ann M. Graybiel.

Ethics declarations

Competing interests

Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.

Supplementary information

Supplementary Methods

Includes references for Supplementary Figure Legends, Supplementary Table 1, (Summary of training schedules and recording yield for the seven rats included in the study) and Supplementary Table 2 (Training sessions included in each learning stage). (DOC 185 kb)

Supplementary Figure 1

Recording sites and unit classification methods. (PDF 1021 kb)

Supplementary Figure 2

Averaged per-neuron spike frequency plots constructed as in Fig. 2. (PDF 2384 kb)

Supplementary Figure 3

Population firing patterns of striatal neurons of the projection neuron class exhibiting task-related activity near the start of the trial runs and those exhibiting task-related activity near the end of the runs. (PDF 854 kb)

Supplementary Figure 4

Extinction-induced reversal of reconfigured striatal activity is not due to increase in running times during extinction. (PDF 1146 kb)

Supplementary Figure 5

Partial and full extinction training procedures yielded similar behavioral and neural changes. (PDF 1947 kb)

Supplementary Figure 6

Multiple changes in striatal projection neuron activity during successive acquisition, extinction and reacquisition training. (PDF 759 kb)

Supplementary Figure 7

Prediction of behavioral accuracy by a composite neural score combining normalized per-neuron firing rate, proportion of different task-responsive sub-populations, and per-phasic response spike proportion. (PDF 640 kb)

Supplementary Figure 8

Late development of restructuring of striatal spike patterns in rats that learned the task at slow rates. (PDF 1065 kb)

Supplementary Figure 9

Averaged per-neuron spike frequency plots for trials with correct and incorrect behavioral responses. (PDF 2309 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Barnes, T., Kubota, Y., Hu, D. et al. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161 (2005). https://doi.org/10.1038/nature04053

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.