Letter | Published:

Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories

Abstract

Learning to perform a behavioural procedure as a well-ingrained habit requires extensive repetition of the behavioural sequence, and learning not to perform such behaviours is notoriously difficult. Yet regaining a habit can occur quickly, with even one or a few exposures to cues previously triggering the behaviour1,2,3. To identify neural mechanisms that might underlie such learning dynamics, we made long-term recordings from multiple neurons in the sensorimotor striatum, a basal ganglia structure implicated in habit formation4,5,6,7,8, in rats successively trained on a reward-based procedural task, given extinction training and then given reacquisition training. The spike activity of striatal output neurons, nodal points in cortico-basal ganglia circuits, changed markedly across multiple dimensions during each of these phases of learning. First, new patterns of task-related ensemble firing successively formed, reversed and then re-emerged. Second, task-irrelevant firing was suppressed, then rebounded, and then was suppressed again. These changing spike activity patterns were highly correlated with changes in behavioural performance. We propose that these changes in task representation in cortico-basal ganglia circuits represent neural equivalents of the explore–exploit behaviour characteristic of habit learning.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1

    James, W. The Principles of Psychology 104–127 (Dover, New York, 1890)

  2. 2

    Dickinson, A. Actions and habits: the development of behavioural autonomy. Phil. Trans. R. Soc. Lond. B 308, 67–78 (1985)

  3. 3

    Pavlov, I. P. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex (ed. and transl. Anrep, G. V.) (Oxford Univ. Press, London, 1927)

  4. 4

    Packard, M. G. & Knowlton, B. J. Learning and memory functions of the basal ganglia. Annu. Rev. Neurosci. 25, 563–593 (2002)

  5. 5

    Graybiel, A. M. The basal ganglia and chunking of action repertoires. Neurobiol. Learn. Mem. 70, 119–136 (1998)

  6. 6

    Poldrack, R. A. et al. Interactive memory systems in the human brain. Nature 414, 546–550 (2001)

  7. 7

    Jog, M., Kubota, Y., Connolly, C. I., Hillegaart, V. & Graybiel, A. M. Building neural representations of habits. Science 286, 1745–1749 (1999)

  8. 8

    Yin, H. H., Knowlton, B. J. & Balleine, B. W. Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning. Eur. J. Neurosci. 19, 181–189 (2004)

  9. 9

    Olveczky, B. P., Andalman, A. S. & Fee, M. S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3, e153 (2005)

  10. 10

    Kao, M. H., Doupe, A. J. & Brainard, M. S. Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 433, 638–643 (2005)

  11. 11

    Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, Cambridge, Massachusetts, 1998)

  12. 12

    Wilson, C. J. in The Synaptic Organization of the Brain (ed. Shepherd, G. M.) 361–413 (Oxford Univ. Press, New York, 2004)

  13. 13

    Doya, K. & Sejnowski, T. J. in Advances in Neural Information Processing Systems Vol. 7 (eds Tesauro, G., Touretzky, D. S. & Leen, T. K.) 101–108 (MIT Press, Cambridge, Massachusetts, 1995)

  14. 14

    Doya, K. & Sejnowski, T. in The New Cognitive Neurosciences (ed. Gazzaniga, M. S.) 469–482 (MIT Press, Cambridge, Massachusetts, 2000)

  15. 15

    Bouton, M. E. Context and behavioural processes in extinction. Learn. Mem. 11, 485–494 (2004)

  16. 16

    Routtenberg, A. & Kim, H.-J. in Cholinergic–Monoaminergic Interactions in the Brain (ed. Butcher, L. L.) 305–331 (Academic, New York, 1978)

  17. 17

    Myers, K. M. & Davis, M. Behavioral and neural analysis of extinction. Neuron 36, 567–584 (2002)

  18. 18

    Gurney, K., Prescott, T. J. & Redgrave, P. A computational model of action selection in the basal ganglia. I. A new functional anatomy. Biol. Cybern. 84, 401–410 (2001)

  19. 19

    Graybiel, A. M., Aosaki, T., Flaherty, A. W. & Kimura, M. The basal ganglia and adaptive motor control. Science 265, 1826–1831 (1994)

  20. 20

    Djurfeldt, M., Ekeberg, Ö. & Graybiel, A. M. Cortex-basal ganglia interaction and attractor states. Neurocomputing 38–40, 573–579 (2001)

  21. 21

    Frank, M. J., Loughry, B. & O'Reilly, R. C. Interactions between frontal cortex and basal ganglia in working memory: a computational model. Cogn. Affect. Behav. Neurosci. 1, 137–160 (2001)

  22. 22

    Houk, J. C. & Wise, S. P. Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb. Cortex 5, 95–110 (1995)

  23. 23

    Doya, K. Metalearning and neuromodulation. Neural Netw. 15, 495–506 (2002)

  24. 24

    Montague, P. R., Hyman, S. E. & Cohen, J. D. Computational roles for dopamine in behavioural control. Nature 431, 760–767 (2004)

  25. 25

    Tanaka, S. C. et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nature Neurosci. 7, 887–893 (2004)

  26. 26

    Reynolds, J. N. J., Hyland, B. I. & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001)

  27. 27

    Mink, J. W. The basal ganglia: focused selection and inhibition of competing motor programs. Prog. Neurobiol. 50, 381–425 (1996)

  28. 28

    McClure, S. M., Berns, G. S. & Montague, P. R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339–346 (2003)

  29. 29

    Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J., Davis, J. & Beiser, D.) 215–232 (MIT Press, Cambridge, Massachusetts, 1995)

  30. 30

    Dickinson, A. & Balleine, B. W. Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1–18 (1994)

Download references

Acknowledgements

We thank H. F. Hall, P. A. Harlan and C. Thorn for their help. This work was funded by the National Institutes of Health and the Office of Naval Research.

Author information

Correspondence to Ann M. Graybiel.

Ethics declarations

Competing interests

Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.

Supplementary information

Supplementary Methods

Includes references for Supplementary Figure Legends, Supplementary Table 1, (Summary of training schedules and recording yield for the seven rats included in the study) and Supplementary Table 2 (Training sessions included in each learning stage). (DOC 185 kb)

Supplementary Figure 1

Recording sites and unit classification methods. (PDF 1021 kb)

Supplementary Figure 2

Averaged per-neuron spike frequency plots constructed as in Fig. 2. (PDF 2384 kb)

Supplementary Figure 3

Population firing patterns of striatal neurons of the projection neuron class exhibiting task-related activity near the start of the trial runs and those exhibiting task-related activity near the end of the runs. (PDF 854 kb)

Supplementary Figure 4

Extinction-induced reversal of reconfigured striatal activity is not due to increase in running times during extinction. (PDF 1146 kb)

Supplementary Figure 5

Partial and full extinction training procedures yielded similar behavioral and neural changes. (PDF 1947 kb)

Supplementary Figure 6

Multiple changes in striatal projection neuron activity during successive acquisition, extinction and reacquisition training. (PDF 759 kb)

Supplementary Figure 7

Prediction of behavioral accuracy by a composite neural score combining normalized per-neuron firing rate, proportion of different task-responsive sub-populations, and per-phasic response spike proportion. (PDF 640 kb)

Supplementary Figure 8

Late development of restructuring of striatal spike patterns in rats that learned the task at slow rates. (PDF 1065 kb)

Supplementary Figure 9

Averaged per-neuron spike frequency plots for trials with correct and incorrect behavioral responses. (PDF 2309 kb)

Rights and permissions

Reprints and Permissions

About this article

Further reading

Figure 1: T-maze task and behavioural learning.
Figure 2: Plasticity in spike activity patterns of striatal projection neurons.
Figure 3: Multiple changes in projection neuron activity in the sensorimotor striatum during acquisition, extinction and reacquisition training.
Figure 4: Striatal neural activity predictive of behavioural performance.

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.