Covert skill learning in a cortical-basal ganglia circuit

Abstract

We learn complex skills such as speech and dance through a gradual process of trial and error. Cortical-basal ganglia circuits have an important yet unresolved function in this trial-and-error skill learning1; influential ‘actor–critic’ models propose that basal ganglia circuits generate a variety of behaviours during training and learn to implement the successful behaviours in their repertoire2,3. Here we show that the anterior forebrain pathway (AFP), a cortical-basal ganglia circuit4, contributes to skill learning even when it does not contribute to such ‘exploratory’ variation in behavioural performance during training. Blocking the output of the AFP while training Bengalese finches to modify their songs prevented the gradual improvement that normally occurs in this complex skill during training. However, unblocking the output of the AFP after training caused an immediate transition from naive performance to excellent performance, indicating that the AFP covertly gained the ability to implement learned skill performance without contributing to skill practice. In contrast, inactivating the output nucleus of the AFP during training completely prevented learning, indicating that learning requires activity within the AFP during training. Our results suggest a revised model of skill learning: basal ganglia circuits can monitor the consequences of behavioural variation produced by other brain regions and then direct those brain regions to implement more successful behaviours. The ability of the AFP to identify successful performances generated by other brain regions indicates that basal ganglia circuits receive a detailed efference copy of premotor activity in those regions. The capacity of the AFP to implement successful performances that were initially produced by other brain regions indicates precise functional connections between basal ganglia circuits and the motor regions that directly control performance.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Trial-and-error learning in adult birdsong.
Figure 2: Infusing APV into RA reduced song variability reversibly without distorting song structure.
Figure 3: Infusing APV into RA prevents the expression but not the acquisition of learning.
Figure 4: Inactivating LMAN during training prevents both the expression and the acquisition of learning.

References

  1. 1

    Hikosaka, O., Nakamura, K., Sakai, K. & Nakahara, H. Central mechanisms of motor skill learning. Curr. Opin. Neurobiol. 12, 217–222 (2002)

    CAS  Article  Google Scholar 

  2. 2

    Houk, J. C., Adams, J. L. & Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G. ) 249–270 (MIT Press, 1995)

    Google Scholar 

  3. 3

    Suri, R. E. & Schultz, W. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91, 871–890 (1999)

    CAS  Article  Google Scholar 

  4. 4

    Mooney, R. Neural mechanisms for learned birdsong. Learn. Mem. 16, 655–669 (2009)

    Article  Google Scholar 

  5. 5

    Tumer, E. C. & Brainard, M. S. Performance variability enables adaptive plasticity of ‘crystallized’ adult birdsong. Nature 450, 1240–1244 (2007)

    ADS  CAS  Article  Google Scholar 

  6. 6

    Charlesworth, J. D., Tumer, E. C., Warren, T. L. & Brainard, M. S. Learning the microstructure of successful behavior. Nature Neurosci. 14, 373–380 (2011)

    CAS  Article  Google Scholar 

  7. 7

    Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998)

    Google Scholar 

  8. 8

    Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)

    CAS  Article  Google Scholar 

  9. 9

    Reynolds, J. N., Hyland, B. I. & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001)

    ADS  CAS  Article  Google Scholar 

  10. 10

    Fee, M. S. & Goldberg, J. H. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198, 152–170 (2011)

    CAS  Article  Google Scholar 

  11. 11

    Fiete, I. R., Fee, M. S. & Seung, H. S. Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J. Neurophysiol. 98, 2038–2057 (2007)

    Article  Google Scholar 

  12. 12

    Doya, K. & Sejnowski, T. in The New Cognitive Neurosciences (ed. Gazzaniga, M. ) 469–482 (MIT Press, 2000)

    Google Scholar 

  13. 13

    Andalman, A. S. & Fee, M. S. A basal ganglia-forebrain circuit in the songbird biases motor output to avoid vocal errors. Proc. Natl Acad. Sci. USA 106, 12518–12523 (2009)

    ADS  CAS  Article  Google Scholar 

  14. 14

    Warren, T. L., Tumer, E. C., Charlesworth, J. D. & Brainard, M. S. Mechanisms and time course of vocal learning and consolidation in the adult songbird. J. Neurophysiol. 106, 1806–1821 (2011)

    CAS  Article  Google Scholar 

  15. 15

    Olveczky, B. P., Andalman, A. S. & Fee, M. S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3, e153 (2005)

    Article  Google Scholar 

  16. 16

    Hampton, C. M., Sakata, J. T. & Brainard, M. S. An avian basal ganglia-forebrain circuit contributes differentially to syllable versus sequence variability of adult Bengalese finch song. J. Neurophysiol. 101, 3235–3245 (2009)

    Article  Google Scholar 

  17. 17

    Krupa, D. J., Thompson, J. K. & Thompson, R. F. Localization of a memory trace in the mammalian brain. Science 260, 989–991 (1993)

    ADS  CAS  Article  Google Scholar 

  18. 18

    Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W. & O’Reilly, R. C. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neurosci. 10, 126–131 (2007)

    CAS  Article  Google Scholar 

  19. 19

    Balleine, B. W. & Ostlund, S. B. Still at the choice-point: action selection and initiation in instrumental conditioning. Ann. NY Acad. Sci. 1104, 147–171 (2007)

    ADS  Article  Google Scholar 

  20. 20

    Crapse, T. B. & Sommer, M. A. Corollary discharge across the animal kingdom. Nature Rev. Neurosci. 9, 587–600 (2008)

    CAS  Article  Google Scholar 

  21. 21

    Olveczky, B. P., Otchy, T. M., Goldberg, J. H., Aronov, D. & Fee, M. S. Changes in the neural control of a complex motor sequence during learning. J. Neurophysiol. 106, 386–397 (2011)

    Article  Google Scholar 

  22. 22

    Sober, S. J., Wohlgemuth, M. J. & Brainard, M. S. Central contributions to acoustic variation in birdsong. J. Neurosci. 28, 10370–10379 (2008)

    CAS  Article  Google Scholar 

  23. 23

    Leonardo, A. Experimental test of the birdsong error-correction model. Proc. Natl Acad. Sci. USA 101, 16935–16940 (2004)

    ADS  CAS  Article  Google Scholar 

  24. 24

    Vates, G. E., Vicario, D. S. & Nottebohm, F. Reafferent thalamo-‘cortical’ loops in the song system of oscine songbirds. J. Comp. Neurol. 380, 275–290 (1997)

    CAS  Article  Google Scholar 

  25. 25

    Goldberg, J. H. & Fee, M. S. A cortical motor nucleus drives the basal ganglia-recipient thalamus in singing birds. Nature Neurosci. 15, 620–627 (2012)

    CAS  Article  Google Scholar 

  26. 26

    Redgrave, P. & Gurney, K. The short-latency dopamine signal: a role in discovering novel actions? Nature Rev. Neurosci. 7, 967–975 (2006)

    CAS  Article  Google Scholar 

  27. 27

    Turner, R. S. & Desmurget, M. Basal ganglia contributions to motor control: a vigorous tutor. Curr. Opin. Neurobiol. 20, 704–716 (2010)

    CAS  Article  Google Scholar 

  28. 28

    Frank, M. J. Computational models of motivated action selection in corticostriatal circuits. Curr. Opin. Neurobiol. 21, 381–386 (2011)

    CAS  Article  Google Scholar 

Download references

Acknowledgements

We thank L. Frank, A. Doupe, M. Stryker and D. Mets for discussion and comments on the manuscript. This work was supported by National Institutes of Health grant NIDCD R01 and National Institute of Mental Health grant P50. J.D.C. and T.L.W. were supported by National Science Foundation graduate fellowships.

Author information

Affiliations

Authors

Contributions

J.D.C., T.L.W. and M.S.B. designed the experiments. J.D.C. performed the experiments with APV in RA, and T.L.W. performed the experiments with LMAN inactivations. J.D.C. analysed the data. J.D.C. prepared the manuscript, with input from the other authors.

Corresponding author

Correspondence to Jonathan D. Charlesworth.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Information

This file contains Supplementary Figures 1-4. (PDF 732 kb)

PowerPoint slides

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Charlesworth, J., Warren, T. & Brainard, M. Covert skill learning in a cortical-basal ganglia circuit. Nature 486, 251–255 (2012). https://doi.org/10.1038/nature11078

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing