Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Letter
  • Published:

Covert skill learning in a cortical-basal ganglia circuit

Nature volume 486, pages 251–255 (2012)Cite this article

Subjects

Abstract

We learn complex skills such as speech and dance through a gradual process of trial and error. Cortical-basal ganglia circuits have an important yet unresolved function in this trial-and-error skill learning1; influential ‘actor–critic’ models propose that basal ganglia circuits generate a variety of behaviours during training and learn to implement the successful behaviours in their repertoire2,3. Here we show that the anterior forebrain pathway (AFP), a cortical-basal ganglia circuit4, contributes to skill learning even when it does not contribute to such ‘exploratory’ variation in behavioural performance during training. Blocking the output of the AFP while training Bengalese finches to modify their songs prevented the gradual improvement that normally occurs in this complex skill during training. However, unblocking the output of the AFP after training caused an immediate transition from naive performance to excellent performance, indicating that the AFP covertly gained the ability to implement learned skill performance without contributing to skill practice. In contrast, inactivating the output nucleus of the AFP during training completely prevented learning, indicating that learning requires activity within the AFP during training. Our results suggest a revised model of skill learning: basal ganglia circuits can monitor the consequences of behavioural variation produced by other brain regions and then direct those brain regions to implement more successful behaviours. The ability of the AFP to identify successful performances generated by other brain regions indicates that basal ganglia circuits receive a detailed efference copy of premotor activity in those regions. The capacity of the AFP to implement successful performances that were initially produced by other brain regions indicates precise functional connections between basal ganglia circuits and the motor regions that directly control performance.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Trial-and-error learning in adult birdsong.
Figure 2: Infusing APV into RA reduced song variability reversibly without distorting song structure.
Figure 3: Infusing APV into RA prevents the expression but not the acquisition of learning.
Figure 4: Inactivating LMAN during training prevents both the expression and the acquisition of learning.

Similar content being viewed by others

References

  1. Hikosaka, O., Nakamura, K., Sakai, K. & Nakahara, H. Central mechanisms of motor skill learning. Curr. Opin. Neurobiol. 12, 217–222 (2002)

    Article  CAS  Google Scholar 

  2. Houk, J. C., Adams, J. L. & Barto, A. G. in Models of Information Processing in the Basal Ganglia (eds Houk, J. C., Davis, J. L. & Beiser, D. G. ) 249–270 (MIT Press, 1995)

    Google Scholar 

  3. Suri, R. E. & Schultz, W. A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91, 871–890 (1999)

    Article  CAS  Google Scholar 

  4. Mooney, R. Neural mechanisms for learned birdsong. Learn. Mem. 16, 655–669 (2009)

    Article  Google Scholar 

  5. Tumer, E. C. & Brainard, M. S. Performance variability enables adaptive plasticity of ‘crystallized’ adult birdsong. Nature 450, 1240–1244 (2007)

    Article  ADS  CAS  Google Scholar 

  6. Charlesworth, J. D., Tumer, E. C., Warren, T. L. & Brainard, M. S. Learning the microstructure of successful behavior. Nature Neurosci. 14, 373–380 (2011)

    Article  CAS  Google Scholar 

  7. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998)

    MATH  Google Scholar 

  8. Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997)

    Article  CAS  Google Scholar 

  9. Reynolds, J. N., Hyland, B. I. & Wickens, J. R. A cellular mechanism of reward-related learning. Nature 413, 67–70 (2001)

    Article  ADS  CAS  Google Scholar 

  10. Fee, M. S. & Goldberg, J. H. A hypothesis for basal ganglia-dependent reinforcement learning in the songbird. Neuroscience 198, 152–170 (2011)

    Article  CAS  Google Scholar 

  11. Fiete, I. R., Fee, M. S. & Seung, H. S. Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J. Neurophysiol. 98, 2038–2057 (2007)

    Article  Google Scholar 

  12. Doya, K. & Sejnowski, T. in The New Cognitive Neurosciences (ed. Gazzaniga, M. ) 469–482 (MIT Press, 2000)

    Google Scholar 

  13. Andalman, A. S. & Fee, M. S. A basal ganglia-forebrain circuit in the songbird biases motor output to avoid vocal errors. Proc. Natl Acad. Sci. USA 106, 12518–12523 (2009)

    Article  ADS  CAS  Google Scholar 

  14. Warren, T. L., Tumer, E. C., Charlesworth, J. D. & Brainard, M. S. Mechanisms and time course of vocal learning and consolidation in the adult songbird. J. Neurophysiol. 106, 1806–1821 (2011)

    Article  CAS  Google Scholar 

  15. Olveczky, B. P., Andalman, A. S. & Fee, M. S. Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3, e153 (2005)

    Article  Google Scholar 

  16. Hampton, C. M., Sakata, J. T. & Brainard, M. S. An avian basal ganglia-forebrain circuit contributes differentially to syllable versus sequence variability of adult Bengalese finch song. J. Neurophysiol. 101, 3235–3245 (2009)

    Article  Google Scholar 

  17. Krupa, D. J., Thompson, J. K. & Thompson, R. F. Localization of a memory trace in the mammalian brain. Science 260, 989–991 (1993)

    Article  ADS  CAS  Google Scholar 

  18. Atallah, H. E., Lopez-Paniagua, D., Rudy, J. W. & O’Reilly, R. C. Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nature Neurosci. 10, 126–131 (2007)

    Article  CAS  Google Scholar 

  19. Balleine, B. W. & Ostlund, S. B. Still at the choice-point: action selection and initiation in instrumental conditioning. Ann. NY Acad. Sci. 1104, 147–171 (2007)

    Article  ADS  Google Scholar 

  20. Crapse, T. B. & Sommer, M. A. Corollary discharge across the animal kingdom. Nature Rev. Neurosci. 9, 587–600 (2008)

    Article  CAS  Google Scholar 

  21. Olveczky, B. P., Otchy, T. M., Goldberg, J. H., Aronov, D. & Fee, M. S. Changes in the neural control of a complex motor sequence during learning. J. Neurophysiol. 106, 386–397 (2011)

    Article  Google Scholar 

  22. Sober, S. J., Wohlgemuth, M. J. & Brainard, M. S. Central contributions to acoustic variation in birdsong. J. Neurosci. 28, 10370–10379 (2008)

    Article  CAS  Google Scholar 

  23. Leonardo, A. Experimental test of the birdsong error-correction model. Proc. Natl Acad. Sci. USA 101, 16935–16940 (2004)

    Article  ADS  CAS  Google Scholar 

  24. Vates, G. E., Vicario, D. S. & Nottebohm, F. Reafferent thalamo-‘cortical’ loops in the song system of oscine songbirds. J. Comp. Neurol. 380, 275–290 (1997)

    Article  CAS  Google Scholar 

  25. Goldberg, J. H. & Fee, M. S. A cortical motor nucleus drives the basal ganglia-recipient thalamus in singing birds. Nature Neurosci. 15, 620–627 (2012)

    Article  CAS  Google Scholar 

  26. Redgrave, P. & Gurney, K. The short-latency dopamine signal: a role in discovering novel actions? Nature Rev. Neurosci. 7, 967–975 (2006)

    Article  CAS  Google Scholar 

  27. Turner, R. S. & Desmurget, M. Basal ganglia contributions to motor control: a vigorous tutor. Curr. Opin. Neurobiol. 20, 704–716 (2010)

    Article  CAS  Google Scholar 

  28. Frank, M. J. Computational models of motivated action selection in corticostriatal circuits. Curr. Opin. Neurobiol. 21, 381–386 (2011)

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We thank L. Frank, A. Doupe, M. Stryker and D. Mets for discussion and comments on the manuscript. This work was supported by National Institutes of Health grant NIDCD R01 and National Institute of Mental Health grant P50. J.D.C. and T.L.W. were supported by National Science Foundation graduate fellowships.

Author information

Authors and Affiliations

  1. Department of Physiology, W. M. Keck Center for Integrative Neuroscience, and the Neuroscience Graduate Program, University of California, San Francisco, 94143, California, USA

    Jonathan D. Charlesworth, Timothy L. Warren & Michael S. Brainard

Authors
  1. Jonathan D. Charlesworth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Timothy L. Warren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael S. Brainard
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.D.C., T.L.W. and M.S.B. designed the experiments. J.D.C. performed the experiments with APV in RA, and T.L.W. performed the experiments with LMAN inactivations. J.D.C. analysed the data. J.D.C. prepared the manuscript, with input from the other authors.

Corresponding author

Correspondence to Jonathan D. Charlesworth.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Information

This file contains Supplementary Figures 1-4. (PDF 732 kb)

PowerPoint slides

Rights and permissions

Reprints and permissions

About this article

Cite this article

Charlesworth, J., Warren, T. & Brainard, M. Covert skill learning in a cortical-basal ganglia circuit. Nature 486, 251–255 (2012). https://doi.org/10.1038/nature11078

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature11078

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing