Article | Published:

Temporal structure of motor variability is dynamically regulated and predicts motor learning ability

Nature Neuroscience volume 17, pages 312321 (2014) | Download Citation

Abstract

Individual differences in motor learning ability are widely acknowledged, yet little is known about the factors that underlie them. Here we explore whether movement-to-movement variability in motor output, a ubiquitous if often unwanted characteristic of motor performance, predicts motor learning ability. Surprisingly, we found that higher levels of task-relevant motor variability predicted faster learning both across individuals and across tasks in two different paradigms, one relying on reward-based learning to shape specific arm movement trajectories and the other relying on error-based learning to adapt movements in novel physical environments. We proceeded to show that training can reshape the temporal structure of motor variability, aligning it with the trained task to improve learning. These results provide experimental support for the importance of action exploration, a key idea from reinforcement learning theory, showing that motor variability facilitates motor learning in humans and that our nervous systems actively regulate it to improve learning.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from $8.99

All prices are NET prices.

References

  1. 1.

    , & A central source of movement variability. Neuron 52, 1085–1096 (2006).

  2. 2.

    , & Sources of signal-dependent noise during isometric force production. J. Neurophysiol. 88, 1533–1544 (2002).

  3. 3.

    , , , & Motor-output variability: a theory for the accuracy of rapid motor acts. Psychol. Rev. 47, 415–451 (1979).

  4. 4.

    , & Neuronal variability: noise or part of the signal? Nat. Rev. Neurosci. 6, 389–397 (2005).

  5. 5.

    , & A sensory source for motor variation. Nature 437, 412–416 (2005).

  6. 6.

    & Signal-dependent noise determines motor planning. Nature 394, 780–784 (1998).

  7. 7.

    , & Role of uncertainty in sensorimotor control. Phil. Trans. R. Soc. Lond. B 357, 1137–1145 (2002).

  8. 8.

    & The uncontrolled manifold concept: identifying control variables for a functional task. Exp. Brain Res. 126, 289–306 (1999).

  9. 9.

    Optimality principles in sensorimotor control. Nat. Neurosci. 7, 907–915 (2004).

  10. 10.

    & Optimal feedback control as a theory of motor coordination. Nat. Neurosci. 5, 1226–1235 (2002).

  11. 11.

    , & Dissociating variability and effort as determinants of coordination. PLoS Comput. Biol. 5, e1000345 (2009).

  12. 12.

    , & Covert skill learning in a cortical-basal ganglia circuit. Nature 486, 251–255 (2012).

  13. 13.

    , & Vocal experimentation in the juvenile songbird requires a basal ganglia circuit. PLoS Biol. 3, e153 (2005).

  14. 14.

    , & Contributions of an avian basal ganglia-forebrain circuit to real-time modulation of song. Nature 433, 638–643 (2005).

  15. 15.

    & Performance variability enables adaptive plasticity of 'crystallized' adult birdsong. Nature 450, 1240–1244 (2007).

  16. 16.

    & Introduction to Reinforcement Learning (MIT Press, 1998).

  17. 17.

    , & Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996).

  18. 18.

    & Variation of bar-press duration: where do new responses come from? Behav. Processes 72, 215–223 (2006).

  19. 19.

    & The modulation of operant variation by the probability, magnitude, and delay of reinforcement. Learn. Motiv. 42, 221–236 (2011).

  20. 20.

    , , & Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787–801 (2011).

  21. 21.

    , , , & Persistence of motor adaptation during constrained, multi-joint, arm movements. J. Neurophysiol. 84, 853–862 (2000).

  22. 22.

    , , , & Primitives for motor adaptation reflect correlated neural tuning to position and velocity. Neuron 64, 575–589 (2009).

  23. 23.

    , & Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179 (2006).

  24. 24.

    & Reduction in learning rates associated with anterograde interference results from interactions between different timescales in motor adaptation. PLoS Comput. Biol. 6, e1000893 (2010).

  25. 25.

    & Long-term retention explained by a model of short-term learning in the adaptive control of reaching. J. Neurophysiol. 100, 2948–2955 (2008).

  26. 26.

    , & Interference between velocity-dependent and position-dependent force-fields indicates that tasks depending on different kinematic parameters compete for motor working memory. Exp. Brain Res. 163, 400–405 (2005).

  27. 27.

    , & Dissociating timing and coordination as functions of the cerebellum. J. Neurosci. 27, 6291–6301 (2007).

  28. 28.

    , , & Linear hypergeneralization of learned dynamics across movement speeds reveals anisotropic, gain-encoding primitives for motor adaptation. J. Neurophysiol. 105, 45–59 (2011).

  29. 29.

    & Central representation of time during motor learning. Proc. Natl. Acad. Sci. USA 96, 11625–11630 (1999).

  30. 30.

    & A basal ganglia-forebrain circuit in the songbird biases motor output to avoid vocal errors. Proc. Natl. Acad. Sci. USA 106, 12518–12523 (2009).

  31. 31.

    , , & Mechanisms and time course of vocal learning and consolidation in the adult songbird. J. Neurophysiol. 106, 1806–1821 (2011).

  32. 32.

    , , , , & The basal ganglia is necessary for learning spectral, but not temporal, features of birdsong. Neuron 80, 494–506 (2013).

  33. 33.

    , & Learning to learn: environmental consistency modulates motor adaptation rates. Proc. Ann. Symp.: Advances in Comp. Motor Control 7 (2008).

  34. 34.

    , , & Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neurosci. 12, 1062–1068 (2009).

  35. 35.

    , , & The rate of visuomotor adaptation correlates with cerebellar white-matter microstructure. Hum. Brain Mapp. 30, 4048–4053 (2009).

  36. 36.

    et al. Structural and functional bases for individual differences in motor learning. Hum. Brain Mapp. 32, 494–508 (2011).

  37. 37.

    , , & Human memory strength is predicted by theta-frequency phase-locking of single neurons. Nature 464, 903–907 (2010).

  38. 38.

    & Prediction of learning rate from the hippocampal electroencephalogram. Science 200, 1298–1300 (1978).

  39. 39.

    et al. Model-free characterization of brain functional networks for motor sequence learning using fMRI. Neuroimage 39, 1950–1958 (2008).

  40. 40.

    , , & Dynamics of the vocal imitation process: how a zebra finch learns its song. Science 291, 2564–2569 (2001).

  41. 41.

    & Social context–induced song variation affects female behavior and gene expression. PLoS Biol. 6, e62 (2008).

  42. 42.

    & A novel reinforcement model of birdsong vocalization learning. Adv. Neural Inf. Process. Syst. 8, 101–108 (1995).

  43. 43.

    , & Model of birdsong learning based on gradient estimation by dynamic perturbation of neural conductances. J. Neurophysiol. 98, 2038–2057 (2007).

  44. 44.

    , & Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).

  45. 45.

    & Learning of action through adaptive combination of motor primitives. Nature 407, 742–747 (2000).

  46. 46.

    & Shared internal models for feedforward and feedback control. J. Neurosci. 28, 10663–10673 (2008).

  47. 47.

    , & Central contributions to acoustic variation in birdsong. J. Neurosci. 28, 10370–10379 (2008).

  48. 48.

    , & Trial-to-trial variability of single cells in motor cortices is dynamically modified during visuomotor adaptation. J. Neurosci. 29, 15053–15062 (2009).

  49. 49.

    , , , & Modulation of saccadic eye movements by predicted reward outcome. Exp. Brain Res. 142, 284–291 (2002).

  50. 50.

    , , , & Changes in the neural control of a complex motor sequence during learning. J. Neurophysiol. 106, 386–397 (2011).

  51. 51.

    Principal Component Analysis (John Wiley & Sons Ltd, 2005).

Download references

Acknowledgements

We thank G. Sing, J. Brayanov, A. Hadjiosif and L. Clark for help with the analyses and helpful discussions. We thank G. Gabriel and S. Orozco for help with experiments. This work was supported in part by the McKnight Scholar Award, a Sloan Research Fellowship, a grant from the National Institute on Aging (R01 AG041878) to M.A.S. and a McKnight Scholar Award and a grant from the National Institute of Neurological Disorders and Stroke (R01 NS066408) to B.P.Ö.

Author information

Author notes

    • Howard G Wu
    •  & Yohsuke R Miyamoto

    These authors contributed equally to this work.

Affiliations

  1. School of Engineering and Applied Sciences, Harvard University, Cambridge, Massachusetts, USA.

    • Howard G Wu
    • , Yohsuke R Miyamoto
    • , Luis Nicolas Gonzalez Castro
    •  & Maurice A Smith
  2. Department of Organismic and Evolutionary Biology, Harvard University, Cambridge, Massachusetts, USA.

    • Bence P Ölveczky
  3. Center for Brain Science, Harvard University, Cambridge, Massachusetts, USA.

    • Bence P Ölveczky
    •  & Maurice A Smith

Authors

  1. Search for Howard G Wu in:

  2. Search for Yohsuke R Miyamoto in:

  3. Search for Luis Nicolas Gonzalez Castro in:

  4. Search for Bence P Ölveczky in:

  5. Search for Maurice A Smith in:

Contributions

Y.R.M., B.P.Ö. and M.A.S. designed the reward-based learning experiments. H.G.W. and M.A.S. designed the error-based learning experiments. H.G.W., L.N.G.C. and M.A.S. designed the variability reshaping experiment. H.G.W., Y.R.M. and M.A.S. analyzed the data. Y.R.M., H.G.W., B.P.Ö. and M.A.S. wrote the paper.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Maurice A Smith.

Integrated supplementary information

Supplementary information

PDF files

  1. 1.

    Supplementary Text and Figures

    Supplementary Figures 1–11

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/nn.3616