Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

The dissociable effects of punishment and reward on motor learning

Nature Neuroscience volume 18, pages 597–602 (2015)Cite this article

Subjects

Abstract

A common assumption regarding error-based motor learning (motor adaptation) in humans is that its underlying mechanism is automatic and insensitive to reward- or punishment-based feedback. Contrary to this hypothesis, we show in a double dissociation that the two have independent effects on the learning and retention components of motor adaptation. Negative feedback, whether graded or binary, accelerated learning. While it was not necessary for the negative feedback to be coupled to monetary loss, it had to be clearly related to the actual performance on the preceding movement. Positive feedback did not speed up learning, but it increased retention of the motor memory when performance feedback was withdrawn. These findings reinforce the view that independent mechanisms underpin learning and retention in motor adaptation, reject the assumption that motor adaptation is independent of motivational feedback, and raise new questions regarding the neural basis of negative and positive motivational feedback in motor learning.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Experimental design.
Figure 2: Punishment led to faster learning, while reward caused greater retention during motor adaptation.
Figure 3: Replication of the double dissociation between reward and punishment using a one-target design.
Figure 4: Direct negative feedback related to poor performance is the critical factor that increases learning rate.

Similar content being viewed by others

References

  1. Daw, N.D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616 (2002).

    PubMed  Google Scholar 

  2. Frank, M.J. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J. Cogn. Neurosci. 17, 51–72 (2005).

    PubMed  Google Scholar 

  3. den Ouden, H.E. et al. Dissociable effects of dopamine and serotonin on reversal learning. Neuron 80, 1090–1100 (2013).

    CAS  PubMed  Google Scholar 

  4. Robinson, O.J., Frank, M.J., Sahakian, B.J. & Cools, R. Dissociable responses to punishment in distinct striatal regions during reversal learning. Neuroimage 51, 1459–1467 (2010).

    PubMed  Google Scholar 

  5. Frank, M.J., Seeberger, L.C. & O'Reilly, R. C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940–1943 (2004).

    CAS  PubMed  Google Scholar 

  6. Wächter, T., Lungu, O.V., Liu, T., Willingham, D.T. & Ashe, J. Differential effect of reward and punishment on procedural learning. J. Neurosci. 29, 436–443 (2009).

    PubMed  PubMed Central  Google Scholar 

  7. Abe, M. et al. Reward improves long-term retention of a motor memory through induction of offline memory gains. Curr. Biol. 21, 557–562 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  8. Wolpert, D.M., Diedrichsen, J. & Flanagan, J.R. Principles of sensorimotor learning. Nat. Rev. Neurosci. 12, 739–751 (2011).

    CAS  PubMed  Google Scholar 

  9. Doya, K. Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 10, 732–739 (2000).

    CAS  PubMed  Google Scholar 

  10. Mazzoni, P. & Krakauer, J.W. An implicit plan overrides an explicit strategy during visuomotor adaptation. J. Neurosci. 26, 3642–3645 (2006).

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Shadmehr, R. & Krakauer, J.W. A computational neuroanatomy for motor control. Exp. Brain Res. 185, 359–381 (2008).

    PubMed  PubMed Central  Google Scholar 

  12. Huang, V.S. & Krakauer, J.W. Robotic neurorehabilitation: a computational motor learning perspective. J. Neuroeng. Rehabil. 6, 5 (2009).

    PubMed  PubMed Central  Google Scholar 

  13. Krakauer, J.W. Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr. Opin. Neurol. 19, 84–90 (2006).

    PubMed  Google Scholar 

  14. Galea, J.M., Vazquez, A., Pasricha, N., Orban de Xivry, J.J. & Celnik, P. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb. Cortex 21, 1761–1770 (2011).

    PubMed  Google Scholar 

  15. Rabe, K. et al. Adaptation to visuomotor rotation and force field perturbation is correlated to different brain areas in patients with cerebellar degeneration. J. Neurophysiol. 101, 1961–1971 (2009).

    CAS  PubMed  Google Scholar 

  16. Moulton, E.A. et al. Aversion-related circuitry in the cerebellum: responses to noxious heat and unpleasant images. J. Neurosci. 31, 3795–3804 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  17. Ernst, M. et al. Decision-making in a risk-taking task: a PET study. Neuropsychopharmacology 26, 682–691 (2002).

    PubMed  Google Scholar 

  18. McCormick, D.A. & Thompson, R.F. Cerebellum: essential involvement in the classically conditioned eyelid response. Science 223, 296–299 (1984).

    CAS  PubMed  Google Scholar 

  19. Hester, R., Murphy, K., Brown, F.L. & Skilleter, A.J. Punishing an error improves learning: the influence of punishment magnitude on error-related neural activity and subsequent learning. J. Neurosci. 30, 15600–15607 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  20. Hadipour-Niktarash, A., Lee, C.K., Desmond, J.E. & Shadmehr, R. Impairment of retention but not acquisition of a visuomotor skill through time-dependent disruption of primary motor cortex. J. Neurosci. 27, 13413–13419 (2007).

    CAS  PubMed  PubMed Central  Google Scholar 

  21. Richardson, A.G. et al. Disruption of primary motor cortex before learning impairs memory of movement dynamics. J. Neurosci. 26, 12466–12470 (2006).

    CAS  PubMed  PubMed Central  Google Scholar 

  22. Beierholm, U. et al. Dopamine modulates reward-related vigor. Neuropsychopharmacology 38, 1495–1503 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  23. Niv, Y., Daw, N.D., Joel, D. & Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 191, 507–520 (2007).

    CAS  Google Scholar 

  24. Awenowicz, P.W. & Porter, L.L. Local application of dopamine inhibits pyramidal tract neuron activity in the rodent motor cortex. J. Neurophysiol. 88, 3439–3451 (2002).

    CAS  PubMed  Google Scholar 

  25. Hosp, J.A., Pekanovic, A., Rioult-Pedotti, M.S. & Luft, A.R. Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J. Neurosci. 31, 2481–2487 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Hosp, J.A. & Luft, A.R. Dopaminergic meso-cortical projections to M1: role in motor learning and motor cortex plasticity. Front. Neurol. 4, 145 (2013).

    PubMed  PubMed Central  Google Scholar 

  27. Shmuelof, L. et al. Overcoming motor “forgetting” through reinforcement of learned actions. J. Neurosci. 32, 14617–14621 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  28. Thabit, M.N. et al. Momentary reward induce changes in excitability of primary motor cortex. Clin. Neurophysiol. 122, 1764–1770 (2011).

    PubMed  Google Scholar 

  29. Krakauer, J.W. Motor learning and consolidation: the case of visuomotor rotation. Adv. Exp. Med. Biol. 629, 405–421 (2009).

    PubMed  PubMed Central  Google Scholar 

  30. Donchin, O., Francis, J.T. & Shadmehr, R. Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. J. Neurosci. 23, 9032–9045 (2003).

    CAS  PubMed  PubMed Central  Google Scholar 

  31. Thoroughman, K.A. & Shadmehr, R. Learning of action through adaptive combination of motor primitives. Nature 407, 742–747 (2000).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. Taylor, J.A. & Ivry, R.B. The role of strategies in motor learning. Ann. NY Acad. Sci. 1251, 1–12 (2012).

    PubMed  Google Scholar 

  33. Torrubia, R., Avila, C., Molto, J. & Caseras, X. The Sensitivity to Punishment and Sensitivity Reward Questionnaire (SPSRQ) as a measure of Gray's anxiety and impulsivity dimensions. Pers. Indiv. Differ. 31, 837–862 (2001).

    Google Scholar 

  34. Smith, M.A., Ghazizadeh, A. & Shadmehr, R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179 (2006).

    PubMed  PubMed Central  Google Scholar 

  35. Tseng, Y.W., Diedrichsen, J., Krakauer, J.W., Shadmehr, R. & Bastian, A.J. Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J. Neurophysiol. 98, 54–62 (2007).

    PubMed  Google Scholar 

  36. Galea, J.M., Ruge, D., Buijink, A., Bestmann, S. & Rothwell, J.C. Punishment-induced behavioral and neurophysiological variability reveals dopamine-dependent selection of kinematic movement parameters. J. Neurosci. 33, 3981–3988 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  37. Dayan, P. & Daw, N.D. Decision theory, reinforcement learning, and the brain. Cogn. Affect. Behav. Neurosci. 8, 429–453 (2008).

    PubMed  Google Scholar 

  38. Izawa, J. & Shadmehr, R. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput. Biol. 7, e1002012 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  39. Wu, H.G., Miyamoto, Y.R., Castro, L.N., Olveczky, B.P. & Smith, M.A. Temporal structure of motor variability is dynamically regulated and predicts motor learning ability. Nat. Neurosci. 17, 312–321 (2014).

    CAS  PubMed  PubMed Central  Google Scholar 

  40. Nichols, R.A. Serotonin, presynaptic 5-HT(3) receptors and synaptic plasticity in the developing cerebellum. J. Physiol. (Lond.) 589, 5019–5020 (2011).

    CAS  Google Scholar 

  41. Kahneman, D. & Tversky, A. Prospect theory – analysis of decision under risk. Econometrica 47, 263–291 (1979).

    Google Scholar 

  42. De Martino, B., Camerer, C.F. & Adolphs, R. Amygdala damage eliminates monetary loss aversion. Proc. Natl. Acad. Sci. USA 107, 3788–3792 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  43. Kahneman, D. & Tversky, A. Choices, values, and frames. Am. Psychol. 39, 341–350 (1984).

    Google Scholar 

  44. Mercer, J. Prospect theory and political science. Annu. Rev. Polit. Sci. 8, 1–21 (2005).

    Google Scholar 

  45. Dayan, E., Averbeck, B.B., Richmond, B.J. & Cohen, L.G. Stochastic reinforcement benefits skill acquisition. Learn. Mem. 21, 140–142 (2014).

    PubMed  PubMed Central  Google Scholar 

  46. Haith, A.M. & Krakauer, J.W. Model-based and model-free mechanisms of human motor learning. Adv. Exp. Med. Biol. 782, 1–21 (2013).

    PubMed  PubMed Central  Google Scholar 

  47. Orban de Xivry, J.J., Criscimagna-Hemminger, S.E. & Shadmehr, R. Contributions of the motor cortex to adaptive control of reaching depend on the perturbation schedule. Cereb. Cortex 21, 1475–1484 (2011).

    PubMed  Google Scholar 

  48. Huang, V.S., Haith, A., Mazzoni, P. & Krakauer, J.W. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787–801 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Patton, J.L. & Mussa-Ivaldi, F.A. Robot-assisted adaptive training: custom force fields for teaching movement patterns. IEEE Trans. Biomed. Eng. 51, 636–646 (2004).

    PubMed  Google Scholar 

  50. Diedrichsen, J., Hashambhoy, Y., Rane, T. & Shadmehr, R. Neural correlates of reach errors. J. Neurosci. 25, 9919–9931 (2005).

    CAS  PubMed  PubMed Central  Google Scholar 

  51. Tanaka, H., Sejnowski, T.J. & Krakauer, J.W. Adaptation to visuomotor rotation through interaction between posterior parietal and motor cortical areas. J. Neurophysiol. 102, 2921–2932 (2009).

    PubMed  PubMed Central  Google Scholar 

  52. Galea, J.M., Sami, S.A., Albert, N.B. & Miall, R.C. Secondary tasks impair adaptation to step- and gradual-visual displacements. Exp. Brain Res. 202, 473–484 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  53. Galea, J.M., Vazquez, A., Pasricha, N., de Xivry, J.J. & Celnik, P. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb. Cortex 21, 1761–1770 (2011).

    PubMed  Google Scholar 

  54. Ingram, J.N., Flanagan, J.R. & Wolpert, D.M. Context-dependent decay of motor memories during skill acquisition. Curr. Biol. 23, 1107–1112 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

This work was supported by a Birmingham Fellow research fellowship (J.M.G.). We thank P. Celnik for comments on a previous version of the manuscript.

Author information

Authors and Affiliations

  1. School of Psychology, University of Birmingham, Birmingham, UK

    Joseph M Galea

  2. Sobell Department for Motor Neuroscience and Movement Disorders, Institute of Neurology, University College London, London, UK

    Elizabeth Mallia & John Rothwell

  3. Insititute of Cognitive Neuroscience, University College London, London, UK

    Jörn Diedrichsen

Authors
  1. Joseph M Galea
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Elizabeth Mallia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. John Rothwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jörn Diedrichsen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.M.G. designed experiment 1. J.M.G. and J.D. designed experiment 2. J.M.G. and E.M. performed research. J.M.G., E.M. and J.D. analyzed data. J.M.G., E.M., J.R. and J.D. wrote the paper.

Corresponding author

Correspondence to Joseph M Galea.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Single-rate state-space model (SSM) prediction across experiment 2.

(a) Average angular reach direction (º) in Experiment 2 for random positive (blue), reward (red), punishment (black) groups. Data is first averaged over Epoch (average across 8 trials), then across participants. Dashed/solid vertical lines indicate short rest periods between blocks (<1minute). (b) Predicted angular reach direction from the SSM fitted separately to each experimental phase (separated by solid vertical lines). Although epoch data is shown, the model was estimated using trial-by-trial reach direction data. Note that washout was estimated simply for presentation purposes. Solid lines = mean, shaded areas = SEM.

Supplementary Figure 2 A fixed generalization function leads to a slower rate of learning across groups

The SSM model assumed that learning did not generalise between the 8-target positions used in experiment 2. Yet, previous work has suggested that generalisation can take place between targets that are 45° apart. Therefore, we assumed a fixed generalisation function of: 0° = 1, 45° = 0.44, >90° = 0.2. This generalisation function was then used to estimate the learning rate for each phase/participant as before. As expected, the inclusion of generalisation led to substantially lower estimates of the learning rate. However similarly to the original model, punishment (black) led to significantly faster learning (SSM parameter B) compared to either the reward (p = 0.032; red) or random positive (p = 0.030; blue) group (F(2,41) = 4.65, p = 0.016). Mean ± SEM. *p<0.05.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1 and 2 and Supplementary Table 1 (PDF 561 kb)

Supplementary Methods Checklist (PDF 386 kb)

Rights and permissions

Reprints and Permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Galea, J., Mallia, E., Rothwell, J. et al. The dissociable effects of punishment and reward on motor learning. Nat Neurosci 18, 597–602 (2015). https://doi.org/10.1038/nn.3956

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3956

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing