A common assumption regarding error-based motor learning (motor adaptation) in humans is that its underlying mechanism is automatic and insensitive to reward- or punishment-based feedback. Contrary to this hypothesis, we show in a double dissociation that the two have independent effects on the learning and retention components of motor adaptation. Negative feedback, whether graded or binary, accelerated learning. While it was not necessary for the negative feedback to be coupled to monetary loss, it had to be clearly related to the actual performance on the preceding movement. Positive feedback did not speed up learning, but it increased retention of the motor memory when performance feedback was withdrawn. These findings reinforce the view that independent mechanisms underpin learning and retention in motor adaptation, reject the assumption that motor adaptation is independent of motivational feedback, and raise new questions regarding the neural basis of negative and positive motivational feedback in motor learning.
This is a preview of subscription content, access via your institution
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Rent or buy this article
Prices vary by article type
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Daw, N.D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616 (2002).
Frank, M.J. Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J. Cogn. Neurosci. 17, 51–72 (2005).
den Ouden, H.E. et al. Dissociable effects of dopamine and serotonin on reversal learning. Neuron 80, 1090–1100 (2013).
Robinson, O.J., Frank, M.J., Sahakian, B.J. & Cools, R. Dissociable responses to punishment in distinct striatal regions during reversal learning. Neuroimage 51, 1459–1467 (2010).
Frank, M.J., Seeberger, L.C. & O'Reilly, R. C. By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306, 1940–1943 (2004).
Wächter, T., Lungu, O.V., Liu, T., Willingham, D.T. & Ashe, J. Differential effect of reward and punishment on procedural learning. J. Neurosci. 29, 436–443 (2009).
Abe, M. et al. Reward improves long-term retention of a motor memory through induction of offline memory gains. Curr. Biol. 21, 557–562 (2011).
Wolpert, D.M., Diedrichsen, J. & Flanagan, J.R. Principles of sensorimotor learning. Nat. Rev. Neurosci. 12, 739–751 (2011).
Doya, K. Complementary roles of basal ganglia and cerebellum in learning and motor control. Curr. Opin. Neurobiol. 10, 732–739 (2000).
Mazzoni, P. & Krakauer, J.W. An implicit plan overrides an explicit strategy during visuomotor adaptation. J. Neurosci. 26, 3642–3645 (2006).
Shadmehr, R. & Krakauer, J.W. A computational neuroanatomy for motor control. Exp. Brain Res. 185, 359–381 (2008).
Huang, V.S. & Krakauer, J.W. Robotic neurorehabilitation: a computational motor learning perspective. J. Neuroeng. Rehabil. 6, 5 (2009).
Krakauer, J.W. Motor learning: its relevance to stroke recovery and neurorehabilitation. Curr. Opin. Neurol. 19, 84–90 (2006).
Galea, J.M., Vazquez, A., Pasricha, N., Orban de Xivry, J.J. & Celnik, P. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb. Cortex 21, 1761–1770 (2011).
Rabe, K. et al. Adaptation to visuomotor rotation and force field perturbation is correlated to different brain areas in patients with cerebellar degeneration. J. Neurophysiol. 101, 1961–1971 (2009).
Moulton, E.A. et al. Aversion-related circuitry in the cerebellum: responses to noxious heat and unpleasant images. J. Neurosci. 31, 3795–3804 (2011).
Ernst, M. et al. Decision-making in a risk-taking task: a PET study. Neuropsychopharmacology 26, 682–691 (2002).
McCormick, D.A. & Thompson, R.F. Cerebellum: essential involvement in the classically conditioned eyelid response. Science 223, 296–299 (1984).
Hester, R., Murphy, K., Brown, F.L. & Skilleter, A.J. Punishing an error improves learning: the influence of punishment magnitude on error-related neural activity and subsequent learning. J. Neurosci. 30, 15600–15607 (2010).
Hadipour-Niktarash, A., Lee, C.K., Desmond, J.E. & Shadmehr, R. Impairment of retention but not acquisition of a visuomotor skill through time-dependent disruption of primary motor cortex. J. Neurosci. 27, 13413–13419 (2007).
Richardson, A.G. et al. Disruption of primary motor cortex before learning impairs memory of movement dynamics. J. Neurosci. 26, 12466–12470 (2006).
Beierholm, U. et al. Dopamine modulates reward-related vigor. Neuropsychopharmacology 38, 1495–1503 (2013).
Niv, Y., Daw, N.D., Joel, D. & Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology (Berl.) 191, 507–520 (2007).
Awenowicz, P.W. & Porter, L.L. Local application of dopamine inhibits pyramidal tract neuron activity in the rodent motor cortex. J. Neurophysiol. 88, 3439–3451 (2002).
Hosp, J.A., Pekanovic, A., Rioult-Pedotti, M.S. & Luft, A.R. Dopaminergic projections from midbrain to primary motor cortex mediate motor skill learning. J. Neurosci. 31, 2481–2487 (2011).
Hosp, J.A. & Luft, A.R. Dopaminergic meso-cortical projections to M1: role in motor learning and motor cortex plasticity. Front. Neurol. 4, 145 (2013).
Shmuelof, L. et al. Overcoming motor “forgetting” through reinforcement of learned actions. J. Neurosci. 32, 14617–14621 (2012).
Thabit, M.N. et al. Momentary reward induce changes in excitability of primary motor cortex. Clin. Neurophysiol. 122, 1764–1770 (2011).
Krakauer, J.W. Motor learning and consolidation: the case of visuomotor rotation. Adv. Exp. Med. Biol. 629, 405–421 (2009).
Donchin, O., Francis, J.T. & Shadmehr, R. Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. J. Neurosci. 23, 9032–9045 (2003).
Thoroughman, K.A. & Shadmehr, R. Learning of action through adaptive combination of motor primitives. Nature 407, 742–747 (2000).
Taylor, J.A. & Ivry, R.B. The role of strategies in motor learning. Ann. NY Acad. Sci. 1251, 1–12 (2012).
Torrubia, R., Avila, C., Molto, J. & Caseras, X. The Sensitivity to Punishment and Sensitivity Reward Questionnaire (SPSRQ) as a measure of Gray's anxiety and impulsivity dimensions. Pers. Indiv. Differ. 31, 837–862 (2001).
Smith, M.A., Ghazizadeh, A. & Shadmehr, R. Interacting adaptive processes with different timescales underlie short-term motor learning. PLoS Biol. 4, e179 (2006).
Tseng, Y.W., Diedrichsen, J., Krakauer, J.W., Shadmehr, R. & Bastian, A.J. Sensory prediction errors drive cerebellum-dependent adaptation of reaching. J. Neurophysiol. 98, 54–62 (2007).
Galea, J.M., Ruge, D., Buijink, A., Bestmann, S. & Rothwell, J.C. Punishment-induced behavioral and neurophysiological variability reveals dopamine-dependent selection of kinematic movement parameters. J. Neurosci. 33, 3981–3988 (2013).
Dayan, P. & Daw, N.D. Decision theory, reinforcement learning, and the brain. Cogn. Affect. Behav. Neurosci. 8, 429–453 (2008).
Izawa, J. & Shadmehr, R. Learning from sensory and reward prediction errors during motor adaptation. PLoS Comput. Biol. 7, e1002012 (2011).
Wu, H.G., Miyamoto, Y.R., Castro, L.N., Olveczky, B.P. & Smith, M.A. Temporal structure of motor variability is dynamically regulated and predicts motor learning ability. Nat. Neurosci. 17, 312–321 (2014).
Nichols, R.A. Serotonin, presynaptic 5-HT(3) receptors and synaptic plasticity in the developing cerebellum. J. Physiol. (Lond.) 589, 5019–5020 (2011).
Kahneman, D. & Tversky, A. Prospect theory – analysis of decision under risk. Econometrica 47, 263–291 (1979).
De Martino, B., Camerer, C.F. & Adolphs, R. Amygdala damage eliminates monetary loss aversion. Proc. Natl. Acad. Sci. USA 107, 3788–3792 (2010).
Kahneman, D. & Tversky, A. Choices, values, and frames. Am. Psychol. 39, 341–350 (1984).
Mercer, J. Prospect theory and political science. Annu. Rev. Polit. Sci. 8, 1–21 (2005).
Dayan, E., Averbeck, B.B., Richmond, B.J. & Cohen, L.G. Stochastic reinforcement benefits skill acquisition. Learn. Mem. 21, 140–142 (2014).
Haith, A.M. & Krakauer, J.W. Model-based and model-free mechanisms of human motor learning. Adv. Exp. Med. Biol. 782, 1–21 (2013).
Orban de Xivry, J.J., Criscimagna-Hemminger, S.E. & Shadmehr, R. Contributions of the motor cortex to adaptive control of reaching depend on the perturbation schedule. Cereb. Cortex 21, 1475–1484 (2011).
Huang, V.S., Haith, A., Mazzoni, P. & Krakauer, J.W. Rethinking motor learning and savings in adaptation paradigms: model-free memory for successful actions combines with internal models. Neuron 70, 787–801 (2011).
Patton, J.L. & Mussa-Ivaldi, F.A. Robot-assisted adaptive training: custom force fields for teaching movement patterns. IEEE Trans. Biomed. Eng. 51, 636–646 (2004).
Diedrichsen, J., Hashambhoy, Y., Rane, T. & Shadmehr, R. Neural correlates of reach errors. J. Neurosci. 25, 9919–9931 (2005).
Tanaka, H., Sejnowski, T.J. & Krakauer, J.W. Adaptation to visuomotor rotation through interaction between posterior parietal and motor cortical areas. J. Neurophysiol. 102, 2921–2932 (2009).
Galea, J.M., Sami, S.A., Albert, N.B. & Miall, R.C. Secondary tasks impair adaptation to step- and gradual-visual displacements. Exp. Brain Res. 202, 473–484 (2010).
Galea, J.M., Vazquez, A., Pasricha, N., de Xivry, J.J. & Celnik, P. Dissociating the roles of the cerebellum and motor cortex during adaptive learning: the motor cortex retains what the cerebellum learns. Cereb. Cortex 21, 1761–1770 (2011).
Ingram, J.N., Flanagan, J.R. & Wolpert, D.M. Context-dependent decay of motor memories during skill acquisition. Curr. Biol. 23, 1107–1112 (2013).
This work was supported by a Birmingham Fellow research fellowship (J.M.G.). We thank P. Celnik for comments on a previous version of the manuscript.
The authors declare no competing financial interests.
Integrated supplementary information
(a) Average angular reach direction (º) in Experiment 2 for random positive (blue), reward (red), punishment (black) groups. Data is first averaged over Epoch (average across 8 trials), then across participants. Dashed/solid vertical lines indicate short rest periods between blocks (<1minute). (b) Predicted angular reach direction from the SSM fitted separately to each experimental phase (separated by solid vertical lines). Although epoch data is shown, the model was estimated using trial-by-trial reach direction data. Note that washout was estimated simply for presentation purposes. Solid lines = mean, shaded areas = SEM.
Supplementary Figure 2 A fixed generalization function leads to a slower rate of learning across groups
The SSM model assumed that learning did not generalise between the 8-target positions used in experiment 2. Yet, previous work has suggested that generalisation can take place between targets that are 45° apart. Therefore, we assumed a fixed generalisation function of: 0° = 1, 45° = 0.44, >90° = 0.2. This generalisation function was then used to estimate the learning rate for each phase/participant as before. As expected, the inclusion of generalisation led to substantially lower estimates of the learning rate. However similarly to the original model, punishment (black) led to significantly faster learning (SSM parameter B) compared to either the reward (p = 0.032; red) or random positive (p = 0.030; blue) group (F(2,41) = 4.65, p = 0.016). Mean ± SEM. *p<0.05.
About this article
Cite this article
Galea, J., Mallia, E., Rothwell, J. et al. The dissociable effects of punishment and reward on motor learning. Nat Neurosci 18, 597–602 (2015). https://doi.org/10.1038/nn.3956
This article is cited by
Scientific Reports (2023)
Reinforcement learning establishes a minimal metacognitive process to monitor and control motor learning performance
Nature Communications (2023)
Visuomotor errors drive step length and step time adaptation during ‘virtual’ split-belt walking: the effects of reinforcement feedback
Experimental Brain Research (2022)
Scientific Reports (2021)
Scientific Reports (2021)