Differentiating effort-related aspects of motivation from reinforcement learning: commentary on Soder et al. “Dose–response effects of d-amphetamine on effort-based decision-making and reinforcement learning”

Subjects

One of the most intensive areas of research and debate in neuropsychopharmacology over the last few decades has been the study of the behavioral functions of brain dopamine (DA) systems. Drugs that alter DA transmission such as DA transport (DAT) inhibitors or DA receptor antagonists are known to affect aspects of appetitive motivation and instrumental behavior. Nevertheless, researchers are still grappling with the details of this involvement, such as the specific aspects of motivation being regulated, and also are trying to determine the conceptual framework that is most useful for understanding this complex area of study. In contemplating this issue, it is vital to consider some of the contributions that the field of neuropsychology has provided to psychopharmacology research. It is important to recognize that global functional descriptors such as “reward”, “reinforcement”, “motivation”, and “motor control” are blunt instruments; these are global and complex processes that can be parsed into several distinct and dissociable aspects. Classical brain manipulations such as lesions or drugs, as well as more recently developed techniques such as optogenetics or chemogenetics, can dissociate these specific aspects from each other by demonstrating that it is possible to substantially alter one function while leaving another largely or completely intact.

The Soder et al. [1] paper in this issue provides a compelling example of the role that neuropsychopharmacology research can play in dissociating aspects of appetitive motivation in humans. These studies were conducted to test the effects of different doses of d-amphetamine that are in the therapeutic range for adult ADHD (i.e., placebo, 10 and 20 mg) on effort-related decision making and reinforcement learning in the same group of healthy human volunteers. In order to assess effort-related aspects of motivation, participants were tested on the Effort Expenditure for Reward Task (EEfRT) task, which is a procedure developed by Treadway et al. in 2009. The EEfRT task measures effort-related decision-making by offering participants the choice between more difficult and vigorous finger tapping to receive a larger monetary reward vs. less effortful finger tapping to receive less reward. Thus, it provides a measure of effort selection based upon the vigorous repetition of a response, though not specifically on force output. This procedure has been used in previous studies of the effort-related motivational effects of d-amphetamine, as well as assessments of these processes in people with depression, schizophrenia, and other psychiatric disorders (for review see ref. [2]). In addition, the same participants were tested on the Probabilistic Reward Task to provide a measure of reinforcement learning [3]. With this task, cartoon face images were presented, and discrimination of the lengths of facial features was differentially reinforced on a probabilistic basis, after which the results were analyzed by a signal detection approach that detected bias towards the more reinforced category. The authors also examined whether individual differences in baseline working memory and willingness to exert effort for monetary reward acted as moderators of the effects of d-amphetamine. Consistent with previous studies, d-amphetamine increased willingness to exert effort, especially at low-to-moderate expected levels of monetary reinforcement. Computational modeling analyses demonstrated that baseline effort expenditure and working memory emerged as moderators of the effect of d-amphetamine on exertion of effort. d-amphetamine increased selection of high-effort choices to a greater extent in people with lower working memory performance and lower baseline effort expenditure, and also had greater effects when expected values of reinforcement were moderate-to-low. In contrast to these clear effects on selection of high effort activity, the same doses of d-amphetamine had no significant effect on reinforcement learning as assessed by the Probabilistic Reward Task.

These results indicate that the effects of d-amphetamine on selection of high-effort activity are not dependent upon actions of the drug on reinforcement learning processes per se, and that these two aspects of motivational and cognitive function can be dissociated. Moreover, they are consistent with the idea that the enhancement of DA transmission induced by d-amphetamine at the doses used is not having some broad or general effect on all aspects of “reward” processes, but rather, is producing differential actions on distinct aspects of appetitive motivation [2]. The Soder et al. results are consistent with rodent studies of effort-based choice involving genetic or pharmacological manipulations of the DAT. Knockdown of the DAT in mice enhanced selection of high effort lever pressing but did not affect reward learning [4]. In rats tested on effort-based choice lever pressing tasks, administration of selective DAT inhibitors consistently increases selection of high effort progressive ratio responding [5, 6], in contrast to drugs that block the norepinephrine and serotonin transporters [5]. Effort-based choice tasks in rodents are increasingly being used to model motivational dysfunctions in psychopathology, and develop potential drug treatments for neuropsychiatric disorders [2, 5, 6]. Questions remain about the precise role of various monoamines in different aspects of effort-related function (e.g., sustained repetitive output, force output, physical vs. cognitive effort). Nevertheless, the Soder et al. paper has important implications for understanding the role of effort-related motivational processes in psychopathology, and could shed light on the neurobiology of psychiatric symptoms such as fatigue, anergia, and lassitude in depression and avolition in schizophrenia [1, 2].

Funding and disclosure

JS has received grants from, and done consulting work for, Shire, Prexa, Chronos, Blackthorn, Lundbeck, and Acadia. There are no competing financial interests, and there is nothing to disclose.

References

  1. 1.

    Soder HE, Cooper JA, Lopez-Gamundi P, Hoots JK, Nunez C, Lawlor VM, et al. Dose–response effects of d-amphetamine on effort-based decision-making and reinforcement learning. Neuropsychopharmacology. 2020. https://doi.org/10.1038/s41386-020-0779-8.

  2. 2.

    Salamone JD, Yohn S, Lopez-Cruz L, San Miguel N, Correa M. Activational and effort-related aspects of motivation: neural mechanisms and implications for psychopathology. Brain 2016;139:1325–47.

    Article  Google Scholar 

  3. 3.

    Pizzagalli DA, Jahn AL, O’Shea JP. Toward an objective characterization of an anhedonic phenotype: a signal-detection approach. Biol Psychiatry. 2005;57:319–27.

    Article  Google Scholar 

  4. 4.

    Cagniard B, Balsam PD, Brunner D, Zhuang X. Mice with chronically elevated dopamine exhibit enhanced motivation, but not learning, for a food reward. Neuropsychopharmacology. 2006;31:1362–70.

    CAS  Article  Google Scholar 

  5. 5.

    Yohn SE, Errante EL, Rosenbloom-Snow A, Sommerville M, Rowland MA, Tokarski K, et al. Blockade of uptake for dopamine, but not norepinephrine or 5-HT, increases selection of high effort instrumental activity: implications for treatment of effort-related motivational symptoms in psychopathology. Neuropharmacology. 2016;109:270–80.

    CAS  Article  Google Scholar 

  6. 6.

    Rotolo RA, Kalaba P, Dragacevic V, Presby RE, Neri J, Robertson E, et al. Behavioral and dopamine transporter binding properties of the modafinil analog (S, S)-CE-158: reversal of the motivational effects of tetrabenazine and enhancement of progressive ratio responding. Psychopharmacology. 2020;237:3459–70.

    CAS  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to John D. Salamone.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Salamone, J.D. Differentiating effort-related aspects of motivation from reinforcement learning: commentary on Soder et al. “Dose–response effects of d-amphetamine on effort-based decision-making and reinforcement learning”. Neuropsychopharmacol. (2021). https://doi.org/10.1038/s41386-020-00930-2

Download citation

Search

Quick links