Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Opponent appetitive-aversive neural processes underlie predictive learning of pain relief

Nature Neuroscience volume 8, pages 1234–1240 (2005)Cite this article

Abstract

Termination of a painful or unpleasant event can be rewarding. However, whether the brain treats relief in a similar way as it treats natural reward is unclear, and the neural processes that underlie its representation as a motivational goal remain poorly understood. We used fMRI (functional magnetic resonance imaging) to investigate how humans learn to generate expectations of pain relief. Using a pavlovian conditioning procedure, we show that subjects experiencing prolonged experimentally induced pain can be conditioned to predict pain relief. This proceeds in a manner consistent with contemporary reward-learning theory (average reward/loss reinforcement learning), reflected by neural activity in the amygdala and midbrain. Furthermore, these reward-like learning signals are mirrored by opposite aversion-like signals in lateral orbitofrontal cortex and anterior cingulate cortex. This dual coding has parallels to 'opponent process' theories in psychology and promotes a formal account of prediction and expectation during pain.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Experimental design and computational model.
Figure 2: Behavioral measures.
Figure 3: Appetitive temporal difference prediction error.
Figure 4: Aversive temporal difference prediction error.
Figure 5: Appetitive relief-related plus aversive exacerbation-related prediction error.

Similar content being viewed by others

References

  1. Cabanac, M. Physiological role of pleasure. Science 173, 1103–1107 (1971).

    Article  CAS  Google Scholar 

  2. Craig, A.D. A new view of pain as a homeostatic emotion. Trends Neurosci. 26, 303–307 (2003).

    Article  CAS  Google Scholar 

  3. Fields, H. State-dependent opioid control of pain. Nat. Rev. Neurosci. 5, 565–575 (2004).

    Article  CAS  Google Scholar 

  4. Solomon, R.L. & Corbit, J.D. An opponent-process theory of motivation. I. Temporal dynamics of affect. Psychol. Rev. 81, 119–145 (1974).

    Article  CAS  Google Scholar 

  5. Konorski, J. Integrative Activity of the Brain: an Interdisciplinary Approach (Chicago, University of Chicago Press, 1967).

    Google Scholar 

  6. Schull, J. A conditioned opponent theory of Pavlovian conditioning and habituation. in The Psychology of Learning and Motivation (ed. Bower, G.) 57–90 (Academic, New York, 1979).

    Google Scholar 

  7. Grossberg, S. Some normal and abnormal behavioral syndromes due to transmitter gating of opponent processes. Biol. Psychiatry 19, 1075–1118 (1984).

    CAS  PubMed  Google Scholar 

  8. Solomon, R.L. The opponent-process theory of acquired motivation: the costs of pleasure and the benefits of pain. Am. Psychol. 35, 691–712 (1980).

    Article  CAS  Google Scholar 

  9. Solomon, R.L. Recent experiments testing an opponent-process theory of acquired motivation. Acta Neurobiol. Exp. (Wars.) 40, 271–289 (1980).

    CAS  Google Scholar 

  10. Dickenson & Dearing, M F. Appetitive-aversive interactions and inhibitory processes. in Mechanisms of Learning and Motivation. (eds. Dickinson, A. & Boakes, R.A.) 203–231 (Erlbaum, Hillsdale, New Jersey, 1979).

    Google Scholar 

  11. Daw, N.D., Kakade, S. & Dayan, P. Opponent interactions between serotonin and dopamine. Neural Netw. 15, 603–616 (2002).

    Article  Google Scholar 

  12. Tanimoto, H., Heisenberg, M. & Gerber, B. Experimental psychology: event timing turns punishment to reward. Nature 430, 983 (2004).

    Article  CAS  Google Scholar 

  13. Barto, A.G. Adaptive critics and the basal ganglia. in Models of Information Processing in the Basal Ganglia (eds. Houk, J.C., Davis, J.L. & Beiser, D.G.) 215–232 (MIT Press, Cambridge, Massachusetts, 1995).

    Google Scholar 

  14. Sutton, R.S. & Barto, A.G. Reinforcement Learning: an Introduction. (MIT Press, Cambridge, Massachusetts, 1998).

    Google Scholar 

  15. Montague, P.R., Dayan, P. & Sejnowski, T.J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).

    Article  CAS  Google Scholar 

  16. Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    Article  CAS  Google Scholar 

  17. O'Doherty, J.P., Dayan, P., Friston, K., Critchley, H. & Dolan, R.J. Temporal difference models and reward-related learning in the human brain. Neuron 38, 329–337 (2003).

    Article  CAS  Google Scholar 

  18. Seymour, B. et al. Temporal difference models describe higher-order learning in humans. Nature 429, 664–667 (2004).

    Article  CAS  Google Scholar 

  19. Dayan, P. & Balleine, B.W. Reward, motivation, and reinforcement learning. Neuron 36, 285–298 (2002).

    Article  CAS  Google Scholar 

  20. Schwartz, A. A reinforcement learning method for maximizing undiscounted rewards. in Proceedings of the Tenth International Conference on Machine Learning. 298–305 (Morgan Kaufmann, San Mateo, California, 1993).

    Google Scholar 

  21. Mahadevan, S. Average reward reinforcement learning: Foundations, algorithms and empirical results. Mach. Learn. 22, 1–38 (1996).

    Google Scholar 

  22. Fields, H.L. Pain modulation: expectation, opioid analgesia and virtual pain. Prog. Brain Res. 122, 245–253 (2000).

    Article  CAS  Google Scholar 

  23. Price, D.D. Psychological Mechanisms of Pain and Analgesia (IASP, Seattle, 1999).

    Google Scholar 

  24. Tanaka, S.C. et al. Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat. Neurosci. 7, 887–893 (2004).

    Article  CAS  Google Scholar 

  25. Ploghaus, A. et al. Dissociating pain from its anticipation in the human brain. Science 284, 1979–1981 (1999).

    Article  CAS  Google Scholar 

  26. Jensen, J. et al. Direct activation of the ventral striatum in anticipation of aversive stimuli. Neuron 40, 1251–1257 (2003).

    Article  CAS  Google Scholar 

  27. McClure, S.M., Berns, G.S. & Montague, P.R. Temporal prediction errors in a passive learning task activate human striatum. Neuron 38, 339–346 (2003).

    Article  CAS  Google Scholar 

  28. Setlow, B., Schoenbaum, G. & Gallagher, M. Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38, 625–636 (2003).

    Article  CAS  Google Scholar 

  29. Daw, N.D. & Touretzky, D.S. Long-term reward prediction in TD models of the dopamine system. Neural Comput. 14, 2567–2583 (2002).

    Article  Google Scholar 

  30. Markowitz, H. The utility of wealth. J. Polit. Econ. 60, 151–158 (1952).

    Article  Google Scholar 

  31. Camerer, C., Loewenstein, G. & Prelec, D. Neuroeconomics: how neuroscience can inform economics. J. Econ. Lit. (in the press).

  32. Rogan, M.T., Leon, K.S., Perez, D.L. & Kandel, E.R. Distinct neural signatures for safety and danger in the amygdala and striatum of the mouse. Neuron 46, 309–320 (2005).

    Article  CAS  Google Scholar 

  33. Watkins, L.R. et al. Neurocircuitry of conditioned inhibition of analgesia: effects of amygdala, dorsal raphe, ventral medullary, and spinal cord lesions on antianalgesia in the rat. Behav. Neurosci. 112, 360–378 (1998).

    Article  CAS  Google Scholar 

  34. Holland, P.C. & Gallagher, M. Amygdala-frontal interactions and reward expectancy. Curr. Opin. Neurobiol. 14, 148–155 (2004).

    Article  CAS  Google Scholar 

  35. O'Doherty, J., Kringelbach, M.L., Rolls, E.T., Hornak, J. & Andrews, C. Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4, 95–102 (2001).

    Article  CAS  Google Scholar 

  36. Small, D.M., Zatorre, R.J., Dagher, A., Evans, A.C. & Jones-Gotman, M. Changes in brain activity related to eating chocolate: from pleasure to aversion. Brain 124, 1720–1733 (2001).

    Article  CAS  Google Scholar 

  37. Glascher, J. & Buchel, C. Formal learning theory dissociates brain regions with different temporal integration. Neuron 47, 295–306 (2005).

    Article  Google Scholar 

  38. Petrovic, P., Kalso, E., Petersson, K.M. & Ingvar, M. Placebo and opioid analgesia–imaging a shared neuronal network. Science 295, 1737–1740 (2002).

    Article  CAS  Google Scholar 

  39. Wager, T.D. et al. Placebo-induced changes in FMRI in the anticipation and experience of pain. Science 303, 1162–1167 (2004).

    Article  CAS  Google Scholar 

  40. Logothetis, N.K., Pauls, J., Augath, M., Trinath, T. & Oeltermann, A. Neurophysiological investigation of the basis of the fMRI signal. Nature 412, 150–157 (2001).

    Article  CAS  Google Scholar 

  41. Stefanovic, B., Warnking, J.M. & Pike, G.B. Hemodynamic and metabolic responses to neuronal inhibition. Neuroimage 22, 771–778 (2004).

    Article  Google Scholar 

  42. Altier, N. & Stewart, J. The role of dopamine in the nucleus accumbens in analgesia. Life Sci. 65, 2269–2287 (1999).

    Article  CAS  Google Scholar 

  43. Horvitz, J.C. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96, 651–656 (2000).

    Article  CAS  Google Scholar 

  44. Smith, A.J., Becker, S. & Kapur, S. A computational model of the functional role of the ventral-striatal D2 receptor in the expression of previously acquired behaviors. Neural Comput. 17, 361–395 (2005).

    Article  Google Scholar 

  45. Ungless, M.A., Magill, P.J. & Bolam, J.P. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science 303, 2040–2042 (2004).

    Article  CAS  Google Scholar 

  46. Roitman, M.F., Wheeler, R.A. & Carelli, R.M. Nucleus accumbens neurons are innately tuned for rewarding and aversive taste stimuli, encode their predictors, and are linked to motor output. Neuron 45, 587–597 (2005).

    Article  CAS  Google Scholar 

  47. Johansen, J.P. & Fields, H.L. Glutamatergic activation of anterior cingulate cortex produces an aversive teaching signal. Nat. Neurosci. 7, 398–403 (2004).

    Article  CAS  Google Scholar 

  48. Gadd, C.A., Murtra, P., De Felipe, C. & Hunt, S.P. Neurokinin-1 receptor-expressing neurons in the amygdala modulate morphine reward and anxiety behaviors in the mouse. J. Neurosci. 23, 8271–8280 (2003).

    Article  CAS  Google Scholar 

  49. Dayan, P. & Abbott, L.F. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, Cambridge, Massachusetts, 2001).

    Google Scholar 

  50. Buchel, C., Holmes, A.P., Rees, G. & Friston, K.J. Characterizing stimulus-response functions using nonlinear regressors in parametric fMRI experiments. Neuroimage 8, 140–148 (1998).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We wish to thank P. Dayan and N. Daw for many helpful discussions and O. Josephs, B. Johanssen and C. Rickard for technical assistance. This research was funded by The Wellcome Trust.

Author information

Authors and Affiliations

  1. Wellcome Department of Imaging Neuroscience, 12 Queen Square, London, WC1N 3BG, UK

    Ben Seymour, John P O'Doherty, Katja Wiech, Richard Frackowiak, Karl Friston & Raymond Dolan

  2. Division of the Humanities and Social Sciences 228-77, California Institute of Technology, Pasadena, 91125, California, USA

    John P O'Doherty

  3. Institute of Child Health, University College London, 30 Guildford Street, London, WC1N 1EH, UK

    Martin Koltzenburg

  4. Neuroimaging Laboratory, Fondazione Santa Lucia, Rome, 00179, Italy

    Richard Frackowiak

Authors
  1. Ben Seymour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. John P O'Doherty
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin Koltzenburg
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Katja Wiech
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Richard Frackowiak
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Karl Friston
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Raymond Dolan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ben Seymour.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Seymour, B., O'Doherty, J., Koltzenburg, M. et al. Opponent appetitive-aversive neural processes underlie predictive learning of pain relief. Nat Neurosci 8, 1234–1240 (2005). https://doi.org/10.1038/nn1527

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn1527

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing