The fundamental biological importance of rewards has created an increasing interest in the neuronal processing of reward information. The suggestion that the mechanisms underlying drug addiction might involve natural reward systems has also stimulated interest. This article focuses on recent neurophysiological studies in primates that have revealed that neurons in a limited number of brain structures carry specific signals about past and future rewards. This research provides the first step towards an understanding of how rewards influence behaviour before they are received and how the brain might use reward information to control learning and goal-directed behaviour.
This article describes how neurons detect rewards, learn to predict future rewards from past experience, and use reward information for learning, choosing, preparing and executing goal-directed behaviour. It also attempts to place the processing of drug rewards within a general framework of neuronal reward mechanisms.
Rewards are defined by their action on behaviour, and are crucial for the survival of the organism. They are vital in the control of homeostasis, sustain learning of new behaviours, the induction of approach behaviour and serve as goals for voluntary, intentional behaviour.
Various neurons detect the occurrence of rewards and reward-predicting stimuli, including those of the ascending dopamine systems, and neurons within the striatum, orbitofrontal cortex and amygdala. Some of these neurons seem to provide a reward prediction error signal that could be used for learning mechanisms, whereas others seem to be involved in the perception of individual rewards or objects that signal rewards.
Some neurons in the striatum and orbitofrontal cortex do not respond directly to rewards but seem to anticipate the occurrence of future rewards. Some neurons process reward information that is dependent on the relative motivational value of the reward.
Neurons in the striatum and different areas of frontal and parietal cortex incorporate information about expected rewards into neuronal activity involved in the production of behaviour leading to reward acquisition. They seem to code the goals of behaviour at the time the behaviour towards the goal is being prepared and executed. Some neurons are active before self-initiated, reward-directed movements and adapt their activity according to ongoing experience.
These studies show that different aspects of reward functions are processed by different neurons in different brain structures. The optimal use of reward information for learning and controlling behaviour requires cooperation between these neuronal reward signals.
The brain structures involved in the processing of natural rewards also seem to be the critical structures for the action of drugs of abuse. One may ask whether such drugs modify existing neuronal responses to natural rewards or constitute rewards in their own right, and as such engage existing neuronal reward mechanisms, directing subjects towards artificially rewarding goals.
This research describes the first steps towards an understanding of how rewards influence behaviour before their receipt and how the brain might use reward information to control learning and goal-directed behaviour.
Access optionsAccess options
Subscribe to Journal
Get full journal access for 1 year
only $22.08 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Fibiger, H. C. & Phillips, A. G. in Handbook of Physiology—The Nervous System Vol. IV (ed. Bloom, F. E.) 647– 675 (Williams and Wilkins, Baltimore, Maryland, 1986 ).
Wise, R. A. & Hoffman, D. C. Localization of drug reward mechanisms by intracranial injections. Synapse 10, 247–263 (1992).
Robinson, T. E. & Berridge, K. C. The neural basis for drug craving: an incentive-sensitization theory of addiction. Brain Res. Rev. 18, 247–291 (1993).
Robbins, T. W. & Everitt, B. J. Neurobehavioural mechanisms of reward and motivation. Curr. Opin. Neurobiol. 6, 228–236 (1996)
Louilot, A., LeMoal, M. & Simon, H. Differential reactivity of dopaminergic neurons in the nucleus accumbens in response to different behavioural situations. An in vivo voltammetric study in free moving rats. Brain Res. 397, 395–400 ( 1986).
Church, W. H. & Justice, J. B. Jr, Neill, D. B. Detecting behaviourally relevant changes in extracellular dopamine with microdialysis. Brain Res. 41, 397–399 (1987).
Young, A. M. J., Joseph, M. H. & Gray, J. A. Increased dopamine release in vivo in nucleus accumbens and caudate nucleus of the rat during drinking: a microdialysis study. Neuroscience 48, 871– 876 (1992).
Wilson, C., Nomikos, G. G., Collu, M. & Fibiger, H. C. Dopaminergic correlates of motivated behaviour: importance of drive. J. Neurosci. 15, 5169–5178 (1995).
Fiorino, D. F., Coury, A. & Phillips, A. G. Dynamic changes in nucleus accumbens dopamine efflux during the Coolidge effect in male rats. J. Neurosci. 17, 4849–4855 (1997).
Thut, G. et al. Activation of the human brain by monetary reward. NeuroReport 8, 1225–1228 (1997).
Rogers, R. D. et al. Choosing between small, likely rewards and large, unlikely rewards activates inferior and orbital prefrontal cortex. J. Neurosci. 19, 9029–9038 ( 1999)
Elliott, R., Friston, K. J. & Dolan, R. J. Dissociable neural responses in human reward systems . J. Neurosci. 20, 6159– 6165 (2000)
Pavlov, P. I. Conditioned Reflexes (Oxford Univ. Press, London, 1927 ).
Rescorla, R. A. & Wagner, A. R. in Classical Conditioning II: Current Research and Theory (eds Black, A. H. and Prokasy, W. F.) 64–99 (Appleton Century Crofts, New York, 1972).
Mackintosh, N. J. A theory of attention: variations in the associability of stimulus with reinforcement . Psychol. Rev. 82, 276– 298 (1975).
Pearce, J. M. & Hall, G. A model for Pavlovian conditioning: variations in the effectiveness of conditioned but not of unconditioned stimuli . Psychol. Rev. 87, 532– 552 (1980).
Schultz, W. Responses of midbrain dopamine neurons to behavioural trigger stimuli in the monkey. J. Neurophysiol. 56, 1439– 1462 (1986).
Romo, R. & Schultz, W. Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements . J. Neurophysiol. 63, 592– 606 (1990).
Schultz, W. & Romo, R. Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioural reactions . J. Neurophysiol. 63, 607– 624 (1990).
Ljungberg, T., Apicella, P. & Schultz, W. Responses of monkey dopamine neurons during learning of behavioural reactions. J. Neurophysiol. 67, 145–163 (1992).
Strecker, R. E. & Jacobs, B. L. Substantia nigra dopaminergic unit activity in behaving cats: effect of arousal on spontaneous discharge and sensory evoked activity. Brain Res. 361 , 339–350 (1985).
Horvitz, J. C. Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events. Neuroscience 96, 651– 656 (2000).
Mirenowicz, J. & Schultz, W. Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli. Nature 379, 449–451 ( 1996).
Guarraci, F. A. & Kapp, B. S. An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential Pavlovian fear conditioning in the awake rabbit. Behav. Brain Res. 99, 169–179 ( 1999).
Schultz, W. Activity of dopamine neurons in the behaving primate. Semin. Neurosci. 4, 129–138 ( 1992).
Redgrave, P., Prescott, T. J. & Gurney, K. Is the short-latency dopamine response too short to signal reward? Trends Neurosci. 22, 146– 151 (1999).
Hollerman, J. R. & Schultz, W. Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neurosci. 1, 304–309 (1998).
Schultz, W. Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1–27 (1998 ).
Schultz, W. & Dickinson, A. Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23, 473– 500 (2000).
Mirenowicz, J. & Schultz, W. Importance of unpredictability for reward responses in primate dopamine neurons. J. Neurophysiol. 72, 1024–1027 (1994).
Schultz, W., Dayan, P. & Montague, R. R. A neural substrate of prediction and reward. Science 275, 1593–1599 ( 1997).
Montague, P. R., Dayan, P. & Sejnowski, T. J. A framework for mesencephalic dopamine systems based on predictive Hebbian learning. J. Neurosci. 16, 1936–1947 (1996).
Sutton, R. S. & Barto, A. G. Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. 88, 135–170 (1981). This paper proposed a very effective 'temporal difference' reinforcement learning model that computes a prediction error over time. The teaching signal incorporates primary reinforcers and conditioned stimuli, and resembles in all aspects the response of dopamine neurons to rewards and conditioned, reward-predicting stimuli, although dopamine neurons also report novel stimuli.
Tesauro, G. TD-Gammon, a self-teaching backgammon program, achieves master-level play . Neural Comp. 6, 215–219 (1994).
Suri, R. E. & Schultz, W. Learning of sequential movements by neural network model with dopamine-like reinforcement signal. Exp. Brain Res. 121, 350–354 (1998).
Suri, R. & Schultz, W. A neural network with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience 91, 871–890 (1999).
Schultz, W. & Romo, R. Responses of nigrostriatal dopamine neurons to high intensity somatosensory stimulation in the anesthetized monkey . J. Neurophysiol. 57, 201– 217 (1987).
Abercrombie, E. D., Keefe, K. A., DiFrischia, D. S. & Zigmond, M. J. Differential effect of stress on in vivo dopamine release in striatum, nucleus accumbens, and medial frontal cortex. J. Neurochem. 52, 1655–1658 (1989).
Hikosaka, O., Sakamoto, M. & Usui, S. Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. J. Neurophysiol. 61, 814–832 ( 1989).
Apicella, P., Ljungberg, T., Scarnati, E. & Schultz, W. Responses to reward in monkey dorsal and ventral striatum. Exp. Brain Res. 85, 491–500 ( 1991).
Apicella, P., Scarnati, E. & Schultz, W. Tonically discharging neurons of monkey striatum respond to preparatory and rewarding stimuli. Exp. Brain Res. 84, 672–675 (1991).
Lavoie, A. M. & Mizumori, S. J. Y. Spatial-, movement- and reward-sensitive discharge by medial ventral striatum neurons of rats. Brain Res. 638, 157–168 ( 1994).
Bowman, E. M., Aigner, T. G. & Richmond, B. J. Neural signals in the monkey ventral striatum related to motivation for juice and cocaine rewards. J. Neurophysiol. 75, 1061–1073 (1996).
Shidara, M., Aigner, T. G. & Richmond, B. J. Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J. Neurosci. 18, 2613–2625 ( 1998).
Matsumura, M., Kojima, J., Gardiner, T. W. & Hikosaka, O. Visual and oculomotor functions of monkey subthalamic nucleus. J. Neurophysiol. 67, 1615–1632 (1992).
Schultz, W. Activity of pars reticulata neurons of monkey substantia nigra in relation to motor, sensory and complex events. J. Neurophysiol. 55, 660–677 (1986).
Niki, H., Sakai, M. & Kubota, K. Delayed alternation performance and unit activity of the caudate head and medial orbitofrontal gyrus in the monkey. Brain Res. 38, 343–353 ( 1972).
Niki, H. & Watanabe, M. Prefrontal and cingulate unit activity during timing behaviour in the monkey. Brain Res. 171 , 213–224 (1979).
Rosenkilde, C. E., Bauer, R. H. & Fuster, J. M. Single cell activity in ventral prefrontal cortex of behaving monkeys. Brain Res. 209, 375 –394 (1981).
Watanabe, M. The appropriateness of behavioural responses coded in post-trial activity of primate prefrontal units. Neurosci. Lett. 101, 113–117 (1989).
Tremblay, L. & Schultz, W. Reward-related neuronal activity during go–no go task performance in primate orbitofrontal cortex. J. Neurophysiol. 83, 1864–1876 (2000).
Niki, H. & Watanabe, M. Cingulate unit activity and delayed response. Brain Res. 110, 381– 386 (1976).
Nishijo, H., Ono, T. & Nishino, H. Single neuron responses in amygdala of alert monkey during complex sensory stimulation with affective significance. J. Neurosci. 8, 3570–3583 ( 1988).A paper written by one of the few groups investigating neurons in the primate amygdala in relation to reward-related stimuli. They reported a number of different responses to the presentation of natural rewards.
Nakamura, K., Mikami, A. & Kubota, K. Activity of single neurons in the monkey amygdala during performance of a visual discrimination task. J. Neurophysiol. 67, 1447–1463 (1992).
Burton, M. J., Rolls, E. T. & Mora, F. Effects of hunger on the responses of neurons in the lateral hypothalamus to the sight and taste of food. Exp. Neurol. 51, 668–677 ( 1976).
Tremblay, L. & Schultz, W. Relative reward preference in primate orbitofrontal cortex. Nature 398, 704– 708 (1999).
Pratt, W. E. & Mizumori, S. J. Y. Characteristics of basolateral amygdala neuronal firing on a spatial memory task involving differential reward . Behav. Neurosci. 112, 554– 570 (1998).
Thorpe, S. J., Rolls, E. T. & Maddison, S. The orbitofrontal cortex: neuronal activity in the behaving monkey. Exp. Brain Res. 49, 93– 115 (1983).
Rolls, E. T., Murzi, E., Yaxley, S., Thorpe, S. J. & Simpson, S. J. Sensory-specific satiety: food-specific reduction in responsiveness of ventral forebrain neurons after feeding in the monkey . Brain Res. 368, 79–86 (1986).
Rolls, E. T., Sienkiewicz, Z. J. & Yaxley, S. Hunger modulates the responses to gustatory stimuli of single neurons in the caudolateral orbitofrontal cortex of the macaque monkey. Eur. J. Neurosci. 1, 53– 60 (1989).
Rolls, E. T., Scott, T. R., Sienkiewicz, Z. J. & Yaxley, S. The responsiveness of neurons in the frontal opercular gustatory cortex of the macaque monkey is independent of hunger. J. Physiol. 397, 1–12 (1988).
Apicella, P., Legallet, E. & Trouche, E. Responses of tonically discharging neurons in the monkey striatum to primary rewards delivered during different behavioural states . Exp. Brain Res. 116, 456– 466 (1997).
Apicella, P., Ravel, S., Sardo, P. & Legallet, E. Influence of predictive information on responses of tonically active neurons in the monkey striatum. J. Neurophysiol. 80, 3341– 3344 (1998).
Apicella, P., Legallet, E. & Trouche, E. Responses of tonically discharging neurons in monkey striatum to visual stimuli presented under passive conditions and during task performance. Neurosci. Lett. 203, 147– 150 (1996).
Williams, G. V., Rolls, E. T., Leonard, C. M. & Stern, C. Neuronal responses in the ventral striatum of the behaving monkey. Behav. Brain Res. 55, 243–252 (1993).
Aosaki, T. et al. Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioural sensorimotor conditioning. J. Neurosci. 14, 3969–3984 (1994).
Apicella, P., Scarnati, E., Ljungberg, T. & Schultz, W. Neuronal activity in monkey striatum related to the expectation of predictable environmental events. J. Neurophysiol. 68, 945–960 (1992).
Schultz, W., Apicella, P., Scarnati, E. & Ljungberg, T. Neuronal activity in monkey ventral striatum related to the expectation of reward. J. Neurosci. 12, 4595– 4610 (1992).
Schoenbaum, G., Chiba, A. A. & Gallagher, M. Orbitofrontal cortex and basolateral amygdala encode expected outcomes during learning. Nature Neurosci. 1, 155–159 (1998).
Hikosaka, K. & Watanabe, M. Delay activity of orbital and lateral prefrontal neurons of the monkey varying with different rewards. Cerebral Cortex 10, 263–271 (2000).
Hollerman, J. R., Tremblay, L. & Schultz, W. Influence of reward expectation on behavior-related neuronal activity in primate striatum. J. Neurophysiol. 80, 947–963 (1998).
Tremblay, L., Hollerman, J. R. & Schultz, W. Modifications of reward expectation-related neuronal activity during learning in primate striatum. J. Neurophysiol. 80, 964–977 ( 1998).
Tremblay, L. & Schultz, W. Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. J. Neurophysiol. 83, 1877–1885 (2000).
Dickinson, A. & Balleine, B. Motivational control of goal-directed action. Animal Learn. Behav. 22, 1– 18 (1994).
Okano, K. & Tanji, J. Neuronal activities in the primate motor fields of the agranular frontal cortex preceding visually triggered and self-paced movement. Exp. Brain Res. 66, 155–166 (1987).
Romo, R. & Schultz, W. Neuronal activity preceding self-initiated or externally timed arm movements in area 6 of monkey cortex. Exp. Brain Res. 67, 656–662 (1987).
Kurata, K. & Wise, S. P. Premotor and supplementary motor cortex in rhesus monkeys: neuronal activity during externally- and internally-instructed motor tasks. Exp. Brain Res. 72, 237– 248 (1988).
Schultz, W. & Romo, R. Neuronal activity in the monkey striatum during the initiation of movements. Exp. Brain Res. 71, 431–436 (1988).
Watanabe, M. Reward expectancy in primate prefrontal neurons. Nature 382, 629–632 (1996). The first demonstration that expected rewards influence the behaviour-related activity of neurons in a manner that is compatible with a goal-directed account. Neurons in primate dorsolateral prefrontal cortex show different activities depending on the expected reward during a typical prefrontal spatial delayed response task.
Leon, M. I. & Shadlen, M. N. Effect of expected reward magnitude on the responses of neurons in the dorsolateral prefrontal cortex of the macaque . Neuron 24, 415–425 (1999).
Kawagoe, R., Takikawa, Y. & Hikosaka, O. Expectation of reward modulates cognitive signals in the basal ganglia. Nature Neurosci. 1, 411 –416 (1998).
Liu, Z. & Richmond, B. J. Response differences in monkey TE and perirhinal cortex: stimulus association related to reward schedules . J. Neurophysiol. 83, 1677– 1692 (2000).The first demonstration of the prominent reward relationships in neurons of temporal cortex. Neuronal responses to task stimuli in the primate perirhinal cortex were profoundly affected by the distance to reward, whereas neuronal responses in the neighbouring TE area predominantly reflected the visual features of the stimuli.
Platt, M. L. & Glimcher, P. W. Neural correlates of decision variables in parietal cortex. Nature 400, 233–238 (1999).Neurons in primate parietal association cortex were sensitive to two key variables of game theory and decision making: the quantity and the probability of the outcomes. Based on studies of choice behaviour and decision making in human economics and animal psychology, this is the first application of principles of decision theory in primate neurophysiology.
Shima, K. & Tanji, J. Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282, 1335–1338 (1998). Neurons in the primate cingulate motor area were active when animals switched to a different movement when continuing to perform the current movement would have produced less reward. This study is interesting from the point of view of movement selection and also for the influence of rewards on behavioural choices.
Kelley, A. E. Functional specificity of ventral striatal compartments in appetitive behaviours . Ann. NY Acad. Sci. 877, 71– 90 (1999).
Carelli, R. M., King, V. C., Hampson, R. E. & Deadwyler, S. A. Firing patterns of nucleus accumbens neurons during cocaine self-administration . Brain Res. 626, 14–22 (1993).The first demonstration of drug-related changes in neuronal activity in one of the key structures of drug addiction, the nucleus accumbens. Two important neuronal patterns were found in the rat: activity preceding lever presses that led to acquisition of the drug and activity following drug delivery. These findings have been consistently reproduced by other goups.
Chang, J. Y., Sawyer, S. F., Lee, R. S. & Woodward, D. J. Electrophysiological and pharmacological evidence for the role of the nucleus accumbens in cocaine self-administration in freely moving rats. J. Neurosci. 14, 1224–1244 (1994).
Carelli, R. M. & Deadwyler, S. A. A comparison of nucleus accumbens neuronal firing patterns during cocaine self-administration and water reinforcement in rats. J. Neurosci. 14, 7735–7746 (1994).
Chang, J. Y., Janak, P. H. & Woodward, D. J. Comparison of mesocorticolimbic neuronal responses during cocaine and heroin self-administration in freely moving rats. J. Neurosci. 18, 3098–3115 (1998).
Carelli, R. M., Ijames, S. G. & Crumling, A. J. Evidence that separate neural circuits in the nucleus accumbens encode cocaine versus 'natural' (water and food) reward. J. Neurosci. 20, 4255–4266 (2000).
Chang, J. Y., Paris, J. M., Sawyer, S. F., Kirillov, A. B. & Woodward, D. J. Neuronal spike activity in rat nucleus accumbens during cocaine self-administration under different fixed-ratio schedules. J. Neurosci. 74, 483– 497 (1996).
Peoples, L. L., Uzwiak, A. J., Gee, F. & West, M. O. Operant behaviour during sessions of intravenous cocaine infusion is necessary and sufficient for phasic firing of single nucleus accumbens neurons. Brain Res. 757, 280–284 ( 1997).
West, M. O., Peoples, L. L., Michael, A. J., Chapin, J. K. & Woodward, D. J. Low-dose amphetamine elevates movement-related firing of rat striatal neurons. Brain Res. 745, 331–335 (1997).
Peoples, L. L. & West, M. O. Phasic firing of single neurons in the rat nucleus accumbens correlated with the timing of intravenous cocaine self-administration. J. Neurosci. 16, 3459–3473 (1996).
Peoples, L. L., Gee, F., Bibi, R. & West, M. O. Phasic firing time locked to cocaine self-infusion and locomotion: dissociable firing patterns of single nucleus accumbens neurons in the rat. J. Neurosci. 18, 7588–7598 (1998).
Calabresi, P., Maj, R., Pisani, A., Mercuri, N. B. & Bernardi, G. Long-term synaptic depression in the striatum: physiological and pharmacological characterization. J. Neurosci. 12, 4224–4233 (1992).
Otmakhova, N. A. & Lisman, J. E. D1/D5 dopamine receptor activation increases the magnitude of early long-term potentiation at CA1 hippocampal synapses. J. Neurosci. 16, 7478–7486 (1996).
Wickens, J. R., Begg, A. J. & Arbuthnott, G. W. Dopamine reverses the depression of rat corticostriatal synapses which normally follows high-frequency stimulation of cortex in vitro. Neuroscience 70, 1– 5 (1996).
Calabresi, P. et al. Abnormal synaptic plasticity in the striatum of mice lacking dopamine D2 receptors. J. Neurosci. 17, 4536–4544 (1997).
The crucial contributions of the collaborators in my own cited work are gratefully acknowledged, as are the collegial interactions with Anthony Dickinson (Cambridge) and Masataka Watanabe (Tokyo). Our work was supported by the Swiss National Science Foundation, European Community, McDonnell–Pew Program, Roche Research Foundation and British Council, and by postdoctoral fellowships from the US NIMH, FRS Quebec, Fyssen Foundation and FRM Paris.
ENCYCLOPEDIA OF LIFE SCIENCES
- GOAL-DIRECTED BEHAVIOUR
Behaviour controlled by representation of a goal or an understanding of a causal relationship between behaviour and attainment of a goal.
Positive reinforcers (rewards) increase the frequency of behaviour leading to their acquisition. Negative reinforcers (punishers) decrease the frequency of behaviour leading to their encounter and increase the frequency of behaviour leading to their avoidance.
- PAVLOVIAN (CLASSICAL) CONDITIONING
Learning a predictive relationship between a stimulus and a reinforcer — does not require an action by the agent.
- OPERANT (INSTRUMENTAL) CONDITIONING
Learning a relationship between a stimulus, an action and a reinforcer conditional on an action by the agent.
Reduction and cessation of a predictive relationship and behaviour following the omission of a reinforcer (negative prediction error).
- MOTIVATIONAL VALUE
A measure of the effort an agent is willing to expend to obtain an object signalling reward or to avoid an object signalling punishment.
About this article
Nature Neuroscience (2019)
Neurons in rat orbitofrontal cortex and medial prefrontal cortex exhibit distinct responses in reward and strategy-update in a risk-based decision-making task
Metabolic Brain Disease (2019)
Dissociable Effects of Cocaine Dependence on Reward Processes: The Role of Acute Cocaine and Craving
Associating a product with a luxury brand label modulates neural reward processing and favors choices in materialistic individuals
Scientific Reports (2017)
The Human Coparental Bond Implicates Distinct Corticostriatal Pathways: Longitudinal Impact on Family Formation and Child Well-Being