Gregory D. Horwitz is in the Vision Center Laboratory and Edward M. Callaway is in the Systems Neurobiology Laboratories at the Salk Institute for Biological Studies, La Jolla, California 92037, USA. horwitz@salk.edu
Monkeys and humans work harder for immediate than for distant rewards. How are associations between reward immediacy and sensory stimuli established in the brain? A recent study suggests a crucial role for dopamine-mediated activity in the rhinal cortex.
Motivation is greatest when previously invested effort is significant and when the payoff is near. The typical first-year Ph.D. student, for example, would happily spend a week carousing with college friends in Las Vegas, despite the expense of graduating a week later than expected. Once graduation is in sight, the same student might put aside spinning the roulette wheel for the centrifuge. This behavior is not strictly rational, but we recognize it as a basic part of human nature, or more precisely, a product of how our brains work.
Recent studies of monkey behavior have shown that monkeys also step up their performance when a reward is imminent1,
2,
3. If they know they have to perform several trials of a behavioral task correctly to earn a reward, they make errors more frequently than if a single correct trial triggers an immediate reward. In the 17 August issue of the Proceedings of the National Academy of Sciences, Liu et al.3 show that expression of the D2 dopamine receptor in the rhinal cortex is critical for this behavior and, in so doing, they provide the first-ever demonstration that experimental manipulation of a neural circuit at the level of gene expression can cause profound and specific changes in the cognitive behavior of a nonhuman primate.
Monkeys in the Liu et al. experiment were seated in front of a video projection screen and trained to release a bar when a spot of light changed color. An additional visual cue informed them of whether or not they would be rewarded at the end of the trial, and if not, how many trials they had to complete successfully before the reward would come. Once they had learned the association between the visual cues and the reward schedule, the monkeys became lackadaisical and error-prone on trials they knew would not end in a reward and diligent when they knew the reward was at hand. This natural preference for immediate over deferred gratification provided a valuable entry point for the study of neural structures underlying the association between visual stimuli and reward expectation.
Previously, these authors had shown that the rhinal (perirhinal and entorhinal) cortex, a centimeter-long strip of cortex tucked up beneath the temporal lobes, is a critical component of the neural circuitry underlying these associations2. Lesions of the rhinal cortex prevented monkeys from learning associations between cues and rewards or acting on previously learned associations. Counterintuitively, monkeys with lesions made fewer errors than control animals, as if they believed that every trial might end in a reward.
Lesion experiments such as these can be difficult to interpret for a number of reasons. First, many different changes in the underlying neural circuitry may give rise to identical changes in the measured behavior. Second, behavioral deficits can evolve as plastic mechanisms compensate for the ablated cortex. Third, surgical lesions cannot reveal which components of the ablated circuit were relevant to the behavior of the intact animal. For example, a particular neuronal subtype or biochemical pathway may be a key determinant of the behavior, but standard lesions cannot target these selectively.
Convergent evidence indicates that the neurotransmitter dopamine is important for mediating associations between stimulus events and rewards. Dopamine antagonists interfere with learning, synaptic plasticity and motivation, and the electrical discharge of dopaminergic neurons in the ventral tegmentum reflects discrepancies between expected and obtained rewards4,
5. The role of dopamine has conventionally been probed with drugs that enhance or suppress dopamine signaling, but these drugs may lack receptor subtype specificity and their actions last only minutes to hours − not long enough to test for effects on learning that evolves over weeks.
To avoid the problems inherent in the conventional pharmacological approach, Liu et al. used a new technique for suppressing dopamine signaling that was previously used to probe other brain regions in mice6,
7,
8. They injected an antisense DNA construct targeting the D2 dopamine receptor into the monkeys' rhinal cortex. The DNA was complexed with cationic lipids to facilitate its incorporation into the neurons, and the antisense DNA sequence was designed such that, when transcribed, the resulting RNA would interfere with the mRNA transcribed from the normal D2 receptor genes. This resulted in a decrease in D2 dopamine receptor expression in the rhinal cortex of these monkeys. Control monkeys were given an injection of antisense DNA targeting and reducing expression of the N-methyl-D-aspartate (NMDA) receptor. Both groups of monkeys were then tested to determine whether they could learn to associate a new visual cue with the immediacy of reward.
Monkeys treated with NMDA receptor antisense DNA or a control cationic lipid with no DNA rapidly learned to associate visual cues (here, bar length) with reward immediacy, as evidenced by their relatively poor performance when the cue informed them that the reward was several trials away (Fig. 1, left column). Monkeys treated with D2 receptor antisense DNA, on the other hand, failed to learn this association and performed the task well irrespective of the cue, similar to monkeys whose rhinal cortex had been surgically ablated (Fig. 1, right column). This result is strong evidence that dopamine signaling in rhinal cortex is important for the association of visual cues and rewards.
Figure 1. Effects of reduced dopamine receptor function in rhinal cortex on learned associations between visual cues and reward.
Monkeys were trained to associate a visual cue, here the length of the white bar on the display, with the number of trials remaining before a reward. Under normal circumstances (left), the monkeys' behavior correlated with the number of trials remaining. Performance was best (90% correct) when the visual cue (short bar) indicated that a reward would come immediately after a successful trial (bottom, left); when the length of the bar indicated that more trials would be required, performance was worse. Injection of NMDA receptor antisense DNA into the rhinal cortex had no effect on the learning of these associations. Injection of D2 dopamine receptor antisense DNA, in contrast, had a clear effect (right). These monkeys failed to associate the visual cues with reward and performed well (90% correct) regardless of the visual cue.
The cationic lipid used for DNA delivery in these experiments allowed Liu et al. to go a step further, and show that the effects were not permanent and could be repeated. DNA delivery with cationic lipids is suboptimal for mediating gene therapy in humans because gene expression is not maintained for more than a few weeks or months, presumably because of DNA degradation7,
9. Liu et al. exploited this feature of the technique by showing that monkeys could recover and learn new associations as the expression of antisense DNA wore off. It is likely that this success will inspire other investigators to use cationic lipids for future genetic manipulations in primates. For example, this method could be used to deliver genes to manipulate or probe specific components of the neural circuits underlying early visual processing and perception.
These results clearly show the importance of dopamine signaling in the monkeys' behavior, but a few outstanding issues remain. One crucial question is whether the dopamine receptor manipulation caused a deficit in learning specifically or in the readout or maintenance of the memory trace. In the previous lesion studies, animals not only failed to learn new associations but also did not distinguish among previously learned visual cues2. It is important to know whether this was also true of monkeys treated with the dopamine receptor antisense DNA.
At least superficially, the monkeys' behavior was affected similarly by reduction of dopamine receptor expression and by surgical lesions. This raises the issue of how the reduction of dopamine receptor expression caused its effects. A likely possibility is that dopamine-mediated communication is required for making associations at a cellular level between neurons carrying cue-related and reward-related activity. Consistent with this idea, dopamine has been implicated in both long-term potentiation and depression10,
11. It would be helpful, however, to know how electrical activity in the rhinal cortex was changed by the introduction of D2 receptor antisense DNA. One would hope that there was not a nonspecific effect, for example, on overall activity levels that approximated the effects of a simple lesion. Subtle changes in the relationships between activity and behavior are more likely, and observation of such changes would provide additional insight into the mechanisms of dopamine action.
Although there is still much to be learned, these studies set the stage for more detailed investigations of the role of dopamine in reward learning. The candidate mechanisms have been substantially narrowed and an important brain region clearly implicated. In addition, these studies have broader implications for studies of brain function. The first successful genetic manipulation of a well-defined behavior in a primate model is a milestone that will have implications far beyond the rhinal cortex.
