INTRODUCTION

Drug addiction is a chronic disorder (McLellan et al, 2000; O’Brien, 2008), where even following long abstinence periods, exposure to drug-associated cues can trigger craving or drug-seeking behaviors, increasing the relapse risk (eg, O’Brien et al, 1998; Childress et al, 1999; Everitt and Robbins, 2005; Volkow et al, 2008). Reducing the impact of these cues is postulated to reduce relapse in at-risk populations (Hammersley, 1992; Drummond, 2001; Nic Dhonnchadha and Kantak, 2011), yet extinction therapy has not yet shown high efficacy (Bouton, 2002; O’Brien, 2008).

Cue-induced relapse has been modeled in experimental animals in a variety of ways, where drug-seeking behavior can be precipitated by cues that previously predicted drug availability, despite responding for drug having undergone extensive extinction training (de Wit and Stewart, 1981; Epstein et al, 2006). The return of drug-seeking behavior at test indicates that the expression of extinction has been disrupted and also provides evidence that the original stimulus-reward associations remain intact. Thus, theories of extinction learning suggest that extinction involves active, new learning regarding the prediction of non-rewarded behavior that competes, albeit sometimes unsuccessfully, with the original learning for behavioral control (Pavlov, 1927; Delamater, 1996; Rescorla, 1996, 1997). In the present studies, we tested the hypothesis that strengthening the extinction of drug-paired stimuli may reduce or prevent a normally enduring propensity for drug-seeking behavior to re-emerge.

Both behavioral and pharmacological treatments have been shown to improve extinction of conditioned fear or responding for food (Rescorla, 2000, 2006; Cain et al, 2004; Morris and Bouton, 2007; Janak and Corbit, 2010). Error-correction learning models argue that learning occurs when there is a discrepancy between what is expected, based on the presence of predictors, such as conditioned cues, and what actually occurs (reward, in appetitive conditioning; Kamin, 1968; Rescorla and Wagner, 1972; Schultz and Dickinson, 2000). This account predicts that the presentation of multiple reward predictors followed by the absence of reward (extinction) would generate a larger prediction error than the presentation of a single predictor, alone, and consequently result in improved extinction learning (Janak and Corbit, 2010; Rescorla, 2000, 2006). As this effect has not been tested in a substance abuse context, our first aim was to test whether simultaneous presentation of multiple, non-reinforced stimuli would enhance extinction of a cocaine-seeking response.

Large prediction errors may promote learning by engaging the attentional mechanisms that contribute to the establishment of long-term memories. Neuromodulators, including norepinephrine (NE), have been proposed to regulate learning and memory formation (McGaugh, 2004; Arnsten, 2009). Accordingly, arousal and the associated noradrenergic effects are thought to be particularly important for modifying performance when reinforcement contingencies change, as is the case during extinction (eg, see Sara et al, 1994; Usher et al, 1999). A role for NE in extinction learning has been demonstrated for extinction of conditioned fear (Cain et al, 2004; Morris and Bouton, 2007) and operant responding for food (Janak and Corbit, 2010). Similarly, recent studies found that NE is important for consolidation of extinction in both cocaine-conditioned place preference (Bernardi and Lattal, 2010; Bernardi et al, 2009; Brenhouse et al, 2010) and self-administration procedures (LaLumiere et al, 2010) using post-training administrations of adrenergic drugs. We hypothesized that the benefit to extinction produced by extinction of a compound cue may also rely on NE activity and, therefore, examined if extinction of cocaine seeking could be enhanced or impaired by pre-training manipulations of the NE system.

Notably, evidence suggests that relapse risk remains high or even grows (‘incubates’) over periods of abstinence (Grimm et al, 2001; Bossert et al, 2005; Lu et al, 2004; Bedi et al, 2011). Therefore, rather than evaluating drug-seeking behavior immediately after extinction training, we conducted our tests 4 weeks later to allow potential spontaneous recovery of responding and to assess the longevity of any effects on extinction following this ‘incubation’ period.

Our findings indicate that compound stimulus presentation or reduced NE reuptake during extinction learning reduced the return of drug seeking when animals were subsequently re-exposed to previously cocaine-predictive cues. In addition, the capacity of compound stimulus extinction to reduce drug-seeking behavior was diminished in rats administered the β-adrenergic receptor antagonist propranolol, providing further evidence that NE contributes to this effect.

MATERIALS AND METHODS

The Effects of Extinguishing a Compound Stimulus on Spontaneous Recovery of Cocaine-Seeking Behavior

Subjects and apparatus

The subjects were 12 male Sprague Dawley rats (Charles River, Hollister, CA) weighing approximately 300 g at the beginning of the experiment. In all, 2 animals were excluded for catheter failure or failure to acquire the lever-press response leaving 10 animals in the behavioral analyses. All procedures were approved by the Institutional Animal Care and Use Committee of the Ernest Gallo Clinic and Research Center at the University of California, San Francisco, and were conducted according to the NIH Guide for the Care and Use of Laboratory Animals. Training and testing took place in Med Associates (East Fairfield, VT) operant chambers housed within sound- and light-attenuating shells. The chambers contained retractable levers that could be inserted at the left and right sides of the chamber. The boxes also contained two white key lights (one over each lever), a white noise generator, and a solenoid that, when activated, delivered a 5 Hz clicker stimulus. The auditory stimuli were adjusted to 80 dB in the presence of background noise (60 dB provided by a ventilation fan). A single lever extension and illumination of a houselight mounted on the top-center of the wall opposite the levers signaled the session start. Computers equipped with MED-PC software controlled the equipment and recorded data.

Surgery

Rats were anesthetized with isoflurane (5% induction, 1–2% maintenance in oxygen) and unilateral, 0.64-mm-OD silastic catheters inserted 2.7–3 cm into the right jugular vein. The catheters were secured to the vein with 4-0 silk-braided sutures and then routed subcutaneously to the back, and attached to a coupling assembly externalized between the scapula. Injection pumps (located outside of the sound-attenuating shell) were connected to a liquid swivel (Instech, Plymouth Meeting, PA) via polyethylene (20) tubing encased in steel spring leashes (Plastics One, Roanoke, VA) on a counter-balance arm. These were secured to each rat and allowed free movement in the experimental chambers. Catheters were flushed daily with heparinized saline (0.4 ml of 10 IU) to maintain patency.

Procedure

Lever training

One week after catheter implantation, rats were food-deprived for 24 h before the initial lever training. Thereafter they had free access to food and water while in the home cage. A summary of the training procedures is provided in Figure 1 and the experimental design is outlined in Figure 2a. Rats were trained to self-administer cocaine (Sigma, St Louis, MO) during 2 h experimental sessions. At the beginning of each session the catheter was flushed with 0.2 ml of sterile saline and connected to the infusion leash. In the initial session, lever depression resulted in intravenous cocaine delivery (0.8 mg/kg/2 s infusion, dissolved in sterile saline vehicle). Each drug infusion was followed by a 20 s time out period. During the 20 s post-response interval, the lever was available but responding had no programmed consequences. Rats were maintained on this schedule until they received a minimum of 10 infusions for 2 consecutive days. Not more than 24 infusions were permitted on any single day. At the end of the session, rats were removed from the chambers and the catheters were flushed with heparinized saline before being returned to the home cage.

Figure 1
figure 1

Experimental design. (a) Timeline of the experiments. (b) Summary of the discriminative stimulus procedure. Presentations of the discriminative stimuli (30 s light, white noise and clicker presented in pseudorandom order separated by an inter-trial interval averaging 270 s) signaled the availability of cocaine reward if the lever-press response was performed. At the end of training responding was reinforced according to a random ratio 4 schedule; ie on average, every fourth response during the stimulus-period resulted in cocaine (i.v.; 0.8 mg/kg/2 s infusion). Lever presses in the absence of the stimuli had no programmed consequences.

PowerPoint slide

Full size image
Figure 2
figure 2

Extinction of a stimulus compound reduces spontaneous recovery of cocaine seeking relative to extinction of a single stimulus. (a) Summary of the within-subjects experimental design. All rats were trained that performance of the lever-press response in the presence of any of the three stimuli resulted in cocaine delivery. Responding during all three stimuli was extinguished before trials in which the stimuli were presented either alone or as part of a compound. Finally, spontaneous recovery of responding in the presence of the stimuli that had been extinguished either alone or as part of a compound was assessed 4 weeks later. (b) Mean lever presses performed per 30 s stimulus or in the pre-stimulus interval of equal length (30 s) on the final day of acquisition. Stimulus identity (ie, ‘to-be compound’ or ‘to-be single’) refers to the treatment that would be given in the third extinction session (noise or clicker; counterbalanced). *Responding during the stimuli was significantly greater than during the pre-stimulus interval; ^responding during the to-be single and to-be compound stimuli was not different. (c) Mean lever presses per stimulus for the initial extinction sessions (days 1 and 2) and the single stimulus trials in the first half of day 3. *Responding was significantly reduced from the first extinction session and ^responding to the different stimuli did not differ. (d) Mean lever presses per stimulus during the last half of the final extinction session, in which two of the three stimuli were presented in compound and the remaining stimulus continued to be presented alone, and in the test conducted 4 weeks later. Presentation of a stimulus compound increased responding during extinction; however, when the element of that compound was tested alone 4 weeks later, animals responded less to the stimulus that was extinguished in compound than to the stimulus extinguished alone. No cocaine was delivered during extinction or test. *Indicates a significant difference in number of lever presses; p<0.05; **p<0.01; N=10.

PowerPoint slide

Full size image

Acquisition

The next day discrimination training began. A discriminative stimulus procedure was used because, in contrast to a conditioned stimulus signaling drug delivery, a discriminative stimulus signals drug availability should the appropriate behavioral response be executed. To achieve this, stimulus presentations were spaced throughout the session and only responding during these stimuli was reinforced. Further, the stimulus did not co-terminate with reward delivery and the more presses performed during the stimulus, the higher the probability of reward. Thus, multiple rewards were possible during each trial to promote maximum control of behavior by the stimulus (Weiss et al, 2003). This paradigm may more closely match the circumstances that a recovering addict faces; ie, that drug-associated cues may be encountered periodically but that, once initiated, there is a strong causal relationship between performance of drug-seeking behaviors and drug delivery. Each session contained 8, 30-s presentations of a light (two keylights illuminated), white noise, and clicker stimulus (24 trials in total). During each stimulus, lever pressing resulted in cocaine delivery according to a random ratio (RR) schedule (described below) followed by a 20 s time out imposed following each infusion to avoid overdose. Responses in the absence of the stimuli had no programmed consequence. In order to aid acquisition, on the first discrimination training day, the lever was only available during the stimulus interval, that is it was inserted at the beginning and retracted at the end of each stimulus. On subsequent days the lever was present throughout the session, but responding was only reinforced during stimulus presentations according to a RR schedule that was increased across days (days 1–5: RR1; days 5–10: RR2; days 11–15: RR4). Stimuli were presented in a pseudorandom order and the inter-trial interval was variable but on average was 4.5 min. Rats were required to earn a minimum of 10 infusions during acquisition sessions with the discriminative stimuli for that session to be counted toward the training total of 15 sessions. While not all rats made this criterion in early sessions, once this criterion was met, rats rarely failed to meet criteria in subsequent sessions. Most rats far exceeded this criterion earning the maximum allowable infusions (24) each day. Rats were trained for a total of 15 days wherein they received at least 10 infusions.

Extinction phase 1

On the following 2 days all rats received sessions identical to the acquisition sessions (eight presentations of each of the three stimuli) except that saline rather than cocaine was delivered when the response requirement was met within the stimuli. In anticipation of the upcoming compound stimulus trials, the initial extinction is important as it reduces the chances that a more salient stimulus may overshadow a less salient stimulus thus retarding rather than facilitating extinction learning (for discussion, see Rescorla, 2006). Furthermore, in the following experiments involving pharmacological treatments, initial extinction reduces the likelihood that these treatments will affect reconsolidation of the original learning rather than extinction.

Extinction phase 2

At the end of phase 1, all stimuli had undergone identical treatment. On the following day, the rats received 12 single stimulus trials (4 of each stimulus) followed by 6 further non-reinforced presentations of each of 2 trial types, AX− and Y−. The two auditory stimuli served as stimuli X and Y in a counterbalanced fashion and the light stimulus served as stimulus A. The entire session had 24 trials as in previous sessions and each stimulus had an equal number of presentations.

Test

Rats remained in their home cage for 4 weeks before testing. The catheters were flushed daily during this period. The test session contained four presentations each of the white noise and clicker stimuli in pseudorandom order. The rats received infusions of saline during this session if their catheters remained patent. Otherwise they were attached to the leash assembly and the pumps were activated as in training, but the syringe was disconnected from the pump.

Does Atomoxetine Treatment Mimic the Effects of Compound Stimulus Presentation on Extinction of Cocaine-Seeking Behavior?

Subjects and apparatus

A total of 24 male Sprague Dawley rats were maintained and trained in the same manner as those above. In all, 3 animals were excluded because of catheter failure or failure to acquire response, leaving 11 animals in the atomoxetine group and 10 animals in the saline group.

Procedure

Acquisition and extinction phase 1

To facilitate acquisition, rats received a single lever-training session (in the absence of any cue) wherein a lever-press response was reinforced with a single, 45 mg food pellet (BioServ, Frenchtown, NJ) until the rats earned a maximum of 50 pellets. All subsequent training, including lever training, acquisition, and extinction phase 1, were identical to those described above. The rats received a saline injection before the final acquisition session in order to habituate them to the injection procedure.

Extinction phase 2

On the basis of lever responding in training and the previous extinction sessions, rats were divided into two groups with approximately even response rates (Figure 3b). On the following day, rats received an injection of either saline (1 ml/kg, ip) or atomoxetine (1 mg/kg, ip; Tocris, Ellisville, MO; dissolved in saline and administered in a volume of 1 ml/kg) 45 min before the beginning of the third extinction session. This dose was chosen based on previous studies that found this dose to be effective in reducing spontaneous recovery of responding for food (Janak and Corbit, 2010). We predicted that atomoxetine treatment would augment stimulus processing and extinction learning, and so the rats received further extinction of a single stimulus (six trials; clicker or noise balanced within groups and total responding for the assigned stimulus in previous extinction sessions was approximately equated across groups).

Figure 3
figure 3

Atomoxetine treatment enhances extinction of cocaine seeking. (a) Summary of the experimental design. (b) Mean lever presses performed per 30 s stimulus or in the 30 s pre-stimulus interval on the final acquisition day for rats in the saline or atomoxetine groups; note that atomoxetine was given 45 min prior to the third extinction session. *Responding during the stimuli was significantly greater than during the pre-stimulus interval and ^responding during the stimuli was not different between groups. (c) Mean lever presses per stimulus for the initial extinction sessions; atomoxetine was given 45 min before the third extinction session. *Responding was significantly lower than on day 1. ^Responding did not differ between groups. (d) Mean lever presses per stimulus during the final extinction session following atomoxetine treatment and during the test session conducted drug-free 4 weeks later. Rats treated with atomoxetine showed reduced responding at test compared with rats treated with saline. **Indicates a significant difference in the number of lever presses; p<0.01. N=11 for the atomoxetine group and 10 for the saline group.

PowerPoint slide

Full size image

Test

Animals remained in their home cage and catheters were flushed daily until the test for spontaneous recovery of responding 4 weeks later. The test consisted of six presentations of the same stimulus that was presented in the final extinction session and the number of lever-press responses were recorded.

Does Propranolol Block the Benefit to Extinction Achieved Through Extinction of a Stimulus Compound?

Procedure

Acquisition and extinction phase 1: A total of 24 male Sprague Dawley rats were maintained and trained in an identical fashion to that described above until propranolol treatment just before the final extinction session. In all, 4 animals were excluded for catheter failure leaving 10 animals in each group. Our working hypothesis was that the compound stimulus presentation increases noradrenergic activity and reducing this effect (via blockade of β-adrenergic receptors) would occlude the benefit of compound stimuli to extinction. To test this hypothesis, groups of rats were given either 0 or 5 mg/kg of propranolol (propranolol hydrochloride; Sigma; dissolved in sterile water) before a final extinction session containing six presentations of a stimulus compound (click or noise paired with light; balanced within groups). This dose was chosen based on our previous studies that found 5.0 mg/kg to significantly reduce the impact of compound stimulus presentation on extinction of responding for food reward (Janak and Corbit, 2010).

Test: Animals were tested 4 weeks later for spontaneous recovery of responding. The test consisted of six presentations of the auditory stimulus that had been part of the compound given in the final extinction session.

Data analyses: Data are presented as means (±SEM). Data were analyzed with repeated measures analysis of variance (ANOVA) or one-way ANOVA to further examine significant main effects or interactions.

RESULTS

Compound Stimulus Extinction Reduces Spontaneous Recovery of Responding for Cocaine

The aim of this experiment was to test whether presentation of a compound of extinguished stimuli would improve extinction learning, reducing future spontaneous recovery, when compared with extinction of a single stimulus alone. Rats underwent 15 days of acquisition training wherein a lever-press response in the presence of one of three discrete stimuli resulted in the delivery of cocaine (0.8 mg/kg/infusion, i.v.). On the final day of acquisition, the mean lever presses per stimulus (±SEM) were 3.4±0.5, 3.6±0.4, and 4.0±0.6 during the light, white noise, and clicker stimuli, respectively. There were no reliable differences in responding between the light, click, or white noise stimuli during acquisition [F<1]. Responding during the pre-stimulus period (30 s) was 0.8±0.25. These data are illustrated in Figure 2b as the mean response rates for the stimuli that would subsequently be presented as either single or compound during extinction.

Reberg (1972) noted that even when two stimuli have themselves been extinguished individually, presenting them together, in compound, yielded greater responding than that observed to either stimulus presented alone indicating that extinction does not entirely eliminate the original excitatory conditioning. These data suggest that compound stimulus presentation potentially reintroduces a prediction error even for previously extinguished stimuli that could, in turn, strengthen extinction learning beyond what might be achieved through extinction of the stimuli alone. As such, rats subsequently underwent two extinction sessions that were identical to the acquisition sessions except that saline was substituted for cocaine. In the third extinction session, conditions were modified such that rats received 12 single stimulus trials (4 of each stimulus) followed by 6 further non-reinforced presentations of a compound stimulus (AX−) interspersed with 6 individual stimuli (Y−) presentations. Thus, two of the stimuli (A & X) were extinguished together (in compound), whereas the third stimulus (Y) was always presented alone. Importantly, all stimuli were presented an identical number of times during acquisition and both phases of extinction training, thus, the only difference between stimuli was that for stimulus X some of the trials were in compound with stimulus A, whereas stimulus Y always occurred alone. The efficacy of extinction of the compound stimulus (AX) vs the single stimulus (Y) was assessed 4 weeks later by returning the animals to the self-administration chambers and examining spontaneous recovery of responding in the presence of either of two stimuli (X vs Y; see Figure 2a).

Data in Figure 2c illustrate that responding in the presence of all three stimuli was significantly decreased during extinction training. Importantly, during the initial extinction sessions (before compound stimulus trials), responding during each of the stimuli when presented singly was low and not significantly different [Figure 2c; F(1,9)=2.1, p>0.05]. The data of primary interest are presented in Figure 2d, which illustrates responding during the final extinction trials and test for spontaneous recovery of responding conducted 4 weeks later. As predicted by the work of Reberg (1972), data shown in Figure 2d, the final extinction day, responding to the single stimulus (Y) remained low, whereas responding to the compound (AX) was higher. Critically, the pattern reversed at test where responding to the stimulus that had been presented as part of the compound (X) was low and responding to the stimulus extinguished alone (Y) increased. This description is confirmed by the statistical analyses, which revealed no effect of experimental phase [extinction vs test; F(1,9)=0.07, p<0.05] or stimulus [F(1,9)=1.1, p>0.05], but a significant interaction between these factors [F(1,9)=11.0, p<0.01]. To further explore this interaction we compared responding with the two stimulus types in each phase. During extinction, responding was higher for the compound than for the single stimulus [F(1,9)=6.7, p<0.05]. In contrast, at test responding was lower for the stimulus extinguished as part of a compound compared with the stimulus extinguished alone [F(1,9)=14.5, p<0.01]. This analysis revealed that extinction of a stimulus compound can improve extinction and reduce spontaneous recovery of responding to the elements of that compound (X) relative to spontaneous recovery in response to a stimulus (Y) previously extinguished alone.

Augmenting NE Enables Extinction of a Single Stimulus to Reduce Spontaneous Recovery of Responding for Cocaine

Prior studies have found evidence that NE facilitates extinction of conditioned fear (Cain et al, 2004; Morris and Bouton, 2007) or instrumental responding for food under a protocol analogous to the present studies (Janak and Corbit, 2010). We therefore tested the hypothesis that extinction of cocaine seeking would also be enhanced by augmenting the noradrenergic system with the NE reuptake inhibitor, atomoxetine.

A separate cohort of rats was trained to respond for cocaine reinforcement in the discriminative stimulus procedure and then underwent 2 days of extinction training where each stimulus was presented individually, as described above (see Figure 3a). On the following day, rats were divided into two groups matched for their response rates in the initial extinction sessions and received either atomoxetine (1.0 mg/kg, ip) or saline (1 ml/kg, ip) 45 min before the third extinction session where, in contrast to Experiment 1, a single stimulus (counterbalanced white noise or clicker) was extinguished alone. We reasoned that if compound stimulus presentation during extinction acts by increased noradrenergic activity, which ultimately benefits extinction learning, administering a drug that increases noradrenergic activity should mimic the effects seen with extinction of a compound stimulus despite animals undergoing extinction of a single cue. Response rates at the end of acquisition and initial extinction are shown in Figure 3b and c. Comparison of drug treatment effects on responding in extinction vs when the rats were tested 4 weeks later, shown in Figure 3d, indicated no effect of group [F(1,19)=2.4, p>0.05]; however, there was an effect of experimental phase [F(1,19)=10.5, p<0.01] and an interaction between these factors [F(1,19)=6.5, p<0.05]. As we were interested in knowing whether atomoxetine treatment during extinction would prevent future spontaneous recovery, we also compared responding in extinction to that seen at test (4 weeks later) for each treatment group. This analysis confirmed an increase in responding by rats treated with saline [F(1,9)=10.8, p<0.01], but not for those treated with atomoxetine, which failed to show spontaneous recovery at test 4 weeks later [F(1,10)<1]. These results confirm our prediction that atomoxetine given before extinction training prevents later spontaneous recovery of cocaine-seeking behavior.

Reduced Spontaneous Recovery Following Compound Stimulus Extinction is Dependent upon β-Adrenergic Receptor Activation

β-Adrenergic receptor activation contributes to post-synaptic changes that underlie NE modulation of long-term memory formation (Sullivan et al, 2000; Abraham et al, 2008; Pu et al, 2009). Further, β-receptor antagonists, such as propranolol, block the beneficial memory effects typically seen with NE activation (Cahill et al, 1994; Berlau and McGaugh, 2006). The results observed following treatment with atomoxetine before extinction suggest that increasing NE availability can enhance extinction learning, similar to the results observed following extinction of a stimulus compound. Nonetheless, these results do not provide direct evidence that compound stimulus presentation recruits noradrenergic activity. We next tested the hypothesis that the benefit to extinction achieved with compound stimulus presentation requires β-adrenergic receptor activation by examining if the effects of compound stimulus presentation, observed above, can be blocked by the β-adrenergic receptor antagonist propranolol (Figure 4a). Thus, we predicted that rats pre-treated with propranolol (5 mg/kg, ip; dose based on our previous work) before the last extinction day would show greater spontaneous recovery at test 4 weeks later than those treated with saline (1 ml/kg, ip) despite compound extinction training.

Figure 4
figure 4

Propranolol treatment blocks the benefit to extinction produced by extinction of a stimulus compound. (a) Summary of the experimental design. (b) Mean lever presses performed per 30 s stimulus or in the 30 s pre-stimulus interval on the final acquisition day for rats in the saline and propranolol groups; note that propranolol treatments were given 30 min before the third extinction session. *Responding during the stimuli was significantly greater than during the pre-stimulus interval and ^responding during the stimuli was not different between groups. (c) Mean lever presses per stimulus for the initial extinction sessions. *Responding was significantly lower than on day 1. ^Responding did not differ between groups. (d) Mean lever presses per stimulus during the final extinction session, in which two of the three stimuli were presented in compound, and during the test session, in which the stimuli were presented alone, conducted drug-free 4 weeks later. In saline treated animals, compound stimulus presentation produced robust responding in extinction and reduced responding when the element was tested 4 weeks later. In contrast, although compound stimulus presentation still produced robust responding in extinction, the group treated with 5.0 mg/kg of propranolol showed greater responding when the element of the compound subsequently tested drug-free. *p<0.05, N=10 per group.

PowerPoint slide

Full size image

Responding decreased across extinction days (Figure 4c). Propranolol treatment, itself, did not affect responding as both groups showed similar response rates during the compound stimulus presentations on extinction day 3 (Figure 4d). However, examination of the test data (Figure 4d) reveals that animals previously treated with saline show little responding in the test session, but that rats previously treated with propranolol show greater spontaneous recovery. Accordingly, the statistical analyses reveal no effect of experiment phase [F(1,18)<1] or group [F(1,18)=3.9, p>0.05], but an interaction between these factors was indicated [F(1,18)=5.2, p<0.05]. As the rats received a stimulus compound in extinction and a single stimulus at test, there was a possibility that response rates would differ between phases regardless of experimental treatment; thus, to further explore the observed interaction, we compared performance of the two groups in extinction and at test. There was no difference between rats treated with saline and those treated with propranolol in response to the compound in the final extinction session [F(1,18)<1]. However, when the impact of this treatment was assessed 4 weeks later, rats previously given propranolol responded significantly more than those treated with saline [F(1,18)=7.5, p<0.05], suggesting that propranolol interferes with the benefit that compound stimulus extinction would otherwise provide for improving extinction learning.

DISCUSSION

Consistent with error-correction models of learning, we found that presentation of a stimulus compound during extinction improved extinction of cocaine-seeking behavior evidenced by decreased spontaneous recovery observed 4 weeks later compared with an equivalent number of single stimulus presentations. Previous work has shown that stimulus compounds have important effects on behavior, for example, producing large reinstatement effects (Panlilio et al, 1996; See et al, 1999), which indicates increased behavioral arousal. However, the impact of such treatment on subsequent responding on extinction learning per se has not been investigated. The current data extend previous work with food or shock reinforcement, suggesting that this effect that can be observed over a wide range of behaviors and reinforcers (Rescorla, 2006; Janak and Corbit, 2010). Further, this effect appears to be mediated, at least in part, via β-adrenergic receptor activation.

Salient events or changing environmental contingencies, including extinction, have been shown to increase firing rates of noradrenergic neurons and NE efflux in forebrain structures (Foote et al, 1980; Sara and Segal, 1991; Mingote et al, 2004). In one study, locus coeruleus (the major source of forebrain NE) firing was increased following a shift to an extinction contingency, but this response rapidly decreased as extinction trials continued (Sara et al, 1994). These data raise the possibility that locus coeruleus activity and consequent NE release is low during prolonged extinction. Given the role of NE in allocation of attention and formation of long-term memories, low NE activity may partially explain why extinction learning is not as stable as excitatory learning across time. In partial support to this hypothesis, extinction of non-drug reinforcers was enhanced by drug treatments that increase NE (Cain et al. 2004; Janak and Corbit, 2010). Accordingly, we hypothesized that enhancing NE during extinction training would improve extinction of cue-driven cocaine-seeking behavior. Consistent with this prediction we observed that the NE reuptake blocker atomoxetine augmented the extinction of a single stimulus and reduced spontaneous recovery of cocaine-seeking behavior. These findings compliment and extend previous demonstrations that atomoxetine reduces both cue-supported responding under a second-order schedule of reinforcement and cue-induced reinstatement (Economidou et al, 2011) by demonstrating that the effects of atomoxetine persist even when animals are tested drug-free.

It is possible that the compound stimulus effects described above might also be explained by increased NE activity. Specifically, as the response of locus coeruleus neurons under extinction conditions is transient (Sara et al, 1994), single stimulus trials may only weakly activate locus coeruleus neurons. Thus, we predicted that the novel introduction of a stimulus compound may produce a greater NE response. To test this hypothesis, in experiment 3, we demonstrated that administration of propranolol, an β-adrenergic receptor antagonist, before the critical compound stimulus extinction trials diminished the benefit of the compound. Together, these data suggest that the enhancement of extinction learning following compound stimulus presentations may critically rely on NE activity at β-adrenergic receptors. We have previously shown that propranolol treatment before extinction of responding for food under a single stimulus is without effect, and so it is unlikely that the effect observed with the stimulus compound is due to some general impairment in extinction (Janak and Corbit, 2010). Rather, these results suggest that the stimulus compound is better able to recruit NE activity, and that the ability of the stimulus compound to enhance extinction requires NE activity.

Intriguingly, the ability of atomoxetine to enhance extinction of cocaine-seeking behavior may seem to be in apparent contrast to previous studies in which yohimbine, an α2-adrenergic receptor antagonist that increases NE release, has been used as a pharmacological stressor. In fact, yohimbine has been used to produce reinstatement (Lee et al, 2004; Shepard et al, 2004) as well as potentiate the cue-induced reinstatement of cocaine seeking (Feltenstein and See, 2006). The ability of yohimbine or footshock, but not cocaine itself, to reinstate cocaine seeking can be attenuated by treatment with propranolol, α2-receptor agonists, or α1-receptor antagonists indicating reliance on noradrenergic activity (Shaham et al, 2000; Mantsch et al, 2010; Le et al, 2011). While we did not observe an increase in cocaine seeking following administration of atomoxetine, some concern remains over whether drug treatments that augment noradrenergic activity may pose a risk rather than a benefit in promoting abstinence in recovering addicts. Importantly, yohimbine and atomoxetine differ in their mechanisms of action. Whereas yohimbine leads to widespread NE release including in brain regions implicated in stress (eg, the extended amygdala and bed nucleus of the stria terminalis; Forray et al, 1997; Galvez et al, 1996; Shaham et al, 2000), atomoxetine, by blocking reuptake, increases synaptic availability of NE only in regions where it is released endogenously in relation to environmental events. Moreover, both conditioned cues, footshock, and yohimbine can induce reinstatement and so it is not surprising that there is some, but not complete, overlap in the brain regions that mediate this capacity (eg, see Buffalari and See, 2011). Indeed, distinct corticostriatal pathways control the promotion and inhibition of cocaine seeking (Peters et al, 2008). Thus, as the neural circuitry mediating extinction is engaged, NE may preferentially promote plasticity related to extinction rather than drug-seeking. Future studies that identify the brain regions required for the extinction-enhancing effects that we report here vs those mediating stress-induced reinstatement will be particularly interesting in this regard.

As the ability of drugs such as yohimbine to induce reinstatement is clear, the long-term effects of such treatments are rarely examined. Of note, yohimbine has been shown to slow extinction of cocaine behaviors (Davis et al 2008; Kupferschmidt et al, 2009), which could pose a significant therapeutic limitation; yet, following repeated administration, a reduced capacity to produce a subsequent reinstatement has been observed (Kupferschmidt et al, 2009). Thus, a certain amount of arousal may be required to establish lasting extinction memories, which if carefully monitored within a clinical setting, may permit drugs such as atomoxetine to safely promote abstinence.

Learning, Consolidation, and Reconsolidation

Previous work in both aversive and appetitive procedures has demonstrated that noradrenergic activity is important not only for memory acquisition but also for post-training memory processes. For example, LaLumiere et al (2010) found that repeated post-training administration of the β2-adrenergic agonist clenbuterol within the infralimbic cortex enhanced retention of extinction, as evidenced by lower responding in subsequent sessions. Other past work has found that interference with the NE system can disrupt stable memory formation as post-retrieval administration of propranolol can impair cocaine-conditioned place preference (Bernardi et al, 2006, 2009; Fricks-Gleason and Marshall, 2008) or conditioned reinforcement (Milton et al, 2008). These effects have been attributed to disrupted reconsolidation. Thus, it is clear that the NE system is important at multiple stages of memory formation. The current results extend those found with post-training drug treatments that target the consolidation or reconsolidation of extinction and suggest that a single pre-training treatment that activates the NE system can also promote the extinction of a cocaine-seeking response and its stimulus control and that this effect is persistent, lasting at least 4 weeks.

Together with the memory enhancements seen with post-training augmentation of NE signaling, it is likely that there is some increase in endogenous NE activity during extinction training, blockade of which impairs extinction, and that post-training pharmacological activation of β-adrenergic receptors is additive to that occurring in the session itself (LaLumiere et al, 2010). Thus, it could be argued that the current results can be explained by enhanced consolidation. To test this possibility, we directly compared the effects of pre- vs post-training administration of atomoxetine on spontaneous recovery of food-seeking (Janak and Corbit, 2010) and found that while post-training treatment reduced spontaneous recovery relative to animals treated with saline (consistent with improved consolidation) rats that received pre-training atomoxetine showed significantly less spontaneous recovery than those receiving the same dose following the extinction session. This result is difficult to explain solely in terms of enhanced consolidation and strongly suggests that NE signaling can also enhance extinction learning. This is consistent with previous demonstrations that pre-training but not post-training β-adrenergic receptor blockade can impair extinction (Mueller et al, 2008; LaLumiere et al, 2010). Together, these results are most consistent with the view that NE can promote learning-related plasticity, but that the type of learning that is either enhanced or impaired will depend on the details of the experimental design. For example, it has previously been shown that the same pharmacological agent can either promote or impair memory (by acting on either the extinction or reconsolidation process) based upon the amount of memory reactivation resulting from long or short training sessions (see Lee et al, 2006).

Receptor Specificity

NE function involves actions distributed across multiple receptor subtypes with different anatomical distributions and, as such, the response of different brain regions to NE or selective agonists or antagonists will depend on the local receptor compliment. These differences allow for local specificity of NE action despite widespread release of NE (Berridge and Waterhouse, 2003), but nonetheless complicate the investigation of NE effects as well as the choice of an ideal therapeutic agent.

An increasing number of adrenergic compounds have been tested and found to have some effect on extinction (for a recent review, see Mueller and Cahill, 2010). Importantly, many of these effects ultimately rely on action at the β-adrenergic receptors (Ferry et al, 1999; Roozendaal et al, 2002; Berlau and McGaugh, 2006; Mueller et al, 2008; LaLumiere et al, 2010). Thus, taken together with our propranolol results, it is likely that the results reported here with atomoxetine result from increases in synaptic NE levels that allow for greater potential activity at the β-adrenergic receptor. Accordingly, we would predict that direct β-adrenergic receptor agonists, such as clenbuterol, would have similar effects.

Clinical Applications

One of the obstacles for drug abuse treatment is that risk of relapse to drug use is high even following long abstinence periods (McLellan et al, 2000). Drug-associated cues are one factor thought to contribute to relapse risk and, thus, reducing the ability of such stimuli to trigger drug-seeking behaviors could significantly improve treatment outcomes (Marlatt, 1990; O’Brien et al, 1998; Taylor et al, 2009). A major goal of extinction research is to discover methods for reducing the capacity of drug-associated cues to elicit drug-seeking behavior by strengthening extinction learning. The rat experiments modeling cocaine addiction described here suggest that increasing behavioral arousal through the presentation of a stimulus compound or increasing NE activity through atomoxetine treatment can enhance the extinction of cocaine-related stimuli and reduce the spontaneous recovery of cocaine-seeking behavior at least a month later.

Together, these data suggest that either direct or indirect NE agonists administered in conjunction with extinction-based therapies could improve extinction of drug-associated cues and thus the efficacy or longevity of these treatments to abate relapse in human addicts. In relation to the indirect NE agonist atomoxetine, it is interesting to note that chronic cocaine self-administration has been shown to upregulate the NE transporter, which may result in less available NE under conditions where drug seeking is often expressed (Macey et al, 2003). Further this NE transporter upregulation may also manifest as some of the cognitive impairments that are apparent during early abstinence (Pace-Schott et al, 2008; Volkow et al, 2011), which can decrease the efficacy of cognitive-based treatment received during early abstinence. Accordingly, increasing NE activity, for example with atomoxetine, may ameliorate these cognitive deficits and improve the efficacy of cognitive-based therapy. A further benefit of a drug, such as atomoxetine, particularly when administered systemically, is that it would amplify the endogenous NE signal by blocking reuptake in locations with active NE release in relation to the behavioral task rather than indiscriminately activating NE receptors throughout the brain. In a clinical setting, this could result in a better-tolerated side-effect profile. This distinction is also important because NE augments both glutamatergic and GABAergic signaling, and so it is difficult to predict the effect of universally increasing potentially opposing pathways with a direct agonist. Finally, the identification of drugs that impair extinction, such as propranolol, could be important for avoiding drug treatments that could be counterproductive to other therapeutic interventions.

CONCLUSIONS

We find that the extinction of cocaine-paired stimuli can be enhanced by concurrent presentation of multiple cocaine-associated stimuli, in compound, as well as through reduced NE reuptake. The fact that a single administration of an indirect NE agonist can facilitate behavioral effects at least 1 month later indicates that NE activity can powerfully influence the stability and longevity of extinction learning. Future studies are needed to identify the neuroanatomical locus of these effects.