INTRODUCTION

Opioid addicts show attentional bias and drug craving in response to drug-related cues, phenomena that may contribute significantly to relapse vulnerability following abstinence (Hyman et al, 2007; Lubman et al, 2008; Marissen et al, 2006). Similarly, rats with a history of chronic morphine or cocaine exposure show increased conditioned place preference (CPP) for drug-associated stimuli as compared with naive rats (Harris and Aston-Jones, 2001, 2003a; Harris et al, 2007b). In contrast, preference is decreased for cues associated with natural rewards such as food during drug abstinence in rats (Harris and Aston-Jones, 2003b; Harris et al, 2007b). This altered hedonia model mirrors the concomitant reduction in interest for natural rewards observed in human opioid addicts, and may reflect an important aspect of the addiction process, particularly given the persistent nature of these adaptations even after protracted periods of abstinence (Lubman et al, 2009). The current studies investigated the behavioral mechanisms underlying increased interest in drug-related stimuli during drug abstinence.

One possible explanation for increased drug-induced place preference during abstinence is that the negative affective state of abstinence directly causes increased preference for drug-associated cues, even if the drug-cue associations were formed before dependence. An alternative explanation is that a motivational shift during abstinence increases the acquisition of drug-cue associations due to intensification of the drug’s rewarding properties. In this case, enhanced drug-induced place preference would be expressed only if cues are paired with drug during abstinence because the animal must re-experience the drug during the abstinent state for the incentive value of the reward to be learned and updated. This would indicate an important role for incentive learning, terminology used to describe the process by which a shift in motivation causes changes to instrumental performance, but only if an animal has previous experience with the reinforcer in that particular motivational state (Balleine, 2001; Dickinson and Balleine, 2002). This incentive-motivational theory is supported by the finding that hungry rats only show increased food-seeking in an instrumental conditioning paradigm if they previously had the opportunity to self-administer food in the deprived state (Balleine, 1992). Similarly, animals trained to self-administer heroin show increased drug-seeking during withdrawal only if they previously were given the opportunity to self-administer heroin in a state of withdrawal (Hutcheson et al, 2001). In this view, abstinence/withdrawal creates a motivation state that enhances the incentive value of drug, increasing subsequent drug-seeking and preference for cues associated with the drug. However, Pavlovian-conditioned responses show direct sensitivity to shifts in primary motivation (Balleine, 2005; Balleine, 2001). For example, hungry rats show increased magazine entries (a Pavlovian association) in an instrumental conditioning paradigm without previous exposure to food during deprivation, despite no change in instrumental responding (Balleine, 1992).

The current studies sought to determine whether incentive learning in a Pavlovian conditioning paradigm underlies increased preference for morphine-associated stimuli during protracted abstinence (Harris and Aston-Jones, 2001, 2003a). We investigated whether such increased preference is dependent upon the occurrence of morphine-cue conditioning during protracted abstinence, or whether morphine-cue conditioning before dependence and abstinence is sufficient for subsequent enhancement of preference when tested in the abstinent state. In our laboratory’s previous studies of increased morphine place preference during abstinence, rats first underwent 2 weeks of drug dependence (by subcutaneous morphine pellets or placebo pellets for control animals) and 2–5 weeks of forced drug abstinence, followed by morphine place conditioning (Harris and Aston-Jones, 2003a). Here, rats underwent morphine place conditioning first, and then were exposed to morphine dependence and abstinence; tests for morphine preference were given every 1–2 weeks to monitor changes in preference throughout dependence and abstinence (with no additional drug-cue conditioning). We also investigated whether the priming and locomotor-depressant properties of acute morphine are altered during protracted abstinence to further determine how drug responses are changed following dependence and abstinence. Results show that drug abstinence alone was not sufficient to drive increased preference for morphine-associated stimuli, but that incentive learning during Pavlovian conditioning in the abstinent state was needed to produce the increased response. We also show that incentive learning processes are operative with long-term shifts in motivation state, during protracted drug abstinence.

MATERIALS AND METHODS

Animals

Male Sprague Dawley rats (initial weight 250–300 g; Harlan, Indianapolis, IN) were group-housed (2–3 per cage) in an animal facility at the University of Pennsylvania, under a 12-h normal light/dark cycle with food and water available ad libitum. All experiments were approved by the Institutional Animal Care and Use Committee at the University of Pennsylvania and conducted according to specifications of the National Institutes of Health as outlined in the Guide for the Care and Use of Laboratory Animals.

Drugs

Morphine pellets (75 mg each), placebo pellets, and morphine sulfate powder were provided by the National Institute on Drug Abuse. Morphine sulfate powder was dissolved in sterile saline and administered at 8 mg/kg (i.p.).

Conditioned Place Preference

The CPP apparatus was constructed out of PVC and contained two distinct chambers separated by a removable partition. One chamber had a grid floor with black walls, and the other had an expanded metal (mesh) floor with black and white stripes on the walls. On preconditioning and test days, the partition had an opening to allow the rat access to both chambers. On conditioning days, this was replaced by a solid partition. Each chamber was equipped with four photobeams (Med Associates, St Albans, VT) to record time and activity automatically. During a preconditioning session, rats were allowed to freely explore both chambers for 15 min to ensure no initial bias. Rats were then assigned a chamber to be paired with morphine in an unbiased counterbalanced design. Rats were given either morphine (8 mg/kg, i.p.) or saline (1 ml/kg) and immediately confined to one chamber for 30 min; these injections alternated over 6 or 8 conditioning sessions with drug or saline always given in the same chamber (six sessions: 3 days of twice daily injections of morphine, or saline alternated in the morning and afternoon; eight sessions: 8 days of daily injections of morphine or saline alternated across days). There was no difference in preference between groups that received 6 or 8 sessions, and we combined data here for these two procedures; this is consistent with previous findings (Bardo et al, 1995). Two days after conditioning, animals were tested for preference by allowing access to both chambers for 15 min.

Morphine Dependence

Two 75-mg morphine pellets were subcutaneously implanted under halothane or isofluorane anesthesia (4–5% for the 3-min procedure). Non-dependent rats were implanted at the same time with placebo pellets. Signs of spontaneous physical withdrawal typically began 10–14 days after implantation as the pellets dissolved (Gold et al, 1994); the remnants of the pellets were removed after 14 days. Animals then underwent forced abstinence for up to 8 weeks.

Experimental Design

Following acclimation to the housing facility, rats were exposed to the CPP apparatus during a preconditioning session, 6 or 8 morphine/saline conditioning sessions, and a test session (Figure 1). Two days later, they were implanted with morphine or placebo pellets (to induce dependence in morphine-pelleted animals) for 2 weeks, followed by forced abstinence. Previous studies show that signs of dependence in morphine-pelleted rats begin to dissipate 10–14 days after implantation, as the pellets dissolve (Gold et al, 1994). Rats were periodically tested in the CPP apparatus throughout abstinence to assess changes in expression of morphine place preference during dependence or abstinence. Morphine place preference expression is persistent and does not show extinction when tested every 2 weeks, over at least 12 weeks (Mueller et al, 2002). Here, rats were tested after 1 week of dependence, 1 week of abstinence, and 3 weeks of abstinence. Immediately before a preference test after 4 weeks of abstinence, animals were given a morphine prime (8 mg/kg) to assess the cueing properties of morphine; such a prime typically results in an increase in the amount of time spent in the morphine-associated chamber (Harris and Aston-Jones, 2001). They were then tested in a drug-free state again at 4 or 6 weeks of abstinence. Animals were subsequently re-conditioned; that is, they were given 6 or 8 additional morphine/saline conditioning sessions (in which morphine was paired with the same chamber as initially), and tested for morphine place preference once again.

Figure 1
figure 1

Increased preference for a morphine-paired environment is expressed only when drug-cue conditioning occurs during the abstinent state. Upper panel (experimental timeline): Rats were given initial morphine place conditioning (CPP, 8 mg/kg), followed by 2 weeks of morphine dependence (by subcutaneous morphine pellets) and abstinence; control animals were implanted with placebo pellets. Animals were then re-conditioned for morphine place preference after 6–8 weeks of abstinence. Middle panel: Rats were tested for morphine preference (CPP tests) every 1–2 weeks throughout dependence and abstinence. The arrows for tests A-G align with the timeline above and with the preference scores below. Lower panel: Means (±SEMs) for morphine place preference (amount of time spent on morphine-paired side minus time spent on saline-paired side, in sec) are shown for morphine- and placebo-pelleted animals during 15-min preference tests (left y axis). The right y axis shows the corresponding time spent in the morphine-paired chamber during the 15-min (900-sec) tests. There were no differences between morphine- and placebo-pelleted rats (n=14 each) for expression of morphine place preference following the initial conditioning sessions (pre-dependence) while the pellets were implanted (dependence), or during protracted abstinence (1, 3, and 4–6 weeks). However, post-dependent rats (morphine-pelleted) showed enhanced morphine place preference following the re-conditioning sessions at 6–8 weeks of protracted abstinence as compared with their previous test (**p<0.01) and as compared with non-dependent rats (placebo-pelleted; #p<0.05). Placebo-pelleted rats did not show a change in morphine place preference following re-conditioning.

PowerPoint slide

Full size image

Statistical Analyses

We used the terminology ‘conditioned place preference’ for within-group comparisons of the time in each chamber in the preconditioning test vs the initial test following conditioning; these comparisons were done using a mixed-model ANOVA. In complementary but distinct analyses, ‘morphine place preference’ is the terminology used for the time spent in the morphine-paired chamber minus the time spent in the saline-paired chamber. For between-group comparisons, morphine place preference and locomotor data were analyzed using mixed-model ANOVAs to compare the two groups (morphine- and placebo-pelleted) across multiple test sessions (with test session as a repeated measure), with Bonferroni post hoc analysis. Unpaired two-tailed t-tests were used to compare the two groups within a single test session.

RESULTS

Both groups of rats (morphine- and placebo-pelleted) showed a CPP after initial conditioning with morphine. There was a significant difference for time spent in the drug- vs saline-paired chambers between the preconditioning baseline and initial postconditioning test (p<0.001; placebo F1,26=34.20; morphine F1,26=18.01). For both groups, post hoc analyses revealed no difference in chamber times during the preconditioning baseline (p>0.05), but a significant difference in chamber times post conditioning (p<0.001).

In our paradigm, rats conditioned only before dependence did not exhibit more place preference (morphine side compared with saline side) than non-dependent rats when tested at any time during morphine dependence or subsequent abstinence. However, these same dependent animals expressed substantially increased morphine place preference after being re-conditioned during protracted abstinence, so that they experienced the drug-cue pairings in the protracted abstinence state (Figure 1). Similar re-conditioning evoked no increase in morphine place preference in the non-dependent group (Figure 1).

Before morphine or placebo pellet implantation, there was no difference between the two groups in morphine place preference after initial conditioning (n=14 in each group). Following pellet implantation, the two groups continued to show no significant differences in morphine place preference during dependence or abstinence, when tested every 1–2 weeks. However, re-conditioning after 6–8 weeks of abstinence revealed a marked difference between groups: post-dependent rats (previously morphine-pelleted) showed significantly increased morphine preference, both as compared with the previous test session and non-dependent rats. A comparison of place preferences on the last two tests showed a significant effect for test session between groups (F1,26=5.16; p=0.032) and a significant interaction between test session and group (F1,26=8.04; p=0.009). Post hoc analyses showed that only post-dependent rats had significantly enhanced preference after re-conditioning, as compared with their previous test (p<0.01). Unpaired t-tests for each of the six test sessions showed that post-dependent rats had significantly different preference from non-dependent rats (previously placebo-pelleted) only after re-conditioning (t26=2.12; p=0.044; Figure 1). There were no differences between groups for locomotor activity (number of beam breaks) during the CPP test sessions following initial conditioning (p=0.36) or re-conditioning (p=0.88).

To determine whether the cueing properties of acute morphine are altered during protracted abstinence, these same rats and 15 additional rats were given a priming injection of morphine immediately before a CPP test at 4 weeks of abstinence, before they received morphine CPP re-conditioning (between tests E and F in Figure 1). Typically, this injection of morphine ‘primes’ rats to spend more time in the morphine-paired chamber (Harris and Aston-Jones, 2001). Results showed that non-dependent rats (previously placebo-pelleted; n=22) had enhanced morphine place preference during the morphine-prime test session as compared with their previous test, whereas post-dependent rats (previously morphine-pelleted; n=21) did not. ANOVA revealed an effect of test session (F1,41=17.06; p=0.0002), and an interaction between test session and group (F1,41=7.65; p=0.0085), and post hoc analyses showed a significant effect of test session only in non-dependent rats (p<0.001). This confirms that only the non-dependent animals showed a prime-induced increase in morphine preference. A comparison of the increased time spent in the morphine-paired chamber during the morphine-prime test session (morphine place preference on prime session minus baseline place preference from previous test session) was significantly different between post-dependent and non-dependent rats (t41=2.77; p=0.0085; Figure 2a).

Figure 2
figure 2

Priming and locomotor-depressant properties of acute morphine are altered during protracted abstinence. After 4 weeks of abstinence (between tests E and F on Figure 1), rats were given a CPP test immediately following a priming injection of morphine (8 mg/kg). Means (±SEMs) for difference scores following the morphine-prime (morphine-prime test session minus the previous test session) are shown for morphine- and placebo-pelleted animals during 15-min CPP tests. (a) Morphine place preference (amount of time spent on morphine-paired side minus time spent on saline-paired side, in sec) was increased in non-dependent rats (previously placebo-pelleted; n=22), but not in post-dependent rats (previously morphine-pelleted; n=21) following morphine-prime as compared with the previous test session. Post-dependent and non-dependent rats were significantly different (**p<0.01). Difference scores for the time spent in the morphine-paired chamber can be calculated by halving the difference scores shown here for place preference. (b) Locomotor activity (numbers of beam breaks) was depressed in non-dependent rats during the morphine-prime session as compared with the previous test session, but not in post-dependent rats. Post-dependent and non-dependent rats were significantly different (*p<0.05).

PowerPoint slide

Full size image

The locomotor properties of acute morphine were altered during protracted abstinence, in addition to the above change in the priming properties of acute morphine. Non-dependent rats showed significantly decreased locomotor activity (number of beam breaks) during the morphine-prime test session as compared with the previous session, whereas post-dependent rats did not. ANOVA revealed an effect of test session (F1,41=19.67; p<0.0001) and an interaction between test session and group (F1,41=5.69; p=0.02), and post hoc analyses showed a significant effect only in non-dependent rats (p<0.001). This confirms that acute morphine decreased locomotor activity only in non-dependent rats. A comparison of the change in locomotor activity during the morphine-prime test session (locomotion on prime session minus baseline locomotion from previous test session) showed significant differences between the non-dependent and post-dependent groups (t41=2.39; p=0.022; Figure 2b).

DISCUSSION

The current results show that morphine place preference is enhanced during protracted abstinence when drug-stimulus associations are experienced in the abstinent state, but not when drug-stimulus associations are formed before dependence and abstinence. When rats were conditioned for morphine place preference before the induction of morphine dependence, preference expression was unaltered throughout dependence and abstinence (Figure 1). However, morphine place preference was increased in post-dependent rats, as compared with the previous test and non-dependent rats, after they were re-conditioned for morphine CPP in the abstinent state (following 6–8 weeks of abstinence). Previous animal studies found that pre-exposure to morphine or cocaine increases subsequent place preference for those drugs (Gaiardi et al, 1991; Harris and Aston-Jones, 2001, 2003a; Harris et al, 2007b; Lett, 1989; Shippenberg and Heidbreder, 1995; Shippenberg et al, 1996; Simpson and Riley, 2005). However, the current results extend those studies by showing that enhanced morphine preference during protracted abstinence is dependent upon experiencing drug-cue pairings specifically in the abstinent state, indicating an important role of incentive learning for Pavlovian-conditioned responses.

Incentive learning has been observed predominantly in instrumental conditioning studies. As an example, rats trained to make an instrumental response for food while in a low-deprivation state and then shifted to a high-deprivation state (hunger) and tested in extinction conditions do not show increased operant performance (food-seeking) unless they have had previous experience with food reinforcement in the current deprivation state (Balleine, 1992). Importantly, these same hungry rats with no previous experience in the deprivation state show increased magazine entries (a Pavlovian association) despite no change in instrumental responding; this implied that incentive learning occurred only for instrumental conditioning. A similar necessity for incentive learning has been observed with opiate self-administration. Opiate self-administration is increased in rats and monkeys during acute withdrawal (by opioid antagonism) or in response to withdrawal-associated stimuli that also elicit withdrawal-like symptoms (Goldberg et al, 1969, 1971; Kenny et al, 2006; Negus, 2006). This withdrawal-induced instrumental response requires incentive learning about the drug in the withdrawal state, as supported by the finding that rats trained to make an instrumental response for heroin and then shifted to a high-motivation state (acute withdrawal from chronic morphine) and tested in extinction conditions do not show increased drug-seeking unless they are given previous experience with heroin-taking in the withdrawal state (Hutcheson et al, 2001). These and similar findings led to the hypothesis that incentive learning theory occurs specifically in instrumental conditioning (Balleine, 1992, 2005; Balleine, 2001). This theory posits that an animal’s instrumental performance is not solely dependent on their motivational state, but also on the incentive properties of the reinforcer that must be learned through experience with the reinforcer in the current drive state (Balleine, 2001; Dickinson and Balleine, 2002).

Here, we extend incentive learning theory by showing that incentive learning also applies to Pavlovian-conditioned stimuli, at least in some circumstances. Our results are significant in showing that the motivational state of morphine abstinence alone is not sufficient to drive increased Pavlovian responding (enhanced morphine place preference), but causes a change in the rewarding properties of morphine so that incentive learning about morphine-related stimuli is increased when experienced in that state. Together, these results for incentive learning are significant clinically because addicts typically re-experience drugs during abstinence many times, particularly during initial drug experimentation that leads to addiction, so that this mechanism may be a substantial element driving increases in drug craving seen in addiction.

Prior studies of incentive learning examined instrumental responding during acute deprivation or withdrawal states (Balleine, 1992; Hutcheson et al, 2001). Our studies extend these prior findings not only by showing that incentive learning occurs for Pavlovian-conditioned associations, but also that incentive learning occurs during long-term shifts in motivation (protracted withdrawal), adding to the acute deprivation/withdrawal findings from previous incentive learning studies. This is significant clinically as it means that drug-associated cues may become more salient when re-experienced even after long periods of drug abstinence.

Our experiments also indicate that the subjective properties of acute morphine are altered following dependence and protracted abstinence. Whereas non-dependent rats responded to a morphine-prime given before a preference test by showing increased time in the morphine-paired chamber, post-dependent (abstinent) rats showed no changes in place preference (Figure 2a). The loss of priming might be explained by a dissociation of morphine with the chamber when these animals were exposed to chronic morphine (pellets) in their home cage during the dependence phase. Receiving morphine (an unconditioned stimulus, US) in the absence of the paired chamber (a conditioned stimulus, CS) might have caused a weakening of the original US–CS association without changing the individual values of the US or CS. Alternatively, the absence of the priming response may indicate that the subjective feeling of acute morphine was altered by protracted abstinence in such a way that the animals no longer associated it with the morphine received during initial conditioning. In any case, these results show that the cueing properties of acute morphine are radically altered by dependence and abstinence.

During protracted abstinence, the locomotor response to acute morphine was also changed during the prime test session (Figure 2b). Tolerance to the depressant effects of morphine following several weeks of morphine abstinence has been reported previously (Bartoletti et al, 1983; Vanderschuren et al, 1997). The absence of the locomotor-depressant effects of acute morphine in post-dependent animals indicates that tolerance/habituation to some properties of morphine persist even after 6–8 weeks of abstinence. In addition, this tolerance is interesting in light of the fact that the rewarding properties are enhanced, or sensitized, indicating dissociation among different properties of morphine.

The increased incentive motivation that drives enhanced morphine preference during protracted drug abstinence may be attributed to a negative allostatic state involving adaptations of reward or stress systems in the brain (reviewed by Aston-Jones and Harris, 2004; Koob and Le Moal, 2008; Martin-Fardon et al, 2010; Smith and Aston-Jones, 2008; Weiss et al, 2001). Increased morphine preference and decreased food preference in protracted abstinence animals have been associated with increased Fos activation in stress-related brain areas, such as extended amygdala and nucleus tractus solitarius (Harris and Aston-Jones, 2003a; Harris and Aston-Jones, 2007a). In addition, previous exposure to stress has been shown to increase the subsequent acquisition of morphine place preference, an effect that is attenuated by corticosterone inhibition during CPP acquisition, but not during stress exposure (Der-Avakian et al, 2005; Will et al, 1998). Alterations in reward circuits (eg, dopaminergic responses to acute morphine) have also been observed in post-dependent rats during abstinence (Acquas and Di Chiara, 1992; Diana et al, 1999; Georges et al, 2006). Long-term changes in stress or reward neurocircuitry that persist during protracted abstinence may increase the reinforcing properties of morphine due to affective contrast of the drug as compared with the pre-existing negative allostatic state. This, in turn, drives increased preference for stimuli associated with morphine during protracted abstinence via incentive learning processes.

CONCLUSIONS

We found that incentive learning during protracted morphine abstinence drives increased preference for stimuli associated with morphine. Therefore, incentive learning is an important mechanism for updating not only instrumental actions but also Pavlovian-conditioned responses, which have an integral role in the addiction process in humans. Incentive learning that occurs with repeated drug exposure following periods of abstinence may be a key behavioral mechanism underlying the ability for drug-associated stimuli to invoke excessive attentional bias and drug craving in addicts. During the transition from casual drug use to addiction, drug- and abstinence-induced changes in the brain may trigger incentive learning for drug-stimulus associations, increasing subsequent cue-induced craving and drug intake to further strengthen the feed-forward addiction cycle.

FUNDING AND DISCLOSURE

Dr G Aston-Jones has received compensation from Elsevier B.V. and Ironwood Pharmaceuticals. However, no compensation was received for compounds or devices used in this research. Dr RJ Smith declares no conflicts of interest.