Learning accounts of addiction and obesity emphasize the persistent power of Pavlovian reward cues to trigger craving and increase relapse risk. While extinction can reduce conditioned responding, Pavlovian relapse phenomena—the return of conditioned responding following successful extinction—challenge the long-term success of extinction-based treatments. Translational laboratory models of Pavlovian relapse could therefore represent a valuable tool to investigate the mechanisms mediating relapse, although so far human research has mostly focused on return of fear phenomena. To this end we developed an appetitive conditioning paradigm with liquid food rewards in combination with a 3-day design to investigate the return of appetitive Pavlovian responses and the involved neural structures in healthy subjects. Pavlovian conditioning (day 1) was assessed in 62 participants, and a subsample (n = 33) further completed extinction (day 2) and a reinstatement test (day 3). Conditioned responding was assessed on explicit (pleasantness ratings) and implicit measures (reaction time, skin conductance, heart rate, startle response) and reinstatement effects were further evaluated using fMRI. We observed a return of conditioned responding during the reinstatement test, evident by enhanced skin conductance responses, accompanied by enhanced BOLD responses in the amygdala. On an individual level, psychophysiological reinstatement intensity was significantly anticorrelated with ventromedial prefrontal cortex (vmPFC) activation, and marginally anticorrelated with enhanced amygdala-vmPFC connectivity during late reinstatement. Our results extend evidence from return of fear phenomena to the appetitive domain, and highlight the role of the vmPFC and its functional connection with the amygdala in regulating appetitive Pavlovian relapse.
Learning about environmental cues that signal desirable outcomes constitutes an important mechanism to flexibly adapt behavior and foster survival. However, learning theories of addiction and obesity emphasize the persistent power of Pavlovian reward cues (conditioned stimuli, CS+)—a beer brand label in the super market or the smell of a freshly-baked cake—to trigger the desire for the associated drug/food (unconditioned stimulus, US), drive habits, and increase the risk of relapse long after abstinence1,2,3,4.
Although extinction—repeatedly presenting a CS+ without the US—reduces conditioned responding5, it does not “erase” the original cue-reward association, but induces new, highly context-dependent inhibitory learning6. Several Pavlovian relapse phenomena—a return of conditioned responding towards the extinguished CS+—originate from these properties of extinction, including the mere passage of time (spontaneous recovery), an unpredicted encounter with the US (reinstatement), or a change in context (renewal)6,7. Reinstatement, the return of conditioned responding towards an extinguished CS after an unpredicted encounter with the US, is well documented in rodents8,9.
From a clinical perspective, this challenges the efficacy of cue-exposure therapy to prevent relapse despite reducing cue reactivity in the clinic10,11. Translational laboratory models of human Pavlovian relapse could therefore represent a valuable tool to investigate the mechanisms that mediate relapse and develop new techniques to counteract it12. Anxiety research experimentally investigates return of fear following extinction on multiple response systems, including psychophysiological measures (skin conductance, fear-potentiated startle) and neuroimaging13,14,15. Conversely, experimental research on appetitive Pavlovian conditioning and relapse in humans is still in its infancy16,17,18. This research gap has been explained by difficulties to find universally-rewarding USs and a lack of established measures sensitive to appetitive responses19–21.
To this end we developed a differential delay conditioning paradigm with liquid food as natural unconditioned stimulus (US) in combination with a 3-day design to evaluate conditioning (day 1), extinction (day 2) and return of conditioned responding following a reinstatement procedure (3 day) while allowing consolidation of learning between sessions. A multimethod approach was used to assess conditioned responses, including explicit (CS pleasantness and US contingency ratings), behavioral (reaction times), and implicit measures (SCRs, heart rate, startle responses). Moreover, return of appetitive conditioned responses was investigated on a neuronal level using fMRI.
Preclinical work points toward an important role of the ventromedial prefrontal cortex (vmPFC) in mediating appetitive Pavlovian relapse after extinction22,23, suggesting it as a central site of extinction memory storage24. This regulatory role is accomplished via projections to key structures involved in reward-related learning, particularly basolateral amygdala and nucleus accumbens (NAcc)25,26. Corroborating animal findings, human neuroimaging confirmed the involvement of the amygdala as well as the ventral striatum, including the NAcc, during appetitive Pavlovian learning with primary rewards27,28. Furthermore, vmPFC activation was related to inhibiting previously learned appetitive responses29, possibly through enhanced functional connectivity with the amygdala28,30. In the fear domain, neuroimaging highlighted an important role for the vmPFC in extinction recall31, and amygdala, hippocampus and vmPFC have been shown to be differentially involved in reinstated fear32,33.
The aims of this study were twofold: (1) to test the hypothesis that appetitive conditioned responses in healthy subjects recover after a reinstatement procedure 24 h after extinction learning; (2) to characterize the neural structures involved in appetitive reinstatement and their relation to individual differences in reinstatement intensity. Based on the outlined findings, we hypothesized that amygdala, NAcc and vmPFC mediate reinstatement and that inhibition of conditioned responses depends on functional amygdala-vmPFC connectivity30.
Materials and methods
Seventy-one healthy, right-handed volunteers recruited from student mailing lists gave written informed consent to participate in the appetitive conditioning session (day 1); the last 36 were followed up for extinction (day 2) and a reinstatement test (day 3). Participants were excluded in case of current or past medical, psychiatric or neurological disorders, drug or alcohol abuse, pregnancy, color blindness or weakness, being on a diet, and allergies or food intolerances to the delivered liquid foods (self-report). All subjects were right-handed and had normal or corrected-to-normal vision. Subjects were required to fast for at least four hours before each session34. A priori defined inclusion criteria (see ‘Behavioral data acquisition’) resulted in a final sample of 62 subjects during conditioning (M (SD)age = 24.42(3.28) years; 35 women) and 33 subjects during extinction and reinstatement test (M (SD)age = 24.06(3.81) years; 18 women). The study was approved by the Ethics Committee of the Charité – Universitätsmedizin Berlin and conducted in accordance with the Declaration of Helsinki.
We used a differential delay conditioning paradigm with liquid food as natural unconditioned stimulus (US) with a multimethod approach over three sessions (conditioning, extinction, and reinstatement; Fig. 1a), 24 h apart to allow consolidation between sessions. Return of conditioned responding on day 3 was assessed after confronting subjects with unsignaled US presentations (reinstatement procedure). We provide publically available source code of the paradigm via Bitbucket (https://bitbucket.org/LearningAndCognition/appreinstatement).
One of four possible juices/smoothies (apple, orange, orange-passionfruit, strawberry-banana) served as US, depending on subject’s preference. US administration consisted of 1 ml of liquid delivered directly into the subjects’ mouth via clear PVC tubes and a programmable syringe pump (World Precision Instruments, Inc., Sarasota, USA). Two different pictures combined with one of two possible tones (400 or 500 Hz; 100-ms duration) served as cues (CS+/CS−, counterbalanced across subjects; see Fig. 1b).
Subjects underwent uninstructed conditioning, extinction, and a reinstatement test, i.e., no information about CS-US contingencies or changes across sessions was provided. Until a final debriefing, subjects were told they were adult controls in an experiment investigating hand-eye coordination and attention in small children, where juice served to keep children’s motivation during the experiment. Subjects were asked to indicate the side of cue appearance via button press with their right index and middle fingers as quickly and accurately as possible.
Conditioning (day 1) consisted of two phases, each with 30 CS+ and 30 CS− trials (total, 120 trials). In each trial, a cue was presented for 4 s to the left or right side of a white fixation cross. The CS+ was paired with US delivery 1 s before cue offset in 50% of the trials; the CS− was never followed by liquid food. To extinguish conditioned responses acquired on day 1, extinction (day 2) comprised one phase with 30 unreinforced CS+ and 30 CS− trials. The reinstatement test (day 3, in MRI) consisted of two identical phases, each with 15 unreinforced CS+/CS− trials. To trigger return of conditioned responding, subjects received three unsignaled, jittered US deliveries within 30 s prior to each phase (reinstatement procedure). Of note, due to the temporal spacing between sessions (24 h) and context changes from laboratory to scanner, spontaneous recovery and renewal might contribute to return of conditioned responding following reinstatement, providing a more ecologically valid model of appetitive Pavlovian relapse, where reinstatement effects inevitably follow some time after treatment (spontaneous recovery) and likely occur in a context different from initial acquisition (renewal). Trial sequences were pseudo-randomized across subjects (see Supplementary Material for further details). The inter-trial interval (ITI) ranged 3.5–12 s (M = 6 s).
Behavioral data acquisition and preprocessing
Thirst and hunger ratings
Thirst and hunger ratings were collected prior to each session on separate 100-mm visual analog scales (VAS) ranging from 0 = ‘not thirsty/hungry at all’ to 100 = ‘very thirsty/hungry’.
Pleasantness of US and CS
US pleasantness was evaluated pre- and post-conditioning, and before the startle test on day 3 by applying a single US delivery, followed by a computerized VAS ranging from −50 = ‘very unpleasant’ to 50 = ‘very pleasant’. Only subjects with mean positive ratings pre-/post-conditioning were included in the study, ensuring US appetence during learning (subjects excluded: n = 6). On-screen pleasantness ratings for CS+ and CS− were acquired on an identical VAS before and after each session (Fig. 1a).
After conditioning, subjects rated the reward probability of each CS on a 100-point VAS (‘Immediately after this picture, I received a sip of juice…’) ranging from ‘never’ to ‘always’. Difference scores [CS+ minus CS−] were calculated as a dimensional awareness indicator, with large positive values implying contingency awareness and values around zero unawareness35. Subjects with notable negative difference scores (≤−20) were excluded from the study (n = 3), as these indicate (explicit) conditioning towards the CS− rather than unawareness. Following a worthwhile reviewer comment, we further explored associations between contingency awareness and conditioning indices on day 1 (see Supplementary Material).
Psychophysiological data acquisition and preprocessing
Psychophysiological data were acquired at 250 Hz using an MR-compatible amplifier (BrainAmp ExG, Brain Products, Munich, Germany).
Skin conductance responses (SCRs) were recorded during all sessions from the participant’s middle phalanges of the left index and middle finger using MR-compatible Ag/AgCl electrodes. Two datasets were excluded because of technical failures during recording or low data quality. Preprocessing and analysis of single-subject data was performed within the PsPM toolbox (version 3.1.1; http://pspm.sourceforge.net/), using the general linear convolution model (GLM) approach. Preprocessing comprised linear interpolation of movement-related artefacts, band-pass filtering (first-order Butterworth, 0.05–5 Hz), downsampling (10 Hz), and normalization to remove between-subject variance in response amplitudes36. Event onsets (CS+/CS−/US) were modeled as stick functions and convolved with a canonical SCR function. GLMs for conditioning and extinction included CS+ and CS− onsets per phase as regressors of interest, and an additional regressor to model the US effect on day 1. To test for return of conditioned responses on day 3, cue onsets of the first five CS+ and CS− trials following unsignaled US deliveries in each phase were modeled as two regressors of interest. US onsets and the remaining cues were modeled as three regressors of no interest. Within each GLM, regressors were fitted to the SC time series, yielding an SCR-amplitude estimate per regressor.
Cue-related modulation of eyeblink and postauricular (PAR) reflexes was assessed by auditory startle tests in each session (Fig. 1a). During the reinstatement session, three unsignaled US deliveries (reinstatement procedure) preceded the startle test. Startle probes consisted of a 50-ms, 90-dB burst of white noise presented binaurally via headphones. The test was initiated by four jittered startle probes while viewing the fixation cross (habituation startles; mean inter-probe-interval 2 s). Thereafter, startle probes were presented in each of four unreinforced CS+/CS− trials with varying stimulus-onset asynchronies (0.5, 1, 1.5, or 2 s). Four additional startle probes were presented during the ITI to reduce startle predictability and avoid a cue-related association. The ITI ranged 9–13 s (M = 10.9 s). Trial order and stimulus side were fully counterbalanced across subjects by assigning one of four fixed trial sequences. Startle responses were recorded with four Ag/AgCl electrodes on left orbicularis oculi and auricularis muscles37,38. Due to technical reasons, startle data from the first 12 participants are missing, and two datasets from the conditioning sample were lost because of technical failures. Electromyography (EMG) signal was notch-filtered at 50 Hz and band-pass filtered (2nd order Butterworth, 28–110 Hz), after which data were rectified and smoothed (3rd order Savitzky-Golay filter) using a moving average of 15 and 20 consecutive data points for PAR and eyeblink reflex, respectively. Startle responses were defined as the difference between the maximum amplitude within 20–115 ms and 10–40 ms after probe onset for eyeblink reflex and PAR, respectively, and startle baseline, i.e., mean EMG activity 50 ms before probe onset (ranges comparable to37,38), and averaged over cue type for group analyses. Eyeblink data from two subjects on day 1 and from one subject on day 3 had to be excluded for low quality.
Acquisition and preprocessing of heart rate (HR) and reaction times (RTs) is provided in the Supplementary Material.
Imaging data acquisition and preprocessing
On day 3, MR data were acquired on a 3 Tesla scanner (Trio, Siemens AG, Erlangen, Germany) using a 32-channel head coil with a standard EPI sequence (40 slices, 3 × 3 mm2 in-plane voxel resolution, TR = 2.09 s, TE = 22 ms, 90° flip angle, 64 × 64 matrix; 192 × 192 mm FOV). Preprocessing and statistical analyses were performed within SPM12 (www.fil.ion.ucl.ac.uk/spm/) implemented in Matlab R2015b (The MathWorks, Inc., Natick, Massachusetts, United States). Preprocessing included slice time correction, realignment to the mean EPI volume, unwarping using the acquired field map, segmentation of the structural T1 image, coregistration of the segmented structural image to the mean EPI, spatial normalization to MNI space based on normalization parameters derived from each subjects’ structural image (2-mm isotropic voxel resolution), and smoothing using a 6-mm full-width at half maximum Gaussian Kernel.
Statistical analysis of behavioral and physiological parameters
Effects of conditioning and extinction
CS pleasantness ratings during conditioning and extinction were analyzed separately with repeated measures ANOVA (rmANOVA) with within-subject factors cue (CS+/CS−) and time (pre/post). SCR, HR and RT during conditioning were analyzed analogously, with the factor time referring to early/late phase of conditioning. Paired t-tests were used to analyze cue differences during the extinction phase and all startle tests.
Differential responding on day 3 was only probed for measures showing a significant conditioning effect on day 1 and successful extinction on day 2. Differential valence ratings and startle responses were evaluated using paired t-tests. For the fMRI reinstatement test, SCRs were estimated for the first five trials per cue after each reinstatement (10 trials/cue) and analyzed using a paired t-test.
Statistical analyses were performed within R version 3.4.339 and rmANOVAs were Greenhouse-Geisser-corrected when necessary. Significant interaction effects were followed by post-hoc t-tests. The alpha level was set at .05 for all analyses and effect sizes were estimated using partial Eta2 (η2p) and Cohen’s d.
Statistical analysis of imaging data (day 3)
An event-related analysis was applied using SPMs GLM approach on two levels. On the subject level, onsets for CS+ and CS− were included for each phase after convolution with the canonical HRF. US onsets and movement parameters were entered as regressors of no interest. Baseline contrasts for CS+ and CS− were computed for each phase and entered into a flexible factorial model on the group level. Reinstatements effects were analyzed by contrasting CS+ vs. CS− across phases. Possible time effects during the reinstatement test were investigated by assessing the cue × phase interaction. Based on evidence showing amygdala and NAcc involvement in appetitive Pavlovian conditioning27,28,40 and the role of the vmPFC in successful extinction recall41,42 and reinstated fear32,33, we applied small volume correction for amygdala, NAcc, and vmPFC at p ≤ .05 FWE-corrected. In our sample (n = 33) we had a power of 0.88 to detect medium effects (d = 0.5) at this threshold (G*Power 343). Bilateral amygdala and NAcc masks were derived from WFU PickAtlas (http://www.fmri.wfubmc.edu/download.htm). For the vmPFC mask, a 10-mm sphere centered on [x = 0, y = 40, z = −12] was used based on previous studies on reinstated fear32,33.
For completeness, exploratory whole-brain analyses at p < .001 uncorrected are provided in the Supplementary Material.
gPPI analysis. Based on previous findings30, we investigated the interplay between amygdala and vmPFC during the reinstatement test and analyzed their cue-dependent functional connectivity using generalized psychophysiological interaction (gPPI) analysis (gPPI toolbox, http://www.nitrc.org/projects/gppi)44. For each participant, the first eigenvariate time series was extracted from the left amygdala seed and deconvolved to generate the neuronal signal45. For each cue type and phase, a PPI term was created by multiplying the respective cue onsets with the neural time series and convolving it with the HRF. The four PPI terms and the seed region time course then entered as regressors into first-level models otherwise similar to the primary analysis first-level GLM. Estimated connectivity parameters for each cue type and phase were analyzed in SPM’s full factorial design on the second level. Cue-specific connectivity was analyzed by contrasting PPI terms for CS+ vs. CS- across phases, and time-dependent changes by interacting cue differences with phase. A region-of-interest (ROI) analysis for the vmPFC was applied at p ≤ .05 FWE-corrected, following the proposed modulatory influence from vmPFC on amygdala activity during aversive41,42 and appetitive30 extinction recall.
Associations with psychophysiological reinstatement effects. In order to link the observed psychophysiological reinstatement effect (SCRs) to brain activation, simple regression analyses were performed within SPM introducing differential SCRs during reinstatement test as a covariate using the contrast images ‘CS+ vs. CS−’, as well as cue-specific connectivity differences (CS+ vs. CS−) from the gPPI analysis in separate SPMs. These analyses were complemented by subgroup analyses based on a median split of differential SCRs, thereby contrasting a low reinstatement (n = 17) with a high reinstatement group (n = 16).
Following a reviewer’s suggestion, an exploratory gPPI analysis using the right NAcc as seed region was applied (see Supplementary Material).
Thirst and hunger
Ratings confirmed thirst and moderate hunger before conditioning (thirst: M (SD) = 63.7(20.6); hunger: M (SD) = 43.3(27.1)), extinction (thirst: M (SD) = 68.7(21.8); hunger: M (SD) = 51.8(29.0)), and reinstatement test (thirst: M (SD) = 65.0(18.6); hunger: M (SD) = 48.4(26.6)).
The perceived pleasantness of the chosen juice/smoothie was high throughout conditioning (M (SD) = 32.32(13.60); Fig. 2a) and before the reinstatement test (M (SD) = 32.61(15.68)), and remained unchanged over sessions (day 1 vs. day 3: Z = −0.08, p = .935; Wilcoxon signed-rank test).
Reward contingencies for CS+ were rated significantly higher than for CS− after conditioning, indicating overall contingency knowledge across subjects that varied to different degrees (M (SD)diff = 30.40(31.58), t(61) = 7.58, p < .001, range: −18.84 to 99.34; Fig. 2a, see Supplementary Material for further details), suggesting that uninstructed conditioning in addition to our cover story allowed for variability regarding explicit learning.
Conditioning and extinction
Analysis of CS pleasantness during conditioning revealed a significant cue × time interaction (F (1,61) = 4.32, p = .042, η2p = 0.07; Fig. 2b). Closer inspection showed that subjective pleasantness of the CS+ increased significantly from pre- to post-conditioning (t(61) = −2.96, p = .004), while CS− pleasantness remained unchanged (t(61) = 0.13, p = .900), resulting in a trendwise differentiation between CS+ and CS− after acquisition (t(61) = 1.98, p = .052) but not at baseline (t(61) = −0.21, p = .840). During extinction, a significant main effect of time (F (1,32) = 4.93, p = .034, η2p = .13) indicated an overall decrease in CS pleasantness, but no cue × time interaction or main effect of cue (all p ≥ .413).
Skin conductance responses (SCRs)
During conditioning, SCRs towards the CS+ were significantly larger compared to the CS− across phases (main effect cue: F (1,59) = 7.08, p = .010, η2p = .11; Fig. 2c). We also observed a significant time effect (F (1,59) = 22.16, p < .001, η2p = .27) due to declining SCRs towards both cue types, but no cue × time interaction (F(1,59) = 0.53, p = .468). As expected, differential SCRs were no longer observed during extinction on day 2 (t(32) = 0.99, p = .329).
Successful conditioning was confirmed by a significantly attenuated eyeblink reflex (t(45) = −3.19, p = .003, d = −.47) as well as a significantly enhanced PAR (t(47) = 2.98, p = .005, d = .46; Fig. 2d) when contrasting startle reflexes during CS+ compared to CS− presentations post-conditioning. After extinction, neither eyeblink reflex nor PAR were differentially modulated by cue type, indicating complete extinction (all p ≥ .894).
Heart rate (HR)
Analysis of HR during conditioning revealed HR increases towards both cue types (main effect time: F (1,58) = 8.03, p = .006, η2p = .12) but no main effect of cue or cue × time interaction (F (1,58) ≤ 1.95, p ≥ .168). HR responses did not differ during extinction (t(32) = −0.82, p = .418).
Reaction times (RTs)
RTs obtained from the stimulus side detection task revealed no significant main or interaction effects during conditioning (F (1,60) ≤ 1.80, p ≥ .185) nor extinction (t(32) = −0.31, p = .760).
The reported conditioning effects remained unchanged when controlling for possible sample effects (conditioning sample vs. 3-day sample).
Behavioral and psychophysiological reinstatement effects
Return of appetitive conditioned responding was investigated for measures showing successful conditioning and extinction, i.e., CS pleasantness ratings, startle responses, and SCRs. Extinguished differences of eyeblink reflex or PAR did not recover at test (p ≥ .500). In contrast, following reinstatement in the fMRI, a significant return of conditioned responding was observed in SCRs, with larger SCRs towards CS+ compared to CS− (t(32) = 2.25, p = .031, d = .39; Fig. 3a). CS pleasantness ratings did not differ significantly at either time point (p ≥ .170), although CS+ pleasantness significantly increased from laboratory to fMRI reinstatement test (t(32) = −2.88, p = .007) while CS− pleasantness remained unchanged (t(32) = −0.83, p = .414).
Neural responses during fMRI reinstatement test
The fMRI reinstatement test was accompanied by a significant differential BOLD response in the left amygdala with stronger activation towards the CS+ compared to the CS− ([x:−26, y:2, z:−26]; Z = 3.82; pFWE ROI = .010, Fig. 3b). The inverse contrast (CS−>CS+) revealed no significant activation differences. We also looked for time dependent effects, as return of conditioned responding may decline with repeated unreinforced CS presentations despite the second reinstatement between phases. We observed a significant decline in differential BOLD response over time (cue × time interaction) in the right NAcc ([x:16, y:8, z:−10]; Z = 3.35, pFWE ROI = .016; Fig. 3c) and trendwise also in the left amygdala ([x:−20, y:−6, z:−18]; Z = 2.23; pFWE ROI = .064), but no differential BOLD response increase over phases.
Neural responses associated with psychophysiological reinstatement effects
We investigated brain regions associated with the psychophysiological reinstatement effect by performing regression and subgroup analyses based on differential SCRs during the reinstatement test. Regression analysis revealed a significant negative correlation between differential SCRs and activation within the vmPFC ([x:−6, y:42, z:−8]; Z = 3.63; pFWE ROI = .022; Fig. 4a), indicating stronger vmPFC activity towards CS+ compared to CS− in subjects showing attenuated differential SCRs. Corroborating this association, directly contrasting participants with high vs. low differential SCRs confirmed significant higher vmPFC activation in the low reinstatement compared to the high reinstatement group ([x:−4, y:42, z:−8], Z = 3.83, pFWE ROI = .011; Fig. S1A).
Functional connectivity between amygdala and vmPFC
Based on the proposed inhibitory role of the vmPFC over amygdala to support successful extinction recall30,41,42, we further investigated cue-dependent functional connectivity between the amygdala and the vmPFC during the reinstatement test. The gPPI analysis showed that, while no significant amygdala-vmPFC connectivity was evident across phases, connectivity during CS+ compared to CS− presentation was significantly enhanced in the second phase of the reinstatement test ([x:8, y:44, z:−16]; Z = 3.49; pFWE ROI = .032). Interestingly, this connectivity was further marginally anticorrelated with the psychophysiological reinstatement effect ([x:6, y:40, z:−16]; Z = 3.25; pFWE ROI = .061; Fig. 4b), i.e., tended to be enhanced in participants with low compared to high differential SCRs during reinstatement test ([x:6, y:40, z:−16], Z = 3.08, pFWE ROI = .077; Fig. S1B).
This study investigated the return of experimentally conditioned appetitive responses in healthy subjects as a translational laboratory model of appetitive Pavlovian relapse. We showed that SCRs recover after a reinstatement procedure 24 h after extinction and provide evidence for opposing roles of amygdala and vmPFC in mediating Pavlovian relapse. During the reinstatement test, amygdala activation towards the CS+ was enhanced, while psychophysiological reinstatement intensity was significantly anticorrelated with vmPFC activation and marginally with enhanced amygdala-vmPFC connectivity observed during late reinstatement.
Amygdala and NAcc activity during appetitive Pavlovian relapse
The reinstatement test showed increased BOLD responses in the left amygdala towards CS+ compared to CS− presentations. Amygdala activity was present over both test phases, declining trendwise over time. Preclinical studies have demonstrated the central role of the amygdala in appetitive Pavlovian learning25,46 and cue-induced relapse in animal models of drug addiction47. Corroborating animal findings showing the relevance of the amygdala in the formation of CS-US associations48, human neuroimaging studies have repeatedly observed amygdala activation during appetitive Pavlovian learning with primary rewards27,28,40. The increased amygdala activation present in our study therefore likely reflects retrieval of the original CS−US association. In line with our results, enhanced amygdala activation towards a previously extinguished fear cue has also been observed following unsignaled aversive US presentations32,33,49 or context changes50,51. Enhanced amygdala activation has further been observed towards an extinguished monetary CS+ following a reactivation procedure 24 h after extinction in healthy controls30. We further observed time-dependent differential NAcc activity due to CS+ related declines in BOLD responses from early to late reinstatement, suggesting a more transient involvement of this structure during the reinstatement test. Animal and human work have identified the NAcc within the ventral striatum as a key structure in the brain’s reward circuit, being involved in reward processing and reward-related learning52,53. In rodents, expression of conditioned approach behavior towards food or drug cues depend on an intact NAcc54,55,56 and human neuroimaging has shown increased BOLD responses in the ventral striatum towards cues predicting primary rewards27,28,40,57.
vmPFC mediated inhibition of appetitive conditioned responses
In contrast to enhanced amygdala and NAcc activity, we did not observe significant vmPFC activation towards CS+ compared to CS− presentations during the reinstatement test. Differential BOLD responses in this region were instead inversely related to psychophysiological reinstatement intensity (i.e., differential SCRs), whereby increased vmPFC involvement was only present in subjects experiencing weak and not in those showing strong reinstatement effects. This finding directly adds to animal evidence supporting a role for the vmPFC in inhibiting maladaptive learned associations25,26. Rodent studies have demonstrated that lesions of the infralimbic (IL) cortex as the homolog region of the human vmPFC do not impair acquisition or within-session extinction of appetitive Pavlovian responses, but impair retrieval of extinction the following day, resulting in increased spontaneous recovery, reinstatement, and renewal22,23. Conversely, optogenetic activation of IL neurons has been shown to suppress the return of appetitive conditioned responses58. Animal models of cue-induced reward seeking after extinction further revealed IL involvement in both drug and natural reward seeking responses26,59,60 and specific CS-responsive neuronal ensembles within the IL have been shown to exert inhibitory control over alcohol seeking61.
In line with our result, human neuroimaging has implicated the vmPFC in successful aversive extinction learning and recall31,32,33,42,50,62, and vmPFC activity41,42,62 and cortical thickness63 scaled inversely with conditioned SCRs. Our results further extend recent evidence linking reduced vmPFC involvement to return of fear following reinstatement32,64. In these studies, differential vmPFC activity was present during simple extinction recall but not after a reinstatement procedure32, and reduced CS− related vmPFC activity during reinstatement test compared to extinction recall was associated with increased SCRs, indicating a “release from inhibition”64. Our results further suggest an important regulatory role for the vmPFC in appetitive Pavlovian relapse. Adding to this, increased ventral vmPFC activity has been shown towards a cue no longer paired with a monetary reward, consistent with an inhibitory signal29.
The rodent vmPFC is widely connected65. Given its strong projections to the amygdala66,67 and previous findings on functional amygdala-vmPFC connectivity during appetitive extinction recall30, we investigated cue-dependent functional connectivity between amygdala and vmPFC during the reinstatement test. While there was no evidence of enhanced connectivity across phases, amygdala-vmPFC coupling during CS+ relative to CS− presentations was significantly enhanced during late reinstatement. On an individual level, amygdala-vmPFC connectivity might be further inversely related to the psychophysiological reinstatement effect, indicated by a marginally significant anticorrelation. These findings add to imaging studies reporting functional amygdala-vmPFC connectivity during fear extinction recall41,42 (but see50) and indicate that amygdala-vmPFC coupling constitutes an important neural correlate of successful extinction recall despite adverse circumstances. In line with this, enhanced cue-dependent amygdala-vmPFC coupling has been observed during appetitive extinction recall in subjects receiving the NMDA receptor agonist D-cycloserine hypothesized to enhance extinction consolidation compared to placebo30. Moreover, increased amygdala-vmPFC connectivity during initial appetitive conditioning seems to attenuate amygdala activity and acquisition of SCRs28. Our finding that amygdala-vmPFC connectivity only emerged during the late reinstatement test is consistent with the proposed disinhibition due to decreased vmPFC activity observed in reinstated fear64. Amygdala-vmPFC connectivity might therefore increase as reinstatement effects decline, in line with declining differential BOLD responses observed over test phases for NAcc and trendwise for amygdala.
Psychophysiological and behavioral measures of conditioned responding
We observed a differential return of conditioned responding during the reinstatement test in an implicit measure (SCRs), providing evidence that human appetitive Pavlovian relapse can be modeled in the laboratory. Most of what is known about Pavlovian relapse effects in humans stems from investigations on return of fear phenomena13,15,68. The relative lack of translational research in the appetitive domain has been explained by difficulties in finding suitable USs and measures sensitive to appetitive conditioned responses19,20,21. In line with a previous study using food US16, we observed successful conditioning and extinction in pleasantness ratings, SCRs and eyelid startle, clearly indicating the validity of our design. Conditioning also resulted in a significantly enhanced PAR, thereby replicating recent evidence demonstrating the sensitivity of this microreflex as an appetitive conditioning index20. By contrast, HR and RTs did not provide sensitive indices of conditioning in our paradigm. Unlike SCRs, neither pleasantness ratings nor startle responses showed significant reinstatement effects. Diverging findings across multiple response measures are also commonly reported in fear reinstatement studies13,69, and multimodal investigation of appetitive conditioning suggests that different conditioning indices are only weakly related on an individual level21. We observed a rather weak conditioning effect in pleasantness ratings with only trendwise differentiation after conditioning. Explicit ratings might primarily reflect cognitive learning components70 as indicated by the influence of contingency awareness in our study (see Supplementary Material), while our uninstructed learning paradigm and cover story was intended to reduce cognitive demands71. Moreover, pleasantness ratings have been shown to be rather insensitive to extinction70,72,73 and, in contrast to SCRs or BOLD responses, might not distinguish experimental groups well28,57. While we observed a robust startle modulation after conditioning, participants may have learned to distinguish the non-reinforced and aversive startle context from appetitive acquisition in this session, potentially impeding assessment of startle modulation on subsequent days. Apart from that, the scanner environment possibly enhanced the return of conditioned responses observed during the fMRI reinstatement test, i.e., a stronger renewal effect might have added to the return of conditioned responding.
Our finding that appetitive conditioned responses returned after unsignaled US presentations 24 h after extinction extends existing evidence on return of fear phenomena in humans to the appetitive research domain. Moreover, our results suggest opposing roles for amygdala and vmPFC in mediating appetitive Pavlovian relapse effects. This is of particular clinical relevance, as drug addiction is associated with heightened amygdala responses towards disease-related cues74, while, at the same time, addicted patients exhibit grey matter volume decreases and activation impairments in vmPFC75,76,77,78,79. Future studies could investigate appetitive Pavlovian conditioning and relapse phenomena in patients with addiction and explore inter-individual differences in these processes as well as its interaction with instrumental responding (e.g., Pavlovian-instrumental transfer (PIT)80). Although correlational, our findings suggest that the vmPFC could be a promising target for novel intervention techniques that aim to counteract appetitive Pavlovian relapse.
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Bouton, M. E. Learning and the persistence of appetite: extinction and the motivation to eat and overeat. Physiol. Behav. 103, 51–58 (2011).
Heinz, A. et al. Targeted intervention: computational approaches to elucidate and predict relapse in alcoholism. NeuroImage 151, 33–44 (2017).
Robinson, T. E. & Berridge, K. C. The neural basis of drug craving: an incentive-sensitization theory of addiction. Brain Res Rev. 18, 247–291 (1993).
Jansen, A. A learning model of binge eating: cue reactivity and cue exposure. Behav. Res Ther. 36, 257–272 (1998).
Pavlov, I. P. Conditioned Reflexes. (Oxford University Press, Oxford, 1927).
Bouton, M. E. Context, ambiguity, and unlearning: sources of relapse after behavioral extinction. Biol. Psychiatry 52, 976–986 (2002).
Bouton, M. E. Context and behavioral processes in extinction. Learn Mem. 11, 485–494 (2004).
Bouton, M. E. & Peck, C. A. Context effects on conditioning, extinction, and reinstatement in an appetitive conditioning preparation. Anim. Learn Behav. 17, 188–198 (1989).
Rescorla, R. A. & Heth, C. D. Reinstatement of fear to an extinguished conditioned stimulus. J. Exp. Psychol. Anim. Behav. Process 1, 88–96 (1975).
Conklin, C. & Tiffany, S. Applying extinction research and theory to cue-exposure addiction treatments. Addiction 97, 155–167 (2002).
Martin, T., Larowe, S. D. & Malcolm, R. Progress in cue exposure therapy for the treatment of addictive disorders: a review update. Open Addict. J. 3, 92–101 (2010).
Milton, A. L. & Everitt, B. J. The persistence of maladaptive memory: addiction, drug memories and anti-relapse treatments. Neurosci. Biobehav. Rev. 36, 1119–1139 (2012).
Haaker, J., Golkar, A., Hermans, D. & Lonsdorf, T. B. A review on human reinstatement studies: an overview and methodological challenges. Learn Mem. 21, 424–440 (2014).
Sevenster, D., Visser, R. M. & D’Hooge, R. A translational perspective on neural circuits of fear extinction: current promises and challenges. Neurobiol. Learn Mem. 155, 113–126 (2018).
Vervliet, B., Craske, M. G. & Hermans, D. Fear extinction and relapse: state of the art. Annu. Rev. Clin. Psychol. 9, 215–248 (2013).
Andreatta, M. & Pauli, P. Appetitive vs. aversive conditioning in humans. Front Behav. Neurosci. 9, 1–8 (2015).
Martin-Soelch, C., Linthicum, J. & Ernst, M. Appetitive conditioning: neural bases and implications for psychopathology. Neurosci. Biobehav. Rev. 31, 426–440 (2007).
Van Gucht, D., Vansteenwegen, D., Beckers, T., Van & den Bergh, O. Return of experimentally induced chocolate craving after extinction in a different context: divergence between craving for and expecting to eat chocolate. Behav. Res. Ther. 46, 375–391 (2008).
Hermann, C., Ziegler, S., Birbaumer, N. & Flor, H. Pavlovian aversive and appetitive odor conditioning in humans: Subjective, peripheral, and electrocortical changes. Exp. Brain Res. 132, 203–215 (2000).
Stussi, Y. & Sander, D. Measuring Pavlovian appetitive conditioning in humans with the postauricular reflex. Psychophysiology 55, e13073 (2018).
Wardle, M. C., Lopez-Gamundi, P. & Flagel, S. B. Measuring appetitive conditioned responses in humans. Physiol. Behav. 188, 140–150 (2018).
Rhodes, S. E. V. & Killcross, S. Lesions of rat infralimbic cortex enhance recovery and reinstatement of an appetitive Pavlovian response. Learn Mem. 11, 611–616 (2004).
Rhodes, S. E. V. & Killcross, A. S. Lesions of rat infralimbic cortex enhance renewal of extinguished appetitive Pavlovian responding. Eur. J. Neurosci. 25, 2498–2503 (2007).
Kalivas, P. W., Peters, J. & Knackstedt, L. Animal models and brain circuits in drug addiction. Mol. Inter. 6, 339–344 (2006).
Quirk, G. J. & Mueller, D. Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology 33, 56–72 (2008).
Peters, J., Kalivas, P. W. & Quirk, G. J. Extinction circuits for fear and addiction overlap in prefrontal cortex. Learn Mem. 16, 279–288 (2009).
Gottfried, J. A., O’Doherty, J. & Dolan, R. J. Appetitive and aversive olfactory learning in humans studied using event-related functional magnetic resonance imaging. J. Neurosci. 22, 10829–10837 (2002).
Klucken, T. et al. Impact of COMT Val158Met-polymorphism on appetitive conditioning and amygdala/prefrontal effective connectivity. Hum. Brain Mapp. 36, 1093–1101 (2015).
Zhang, Z., Mendelsohn, A., Manson, K. F., Schiller, D. & Levy, I. Dissociating value representation and inhibition of inappropriate affective response during reversal learning in the ventromedial prefrontal cortex. eNeuro 2, e0072–15.2015 (2016).
Ebrahimi, C. et al. Combining D-cycloserine with appetitive extinction learning modulates amygdala activity during recall. Neurobiol. Learn Mem. 142, 209–217 (2017).
Fullana, M. A. et al. Fear extinction in the human brain: a meta-analysis of fMRI studies in healthy participants. Neurosci. Biobehav Rev. 88, 16–25 (2018).
Lonsdorf, T. B., Haaker, J. & Kalisch, R. Long-term expression of human contextual fear and extinction memories involves amygdala, hippocampus and ventromedial prefrontal cortex: A reinstatement study in two independent samples. Soc. Cogn. Affect Neurosci. 9, 1973–1983 (2014).
Scharfenort, R., Menz, M. & Lonsdorf, T. B. Adversity-induced relapse of fear: Neural mechanisms and implications for relapse prevention from a study on experimentally induced return-of-fear following fear conditioning and extinction. Transl. Psychiatry 6, e858 (2016).
Astur, R. S., Carew, A. W. & Deaton, B. E. Conditioned place preferences in humans using virtual reality. Behav. Brain Res. 267, 173–177 (2014).
Tapia León, I., Kruse, O., Stalder, T., Stark, R. & Klucken, T. Neural correlates of subjective CS/UCS association in appetitive conditioning. Hum. Brain Mapp. 39, 1637–1646 (2018).
Bach, D. R., Friston, K. J. & Dolan, R. J. An improved algorithm for model-based analysis of evoked skin conductance responses. Biol. Psychol. 94, 490–497 (2013).
Blumenthal, T. D. et al. Committee report: guidelines for human startle eyeblink electromyographic studies. Psychophysiology 42, 1–15 (2005).
Sandt, A. R., Sloan, D. M. & Johnson, K. J. Measuring appetitive responding with the postauricular reflex. Psychophysiology 46, 491–497 (2009).
R Core Team. R: A language and environment for statistical computing. 2017. http://www.r-project.org. Accessed 15 Feb 2018.
O’Doherty, J. P., Deichmann, R., Critchley, H. D. & Dolan, R. J. Neural responses during anticipation of a primary taste reward. Neuron 33, 815–826 (2002).
Phelps, E. A., Delgado, M. R., Nearing, K. I. & LeDoux, J. E. Extinction learning in humans:role of the amygdala and vmPFC. Neuron 43, 897–905 (2004).
Milad, M. R. et al. Recall of fear extinction in humans activates the ventromedial prefrontal cortex and hippocampus in concert. Biol. Psychiatry 62, 446–454 (2007).
Faul, F., Erdfelder, E., Lang, A.-G. & Buchner, A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav. Res. Methods 39, 175–191 (2007).
McLaren, D. G., Ries, M. L., Xu, G. & Johnson, S. C. A generalized form of context-dependent psychophysiological interactions (gPPI): a comparison to standard approaches. Neuroimage 61, 1277–1286 (2012).
Gitelman, D. R., Penny, W. D., Ashburner, J. & Friston, K. J. Modeling regional and psychophysiologic interactions in fMRI: the importance of hemodynamic deconvolution. Neuroimage 19, 200–207 (2003).
Janak, P. H. & Tye, K. M. From circuits to behaviour in the amygdala. Nature 517, 284–292 (2015).
Luo, Y. X., Xue, Y. X., Shen, H. W. & Lu, L. Role of amygdala in drug memory. Neurobiol. Learn Mem. 105, 159–173 (2013).
Tye, K. M., Stuber, G. D., De Ridder, B., Bonci, A. & Janak, P. H. Rapid strengthening of thalamo-amygdala synapses mediates cue-reward learning. Nature 453, 1253–1257 (2008).
Schiller, D., Kanen, J. W., LeDoux, J. E., Monfils, M.-H. & Phelps, E. A. Extinction during reconsolidation of threat memory diminishes prefrontal cortex involvement. PNAS 110, 20040–20045 (2013).
Kalisch, R. et al. Context-dependent human extinction memory is mediated by a ventromedial prefrontal and hippocampal network. J. Neurosci. 26, 9503–9511 (2006).
Agren, T. et al. Disruption of reconsolidation erases a fear memory trace in the human amygdala. Science 337, 1550–1552 (2012).
Haber, S. N. & Knutson, B. The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35, 4–26 (2010).
Daniel, R. & Pollmann, S. A universal role of the ventral striatum in reward-based learning: evidence from human studies. Neurobiol. Learn Mem. 114, 90–100 (2014).
Parkinson, J. A., Cardinal, R. N. & Everitt, B. J. Limbic cortico-ventral system underlying appetitive conditioning. Prog. Brain Res. 126, 263–285 (2000).
Parkinson, J. A. et al. Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function. Behav. Brain Res. 137, 149–163 (2002).
Milton, A. L. & Everitt, B. J. The psychological and neurochemical mechanisms of drug memory reconsolidation: implications for the treatment of addiction. Eur. J. Neurosci. 31, 2308–2319 (2010).
Klucken, T., Wehrum-Osinsky, S., Schweckendiek, J., Kruse, O. & Stark, R. Altered appetitive conditioning and neural connectivity in subjects with compulsive sexual behavior. J. Sex. Med. 13, 627–636 (2016).
Villaruel, F. R. Optogenetic activation of the infralimbic cortex suppresses the return of appetitive Pavlovian-conditioned responding following extinction. Cereb Cortex 28, 4210–4221 (2018).
Pfarr, S. et al. Choice for drug or natural reward engages largely overlapping neuronal ensembles in the infralimbic prefrontal cortex. J. Neurosci. 49, 0026–18 (2018).
Warren, B. L. et al. Distinct Fos-expressing neuronal ensembles in the ventromedial prefrontal cortex mediate food reward and extinction memories. J. Neurosci. 36, 6691–6703 (2016).
Pfarr, S. et al. Losing control: excessive alcohol seeking after selective inactivation of cue-responsive neurons in the infralimbic cortex. J. Neurosci. 35, 10750–10761 (2015).
Hermann, A., Stark, R., Milad, M. R. & Merz, C. J. Renewal of conditioned fear in a novel context is associated with hippocampal activation and connectivity. Soc. Cogn. Affect Neurosci. 11, 1411–1421 (2016).
Milad, M. R. et al. Thickness of ventromedial prefrontal cortex in humans is correlated with extinction memory. PNAS 102, 10706–10711 (2005).
Scharfenort, R. & Lonsdorf, T. B. Neural correlates of and processes underlying generalized and differential return of fear. Soc. Cogn. Affect Neurosci. 11, 612–620 (2016).
Noori, H. R. et al. A multiscale cerebral neurochemical connectome of the rat brain. PLoS Biol. 15, 1–23 (2017).
Chiba, T., Kayahara, T. & Nakano, K. Efferent projections of infralimbic and prelimbic areas of the medial prefrontal cortex in the Japanese monkey, Macaca fuscata. Brain Res. 888, 83–101 (2001).
Ghashghaei, H. T. & Barbas, H. Pathways for emotion: Interactions of prefrontal and anterior temporal pathways in the amygdala of the rhesus monkey. Neuroscience 115, 1261–1279 (2002).
Vervliet, B., Baeyens, F., Van den Bergh, O. & Hermans, D. Extinction, generalization, and return of fear: a critical review of renewal research in humans. Biol. Psychol. 92, 51–58 (2013).
Haaker, J., Lonsdorf, T. B., Thanellou, A. & Kalisch, R. Multimodal assessment of long-term memory recall and reinstatement in a combined cue and context fear conditioning and extinction paradigm in humans. PLoS ONE 8, 1–10 (2013).
Hofmann, W., De Houwer, J., Perugini, M., Baeyens, F. & Crombez, G. Evaluative conditioning in humans: a meta-analysis. Psychol. Bull. 136, 390–421 (2010).
Beckers, T., Krypotos, A. M., Boddez, Y., Effting, M. & Kindt, M. What’s wrong with fear conditioning? Biol. Psychol. 92, 90–96 (2013).
Papachristou, H., Nederkoorn, C., Beunen, S. & Jansen, A. Dissection of appetitive conditioning. Does Impuls. play a role? Appetite 69, 46–53 (2013).
Van Gucht, D., Baeyens, F., Vansteenwegen, D., Hermans, D. & Beckers, T. Counterconditioning reduces cue-induced craving and actual cue-elicited consumption. Emotion 10, 688–695 (2010).
Chase, H. W., Eickhoff, S. B., Laird, A. R. & Hogarth, L. The neural basis of drug stimulus processing and craving: an activation likelihood estimation meta-analysis. Biol. Psychiatry 70, 785–793 (2011).
Ersche, K. D., Williams, G. B., Robbins, T. W. & Bullmore, E. T. Meta-analysis of structural brain abnormalities associated with stimulant drug dependence and neuroimaging of addiction vulnerability and resilience. Curr. Opin. Neurobiol. 23, 615–624 (2013).
Reiter, A. M. F. et al. Behavioral and neural signatures of reduced updating of alternative options in alcohol-dependent patients during flexible decision-making. J. Neurosci. 36, 10935–10948 (2016).
Sebold, M. et al. When habits are dangerous: alcohol expectancies and habitual decision making predict relapse in alcohol dependence. Biol. Psychiatry 82, 847–856 (2017).
Voon, V., Reiter, A., Sebold, M. & Groman, S. Model-based control in dimensional psychiatry. Biol. Psychiatry 82, 391–400 (2017).
Moeller, S. J. & Paulus, M. P. Toward biomarkers of the addicted human brain: using neuroimaging to predict relapse and sustained abstinence in substance use disorder. Prog. Neuro-Psychopharmacol. Biol. Psychiatry 80, 143–154 (2018).
Cartoni, E., Balleine, B. & Baldassarre, G. Appetitive Pavlovian-instrumental transfer: a review. Neurosci. Biobehav. Rev. 71, 829–848 (2016).
Loftus, G. R. & Masson, M. E. Using confidence intervals in within-participant designs. Psychon. Bull. Rev. 1, 476–490 (1994).
Masson, M. E. & Loftus, G. R. Using confidence intervals for graphically based data interpretation. Can. J. Exp. Psychol. 57, 203–220 (2003).
This work was supported by the Deutsche Forschungsgemeinschaft (DFG grants to FS SCHL1969/4–1 and SCHL1969/2–2 as part of FOR1617). CE was supported by an Elsa-Neumann scholarship and a Charité scholarship. We thank Lisa Nitsche and Nathalie Scriba for help with data acquisition.