Introduction

The cost to individuals and societies throughout the world of drug and behavioral addictions remains enormous [1, 2]. Despite this outcome, drug use and problem behaviors are maintained and often self-propagate. Attempts to identify factors predisposing individuals to addiction have proposed pre-existing conditions and traits. These include reactivity to novelty [3] and dopamine (DA) associated impulsivity [4], suggested to respectively predict the initiation and persistence of drug taking [5]. Progressive effects stemming from drug exposure leading to incentive sensitization [6, 7] or recruitment of opponent processes [8], and interactions between pre-existing conditions and drug-induced effects [9, 10] have also been proposed. Depending on the manner and timing of drug exposure, effects could influence the progression and persistence of drug taking in adult users [11] or could manifest as predisposing conditions in individuals exposed in utero or adolescence [12, 13] or even via epigenetic inheritance from a drug taking male parent [14].

While the preponderance of work on addiction has focused on drug-induced neuronal plasticity [15, 16] and how it may impact vulnerable individuals [17], the effects of exposure to non-drug stimuli and particularly experienced reinforcement contingencies have received relatively little attention. As with drug abuse, gambling disorder is a behavioral addiction associated with progressive symptoms of compulsion, loss-of-control, and continued pursuit despite negative consequences [18]. Because the incorporation of uncertainty is central to games of chance and is necessary to maintain their pursuit [19], we asked here whether prolonged intermittent exposure to uncertainty can induce neuroadaptations similar to those produce by abused drugs to enhance drug seeking and taking. Indeed, gambling disorder and drug abuse comorbidity is common [20]. A priming dose of amphetamine can increase motivation to gamble [21] and pathological gamblers show evidence for greater amphetamine-induced striatal DA release [22], suggesting that the two disorders share overlapping mechanisms and are susceptible to cross-sensitization [23, 24]. Some recent preclinical studies have directly supported this possibility, but are limited to demonstrations that exposure to uncertainty enhances the locomotor response to amphetamine [25,26,27].

Thus, in the present experiments, we assessed the effects of prior exposure to uncertainty on a number of biochemical and behavioral effects normally observed in rats previously exposed to psychostimulant drugs. This exposure led to enhanced locomotor and nucleus accumbens (NAcc) DA responding to amphetamine, and heightened levels of drug taking consistent with excessive incentive wanting. These results show in drug naive animals direct effects of exposure to uncertainty on NAcc DA reactivity and drug intake that provide a novel pathway to drug abuse. Additional findings suggest that uncertainty may achieve these effects by mimicking those of abused drugs on the midbrain DA system during intermittent exposure.

MATERIALS AND METHODS

Subjects

Male Long–Evans rats weighing 250–275 g on arrival from Harlan Teklad (Madison, WI) were individually housed with food and water available ad libitum in a 12-h light/12-h dark reverse cycle room. All procedures were initiated following a 5 day acclimation period and conducted during the dark cycle according to an approved Institutional Animal Care and Use Committee protocol. In some cases, rats were surgically prepared to allow for in vivo microdialysis of the NAcc or intravenous (IV) amphetamine self-administration (Supplementary Materials).

Exposure to conditions of uncertainty

To manipulate exposure to uncertainty, rats in different groups were given the opportunity to nose-poke for a non-caloric saccharin solution (0.3% in water) under either certain [fixed-ratio (FR)] or uncertain conditions [variable-ratio (VR)] for 55 1-h sessions given twice/day 6-days/week using procedures previously described [25]. FR and VR ratio schedules were escalated as follows when rats obtained at least 20 saccharin reinforcers in two consecutive sessions: 1, 2, 3, 5, 7, 10, 13, 16, 19, and 20. Rats were maintained on the last ratio (FR/VR 20) for 20–25 sessions. These constituted the period of prolonged exposure to uncertainty in the VR rats. During the saccharin nose-poking sessions, number of nose-pokes, dipper entries, saccharin reinforcers, and sessions spent on each ratio were recorded. These procedures were conducted in operant boxes situated in sound attenuating chambers (Supplementary Materials).

The act of gambling is complex and can incorporate a number of features, including loss chasing, betting, and risk of loss ([27]; and references therein). By exposing rats to the VR condition, the present approach sought to isolate uncertain receipt of a reinforcer as the test variable, reasoning that it is a critical feature of many, if not all, forms of gambling [19]. This does not preclude examination of other features of gambling, chronic exposure to which may also lead to long-lasting neuroadaptations.

Effects of prior exposure to uncertainty

Two to three weeks following the last saccharin nose-poking session, separate groups of rats were tested for their locomotor or NAcc DA response to amphetamine or their self-administration of the drug using a lever press operant.

Locomotor activity

Following a 30-min habituation to open fields (Supplementary Materials), rats were injected with saline (1 ml/kg, IP), placed again in the open fields and locomotion recorded for 30-min. They were then removed, injected with amphetamine (1.0 mg/kg, IP), returned again to the open fields, and their locomotion recorded for an additional 2 h.

Microdialysis—NAcc DA

Rats were slightly anesthetized with isoflurane and concentric microdialysis probes (CMA11, Harvard Apparatus) lowered into the NAcc core for microdialysis testing the following day (2 mm active membrane length protruding from the tip of the guide cannula). The NAcc core was selected because enhanced DA overflow has been reported in this subregion following psychostimulant exposure ([7, 28]; and references therein). Overnight probe equilibration and testing the following day was conducted as described previously [29]. On the test, three baseline samples were collected. Rats were then injected with amphetamine (1 mg/kg, IP) while still tethered for microdialysis and 6 additional samples collected. Dialysates were collected every 20-min and frozen at −80 °C for later assessment of DA levels by HPLC-EC (Supplementary Materials). Probe location in the NAcc core (Figure S1) was confirmed in cresyl violet stained sections as previously described [30].

Amphetamine self-administration

Five to seven days following surgical catheterization, rats connected via a steel-spring tether to a liquid swivel and infusion pump positioned outside an operant box were subjected to a short training period. They were then tested as previously described [29] on a progressive ratio (PR) schedule of reinforcement [31] for their self-administration of amphetamine (100 µg/kg/infusion) in each of six daily 2-h sessions. IV catheters were flushed daily with a sterile 0.9% saline solution containing 30 IU/ml heparin and 250 mg/ml ampicillin in order to maintain patency. Catheter viability was assessed throughout testing. These procedures were conducted in a separate set of operant boxes that differed from those described above for saccharin self-administration in that they had different flooring, bedding, and walls, no dipper, and a lever rather than the nose-poke holes requiring performance of a new operant to obtain amphetamine (Supplementary Materials).

NAcc DA during uncertainty

To further characterize effects produced during uncertainty that may lead to long-lasting changes in neurotransmission and behavior, additional rats in different groups were trained to nose-poke for saccharin as outlined above and subjected to microdialysis of the NAcc at FR/VR ratios 5, 13–16, or 19–20. Rats performing under FR/VR ratios 13 and 16 were grouped as were rats performing under ratios 19 and 20. DA levels in the NAcc core were assessed because midbrain DA neurons show the greatest sustained activation during periods of maximal uncertainty [32], this is measurable as an increase in sustained release in this subregion [33], and the resulting intermittent DA neuron activation produced on each of the 20–25 sessions of prolonged exposure to uncertainty could provide a mechanism for the induction of long-lasting sensitization [7]. Rats were prepared for microdialysis as described above with overnight equilibration taking place in 5 gal plastic buckets located in the operant chamber test room. Dialysates were collected every 2-min and frozen at −80 °C for later assessment of DA levels by HPLC–MS/MS ([34]; Supplementary Materials). On the test session, five samples were collected in the overnight bucket, the rats then moved to the operant chambers with the nose-poke holes occluded and five baseline samples collected, and then the nose-poke holes made available and 30 samples collected during the 1-h saccharin self-administration session. Probe location in the NAcc core (Figure S2) was confirmed in cresyl violet stained sections as previously described [30].

Statistical analyses

Repeated between-within analyses of variance (ANOVA) were used to analyze number of sessions spent, nose-pokes and dipper entries emitted, and saccharin reinforcers obtained on each ratio during exposure, and locomotor counts, NAcc DA, and number of amphetamine infusions obtained during testing when assessing the effects of exposure. Group (FR or VR) was the between factor and schedule ratio, time, or day were the within factors. An independent samples t-test was used to analyze days to criterion during training for amphetamine self-administration. A between-groups ANOVA with group (FR or VR) and ratio (5, 13–16, or 19–20) as the between factors was used to analyze NAcc DA, nose-pokes, dipper entries, and saccharin reinforcers when assessing NAcc DA during exposure. To assess the relationship between NAcc DA and emission of behaviors (nose-pokes and dipper entries) or obtainment of reinforcers at each 2-min bin in these sessions, Pearson correlation coefficients were calculated and tested for significance. When specific group mean comparisons were required following ANOVA, post hoc Scheffé tests were conducted. All calculations were conducted with the IBM SPSS Statistics 22 Module.

Results

Exposure to conditions of uncertainty

As we reported previously [25], following a learning period under the first reinforcement schedule (FR/VR 1) in which rats associated nose-pokes with delivery of saccharin in the dipper, rats reliably nose-poked and consumed the saccharin on all subsequent sessions. Similarly, the number of nose-pokes into the inactive hole declined precipitously in the first few sessions on FR/VR 1 and remained at <10/session on all remaining sessions. Importantly, no differences were observed between the certain (FR) and uncertain (VR) conditions in the number of sessions spent on each ratio under FR/VR 1 (acquisition of this association), in subsequent sessions in which the reinforcement schedule increased, or under the final schedule (FR/VR 20) constituting the period of prolonged exposure to uncertainty in the VR rats (Fig. 1a; F1,46 = 1.83, ns, group; F9,414 = 0.67, ns, group × ratio interaction). Similarly, no group differences were observed throughout exposure in number of nose-pokes (Fig. 1b; F1,46 = 0.13, ns) and dipper entries emitted (Fig. 1c; F1,46 = 1.29, ns) or in the number of saccharin reinforcers obtained (Fig. 1d; F1,46 = 0.40, ns). As expected, as the work demands of the ratios increased over the sessions, the number of nose-pokes correspondingly increased while the number of dipper entries and saccharin reinforcers obtained decreased. A significant effect of ratio was detected for each of these measures (F9,414 = 87.0, nose-pokes, 40.52, dipper entries, 31.51 reinforcers, all p < 0.001) but significant group × ratio interactions were not detected (F9,414 = 0.25, nose-pokes, 0.66, dipper entries, 0.62, reinforcers, all ns). Thus, long-lasting effects observed following this exposure period could not be explained by group differences in any of these measures.

Fig. 1
figure 1

Rats responding for saccharin under Certain (FR) or Uncertain (VR) conditions showed no group differences in number of sessions spent (a), number of nose-pokes (b) and dipper entries (c) emitted, as well as number of saccharin reinforcers (d) obtained on each ratio. Data are shown as group means (±SEM). Numbers at ordinates indicate counts. Numbers in parentheses indicate n/group

Full size image

Effects of prior exposure to uncertainty

Two to three weeks following the last saccharin nose-poking session, separate subgroups of these rats were tested for their locomotor or NAcc DA response to amphetamine or their self-administration of the drug using a lever press operant.

Locomotor activity

As reported previously [25,26,27], prior exposure to uncertainty enhanced the subsequent locomotor response to amphetamine (Fig. 2a). An overall significant group effect was not detected but a significant effect of time (F11,110 = 15.93, p < 0.001) and a significant group × time interaction (F11,110 = 2.04, p < 0.05) were. Post hoc tests showed significant group differences at the peak of the amphetamine locomotor response. No significant effects were detected at baseline prior to the amphetamine challenge.

Fig. 2
figure 2

Rats previously exposed to conditions of Uncertain reinforcement (VR) showed enhanced locomotor (a) and NAcc DA (b) responding to a challenge injection of amphetamine (arrow at abscissa), as well as enhanced work output and self-administration of the drug (c) compared to rats previously exposed to Certain reinforcement (FR). Data are shown as group means (±SEM). Numbers in parentheses indicate n/group. *p < 0.05, **p < 0.01, compared to certain at the specified time

Full size image

NAcc DA

Although increased locomotion is sometimes taken as a proxy of increased midbrain DA activity, a direct demonstration that exposure to uncertainty leads to a greater NAcc DA response to amphetamine has yet to be reported. We show here that exposure to uncertainty in fact sensitizes the ability of amphetamine to subsequently increase NAcc DA overflow (Fig. 2b). Again, an overall significant group effect was not detected but a significant effect of time (F5,90 = 38.23, p < 0.001) and a significant group × time interaction (F5,90 = 3.29, p < 0.05) were. Post hoc tests showed significant group differences at the peak of the NAcc DA response. Absolute baseline levels of DA observed prior to the amphetamine challenge (FR, 100.99 ± 20.44; VR, 80.77 ± 12.04; pg/25 µl; baseline means ± SEMs) did not differ significantly between the two groups (Figure S3 shows additional pre- and post-amphetamine analyses of absolute DA levels).

Amphetamine self-administration

Psychostimulant exposed rats that exhibit sensitized midbrain DA neuron reactivity also show enhanced work output and self-administration of these drugs [7]. We report here that drug naive rats previously exposed to conditions of uncertainty remarkably show the same increase in subsequent work output and self-administration of amphetamine (Fig. 2c). As previously reported following drug exposure [29], no significant group differences were observed in days to criterion during training (FR: 2.78 ± 0.43; VR, 2.57 ± 0.3; t14 = 0.37, ns). However, during PR testing, ANOVA found a significant overall group effect (F1,14 = 4.89, p < 0.05). By day 5 of testing, VR rats previously exposed to uncertainty showed a close to sixfold increase in work output and significantly more amphetamine infusions obtained (p < 0.01) compared to control FR rats previously exposed to certain contingencies between nose-pokes and saccharin reinforcement. Together with the above locomotor and NAcc DA results, these findings indicate that exposure to uncertainty can induce neuroadaptations similar to those produce by abused drugs to enhance drug taking.

NAcc DA during uncertainty

To further characterize effects produced during uncertainty that may lead to the long-lasting changes in neurotransmission and behavior described above, additional rats in different groups were trained to nose-poke for saccharin as outlined above and subjected to microdialysis for NAcc DA at FR/VR ratios 5, 13–16, or 19–20. During the saccharin sessions, rats working under the FR ratios were subjected to certain relationships between nose-pokes and the saccharin reinforcer; whereas, the VR rats were subjected to increasingly uncertain relationships. Noting that uncertainty can be measured as the variance (deviation from the mean) of a ratio probability distribution [35], we first plotted operant variance values for the different fixed and variable ratios experienced during the saccharin sessions. As shown in Fig. 3a, the variance associated with the different escalating FRs remains at 0 as these program a fixed relationship between nose-pokes and payout while the variance associated with the different escalating VRs increases exponentially as these program an increasingly variable relationship. Remarkably, NAcc DA overflow averaged throughout the saccharin sessions tracked this measure of uncertainty (Fig. 3b). FR rats showed NAcc DA levels that remained at baseline despite increasing ratios whereas VR rats showed an exponential increase in DA overflow that paralleled the growing variance of the increasing variable ratios. The ANOVA revealed significant effects of groups (F1,28 = 14.92, p < 0.001) and ratio (F2,28 = 3.71, p < 0.05) and a significant group × ratio interaction (F2,28 = 3.64, p < 0.05). Absolute levels of DA did not differ significantly between the FR and VR conditions in the overnight (1.14 ± 0.14 to 1.91 ± 1.10 mean nM ± SEM) and the baseline (1.15 ± 0.22 to 1.701 ± 0.90 mean nM ± SEM) sampling periods. At ratio 19–20, VR rats showed a >65% increase in DA overflow compared to FR rats (p < 0.001) and showed substantially greater than baseline levels throughout the saccharin session (Fig. 3c-e). Unlike DA, numbers of nose-pokes, dipper entries, and saccharin reinforcers obtained did not track uncertainty (Fig. 3f–h). As in Fig. 1, number of nose-pokes increased with increasing ratios but did so in both the FR and VR conditions. ANOVA revealed a significant effect of ratio (F2,28 = 19.93, p < 0.001) but no significant effect of groups or groups × ratio interaction. Likewise, no significant group differences were detected for dipper entries (F2,28 = 0.63, ns) and saccharin reinforcers obtained (F2,28 = 1.01, ns). To assess further the relationship between DA and emission of behaviors (nose-pokes and dipper entries) and reinforcers obtained, correlations were determined between DA and these measures at 2-min bins during the saccharin test session. As illustrated in Fig. 3i–k for nose-pokes, a simple relationship between DA and expression of behavior was not detected. Low and non-significant correlations were found at all six ratio levels tested (r = 0.03–0.15, ns). In both the certain (FR) and uncertain (VR) conditions, DA levels remained relatively constant with different numbers of nose-pokes emitted per bin. Similar findings were obtained for dipper entries and reinforcers obtained. Together, these findings suggest that during exposure to uncertain conditions, NAcc DA shows a sustained increase in overflow that is not associated with emission of different behaviors but tracks the variance of the ratios scheduled.

Fig. 3
figure 3

During exposure to the certain (FR) and uncertain (VR) saccharin reinforcement contingencies, NAcc DA tracked the variance of the scheduled ratios but not the emission of behavior. a Variance (a measure of uncertainty) associated with the different FR and VR ratios scheduled. b NAcc DA shown as group mean session averages (±SEM) expressed as % of baseline. n/group for ratios 5, 13–16, 19–20: FR, 8, 6, 5; VR: 7, 4, 4. ***p < 0.001, compared to FR at the specified ratio. Time course for these data, as well as overnight (ON) and baseline (BL) values are shown in (c–e). f–h Numbers (group means ± SEM) of nose-pokes, dipper entries, and saccharin reinforcers obtained did not track uncertainty. i–k No significant correlations were detected between the number of nose-pokes emitted and NAcc DA levels observed at each 2-min bin. Lines indicate simple linear regressions

Full size image

DISCUSSION

In the present experiments, we found that prior exposure to conditions of uncertainty enhanced subsequent locomotor and NAcc DA responding to amphetamine and heightened self-administration of the drug. These effects, consistent with effects normally observed in rats previously exposed to psychostimulant drugs, were produced in drug naive animals, demonstrating direct effects of exposure to uncertainty on NAcc DA reactivity and drug intake that provide a novel pathway to drug abuse. The additional finding that NAcc DA tracked uncertainty during operant responding for saccharin provides a likely mechanism, again observed with abused drugs, whereby repeated intermittent increases in DA can lead to long-lasting changes in neurotransmission and behavior.

Substance use and gambling disorders exhibit comorbidity [20]. Cocaine-dependent individuals show a higher prevalence of pathological gambling than observed in the general population [36] and drugs like amphetamine are known to increase motivation to gamble [21] and to elicit enhanced striatal DA release [22] in pathological gamblers. In preclinical studies, rats previously exposed to psychomotor stimulant drugs show enhanced responding to and for amphetamine [7] and increased risky decision-making in a rodent gambling task [37]. Similarly, rats previously exposed to conditions of uncertainty also show enhanced responding to and for amphetamine (present findings), as well as increased risky decision-making [27]. The rodent gambling task used in these studies is thought to capture a bias in decision-making common to both gambling and substance use disorders [27]. The experience of uncertainty also increases the incentive salience of and approach to both normally unattractive and previously conditioned cues [38,39,40]. These findings together with those reported here showing that exposure to uncertainty in and of itself can enhance subsequent NAcc DA responding to and self-administration of amphetamine, support a unified theory of addiction in which excessive incentive wanting drives the maintenance and progression of both drug and behavioral addictions, such as gambling disorder. Thus, repeated intermittent exposure to uncertainty, manipulated here not as a model of gambling but rather as a central feature of games of chance necessary to maintain their pursuit [19], led to incentive sensitization resulting in excessive drug intake.

When characterizing the effects produced during exposure by the uncertain saccharin reinforcement contingencies, we found that NAcc DA showed a sustained increase in overflow that was not associated with emission of behaviors but rather tracked the variance of the scheduled ratios. This is reminiscent of earlier as well as more recent demonstrations of sustained midbrain DA neuron activation measured either as single unit recordings [32] or DA release in the NAcc assessed by voltammetry [33]. In both cases, sustained DA activity was greatest during periods of maximal uncertainty and was dissociable from stimulus-induced phasic activity. While the timescales in these studies (seconds) were vastly different from those in the present study (minutes), similar associations between uncertainty and activity [41] or DA release [42] in dopaminergic regions like the ventral striatum during a gambling task have been reported in human fMRI (2–4 s resolution) and PET (20–30 min resolution) studies, suggesting with the present findings that DA signaling can track uncertainty over a wide temporal range. In these studies, cues associated with reinforcement probabilities ranging from 0 to 1 were manipulated to produce an inverted U function between uncertainty and probability of reinforcement, with p = 0 or 1 associated with no uncertainty, intermediate p’s associated with growing uncertainty, and the central most p = 0.5 associated with the greatest. Here, we describe an operant conditioning approach that supports a similar relationship between uncertainty and sustained DA activation but does so over exponentially increasing levels of uncertainty in a manner that controls for group differences in acquisition and emission of the operant, as well as reinforcements obtained. In this case, the variance associated with the different escalating FR schedules remained at 0 as these programmed a fixed relationship between nose-pokes and payout while the variance associated with the different escalating VR schedules increased exponentially as these programmed an increasingly variable relationship (Fig. 3a), which, again, was associated with an increase in sustained midbrain DA activation measurable as sustained DA overflow in the NAcc.

This relationship between uncertainty and NAcc DA overflow was not reported in a study using a different operant approach. In this study [43], rats were trained to lever press for electrical stimulation of the medial forebrain bundle under a predictable fixed or unpredictable variable interval 12 s reinforcement schedule. Despite evidence that the schedules elicited typically different patterns of behavior, the DA overflow they produced in the NAcc was strikingly similar. It is difficult to reconcile these findings with those described above. As these authors indicate, however, there is evidence that the tonic and phasic DA effects associated with electrical brain stimulation may differ from those associated with the gustatory reinforcers used here and elsewhere [25, 32, 33], possibly rendering it unable to isolate a tonic DA uncertainty signal. Importantly, questions remain about the origin of this signal and its relationship to cue and reinforcer induced phasic DA responses [33, 44,45,46]. Whatever the case, we propose that the sustained DA activation produced on each of the 20–25 intermittent uncertainty sessions in the present experiments mimics what is observed with repeated intermittent psychostimulant injections and thus could provide a mechanism for the induction of long-lasting sensitization [7]. A good candidate for further study in this context is the ventral hippocampus—NAcc—ventral pallidal—ventral tegmental area pathway. This pathway is known to regulate tonic midbrain DA transmission [47] and to participate in sensitization by amphetamine [48].

While the present experiments focused on the NAcc core, enhanced DA overflow has also been reported in the NAcc shell following psychostimulant exposure [49] and during a decision-making task performed under conditions of reward uncertainty [50]. These findings indicate that the sustained activation of midbrain DA neurons observed during periods of uncertainty may also include those neurons projecting to this subregion thereby recruiting their participation as well to the induction of sensitization.