INTRODUCTION

Disorders of the dopamine system, such as schizophrenia, Parkinson's disease, or stimulant addiction, are associated with impairments in different forms of cost/benefit decision making (Rogers et al, 1999; Murphy et al, 2001; Walton et al, 2002; Shurman et al, 2005; Hoffman et al, 2006). One form of decision making that has received considerable attention entails situations where an organism chooses between options that yield either smaller, easily accessible rewards, or more valuable rewards that may be obtained with considerable more effort. It is well established that effort-based decision making is critically dependent on mesolimbic dopamine transmission, as systemic treatment with dopamine antagonists or dopaminergic lesions of the ventral striatum markedly reduces preference for larger rewards associated with a greater effort cost (Salamone et al, 2007; Floresco et al, 2008b; Salamone et al, 2009). Furthermore different nodes of dopaminergic circuits are important in mediating effort-related judgments, including the basolateral amygdala (Floresco and Ghods-Sharifi, 2007; Ghods-Sharifi et al, 2009), anterior cingulate of the prefrontal cortex (Walton et al, 2002), and nucleus accumbens core (Floresco, 2007; Botvinick et al, 2009).

Studies investigating the neurochemical underpinnings of effort-based decision making have almost exclusively employed male subjects. However, it is important to note that ovarian hormones, in particular, estradiol, exert complex modulatory control over dopamine (Segarra et al, 2010; Zhao and Becker, 2010; Jacobs and D’Esposito, 2011a) that may in turn alter cost/benefit evaluations. A wide range of behaviors and neurobiological mechanisms are modulated by estradiol, including neuroprotection of dopamine neurons (Kuppers et al, 2000). Estradiol affects dopamine autoreceptors, enhances binding on D2 dopamine receptors, and increases excitability of receptors on dopamine terminals (Thompson and Moss, 1994; Becker, 1999; Becker and Hu, 2008). All of these mechanisms result in enhanced and/or prolonged effects of dopamine. Additionally, the rapid onset as well as the enduring effects of estradiol suggests both genomic and non-genomic mechanisms mediate estradiol's ability to enhance dopamine transmission (McEwen, 1991; Simoncini and Genazzani, 2003).

Estradiol also exerts dose-dependent effects on cognition mediated by regions thought to be critical for normal decision making including the hippocampus, prefrontal cortex, striatum, and amygdala (Luine et al, 1998; Galea et al, 2001; Holmes et al, 2002; Wide et al, 2004; Sherwin, 2006; Sinopoli et al, 2006; Barha et al, 2010; Barha and Galea, 2010). Specifically low levels of estradiol facilitate, whereas high levels impair prefrontal and hippocampus-dependent learning (Holmes et al, 2002; Wide et al, 2004; Sinopoli et al, 2006; Barha et al, 2010). Indeed some of estradiol's effects on cognition are modulated by its effects on dopaminergic transmission within these regions (Quinlana et al, 2008). Given these findings, estradiol may also impact effort-related judgments that are critically dependent on mesocorticolimbic dopamine circuitry.

There are two known estrogen receptors (ERα and ERβ) encoded by different genes and are widely expressed in the mammalian central nervous system (Shughrue et al, 1997; Shughrue and Merchenthaler, 2000; Warembourg and Leroy, 2004). For example, mRNA expression for both ERα and ERβ was found in the female hippocampus, although more expression of ERα mRNA is detectable in the basolateral amygdala, and only ERβ mRNA in the nucleus accumbens and prefrontal cortex (Shughrue et al, 1997; Shima et al, 2003). Estradiol works genomically and non-genomically on both ER subtypes, and synergistically with neurotransmitters (ie dopamine) to exert their effects. Many of estradiol's effects on neuronal properties are fast and easily reversible, suggesting non-genomic actions, whereas the genomic effects typically occur 24–48 h after administration (Woolley, 1999).

Given estradiol's effects on cognitive domains (Galea et al, 2001; Galea et al, 2008; Barha et al, 2010; Barha and Galea, 2010) the interplay of estradiol and dopamine (Chiodo et al, 1986; Luine et al, 1998; Becker, 1999), and the presence of ERs within key brains regions implicated in cost/benefit decision making (Shimizu and Bray, 1993; Shughrue et al, 1997; Shughrue and Merchenthaler, 2000; Shima et al, 2003), the present study examined the influence of ovarian hormones in female Long-Evans rats using a well-established effort-discounting task (Floresco et al, 2008b). In the present study we used intact and ovariectomized subjects and administered either estradiol, an ERα agonist, ERβ agonist, or both and recorded choice preference during an effort-based decision making task. Given the potentiating effects of estradiol on dopamine along with the presence of ERs within the neurocircuitry implicated in mediating cost/benefit decision making, we hypothesized that alterations in estradiol activity would exert a modulatory effect on effort-based decision making in female rats.

MATERIALS AND METHODS

Animals

Adult female Long-Evans rats (210–244 g, N=16; Charles River, Montreal, QC, Canada) were single housed and maintained to ∼85% of free feeding weight, allowing a gain of 3 g per week for natural growth, and given ad-libitum water. The colony room was maintained on a 12 12 hours light–dark cycle (on at 0700 hours), and testing occurred between 1000–1400 hours. Experimentation was in accordance with the Canadian Council of Animal Care and was approved by the University of British Columbia's Animal Care Committee.

Apparatus

Two sound-attenuating operant chambers (30.5 × 24 × 21 cm; Med-Associates, St. Albans, VT, USA) were used. Each chamber was fitted with two retractable levers and a food receptacle in between to allow for food reinforcement (45 mg sugar pellets; Bioserv, Frenchtown, NJ, USA). A single light located in the top-center of the wall opposite the levers illuminated the chamber. Experimental data were recorded using MedPC software.

Training

Training protocols adapted from Cardinal et al (2000) have been described previously (Floresco et al, 2008b; Ghods-Sharifi and Floresco, 2010). Rats were trained under a fixed-ratio (FR) 1 schedule to a criterion of 50 presses in a 30-min session (one session per day), first for one lever then for the other (counterbalanced left/right between subjects) across 2 consecutive days. Later training occurred on a simplified version of the full task, where rats were randomly presented with one of the two levers over 90 training trials and one press on the lever delivered a single sugar pellet. Before the commencement of the trial, the levers were retracted and the house light turned off. Every 40 s, the house light would illuminate and one of the two levers would be inserted into the chamber. Omissions were scored when subjects failed to respond to the extended lever within 10 s, whereby the lever would retract and the chamber would darken. However, a response on the extended lever would result in the retraction of the lever and the immediate delivery of a single sugar pellet. The house light remained illuminated for another 4 s. For each set of trials, the left or the right lever would be presented once randomly. Before moving on to the full task, a criterion of 80 or more successful trials (⩽ 10 omissions) was reached by each subject for at least four consecutive training sessions.

Decision-Making Task

Effort discounting: Figure 1 illustrates the basic procedures in this task. Each day, animals completed a 32 min session consisting of 48 discrete trials, each separated into four blocks. Each block of trials began with two forced-choice trials. On these trials, only one of the two levers was randomly presented. During the next 10 trials, both levers were presented and rats had a free choice between the two levers. Throughout the inter-trial state, the chamber was in darkness and both levers were retracted. At 40 s intervals, a new trial began along with the illumination of the house light, followed by the extension of both levers 3 s later. One lever was designated as the low reward (LR) lever, and the other as the high reward (HR) lever. These levers were counterbalanced (left/right) between animals, and remained constant for the duration of the experiment. Once the levers were presented, the animal had 25 s to make a response; failing to do so is counted as an omission, and the chamber was reset. A single press of the LR lever resulted in the retraction of both levers and the immediate delivery of two pellets. However, after the first response on the HR lever, the LR lever was immediately retracted, and the HR lever remained inserted in the chamber until completion of the required FR of presses. The FR requirement for the HR lever increased within the session (described below). Upon completion of the FR requirement for the HR, the HR lever retracted, four pellets were immediately delivered 0.5 s apart, and the house light remained on for another 4 s. The chamber then darkened, and was reset to the inter-trial state.

Figure 1
figure 1

Effort-based decision-making task. An illustration of the basic procedures in this task. Each session is comprised of 48 discrete trials. A house light comes on, both levers are presented, and the animal has a free choice between the two levers: the low-reward (LR) lever, and the high-reward (HR) lever. Once presented, the animal has to make a response within 25 s; failing to do so is counted as an omission, and the chamber is reset to the intertrial state (darkness). A single press of the LR lever resulted in the retraction of both levers and the immediate delivery of two pellets. However, after the first response on the HR lever, the LR lever is immediately retracted, and the HR lever remains inserted in the chamber until completion of the required fixed ratio (FR) of presses. Upon completion of the FR requirement for the HR, the HR lever retracts, four pellets are immediately delivered. The chamber then darkens, and is reset to the intertrial state. Additionally, the FR requirement for the HR lever increases within the session. The FR of lever presses required to obtain the HR increases discretely over the four blocks of trials, beginning with 2 presses, then 5, 10, and finally 20 presses, respectively.

PowerPoint slide

Full size image

The FR of the lever presses required to obtain the HR increased over the four blocks of trials, beginning with 2 presses, then 5, 10, and finally 20 presses. On the rare occurrence when a rat failed to complete the required number of presses on the HR lever within 25 s after its extension, the lever retracted without delivery of food, and the chamber was reset. However, the animal's choice was still incorporated into the data analysis. The number of omissions and the average rate of lever pressing (number per second) from each trial block were added together and the total across all four blocks was used for statistical analysis. Training on the task continued until rats as a group (1) chose the HR lever during the first trial block (FR 2) on at least 70% of successful trials and (2) demonstrated stable baseline levels of choice. Stable baseline performance was determined using a procedure described by (Winstanley et al, 2004) in which data from three consecutive sessions were analyzed using a repeated-measures ANOVA with two within-subjects factors (day and trial block). If the effect of trial block was significant at the p<0.05 level but there was no main effect of training day or training day × trial block interaction, animals were judged to have achieved stable performance. Once stable performance was reached, rats received surgery (OVX or Sham). After recovery (1 week), rats were retrained on the task until a stable level of choice was obtained.

Surgery

Subjects were either bilaterally ovariectomized (OVX; N=10) or sham ovariectomized (Sham; N=6). Rats were anesthetized using an initial flow rate of 5% of isofluarane (Boxter, Mississauga, ON, Canada) and 3% during surgery. Five mg/kg Anafen (Merial Canada, Baie-d’Urfe, QC, Canada) and 5 ml Lactated Ringer Solution (Braun Medical, Scarborough, ON, Canada) were administered subcutaneously (s.c.) post-surgery. After 6 days of recovery with ad libitum access to food and water, subjects were once again food restricted and tested daily in the operant chambers.

Hormone Injections

Sham rats did not continue in the study beyond 16 days post-surgery, whereas OVX rats received a series of different doses of estradiol, ER agonists, or vehicle (sesame oil) beginning 15 days after post-surgical training. All injections were administered s.c. 4 h before testing. Rats received an injection of 0.1 ml sesame oil as a control, and then a hormone agonist the following day, and then tested 24 h post-hormone injection (see Figure 2 for a timeline of injections). The order of treatment with low (0.3 μg estradiol benzoate (EB)/0.1 ml oil) and high (10 μg EB/0.1 ml oil) doses of EB were counterbalanced. These doses were chosen because they exert differential effects on other forms of cognition (eg spatial memory; Holmes et al, 2002). We tested the low dose of EB twice to assess consistency of behavioral responses to a hormone agonist injection across time. In order to investigate the contribution of each receptor subtype, propyl-pyrazole triol (PPT; ERα agonist) and diarylpropionitrile (DPN; ERβ agonist) were examined independently and cooperatively (10 μg of each/0.1 ml oil). Following injections of EB, testing of PPT and DPN independently was counterbalanced, followed by an s.c. injection of the combination of these agonists.

Figure 2
figure 2

Timeline of injections following ovariectomy (N=10, within subject). Note: Sham rats discontinued before the start of injections. The two doses of EB were counterbalanced, as well as the ER-specific agonists PPT and DPN. Thus with counterbalancing, N=5 for each type of injection unless specified otherwise. Needle symbols indicate injection days, whereas ‘24 h’ refers to the day immediately following the hormone injection. ‘Stable responding (SR)’ indicates days when animals were tested in the operant chambers between injections in order to achieve SR before the next round of injections. The 3 consecutive days included in the statistical analyses are indicated with lines and include the following: (1) oil control injection, (2) hormone injection, and (3) 24 h post-injection. The number of days of each time segment is included in brackets.

PowerPoint slide

Full size image

Determining Stage of the Estrous Cycle

Daily samples of vaginal cells were taken by lavage before surgery and for the first week of post-surgery testing. The samples were analyzed on slides using a Nikon E600 light microscope. As differences in spatial performance are observed during proestrus compared with other stages of the estrous cycle (Warren and Juraska, 2000; Rummel et al, 2010), we discriminated proestrus vs non-proestrus. Proestrus was defined by the presence of ∼70% round nucleated epithelial cells as described in Marcondes et al (2002).

Data Analyses

The proportion of choices directed towards the HR lever for each block of free-choice trials, factoring in trial omissions was calculated by dividing the number of choices of the HR lever by the total number of successful trials. For within-subjects comparisons (proestrus vs non-proestrus; estradiol or ER agonists vs vehicle) choice data were analyzed using two-way, within subjects’ analyses of variance (ANOVA), with estrous phase or treatment day and trial block (number of presses required on HR lever) as the within-subjects factors. For between-subjects comparisons (Sham vs ovariectomy), the data were analyzed with a three-way mixed ANOVA, with the surgery as a between-subjects factor and phase (pre- or post-surgery) and trial block as within-subjects factors. In each of these analyses, the effect of trial block was always significant (all p's<0.001), and will not be reported further. Rates of pressing on the HR lever (presses per second) were analyzed in a manner similar to choice data, except these values were averaged across the four trial blocks. To investigate the longer lasting actions of treatment with estradiol or ER agonists, data from 24 h post-injection was included in the statistical analysis so that the treatment day effect had three levels (day before, day of, and day after treatment). For each set of treatment comparisons (oil, 0.3 μg EB, 10 μg EB, 10 μg PPT, 10 μg DPN, 10 μg PPT+10 μg DPN) the number of HR lever choices and trial omissions were analyzed with separate two-way, repeated-measures ANOVAs with treatment day and trial block as within subject factors. Post-hoc tests utilized Newman–Keuls comparisons.

RESULTS

Effort Discounting in Intact, Cycling Females

After 26 days of training on the effort-discounting task, intact females displayed stable baseline levels of choice for 3 consecutive days, at which point estrous cycle stage was tracked from sessions 27–41. Four females did not cycle during this test period and are not included in this analysis. All proestrus days were averaged for each rat, as well as the session 24 h before and immediately following proestrus. Analysis of the choice data revealed no significant effect of proestrus (p=0.98) or the proestrus × block interaction (p=0.49). Thus, normal fluctuations in ovarian hormones across the estrous cycle do not appear to exert a major influence on this form of cost/benefit decision making.

Ovariectomy Reduces Effort Discounting

After 46–48 test sessions, all rats underwent surgery (OVX or Sham), and 1 week later were retested on the task. Analysis of the choice data obtained before surgery (sessions 37–39, before unlimited access to food was given) compared with choice data obtained post-surgery (sessions 4–6 to allow for stable responding) revealed a surgery × phase interaction (F(1,13)=5.69, p<0.05), and a surgery × trial block interaction (F(3,39)=3.79, p<0.05). Post-hoc tests indicate that both groups did not differ in their overall choice before surgery (p=0.69), and that Sham rats showed similar levels of discounting pre- and post-surgery (p=0.47). In stark contrast, ovariectomy induced a significant increase in number of choices directed toward the HR lever on trial blocks 5, 10 and 20 (p's<0.01; Figure 3). Thus, ovariectomy induced a dramatic shift in the manner in which female rats allocated their responses, showing a greater tendency to select the high effort/HR option.

Figure 3
figure 3

Effect of surgery on effort-based decision making. Ovariectomy (OVX) caused a significant increase in number of choices directed toward the HR lever compared with Sham rats. N=10 and 6 for OVX and Sham surgery groups, respectively. *Denotes significantly different than post-SHAM at p<0.01. Symbols represent mean+SEM.

PowerPoint slide

Full size image

There was a significant main effect of surgery on the number of omissions (F(1,13)=9.38, p<0.01). Post-hoc tests indicate ovariectomized females commit fewer omissions post-surgery compared with pre-surgery numbers and compared with the Sham rats post-surgery (p<0.01), whereas Sham operated rats did not differ in terms of the number of omissions pre- and post-surgery (p=0.29). Importantly, however, surgery did not affect the rate of lever pressing for either the OVX or Sham group (p=0.82). Thus, even though rats in the Sham group made fewer choices of the high effort/HR lever, when they did choose this lever, they responded as robustly as OVX rats. Group means for all analyses on number of omissions and rate of lever pressing is presented in Table 1.

Table 1 Total Number of Omissions and Average Rate of Lever Pressing
Full size table

High Levels of Estradiol Reduce Preference for Larger, More Costly Rewards

The effect of EB on effort discounting was assessed in OVX females after nine post-OVX training sessions, at which point, these rats displayed stable baseline levels of choice behavior. To investigate the lasting effects of genomic actions by estradiol, choice data from 24 h post-injection was included in the analysis. Thus, choice data was analyzed across three treatment days: oil injection, EB injection, and 24 h post-injection of EB. To determine whether the order of treatment may have led to differential effects of estradiol on decision making, we analyzed choice behavior with order of injections as between-subject factors (low then high EB, high then low EB, PPT then DPN, and DPN then PPT) and did not find a significant effect of order following any of the injections (0.3 μg EB, 10 μg EB, PPT, DPN, or PPT+DPN; all p>0.29). There were no significant effects of the 0.3 μg EB injection on choice data across the treatment days following either the first administration, that was counterbalanced with the 10 μg EB injection, or the second administration (p>0.34; values from the first and second injection of low EB were averaged and are shown in Figure 4a). Additionally, response to an acute administration of low EB (0.3 μg) remained stable over time among the OVX rats in the present experiment as responding did not differ between the first and second exposure (p=0.86). For the high dose of EB (10 μg), analysis of the choice data revealed a significant main effect of treatment day (F(2,18)=7.79, p<0.01 but no significant treatment day × trial block interaction (p=0.21; 4B). Post-hoc tests indicated that the high dose of EB decreased choice for the HR lever 24 h post-injection (p<0.01) but not on the day of injection (p=0.77). Thus, treatment with estradiol induced a delayed and dose-dependent reduction in preference for the high effort/HR option in OVX rats.

Figure 4
figure 4

(a) Effect of a lower dose of estradiol benzoate (EB) on effort-based decision making; there was no significant effect on HR responding with a lower dose of EB (0.3 μg). (b) Effect of a higher dose of EB on effort-based decision making. A higher dose of EB (10 μg) significantly decreased choice for HR lever responding the day after, but not the day of injection. *Denotes p<0.01. Symbols represent mean+SEM. N=10.

PowerPoint slide

Full size image

By this stage of training, baseline omission rates were very low (<1 per rat). There was no significant effect on the number of omissions for low EB (p=0.64). However, the high dose of EB induced a slight, but statistically significant increase in omissions 24 h after injection (F(2,18)=6.38, p<0.01, and Newman–Keuls, p<0.05, Table 1). However, treatment with either the low or high dose of EB did not affect the rate of lever pressing (all p>0.10).

The ERα-Selective Ligand PPT Increases Preference for Larger, More Costly Rewards

Following tests with estradiol, we tested the effects of selective ER agonists on effort discounting in the same OVX females using PPT (ERα; 10 μg) or DPN (ERβ; 10 μg). To investigate the lasting effects of genomic actions by selective ER agonists, choice data from 24 h post-injection was included in the analysis, in a manner similar to the analysis of the data from the EB tests. Analysis of the choice data revealed significant main effects of PPT treatment day (F(2,18)=4.35, p<0.05) and a significant treatment day × trial block interaction (F(6,54)=3.56, p<0.01; Figure 5a). In contrast to what was observed with estradiol, post-hoc tests indicated that PPT actually increased choice of the HR lever on the day of PPT injection relative to vehicle treatment with this effect being statistically significant during the third block where the effort requirement was 10 presses (p<0.02). Furthermore, choice of the HR lever was further augmented 24 h post injection compared with both the day of PPT injection and vehicle treatment (p<0.001).

Figure 5
figure 5

(a) Effect of ERα-selective ligand propyl pyrazole triol (PPT) on effort-based decision making. ERα-selective ligand PPT increased responding on the HR lever compared with the control oil injection on the day of PPT and 24 h post-injection, but only when 10 presses were required. Responding on the HR lever was additionally augmented 24 h post-injection when 10 presses were required (p<0.02). (b) Effect of ERβ agonist diarylpropionitrile (DPN) on effort-based decision making. DPN increased responding on the HR lever 24 h post-injection when 10 presses were required, but not on the day of DPN (p<0.001). (c) Effect of ERα-selective ligand PPT and ERβ agonist DPN combination on effort-based decision making. The combination of PPT and DPN decreased responding on the HR lever 24 h post-injection (p=0.06), but not on the day of PPT+DPN. *Denotes p<0.02. Symbols represent mean+SEM. N=10.

PowerPoint slide

Full size image

Again, omission rates were very low, yet, there was a trend for PPT to decrease trial omissions (main effect of treatment day: F(2,18)=3.40, p=0.056). However, there was no effect of PPT on rate of lever pressing (p=0.66).

The ERβ Agonist DPN Increases Preference for Larger, More Costly Rewards

Activation of ERβ receptors with DPN increased preference for the HR lever. Analysis of the choice data revealed a significant main effect of treatment day (F(2,18)=5.82, p<0.05), and a significant treatment day × trial block interaction (F(6,54)=4.98, p<0.001). As can be observed in Figure 5b, post-hoc analysis indicated that DPN increased responding on the HR lever compared with the control oil injection 24 h post-injection (p<0.001) when 10 presses were required, but not on the day of DPN injection (p>0.10). There was no significant effect of DPN on number of omissions (p=0.24) or rate of lever pressing (p=0.18).

PPT and DPN Administered Together Decreased the Effort Exerted to Obtain a Larger Food Reward

The effects of treatment with estradiol on effort discounting were opposite to those observed following treatment with receptor selective agonists, in that estradiol decreased preference of the HR lever, whereas treatment with either ERα or β receptor agonists actually increased preference for this option. To probe these opposing findings further, we assessed the effects of treatment with a combination of 10 μg PPT and 10 μg DPN on effort discounting (Figure 5c). Analysis of the choice data revealed a statistical trend for the main effect of treatment day (F(2,14)=3.11, p=0.07), but not a day × trial block interaction (p=0.32). Based in part on our previous findings with estradiol, we had expected that a priori PPT+DPN would affect choice behavior 24 h after injection. We therefore conducted a subsequent exploratory analysis on the main effect. This analysis revealed that treatment with PPT + DPN decreased choice of the HR lever 24 h after treatment, relative to the vehicle treatment day, with this effect approaching statistical significance (p=0.06). However, these treatments did not induce a reliable alteration in choice on the day of injection (p=0.34). Thus, even though selective activation of either ERα or β receptors alone induce an effect on choice opposite to that of estradiol, activation of both receptors simultaneously produced an effect that resembled that induced by estradiol (10 μg EB) treatment. The PPT+DPN cocktail did not alter the number of omissions or rate of lever pressing (all p<0.74).

DISCUSSION

To our knowledge, this is the first study to demonstrate that estradiol exerts modulatory control over cost/benefit decision making in female rats. Following ovariectomy, female rats were more likely to choose the high effort/HR option than rats who received a control sham surgery, suggesting that ovarian hormones reduce the preference to work harder to obtain a larger reward. This appears to be mediated in part by estradiol, as administration of a high level of estradiol in ovariectomized rats resulted in a delayed reduction in choice of the high effort/HR option. Interestingly PPT (ERα agonist) and DPN (ERβ agonist) increased preference to exert more effort for a larger reward when administered separately to ovariectomized females, but when given together, induced a change in decision making that resembled that induced by high levels of estradiol. In all cases, the behavioral effects of estradiol and the combination of PPT with DPN were observed 24 h post-injection, suggesting that genomic mechanisms contributed to the lasting effects of ERs on effort-based decision making. Importantly, these effects cannot be easily attributed to changes in motor performance or more general alterations in motivation, as the rate of lever pressing was not significantly different between any treatment conditions. Thus, these data suggest that high levels of estradiol, acting simultaneously on both ERα and β receptors appear to shift biases related to cost/benefit decision making, making animals more averse to effort costs that may be associated with larger rewards.

Ovariectomized Rats Chose to Exert More Effort for a Larger Reward and this Effect was Reversed by a High Dose of Estradiol

Following ovariectomy, rats exhibited an increased preference to work harder for a larger reward than sham-operated controls. In turn, choice of the HR lever was reduced following treatment with high estradiol, indicating that higher levels of estradiol bias the decision criterion of the rats away from higher cost options. Although the specific mechanisms through which estradiol may act to exert these effects are unclear, they may be related to alterations in dopamine transmission (discussed below). As has been demonstrated with male rats, the task used in the present study (like other forms of effort-based decision making) is exquisitely sensitive to manipulations of the dopamine system or some of its key terminal regions such as the nucleus accumbens, anterior cingulate cortex, or basolateral amygdala (Salamone et al, 1994; Floresco and Ghods-Sharifi, 2007; Salamone et al, 2007; Ostlund et al, 2011). For example, systemic administration of dopamine antagonists or lesions of dopamine terminals in the NAc induce a pronounced decrease in preference for larger rewards associated with a greater effort cost, but do not affect a general preference for larger vs smaller rewards in males (Cousins et al, 1994; Salamone et al, 1994; Denk et al, 2005; Salamone et al, 2007; Salamone et al, 2009). However, it is interesting to note that increases in dopamine transmission with amphetamine can exert biphasic, dose-dependent effects on effort-based decision making in male rates. Higher doses of amphetamine reduce preference for high effort/HR options on the effort-discounting task used here, whereas lower doses have the opposite effect (Floresco et al, 2008b). This suggests that an inverted U relationship exists between dopamine levels and choice for the high effort/HR option in males. Indeed, several types of steroid hormones and neurotransmitters have been reported to modulate behavior and cognition in a similar biphasic manner in both sexes (Holmes et al, 2002; Floresco and Magyar, 2006; Cools and D’Esposito, 2011; Jacobs and D’Esposito, 2011b; Rajagovindan and Ding, 2011).

Estradiol enhances and/or prolongs effects of dopamine through several mechanisms including effects on autoreceptors, enhanced binding on D2 receptors, and increased sensitivity of these receptors in dopamine terminal regions in females (Thompson and Moss, 1994; Becker, 1999; Becker and Hu, 2008). Female rats have greater sensitivity to dopamine-induced alterations in behavior and enhanced levels of drug-evoked dopamine increase (Becker, 1999). Importantly, estradiol-induced alterations in dopamine levels are behaviorally relevant. Estradiol modulates both motivational and reinforcing properties of cocaine (Lynch and Carroll, 1999; Lynch et al, 2000; Carroll et al, 2002; Hu et al, 2004), where naturally high levels of estradiol enhanced reinforcement of cocaine, as well as motivation to self-administer cocaine in female rats (Roberts et al, 1989). Together, this suggests the dopamine system of intact females may be more sensitive than males, and under basal conditions may bias a choice strategy more similar to a male in a hyper-dopaminergic state. It is important to note, however, that the influence of estradiol on PFC-mediated tasks depends on pre-existing basal levels of dopaminergic transmission (Jacobs and D’Esposito, 2011b). With respect to the present study, increased preference for the HR lever induced by ovariectomy may have been caused by slight decreases in mesolimbic dopamine transmission. Conversely, the reduced tendency to select the high effort option following estradiol replacement may be linked to an increase in dopamine release, biasing choice in a manner similar to amphetamine administration (Yu et al, 2009).

Despite the above-mentioned findings, there is other evidence to suggest that ovariectomy may enhance dopamine activity. Following ovariectomy, a decrease in dopamine transporter levels (Bossé et al, 1997) and D2 receptor expression has been observed (Bazzett and Becker, 1994), which would be expected to lead to an increase in dopamine tone. This notion is consistent with recent studies on the effects of ovariectomy and estradiol treatment on latent inhibition in female rats (Arad and Weiner, 2010a, 2010b). In that study, ovariectomy impaired latent inhibition following a weak-conditioning procedure, in a manner similar to that induced by amphetamine, suggesting that this manipulation may have augmented dopamine activity. These authors also demonstrated that under some conditions, exogenous estradiol could exert antipsychotic (ie anti-dopaminergic) properties in ovariectomized rats. As mentioned above, moderate increases in dopamine activity induced by lower doses of amphetamine can increase preference for larger, more costly rewards in male rats (Floresco et al, 2008a), similar to what was observed following ovariectomy in the present study. It is also well established that antipsychotic dopamine antagonists have the opposite effect on effort-based decision making in male rats (Denk et al, 2005; Salamone et al, 2007; Floresco et al, 2008b; Salamone et al, 2009), resembling the effects of estradiol treatment reported here. Therefore, it is equally plausible that the effects of ovariectomy and estradiol treatment on decision making may be the result of an increase and decrease in mesolimbic dopamine activity, respectively. Further research on this topic, assessing the effects of different doses of dopamine agonists and antagonists in intact and ovariectomized rats, is required to resolve this issue. Nevertheless, when viewed collectively, these data indicate that circulating levels of estradiol have a considerable impact on cost/benefit decision making in the female, and that these effects may be mediated in part through alterations in dopaminergic transmission.

ERα or ERβ Agonists Increase Preference for Larger, More Costly Rewards

ERα (PPT) or ERβ (DPN) agonists independently result in greater preference for the high effort/HR lever. Interestingly, estradiol (which activates both ERα and ERβ), as well as the combination of PPT and DPN, had the opposite effect on choice behavior, suggesting that activating each receptor independently affects cost/benefit decision making differently than concurrent activation of both receptors. The two subtypes of ERs have a great deal of overlap in function and location, but also have their own separate and unique effects (Matthews and Gustafsson, 2003). Furthermore, dimerization of ERα and ERβ together can lead to very different effects than activation of each receptor independently (Matthews and Gustafsson, 2003), suggesting the pharmacological downstream effects of these two receptors would likely be different. For example, hippocampal-mediated cognition was not altered in ERαKO mice (Fugger et al, 2000), but was impaired in ERβKO mice (Rissman et al, 2002), suggesting differential roles for each receptor in mediation of cognition. ERα and ERβ may have similar roles in modulating effort-based decision making due to the similar effects on behavior. However, additional doses of individual agonists would need to be examined to further elucidate specific roles of each receptor subtype.

It is important to note that the length of time following ovariectomy can influence the brain's ability to respond to estrogens (Bossé et al, 1997; Simpkins et al, 1997; Mohamed and Abdel-Rahman, 2000). In the present study, all subjects received doses of estradiol or specific receptor agonists in a counterbalanced fashion. Although there were 72 days between OVX and the final day of testing, all animals consistently were given a hormone or agonist injection intermittently throughout the experiment (Figure 2). Thus, potential cumulative effects of the repeated injections over time, as well as differential responsiveness following OVX long term, cannot be ruled out as contributing factors to our findings. However we do not believe that a lack of responsiveness to estrogens due to length of time after ovariectomy or cumulative injections had a significant role in our findings for a number of reasons. First, our animals were behaviorally responsive to both PPT and DPN when administered independently and in combination. In particular, after treatment with both agonists, rats displayed alterations in behavior in a similar pattern comparable to the high dose of EB (10 μg) earlier in the injection timeline. At the time of PPT and DPN injections 71 days had elapsed since ovariectomy, yet females were behaving in a similar manner when tested with high EB 17 or 26 days post-OVX. Secondly, we found that response to an acute administration of low EB (0.3 μg) remained stable over time among the OVX rats in the present experiment. Specifically, the 0.3 μg EB was administered twice throughout the injection regimen, and response did not differ between the first and second exposure (p=0.86), indicating that the time since OVX did not alter the response to EB on effort-based decision making (10 and 19 days between the two 0.3 μg EB test days). Past studies have found the effects of long-term OVX to be specific to both the region and neurochemical of interest (Singh et al, 1994; Bossé et al, 1997; Mohamed and Abdel-Rahman, 2000; Barker and Galea, 2009). Thus, the anterior cingulate, BLA, and NAc may have been differentially affected by OVX and estradiol treatment. It is also possible that the intermittent injections of EB and ER agonists maintained priming of the brain to subsequent estradiol administration so that subjects remained behaviorally responsive to estrogens despite the extended time since OVX. In this regard, the average amount of time between hormone exposure was 10.6 days (range, 8–15 days), which may have been sufficient to keep the brain responsive to estradiol over the post-OVX testing period.

The present results suggest a role for both subtypes of ER in alterations in effort-based decision making following ovariectomy; however, it remains unknown if ERα and ERβ agonists exert their effects through similar mechanisms despite their comparable effects on cost/benefit decision making. Thus, the effects of stimulating ERα and ERβ independently appear to produce near identical behavioral changes, whereas simultaneous stimulation of both ERs produce opposing effects on effort-based choice. Differential distribution of each subtype of receptor within neurocircuitry implicated in effort-based decision making may explain, in part, differential effects of each agonist examined in the present study. For example, mRNA expression for both ERα and ERβ was found in the female hippocampus, whereas more expression of ERα mRNA is detectable in the basolateral amygdala, and only ERβ mRNA in the nucleus accumbens and prefrontal cortex (Shughrue et al, 1997; Shima et al, 2003). Further research using local administration of receptor-specific estradiol agonists within different brain regions may help to clarify the mechanisms through which these individual receptors may exert their effects on decision making.

Other Potential Mechanisms Through Which Estradiol May Mediate Decision Making

In the current study we examined both intact and ovariectomized females, which would abolish fluctuating levels of the ovarian hormones estradiol and progesterone. Although we did not observe variations in effort-related choice across the estrous cycle in the present study, this may have been due to a couple of reasons. First we tested all animals between 1000–1400 hours, thus potentially missing the afternoon peak in estradiol that occurs during proestrus. Given the fact that we found alterations in effort-based decision making after a high dose of estradiol, it is possible we may have observed changes in decision making that were associated with estrous phase if testing corresponded with the natural peak in estradiol levels. Second, during the estrous cycle there are many other ovarian hormones at play, notably progesterone and progesterone's interactions with estradiol may also affect this reward-related cognition. Interestingly, Russo et al (2003) found that estradiol with progesterone potentiated conditioned place preference in ovariectomized rats, and these changes coincided with increased dopamine levels within the nucleus accumbens. In pre-menopausal humans, females in the late follicular phase (estradiol levels are higher compared with levels of progesterone) report greater liking of drugs of abuse and more euphoria (White et al, 2002; Becker and Hu, 2008). Thus, although the emphasis of this study was to investigate the modulatory control that estradiol exerts on cost/benefit decision making mediated by dopaminergic transmission, it is possible that the effects of ovariectomy reported here may also have been caused by reduced levels of progesterone. However, the interplay between the ovarian hormones estradiol and progesterone and their impact on dopamine is not fully understood and requires further investigation.

Glutamatergic transmission also has a central role in modulating effort-based decision making (Floresco et al, 2008b), and estradiol influences glutamate transmission within certain nodes of mesocorticolimbic circuitry that regulates these processes. Specifically, within the basolateral amygdala estradiol decreases excitatory postsynaptic potential amplitude (Womble et al, 2002). Within the nucleus accumbens core, ovariectomy resulted in decreased glutamate levels in morphine tolerant female rats, but did not affect glutamate levels in the male, thus, an estrogen-sensitive mechanism may modulate excitatory amino acid release in the nucleus accumbens in the female rodent (Mousavia et al, 2007). Changes in the excitability of neurons within the amygdala or accumbens induced by fluctuations in estradiol levels may alter the manner in which these regions process information about costs and benefits, leading to differential biases towards or away from larger, more costly rewards. Thus, it is important to acknowledge potential effects of ERα and ERβ stimulation on glutamate, in addition to the alterations on dopamine in the present study. Future studies should address other key neurotransmitters, such as glutamate, implicated in effort-based decision making, as well as the role of additional hormones, such as progesterone.

Conclusions

Ovariectomy dramatically increased, whereas high estradiol decreased, the preference to exert more effort to obtain a larger food reward. Thus, the results of the present study indicate that estradiol exerts a pronounced impact on effort-related judgments, biasing choice behavior towards smaller, yet more easily accessible rewards. Estradiol works through both ERα and ERβ, albeit in opposing directions from both combined, to alter effort-based decision making in female rats. More research is needed to further elucidate the role of estradiol on cost/benefit decision making, as the presence of other gonadal hormones, as well as complex interactions with key neurotransmitters are likely mediating the observed behavioral effects. Increased knowledge of the role of estradiol in females relating to cost/benefit decision making can inform numerous cognitive domains, especially psychopathologies that are coupled with problems in cognition (ie schizophrenia, substance-use disorders) as well as natural cognitive decline associated with aging and certain hormone replacement therapies.