Approach-avoidance conflict is induced when an organism encounters a stimulus that carries both positive and negative attributes. Accumulating evidence implicates the ventral hippocampus (VH) in the detection and resolution of approach-avoidance conflict, largely on the basis of maze-based tasks assaying innate and conditioned responses to situations of conflict. However, its role in discrete trial approach-avoidance decision-making has yet to be elucidated. In this study, we designed a novel cued operant conflict decision-making task in which rats were required to choose and respond for a low reward option or high reward option paired with varying shock intensities on a differential reinforcement of low rates of responding schedule. Post training, the VH was chemogenetically inhibited while animals performed the task with the usual outcomes delivered, and with the presentation of cues associated with the reward vs. conflict options only (extinction condition). We found that VH inhibition led to an avoidance of the conflict option and longer latency to choose this option when decision-making was being made on the basis of cues alone with no outcomes. Consistent with these findings, VH-inhibited animals spent more time in the central component of the elevated plus maze (EPM), indicating a potential deficit in decision-making under innate forms of approach-avoidance conflict. Taken together, these findings implicate the VH in cue-driven approach-avoidance decisions in the face of motivational conflict.
Many decisions such as choosing a restaurant to dine in, or foraging for food in the wild require the organism to accumulate and integrate information across multiple domains that include the reward value (e.g., food), effort (e.g., distance), and potential punishment (gastric malaise, predation). Much progress has been made in the quantification of, and delineation of the neural substrates governing cost-benefit decision-making that manipulates the magnitude, probability, effort, and timing of obtaining reward [1, 2]. However, less research has been directed at quantifying decisions that are made under an approach-avoidance conflict as a result of exposure to competing/mixed outcomes, despite historical interest in the subject [3,4,5,6,7]. Furthermore, the underlying neural circuitry of approach-avoidance decision-making remains to be elucidated, an important undertaking given that approach-avoidance decisions are likely dysregulated in psychiatric diseases [8,9,10].
The ventral hippocampus (VH) has emerged more recently as a key area involved in emotional regulation, distinct from the role of its dorsal counterpart in spatial learning and memory . The VH has been implicated in the control of anxiety [11,12,13], extinction of fear conditioning , extinction of active avoidance , in addition to its role in regulating maze-based cued approach-avoidance decisions [16,17,18], and foraging-related and operant-based approach-avoidance decisions in humans [19, 20]. However, its role in operant decision-making in preclinical studies, in which animals have to make discrete choices under approach-avoidance conflict has not been well studied, with some studies reporting an absence of effect with anterior hippocampus inactivation (in marmosets) . One important consideration is that the majority of studies revealing significant contributions of VH in emotional regulation have utilised tests that are conducted in extinction-like scenarios, involving the presentation of conditioned or innate stimuli in the absence of outcomes, or exposure to situations of potential, but no actual threat. It is therefore unclear if the VH is critically involved in decision-making tasks that involve reward and punishment outcomes, alongside the orbitofrontal cortex (OFC), insula, and subgenual ACC [22, 23], or whether the VH is selectively involved in cue-driven decision-making.
We therefore developed a novel operant cued conflict-based decision-making task that enabled the investigation of the role of the VH in decision-making that involves choosing between two cued options under conditions in which the outcomes are delivered, and not delivered (in extinction). Given the evidence discussed above, we predicted that chemogenetic inhibition of excitatory neurons of VH would selectively disrupt cued approach-avoidance choice decisions made under extinction conditions. Animals were first trained in multiple blocks of increasing approach-avoidance conflict, to choose between a high reward option with varying shock intensities (conflict) on one lever, and a low reward option (reward only) on another lever. Varying the shock intensity between blocks allowed us to observe the point of subjective equality (PSE), the conflict level at which animals exhibited equal preference between the conflict and low reward options, and discrimination sensitivity (DS), the animals’ ability to differentiate responding between blocks. In addition, unlike most other decision-making tasks, our task was based on a differential reinforcement of low rates of responding (DRL) schedule of reinforcement, in which animals were required to wait for at least 6 s before responding to obtain the outcome. This ensured that animals were given sufficient time to appraise the cues associated with the conflict and reward options, and to minimize the incidence of impulsive choices, since HC lesions have previously been shown to induce impulsive choice in a delayed discounting task . Furthermore, the DRL schedule enabled us to calculate deviations from optimality .
Materials and methods
Subjects were 22 male Long Evans rats (Charles River, QC, Canada), weighing ~400–600 g at the time of surgery. They were housed in groups of two in a 21 °C room, under a 12 h light/dark cycle (lights on at 7:00 A.M.). Water was available ad libitum throughout the experiment. Before commencing behavioral testing, food was restricted to maintain animals’ weights between 90% of their free-feeding weight to promote motivation to engage in behavioral tasks. All behavioral testing took place in accordance with the ethical and legal requirements under Ontario’s Animals for Research Act, the Canadian Council of Animal Care, and approval of the Local Animal Care Committee.
Rats were divided into two groups (n = 11 hM4Di-injected group; n = 11 GFP-injected controls) and underwent bilateral adeno-associated virus (AAV) injection surgeries. Rats were anaesthetized with isoflurane gaseous anesthetic (5% for induction, 2–3% for maintenance, Benson Medical, ON, Canada) and placed in a stereotaxic frame (Stoelting Co, IL). Briefly, an incision was made at the skull midline and small holes were created via a dental drill to target the VH bilaterally (AP −5.8 mm; ML ± 5.4 mm; DV −6.5 mm). A 10 μl Hamilton syringe was then lowered into the VH for bilateral infusion of 0.5 μl of AAV containing Designer Receptors Exclusively Activated by Designer Drugs hM4Di (AAV8-CAMKII-hM4Di-mCherry, Addgene, MA) or a control virus (AAV8-CAMKII-GFP, Addgene, MA). The AAV delivery occurred over a period of 5 min with controlled electronic microinjector (Stoelting Co, IL). Once injection was completed, the syringe remained at the delivery site for an additional 5 min to allow AAV particles to diffuse away from the injector tip before the scalp was stitched back together. After surgery, the animals were allowed a recovery period of a minimum of 7 days with postsurgical care and food available ad libitum before behavioral testing.
All rats received an intraperitoneal (IP) injection of either 0.9% saline and clozapine-N-oxide dihydrochloride (CNO, R&D Systems, MN, dissolved in sterile saline at a volume of 1 ml/kg) 45 min prior to outcome and extinction test sessions. For the elevated plus maze (EPM) sessions, all rats received an IP injection of CNO (1 mg/kg) 45 min prior to the test session.
Testing took place in eight operant chambers (30.5 cm L × 24.1 cm W × 29.2 cm H, Med Associates, VT) contained within a sound-attenuated box with a ventilation fan. Each chamber had two 4 cm wide retractable levers positioned on the sidewall, located 2 cm either side of a central food well (magazine) and 5 cm above the floor composed of parallel shock bars. A light panel was positioned 3 cm above each lever. The shock bars were connected to shock generators, which produced mild foot shocks (0.5 s, 0.05–0.65 mA). A pellet dispenser connected to the central food well delivered 45 mg chocolate sucrose pellets (TestDiet, MO) as reward stimuli, and an infrared detector monitored every nose poke entry into the food well. The chamber was illuminated by a 1.8 W, 17 V house light located at the top right corner of the chamber.
All rats received two magazine training sessions in which they were placed in the operant box for 1 h and received sucrose pellets on a variable interval 20 s schedule of reinforcement. Rats were then trained to lever press for sucrose pellets under a Fixed Ratio 1 schedule. To avoid side bias, the lever to be inserted (left vs. right) was randomly chosen until 50 rewards were obtained from each side. A session ended after 30 min, or after animals pressed both levers 50 times. Rats were moved to the next phase of training after two consecutive sessions with at least 40 presses on each side.
Differential reinforcement of low rates of responding (DRL) schedule
All rats were trained to respond for a reward pellet under a delayed DRL schedule of reinforcement following lever press training (Fig. 1A). Each trial started with the insertion of the left or right lever and rats were required to wait a fixed minimum time (6 s) before responding to obtain a single sucrose pellet. If the subject responded prematurely (incorrect response), then the lever was retracted without reward and the trial ended 0.5 s after retraction. When a correct response was executed, the lever retracted immediately, and a reward pellet was delivered to the magazine after a brief (0.5 s) delay. The 0.5 s delay was implemented in order to accommodate the introduction of shocks (0.5 s duration) in the next phase of training, such that the shock delivery co-terminated with the reward delivery. After 30 sessions, training was halted for the rats to undergo surgical infusions of the hM4Di/GFP construct.
DRL two alternative forced choice training
Following the DRL training, all rats underwent the two alternative forced choice training in which responding on one of two extended levers (side of lever counterbalanced) delivered one sucrose pellet under the same DRL schedule (reward only option), while responding on the other lever led to a mixed outcome (two pellets and shock − conflict option). Training took place in a number of steps, with the initial step involving the administration of three blocks of a minimum of 30 trials per session, with each block representing increasing shock intensities associated with the conflict option (0.25, 0.30, 0.35 mA, Fig. 1B). Each block began with 10 forced choice trials in which one of the two levers was presented in random order (5 trials each), and animals were required to respond under DRL6s to receive a single reward pellet or a mixed outcome (two reward pellets and shock). Responding on the lever led to the retraction of lever for 0.5 s, and the delivery of associated outcomes, followed by a re-extension of the lever to start the next trial. If a response was not emitted within 12 s, the lever was retracted, and the intended outcome for that trial was delivered. Importantly, the extension of the levers was accompanied by the presentation of a light stimulus positioned over each lever, signalling the outcome associated with the lever. A constant light was paired with the reward only option and a flickering light was paired with the mixed outcome (conflict) option. The light associated with the conflict option flickered at 1.33 HZ at the lowest shock intensity (0.25 mA), with a decrease in the rate of flickering with increasing shock intensity/block (0.30 mA:1 Hz, 0.35 mA: 0.8 Hz). Following the forced choice trials, animals were administered free choice trials in which two levers were presented, and animals were required to respond on one of the two levers under DRL 6 s to receive the associated outcome, with the 12 s response deadline removed. In this phase, a minimum of 20 correct responses was required (waiting >6 s to respond on either lever) to advance to the next block. Animals underwent further training steps (Fig. 1B, C, Supplementary Methods) until stable choice performance was established in training sessions consisting of seven blocks of a minimum of 30 trials (10 forced, >20 free choice) marked by increasing shock intensity (0.05–0.65 mA, in increments of 0.1 mA), and a light cue of decreasing frequencies (4,2,1.33, 1, 0.8, 0.66, 0.57 Hz) accompanying the conflict option.
DRL choice outcome test
For the DRL choice outcome test sessions (Fig. 1D), half of the rats underwent three consecutive sessions of DRL choice test under the effect of 1 mg/kg CNO, then three consecutive sessions without injections (for drug washout), followed by three consecutive sessions with saline injections, while the other half received the drug and saline injections in reverse order. The test sessions were identical in structure to the training sessions, with the exception that the first 10 forced choice trials were not administered. Therefore, in each session, animals were required to complete a minimum of 20 free choice trials per block (seven blocks, total of 140 trials minimum).
DRL choice extinction test
Following the DRL choice outcome tests, animals were retrained on a shorter version of the task in preparation for the extinction test (Fig. 1E). The free choice phase was now reduced to achieving ten correct responses before proceeding to the next block (i.e., a minimum of 70 trials/session). After ten sessions, animals were tested in the DRL choice task for one session under extinction conditions (Fig. 1F), in which the session proceeded as during training (same temporal structure, and same minimum number of correct trials), with the exception that no outcomes were delivered. Half of the animals received saline while the other half received 1 mg/kg CNO prior to the extinction test session. Animals were then retrained in the shorter DRL choice task for three further sessions, and received one further test in extinction with the drug/saline injections reversed.
Elevated plus maze (EPM)
All rats underwent a standard ethological test that measures innate expressions of approach-avoidance conflict in EPM, which consists of two open arms (40 cm L × 10 cm W × 2 cm H) and two closed arms (40 cm L × 10cm W × 30 cm H) with a central platform (10 cm L × 10 cm W). Each rat was placed individually in the maze for 5 min, beginning at the central hub of the maze facing an open arm. The arm entries and time spent in each arm were measured and compared. This task was performed prior to the delayed DRL choice test with CNO injections.
Following the final extinction test under the influence of either saline or CNO (1 mg/kg), all hM4Di rats were anaesthetized with a lethal dose of pentobarbital and perfused intracardially with 0.9% saline and then a 4% paraformaldehyde solution. The brains were then extracted and kept in 4% paraformaldehyde and 20% sucrose solution for 24 h, and were cut into 50-μm thick coronal sections using a vibratome (Leica VT1200S). All sections then underwent immune staining for cfos, according to established protocols in our laboratory (see Supplementary Methods), and mounted on gelatin-coated slides and air dried before being cover slipped with Fluoroshield Mounting medium with DAPI for nuclear staining (Abcam, UK).
Cell imaging and counting
hM4Di and GFP expression, and c-Fos immunoreactivity were visualized at ×4, ×10, ×20 and ×40 magnification using the NIKON Ni-U upright florescent microscope (NIKON, NY). GFP expressing cells and c-Fos positive (+) cells conjugated with TSA-fluorescein were visualized using the FITC filter, while hM4Di-expressing cells were visualized using the TexRed filter. Qualitative analysis of the extent of the hM4Di expression was achieved by superimposing ×4 images from each animal and generating an intensity mapping of the areas most consistently expressing hM4Di. Quantification of c-fos+ cells, and cells double labeled with cfos and hM4Di/mCherry was achieved using two images of coronal sections of the VH taken at ×4 and ×20 magnification from each animal. VH subregions were demarcated into six regions of interest [vCA1 (vCA1v, vCA1d), vCA2, vCA3 (vCA3v, vCA3d), vDG), and the number of cfos+, and cfos + mCherry-positive cells within those boundaries were counted using ImageJ  and Fiji  software.
Data were analysed using statistical software R  and lme4 . DRL responses from every three sessions of acquisition training and choice tests (except the extinction test) were pooled to obtain more representative probability distributions. These distributions were fitted with exponential inverse-gaussian using a custom MATLAB script that uses maximum likelihood estimation  and fit parameters were used to calculate a number of DRL parameters including the wait times/inter-response times (IRT), timing uncertainty (coefficient of variation, CV) and optimality related measures (Expected Reward Rate (ERR), Percentage of Maximum Expected Reward Rate (PMERR)). For acquisition data, DRL parameters for the first and last three sessions were compared to assess learning. For DRL choice test data, DRL parameters for each lever option were calculated separately (conflict and reward only) for each block. However, since the number of reward only responses at lower shock intensities and number of conflict responses at higher shock intensities were few in number, we analysed the % correct responses (responses with >6 s wait time) and DRL responses from the three highest intensity blocks (0.45–0.65 mA) for the reward only option and the three lowest intensity blocks (0.05–0.25 mA) for the conflict option. Additionally, preference for the reward only option was calculated by dividing the number of reward only responses by the total number of responses in each block. PSE and DS were calculated by fitting psychometric curves using a custom Matlab script  to the preference for the reward only option for each subject. Linear mixed modelling was then employed to analyse all parameters for both conflict and reward only choices in the outcome and extinction test sessions, using the intensity of the shock (×7), experiment (outcome, extinction sessions), drug (saline, CNO), and virus (hM4Di-mCherry or GFP) as fixed effects, and intercepts for subjects as random effects. Post-hoc analysis was conducted by using emmeans package with Tukey familywise correction. Degrees of freedom were calculated using the Satterthwaite method.
Finally, random data sampling was carried out to establish that significant effects found between outcome and extinction conditions were not driven by differences in trial numbers between the two conditions (Supplemental Information, Table S1).
CNO-induced reduction in Cfos+ cells in the vCA3v and vCA1d
All animals showed robust expression of hM4Di (mCherry+ cells) or GFP within the confines of VH (Fig. 2A–D), with consistent transfection of the vCA3, vCA2, and vCA1. Analysis of the number of cfos positive (+) cells in the VH of saline-injected hM4Di-expressing animals revealed the greatest and least levels of cfos activation to be in the vCA1 and vCA3d, respectively. To characterize the inhibition pattern more fully (Fig. 2E), analysis of the number of cfos+ cells following the final DRL choice test revealed that the group injected with CNO exhibited reduced amount of cfos+ cells in the vCA1d (t(5) = 3.12, p = 0.026) and vCA3v (t(5) = 3.309, p = 0.021,) but not in vCA1v (t(5)=0.82, p = 0.45), vCA3d (t(5)=0.97, p = 0.37), vCA2 (t(5)=1.44, p = 0.20), or vDG (t(5) = 1.29, p = 0.129) compared to those injected with saline. Analysis of the number of double labeled (cfos+/hM4Di (mCherry)+ cells) also confirmed the same pattern of results (Fig. 2F), with a significant reduction in the number of double labeled cells in the vCA1d (t(5) = 3.19, p = 0.021) and vCA3v (t(5) = 3.05, p = 0.028) areas, but no other areas (all p > 0.14). No significant differences were observed in the number of non-hM4Di cfos+ cells between the CNO- and saline-injected animals (all p > 0.08, CNO: 4.19 ± 1.0, SAL: 3.75 ± 0.97 across all subareas).
Acquisition of DRL schedule of responding for reward
Comparisons of performance measures in the first and last three sessions of responding under a 6 s DRL schedule revealed that animals improved their performance as training progressed (Fig. 3), with a decrease in average wait time (F(1, 21.244) = 54.350, p < 0.0001, Fig. 3B) and timing uncertainty (CV, F(1, 17.349) = 263.812, p < 0.0001, Fig. 3C), as well as increases in expected reward rate (ERR, F(1, 19.988) = 515.867, p < 0.0001, Fig. 3D) and proximity to optimal duration (PMERR, F(1, 42) = 15.611, p < 0.001, Fig. 3E).
Acquisition of DRL choice task
In this phase of training rats were trained to choose between lever pressing on one lever for the reward only option (one pellet) and on another for a mixed outcome conflict option (two pellets and shock). The shock intensity in the conflict option was increased across seven blocks (0.05 mA −> 65 mA) to examine the change in preference for the reward only option. After extensive training, animals exhibited discriminative responding between blocks (Fig. 4A), as evidenced by a significant reduction in the preference of reward-only option at lower shock intensities (0.05 mA (p < 0.0001), 0.15 mA (p < 0.0001), 0.25 mA (p < 0.0001), 0.35 mA (p < 0.0001), and 0.45 mA (p < 0.001) when performance averages of the first three and last three sessions of training were compared (F shock × session (6, 286) = 5.17, p < 0.0001). Crucially, the DRL choice acquisition pattern of animals belonging to the hM4Di and GFP-control groups was not significantly different (all virus group interactions p > 0.29).
VH inhibition increases the preference of reward only option, but only under extinction conditions
We then assessed the effect of chemogenetically inhibiting the VH on DRL choice performance with outcomes, and under extinction in which only the cues associated with the outcomes were presented. All animals received injections of saline or CNO (1 mg/kg) in a within-subject design, prior to each of six sessions of 7-block DRL choice testing with outcomes, and each of two sessions under extinction conditions (Fig. 1 Testing). Chemogenetic VH inhibition significantly increased the preference of reward-only option across the seven blocks for the extinction test only, when compared to their performance after saline injections (Fig. 4B,F drug × virus × experiment (1, 553.13) = 5.24, p < 0.03, post-hoc: p < 0.01). In contrast, VH inhibition did not alter choice performance when animals experienced the delivery of the outcomes (p = 0.47). Similarly, CNO injections did not induce significant changes in DRL choice performance in the control-GFP group in either of the experimental conditions (both p > 0.32, Fig. 4C). When we compared choice decisions made under extinction and with outcomes separately for the hM4Di and GFP groups, we found saline-injected hM4Di animals decreased their preference for the reward only option in extinction, compared to outcome conditions (F drug × experiment (1,270) = 5.33, p < 0.03, post-hoc: p < 0.05), but this change was absent in CNO-injected animals (p = 0.19). In the GFP group, however, animals decreased their preference for the reward only option specifically at the two highest shock intensities under extinction conditions (F experiment (1,287.45)=6.27, p < 0.02, F experiment x shock (6,283.13) = 3.97, p < 0.001, post-hoc; 0.55 mA, p < 0.001, 0.65 mA, p < 0.01), irrespective of drug injection (all drug effects p > 0.22). Thus, VH inhibition led to a significant change in choice preference under extinction conditions.
Chemogenetic VH inhibition in the hM4Di group also led to a decrease (leftward shift) in PSE during extinction (Fig. 4D, F drug × virus × experiment (1, 61.32) = 4.13, p < 0.05, post-hoc: p < 0.005), but not during the outcome test (p = 0.56) when compared with performance after saline injections. Furthermore, when the PSE from testing under outcome and extinction conditions was compared, saline-injected hM4Di animals had lower PSE values under extinction (p < 0.02). In contrast, no change in PSE was observed in either drug condition (CNO or saline), nor experimental condition (extinction vs. outcome) in the control-GFP group (all p > 0.28, Fig 4E). The DS was not altered under any of the conditions in the hM4Di and GFP groups (all p > 0.15). These results indicate that VH inhibition altered the parity between reward- and shock-associated cues (increased avoidance of the conflict cues) but not the rats’ ability to discriminate between cues signaling shock intensities under extinction conditions.
VH inhibition did not impact other performance measures such as the overall percentage of completed trials over the minimum number or trials required to progress to the next block (Fig. 4D, E, no interactions between drug, virus and shock, all p > 0.11), percentage of correct trials for the reward only option (no significant interactions between drug, virus and shock, all p > 0.82) during outcomes-present or extinction conditions. However, VH inhibition increased the percentage of correct responses for the conflict option, but only in the extinction test (Fig. 4B, F drug × virus × experiment (1, 231) = 8.884, p < 0.005, post-hoc: p < 0.001).
VH inhibition increases the average wait time for conflict option, but only in extinction
To further elucidate the response patterns for the conflict and reward only options, the IRT and other DRL parameters were analyzed separately. Analysis of the conflict option response data revealed that CNO-induced VH inhibition in the hM4Di group increased the average IRTs during the extinction test, as compared to when the same animals were injected with saline (Fig. 5A, C, F drug × virus × experiment (1, 231) = 10.95, p < 0.003, post-hoc: p < 0.0001), and when compared to the IRTs during the outcome test (p = 0.0002). VH inhibition did not affect average IRTs in the outcome test (p = 0.84), when compared to saline injection conditions. CNO injections in the control-GFP group failed to induce any changes in the average IRTs during tests with outcomes (p = 0.35) or in extinction (p = 0.41). Furthermore, control-GFP group IRTs in the extinction test did not differ from those in the outcome test (p = 0.29). Unlike responding for the conflict option, there was no effect of VH inhibition or CNO injections on average IRTs (p > 0.09 for all analyses) in either the hM4Di or GFP group for the reward only option (Fig. 5B, D). Thus, animals waited longer to respond to the conflict cue but not reward cue when the glutamatergic cells in their VH were inhibited.
Generally, across both hM4Di and GFP groups, the CV (F(1, 231) = 8.2160, p < 0.01) and PMERR (F(1, 231) = 4.129, p < 0.05) for the conflict option were higher during the extinction test compared to the outcome test. Similarly, the CV (F(1, 230.877) = 19.087, p < 0.001) was increased for the reward only option, while PMERR (F(1, 218.73) = 5.617, p < 0.03) and ERR were decreased (F(1, 218.892) = 25.0612, p < 0.0001). Notably, there was no effect of group or treatment, or interactions between these for CV, ERR, or PMERR (p > 0.082).
VH inhibition increases the time spent in the central compartment of EPM
We also assessed the effect of VH inhibition on EPM, an ethological test of anxiety. Independent sample t-test comparisons between the virus groups under the effect of CNO injection revealed that the time spent in open and closed arms for the hM4Di group was not significantly different to the control-GFP group; (open: t(20) = 0.001, p = 0.99, closed: t(20) = 0.26, p = 1.15, Fig. 6B). On the other hand, time spent in the central compartment was significantly higher for the hM4Di group compared to the control-GFP group (t(20) = 3.24, p = 0.004). Analyses of the number of entries into the open and closed arms revealed no significant differences between the hM4Di and GFP groups (Fig. 6C open: t(20) = 0.46, p = 0.65, closed: t(20) = 0.48, p = 0.63).
In the present study, we trained rats in a novel cued operant approach-avoidance conflict choice task in which rats chose between a high reward option with varying intensities of shock and a low reward option that was never paired with shock. Following steady state performance, the excitatory projections from the VH were chemogenetically inhibited, and its effect on choice performance with the outcomes present, and in the presence of cues only, was examined. VH inhibition led to an overall increase in preference for the low reward option across different shock intensities, but only during cued decision-making when outcomes were not presented (i.e. extinction). VH inhibition also selectively increased the average duration animals waited before pressing the conflict option, but not the reward-only option, indicating that the VH-mediated effect was specific to the presence of an approach-avoidance conflict. The lack of an effect during testing with outcomes suggests that, unlike manipulations of basolateral amygdala [32, 33], OFC , or rostromedial tegmental nucleus , the VH is not engaged in the online calculation of outcome values in guiding decision-making; rather, it is critical during cued, but non-reinforced decision-making under motivational conflict.
One of the major goals of this study was to investigate approach-avoidance conflict in a goal-directed two-forced choice task in which parity between reward and shock can be calculated by presenting animals with two options: the “conflict option” associated with the delivery of high reward with varying levels of shock to vary the degree of approach-avoidance conflict, and “reward only option” associated with low reward. Many established rat decision-making tasks such as the rat gambling task , Iowa gambling task , delay discounting task , probabilistic discounting task  (for review ), risky decision task  and two-lever operant conflict task  also allow animals to make a choice between multiple options associated with differing reward and punishment (timeout, delay, effort, shock) contingencies. However, most are designed to capture risk-taking behavior that hinges on the presence of uncertainty, rather than an approach-avoidance conflict, with the use of probabilistic delivery of aversive outcomes (e.g., shock). We opted to manipulate shock intensity rather than probability of shock to ensure that the conflict option produces choice conflict in every trial. Additionally, outcomes were delivered following a forced wait time (DRL schedule) in order to minimize the occurrence of impulsive and/or perseverative responding, and to ensure sufficient exposure to the cues signaling each option. Controlling for impulsivity was particularly important in the present study, as an increase in impulsive responding has been previously documented following HC lesions [24, 42], and would potentially have generated chance level responding in choice decision-making across all levels of shock due to attenuated deliberation of the cued options, and DS in VH-inhibited animals. In addition, the DRL schedule has a well-defined optimal wait time that is independent of the value of the outcome [30, 43]. Maximizing the reward rate under this schedule depends on the trade-off between reward probability based on timing uncertainty and the average wait time. This allowed us to quantify the optimal wait time for each option based on timing uncertainty.
Our data implicate the VH selectively in guiding approach-avoidance choice decisions based on cue information alone (in the absence of outcomes), which is in accord with accumulating evidence that the VH may be critical in the processing of cues that signal potential threat, or conflicting outcomes, rather than actual threat/outcomes. Existing ethological tests of anxiety, (e.g., EPM, open field) and cue-based AA decision-making tasks that the VH is strongly linked to [16,17,18, 20, 42, 44] are all administered in the presence of innate or learned environmental cues (open and closed arms, cues in the maze/screen) that predict or threaten the delivery of future appetitive or aversive outcomes to guide decision-making, without the actual subjective experience of the outcomes. Together with more established evidence of the role of the VH in the contextual control of appetitive responses [45,46,47,48], and extinction of cued fear or active avoidance responses [14, 15], these findings lend substantial support to the view that the VH is preferentially engaged in mediating cued control over appetitively and aversively motivated behaviors and decisions, but not in goal-directed behavior driven by outcome value. This view is further corroborated by a recent marmoset study in which it was shown that GABAR-mediated inactivation of the anterior hippocampus failed to induce any changes in responding for the preferred but punished (conflict) option in a two-choice decision-making task in which responding was guided predominantly by a probabilistic delivery of aversive outcomes during cued reward-seeking, . In humans, approach-avoidance tasks involving virtual foraging (foraging for reward tokens in the presence of a sleeping predator)  and cue-based operant-like responding (making approach/avoid button presses to visual stimuli associated with reward or punishment)  have been associated with anterior HPC involvement. A common feature of these tasks is that participants do not know the ultimate outcome on each trial while foraging (whether the predator will wake up) or deciding which button to press (whether approaching will lead to reward/punishment). There are perhaps some similarities between this task feature and an extinction condition in which a rat maintains responding in the absence of an outcome.
In seeking to further understand the contrasting effects of VH inhibition on reinforced and non-reinforced (extinction) decision-making, the ‘task state’ theory [49, 50] is worth considering. According to this account, the OFC plays a role in determining the ‘current state’ or location in a cognitive map of the task, which encapsulates the task-specific structure, associations between cues, outcomes and actions, and importantly, in distinguishing states that are perceptually imperceptible, yet different, as in a reinforced vs. extinction session. It has been suggested that the VH works together with the OFC in the representation of this map  and that VH output is required for the OFC to encode state representation . In the present study, VH inhibition caused animals to generate similar choice preference in the extinction session to that in the reinforced (outcome) session, while saline injections increased the preference of conflict choice during the extinction session, potentially reflecting a failure of the VH-inhibited animals to discern the subtle change in task state with the extinction session, and an inability of these rats to update the change in outcome contingencies, which is strongly reminiscent of the previously reported effects of whole HPC or OFC lesions on attenuating extinction [47, 53, 54]. However, while compelling, this account cannot fully explain the observed increase in the animal’s wait times selectively for the conflict option in the extinction session, but not the outcome test sessions. Further research is warranted to further probe the exact role of the VH in task state representation.
The specificity of the increase in the wait times of the VH-inhibited animals for the conflict option in the extinction session indicates that the observed effect is related to decision-making in the presence of a cued approach-avoidance conflict, rather than a corollary of a motor deficit, timing deficit, or general behavioral inhibition. The time spent before responding can be conceived as a measure of confidence in the choice  and its increase highlights a possible disruption in decision-making . Similarly, we observed an increase in the time spent in the central component of the EPM with VH inhibition, which, unlike the times spent in the open or closed arms, has been previously correlated to measures of decision-making processes rather than anxiety . Together, these findings implicate the VH, and its excitatory glutamatergic neurons, in the impairment of decision-making under cued approach-avoidance conflict in the absence of outcomes.
The observed avoidance of the cued conflict option and leftward shift of the PSE in the absence of any reinforcement indicates that in VH-inhibited animals, the perceived parity between the reward- and shock-associated cues had become dysregulated. We believe these data to further indicate that the perceived negative valence of the conflict cue may have increased under extinction, implicating the VH in facilitating cued approach decisions in the face of motivational conflict under normal circumstances, consistent with the role that we had previously attributed to the vCA1, and not vCA3/DG subregions of the VH in a maze-based cued approach-avoidance conflict task that was also administered under extinction conditions [16, 57]. An alternative account of an increase in the perceived positive valence of the low reward option is incompatible with the current results as the conflict cue signals a higher value of reward. Of note is the observation that under extinction conditions, the control groups exhibited the opposite pattern of preferring the conflict option across all shock intensities in the hM4Di saline condition. This shift in preference for the conflict option was accompanied by an increase in PSE only in the hM4Di saline control condition, which may be reflective of the value of the shock-associated cue extinguishing faster than the reward-associated cue. The absence of a change in PSE in the GFP-control groups is most likely due to the change in preference for the conflict option occurring only at the highest shock levels, as a change in PSE represents altered preference across all shock intensities.
Finally, while we observed a VH inhibition-induced alteration in a decision-making parameter (central compartment time) in the EPM, we failed to observe the widely reported anxiolytic effect of VH lesions/inactivation [11, 58,59,60,61,62] in the present study. It is possible that the more targeted inhibition of glutamatergic cells residing in a small band of vCA3 and vCA1 (vCA1d) in the present study, as opposed to larger, non-cell type specific inactivation of VH cells employed in previous studies, led to a more focal effect of disrupting decision-making parameters of approach-avoidance conflict processing. Existing research remains inconclusive on the effect of subfield-specific manipulations on ethological tests of anxiety, particularly with reports of varying effectiveness of vDG and vCA1 disruption on anxiety [16, 57, 63, 64]. The existence of functional/anatomical/genetic heterogeneity in the vCA1 itself is well documented, with subpopulations of neurons that are differentially responsive to anxiety, spatial, and goal-directed tasks (e.g., [65, 66]), thus raising the possibility that behavioral expressions of anxiety, and decision-making parameters of approach-avoidance conflict may be subserved by distinct neuronal subdomains with in the VH.
In conclusion, using a novel cued approach-avoidance conflict decision-making task combined with a chemogenetic approach that allowed selective targeting of excitatory neurons in the VH, the present study extends, and offers novel insight into our understanding of VH function in approach-avoidance conflict processing in a number of important ways. First, we have demonstrated that the excitatory neurons within a band of the vCA3 and vCA1 are preferentially involved in choice decisions that are made on the basis of conflicting cues in the absence of outcomes. Second, by allowing the independent measurement of two decision-making parameters: wait time and choice, the present task enabled us to identify a subarea in the VH that is critical in the deliberation and initiation of cue-elicited decision-making under two conflict situations that involve animals being presented with a decision of choosing between a potential small reward over large reward and punishment, or choosing safety (closed arms of EPM) over potential reward and threat (open arms). These findings have implications for neuropsychiatric diseases in which approach-avoidance conflict decision-making is likely aberrant [8, 9].
Funding and disclosure
This study was funded by the Natural Sciences and Engineering Research Council of Canada (N.S.E.R.C, 402642) and Canadian Institutes of Health Research (390877). We declare that there are no competing financial interests with the work described.
Rangel A, Camerer C, Montague PR. A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci. 2008;9:545–56.
Bailey MR, Simpson EH, Balsam PD. Neural substrates underlying effort, time, and risk-based decision making in motivated behavior. Neurobiol Learn Mem. 2016;133:233–56.
Smith S, Guthrie ER. General psychology in terms of behavior. New York, NY: D. Appleton & Co.; 1921.
Brown JS. Generalized approach and avoidance responses in relation to conflict behavior. Doctoral dissertation. Yale University, 1940.
Brown JS. The generalization of approach responses as a function of stimulus intensity and strength of motivation. J Comp Psychol. 1942;33:209.
Brown JS. Gradients of approach and avoidance responses and their relation to level of motivation. J Comp Physiol Psychol. 1948;41:450.
Miller NE, Murray EJ. Displacement and conflict; learnable drive as a basis for the steeper gradient of avoidance than of approach. J Exp Psychol. 1952;43:227–31.
Nguyen D, Schumacher A, Erb S, Ito R. Aberrant approach-avoidance conflict resolution following repeated cocaine pre-exposure. Psychopharmacology. 2015;232:3573–83.
Aupperle RL, Melrose AJ, Francisco A, Paulus MP, Stein MB. Neural substrates of approach-avoidance conflict decision-making. Hum Brain Mapp. 2015;36:449–62.
Fricke K, Vogel S. How interindividual differences shape approach-avoidance behavior: relating self-report and diagnostic measures of interindividual differences to behavioral measurements of approach and avoidance. Neurosci Biobehav Rev. 2020;111:30–56.
Bannerman DM, Rawlins JNP, McHugh SB, Deacon RM, Yee BK, Bast T, et al. Regional dissociations within the hippocampus—memory and anxiety. Neurosci Biobehav Rev. 2004;28:273–83.
Jimenez JC, Su K, Goldberg AR, Luna VM, Biane JS, Ordek G, et al. Anxiety cells in a hippocampal-hypothalamic circuit. Neuron 2018;97:670–83.
Padilla-Coreano N, Bolkan SS, Pierce GM, Blackman DR, Hardin WD, Garcia-Garcia AL, et al. Direct ventral hippocampal-prefrontal input is required for anxiety-related neural activity and behavior. Neuron. 2016;89:857–66.
Sierra-Mercado D, Padilla-Coreano N, Quirk GJ. Dissociable roles of prelimbic and infralimbic cortices, ventral hippocampus, and basolateral amygdala in the expression and extinction of conditioned fear. Neuropsychopharmacology. 2011;36:529–38.
Çavdaroğlu B, Toy J, Schumacher A, Carvalho G, Patel M, Ito R. Ventral hippocampus inactivation enhances the extinction of active avoidance responses in the presence of safety signals but leaves discrete trial operant active avoidance performance intact. Hippocampus. 2020. https://doi.org/10.1002/hipo.23202.
Schumacher A, Villaruel FR, Ussling A, Riaz S, Lee ACH, Ito R. Ventral hippocampal CA1 and CA3 differentially mediate learned approach-avoidance conflict processing. Curr Biol. 2018;28:1318–24.e4.
Schumacher A, Vlassov E, Ito R. The ventral hippocampus, but not the dorsal hippocampus is critical for learned approach-avoidance decision making. Hippocampus. 2016;26:530–42.
Ito R, Lee ACH. The role of the hippocampus in approach-avoidance conflict decision-making: evidence from rodent and human studies. Behav Brain Res. 2016;313:345–57.
Bach DR, Guitart-Masip M, Packard PA, Miró J, Falip M, Fuentemilla L, et al. Human hippocampus arbitrates approach-avoidance conflict. Curr Biol. 2014;24:541–7.
O’Neil EB, Newsome RN, Li IHN, Thavabalasingam S, Ito R, Lee ACH. Examining the role of the human hippocampus in approach-avoidance decision making using a novel conflict paradigm and multivariate functional magnetic resonance imaging. J Neurosci. 2015;35:15039–49.
Wallis CU, Cockcroft GJ, Cardinal RN, Roberts AC, Clarke HF. Hippocampal interaction with area 25, but not area 32, regulates marmoset approach–avoidance behavior. Cereb Cortex. 2019. https://doi.org/10.1093/cercor/bhz015.
Talmi D, Pine A. How costs influence decision values for mixed outcomes. Front Neurosci. 2012;6:1–21.
Park SQ, Kahnt T, Rieskamp J, Heekeren HR. Neurobiology of value integration: when value impacts valuation. J Neurosci. 2011;31:9307–14.
Cheung THC, Cardinal RN. Hippocampal lesions facilitate instrumental learning with delayed reinforcement but induce impulsive choice in rats. BMC Neurosci. 2005. https://doi.org/10.1186/1471-2202-6-36.
Freestone DM, Church RM. Optimal timing. Curr Opin Behav Sci. 2016;8:276–81.
Schneider CA, Rasband WS, Eliceiri KW. NIH Image to ImageJ: 25 years of image analysis. Nat Methods. 2012;9:671–75.
Schindelin J, Arganda-Carreras I, Frise E, Kaynig V, Longair M, Pietzsch T, et al. Fiji: an open-source platform for biological-image analysis. Nat Methods. 2012;9:676–82.
Team R. Core. R: A language and environment for statistical computing. Ind Commer Train. 2019. https://doi.org/10.1108/eb003648.
Bates D, Maechler M, Bolker B, Walker S. Package“lme4”. J Stat Softw. 2015. https://doi.org/10.18637/jss.v067.i01.
Cavdaroglu B, Zeki M, Balci F. Time-based reward maximization. Philos Trans R Soc B Biol Sci. 2014;369:20120461–20120461.
Wichmann FA, Hill NJ. The psychometric function: I. Fitting, sampling, and goodness of fit. Percept Psychophys. 2001;63:1293–313.
Orsini CA, Trotta RT, Bizon JL, Setlow B. Dissociable roles for the basolateral amygdala and orbitofrontal cortex in decision-making under risk of punishment. J Neurosci. 2015;35:1368–379.
Orsini CA, Hernandez CM, Singhal S, Kelly KB, Frazier CJ, Bizon JL, et al. Optogenetic inhibition reveals distinct roles for basolateral amygdala activity at discrete time points during risky decision making. J Neurosci. 2017;37:11537–48.
Vento PJ, Burnham NW, Rowley CS, Jhou TC. Learning from one’s mistakes: a dual role for the rostromedial tegmental nucleus in the encoding and expression of punished reward seeking. Biol Psychiatry. 2017;81:1041–9.
Zeeb FD, Robbins TW, Winstanley CA. Serotonergic and dopaminergic modulation of gambling behavior as assessed using a novel rat gambling task. Neuropsychopharmacology. 2009;34:2329–43.
van den Bos R, Lasthuis W, den Heijer E, van der Harst J, Spruijt B. Toward a rodent model of the Iowa gambling task. Behav Res Methods. 2006;38:470–8.
Evenden JL, Ryan CN. The pharmacology of impulsive behaviour in rats: the effects of drugs on response choice with varying delays of reinforcement. Psychopharmacology. 1996;128:161–70.
Cardinal RN, Howes NJ. Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci. 2005;6:37.
Winstanley CA, Floresco SB. Deciphering decision making: variation in animal models of effort- and uncertainty-based choice reveals distinct neural circuitries underlying core cognitive processes. J Neurosci. 2016;36:12069–79.
Simon NW, Gilbert RJ, Mayse JD, Bizon JL, Setlow B. Balancing risk and reward: a rat model of risky decision making. Neuropsychopharmacology. 2009;34:2208–17.
Oberrauch S, Sigrist H, Sautter E, Gerster S, Bach DR, Pryce CR. Establishing operant conflict tests for the translational study of anxiety in mice. Psychopharmacology. 2019;236:2527–41.
Abela AR, Dougherty SD, Fagen ED, Hill CJR, Chudasama Y. Inhibitory control deficits in rats with ventral hippocampal lesions. Cereb Cortex. 2013;23:1396–409.
Freestone DM, Balcı F, Simen P, Church RM. Optimal response rates in humans and rats. J Exp Psychol Anim Learn Cogn. 2015;41:39–51.
Bannerman DM, Deacon RM, Offen S, Friswell J, Grubb M, Rawlins JNP. Double dissociation of function within the hippocampus: spatial memory and hyponeophagia. Behav Neurosci. 2002;116:884–901.
Bannerman DM, Yee BK, Good MA, Heupel MJ, Iversen SD, Rawlins JNP. Double dissociation of function within the hippocampus: a comparison of dorsal, ventral, and complete hippocampal cytotoxic lesions. Behav Neurosci. 1999;113:1170–88.
Sutherland RJ, McDonald RJ, Hill CR, Rudy JW. Damage to the hippocampal formation in rats selectively impairs the ability to learn cue relationships. Behav Neural Biol. 1989;52:331–56.
Ito R, Everitt BJ, Robbins TW. The hippocampus and appetitive Pavlovian conditioning: effects of excitotoxic hippocampal lesions on conditioned locomotor activity and autoshaping. Hippocampus. 2005;15:713–21.
Riaz S, Schumacher A, Sivagurunathan S, Van Der Meer M, Ito R. Ventral, but not dorsal, hippocampus inactivation impairs reward memory expression and retrieval in contexts defined by proximal cues. Hippocampus. 2017;27:822–36.
Wilson RC, Takahashi YK, Schoenbaum G, Niv Y. Orbitofrontal cortex as a cognitive map of task space. Neuron 2014;81:267–79.
Bradfield LA, Dezfouli A, Van Holstein M, Chieng B, Balleine BW. Medial orbitofrontal cortex mediates outcome retrieval in partially observable task situations. Neuron. 2015;88:1268–80.
Wikenheiser AM, Schoenbaum G. Over the river, through the woods: cognitive maps in the hippocampus and orbitofrontal cortex. Nat Rev Neurosci. 2016;17:513–23.
Wikenheiser AM, Marrero-Garcia Y, Schoenbaum G. Suppression of ventral hippocampal output impairs integrated orbitofrontal encoding of task structure. Neuron. 2017;95:1197–207.
Jarrard LE, Isaacson RL. Hippocampal ablation in rats: effects of intertrial interval. Nature. 1965;207:109–10.
Butter CM, Mishkin M, Rosvold HE. Conditioning and extinction of a food-rewarded response after selective ablations of frontal cortex in rhesus monkeys. Exp Neurol. 1963;7:65–75.
De Martino B, Fleming SM, Garrett N, Dolan RJ. Confidence in value-based choice. Nat Neurosci. 2013;16:105–10.
Boguszewski P, Zagrodzka J. Emotional changes related to age in rats—a behavioral analysis. Behav Brain Res. 2002;133:323–32.
Yeates DCM, Ussling A, Lee ACH, Ito R. Double dissociation of learned approach–avoidance conflict processing and spatial pattern separation along the dorsoventral axis of the dentate gyrus. Hippocampus. 2019. https://doi.org/10.1002/hipo.23182.
Adhikari A. Distributed circuits underlying anxiety. Front Behav Neurosci. 2014;8:112.
Bannerman D, Grubb M, Deacon RM, Yee B, Feldon J, Rawlins JN. Ventral hippocampal lesions affect anxiety but not spatial learning. Behav Brain Res. 2003;139:197–213.
Kjelstrup KG, Tuvnes FA, Steffenach HA, Murison R, Moser EI, Moser MB. Reduced fear expression after lesions of the ventral hippocampus. Proc Natl Acad Sci USA. 2002. https://doi.org/10.1073/pnas.152112399.
Trivedi MA, Coover GD. Lesions of the ventral hippocampus, but not the dorsal hippocampus, impair conditioned fear expression and inhibitory avoidance on the elevated T-maze. Neurobiol Learn Mem. 2004;81:172–84.
Fanselow MS, Dong HW. Are the dorsal and ventral hippocampus functionally distinct structures? Neuron. 2010;65:7–19.
Kheirbek MA, Drew LJ, Burghardt NS, Costantini DO, Tannenholz L, Ahmari SE, et al. Differential control of learning and anxiety along the dorsoventral axis of the dentate gyrus. Neuron. 2013;77:955–68.
Engin E, Smith KS, Gao Y, Nagy D, Foster RA, Tsvetkov E, et al. Modulation of anxiety and fear via distinct intrahippocampal circuits. Elife. 2016;5:e14120.
Dong HW, Swanson LW, Chen L, Fanselow MS, Toga AW. Genomic-anatomic evidence for distinct functional domains in hippocampal field CA1. Proc Natl Acad Sci USA. 2009. https://doi.org/10.1073/pnas.0812608106.
Ciocchi S, Passecker J, Malagon-Vina H, Mikus N, Klausberger T. Selective information routing by ventral hippocampal CA1 projection neurons. Science. 2015;348:560–3.
Paxinos G, Watson C. The rat brain in stereotaxic coordinates. San Diego, CA: Acad Press. 1998. p. 1–474.
We are grateful to Dr. Maithe Arruda Carvalho for providing the training and use of her Nikon-Ni U fluorescent microscope.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Çavdaroğlu, B., Riaz, S., Yeung, E.H.L. et al. The ventral hippocampus is necessary for cue-elicited, but not outcome driven approach-avoidance conflict decisions: a novel operant choice decision-making task. Neuropsychopharmacol. 46, 632–642 (2021). https://doi.org/10.1038/s41386-020-00898-z
This article is cited by
Blocking μ-opioid receptors attenuates reinstatement of responding to an alcohol-predictive conditioned stimulus through actions in the ventral hippocampus
Two opposing hippocampus to prefrontal cortex pathways for the control of approach and avoidance behaviour
Nature Communications (2022)
Parallel ventral hippocampus-lateral septum pathways differentially regulate approach-avoidance conflict
Nature Communications (2022)