Introduction

Decision neuroscience has made significant progress in identifying neurobiological correlates of value representations using paradigms involving simple choices between two stimuli based on underlying preferences1,2. A value network involving a fronto-striatal circuit including the ventral striatum (VS) and the ventromedial prefrontal cortex (vmPFC), and posterior cingulate cortex (PCC) has been proposed3,4. An unsolved question is where the value signals processed by this network come from, particularly for complex stimuli.

One influential conceptualization of preference construction proposes multiple steps including 1. retrieval of relevant experiences with stimuli in the choice set, 2. comparison of relevant attributes to reach a decision value and 3. imagining future consequences of potential choices, that can be categorized as memory-related processes (retrospective or prospective)5,6.

A long line of work in cognitive neuroscience shows the importance of the medial temporal lobe (MTL) in these processes7. The involvement and interaction of the MTL with the value network only recently attracted attention8,9,10. Wimmer and Shohamy (2012) show MTL involvement in the value transfer of rewarded stimuli by associative learning that biases later decisions on non-rewarded stimuli. Barron, Dolan, and Behrens (2013) show activity in the hippocampus, in addition to medial prefrontal cortex, when subjects were asked to indicate preferences for novel food items based on familiar, but previously uncombined tastes. Gluth et al. (2015) show that choices are limited by memory constraints, which is associated with functional connectivity between the hippocampus and vmPFC11. Work motivated by the hippocampus’ involvement in imagining future experiences12,13 find that participants asked to imagine future events make more patient value-related decisions across time, which correlates with stronger activity in a set of brain regions including the hippocampus14. Impairment of these structures relates to more impatient choices, as shown in patients with subjective cognitive impairments regarded as a pre-stage of neurodegenerative disorders15.

These studies suggest the involvement of the hippocampus and memory processes in value-related decision-making, but do not provide conclusive evidence that these processes are needed for such decisions. Such evidence requires comparing value-related decision-making abilities in the absence or impairment of these brain regions. Finding such differences would substantiate psychological models of decision-making involving memory processes and extend our understanding of the neural value network and the origins of value signals for complex options. Work that established the role of the ventromedial frontal region as crucial in the value network used this method: Patients with damage in these areas perform poorly in value-related decisions compared both to healthy controls, as well as patients with lesions elsewhere in the frontal cortex16,17.

Given these findings, we asked whether patients with hippocampal sclerosis are impaired in making consistent value-based decisions. Hippocampal sclerosis is a key neuropathological feature in patients with medial temporal lobe epilepsy18, with neurosurgical removal of the medial temporal lobe showing a high seizure-free rate. These patients show neuropsychological deficits mainly in the memory domain19,20. To control for other epilepsy-related factors, like anticonvulsive medication or social effects of having seizure, we included in addition to healthy controls (CON), a control group of patients with lesions outside of the temporal lobe (extratemporal group ETL). We tested the effect on value-based decisions with binary choices among familiar food products. Our measure of choice quality was transitivity, the degree to which preferences are internally consistent (in our case between different candy bars). If a person chose Rolo over Bounty, and Bounty over Mars, choice transitivity requires they pick Rolo over Mars21. Decision neuroscience uses this metric to quantify choice quality16,17,22,23. As in this previous research, we included a pairwise judgment (rather than preference) task as a control, presenting respondents with pairs of numbers and asking them to judge which of the two is larger. This protocol was similar to that used to establish the necessary role of the vmPFC in value-related decisions16. Thus, selective differences in patients with MTL damage in value-based choices between candy bars compared to numerical decisions should provide strong evidence for the involvement of the hippocampus.

Results

Figure 1 qualitatively shows that MTL patients make a greater percentage of intransitive choices compared to the two control groups in the preference task, but not in the control task (mean (sd) percentages for the preference task: CON: 2.75% (1.41%); ETL: 3.37% (2.16%); MTL: 6.07% (5.02%); median percentages: CON: 2.94%; ETL 2.72%; MTL: 4.56%; mean (sd) percentages for the control task: CON: 0.14% (0.20%); ETL: 1.00% (2.54%); MTL: 0.50% (0.53%), median percentages: CON: 0.04%; ETL: 0.00%; MTL: 0.36%. Quantitatively we used a linear mixed model regressing log transformed intransitivity percentages from both tasks on an interactive model of group and task factors with orthogonal group contrasts. The interactions in this model compared the difference in error rates across the two tasks using the non-lesion control group as the baseline and first comparing the ETL group difference to this group and then the MTL group to both control groups. The MTL-group task interaction was b = – 0.06, t(91) = –2.98, p = 0.004. The difference between degree of intransitivity between the preference and control task did not differ significantly between the two control groups (linear mixed model with orthogonal contrasts ETL-group task interaction b = – 0.04, t(91) = 0.97, p = 0.333). We chose this analytical approach of comparing differences between the two tasks over analyzing rates of intransitivity separately across the tasks due to the lower error rates in the control task. We discuss potential consequences of the imbalance in intransivities between the two tasks in the Discussion.

Figure 1
figure 1

Mean percentage of intransitive choices per group in each task (nMTL = 31, nC = 30, nETL = 30). Error bars represent SEM.

Full size image

For a random subset of MTL-patients with available MRIs (n = 16; the remaining MTL-patients were not included due to lack of available data) we determined the ratio of compromised hippocampal volume to total volume (lateral damage index; LDI) and correlated this individual difference variable with the percentage of intransitive choices observed for these participants. We used a non-parametric correlation coefficient that is insensitive to outliers because it is calculated using rank order. We found a strong and significant relationship between these two variables, as shown in Fig. 2 (Spearman-rho = 0.676; F(1, 14) = 11.78, p = 0.004; n = 16), such that the larger the lesion volume, the less consistent were the value-based choices. We did not find any lateralized differences in proportion of intransitivity comparing subjects with more intact left MTLs to those with more intact right MTLs (t(14) = 0.731, p = 0.478). Unfortunately however these results were severely underpowered due to the small sample size and we are not able to report more detailed analyses on the effects of lesion extent.

Figure 2
figure 2

(a) Relationship between hippocampal lesion volume and intransitive choices. Scatterplot of compromised hippocampal volume (LDI - as a ratio of total volume quantified as the laterality index described in Equation 2) against percentage of intransitive choices for a random subset of n = 16 of the MTL patients. Smoothing is done locally (loess) with α = 2. The observed robust nonparametric rank order correlation rho = 0.676, p = 0.004. (b) Representative image of a single patient with right hippocampal sclerosis using radiological convention. Arrows highlight the MTL atrophy on the right hippocampus. This patient has a hippocampal ratio of 0.24.

Full size image

To provide context for interpreting the observed frequencies of intransitivity, we conducted a series of simulations that use a random utility model with a stochastic term added to the utility of the options, such that the probability of choosing option A (p(A)) in a decision between A and B is:

$$p(A)=\frac{1}{1+{e}^{((1-\alpha )u(B)+\alpha \varepsilon )-((1-\alpha )u(A)+\alpha \varepsilon )}}$$
(1)

where u(A) and u(B) represent the utilities of options A and B, α represented the proportion (between 0 and 1) of the observed utility due to random error, and ε is the random error. It can be shown analytically that the maximum proportion of intransitive triples is 0.25 (Fig. 3, also see the discussion section of Tversky, 1969). Our question of interest is the effect of α, the proportion of random error upon intransitivity. Our hypothesis is that the degree of MTL patients’ hippocampal sclerosis increases α, since access to past experiences that would normally be called on to make a choice is impaired. We simulated how the proportion of intransitive triples increases as noise in utilities increases. The effect was non-linear (Fig. 4), and the observed intransitivities in the MTL group corresponded to an α of 0.3, i.e., approximately 30 percent of the computed utility values in Equation 1 were based on random error.

Figure 3
figure 3

Tree diagram indicating possible intransitive paths from three binary choices. Each “ > ” indicates a choice of one candy bar over the other. Of the eight potential choice patterns that three options can result in only two of them are intransitive (25%). This provides the theoretical upper limit for the percentage of intransitivities.

Full size image
Figure 4
figure 4

Mean percentage of intransitivities at different noise levels (α), based on 1000 simulations at each noise level. Error bars indicate standard errors. At maximum noise level (random choice) the percentage of intransitivities approaches the theoretical upper limit of 25%. The increase to this limit is non-linear. The MTL-group’s behavior aligns with a noise level of 30%.

Full size image

We can test several alternative explanations to our account of random error in value construction for our data. One alternative explanation is that respondents retain explicit episodic memory of previous value comparisons during the task, and do not perform value construction for the two options of each pairwise choice. Under this account, non-MTL respondents may have better memory for their choices made earlier in the task, and this better episodic memory prevents intransitive choices. This account would suggest that the rate of intransitivities declines over time, as previous choices are remembered and used to avoid intransitive later choices. We might expect this decline in intransitivities over choice trials would differ for the MTL and non-MTL groups. We tested this hypothesis by regressing whether or not a triplet was intransitive on the trial number of the last seen trial in that triplet. The third trial in the triplet was chosen specifically because under this account the third trial is the critical one in which the choice would rely on the recollection of the choices from the previous two trials. We found no increase in the probability of a triplet being intransitive depending on when the subjects saw the last trial in that triplet (b = 0.027, z = 0.79, p = 0.427) nor was this trend different for the MTL group (b = 0.032, z = −0.78, p = 0.434).

An account emphasizing episodic memories of previous choices during the task makes a more specific hypothesis: It would predict that the probability of instransitivity depends on the delay (number of trials) between the choices involving the items that define an intransitive triplet. To test this we checked whether a triplet was more likely to be intransitive depending on the variance in the trial numbers involved in that triplet. We found that the further apart from each other the three choices in a triplet were made the more likely they were to be intransitive (b = 0.109, z = 3.40, p = 0.007). Crucially, however, this pattern was not different for the MTL group (b = −0.049, z = 1.25, p = 0.213). That is, the group differences in intransivity cannot be explained by impairments of episodic memories during the task. Importantly neither of these two analyses were intended to rule out the role of mnemonic processes overall in explaining the hippocampal contribution to value-based decisions. They eliminated a memory account that relies only on episodic memories of task performance and potentially support a role that relies on associative processes that rely on longer-term episodic memories in constructing value representations.

Another alternative explanation involves group differences in speed-accuracy tradeoff. One could hypothesize that intransitive choices result from speedy decisions and the MTL group makes more of them because they are more careless and faster in their choices. To test this, we examined response latencies of the choices, and the relationship between responses latencies and intransitivities for MTL and non-MTL groups. Contrary to a speed-accuracy tradeoff to explain intransitive choices, we found that slower (rather than faster) trials were more likely to be involved in intransitive triplets (b = 0.441, t(16985) = 4.40, p = 0.00001) for all groups, and that this did not differ for the MTL group (i.e., no interaction with this group: b = −0.0846, t(16985) = −0.62, p = 0.535, though there was a quadratic effect of time for the ETL group b = −0.382, t(16985) = −2.69, p = 0.007). Moreover, the MTL group actually had a significantly slower average response time per trial (b = 0.301, t(88) = 2.11, p = 0.038). Together, these results suggest that response times are reflective of an associative valuation process. Intransitive triplets accompanied more effortful and longer responding, eliminating the possibility that they rise from quick and careless decisions as could have been suggested by a speed-accuracy tradeoff.

Notably both the speed accuracy tradeoff and the effect of the trial number of the last trial in a triplet on intransitivity is the same for the numbers task as it is for the choice task, suggesting that the two tasks shared some similarities. Finally, we examined whether there were any idiosyncratic effects on preference intransitivity associated with specific stimuli (candy bars). We found no significant differences in the average number of intransitive triplets each candy bar was involved in (F(1, 90) = 0.003, p = 0.955). Though we did not have any additional measures of candy-bar-familiarity this analysis suggests that the MTL patients did not have systematic problems with any specific bar.

In combination, these analyses suggested that the observed increase in transitivity violations for respondents with MTL lesions in the preference task but not number-comparison task, in a way that is related to the volume of hippocampal lesions, suggests a failure in value-related associations in this group.

Discussion

We provide support that brain regions associated with memory-related associative processes play a critical role in value-based decision-making. Hippocampal lesions were associated with an increase in intransitive value-based choices, and the degree of intransitivity is related to magnitude of the damage to the hippocampus (LDI). A control task not involving value-based processes did not show these effects, nor do respondents who have lesions outside of the medial temporal lobe. These dissociation results implicate a crucial role for the hippocampal areas in preference construction24, a conceptualization in behavioral decision research that contrasts with standard theories of rational choice that implicitly assume stable utility functions and choice options with preexisting values.

While these data provide support for a hippocampal role in value-based decision-making they have some shortcomings as well. First, the control task where the subjects were asked to indicate the larger of two numbers, does not appear to be matched in difficulty to the experimental task as indicated by the much lower error rates. This task was chosen for its conceptual similarity to control tasks used in the literature and we tried addressing this discrepancy analytically by modeling the data together rather than separately. Despite our efforts to address this issue the imbalance could suggest that the pattern observed in the value-based decisions is due to an inability to process difficult choices. Though it is hard to delineate whether the ‘difficulty’ of value-based decisions would be dissociated from the hypothesized deficits in associative capabilities future studies call for more extensive piloting in choosing the control task. Second, we are very limited in our analyses of the role of lesion extent because imaging data were only available for a quarter of the clinical sample. Since the patients’ primary reason to visit was not to complete behavioral experiments but to receive healthcare we were restricted in our data collection efforts both in terms of imaging as well as additional neuropsychological tasks. Finally, we assumed familiarity of all experimental stimuli (candy bars) for all subjects but did not have any other tests to check for potential effects of this variable.

Two conceptual clarifications are in order. Our central dependent measure, the frequency of intransitive preferences, has been used before to examine the inability of decision makers to produce a stable representation of the value of choice options with other patient groups16,17. Earlier work, however, using choice intransitivity as a dependent measure did so to identify choice heuristics incompatible with utility maximization25. This resulted in a debate on the correct probabilistic model of transitivity that would account for errors in experimental data and whether that was evidence for a particular mechanism26,27,28. Our use of the term pairwise “transitivity” is not based on these frameworks and our design with two alternatives per choice does not employ such model comparison. We use intransitivity counts, as in other decision neuroscience research, instead, to examine error associated with the construction of value representations.

Second, our use of the term “transitivity” is only marginally related to the extensive literature measuring transitive inference, where a set of premises are learned in the experiment and participants are asked to generalize these learned rules to novel contexts and combinations of stimuli. Transitive inference tasks have been instrumental in establishing the role of the hippocampus in representing organizations of stimulus relations29. Animal lesion studies established the necessity of the hippocampus for transitive inference30,31, and data from humans has confirmed the involvement of this region32,33. However transitive inference paradigms differ from ours, critically, because our respondents are stating their preferences, not learned premises. We do not present participants with transitive relations and ask them to reason following this rule. We ask for their preference between two candy bars. We do not hypothesize that if a participant chooses Snickers over Mars and Mars over Bounty they would also choose Snickers over Bounty because they are instructed that these choices must follow a given transitive relationship. Instead, their transitive choice reflects an anticipation that they will enjoy Snickers more. That is, while a transitive inference task implies a strict ordinal relationship between stimuli thereby recruiting working memory, transitivity of choice as measured by our design relies on values learned over time and presumably relies on the recruitment of associative faculties34.

Despite the evidence for the involvement of the hippocampus in consistent value-based decisions, the delineation of specific cognitive and neural mechanisms provides multiple avenues for future research.

First, the hippocampus is just one part in a larger network of relevant brain areas involved in the retrieval and processing of choice values. A recent review35 suggests hippocampal involvement in a variety of cognitive functions outside of the domain of declarative memory providing two different hypotheses of hippocampal function: The memory modulation hypothesis proposes that representations within the hippocampus may transiently bias other cognitive functions by providing memory input into computations going on in other brain systems e.g. value computations in our task. The adaptive function hypothesis, in contrast, highlights the hippocampus as being an important central processing unit with specific computations carried out in the hippocampal networks themselves, depending on the task at hand.

Our hippocampal patients produce patterns of intransitivity of value-based choice that are similar to those observed in ventromedial prefrontal cortex (vmPFC) patients, suggesting that the associations and memories stored in the hippocampus may serve as inputs to value calculation occurring elsewhere10, potentially in line with the memory modulation hypothesis. The hippocampus is one of the most highly interconnected brain areas36,37. In addition to being directly and monosynaptically connected to the prefrontal cortex, animal work suggests additional indirect connections that are topographically specific and project on functionally distinct prefrontal regions36,37.

This possibility calls for a nuanced investigation of the interactions between hippocampal and prefrontal regions in value-based decision-making. For example, Ranganath and Ritchey38 propose a division of the MTL into two systems for memory-guided behavior: the anterior (AT) and posterior-medial (PM) system. The AT, which is comprised of the perirhinal cortex and anterior parts of the hippocampus and amygdala has strong interconnections with the frontal cortex, has been argued to be involved in familiarity-based cognition, social behavior and saliency. This is also the part of the hippocampus which is most affected in patients with hippocampal sclerosis39. Ranganath and Ritchey suggest that the AT system could facilitate the use of past experiences to inform inferences about the personality and intentions of others. More recently, Zeidman and Maguire40 connected the diverse set of results human AT is involved in by arguing that it might be responsible for scene representations. Their conceptualization of ‘scenes’ includes imagined episodes, which for our task can incorporate anticipated pleasure of choosing different bars. While very speculative and cannot be proven with the data available in this paper alone such inferential abilities specific to distinct regions in the MTL along with the connection to the ventromedial prefrontal cortex may play a role in value-based decisions.

On the other hand, in line with an adaptive function hypothesis, deficits in consistent choices might be due to hippocampus-specific computations. For example22 showed that vmPFC lesioned patients differ from normal controls in their external information search (revealing different pieces of information about choice options by clicking on them), in ways that could be attributed to diminished planning capacity. Perhaps this planning capacity relies on hippocampus-specific computations. An interesting topic of research would be whether vmPFC patients exhibit deficits in different mnemonic processes.

A second future research topic are potential compensation mechanisms in patients with chronic hippocampal lesions. It is well-known that chronic brain lesions may lead to compensatory shifts in neural processes, e.g. in the domain of language processing41. The application of neuroimaging methods during a value-based decision task in these patients could provide answers to these questions. Along these lines, a more comprehensive neuropsychological battery ruling out group differences in other potential cognitive processes (e.g. attention, engagement, familiarity with bars etc.) would restrict the behavioral differences to the hypothesized associative processes. While we tried to address some potential questions with sophisticated behavioral analyses we did not have access to such data. More extensive control tasks would further constrain the implicated cognitive functions to value-related processes, as many participants from all groups performed at ceiling in our number comparison control task limiting its conclusiveness.

Third, although patients with temporal lobe epilepsy and hippocampal sclerosis do show neuropsychological deficits especially in the domain of declarative memory, the amount to which these deficits occur varies strongly between patients20. Future research combining in-depth neuropsychological testing together with value-based choice tasks may shed light on the specific cognitive components underlying the observed range of decision deficits.

Our results suggest a potentially critical role for the hippocampus in the construction of the value of choice options. Most decisions require the construction of value based on past experience. Even a previously experienced option, like a favorite dish in a familiar restaurant, requires us to compare recollections of the value of that option to newly available options such as tonight’s specials. A better understanding of both internal and external inputs to preference construction processes and their aggregation and comparison will allow us to comprehend and model how the brain calculates value and makes consistent choices.

Methods

The study was approved by the local ethics committee of the University of Bonn and the Institutional Review Board at Columbia University (IRB-AAAB1301). All subjects gave their written informed consent and all experiments were performed in accordance with relevant guidelines and regulations.

A total of 91 respondents participated. Thirty-one patients (15 female; mean age 47.74 with SD 2.56; Table 1) suffering from medial temporal lobe epilepsy with clinically diagnosed uni-lateral (left:n = 14;right:n = 8) or bilateral (n = 9) hippocampal sclerosis from the presurgical program at the Department of Epileptology in Bonn were included in the study. Different from patients with lesions in the vmPFC16 who can have lesions with little overlap to each other, the lesion locations in MTL patients are very similar. This makes lesion volume a better individual difference marker, as described below. Two control groups consisted of thirty patients with extratemporal lobe epilepsy (14 female; mean age 43.10 with SD 2.60; ETL group; Table 1) and thirty healthy control subjects (15 female; mean age 51.40 with SD 2.60; CON group; Table 1), respectively.

Table 1 Demographics.
Full size table

Each respondent made a series of choices between pairs of 20 candy bars, presented pictorially on a computer. Each pairwise combination was presented once, resulting in (20 × 19)/2 = 190 choices for each participant, with a different random order. In a control task, subjects were presented with pairs of numbers, drawn from the range of one to twenty, and had to judge which number was larger. We computed judgment inconsistency across triplets of comparison identically for the two tasks. Subjects knew that they would receive their candy bar of choice from one randomly selected choice trial, in addition to a participation fee of 10 € at the end of all testing.

Our focal dependent measure was the proportion of intransitive choices. A triplet is intransitive if (i) A was chosen over B and B was chosen over C, yet C was chosen over A or (ii) if B was chosen over A and C was chosen over B, yet A was chosen over C.

The proportion of intransitive choices was obtained by dividing the number of intransitive triplets by the total number of triplets. Analytically, it can be shown that the maximum level of intransitivities (those produced by a random responder) is 25% of all triplets. We report the results of simulations that demonstrate the non-linear relationship between number of intransitive choices and response error.

We also obtained, for a random subgroup of the patients with unilateral hippocampal sclerosis (n = 16), a 3D-T1 weighted high-resolution data set (MP-RAGE, voxel size 1 × 1 × 1mm, repetition time 1570ms, echo time 3.42ms, flip angle 15°, field of view 256 mm × 256 mm) for volumetric measurement of the hippocampus. This was done in a fully automated manner by means of the FreeSurfer image analysis suite (Version 5.1.0, Martinos Center, Harvard University, Boston, MA, USA.; FreeSurfer, RRID:SCR_001847)42,43. Because of the high variance in total hippocampal volume between individuals, we used a lateral damage index of hippocampal volume to express the extent of unilateral hippocampal damage in our MTL group:

$$LDI=abs(\frac{{V}_{Hipp{o}_{L}}-{V}_{Hipp{o}_{R}}}{{V}_{Hipp{o}_{L}}+{V}_{Hipp{o}_{R}}})$$
(2)

This lateral damage index can obviously be only assessed for subjects with unilateral hippocampal sclerosis.

Experimental Design and Statistical Analysis

Our sample size was constrained by the availability of MTL patients with the appropriate lesion. Still, a power analysis based on effect sizes for healthy participants in the literature suggested we were well powered. Assuming a base proportion of 3% of intransitivities for healthy controls44 and the same for ETL patients in contrast to twice this amount for the MTL patients, a large (and therefore conservative) estimate (i.e. an effect size of f = 0.4), we would need a total of 60 participants for a power level of 0.95. Our sample with at least 30 subjects per group was well above this.

To perform statistical analysis on our focal behavioral dependent measure, the intransitivity proportions were log transformed to avoid non-normal distributions and unequal variances between the tasks (Bartlett test K2(1) = 58, p < 0.001 pre transformation; K2(1) = 3.4, p = 0.07 post-transformation). Based on model comparisons (details of compared models can be found under Group difference in intransitivities > Summary Statics in the link provided for the analysis report), a linear mixed model was deemed the appropriate analysis having compared it to simpler models with no random effects. The contrasts of this model were orthogonalized to allow a direct comparison of the ETL group to the healthy controls and of the MTL group to both non-lesion and the ETL control groups together.

Statistical analyses were performed using R (Version 3.3.2; R Project for Statistical Computing, RRID:SCR_001905) for Mac. We use a two-tailed p-value of 0.05 as our criterion for statistical significance. The details of multilevel models are reported in the Results section.

Code availability

All analysis code can be found under https://github.com/zenkavi/TransitivityOpen with a full analysis report including additional figures depicting individual data points under https://zenkavi.github.io/TransitivityOpen/Transitivity_OpenAnalyses.nb.html

Data availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.