Evidence for hippocampal dependence of value-based decisions

Consistent decisions are intuitively desirable and theoretically important for utility maximization. Neuroeconomics has established the neurobiological substrate of value representation, but brain regions that provide input to this network is less explored. The constructed-preference tradition within behavioral decision research gives a critical role to associative cognitive processes, suggesting a hippocampal role in making consistent decisions. We compared the performance of 31 patients with mediotemporal lobe (MTL) epilepsy and hippocampal lesions, 30 patients with extratemporal lobe epilepsy, and 30 healthy controls on two tasks: binary choices between candy bars based on their preferences and a number-comparison control task where the larger number is chosen. MTL patients made more inconsistent choices than the other two groups for the value-based choice but not the number-comparison task. These inconsistencies correlated with the volume of compromised hippocampal tissue. These results add to increasing evidence on a critical involvement of the MTL in preference construction and value-based choices.

regions. Finding such differences would substantiate psychological models of decision-making involving memory processes and extend our understanding of the neural value network and the origins of value signals for complex options. Work that established the role of the ventromedial frontal region as crucial in the value network used this method: Patients with damage in these areas perform poorly in value-related decisions compared both to healthy controls, as well as patients with lesions elsewhere in the frontal cortex 16,17 .
Given these findings, we asked whether patients with hippocampal sclerosis are impaired in making consistent value-based decisions. Hippocampal sclerosis is a key neuropathological feature in patients with medial temporal lobe epilepsy 18 , with neurosurgical removal of the medial temporal lobe showing a high seizure-free rate. These patients show neuropsychological deficits mainly in the memory domain 19,20 . To control for other epilepsy-related factors, like anticonvulsive medication or social effects of having seizure, we included in addition to healthy controls (CON), a control group of patients with lesions outside of the temporal lobe (extratemporal group ETL). We tested the effect on value-based decisions with binary choices among familiar food products. Our measure of choice quality was transitivity, the degree to which preferences are internally consistent (in our case between different candy bars). If a person chose Rolo over Bounty, and Bounty over Mars, choice transitivity requires they pick Rolo over Mars 21 . Decision neuroscience uses this metric to quantify choice quality 16,17,22,23 . As in this previous research, we included a pairwise judgment (rather than preference) task as a control, presenting respondents with pairs of numbers and asking them to judge which of the two is larger. This protocol was similar to that used to establish the necessary role of the vmPFC in value-related decisions 16 . Thus, selective differences in patients with MTL damage in value-based choices between candy bars compared to numerical decisions should provide strong evidence for the involvement of the hippocampus. Figure 1 qualitatively shows that MTL patients make a greater percentage of intransitive choices compared to the two control groups in the preference task, but not in the control task (mean (sd) percentages for the preference task: CON: 2.75% (1.41%); ETL: 3.37% (2.16%); MTL: 6.07% (5.02%); median percentages: CON: 2.94%; ETL 2.72%; MTL: 4.56%; mean (sd) percentages for the control task: CON: 0.14% (0.20%); ETL: 1.00% (2.54%); MTL: 0.50% (0.53%), median percentages: CON: 0.04%; ETL: 0.00%; MTL: 0.36%. Quantitatively we used a linear mixed model regressing log transformed intransitivity percentages from both tasks on an interactive model of group and task factors with orthogonal group contrasts. The interactions in this model compared the difference in error rates across the two tasks using the non-lesion control group as the baseline and first comparing the ETL group difference to this group and then the MTL group to both control groups. The MTL-group task interaction was b = -0.06, t(91) = -2.98, p = 0.004. The difference between degree of intransitivity between the preference and control task did not differ significantly between the two control groups (linear mixed model with orthogonal contrasts ETL-group task interaction b = -0.04, t(91) = 0.97, p = 0.333). We chose this analytical approach of comparing differences between the two tasks over analyzing rates of intransitivity separately across the tasks due to the lower error rates in the control task. We discuss potential consequences of the imbalance in intransivities between the two tasks in the Discussion.

Results
For a random subset of MTL-patients with available MRIs (n = 16; the remaining MTL-patients were not included due to lack of available data) we determined the ratio of compromised hippocampal volume to total volume (lateral damage index; LDI) and correlated this individual difference variable with the percentage of intransitive choices observed for these participants. We used a non-parametric correlation coefficient that is insensitive to outliers because it is calculated using rank order. We found a strong and significant relationship between these two variables, as shown in Fig. 2 (Spearman-rho = 0.676; F(1, 14) = 11.78, p = 0.004; n = 16), such that the larger the lesion volume, the less consistent were the value-based choices. We did not find any lateralized differences in proportion of intransitivity comparing subjects with more intact left MTLs to those with more intact right MTLs (t(14) = 0.731, p = 0.478). Unfortunately however these results were severely underpowered due to the small sample size and we are not able to report more detailed analyses on the effects of lesion extent. To provide context for interpreting the observed frequencies of intransitivity, we conducted a series of simulations that use a random utility model with a stochastic term added to the utility of the options, such that the probability of choosing option A (p(A)) in a decision between A and B is: where u(A) and u(B) represent the utilities of options A and B, α represented the proportion (between 0 and 1) of the observed utility due to random error, and ε is the random error. It can be shown analytically that the maximum proportion of intransitive triples is 0.25 (Fig. 3, also see the discussion section of Tversky, 1969). Our question of interest is the effect of α, the proportion of random error upon intransitivity. Our hypothesis is that the degree of MTL patients' hippocampal sclerosis increases α, since access to past experiences that would normally be called on to make a choice is impaired. We simulated how the proportion of intransitive triples increases as noise in utilities increases. The effect was non-linear ( Fig. 4), and the observed intransitivities in the MTL group corresponded to an α of 0.3, i.e., approximately 30 percent of the computed utility values in Equation 1 were based on random error. We can test several alternative explanations to our account of random error in value construction for our data. One alternative explanation is that respondents retain explicit episodic memory of previous value comparisons during the task, and do not perform value construction for the two options of each pairwise choice. Under this account, non-MTL respondents may have better memory for their choices made earlier in the task, and this better episodic memory prevents intransitive choices. This account would suggest that the rate of intransitivities declines over time, as previous choices are remembered and used to avoid intransitive later choices. We might expect this decline in intransitivities over choice trials would differ for the MTL and non-MTL groups. We tested this hypothesis by regressing whether or not a triplet was intransitive on the trial number of the last seen trial in that triplet. The third trial in the triplet was chosen specifically because under this account the third trial is the critical one in which the choice would rely on the recollection of the choices from the previous two trials. We found no increase in the probability of a triplet being intransitive depending on when the subjects saw the last trial in that triplet (b = 0.027, z = 0.79, p = 0.427) nor was this trend different for the MTL group (b = 0.032, z = −0.78, p = 0.434).
An account emphasizing episodic memories of previous choices during the task makes a more specific hypothesis: It would predict that the probability of instransitivity depends on the delay (number of trials) between the choices involving the items that define an intransitive triplet. To test this we checked whether a triplet was more likely to be intransitive depending on the variance in the trial numbers involved in that triplet. We found that the further apart from each other the three choices in a triplet were made the more likely they were to be intransitive (b = 0.109, z = 3.40, p = 0.007). Crucially, however, this pattern was not different for the MTL group (b = −0.049, z = 1.25, p = 0.213). That is, the group differences in intransivity cannot be explained by impairments of episodic memories during the task. Importantly neither of these two analyses were intended to rule out the role of mnemonic processes overall in explaining the hippocampal contribution to value-based decisions. They eliminated a memory account that relies only on episodic memories of task performance and potentially support a role that relies on associative processes that rely on longer-term episodic memories in constructing value representations.
Another alternative explanation involves group differences in speed-accuracy tradeoff. One could hypothesize that intransitive choices result from speedy decisions and the MTL group makes more of them because they are more careless and faster in their choices. To test this, we examined response latencies of the choices, and the relationship between responses latencies and intransitivities for MTL and non-MTL groups. Contrary to a speed-accuracy tradeoff to explain intransitive choices, we found that slower (rather than faster) trials were more likely to be involved in intransitive triplets (b = 0.441, t(16985) = 4.40, p = 0.00001) for all groups, and that this did not differ for the MTL group (i.e., no interaction with this group: b = −0.0846, t(16985) = −0.62, p = 0.535, though there was a quadratic effect of time for the ETL group b = −0.382, t(16985) = −2.69, p = 0.007). Moreover, the MTL group actually had a significantly slower average response time per trial (b = 0.301, t(88) = 2.11, p = 0.038). Together, these results suggest that response times are reflective of an associative valuation process.  Intransitive triplets accompanied more effortful and longer responding, eliminating the possibility that they rise from quick and careless decisions as could have been suggested by a speed-accuracy tradeoff.
Notably both the speed accuracy tradeoff and the effect of the trial number of the last trial in a triplet on intransitivity is the same for the numbers task as it is for the choice task, suggesting that the two tasks shared some similarities. Finally, we examined whether there were any idiosyncratic effects on preference intransitivity associated with specific stimuli (candy bars). We found no significant differences in the average number of intransitive triplets each candy bar was involved in (F(1, 90) = 0.003, p = 0.955). Though we did not have any additional measures of candy-bar-familiarity this analysis suggests that the MTL patients did not have systematic problems with any specific bar.
In combination, these analyses suggested that the observed increase in transitivity violations for respondents with MTL lesions in the preference task but not number-comparison task, in a way that is related to the volume of hippocampal lesions, suggests a failure in value-related associations in this group.

Discussion
We provide support that brain regions associated with memory-related associative processes play a critical role in value-based decision-making. Hippocampal lesions were associated with an increase in intransitive value-based choices, and the degree of intransitivity is related to magnitude of the damage to the hippocampus (LDI). A control task not involving value-based processes did not show these effects, nor do respondents who have lesions outside of the medial temporal lobe. These dissociation results implicate a crucial role for the hippocampal areas in preference construction 24 , a conceptualization in behavioral decision research that contrasts with standard theories of rational choice that implicitly assume stable utility functions and choice options with preexisting values.
While these data provide support for a hippocampal role in value-based decision-making they have some shortcomings as well. First, the control task where the subjects were asked to indicate the larger of two numbers, does not appear to be matched in difficulty to the experimental task as indicated by the much lower error rates. This task was chosen for its conceptual similarity to control tasks used in the literature and we tried addressing this discrepancy analytically by modeling the data together rather than separately. Despite our efforts to address this issue the imbalance could suggest that the pattern observed in the value-based decisions is due to an inability to process difficult choices. Though it is hard to delineate whether the 'difficulty' of value-based decisions would be dissociated from the hypothesized deficits in associative capabilities future studies call for more extensive piloting in choosing the control task. Second, we are very limited in our analyses of the role of lesion extent because imaging data were only available for a quarter of the clinical sample. Since the patients' primary reason to visit was not to complete behavioral experiments but to receive healthcare we were restricted in our data collection efforts both in terms of imaging as well as additional neuropsychological tasks. Finally, we assumed familiarity of all experimental stimuli (candy bars) for all subjects but did not have any other tests to check for potential effects of this variable.
Two conceptual clarifications are in order. Our central dependent measure, the frequency of intransitive preferences, has been used before to examine the inability of decision makers to produce a stable representation of the value of choice options with other patient groups 16,17 . Earlier work, however, using choice intransitivity as a dependent measure did so to identify choice heuristics incompatible with utility maximization 25 . This resulted in a debate on the correct probabilistic model of transitivity that would account for errors in experimental data and whether that was evidence for a particular mechanism [26][27][28] . Our use of the term pairwise "transitivity" is not based on these frameworks and our design with two alternatives per choice does not employ such model comparison. We use intransitivity counts, as in other decision neuroscience research, instead, to examine error associated with the construction of value representations.
Second, our use of the term "transitivity" is only marginally related to the extensive literature measuring transitive inference, where a set of premises are learned in the experiment and participants are asked to generalize these learned rules to novel contexts and combinations of stimuli. Transitive inference tasks have been instrumental in establishing the role of the hippocampus in representing organizations of stimulus relations 29 . Animal lesion studies established the necessity of the hippocampus for transitive inference 30,31 , and data from humans has confirmed the involvement of this region 32,33 . However transitive inference paradigms differ from ours, critically, because our respondents are stating their preferences, not learned premises. We do not present participants with transitive relations and ask them to reason following this rule. We ask for their preference between two candy bars. We do not hypothesize that if a participant chooses Snickers over Mars and Mars over Bounty they would also choose Snickers over Bounty because they are instructed that these choices must follow a given transitive relationship. Instead, their transitive choice reflects an anticipation that they will enjoy Snickers more. That is, while a transitive inference task implies a strict ordinal relationship between stimuli thereby recruiting working memory, transitivity of choice as measured by our design relies on values learned over time and presumably relies on the recruitment of associative faculties 34 .
Despite the evidence for the involvement of the hippocampus in consistent value-based decisions, the delineation of specific cognitive and neural mechanisms provides multiple avenues for future research.
First, the hippocampus is just one part in a larger network of relevant brain areas involved in the retrieval and processing of choice values. A recent review 35 suggests hippocampal involvement in a variety of cognitive functions outside of the domain of declarative memory providing two different hypotheses of hippocampal function: The memory modulation hypothesis proposes that representations within the hippocampus may transiently bias other cognitive functions by providing memory input into computations going on in other brain systems e.g. value computations in our task. The adaptive function hypothesis, in contrast, highlights the hippocampus as being an important central processing unit with specific computations carried out in the hippocampal networks themselves, depending on the task at hand. Our hippocampal patients produce patterns of intransitivity of value-based choice that are similar to those observed in ventromedial prefrontal cortex (vmPFC) patients, suggesting that the associations and memories stored in the hippocampus may serve as inputs to value calculation occurring elsewhere 10 , potentially in line with the memory modulation hypothesis. The hippocampus is one of the most highly interconnected brain areas 36,37 . In addition to being directly and monosynaptically connected to the prefrontal cortex, animal work suggests additional indirect connections that are topographically specific and project on functionally distinct prefrontal regions 36,37 .
This possibility calls for a nuanced investigation of the interactions between hippocampal and prefrontal regions in value-based decision-making. For example, Ranganath and Ritchey 38 propose a division of the MTL into two systems for memory-guided behavior: the anterior (AT) and posterior-medial (PM) system. The AT, which is comprised of the perirhinal cortex and anterior parts of the hippocampus and amygdala has strong interconnections with the frontal cortex, has been argued to be involved in familiarity-based cognition, social behavior and saliency. This is also the part of the hippocampus which is most affected in patients with hippocampal sclerosis 39 . Ranganath and Ritchey suggest that the AT system could facilitate the use of past experiences to inform inferences about the personality and intentions of others. More recently, Zeidman and Maguire 40 connected the diverse set of results human AT is involved in by arguing that it might be responsible for scene representations. Their conceptualization of 'scenes' includes imagined episodes, which for our task can incorporate anticipated pleasure of choosing different bars. While very speculative and cannot be proven with the data available in this paper alone such inferential abilities specific to distinct regions in the MTL along with the connection to the ventromedial prefrontal cortex may play a role in value-based decisions.
On the other hand, in line with an adaptive function hypothesis, deficits in consistent choices might be due to hippocampus-specific computations. For example 22 showed that vmPFC lesioned patients differ from normal controls in their external information search (revealing different pieces of information about choice options by clicking on them), in ways that could be attributed to diminished planning capacity. Perhaps this planning capacity relies on hippocampus-specific computations. An interesting topic of research would be whether vmPFC patients exhibit deficits in different mnemonic processes.
A second future research topic are potential compensation mechanisms in patients with chronic hippocampal lesions. It is well-known that chronic brain lesions may lead to compensatory shifts in neural processes, e.g. in the domain of language processing 41 . The application of neuroimaging methods during a value-based decision task in these patients could provide answers to these questions. Along these lines, a more comprehensive neuropsychological battery ruling out group differences in other potential cognitive processes (e.g. attention, engagement, familiarity with bars etc.) would restrict the behavioral differences to the hypothesized associative processes. While we tried to address some potential questions with sophisticated behavioral analyses we did not have access to such data. More extensive control tasks would further constrain the implicated cognitive functions to value-related processes, as many participants from all groups performed at ceiling in our number comparison control task limiting its conclusiveness.
Third, although patients with temporal lobe epilepsy and hippocampal sclerosis do show neuropsychological deficits especially in the domain of declarative memory, the amount to which these deficits occur varies strongly between patients 20 . Future research combining in-depth neuropsychological testing together with value-based choice tasks may shed light on the specific cognitive components underlying the observed range of decision deficits.
Our results suggest a potentially critical role for the hippocampus in the construction of the value of choice options. Most decisions require the construction of value based on past experience. Even a previously experienced option, like a favorite dish in a familiar restaurant, requires us to compare recollections of the value of that option to newly available options such as tonight's specials. A better understanding of both internal and external inputs to preference construction processes and their aggregation and comparison will allow us to comprehend and model how the brain calculates value and makes consistent choices.

Methods
The study was approved by the local ethics committee of the University of Bonn and the Institutional Review Board at Columbia University (IRB-AAAB1301). All subjects gave their written informed consent and all experiments were performed in accordance with relevant guidelines and regulations.
A total of 91 respondents participated. Thirty-one patients (15 female; mean age 47.74 with SD 2.56; Table 1) suffering from medial temporal lobe epilepsy with clinically diagnosed uni-lateral (left:n = 14;right:n = 8) or bilateral (n = 9) hippocampal sclerosis from the presurgical program at the Department of Epileptology in Bonn were included in the study. Different from patients with lesions in the vmPFC 16  Each respondent made a series of choices between pairs of 20 candy bars, presented pictorially on a computer. Each pairwise combination was presented once, resulting in (20 × 19)/2 = 190 choices for each participant, with a different random order. In a control task, subjects were presented with pairs of numbers, drawn from the range of one to twenty, and had to judge which number was larger. We computed judgment inconsistency across triplets of comparison identically for the two tasks. Subjects knew that they would receive their candy bar of choice from one randomly selected choice trial, in addition to a participation fee of 10 € at the end of all testing.
Our focal dependent measure was the proportion of intransitive choices. A triplet is intransitive if (i) A was chosen over B and B was chosen over C, yet C was chosen over A or (ii) if B was chosen over A and C was chosen over B, yet A was chosen over C.
The proportion of intransitive choices was obtained by dividing the number of intransitive triplets by the total number of triplets. Analytically, it can be shown that the maximum level of intransitivities (those produced by a random responder) is 25% of all triplets. We report the results of simulations that demonstrate the non-linear relationship between number of intransitive choices and response error.
We also obtained, for a random subgroup of the patients with unilateral hippocampal sclerosis (n = 16), a 3D-T1 weighted high-resolution data set (MP-RAGE, voxel size 1 × 1 × 1mm, repetition time 1570ms Experimental Design and Statistical Analysis. Our sample size was constrained by the availability of MTL patients with the appropriate lesion. Still, a power analysis based on effect sizes for healthy participants in the literature suggested we were well powered. Assuming a base proportion of 3% of intransitivities for healthy controls 44 and the same for ETL patients in contrast to twice this amount for the MTL patients, a large (and therefore conservative) estimate (i.e. an effect size of f = 0.4), we would need a total of 60 participants for a power level of 0.95. Our sample with at least 30 subjects per group was well above this.
To perform statistical analysis on our focal behavioral dependent measure, the intransitivity proportions were log transformed to avoid non-normal distributions and unequal variances between the tasks (Bartlett test K 2 (1) = 58, p < 0.001 pre transformation; K 2 (1) = 3.4, p = 0.07 post-transformation). Based on model comparisons (details of compared models can be found under Group difference in intransitivities > Summary Statics in the link provided for the analysis report), a linear mixed model was deemed the appropriate analysis having compared it to simpler models with no random effects. The contrasts of this model were orthogonalized to allow a direct comparison of the ETL group to the healthy controls and of the MTL group to both non-lesion and the ETL control groups together.
Statistical analyses were performed using R (Version 3.3.2; R Project for Statistical Computing, RRID:SCR_001905) for Mac. We use a two-tailed p-value of 0.05 as our criterion for statistical significance. The details of multilevel models are reported in the Results section.
Code availability. All analysis code can be found under https://github.com/zenkavi/TransitivityOpen with a full analysis report including additional figures depicting individual data points under https://zenkavi.github.io/ TransitivityOpen/Transitivity_OpenAnalyses.nb.html Data availability. The data that support the findings of this study are available from the corresponding author upon reasonable request.