Introduction

The framing effect is a decision-making bias in which presenting the same option in different ways (e.g., positive vs. negative) can reverse people’s risk preference (i.e., risk aversion in the positive frame; risk seeking in the negative frame)1,2. After Kahneman and Tversky’s seminal paper introduced risky choice framing in 1981, the classic paradigm to examine the framing effect is using written vignettes that describe hypothetical scenarios and presents a choice between a sure and a risky outcome in positive and negative frames1,2,3,4. For example, in the well-known Asian disease problem, a majority of participants (72%) were risk averse when programs were positively framed based on a description of lives saved; however, preference reversal occurred when the same outcomes were negatively framed based on a description of lives lost (78%)5. Prospect theory accounts for this as a consequence of participants coding positive outcomes as gains and negative outcomes as losses, inducing risk averse and risk seeking preferences respectively5,6.

In recent years, another novel personal framing decision-making task introduced by De Martino et al.’s7 has been widely used in neuroimaging and genetic studies7,8,9,10,11,12. In this gambling task, participants were endowed with an initial amount (e.g., $50). They were then asked to choose between a sure amount and a gamble. The gamble was represented graphically using a pie chart that showed the chances of winning and losing as portions of the pie. The sure option was framed in one of two ways (e.g., keep $20 or lose $30 of the initial endowment). Although the two formulations are obviously equivalent, the sure option was chosen more often when it was framed positively (with the word ‘keep’) than when it was framed negatively (with the word ‘lose’) than the gamble option. This reward-based gambling task is similar to the vignette-based task in the construct.

To date, many studies have examined how individual differences in framing effect can be explained by psychological and biological traits. Specifically, some studies using different vignette-based tasks provide evidence that individual differences in personality are correlated with the framing effect13,14,15; for example, people who are high in need for cognition are less sensitive to the framing effect16,17. Similarly, the framing effect seen in risky task scenarios relates to various “big five” personality traits; for instance, people scoring low on openness have a tendency to show the risk preference reversal18. More importantly, several genetic studies using reward-based gambling task have found that genetic factors have a strong impact on the individual difference in susceptibility to framing. For example, individuals homozygous for the short (s) allele at the serotonin transporter gene-linked polymorphic region (5-HTTLPR) are more sensitive to framing compared with the long (la) group using a reward-based gambling task in the gain domain7,9,11. Recently, another genetic study found that the Met allele of COMT Val158Met polymorphism (rs4680) carriers show a greater framing effect than the Val/Val homozygotes as the former gambled more than the latter in the loss frame. Moreover, the gene-behavior association are mediated by resting-state functional connectivity between orbitofrontal cortex and bilateral amygdala19. Taken together, these studies indicate that there are stable individual traits that are associated with individual differences in susceptibility to framing effects and framing bias seems to be biologically determined. The underlying assumption therefore, is that individual susceptibility to framing manipulation is trait based and can be measured using different experimental tasks.

However, the convergent validity of these tasks has yet to be examined (e.g., reward-based gambling tasks in the gain domain and vignette-based tasks in the loss domain). Convergent validity is agreement between measures of the same construct assessed by different methods20,21. A successful evaluation of convergent validity shows that a test of a concept is highly correlated with other tests designed to measure the same concept. For instance, the scores a test of mathematics skills can be correlated with scores on other tests that are also designed to measure basic mathematical ability. High positive correlations between the test scores would be evidence of convergent validity. If measures from different methods are not correlated with each other, these methods may measure different traits rather than the same trait.

Notably, the two classic paradigms differ in task domains and task contexts (e.g., vignette-based task in the loss domain or gambling task in the gain domain), which may result in different framing effects. In the present study, domain meant the valence of the problem context (e.g., potential gain or potential loss in a final outcome), whereas frames represented how the advantage of the option (positive) or the disadvantage of the option (negative) was portrayed in the description outcome expectation22. Framing effects in hypothetical written vignettes are always observed in a loss problem domain (e.g., Asian disease problem, cancer problem and job loss problem)23, but research has not examined the variety of framing effects in a similar gain problem domain. Many studies on decision-making have shown that people weigh losses more heavily than equivalent gains and thus individuals’ risk-taking attitudes are diverse in the domain of gains and losses24,25,26. A recent study using a modified gambling task also found stable framing effects in the gain and loss domains (i.e., the total amount will be won or lost as a prerequisite in the context), but they appear to be associated with differential brain responses22. It is worth noting that a difference in risk attitude in different domains (i.e., risk aversion in a gain domain and risk seeking in a loss domain) is usually termed a reflection effect (i.e., whether the outcomes are gains or losses)27. Reflection effects involve real gains and losses, while framing effects involve the same choice outcomes as though they are gains or losses27,28,29. From this perspective, another question of interest in this study is whether an individual’s framing effect in a gain domain is correlated with the framing effect in a loss domain.

To address this question, we used multiple measures of framing to test their convergent validity. Specifically, a within-subjects design was used to examine the correlations among the framing effects across different task contexts and domains. That is, we conducted a modified reward-based gambling task22 and vignette-based task2 to test framing effects in the gain and loss domains, respectively. Importantly, the main purpose of this study was to investigate whether framing effects in two classic framing manipulations (i.e., hypothetical written vignettes-based task in the loss domain and real gambling task in the gain domain) were correlated. In addition, we tested the relationship between the framing effect in the gain domain and the loss domain and the correlation between the framing effect in the gambling task and the vignette-based task. This allowed us to infer whether the framing effect was the same across contexts and domains or, alternatively, whether it was modulated by contexts and domains. Based on a vast literature that framing effect was correlated with certain personalities and genes, we predicted that the two classic paradigms designed to measure framing should be highly correlated with each other. On the other hand, since the two paradigms might involve different psychological processes, for example, Fagley and Miller (1997) demonstrated that several variables influence the effect of framing, including features of the decision problem such as problem contexts or who the affected people will be30, we predicted that the two types of tasks would not be correlated with each other.

Materials and Methods

Participants

One hundred and fifty-nine volunteers (80 male; mean age ± SD, 20.90 ± 1.88 years) participated in task 1 and task 2. A set of comprehension questions was presented to check if participants understood the game instructions. All participants answered the questions correctly. Participants received a base payment (¥25, about $4) and extra earnings from the gambling tasks, depending on their performance. The study was approved by the Ethics Committee of the School of Psychology at South China Normal University. The methods were carried out in accordance with the approved guidelines. Written, informed consent was obtained from each participant and all participants were informed of their right to discontinue participation at any time. During the debriefing session, the majority of subjects were found to be unaware of any biasing effects that followed the experiment.

Experimental paradigms

Reward-based gambling task

In task 1, we used a modified gambling task22 to examine how positive and negative frames influenced risk-taking behavior in both gain and loss domains. During an instruction period, participants were familiarized with the task and given ten practice trials.

There were two different blocks, including a gain and loss block. Each block had a positive and negative frame. In the gain block (Fig. 1A), on each trial, participants were first shown a text message (for 2 s) indicating the starting amount of money that they would receive. Participants were informed that they would not be able to retain the whole of this initial amount, but would next have to choose between a sure and a gamble option presented in the context of two different frames. The sure amount was framed as the amount of the initial endowment participants would keep in the positive frame condition or as the amount of the initial endowment participants would lose in the negative frame. In the loss block (Fig. 1B), participants were first shown a message indicating the starting amount of money that they would lose. The sure amount was framed as the amount participants would save in the positive frame or as the amount participants would still lose in the negative frame. The gamble option was represented by a pie chart depicting the frequency of winning or losing in each trial. Participants in the gain domain block were told that they would gain all initial money if they won the gamble and would gain nothing if they lost the gamble. In the loss block, they would lose no money if they won the gamble and would lose the entire amount at stake if they lost the gamble. The expected values of the sure and gamble options were equivalent in each trial and also mathematically equivalent between frames. The sure option was presented on the left side of the screen and the gamble option was presented on the right side of the screen. Participants were given 4 s to respond by pressing the left or right button. They were also told that during the task they would not receive feedback concerning the outcomes of their decisions. The inter-trial interval was 2 seconds.

Figure 1
figure 1

The gambling paradigm.

At the beginning of each trial, a text message indicating the starting amount of money was presented for 2 s. The amount was a gain in the gain domain (A) and a loss in the loss domain (B). The participants were informed that they may or may not receive this initial amount of money, depending on their subsequent choices. Then a sure option was shown on the left of the screen and a gambling option was shown on the right of the screen for 4 s. The sure option was presented in a positive frame or negative frame in both domains, while a gambling option was presented as a pie chart depicting the frequency of winning (i.e., keep all of the initial reward in the gain domain or lose nothing in the loss domain) or losing (i.e., receive nothing in the gain domain or lose the initial amount in the loss domain). Participants were asked to make a decision within 4 s without feedback.

Full size image

Four different starting amounts were used in the task (¥25, ¥50, ¥75 and ¥100) and there were four different percentages of winning or losing in a given trial (20%, 40%, 60% and 80%). Each combination of the starting amounts and the percentages was presented twice, resulting in a total of 64 game trials (32 positive frames, 32 negative frames) in each block. In two blocks, the starting amounts and the percentages of winning or losing were both fully balanced between the frame conditions. The gamble option was identical for both frames and represented by a pie chart depicting the frequency of winning or losing and experimental conditions were randomized within the blocks. We also included 12 catch trials in each block to ensure that participants remained actively engaged in the decision-making task throughout the course of the experiment7. In these catch trials, in both frames, expected outcomes for the sure and gamble options were markedly unbalanced: in half of the trials (‘gamble weighted’) the gamble option was highly preferable (e.g., 95% probability of winning by taking the gamble option versus a sure choice of 50% of the initial amount) and for the other half of trials (‘sure weighted’) the sure option was preferable (e.g., 5% probability of winning by taking the gamble option versus a sure choice of 50% of the initial amount). As in the main experimental trials, the catch trials were also presented in either a positive or a negative frame. Participants were told that their performance in the task determined how much they would be awarded at the end of the task. One trial in the gain domain and one trial in the loss domain were randomly chosen and implemented. Unknown to the participants, if the final outcome was losing, participants received no reward. Thus, losing money meant winning no money in the end.

Vignette-based task

In task 2, we used a modified vignette-based task2 to examine how positive and negative frames influenced risk-taking behavior in both gain and loss domains. Before the task, participants were familiarized with the task and given four practice trials.

At the beginning of each trial, the social problem with initial amounts was shown (for 4 s) and then participants had to choose between a sure and a gamble option presented in the context of two different frames within 6 s. There were 20 trials (16 game trials and 4 catch trials) in each block. In the gain block (Fig. 2A), a novel gold discovery task was used to investigate the framing effect in the gain domain. These were presented in one scenario and two different size contexts (i.e., ‘600/6 tons of gold have been found, but gold mining would mean suffering a certain amount of loss’). Half of the choice problems were framed positively in terms of the expected amount of gold saved while half of the problems were framed negatively in terms of the expected amount of gold lost. So 16 trials included 2 framing conditions × 1 scenario of gold discovery × 2 starting amounts × 4 repetitions. The probability of a successful outcome in the gamble option was 1/3. In the loss block (Fig. 2B), a modified life-death problem was used to investigate the framing effect in the loss domain. These modified life-death problems were presented in two scenarios: disease infection and terrorist kidnapping, with two different group sizes (e.g., ‘600/6 people have been kidnapped by terrorists and all hostages would die if the rescue were not carried out’). Half of the choice problems were framed positively in terms of the expected number of lives saved while half of the problems were framed negatively in terms of the expected number lives lost. Thus 16 trials included 2 framing conditions × 2 scenarios of the life-death problem × 2 starting amounts × 2 repetitions. The probability of the gamble options was constant as in the gain block (i.e., 1/3). In two blocks, the starting amounts and the probabilities of gaining or losing were both fully balanced between the framing conditions and experimental conditions were randomized within the blocks.

Figure 2
figure 2

The vignette-based paradigm.

At the beginning of each trial, the social problem with initial amount was shown for 4 s. The amount was a gain in the gain domain (A) and a loss in the loss domain (B). The participants were informed that the social problem with initial amount may or may not happen, depending on their subsequent choices. Then a sure option was shown on the left of the screen and a gambling option was shown on the right of the screen for 6 s. The sure option was presented in a positive frame or negative frame in both domains, while a gambling option was depicted as the frequency of a successful outcome (i.e., keep all of the initial amount in the gain domain or lose nothing in the loss domain with probability 1/3) or losing (i.e., save nothing in the gain domain or lose the initial amount in the loss domain with probability 2/3). Participants were asked to make a decision within 6 s without feedback.

Full size image

We also included 4 catch trials in each block as in task 1. In these catch trials, in both the gain and loss frames, expected outcomes for the sure and gamble option were markedly unbalanced: half of the trials were ‘gamble weighted’ and the other half of trials were ‘sure weighted’. The sure option was presented on the left side of the screen and the gamble option was presented on the right side of the screen. Participants were told that during the task they would not receive feedback concerning the outcomes of their decisions. All blocks were presented in a fixed order (i.e., gambling task in the gain domain; vignette-based task in the loss domain; gambling task in the loss domain; vignette-based task in the gain domain).

Data analysis

Eight participants’ data were removed from the analysis because their accuracy on catch trials in the two tasks was 3 standard deviations lower than the average, leaving a final sample of n = 151 (77 male) for analysis. The remaining subjects showed high accuracy on catch trials (mean ± SD > 0.85 ± 0.17), indicating their continued engagement with the task throughout the experiment. To examine the framing effect, two-way repeated measures ANOVAs using the domain (gain/loss) and the frame (positive/negative) as independent factors and the frequency of gambling choices as the dependent variable was performed to analyze the data from the gambling task and the vignette-based task separately. In these analyses, the Greenhouse–Geisser correction for non-sphericity was applied where appropriate. Additionally, each framing manipulation had a positive and negative frame that was presented in separate parts of the battery. Thus, using the frequency of choosing the gambling option in the negative frame minus the frequency of choosing the gambling option in the positive frame as an index of the framing effect, we further investigated the relationship among framing effects across different domains and different contexts.

Results

Framing effect in the gambling task

In the gambling task, participants revealed a preference for gambling choices in the negative frame (53.8% ± 1.8%, mean ± SE) compared to the positive frame (42.3% ± 1.8%, F(1,150) = 190.610, p < 0.001, ηp2 = 0.560). There was also a significant difference between the loss (46% ± 1.9%) and gain domain (50.1% ± 2.1%, F(1,150) = 4.932, p = 0.028, ηp2 = 0.032). The interaction effect was not significant (F(1,150) = 1.496, p = 0.223, ηp2 = 0.010), suggesting no difference in the degree of the framing effect in the loss domain and the gain domain. However, paired sample t tests showed that the frequency of the gambling option in the negative frame (52.2% ± 2%) was significantly higher than that in the positive frame (39.9% ± 1.9%, t152 = 10.509, p < 0.001, d = 0.855; Fig. 3D) in the loss domain and similarly, the frequency of the gambling option in the negative frame (55.5% ± 2.1%) was significantly higher than that in the positive frame (44.9% ± 2.1%, t152 = 10.611, p < 0.001, d = 0.864; Fig. 3A) in the gain domain, suggesting that framing effect is significant either in the gain domain or in the loss domain. Across different percentages (20%, 40%, 60%, 80%) and starting amounts (¥25, ¥50, ¥75, ¥100), the frequency of choosing the gambling option in the negative frame was significantly higher than that in the positive frame (p < 0.01), both in the gain domain and the loss domain.

Figure 3
figure 3

Behavioral results.

In the gambling task (A) and vignette-based task (B), the frequency of gambling choices (mean, SE) was significantly higher in the negative frame than in the positive frame, regardless of domain. The framing effect in the gambling task in the loss domain showed no significant correlation with that in the vignette-based task in the gain domain (C). The framing effect in the gambling task and the framing effect in the vignette-based task were significantly correlated (D). The framing effect in the gain domain and the framing effect in the loss domain were significantly correlated (E).

Full size image

Framing effect in the vignette-based task

In the vignette-based task, we found behavior patterns similar to those seen in the gambling task. The frequency of gambling choices was significantly higher in the negative frame (50.6% ± 2.5%) than in the positive frame (40.4% ± 2.4%, F(1,150) = 40.088, p < 0.001, ηp2 = 0.211). There was a significant difference between the gain (39.5% ± 2.8%) and loss domain (51.5% ± 2.6%, F(1,150) = 17.776, p < 0.001, ηp2 = 0.106). The interaction effect was not significant (F(1,150) = 0.287, p = 0.593, ηp2 = 0.002), suggesting that the degree of the framing effect in the loss domain and the gain domain did not differ. However, paired sample t tests showed that the frequency of the gambling option in the negative frame (57% ± 2.9%) was significantly higher than that in the positive frame (46% ± 2.8%, t152 = 5.108, p < 0.001, d = 0.416; Fig. 3B) in the loss domain and similarly, the frequency of the gambling option in the negative frame (44.3% ± 3%) was significantly higher than that in the positive frame (34.7% ± 2.9%, t152 = 4.921, p < 0.001, d = 0.405, Fig. 3E) in the gain domain, suggesting that framing effect is significant either in the gain domain or in the loss domain. Across different starting amounts (600/6), the frequency of choosing the gambling option in the negative frame was significantly higher than that in the positive frame (p < 0.001).

Correlations between the two measurements

First, we found no significant correlation between the framing effect of the vignette-based task in the loss domain and the framing effect of the gambling task in the gain domain (Spearman correlation: rs = 0.056, p = 0.498, n = 151; Fig. 3C). Second, there was a significant correlation between the framing effect of the gambling task and the vignette task (rs = 0.167, p = 0.040, n = 151; Fig. 3D) when combining the gain and loss domains and between the loss domain and the gain domain (rs = 0.235, p = 0.004, n = 151; Fig. 3E) when combining the gambling and vignette tasks. Third, a significant correlation was found between the gain domain and the loss domain in the gambling task (Pearson correlation: r = 0.164, p = 0.045, n = 151) and in the vignette-based task (rs = 0.178, p = 0.029, n = 151), respectively. But we observed no significant correlation between the gambling task and vignette-based task in the loss domain (rs = 0.144, p = 0.078, n = 151) or gain domain (rs = 0.027, p = 0.742, n = 151), respectively.

Discussion

Across two different task contexts and domains, we demonstrate significant framing effects, suggesting that the framing effect can be robustly elicited in either vignette-based or gambling tasks in different domains. Importantly, we found no significant correlation between two commonly used measures–a vignette-based task in the loss domain and a gambling task in the gain domain–suggesting that framing effects may vary across task contexts and the valence of domains. Such low correlation between two classic measures could be used as evidence against their convergent validity. It raises the concern that whether these two classic paradigms both are valid methods to measure individual differences in framing effects during decision making or not.

Our correlation results lead us to conclude that an individual’s framing effect may be modulated by different task contexts and the valence of domains13,31. Although numerous studies have found that individual differences such as gender and thinking styles (e.g., rational and intuitive) are correlated with the framing effect32,33,34,35, there are still some divergent results, with no evidence that need for cognition and gender differences are predictors of risk attitudes15,36,37,38. One possibility is that task contexts and domains may modulate the magnitude of the framing effect. For example, participants were not paid in the vignette-based task and it looked much more abstract than the gambling task. Decision makers in personal-related tasks (e.g., reward-based gambling task) may need more consideration. Previous study has found that the requirement to think more about a choice is believed to prompt additional cognitive processing regarding the options16. Thus, cognitive psychologists, decision scientists and economists should pay much attention to different framing measurements when discussing the framing effect on decisions.

There is now an increasing interest in the knowledge of two physical substrates responsible for behavior: the brain and the genome39,40. Recently, a meta-analysis of neuroimaging studies of risky choice framing effects has found that three brain regions are the key neural correlates of framing effects in life-death, monetary and other types of problems, such as the right inferior frontal gyrus (IFG), the left anterior cingulate cortex (ACC) and the left amygdala41. Intriguingly, these investigations also provided evidence that framing effect in two common measures (i.e., a vignette-based task and a gambling task) involves differential activation of separate neural substrates. While left amygdala and ACC played an important role in gambling tasks in the gain domain7,8,9,10, the right dorsolateral prefrontal cortex, parietal cortex and right IFG had influence on vignette-based tasks in the loss domain2,42. Although it is unclear whether these regions are specifically related with the framing effect in different tasks, the results seem to suggest that different psychological processes are engaged in different framing tasks.

In addition, as mentioned previously, existing research has shown that the genetic factors influence the framing effect9,11,19, which suggests that the framing effect for an individual is stable as a trait. But one limitation of these studies is the use of one task in one domain to test small samples of individuals’ risk attitudes, which may not provide enough evidence to conclude that individual differences can predict people’s framing effect. The low convergent validity of two classic framing measures is a potential limitation to the understanding of framing effect as a trait. Thus, the current findings argue that individual differences research using behavioral tasks should assess the convergent validity of different behavioral tasks. In this way, it may provide more evidence that people’s susceptibility to the framing of choice outcomes is a trait and has genetic basis, since convergent validity means the instrument is related to other measures of the same construct and related constructs43. Moreover, recent genetic study using a delayed discounting task found that oxytocin and estrogen receptor polymorphisms temper accelerated cellular aging in young females who are more inclined to make impatient choices. However, the authors admitted that it is unable to conclusively disentangle two potential underlying mechanisms of their findings, namely, impatience leads to the interaction effect between leukocyte telomere length and genetic sensitivity or, alternatively, such biological factors lead to impatience (state-dependent model)44. Thus, in order to effectively explain the relationship between the genes and individual susceptibility to a stable decision bias, further research is needed to test whether the genetic factor is associated with such decision bias under different tasks.

It is surprising that participants’ framing effects in the gain domain were accordant with those in the loss domain when combining two tasks. There was a similar positive correlation between the gain and loss domains in the gambling task and that in the vignette-based task as well. These findings may be in line with a previous study showing that what really matters in risk taking behavior is the different descriptions of outcome, not the valence of task domains45. The correlation of framing effects between the domain of gains and losses may further support the threat and opportunity hypothesis—that is, if decision makers are able to translate an objective threat (i.e., a description of loss in a loss problem domain) into an opportunity (i.e., an expectation of positive outcome), then it could be expected that they could translate an objective opportunity (i.e., a description of gain in a gain problem domain) into a threat (i.e., an expectation of negative outcome), because individuals could shift the reference point based on the valence of domains46. In addition, regardless of the valence of domains, the framing effect in the gambling task was correlated with the effect in the vignette-based task, suggesting that people’s risk-taking attitude in different tasks may develop in the same way. However, previous studies have provided evidence that decision makers motivated by incentives are more sensitive to framing effects3,5, such as in the gambling task. Notably, we found no correlation between the two tasks in the gain domain or between the two tasks in the loss domain, meaning that task context and domain may have an interactive effect on the magnitude of the framing effect. It is not clear how participants would encode the context and domain variables in the two different tasks. Kühberger et al.’s29 meta-analysis introduced reflection versus framing, as well as context, as factors. Inspection of their Table 3 is informative for the present paper and shows what also is found here: framing is more relevant to risk attitude than reflection, while the domains and contexts also have an influence, but mainly as nuisance variables29. In sum, our study suggests that individual differences in the framing effect are task-dependent (e.g., task contexts and domains).

There are several limitations that are worth mentioning. First, our research relies on within-subjects designs to test for framing effects. It is not clear what the results would have looked like in a more customary one-shot framing study. Using both positive and negative frames for each participant could promote hypothesis guessing of strategies or could lead to attenuation of framing effects. However, consistent with previous studies using repeated trials, we nevertheless found framing effects. Using repeated trials might allow researchers to have more reliable measures of the framing effect. Besides, most subjects were found to be unaware of any biasing effects that followed the experiment during the debriefing session. Second, one of our key findings is a lack of correlation between the vignette-based task in the loss domain and the gambling task in the gain domain. The two classic tasks have been used to measure individual differences in the framing effect. The underlying mechanisms explaining why the two measures were not correlated with each other remain unclear.

The current findings have a wide range of implications for our understanding of how a person’s framing effect might interact with the valence of domains and task context. If the framing effect is task-dependent, future studies should be cautious about drawing conclusions based on correlations involving individual differences in framing. In order to better examine the framing effect, future research should further examine the underlying mechanisms mediating the framing effect in gain and loss domains across different task contexts.

Additional Information

How to cite this article: Zhen, S. and Yu, R. All framing effects are not created equal: Low convergent validity between two classic measurements of framing. Sci. Rep. 6, 30071; doi: 10.1038/srep30071 (2016).