Abstract
Human information processing is not always rational but influenced by prior attitudes, a phenomenon commonly known as motivated reasoning. We conducted two studies (N1 = 556, N2 = 1198; UK samples) investigating motivated reasoning in the context of climate change with a focus on individual differences as potential moderating factors. While previous research investigated motivated reasoning regarding the debate whether climate change is anthropogenic, we focused on current discourses about the effectiveness of different countermeasures. To this end, participants evaluated fictitious scientific data on the effectiveness of regulations to reduce CO2 emissions. In both studies, participants exhibited motivated reasoning as indicated by the observation that prior attitudes about CO2 reduction policies predicted evaluation of the scientific data. The degree of motivated reasoning was not related to individual difference variables, namely the ability to understand and reason with numbers (Numeracy), the willingness to show this ability (Need for Cognition), and the tendency to maximize one’s individual utility (Dark Factor of Personality). However, numeracy was associated with a less biased interpretation of the presented information. Our research demonstrates that motivated reasoning is a general phenomenon, and points to numerical training as one way to improve reasoning.
Similar content being viewed by others
Introduction
While there is wide scientific consensus on the anthropogenic nature of climate change1 and the need to act2, there is a vivid debate about the best and most effective ways of addressing the current global crisis. Crucially, this debate is not only relevant to scientists, activists, and politicians, but also to the public. In order to adjust their behavior and to demand political action, individuals need to understand the existing scientific evidence and the multilayered consequences of different mitigation policies3,4. However, the positions that individuals hold regarding these issues are overshadowed by their values and worldviews5, potentially undermining the rational discourse. As it seems, not everyone is willing to follow “the unforced force of the better argument”6, p. 305. In psychology, this is commonly known as motivated reasoning. Against this background, the present studies were aimed at investigating motivated reasoning in the context of climate change and the influence that different dimensions of individual differences have on the degree of biased reasoning. In particular, we tested whether motivated reasoning is influenced by the ability to arrive at a correct interpretation of the presented numerical information (Numeracy), the willingness to show this ability (Need for Cognition), and the tendency to maximize one’s individual utility (Dark Factor of Personality).
Motivated reasoning
Motivated reasoning denotes the phenomenon that human information processing is shaped by the individual’s motives, goals, and attitudes instead of being rational and objective7,8. That is, “people are more likely to arrive at those conclusions that they want to arrive at”8, p. 495. When motivated reasoning manifests itself in the context of evaluating scientific evidence on a given topic, it is referred to as motivated science reception or motivated rejection of science9,10,11. Over the last decades, researchers have investigated various aspects of motivated reasoning. For instance, it has been shown that individuals tend to seek out information that confirms their prior attitudes12,13. In addition, attitudes drive how individuals deal with new information that they are being confronted with: While information that is congruent with one’s attitudes is readily being accepted (motivated acceptance), information contradicting one’s attitudes is evaluated more critically14,15 and may trigger the active search for further congruent information (motivated rejection)16. In many situations, it is the combination of motivated acceptance and motivated rejection that can be considered to lie at the heart of motivated reasoning. Motivated reasoning has been demonstrated on a wide range of topics, such as capital punishment17, pacifism18, and nanotechnology19.
Importantly, there are several studies investigating motivated reasoning in the context of climate change20,21,22,23,24,25,26,27,28,29. All of these studies have found evidence for motivated reasoning and have thus contributed to our understanding of the current debates on climate change and environmental policies. Nevertheless, there are certain limitations to be noted. To begin with, all above-mentioned studies except from one26 were based on US samples, limiting the cross-cultural validity of the results. Importantly, the strong partisan divide on climate change beliefs, which can be observed in the US30, does not equally translate to other countries around the globe. Arguably, the lack of sample diversity is particularly problematic as it is linked to two further limitations. First, many of the above-mentioned studies use political orientation as a proxy for climate change attitudes instead of directly measuring these attitudes21,23,26,27,29, which seems reasonable in a situation in which there is a strong polarization along party lines in a particular country. However, political orientation is an extremely broad category, making it difficult to determine what kind of goals or motives actually drive motivated reasoning in these situations13,30,31. Second, several of the above-mentioned studies were restricted to investigating motivated reasoning regarding the question whether climate change is anthropogenic or not20,21,22,28. Again, this seems reasonable given the relatively high percentage of individuals in the US denying the anthropogenicity of climate change. Taken together, it remains unclear whether the current state of evidence on the mechanisms and moderators of motivated reasoning is limited to highly identity-relevant categories such as political orientation and to heated debates about the anthropogenicity of climate change. At the same time, the current debates in the public and political realm in many countries go beyond the general question about anthropogenicity and involve discussing the pros and cons of specific policies. Therefore, it is important to investigate whether and under what conditions motivated reasoning occurs with respect to these issues. Finally, much of the support for motivated reasoning in the context of climate change is derived from correlational evidence (e.g., between political orientation and policy support or risk perception21,24,25,27) and is consequently not based on performance on an actual task with objectively correct or wrong responses. The latter has the advantage that it allows disentangling motivated reasoning and unbiased reasoning26.
Against this background, we conducted two studies on motivated reasoning in the context of climate change that account for these limitations. Hence, we recruited participants in a non-US English-speaking country (United Kingdom) in order to ensure the cross-cultural validity of the existing evidence. In addition, and building on a similar study design used for investigating motivated reasoning in the context of the COVID-19 pandemic32, we asked participants about their attitude towards stricter regulations to reduce CO2 emissions and presented them with the task to evaluate two fictitious studies, one demonstrating that such regulations lead to a decrease of CO2 emissions and one demonstrating that stricter regulations lead to an increase of CO2 emissions. The fact that the outcome of the fictitious studies was manipulated within-subjects distinguishes our study from previous attempts to study motivated reasoning in the context of climate change using a performance-related measure26,29. Importantly, manipulating the study outcome within-subjects enabled us to calculate an individual bias score and to relate this individual bias score to other factors potentially influencing the degree of motivated reasoning (for details, see the method section). More specifically, we calculated a directional bias score, indicating both the direction and the degree of biased reasoning. A correlation between the directional bias score and attitude indicates that individuals engage in motivated reasoning. Based on the extant literature describing motivated reasoning in various contexts, we formulated the following hypothesis regarding the directional bias score:
H1: People show motivated reasoning in the context of climate change. More specifically, people evaluate data on the effectiveness of stricter regulations to reduce C02 emissions in line with their prior attitude towards such regulations.
Furthermore, we also calculated an absolute bias score, indicating the degree of bias irrespective of the direction. The absolute bias score reflects the extent to which participants give the same response to both studies. A high absolute bias score indicates that participants do not recognize that the studies in fact show opposite patterns (i.e., that one study shows an increase while the other study shows a decrease of C02 emissions). A low absolute bias score indicates that participants are able to differentiate between the two fictitious studies and understand that the two studies show different outcomes. The absolute bias score can be seen as an indicator of biased and incorrect reasoning independently from the participants’ attitude. To be clear, the absolute bias score does not reflect motivated reasoning (in the sense of attitude-consistent reasoning). Analyzing the absolute bias score is interesting because it should be related to participants’ ability to understand and to reason with numbers (for more details see below)32.
The possibility to calculate a directional bias score as well as an absolute bias score within a single paradigm constitutes a crucial advantage of our method, as it enables us to investigate biased reasoning in more detail. More specifically, we can investigate the degree of attitude-consistent reasoning (i.e., the correlation of attitude and the directional bias score) and potential factors moderating this relationship (see below). At the same time, we can also investigate the degree of biased reasoning independently of the direction (i.e., the absolute bias score) and potential moderating factors (see below). While the former sheds light on motivated reasoning, the latter allows a better understanding of the general extent of biased or incorrect reasoning.
Potential factors influencing motivated reasoning
Although motivated reasoning is generally considered to be a universal phenomenon, it is reasonable to assume that the degree of motivated reasoning depends on a wide range of boundary conditions33. In the context of the present studies, we sought to investigate three dimensions of individual differences that could potentially be related to the degree of motivated reasoning (for an overview, see Fig. 1). First, in order to arrive at an unbiased interpretation of new information, individuals need to have the necessary abilities to do so. As the fictitious studies that participants had to evaluate required them to reason with numbers, the ability component was operationalized in terms of numeracy. Second, even if individuals have the necessary abilities to arrive at an unbiased conclusion, they must be willing to use these abilities. Arguably, this willingness might depend on their thinking style as expressed through their Need for Cognition. Third, independently from their abilities and their willingness to use these abilities, the degree to which individuals engage in motivated reasoning might also depend on the general tendency to maximize one’s individual utility irrespective of the existing evidence, which is captured by the Dark Factor of Personality. These three dimensions of individual differences and the related research questions are described in detail below.
Study Overview. The present studies investigated the relationship between climate change attitudes and two forms of biased reasoning, as expressed in the directional bias and the absolute bias. In addition, it was tested whether this relationship as well as the degree of biased reasoning are influenced by three dimensions of individual differences: the ability to arrive at a correct interpretation of the presented information (Numeracy), the willingness to show this ability in a given situation (Need for Cognition), and the stable tendency to maximize one’s individual utility (Dark Factor of Personality).
Numeracy
Numeracy describes the ability to understand and to reason with numbers. Research has demonstrated that higher numeracy is associated with making better and more informed decisions34,35. For instance, individuals with higher numeracy have better financial literacy36,37,38 and better physical and mental health39,40,41. In the case of motivated reasoning, however, the picture is more complicated: On the one hand, it has been suggested that the degree of attitude-consistent judgment is more pronounced among highly numerate individuals, arguably because these individuals are better at finding rationalizations for their prior attitudes21,25,26,42,43,44. On the other hand, there are also studies which found no clear effects of numeracy on the degree of attitude-consistent judgment45,46 or even evidence in favor of reduced attitude-consistent judgment among highly numerate individuals47,48, providing support for the idea that numeracy can be an antidote to inaccurate and biased reasoning. The assumed reason behind such findings is that numeracy should enable individuals to consider the available evidence instead of simply falling for processes of motivated acceptance and motivated rejection. Note also that a study using the same procedure as the present investigation32 found no effect of numeracy on the degree of attitude-consistent judgment but observed that higher numeracy was associated with a reduced absolute bias. In other words, this study suggests that it may be worth investigating different kinds of biased reasoning in order to paint a nuanced picture of the role of numeracy. Given the two opposite theoretical perspectives and conflicting evidence, we formulated the following research question:
Research Question 1: Is the individual’s ability to understand and to reason with numbers (i.e., numeracy) related to (a) the degree of attitude-consistent judgment (i.e., the relation between attitude and directional bias) and (b) to the degree of absolute bias?
Need for Cognition
Need for Cognition (NFC) is defined as “the tendency for an individual to engage in and enjoy thinking”49. Individuals with a high NFC prefer complex and intellectually challenging to simple problems, while individuals with a low NFC tend to avoid tasks that require deliberation and mental effort50,51. In line with this, it has been demonstrated that individuals with a high NFC show a higher interest in science52 and are more likely to be driven by accuracy goals compared to individuals with a low NFC53, suggesting that motivated reasoning might be reduced among individuals with a high NFC, arguably because these individuals are motivated to evaluate the available evidence thoroughly instead of simply accepting or rejecting it based on their prior attitudes. At the same time, evidence also suggests that individuals with a high NFC tend to have stronger and less ambivalent attitudes that are more resistant to change54,55, arguably precisely because of their tendency to base their opinions on an effortful analysis of the available evidence51, which supports the idea that a high NFC might be associated with more pronounced motivated reasoning under certain circumstances. In the context of climate change, studies suggest a link between a higher NFC and a higher probability to accept the anthropogenicity of climate change56 as well as more pronounced pro-environmental attitudes and goals57. However, a recent study that directly tested the influence of NFC on the degree of motivated reasoning in the context of climate change did not find any significant effects20. Given the ambiguous empirical evidence and given the theoretical plausibility of different outcomes, we formulated the following research question:
Research Question 2: Is the individual’s Need for Cognition related to (a) the degree of attitude-consistent judgment (i.e., the relation between attitude and directional bias) and (b) to the degree of absolute bias?
Dark Factor of Personality
The Dark Factor of Personality (D) describes “the general tendency to maximize one’s individual utility – disregarding, accepting, or malevolently provoking disutility for others – accompanied by beliefs that serve as justifications”58. D is conceptualized to capture the common variance of dark personality traits such as Machiavellianism, egoism, and psychopathy58,59. It has been observed that individuals high in D hold problematic epistemic beliefs in the sense that they have a strong conviction that truth is political and ultimately a matter of power, that they believe that they can trust their intuition when evaluating facts, and that they have a low need to ensure that their beliefs align with the available evidence; in turn, these epistemic beliefs are associated with embracing conspiracy theories, a reduced ability to discern fake news from real news, and a lower probability of following public health recommendations in the context of the COVID-19 pandemic60,61,62. To our knowledge, D has not been investigated in the context of motivated reasoning and/or climate change. Given the above-mentioned evidence and given that scoring high on the dark triad (i.e., a related albeit distinct personality construct) is associated with a weaker conviction that climate change is anthropogenic63 as well as reduced pro-environmental behavioral intentions64, it nevertheless seems plausible to assume that D might play a role in the context of the present studies. In particular, one might assume that individuals high in D willingly accept information that suits their agenda and reject information that does not. As there is no direct evidence for this hypothesis yet, however, we formulated the following open research question:
Research Question 3: Is the individual’s Dark Factor of Personality score related to (a) the degree of attitude-consistent judgment (i.e., the relation between attitude and directional bias) and (b) to the degree of absolute bias?
Study 1
Method
Ethics statement
In Germany, where the principal investigators are located, ethics approval is deemed unnecessary according to national regulations for studies as the present ones65. Nevertheless, both Study 1 and Study 2 were conducted in full accordance with the Declaration of Helsinki, as well as the ethical guidelines provided by the German Psychological Society (DGPs)66. Of course, this also included obtaining informed consent from all participants before they were able to take part in this study.
Participants
As correlations between two variables can be assumed to stabilize at about 250 participants67 and as it has been recommended to at least double sample size when investigating interactions with an additional factor68, we aimed for a final sample size in Study 1 of at least 500 participants. Participants were recruited via Prolific (www.prolific.com). All data exclusions, manipulations, and measures are reported. The study was programmed on SoSciSurvey69.
Session 1: prescreening
In Session 1, participants were prescreened for their attitude towards stricter regulations to reduce CO2 emissions. As we were not sure how attitude would be distributed among participants and as we wanted to make sure to be able to invite a reasonable number of participants on each point of the attitude spectrum for the main study, we decided to collect data from at least 900 participants. In total, 902 participants completed the prescreening. Participation was only possible for Prolific users with an approval rate of at least 98% who were fluent in English and currently living in the United Kingdom. Participants were compensated with £1.70. Session 1 lasted about ten minutes. Data collection took place from August 11 to August 12, 2022.
Session 2: main study
To account for exclusions and potential dropout between sessions, we decided to invite 721 participants for Session 2. An equal distribution of participants across the seven scale points of the regulation attitude item would have implied inviting 103 participants per scale point for Session 2. Whenever we had 103 or fewer participants on one scale point of the regulation attitude item, all of these participants were invited for Session 2; whenever we had more than 103 participants on one of the seven points of the scale, we randomly drew a subset of participants (for more information about the participants who were selected for participation in Session 2 see the Supplemental Material S2 and S3).
From the total of 721 participants that were invited to participate in Session 2, 673 participants completed the study. Participants who did not pass all attention checks (N = 98) were excluded from the analysis. We also excluded participants who had noticeable short (i.e., less than one third of the median response time; N = 14) or long (i.e., more than six times the median response time; N = 5) response times, indicating careless responding. Thus, the final sample consisted of 556 participants (Mage = 43.40, SD = 14.05, 18–80 years; 226 male, 323 female, 3 other, 4 prefer not to say; see Supplemental Material S4–S6 for information about political orientation, ethnicity, and education). Participants were compensated with £1.05 upon study completion. Session 2 lasted about six minutes. Data collection took place from August 16 to August 18, 2022.
Materials
Attitude towards stricter regulations to reduce CO2 emissions
The attitude towards regulations to reduce CO2 emissions was measured using one item created for the purpose of the study (“Governments should enforce stricter regulations to decrease CO2 emissions, even if these regulations restrict the freedom of the individual”). We additionally assessed three facets of attitude strength (all items were measured on 7-point Likert-type scales ranging from 1 to 7)70: attitude certainty (“How certain are you of your views about regulations to reduce CO2 emissions?” “How sure are you that your opinion about regulations to reduce CO2 emissions is right?”), attitude importance (“To you personally, how important is the issue of regulations to reduce CO2 emissions?” “Personally, how much do you care about the issue of regulations to reduce CO2 emissions?”), and ego-involvement (“How central is your attitude toward regulations to reduce CO2 emissions to your self-image?” “How representative of your values is your attitude toward regulations to reduce CO2 emissions?”). The internal consistency of the 6-item attitude strength scale was good, α = 0.85. As they were not part of the main research questions and yielded no significant results, the analyses regarding attitude strength are reported in the Supplemental Material S8 (Study 1) and S19 (Study 2).
Studies on the effectiveness of stricter regulations to reduce CO2 emissions
Participants evaluated two fictitious studies on the effectiveness of stricter regulations to reduce CO2 emissions (see Fig. 2): one study demonstrating that stricter regulations to reduce CO2 emissions are effective (pro-regulation study) and one study demonstrating that stricter regulations to reduce CO2 emissions are counterproductive (anti-regulation study). The results of the studies were presented in two-by-two contingency tables adapted from a previous study32. As the study outcome was manipulated within-subjects, two different sets of numbers were used. These are designed in a way so that the Binomial Effect Size Display (BESD)71 is equivalent across conditions (BESD = 8.8% in the pro-regulation studies, BESD = -8.8% in the anti-regulation studies)72. Furthermore, all contingency tables show that there were more cities/districts in which the CO2 emissions increased (77%) than cities/districts in which the CO2 emissions decreased (23%). Thus, the overall trend was held constant across the two sets of numbers.
Study Conditions. Each participant evaluated the results of two fictitious studies: one study demonstrating that stricter regulations to reduce CO2 emissions are effective (pro-regulation study) and one study demonstrating that stricter regulations to reduce CO2 emissions are counterproductive (anti-regulation study). Which of the two sets of numbers was used for the pro-regulation study and for the anti-regulation study was counterbalanced across participants (Stimulus Set A vs. Stimulus Set B). Whether participants first evaluated the pro-regulation study or the anti-regulation study was counterbalanced across participants.
Most importantly, the numbers in the contingency tables were designed in a way so that superficial processing (i.e., comparing absolute numbers between two cells instead of ratios) easily leads to the wrong interpretation43. Take the pro-regulation study in Stimulus Set A, for instance: When looking at the absolute numbers in the upper left and lower left cell only, there are more cities/districts with stricter regulations to reduce CO2 emissions (223) than cities/districts without stricter regulations to reduce CO2 emissions (107) that show an increase in CO2 emissions, which can lead to the wrong conclusion that stricter regulations are counterproductive. In fact, however, there is an increase in CO2 emissions in 223 out of 298 schools with stricter regulations (74.8%) compared to an increase in CO2 emissions in 107 out of 128 cities/districts without stricter regulations (83.6%), indicating that stricter regulations are an overall successful intervention. The logic behind the numbers in the anti-regulation study is exactly the same, leading to the opposite conclusion (for details, see the Supplemental Material S11).
Participants were asked to indicate what can be concluded from the presented studies on a 6-point Likert scale (ranging from 1 = “The package of stricter regulations enforced in the cities/districts leads to an increase in CO2 emissions” to 6 = “The package of stricter regulations enforced in the cities/districts leads to a decrease in CO2 emissions”). Note that participants may legitimately differ regarding their estimation of the strength of the evidence provided by the studies (i.e., response options 4–6 are correct in the case of the pro-regulation study and response options 1–3 are correct in the case of the anti-regulation study). As the ratios in the pro-regulation and the anti-regulation study are exactly reversed and the BESD is equivalent (see above), however, an unbiased observer should show a symmetric response pattern (e.g., tick response option “4” when evaluating the pro-regulation study and option “3” when evaluating the anti-regulation study). Differently put, an asymmetric response pattern indicates a bias (see Fig. 3).
Bias Scores. The figure depicts three potential response patterns of an unbiased observer, an observer with a pro-regulation bias, and an observer with an anti-regulation bias. The anti-regulation/pro-regulation bias score or directional bias score is the difference between the response to the pro-regulation study and the reversely coded response to the anti-regulation study (unbiased observer: 5–5 = 0; observer with pro-regulation bias: 6–4 = 2; observer with anti-regulation bias: 4–6 = –2). The directional bias score indicates both the direction and the degree of the bias. By taking the absolute value of the directional bias score, one can calculate an absolute bias score, which indicates the degree of the bias irrespective of the direction (unbiased observer: 0, observer with pro-mask bias: 2, observer with anti-mask bias: 2).
Based on this assumption, two dependent variables can be calculated. The anti-regulation/pro-regulation bias score or directional bias score is the difference between the response to the pro-regulation study and the reversely coded response to the anti-regulation study. A positive value indicates a pro-regulation bias, while a negative value indicates an anti-regulation bias. More extreme values indicate a more extreme bias. That is, the directional bias score indicates both the direction and the degree of the bias. As the directional bias score combines the participants’ response to attitude-consistent and attitude-inconsistent information, it mirrors the joint effects of motivated acceptance and motivated rejection. In case participants engage in motivated reasoning in the context of climate change (i.e., show attitude-consistent judgment), individuals who support stricter regulations to reduce CO2 emissions should on average show a positive directional bias while individuals who are against stricter regulations should on average show a negative directional bias. Note that a zero directional bias score does not necessarily indicate a correct evaluation of the studies, as this score can also result from a symmetric, but incorrect evaluation of the studies. By taking the absolute value of the directional bias score, one can calculate an absolute bias score, which indicates the degree of the bias irrespective of the direction. That is, the absolute bias score mirrors the degree of asymmetry when evaluating the two fictitious studies. In other words, it mirrors whether participants are able to differentiate between the two fictitious studies and whether they understand that the two studies show different outcomes. In case participants supporting and participants opposing stricter regulations to reduce CO2 emissions are equally biased, however, there will be no association between attitude and absolute bias. Apart from that, one can calculate the study evaluation accuracy, i.e., whether a study is evaluated correctly (1) or not (0).
Numeracy
Numeracy was measured using an 8-item scale73. The scale includes items such as “In the Big Bucks Lottery, the chances of winning a $10.00 prize are 1%. What is your best guess about how many people would win a $10.00 prize if 1,000 people each buy a single ticket from Big Bucks?”. The numeracy score for each participant equals the number of correct responses, ranging from zero to eight.
Need for Cognition
NFC was measured using an 18-item scale74. The scale includes items such as “I would prefer complex to simple problems” or “I find satisfaction in deliberating hard and for long hours”. Items were rated on a 9-point Likert scale ranging from 1 (very strong disagreement) to 9 (very strong agreement). The internal consistency of the 18-item scale was excellent, α = 0.94.
Dark Factor of Personality
D was measured using an 16-item scale75. The scale includes items such as “My own pleasure is all that matters” or “Payback needs to be quick and nasty”. Items were rated on a 5-point Likert scale ranging from 1 (strongly disagree) to 5 (strongly agree). The internal consistency of the 16-item scale was good, α = 0.85.
Procedure
Session 1: prescreening
After providing informed consent, participants answered demographic questions regarding age, gender, political orientation, ethnicity, and education. Next, participants were asked about their attitude towards stricter regulations to reduce CO2 emissions, followed by the scales measuring the NFC, D, and numeracy. Session 1 included one attention check item (for details, see Supplemental Material S1). After completing the prescreening, participants were informed about the purpose of the study and informed that they might receive an invitation to take part in Session 2 in a couple of days.
Session 2: main study
After providing informed consent, participants were told that they would be presented with the results of two scientific studies investigating the effectiveness of stricter regulations to decrease CO2 emissions (for detailed instructions, see online materials). Moreover, participants were told that the studies were conducted in two European countries (although the names of the two countries would not be disclosed for standardization purposes) – and that it will be their task to evaluate whether the studies support the conclusion that stricter regulations are an overall effective intervention or not. Whether participants first evaluated the pro-regulation study or the anti-regulation study was counterbalanced across participants. Which of the two sets of numbers was used for the pro-regulation study and for the anti-regulation study was counterbalanced across participants. The two counterbalancing factors did not influence the results, that is, there were no order effects, neither in Study 1 nor in Study 2 (see the Supplemental Material S7 and S18). After evaluating the two studies, participants responded to a multiple-choice attention check item (for details, see Supplemental Material S1). Participants were debriefed after completing Session 2. More specifically, participants were informed that the two studies were fictitious and that the existing scientific evidence indicates that stricter regulations can be considered a powerful means to reduce CO2 emissions.
Results
Study evaluation
Overall, study evaluation accuracy rates were rather low, confirming the validity of our stimulus material as posing difficult statistical problems: 50.00% of the participants evaluated the pro-regulation study correctly (i.e., selected 4, 5, or 6 on the 6-point Likert scale), and 58.09% of the participants evaluated the anti-regulation study correctly (i.e., selected 1, 2, or 3). That is, the anti-regulation study was significantly more likely to be evaluated correctly than the pro-regulation study, t(555) = 2.77, p = 0.006, d = 0.12.
There was no significant difference between the time participants spent on evaluating the results of the pro-regulation study (Mpro = 117.08 s, SD = 121.84, Mdn. = 93 s) and the results of the anti-regulation study (Manti = 110.88 s, SD = 81.04, Mdn. = 90 s), t(555) = 1.11, p = 0.266, d = 0.05. The fact that participants spent clearly more than one minute on average on evaluating the results indicates that participants engaged with the material and took the task seriously.
Biased reasoning and zero-order correlations
Attitude towards stricter regulations to reduce CO2 emissions was significantly correlated with the directional bias, indicating that participants showed attitude-consistent judgment (i.e., engaged in motivated reasoning) in the context of climate change, r = 0.09, p = 0.037. Zero-order correlations between variables are displayed in Table 1.
Potential factors related to biased reasoning
To investigate our research questions, we ran separate hierarchical regression analyses for each individual difference variable (i.e., numeracy, NFC, D). Note that we decided to run separate hierarchical regression analyses because we were mainly interested in the individual and separate contribution of each of the three factors. The predictors in each regression analysis were the attitude towards stricter regulations to reduce CO2 emissions (z-standardized, Step 1), the respective individual difference variable (z-standardized, Step 2), and the product term of the two variables (entered in Step 3). To check whether the results for each individual difference variable remain the same when the influence of the other predictors is controlled, we ran further hierarchical regression analyses in which we entered the other two individual difference variables as predictors in the first step. This did not change the results in meaningful ways (see Supplemental Material S9).
We ran separate regression analyses for the directional bias and the absolute bias as the criterion. The regression analyses using directional bias as the criterion help to determine whether the degree of attitude-consistent judgment (i.e., motivated reasoning) is moderated by the above-mentioned factors. In these regression analyses, the key question is whether the relationship between attitude and directional bias is moderated by the additional factor under investigation (i.e., whether the interaction effect is significant). The regression analyses using absolute bias as the criterion help to determine whether the degree of bias (irrespective of the direction) is predicted by the above-mentioned factors. In these regression analyses, the key question is whether the additional factor under investigation has an effect on the degree of absolute bias (i.e., whether the main effect is significant). That is, although we apply the same analytic strategy for both criteria in order to make the results easily comparable, the effect of interest differs depending on the used criterion. Tables for all regression analyses can be found in the Supplemental Material S9. Finally, we ran logistic regression analyses on study evaluation accuracy to investigate whether the likelihood of correct study interpretation is moderated by the above-mentioned factors. The results are presented in the Supplemental Material S10.
Directional bias
Attitude positively predicted the directional bias score, B = 0.18, SEB = 0.09, p = 0.037, R2 = 0.01. Numeracy was negatively related to the directional bias score, B = − 0.21, SEB = 0.09, p = 0.018, ΔR2 = 0.01. The interaction between attitude and numeracy was not significant, B = 0.15, SEB = 0.08, p = 0.073, ∆R2 = 0.006. In the next regression analysis, Need for Cognition was entered in the second step, yielding no significant association, B = -0.05, SEB = 0.09, p = 0.585, ΔR2 = 0.001. The interaction between attitude and Need for Cognition did not explain additional variance, B = 0.04, SEB = 0.08, p = 0.612, ΔR2 < 0.001. A similar pattern of results was found for the Dark Factor. It neither predicted the bias score, B = − 0.08, SEB = 0.09, p = 0.394, ΔR2 = 0.001, nor was an interaction present, B = 0.02, SEB = 0.09, p = 0.777, ΔR2 < 0.001. To check for higher-order interactions involving attitude as one of the predictors, we ran an additional regression analysis with all variables and the product term of these variables as predictors. No three-way interaction was observed that involved attitude as one of the predictors, all ps > 0.249 (for details, see the Supplemental Material S9).
Absolute bias
Attitude towards stricter regulations was unrelated to the absolute bias score, B = − 0.03, SEB = 0.06, p = 0.614, R2 < 0.001. Numeracy, entered as an additional predictor in the second step, was negatively related to the absolute bias score, B = − 0.17, SEB = 0.06, p = 0.007, ΔR2 = 0.01. The interaction between attitude and numeracy was not significant, B = − 0.06, SEB = 0.06, p = 0.313, ∆R2 = 0.002. In our second analysis, Need for Cognition did not predict the absolute bias score, B = − 0.03, SEB = 0.06, p = 0.653, R2 < 0.001, but the interaction between attitude and Need for Cognition was significant, B = − 0.12, SEB = 0.06, p = 0.043, ∆R2 = 0.007. The Dark Factor was unrelated to the absolute bias score, B = − 0.02, SEB = 0.06, p = 0.775, ΔR2 < 0.001, and no interaction between attitude and the Dark Factor was observed, B = 0.08, SEB = 0.06, p = 0.212, ∆R2 = 0.003. To check for higher-order interactions involving attitude as one of the predictors, we ran an additional regression analysis with all variables and the product term of these variables as predictors. We found no indication for the presence of three-way interactions that involved attitudes, all ps > 0.546 (for details, see the Supplemental Material S9).
Bayes regressions
We completed our analyses with Bayes linear regression analyses as they allow for an interpretation of the degree of evidence favoring the null hypothesis76,77,78. Of particular interest regarding the directional bias were comparisons between the interactions between attitude and the focal individual difference variable versus the null model. The results (see Fig. 4) indicate moderate support for the null hypothesis in models with the Need for Cognition or the Dark Factor. Results were inconclusive for Numeracy. Of particular interest regarding the absolute bias were comparisons between the main effects of the focal individual difference variable and the null model. These analyses indicate moderate support for the null hypothesis in models with the Need for Cognition or the Dark Factor, as well as moderate support for the alternative hypothesis that numeracy is negatively linked to the absolute bias.
Bayes Linear Regression Results. Results of Bayes Regression Analyses with JASP default priors. For results on the directional bias as the criterion, BF10 for the model with the interaction between attitude and the focal individual difference variable versus the null model that includes the two main effects is plotted. For results on the absolute bias as the criterion, BF10 for the model with the focal individual difference variable versus the null model that includes attitude is plotted. For BF10 < 1, evidence favors the null hypothesis, for BF10 > 1, evidence favors the alternative hypothesis. Descriptive labels regarding the strength of evidence follow extant recommendations88,89.
Discussion
Study 1 revealed three important findings. First, participants engaged in motivated reasoning in the context of climate change, confirming the results from previous studies20,21,22,23,24,25,26,27,28,29. The second important finding concerns the role of numeracy. On the one hand, we found that higher numeracy was associated with a reduced absolute bias (i.e., a reduced degree of asymmetric reasoning), indicating that more numerate individuals were better at differentiating the outcome of the two fictitious studies. This result is in line with previous research using the absolute bias score as a dependent variable in a different context32. On the other hand, numeracy did not significantly moderate the relationship between attitude and directional bias. Note, however, that this nonsignificant finding was inconclusive as indicated by the Bayes Factors, making definitive interpretations difficult. Third, the other factors investigated in the present study (i.e., Need for Cognition, Dark Factor of Personality) did not influence the participants’ performance. More specifically, none of the two factors moderated the relationship between attitude and directional bias (i.e., the degree of attitude-consistent judgment) and none of the two factors had an influence on the degree of the absolute bias (i.e., the degree of asymmetric reasoning regarding the two fictitious studies). The finding that Need for Cognition moderated the relationship between attitude and absolute bias was not part of our main research questions and will therefore not considered further in the following. In order to shed more light on the role of numeracy and to confirm the observation that none of the other two factors seem to play a major role for motivated reasoning in the context of climate change, we decided to run a preregistered conceptual replication study with a larger sample.
Study 2
Method
Participants
We aimed for a final sample of at least twice the size in the main study (i.e., at least 1100 participants). The study was preregistered (https://doi.org/10.17605/osf.io/94fn3). Participants were recruited via Prolific (www.prolific.com). Participants provided informed consent. All data exclusions, manipulations, and measures are reported. The study was programmed on SoSciSurvey69.
Session 1: prescreening
In Session 1, participants were prescreened for their attitude towards stricter regulations to reduce CO2 emissions. In total, 1604 participants completed the prescreening. Participation was only possible for Prolific users with an approval rate of at least 98% who were fluent in English and currently living in the United Kingdom. Participants who had already participated in Study 1 were not eligible for participation in Study 2. Participants were compensated with £1.65. Session 1 lasted about ten minutes. Data collection took place on January 11, 2023.
Session 2: main study
To account for exclusions and potential dropout between sessions, we decided to invite 1450 participants for Session 2. An equal distribution of participants across the seven scale points of the regulation attitude item would have implied inviting 207 participants per scale point for Session 2. Whenever we had 207 or fewer participants on one scale point of the regulation attitude item, all of these participants were invited for Session 2; whenever we had more than 207 participants on one of the seven points of the scale, we randomly drew a subset of participants (for more information about the participants who were selected for participation in Session 2 see the Supplemental Material S13 and S14).
From the total of 1450 participants that were invited to participate in Session 2, 1371 participants completed the study. Participants who did not pass all attention checks (N = 160) were excluded from the analysis. We also excluded participants who had noticeable short (i.e., less than one third of the median response time; N = 7) or long (i.e., more than six times the median response time; N = 6) response times, indicating careless responding. Thus, the final sample consisted of 1198 participants (Mage = 41.34, SD = 13.39, 18–81 years; 550 male, 637 female, 8 other, 3 prefer not to say; see Supplemental Material S15–S17 for information about political orientation, ethnicity, and education). Participants were compensated with £1.05 upon study completion. Session 2 lasted about six minutes. Data collection took place from January 22 to January 25, 2023.
Materials and procedure
As Study 2 was designed to be a conceptual replication study, we closely followed the design and procedure of Study 1. As the correlation between attitude and directional bias was quite low in Study 1 (r = 0.09) and as the number of participants who opposed stricter regulations was clearly lower than the number of participants who supported stricter regulations, we implemented three changes hoping to strengthen the motivated reasoning effect and to achieve a slightly more balanced sample regarding attitude. First, the attitude item was changed from “Governments should enforce stricter regulations to decrease CO2 emissions even if these regulations restrict the freedom of the individual” to “Governments should enforce stricter regulations to decrease CO2 emissions, which restrict the freedom of the individual”. Second, we changed the labels on the two-by-two contingency tables from “Cities/Districts with stricter regulations” versus “Cities/Districts without stricter regulations” (see Fig. 2) to “Cities/Districts with regulations restricting the freedom of the individual” versus “Cities/Districts without regulations restricting the freedom of the individual”. Third, the prescreening (Session 1) only included the attitude item but not the six items measuring attitude strength; the attitude item was presented again and this time together with the six items measuring attitude strength in the beginning of Session 2 directly before evaluating the two fictitious studies. A fourth change was implemented in order to allow additional exploratory analyses (for results, see the Supplemental Material S22): Below each of the two fictitious studies, we added two more items, one item measuring how confident participants are that their response to the fictitious study was correct (“How confident are you that your answer is correct?”; 6-point Likert scale ranging from 1 (very unconfident) to 6 (very confident); see79) and one item measuring how trustworthy participants find the study’s result (“How trustworthy do you think the results of this study are?”; 6-point Likert scale ranging from 1 (not at all trustworthy) to 6 (very trustworthy); see80).
The rest of the study materials as well as study procedure were identical to Study 1 (for the attention check items, see the Supplemental Material S12). The answers on the attitude items in Session 1 and Session 2 were highly correlated, r = 0.744, p < 0.001. The statistical analyses were conducted with the answers on the attitude item from Session 2. The internal consistency of the 6-item attitude strength scale was good, α = 0.85. The internal consistency of the NFC scale was excellent, α = 0.94. The internal consistency of the scale measuring D was good, α = 0.86.
Results
Study evaluation
Overall, study evaluation accuracy rates were rather low, confirming the validity of our stimulus material as posing difficult statistical problems: 43.82% of the participants evaluated the pro-regulation study correctly (i.e., selected 4, 5, or 6 on the 6-point Likert scale), and 57.10% of the participants evaluated the anti-regulation study correctly (i.e., selected 1, 2, or 3). That is, the anti-regulation study was significantly more likely to be evaluated correctly than the pro-regulation study, t(1197) = -6.67, p < 0.001, d = 0.19.
There was no significant difference between the time participants spent on evaluating the results of the pro-regulation study (Mpro = 87.83 s, SD = 67.79, Mdn. = 73 s) and the results of the anti-regulation study (Manti = 93.54 s, SD = 88.58, Mdn. = 73 s), t(1197) =− 1.89, p = 0.059, d = 0.05. The fact that participants spent more than a minute on average on evaluating the results indicates that participants engaged with the material and took the task seriously.
Biased reasoning and zero-order correlations
Attitude towards stricter regulations to reduce CO2 emissions was significantly correlated with the directional bias, indicating that participants showed attitude-consistent judgment (i.e., engaged in motivated reasoning) in the context of climate change, r = 0.11, p < 0.001. Zero-order correlations between variables are displayed in Table 2.
Potential factors related to biased reasoning
As in Study 1, we ran separate hierarchical regression analyses for each individual difference variable (i.e., numeracy, NFC, D). Again, we also ran further hierarchical regression analyses in which we entered the other two individual difference variables as predictors in the first step to check whether the results for each individual difference variable remain the same when the influence of the other predictors is controlled. Again, this did not change the results in meaningful ways (see Supplemental Material S20). Note that the preregistration stated that we would run one-step regression analyses with attitude, one of the three factors potentially influencing motivated reasoning (i.e., numeracy, NFC, D), and the product term of the two variables as predictors and bias as the criterion. However, we decided that conducting hierarchical regression analyses would be easier to report and understand. Note that the results do not change when running the preregistered one-step analyses instead of the hierarchical regression analyses. Tables for all regression analyses can be found in the Supplemental Material S20. Finally, we ran logistic regression analyses on study evaluation accuracy. The results are presented in the Supplemental Material S21.
Directional bias
Again, attitude positively predicted the directional bias score, B = 0.23, SEB = 0.06, p = < 0.001, R2 = 0.01. Numeracy did not predict the directional bias score, B =− 0.06, SEB = 0.06, p = 0.311, ΔR2 = 0.001, and no interaction between attitude and numeracy was observed, B = 0.01, SEB = 0.06, p = 0.907, ∆R2 < 0.001. In the subsequent regression analysis, Need for Cognition was entered in the second step, yielding no significant association, B = 0.01, SEB = 0.06, p = 0.908, ΔR2 < 0.001. The interaction between attitude and Need for Cognition did not explain additional variance, B = − 0.01, SEB = 0.06, p = 0.832, ΔR2 < 0.001. Likewise, the Dark Factor did neither predict the bias score, B = 0.07, SEB = 0.06, p = 0.233, ΔR2 = 0.001, nor was an interaction present, B = 0.06, SEB = 0.06, p = 0.275, ΔR2 = 0.001. To check for higher-order interactions involving attitude as one of the predictors, we ran an additional regression analysis with all variables and the product term of these variables as predictors. No three-way interactions that involved attitudes were observed, all ps > 0.159 (for details, see the Supplemental Material S20).
Absolute bias
Attitude negatively predicted the absolute bias score, B = -0.11, SEB = 0.04, p = 0.011, R2 = 0.005. When numeracy was entered as an additional predictor in the second step, higher numeracy predicted a reduced absolute bias score, B = -0.18, SEB = 0.04, p < 0.001, ΔR2 = 0.02. The interaction between attitude and numeracy was not significant, B = − 0.07, SEB = 0.04, p = 0.091, ∆R2 = 0.002. Our second analysis involved the Need for Cognition, which did not predict the absolute bias score when entered in the second step, B = -0.03, SEB = 0.04, p = 0.461, R2 < 0.001, and the interaction between attitude and Need for Cognition was not significant, B = − 0.04, SEB = 0.04, p = 0.337, ∆R2 = 0.001. The Dark Factor was unrelated to the absolute bias score, B = − 0.06, SEB = 0.04, p = 0.144, ΔR2 = 0.002, and no interaction between attitude and the Dark Factor was observed, B = < 0.01, SEB = 0.04, p = 0.986, ∆R2 < 0.001. To check for higher-order interactions involving attitude as one of the predictors, we ran an additional regression analysis with all variables and the product term of these variables as predictors. Again, none of the three-way interactions that involved attitudes as a predictor yielded a significant higher-order effect, all ps > 0.270 (for details, see the Supplemental Material S20).
Bayes regressions
We again completed our analyses with Bayes linear regression analyses to gain additional insights regarding the degree of evidence favoring the null hypothesis76,77,78. Bayes analyses were first conducted with the directional bias score as the criterion and the interactions between attitude and the individual difference variable vs. the null model as the effect of interest. For all three individual difference variables the results indicate moderate support for the null hypothesis (see Fig. 4). Next, analyses were conducted with the absolute bias score as the criterion and the association with the individual difference variable vs. the null model as the effect of interest. These analyses indicate moderate support for the null hypothesis in models with the Need for Cognition or the Dark Factor the predictor, as well as very strong support for the hypothesis that numeracy is negatively linked to the absolute bias.
General discussion
The present research had two main goals. The first goal was to investigate whether individuals engage in motivated reasoning in the context of climate change using a design that accounts for the limitations of previous studies. In order to allow a nuanced analysis of motivated reasoning, we calculated two biases, a directional and an absolute bias. The second goal was to examine potential moderating factors of motivated reasoning. More specifically, we sought to investigate the influence of the ability to perform the numerical-statistical task at hand (operationalized in terms of numeracy), the willingness to use this ability (operationalized in terms of Need for Cognition), and the stable tendency to maximize one’s individual utility (operationalized in terms of the Dark Factor of Personality).
Regarding the first goal, we found clear evidence across two studies that participants engaged in motivated reasoning when evaluating studies on the effectiveness of stricter regulations to reduce CO2 emissions. In particular, the more strongly participants opposed stricter regulations, the more strongly they exhibited a directional bias towards interpreting the studies as showing that stricter regulations are counterproductive. Conversely, the more strongly participants favored stricter regulations, the more strongly they exhibited a directional bias towards interpreting the studies as showing that stricter regulations are effective. These results are in line with the previous studies on motivated reasoning in the context of climate change20,21,22,23,24,25,26,27,28,29. Importantly, the present research does not only confirm these results but also extends the existing body of knowledge in several ways: We demonstrated motivated reasoning in the context of climate change when evaluating scientific evidence on the effectiveness of specific policies to reduce CO2 emissions instead of merely addressing the question whether climate change is anthropogenic. Thus, motivated reasoning also extends to more nuanced discourses about the pros and cons of specific policies. Furthermore, we demonstrated motivated reasoning when directly measuring participants’ attitude on stricter regulations instead of using political orientation as a proxy. Thus, motivated reasoning is not limited to highly identity-relevant social categories such as political orientation. With respect to the latter, note that political orientation was – as indicated by the zero-order correlations (see Table 1 and Table 2) – not associated with directional bias, suggesting that it is important to address the attitudes underlying motivated reasoning rather than using broad identity categories, at least when investigating a topic that is less polarized along party lines9,81.
Regarding our findings for motivated reasoning, two more aspects deserve attention. On the one hand, and as pointed out in the introduction, motivated reasoning is usually seen as a combination of the motivated acceptance of attitude-consistent information and the motivated rejection of attitude-inconsistent information. Accordingly, our directional bias score combines the participants’ responses to attitude-consistent and attitude-inconsistent information in a joint estimate. Nevertheless, future research using different designs and bias estimates could strive to disentangle the processes of motivated acceptance and motivated rejection and their contribution to biased reasoning. On the other hand, the correlation between attitude and the directional bias score was robust but relatively small (about 0.10 in both studies), suggesting that motivated reasoning effects in the context of evaluating policies to reduce CO2 emissions exist but might be somewhat limited in size. Interestingly, a study on motivated reasoning in the context of mask mandates during the COVID-19 pandemic, which used the same design as the present studies, found a substantially higher correlation between attitude and directional bias of 0.3432. This indicates that the size of motivated reasoning effects may vary depending on contextual factors such as the topic under investigation. Note, however, that even relatively small effects as observed in the present studies can be of great practical importance especially when they apply to large groups of individuals82. Simply put, as motivated reasoning is usually assumed to be a general phenomenon, its societal impact can be large even if the effect itself is small.
Regarding the second goal, first note that the low correlations between the three dimensions of individual differences (see Tables 1 and 2) confirm our idea that these dimensions are indeed relatively distinct from one another. As far as the influence of these three variables on motivated reasoning is concerned, the results across our two studies are remarkably similar: In short, only numeracy was clearly associated with the degree of biased reasoning in the sense that higher numeracy predicted a reduced absolute bias. Neither NFC nor D were related to the directional or the absolute bias.
The finding that higher numeracy is associated with a reduced absolute bias is in line with previous research demonstrating that numeracy can lead to more objective reasoning and better decision-making even in the context of highly politicized issues32,47,48. At the same time, the finding that numeracy did not moderate the relationship between attitude and directional bias contradicts those studies, which have found more pronounced motivated reasoning among highly numerate individuals21,25,26,42,43. Although the latter studies have been criticized on methodological and theoretical grounds44, further investigating under which circumstances numeracy is associated with reduced and under which circumstances it is associated with exacerbated motivated reasoning is an important topic for future research. Note that the mechanisms potentially underlying the contradictory findings (i.e., using numerical-statistical abilities to arrive at a more objective evaluation of evidence versus using these abilities to rationalize prior beliefs) are not mutually exclusive; hence, which mechanism is more dominant may depend on various contextual factors and boundary conditions. Against this background, the present research also underlines the importance of a nuanced analysis of motivated reasoning: Rather than investigating motivated reasoning as a single construct, it may be advisable to look at different biases that elucidate different aspects of motivated reasoning, such as the directional and the absolute bias used in the present study. In this respect, our results suggest that numeracy is not associated with the degree of attitude-consistent reasoning but with a better ability to understand that the two fictitious studies show different outcomes.
Given that we have found consistent evidence that higher numeracy is associated with reduced biased reasoning, it appears essential to foster statistical education in the general population83,84: If individuals lack the necessary abilities to understand the scientific evidence that is being discussed in the public discourse and that underlies policy decisions, they potentially also lack the ability to act as autonomous and responsible citizens who do their part to address global challenges such as climate change. In other words, our findings point to the importance of systemic reforms that provide an environment in which individuals can develop the skills that are needed in the complex modern world: While motivated reasoning may be a universal cognitive mechanism, the degree to which individuals correctly interpret scientific evidence will crucially depend on their access to appropriate educational environments. In the short run, it also seems important to support numerical information with visualizations and explanations that are easy to understand and that can help to bridge existing knowledge gaps85,86.
The encouraging result that higher numeracy is associated with a reduced absolute bias is further underlined by the observation that degree of motivated reasoning does not vary as a function of the individuals’ NFC or D, that is, as a function of two individual difference variables, which would be much harder to change. With respect to NFC, our results are in line with a previous study which also found no significant effects20. Thus, the present research lends credibility to the idea that the positive effects of a high NFC (i.e., being more likely to be driven by accuracy goals)53 might be counterbalanced by the negative effects of a high NFC (i.e., the tendency to have stronger and less ambivalent attitudes that are more resistant to change)54,55. With respect to D, one may speculate that the problematic epistemic beliefs which are fueled by a pronounced Dark Factor60,61,62 lead to cognitive processes that are conceptually distinct from motivated reasoning. More specifically, one could assume that individuals high in D are skeptical about scientific evidence in general because they believe, for instance, that truth is political and ultimately a matter of power, while motivated reasoning refers to biased information processing when being confronted with specific scientific evidence. However, further investigating these potential differences could be another important topic for future research.
Three aspects of the stimulus material need emphasis. First, evaluation accuracy was relatively low across both studies: Only about half of the participants indicated correctly whether the contingency tables showed that stricter regulations are effective or counterproductive. This demonstrates that analyzing contingency tables indeed poses a difficult statistical problem to individuals and that the correct interpretation of the provided data was not obvious. As real scientific evidence that individuals might encounter when looking for information about climate change is likely to be difficult to understand, the stimulus material offers a valid test for motivated reasoning. At the same time, it should not be forgotten that understanding data on climate change arguably requires more sophisticated statistical knowledge than the ability to compare the ratios of contingency tables – making the participants’ poor performance even more worrying and providing more evidence for our claim that improving statistical literacy in the general population is highly desirable.
Second, we chose to present the fictitious studies in the form of two-by-two contingency tables because this allowed a well-controlled presentation of the data, ensuring that the relevant information is easily available to participants. In addition, this also helped to make certain that participants’ responses were based on reasoning processes rather than their ability to read and understand texts or their general world knowledge. That being said, we want to emphasize that studies on motivated reasoning should be conducted using different materials to ensure the generalizability of the findings. Importantly, this particularly applies to investigations concerning the factors influencing motivated reasoning: While there is already ample evidence regarding the existence of motivated reasoning in the context of climate change using different kinds of stimulus material20,21,22,23,24,25,26,27,28,29, this is not the case for all factors investigated in the present studies. Hence, we encourage independent replications of our findings using different materials.
Third, in both Study 1 and Study 2 the percentage of participants who evaluated the anti-regulation study correctly was higher than the percentage of participants who evaluated the pro-regulation study correctly. As research clearly indicates that enforcing stricter regulations can be a powerful means to decrease CO2 emissions87, one might assume that participants expected to be presented with evidence supporting the effectiveness of stricter regulations. In case that—at least some—participants based their response on such a prior expectation, however, this would result in exactly the opposite pattern, as a superficial evaluation of the anti-regulation study suggests that stricter regulations are effective, while a superficial evaluation of the pro-regulation study suggests that stricter regulations are counterproductive. That is, in this case, the participants’ performance in the pro-regulation study should have been better than the performance in the anti-regulation study. A possible explanation for the observed pattern could be that the participants’ expectations were not so much shaped by the existing scientific evidence but rather by societal debates, which often question how much of a difference stricter regulations on a local level can actually make when climate change is a global problem. However, this finding deserves further investigation.
In sum, the present research offers good and bad news. The bad news is that motivated reasoning poses a challenge to modern societies facing climate change. Importantly, this does not only apply to the general question whether climate change is anthropogenic, but also to the evaluation of specific scientific evidence. The good news is that the degree to which individuals engage in biased reasoning mainly depends on the individuals’ abilities (such as numeracy), which can in principle be improved through education, and not so much on rather stable personality characteristics (such as the Dark Factor of Personality) or thinking styles (such as Need for Cognition), which are arguably harder to change.
Data availability
Data and analysis code for both Study 1 and Study 2 can be downloaded at https://doi.org/10.17605/osf.io/nvek2.
References
Lynas, M., Houlton, B. Z. & Perry, S. Greater than 99% consensus on human caused climate change in the peer-reviewed scientific literature. Environ. Res. Lett. 16(11), 114005. https://doi.org/10.1088/1748-9326/ac2966 (2021).
IPCC (2021). Climate change 2021: The physical science basis. Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press. https://doi.org/10.1017/9781009157896
Ivanova, D. et al. Quantifying the potential for climate change mitigation of consumption options. Environ. Res. Lett. 15(9), 093001. https://doi.org/10.1088/1748-9326/ab8589 (2020).
Whitmarsh, L., Poortinga, W. & Capstick, S. Behaviour change to address climate change. Curr. Opin. Psychol. 42, 76–81. https://doi.org/10.1016/j.copsyc.2021.04.002 (2021).
Hornsey, M. J., Harris, E. A., Bain, P. G. & Fielding, K. S. Meta-analyses of the determinants and outcomes of belief in climate change. Nat. Clim. Chang. 6(6), 622–626. https://doi.org/10.1038/NCLIMATE2943 (2016).
Habermas, J. Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy (MIT Press, 1996).
Kruglanski, A. W. Motivated social cognition: Principles of the interface. In Social psychology: Handbook of Basic Principles (eds Kruglanski, A. W. & Higgins, E. T.) 493–520 (Guilford Press, 1996).
Kunda, Z. The case for motivated reasoning. Psychol. Bull. 108(3), 480–498. https://doi.org/10.1037/0033-2909.108.3.480 (1990).
Hornsey, M. J. Why facts are not enough: Understanding and managing the motivated rejection of science. Curr. Dir. Psychol. Sci. 29(6), 583–591. https://doi.org/10.1177/0963721420969364 (2020).
Lewandowsky, S. & Oberauer, K. Motivated rejection of science. Curr. Dir. Psychol. Sci. 25(4), 217–222. https://doi.org/10.1177/0963721416654436 (2016).
Rothmund, T., Gollwitzer, M., Nauroth, P. & Bender, J. Motivierte Wissenschaftsrezeption [Motivated Science Reception]. Psychol. Rundsch. 68(3), 193–197. https://doi.org/10.1026/0033-3042/a000364 (2017).
Taber, C. S. & Lodge, M. Motivated skepticism in the evaluation of political beliefs. Am. J. Political Sci. 50(3), 755–769. https://doi.org/10.1111/j.1540-5907.2006.00214.x (2006).
Druckman, J. N. & McGrath, M. C. The evidence for motivated reasoning in climate change preference formation. Nat. Clim. Chang. 9(2), 111–119. https://doi.org/10.1038/s41558-018-0360-1 (2019).
Dawson, E., Gilovich, T. & Regan, D. T. Motivated reasoning and performance on the Wason selection task. Pers. Soc. Psychol. Bull. 28(10), 1379–1387. https://doi.org/10.1177/014616702236869 (2002).
Edwards, K. & Smith, E. E. A disconfirmation bias in the evaluation of arguments. J. Pers. Soc. Psychol. 71(1), 5–24. https://doi.org/10.1037/0022-3514.71.1.5 (1996).
Kunda, Z. Motivated inference: Self-serving generation and evaluation of causal theories. J. Pers. Soc. Psychol. 53(4), 636–647. https://doi.org/10.1037/0022-3514.53.4.636 (1987).
Lord, C. G., Ross, L. & Lepper, M. R. Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. J. Pers. Soc. Psychol. 37(11), 2098–2109. https://doi.org/10.1037/0022-3514.37.11.2098 (1979).
Bender, J., Rothmund, T., Nauroth, P. & Gollwitzer, M. How moral threat shapes laypersons’ engagement with science. Pers. Soc. Psychol. Bull. 42(12), 1723–1735. https://doi.org/10.1177/0146167216671518 (2016).
Kahan, D. M., Braman, D., Slovic, P., Gastil, J. & Cohen, G. Cultural cognition of the risks and benefits of nanotechnology. Nat. Nanotechnol. 4(2), 87–90. https://doi.org/10.1038/nnano.2008.341 (2009).
Caddick, Z. A. & Feist, G. J. When beliefs and evidence collide: psychological and ideological predictors of motivated reasoning about climate change. Think. Reason. 28(3), 428–464. https://doi.org/10.1080/13546783.2021.1994009 (2022).
Drummond, C. & Fischhoff, B. Individuals with greater science literacy and education have more polarized beliefs on controversial science topics. Proc. Natl. Acad. Sci. 114(36), 9587–9592. https://doi.org/10.1073/pnas.1704882114 (2017).
Fischer, H., Huff, M. & Said, N. Polarized climate change beliefs: No evidence for science literacy driving motivated reasoning in a U.S. national study. Am. Psychol. 77(7), 822–835. https://doi.org/10.1037/amp0000982 (2022).
Hart, P. S. & Nisbet, E. C. Boomerang effects in science communication: How motivated reasoning and identity cues amplify opinion polarization about climate mitigation policies. Commun. Res. 39(6), 701–723. https://doi.org/10.1177/0093650211416646 (2012).
Hart, P. S., Nisbet, E. C. & Myers, T. A. Public attention to science and political news and support for climate change mitigation. Nat. Clim. Chang. 5(6), 541–545. https://doi.org/10.1038/nclimate2577 (2015).
Kahan, D. M. et al. The polarizing impact of science literacy and numeracy on perceived climate change risks. Nat. Clim. Chang. 2(10), 732–735. https://doi.org/10.1038/nclimate1547 (2012).
Nurse, M. S. & Grant, W. J. I’ll see it when I believe it: Motivated numeracy in perceptions of climate change risk. Environ. Commun. 14(2), 184–201. https://doi.org/10.1080/17524032.2019.1618364 (2020).
Ripberger, J. T. et al. Bayesian versus politically motivated reasoning in human perception of climate anomalies. Environ. Res. Lett. 12(11), 114004. https://doi.org/10.1088/1748-9326/aa8cfc (2017).
Said, N., Fischer, H. & Anders, G. Contested science: Individuals with higher metacognitive insight into interpretation of evidence are less likely to polarize. Psychon. Bull. Rev. 29, 668–680. https://doi.org/10.3758/s13423-021-01993-y (2022).
Washburn, A. N. & Skitka, L. J. Science denial across the political divide: Liberals and conservatives are similarly motivated to deny attitude-inconsistent science. Soc. Psychol. Pers. Sci. 9(8), 972–980. https://doi.org/10.1177/1948550617731500 (2018).
Bayes, R. & Druckman, J. N. Motivated reasoning and climate change. Curr. Opin. Behav. Sci. 42, 27–35. https://doi.org/10.1016/j.cobeha.2021.02.009 (2021).
Molden, D. C., Bayes, R. & Druckman, J. N. A motivational systems approach to investigating opinions on climate change. Think. Reason. 28(3), 396–427. https://doi.org/10.1080/13546783.2021.1982003 (2022).
Hutmacher, F., Reichardt, R. & Appel, M. The role of motivated science reception and numeracy in the context of the COVID-19 pandemic. Public Underst. Sci. 31(1), 19–34. https://doi.org/10.1177/09636625211047974 (2022).
Cotter, R. G., Lodge, M. & Vidigal, R. The boundary conditions of motivated reasoning. In The Oxford Handbook of Electoral Persuasion (eds Suhay, E. et al.) 66–87 (Oxford University Press, 2020).
Garcia-Retamero, R., Sobkow, A., Petrova, D., Garrido, D. & Traczyk, J. Numeracy and risk literacy: What have we learned so far?. Span. J. Psychol. 22, e10. https://doi.org/10.1017/sjp.2019.16 (2019).
Reyna, V. F., Nelson, W. L., Han, P. K. & Dieckmann, N. F. How numeracy influences risk comprehension and medical decision making. Psychol. Bull. 135(6), 943–973. https://doi.org/10.1037/a0017327 (2009).
Pachur, T. & Galesic, M. Strategy selection in risky choice: The impact of numeracy, affect, and cross-cultural differences. J. Behav. Decis. Mak. 26(3), 260–271. https://doi.org/10.1002/bdm.1757 (2013).
Skagerlund, K., Lind, T., Strömbäck, C., Tinghög, G. & Västfjäll, D. Financial literacy and the role of numeracy–How individuals’ attitude and affinity with numbers influence financial literacy. J. Behav. Exp. Econ. 74, 18–25. https://doi.org/10.1016/j.socec.2018.03.004 (2018).
Traczyk, J. et al. Numerate decision makers don’t use more effortful strategies unless it pays: A process tracing investigation of skilled and adaptive strategy selection in risky decision making. Judgm. Decis. Mak. 13(4), 372–381 (2018).
Apter, A. J. et al. Linking numeracy and asthma-related quality of life. Patient Educat. Couns. 75(3), 386–391. https://doi.org/10.1016/j.pec.2009.01.003 (2009).
Garcia-Retamero, R., Andrade, A., Sharit, J. & Ruiz, J. G. Is patients’ numeracy related to physical and mental health?. Med. Decis. Making 35(4), 501–511. https://doi.org/10.1177/0272989X15578126 (2015).
Huizinga, M. M., Beech, B. M., Cavanaugh, K. L., Elasy, T. A. & Rothman, R. L. Low numeracy skills are associated with higher BMI. Obesity 16(8), 1966–1968. https://doi.org/10.1038/oby.2008.294 (2008).
Kahan, D. M., Landrum, A., Carpenter, K., Helft, L. & Hall Jamieson, K. Science curiosity and political information processing. Polit. Psychol. 38, 179–199. https://doi.org/10.1111/pops.12396 (2017).
Kahan, D. M., Peters, E., Dawson, E. C. & Slovic, P. Motivated numeracy and enlightened self-government. Behav. Public Policy 1, 54–86. https://doi.org/10.1017/bpp.2016.2 (2017).
Tappin, B. M., Pennycook, G. & Rand, D. G. Rethinking the link between cognitive sophistication and politically motivated reasoning. J. Exp. Psychol.: General 150(6), 1095–1114. https://doi.org/10.1037/xge0000974 (2021).
Connor, P., Sullivan, E., Alfano, M. & Tintarev, N. Motivated numeracy and active reasoning in a Western European sample. Behav. Public Policy https://doi.org/10.1017/bpp.2020.32 (2020).
Persson, E., Andersson, D., Koppel, L., Västfjäll, D. & Tinghög, G. A preregistered replication of motivated numeracy. Cognition 214, 104768. https://doi.org/10.1016/j.cognition.2021.104768 (2021).
Lind, T., Erlandsson, A., Västfjäll, D. & Tinghög, G. Motivated reasoning when assessing the effects of refugee intake. Behav. Public Policy https://doi.org/10.1017/bpp.2018.41 (2018).
Tappin, B. M., Pennycook, G. & Rand, D. G. Bayesian or biased? Analytic thinking and political belief updating. Cognition 204, 104375. https://doi.org/10.1016/j.cognition.2020.104375 (2020).
Cacioppo, J. T. & Petty, R. E. The need for cognition. J. Personal. Soc. Psychol. 42(1), 116–131. https://doi.org/10.1037/0022-3514.42.1.116 (1982).
Cacioppo, J. T., Petty, R. E., Feinstein, J. A. & Jarvis, W. B. G. Dispositional differences in cognitive motivation: The life and times of individuals varying in need for cognition. Psychol. Bull. 119(2), 197–253. https://doi.org/10.1037/0033-2909.119.2.197 (1996).
Petty, R. E., Briñol, P., Loersch, C. & McCaslin, M. J. The need for cognition. In Handbook of Individual Differences in Social Behavior (eds Leary, M. R. & Hoyle, R. H.) 318–329 (Guilford Press, 2009).
Feist, G. J. Predicting interest in and attitudes toward science from personality and need for cognition. Personal. Individ. Differ. 52(7), 771–775. https://doi.org/10.1016/j.paid.2012.01.005 (2012).
Arceneaux, K. & Vander Wielen, R. J. The effects of need for cognition and need for affect on partisan evaluations. Polit. Psychol. 34(1), 23–42. https://doi.org/10.1111/j.1467-9221.2012.00925.x (2013).
Haugtvedt, C. P. & Petty, R. E. Personality and persuasion: Need for cognition moderates the persistence and resistance of attitude changes. J. Personal. Soc. Psychol. 63(2), 308–319. https://doi.org/10.1037/0022-3514.63.2.308 (1992).
Thompson, M. M. & Zanna, M. P. The conflicted individual: Personality-based and domain specific antecedents of ambivalent social attitudes. J. Personal. 63(2), 259–288. https://doi.org/10.1111/j.1467-6494.1995.tb00810.x (1995).
Kudrna, J., Shore, M. & Wassenberg, D. Considering the role of “need for cognition” in students’ acceptance of climate change & evolution. Am. Biol. Teacher 77(4), 250–257. https://doi.org/10.1525/abt.2015.77.4.4 (2015).
Barbaro, N., Pickett, S. M. & Parkhill, M. R. Environmental attitudes mediate the link between need for cognition and pro-environmental goal choice. Personal. Individ. Differ. 75, 220–223. https://doi.org/10.1016/j.paid.2014.11.032 (2015).
Moshagen, M., Hilbig, B. E. & Zettler, I. The dark core of personality. Psychol. Rev. 125(5), 656–688. https://doi.org/10.1037/rev0000111 (2018).
Bader, M. et al. Themes of the dark core of personality. Psychol. Assess. 33(6), 511–525. https://doi.org/10.1037/pas0001006 (2021).
Rudloff, J. P. & Appel, M. When truthiness trumps truth: Epistemic beliefs predict the accurate discernment of fake news. J. Appl. Res. Memory Cognition https://doi.org/10.1037/mac0000070 (2022).
Rudloff, J. P., Hutmacher, F. & Appel, M. Beliefs about the nature of knowledge shape responses to the pandemic: Epistemic beliefs, the dark factor of personality, and COVID-19-related conspiracy ideation and behavior. J. Personal. 90(6), 937–955. https://doi.org/10.1111/jopy.12706 (2022).
Rudloff, J. P., Hutmacher, F. & Appel, M. Post-truth epistemic beliefs rooted in the dark factor of personality are associated with higher COVID-19 vaccination refusal. Sci. Rep. 13, 4254. https://doi.org/10.1038/s41598-023-31079-9 (2023).
Pitiruţ, B., Ogunbode, C. & Enea, V. Attitudes towards global warming: The role of anticipated guilt and the dark triad traits. Personal. Individ. Differ. 185, 111285. https://doi.org/10.1016/j.paid.2021.111285 (2022).
Ucar, G. K., Malatyalı, M. K., Planalı, G. Ö. & Kanik, B. Personality and pro-environmental engagements: The role of the dark triad, the light triad, and value orientations. Personal. Individ. Differ. 203, 112036. https://doi.org/10.1016/j.paid.2022.112036 (2023).
German Research Foundation (2023). Statement by an ethics committeee. https://www.dfg.de/en/research_funding/faq/faq_humanities_social_science/index.html
German Psychological Society (2022). Berufsethische Richtlinien [Work ethical guidelines]. https://www.dgps.de/fileadmin/user_upload/PDF/Berufsetische_Richtlinien/BER-Foederation-20230426-Web-1.pdf
Schönbrodt, F. D. & Perugini, M. At what sample size do correlations stabilize?. J. Res. Personal. 47(5), 609–612. https://doi.org/10.1016/j.jrp.2013.05.009 (2013).
Simonsohn, U. No-way interactions. The Winnower https://doi.org/10.15200/winn.142559.90552 (2015).
Leiner, D. J. SoSci Survey (Version 3.1.06) [Computer software]. Available at https://www.soscisurvey.de (2019).
Pomerantz, E. M., Chaiken, S. & Tordesillas, R. S. Attitude strength and resistance processes. J. Personal. Soc. Psychol. 69(3), 408–419. https://doi.org/10.1037/0022-3514.69.3.408 (1995).
Rosenthal, R. & Rubin, D. B. A simple general purpose display of magnitude of experimental effect. J. Educat. Psychol. 74(2), 166–169. https://doi.org/10.1037/0022-0663.74.2.166 (1982).
Lenhard, W. & Lenhard, A. Berechnung von Effektstärken [Calculation of effect sizes]. Psychometrica https://doi.org/10.13140/RG.2.2.17823.92329 (2016).
Weller, J. A. et al. Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. J. Behav. Decis. Making 26(2), 198–212. https://doi.org/10.1002/bdm.1751 (2013).
Cacioppo, J. T., Petty, R. E. & Feng Kao, C. The efficient assessment of need for cognition. J. Personal. Assess. 48(3), 306–307. https://doi.org/10.1207/s15327752jpa4803_13 (1984).
Moshagen, M., Zettler, I. & Hilbig, B. E. Measuring the dark core of personality. Psychol. Assess. 32(2), 182–196. https://doi.org/10.1037/pas0000778 (2020).
Dienes, Z. Using Bayes to get the most out of non-significant results. Front. Psychol. 5, 781. https://doi.org/10.3389/fpsyg.2014.00781 (2014).
van Doorn, J. et al. The JASP guidelines for conducting and reporting a Bayesian analysis. Psychon. Bull. Rev. 28, 813–826. https://doi.org/10.3758/s13423-020-01798-5 (2021).
Quintana, D. S. & Williams, D. R. Bayesian alternatives for common null-hypothesis significance tests in psychiatry: A non-technical guide using JASP. BMC Psychiatry 18(1), 178. https://doi.org/10.1186/s12888-018-1761-4 (2018).
Pieger, E., Mengelkamp, C. & Bannert, M. Metacognitive judgments and disfluency–Does disfluency lead to more accurate judgments, better control, and better performance?. Learn. Instr. 44, 31–40. https://doi.org/10.1016/j.learninstruc.2016.01.01 (2016).
Kuru, O., Pasek, J. & Traugott, M. W. Motivated reasoning in the perceived credibility of public opinion polls. Public Opin. Q. 81(2), 422–446. https://doi.org/10.1093/poq/nfx018 (2017).
Hornsey, M. J. & Fielding, K. S. Attitude roots and Jiu Jitsu persuasion: Understanding and overcoming the motivated rejection of science. Am. Psychol. 72(5), 459–473. https://doi.org/10.1037/a0040437 (2017).
Rosenthal, R. & Rubin, D. B. A note on percent variance explained as a measure of the importance of effects. J. Appl. Soc. Psychol. 9(5), 395–396. https://doi.org/10.1111/j.1559-1816.1979.tb02713.x (1979).
Fong, G. T., Krantz, D. H. & Nisbett, R. E. The effects of statistical training on thinking about everyday problems. Cognit. Psychol. 18(3), 253–292. https://doi.org/10.1016/0010-0285(86)90001-0 (1986).
Gigerenzer, G., Gaissmaier, W., Kurz-Milcke, E., Schwartz, L. M. & Woloshin, S. Helping doctors and patients make sense of health statistics. Psychol. Sci. Public Interest 8(2), 53–96. https://doi.org/10.1111/j.1539-6053.2008.00033.x (2007).
Garcia-Retamero, R. & Galesic, M. Communicating treatment risk reduction to people with low numeracy skills: A cross-cultural comparison. Am. J. Public Health 99(12), 2196–2202. https://doi.org/10.2105/AJPH.2009.160234 (2009).
Peters, E. Beyond comprehension: The role of numeracy in judgments and decisions. Curr. Dir. Psychol. Sci. 21(1), 31–35. https://doi.org/10.1177/0963721411429960 (2012).
Martin, G. & Saikawa, E. Effectiveness of state climate and energy policies in reducing power-sector CO2 emissions. Nat. Climate Change 7(12), 912–919. https://doi.org/10.1038/s41558-017-0001-0 (2017).
Jeffreys, H. The Theory of Probability (Oxford University Press, 1961).
Lee, M. D. & Wagenmakers, E.-J. Bayesian Cognitive Modeling: A Practical Course (Cambridge University Press, 2014).
Funding
Open Access funding enabled and organized by Projekt DEAL. This publication was supported by the “Overhead Funding Program” of the Faculty of Human Sciences of the Julius-Maximilians-Universität Würzburg, Germany.
Author information
Authors and Affiliations
Contributions
F.H.: Conceptualization, Methodology, Investigation, Formal Analysis, Data Curation, Writing (original draft). M.A.: Conceptualization, Formal Analysis, Writing (original draft). R.R.: Conceptualization, Methodology, Investigation, Formal Analysis, Writing (original draft).
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Hutmacher, F., Reichardt, R. & Appel, M. Motivated reasoning about climate change and the influence of Numeracy, Need for Cognition, and the Dark Factor of Personality. Sci Rep 14, 5615 (2024). https://doi.org/10.1038/s41598-024-55930-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-024-55930-9
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
