The Paris Agreement on climate change aims to improve cooperation by allowing governments to set their own commitments. Its success hinges on whether governments and investors believe those national commitments. To assess credibility, we interrogate a large novel sample of climate policy elites with decades of experience and well-placed to evaluate whether nations’ policy pledges are aligned with what they are politically and administratively able to implement. This expert assessment reveals that countries making the boldest pledges are also making the most credible pledges, contrasting theoretical warnings of a trade-off between ambition and credibility. We find that the quality of national political institutions is the largest explanator of the variation in credibility, and Europe’s credibility is exceptionally high. We also find that economic factors, such as the costs and benefits of controlling emissions, are statistically unimportant in explaining the credibility of national pledges to cooperate.
For more than 30 years, diplomats have been trying to advance coordination on climate change policy1,2. Those efforts have taken many different diplomatic forms and reflected many different theories about how to govern global collective problems3,4. Some theories invoke the need for integrated global contracts linked to strict enforcement mechanisms, such as those linked to trade sanctions5. Other theories, and a growing array of evidence, suggest that integrated global contracts are impossible to craft and thus international agreements must be more decentralized and voluntary6,7. Still other, complementary, approaches see cooperation emerging from small groups of committed governments and firms—clubs—then deepening and expanding with effort and experience5,6,8,9.
The Paris Agreement, although it formally did not endorse any theory of change, reflects a shift away from contracting logics to greater roles for individual national initiative and experimentation10,11. Paris is oriented around pledges, known formally as nationally determined contributions (NDCs), and is formally non-punitive. Some studies on the virtues and risks of the non-punitive diplomatic approach emphasize the value of flexibility6,9,12,13,14,15. Others point to the danger that pledge-based systems will merely codify the status quo and not lead to much (or any) deeper cooperation5,16,17. Voluntary commitments raise age-old debates in the study of international law: absent enforcement, can voluntarism inspire countries to deviate from the status quo in ways that solve problems like the need to cut warming pollution18? If governments are free to select their own commitments, won’t pledges of great ambition to adopt costly policies go hand-in-hand with low credibility that deviations from the status quo will occur8,18,19?
Whether this new approach actually has much impact on national policies and emissions hinges on an important empirical research question: what is the credibility of the Paris pledges? Although simple to ask, this question is hard to answer. The formal content of most NDCs is extremely thin, making it difficult for analysts and governments to assess intent and credibility20. (In this paper we focus on NDC pledges related to controlling emissions, but the thinness of content applies as well to other climate-related pledges in NDCs such as those on financing and adaptation to climate impacts.) This opacity stems, in part, from the fact that the formal requirements for NDCs were established through an intergovernmental process based on consensus decision-making8,21,22, which nearly always is a recipe for the lowest common denominator15,23. Some studies have used these pledges to show their potential collective impact on global emissions and warming24,25. Typically, such studies focus on whether the stated ambition of national pledges will be adequate to meet agreed collective goals, such as stopping warming at 2 °C above pre-industrial levels. The question of whether stated pledges are credible lurks in the background, unanswered.
Some organizations track the content and credibility of NDCs, often with the policy goal of putting pressure on laggards26,27,28,29. However, using such tools reliably raises difficult methodological challenges because it is hard, looking from the outside of a national policy process, to evaluate how current and successor national governments will put stated policies into effect. With few exceptions (for example, refs. 29 and 30), no social science discipline has done much to advance theories and methods needed for cross-national explanation that could be used reliably for such purposes.
We offer a new method for assessing the credibility of national policy strategies by tapping a novel source of information: diplomatic and scientific experts who, for decades, have participated in climate policy debates. Often, experts are a useful source of structured information when it is impractical to measure variables of interest directly31. Such settings arise, as in climate policy, where the phenomena under study are highly complex and assessment requires informed judgement and intuition, guided by experience, because formal sources of empirical information such as national laws and regulations are hard to evaluate independently or elusive. This logic has inspired a rich literature that uses methods of expert elicitation32,33,34,35,36.
The value of expert information and reasoning has led behavioural scientists to probe when and how the experience of elites leads them to behave in ways that are different from the general public37,38,39,40,41,42,43,44,45,46. Experience often gives elites special skills for making complex decisions efficiently and reliably, although ‘experience’ is highly specific to the domain of expertise. Elite chess players have heuristics that are extremely valuable for chess but of little value for other board games such as Go47. Thus, while it is important to get expert insights, it is also crucially important to pay close attention to kinds of expertise. Non-elite populations can be used to address some important issues surrounding climate policy such as public willingness to pay for policy and reactions to climate risks48,49,50,51,52,53,54, but for the kinds of policy judgement and assessment that are the focus of this paper, perceptions of elites are distinct and indispensable.
We recruited the elite sample for this analysis with email invitations based on official registrations for Conference of the Parties (COP) sessions of the United Nations Framework Convention on Climate Change (UNFCCC), the main intergovernmental body tasked with addressing the climate crisis, and official lists of authors and reviewers of the IPCC, the body responsible for periodic assessment of climate science. Invitations contained a bespoke link to an online survey. Between September 2020 and January 2021, we recruited 829 respondents from more than 150 countries (response rate 47% on verified invitations, see Methods and Table 1). For nearly all countries, this survey period implicated just the first NDC; for the United States it spans the end of President Trump’s term and the expectations for a Biden administration. (See Supplementary Information for robustness checks for updated NDCs (Supplementary Table 26), including Trump and post-Trump results in the United States (Supplementary Tables 43–52).) To our knowledge, this is by far the largest and most diverse sample of climate policy elites ever polled systematically for their insights into climate policy.
Because experience is domain specific, we distinguish two types of climate expertise: policy experts (‘Negotiators’) drawn from the ranks of UNFCCC delegations and science experts (‘Scientists’) drawn from the IPCC who also participate extensively in COPs but more often as observers. We additionally measure levels of expertise by counting the numbers of COPs each expert has joined.
We gave each subject a battery of questions focused on national pledges (NDCs) and other elements of the Paris Agreement (for the full list of questions see the Supplementary Information). Those included expert evaluation of the likelihood that the NDC pledge submitted by their home country would be honoured—what scholars call ‘compliance.’ We also asked experts to evaluate for their home country and other regions the total effort implied by each submitted pledge—analogous to what is often called ‘ambition.’ Because the Paris Agreement is designed to let countries set their own pledges to reflect their own circumstances, we asked the experts to assess ambition relative to a country’s or region’s economic strength (see question D1_10 in the Supplementary Information). Expected compliance, conditional upon ambition, is credibility.
On average, experts from non-Organisation for Economic Co-operation and Development (OECD) countries consider their home countries’ pledges to be highly ambitious (and no less credible) in contrast with OECD countries (Fig. 1). Within the OECD, Europe is exceptional: experts from that region consider their pledges made to be ambitious and most credible.
We also asked experts to assess the commitments of other countries (Fig. 2). Owing to the large number of countries, we limited the assessment of other countries to a selection of the countries most important for climate mitigation policies. In Europe, we asked experts to evaluate the European Union (EU) rather than individual nations. From this perspective, not only do Europeans see themselves as ambitious and credible, but so do the experts from other regions—the variation is even more pronounced compared with the self-assessments shown in Fig. 1.
We distinguish negotiators from scientists in the responses shown in Figs. 1 and 2 for they have distinct types of expertise. Negotiators are typically government employees and steeped in the art of what is politically and administratively feasible. They have, for their own country, particular knowledge about the quality of policy proposals and the ability to assess from intuition and experience the impact of policies on factors that ultimately matter most, such as emissions. Climate scientists, by contrast, tend to focus on imperatives of stopping climate change and, typically, the inadequacy of efforts (for example, refs. 25,28). There are marked differences in assessments by these experts of their own countries, especially in the OECD countries (Fig. 1). Negotiators are much more optimistic about their home country’s expected compliance than scientists. Non-OECD Asia is the only exception, a finding that we link to the relatively small number of scientists from this region (Supplementary Tables 5 and 8). The differences between negotiators and scientists decrease (often disappear) in Fig. 2 when the negotiators look outside their specific domain of expertise and evaluate other countries’ pledges (see Supplementary Tables 14–17 for details). Policy expertise is domain specific; even the diplomats, when they look beyond the home country they know best, are not much different from scientists who lack extensive policy experience.
Now we turn to plausible explanations through regression analysis. Our dependent variable is expected compliance—that is, credibility— of the home country (Fig. 1). The theoretical literature suggests that ambition should explain compliance. Indeed, much of the scholarship in international relations, surveyed above, suggests that low ambition begets high compliance17,18,55. Yet the logic of the Paris Agreement is based on the opposite expectation—credible pledges will also beget more cooperation and ambition. We include two measures of ambition. First is the experts’ own assessment of ambition considering the economic capabilities of their home country or region (Fig. 1). Second, we alternatively include an independently measured assessment of ambition derived by scientists (to the measure by Robiou du Pont & Meinshausen28; RdP&M) who are focused on what each country must do individually so that the collective efforts add up to the widely discussed goal of stopping warming at 1.5 °C above pre-industrial levels28. These two measures reflect two very distinct ways of conceptualizing the idea of ambition. The former method treats ambition as a ‘political’ concept—a measure of what a country is pledging compared with what the nation’s experts think it is obliged and able to pledge. The second measure of ambition is more ‘objective’ in that it reflects a dispassionate, independent assessment of what each country must do. This latter approach often shades into the concept of ‘science-based targets’, which is rooted in the idea that science can instruct political processes in the level of effort needed to control warming pollution. These two measures let us test which of these two theories about the relationship between ambition and credibility—a political approach or a scientific one—might yield the greatest insight into the policies nations actually adopt.
Our regression also includes a measure of expected damages from the impacts of climate change56. Countries vulnerable to higher damages should be more motivated in their ambition to cut pollution and thus to make credible commitments28,29,57,58. Following the rich literature in comparative politics and law6,29,59,60,61,62,63,64 we include measures of the quality of institutions. Looking to the literature on the political economy of energy and industrial policy65,66,67,68,69, we include a measure of the power of the industry most squarely implicated in cutting carbon pollution: fossil fuels. Finally, we control for the type of expert making the assessment, mindful that expertise is domain specific.
Political and institutional factors have a substantial effect on respondents’ expected compliance of their home country (Table 2). The quality of government is significant in all models: higher-quality government institutions lead to more credibility. The type of political system has consistent (if not always significant) sign and suggests that democratic governments are less credible in the commitments they make. Perhaps this reflects the incentives for politicians to make vague treaty commitments that are attractive to voters who are unable to assess credibility and cost70. Credibility also declines with economic strength, which suggests that economic capabilities beyond institutional quality do not increase credibility. (Although not a subject of intense empirical investigation, it has been widely assumed that richer countries would be more reliable participants in complex policy coordination needed to address collective goals like stopping climate change. Our study finds no support for that proposition.)
The ambition of the pledges is significant in all models but in sharply diverging ways. Respondents’ assessments of ambition are highly correlated with credibility, but ‘objective’ measures of ambition by dispassionate scientists are inversely correlated with credibility. The political approach to ambition travels with credibility; the scientific one does not. Also striking is that all these factors are much more influential than the variables that, from an economic perspective, should be decisive: climate change damages and measures of the size of incumbent high-pollution industries (for example, size of fossil fuel rents and CO2 emissions per capita).
Compared with diplomats, scientists are systematically more pessimistic about compliance—a matter that is outside their domain of expertise (see Supplementary Table 38 for a decomposition of types of scientist, which have no statistically significant effect). Experience (measured by the number of COPs attended) also correlates with rosier assessments of compliance, a finding consistent with studies that suggest the characteristics of an expert are learned on the job and may be sociological as much as individual71.
To complement this regression analysis we also asked the experts to assess the weight of different possible explanations for their home country’s compliance with its current NDC (Fig. 3). These possible explanators, taken from the literature and pre-survey pilot interviews, range from factors directly anchored in climate change goals (for example, solving the collective action problem of warming) to those related to international diplomacy (for example, national reputation) to a variety of co-benefits that are correlated with national energy policy but suggest different motivations for national action (for example, addressing local air pollution). Analysts have long noted that wealthy nations tend to view climate change as a crisis meriting action in its own right, but many emerging and developing countries tend to frame climate action within a broader set of ‘sustainable development’ agendas72. Our results are consistent: for the experts from OECD nations, the single most important explanation for compliance is mitigation of climate change. For the rest of the world, boosting economic growth and mitigation of local environmental pollution are more important.
Theories of international cooperation, including those that have motivated the design of the Paris Agreement, have been hard to test with data. The most critical variables—the national motivation to make and honour commitments—have been particularly hard to measure. Tapping experts steeped in the policy process offers a novel source of insight. Our results point to two major observations. First, as widely expected, credibility and ambition are closely related. But our study reveals contrasting perspectives. When policy experts assess ambition, the countries making the boldest pledges are also making the most credible pledges. This result is among the first systematic evidence that the core logic of the Paris Agreement is working. Paris was designed around the idea that by making pledges non-binding—in contrast with legally binding emission targets and timetables of the Kyoto Protocol—national governments would be more flexible to reveal what they are willing and able to implement. That, coupled with periodic review, offered a way to keep pushing for more ambition and credibility in tandem8,10,11. Not only do we find support for that idea, but our result also contributes to the age-old debate over the value of non-binding legal pledges. Non-binding commitments, with the right supporting institutions, can elicit greater adjustment by countries in part because non-binding instruments are more flexible and better able to accommodate uncertainty6,15,23,55,73. By contrast, when pledges are evaluated against what scientists say is necessary—a process focused on the geophysics of the climate, rather than the political realities of what is possible—the result is the trade-off between ambition and credibility that so many analysts have feared15,17.
Second, our study suggests that credibility is a political and institutional story. Within political science there is a rich literature suggesting that institutional quality—including the ability of organized interest groups to exert disproportionate influence on policy processes—should have a large impact on how a country organizes political support for (and against) policies, and also the surety with which a country can implement policies once decided74,75. Related, there are rich literatures around democratic deliberation as a possible explanation for societal support (and thus credibility) concerning an array of national policies, including those aimed at honouring international commitments76,77,78,79. Still other research suggests that countries with strong redistributive institutions—welfare states—may be better able to make credible commitments because they are more capable of managing otherwise wrenching social changes, such as unemployment in declining industries, caused by national policies aimed at implementing international commitments29,80,81,82.
We look at these factors with two measures of institutional type: quality and political system (Table 2). Institutional quality is associated with higher expected compliance in all of our models. The standard measure of political system—Polity IV scores—suggests that in some models less-democratic countries offer more credible pledges after controlling for institutional quality and ambition. This finding may reflect the high degree of administrative and political control that exists within consolidated autocratic governments83. Research on arms control, economic cooperation and other domains has suggested that governments with high administrative quality and a degree of political insulation may stay the course, implementing complex international commitments even in the face of economic and political shocks18,78. Such hypotheses need closer evaluation in the domain of climate change politics and policies.
In contrast with the political and institutional story, we find that most of the conventional economic factors, such as measures of expected economic damages from climate change, emissions levels or fossil fuel dependency, don’t explain much of the assessed variation in the credibility of NDCs. Strikingly, our measures for climate vulnerability are insignificant. Moreover, our regressions suggest that lower gross domestic product (GDP) is also associated with higher expectations of compliance. This may reflect the impact of capacity-building programmes that have been running for decades84,85. In other domains of international cooperation such programmes have played important roles in boosting compliance by low-income nations8,86. Our results point to the need for additional research on how capacity-building commitments for developing countries link to the credibility of those countries’ pledges.
These two results—about the link between ambition and credibility, and the political and institutional processes associated with credibility—suggest directions for policy and research. For policymakers, our results are consistent with other studies that suggest it is important to distinguish Paris-like processes of cooperation that utilize mechanisms of non-binding self-determined pledges from the formal diplomacy of consensus decision-making8. For scholars, this distinction between actual cooperation on climate change and formal diplomacy suggests it is particularly important to understand how international institutions interact with national policy processes, since these processes seem to explain so much of expected compliance, and credibility is central to effective pledge-based diplomacy. One implication for policymakers is that the machinery of international pledge-and-review will need the capacity to assess (and perhaps enhance) credibility, which may be difficult to achieve via formal intergovernmental decision-making that tends to work by consensus. Other kinds of international institutions may be needed, such as those created by groups of committed first movers on climate policy rather than through global consensus. In addition, those institutions might benefit from policy assessments by experts using methods such as those reported here. These are visions for international institutional machinery that are quite different from the standard UN-based processes.
For decades, climate cooperation has been marked by a lot of diplomacy but not much real action because pledges were non-existent, not particularly ambitious or disingenuous. That is now changing, possibly quickly. The crisis in Ukraine, although it unfolded after our survey closed, is plausibly accelerating that action—especially in the region our survey already identified as most exceptional (Europe)87. With the right methods and theories, a rich research agenda is unfolding as we seek to understand that variation and, in policy processes, shape national action towards more cooperative global outcomes.
Data collection and sample
The research was evaluated and approved by the Ethics Committee of the University of Kassel, Germany. All human participants in our study gave informed consent before participation. To obtain the data for our empirical analyses, we invited climate policy experts via email. The invitation email contained a short introductory text and a link to an online questionnaire provided on the QuestionPro platform. The links were personalized to ensure that the questionnaire is filled out only once by each participant. The first invitation to the survey was sent out on 21 September 2020. This was followed by four reminders with an interval of roughly three weeks each. The final reminder was distributed on 5 January 2021. The possibility to take part in the survey ended on 31 January 2021. To incentivize participation, we offered to share preliminary, descriptive results of the survey within four weeks after participation of the respective respondent.
The sample is made up of climate policy experts from two sources: the UNFCCC (Negotiator sample) and the IPCC (Scientist sample). The Negotiator sample is based on the lists of participants published by the UNFCCC after each COP. For COPs 16–25 (2010–2019), email contacts for individuals who were listed as a party member at least once were taken from previous studies or searched for on the Internet. Individuals who attended the COPs as observer only (and never as party) were not included. The Scientist sample consists of authors or reviewers of the Fifth Assessment Report by the IPCC. The list is available on the IPCC website and the email addresses were obtained through Internet searches. In the regression analyses, we always control for whether an individual is from the Negotiator or the Scientist sample.
A total 978 individuals from 162 countries participated in the survey (700 Negotiators, 278 Scientists), meaning that they answered at least some of the questions. A total of 829 individuals answered the questions relevant to this Article (599 Negotiators, 230 Scientists). In our empirical analyses, the number of observations varies slightly across questions because respondents who did not answer a certain question or answered it with ‘I don’t know’ had to be left out. Additionally, respondents were dropped when relevant information for their respective home country was not available to be used as explanatory variable. To calculate the response rate, we set the number of individuals who answered the questions relevant to this Article in relation to the number of individuals who were contacted and verifiably opened the link to the survey (1,768 in total: 1,313 Negotiators, 455 Scientists). Following this approach, the overall response rate is 46.89% (45.62% Negotiators, 50.55% Scientists). There is no other way for us to calculate the response rate because we do not know how many individuals have actually received and seen the invitation. Many of the contact addresses, some dating back to 2010, are no longer valid or active.
Questionnaire and empirical approach
The survey covered different aspects of international climate policy with a focus on the Paris Agreement. An early version of the survey was pre-tested at the Bonn Climate Change Conference (SB 50) in June 2019. The pre-test was conducted in the form of in-person qualitative interviews among six individuals who have been involved in climate negotiations as either party member or observing party. One pre-test participant, with experience as party member, supported the project further in an advisory role to provide feedback in matters of wording, comprehension and content of questions up until the final version of the survey. All survey questions used in this Article can be found in the Supplementary Information. Definitions and summary statistics of the dependent and explanatory variables can be found in Supplementary Tables 21–23.
The questionnaire started with a short introduction describing the subject of the survey. Here, we also provided a data protection declaration (in line with the General Data Protection Regulation of the EU) and obtained the respondents’ consent to process their data. After that, we asked participants to state their home country. Respondents were told to indicate the country whose climate policy they know best. In most cases the indicated home country aligned with nationality (for 88% of respondents in the Negotiator sample, 87% in the Scientist sample) and delegation membership (90% in the Negotiator sample) of the respondent. The main part of the survey was organized into several blocks of questions. The first part relevant to this Article was concerned with the participants’ assessments of the NDCs submitted under the Paris Agreement. These are the outcome variables of interest for our analysis. More specifically, the questions on the NDCs asked participants about the ambition and the expected compliance regarding the NDC of their home country and other countries. Furthermore, we elicited reasons for the fulfilment of the respective NDCs. In a later part of the survey, we additionally assessed the respondents’ views on the consequences of climate change on future living conditions to be used as control variable. In the final part, we obtained information regarding the participants’ personal background, such as gender, age, nationality, the field in which they have obtained their highest degree of training, the type of organization for which they work and the number of COPs they attended as a party member.
To perform regression analyses, we added a wide set of explanatory variables on the country level to our dataset. These variables were matched to the respective participant based on the indicated home country. The added variables used for the analyses presented in Table 2 can be categorized in the following way: geopolitical background, ambition of NDCs, vulnerability, dependence on the extractive fossil fuel industry, type of government. The geopolitical background is given by OECD membership and can be OECD Europe, OECD rest of the world, non-OECD rest of the world. For ambition of NDCs we rely on the assessments made in ref. 28. (This is not the only potential independent source of ambition. Looking beyond the RDP&M study, in Supplementary Tables 19 and 20, and Supplementary Figs. 2–5 we compare our subjective measures of credibility, based on the assessments of experts, with a variety of other metrics, including from Climate Action Tracker88, Germanwatch89 and from the London School of Economics database of national climate laws and policies90.) Vulnerability (expected damages from climate change and other global challenges) is measured by the 2018 ND-GAIN Index56. To control for a country’s dependence on the extractive fossil fuel industry, we included a variable that is the sum of oil, gas and coal rents expressed as share of GDP based on data by the World Bank91,92,93. Type of government is controlled for by using Polity IV scores along with the World Economic Forum’s measure of government quality59,94. All added variables were standardized (mean = 0, s.d. = 1) to ease interpretation.
For some specifications, we alternatively use variables that were derived directly from the survey. For ambition, respondents gave a subjective evaluation of their home country’s NDC (in relation to economic strength) using a Likert-type scale with five answer categories ranging from ‘(1) Not ambitious at all’ to ‘(5) Very ambitious’ and an ‘I don’t know’ option. ‘Ambition (our survey)’ is constructed as a dummy variable that takes the value 1 if the respondent answered with either 4 or 5 on the Likert-type scale and 0 otherwise. Respondents’ subjective expectations about climate change damages (vulnerability) were elicited by asking them to estimate the consequences of climate change on future living conditions up to 2100 for their home country. The assessments were elicited by means of a Likert-type scale with five answer categories ranging from ‘(1) Extremely large damages’ to ‘(5) No damages’ and an ‘I don’t know’ option. ‘Vulnerability (our survey)’ is constructed as a dummy variable that takes the value 1 if the respondent answered with either 1, 2 or 3 on the Likert-type scale and 0 otherwise. The regression analyses additionally included information on the respondent itself that was elicited in the final part of the survey. Here, we controlled for whether a respondent was from the Negotiator or Scientist sample, whether the respondent works for a national government organization, and how often a respondent attended a COP as party member.
The dependent variable in the presented regressions is a respondents’ assessment of the expected compliance with the NDC pledge for the respective home country. The variable was elicited using a Likert-type scale with five answer categories ranging from ‘(1) Not confident at all’ to ‘(5) Very confident’ and an ‘I don’t know’ option. ‘Compliance NDC’ is constructed as a dummy variable that takes the value 1 if the respondent answered with either 4 or 5 on the Likert-type scale and 0 otherwise. Accordingly, we used binary probit models and report the marginal effects at the mean of the other variables. (The results for North America, dominated by the United States, are affected by the US election in 2020. More detail on how the election outcome influenced the assessments is included in Supplementary Tables 43–52. Regardless of electoral timing, we asked for an evaluation of the formal NDC that the United States had submitted back in 2016.)
For the described dependent variable, we present results from four specifications of estimations of binary probit models in the main paper. In the four specifications, we include different combinations of third-party and survey-collected measures of ambition and vulnerability. Throughout the four specifications our main results regarding the geopolitical background, NDC ambition, vulnerability, institutional quality and respondent background are robust.
The Supplementary Information provides further robustness checks, these include the following: Supplementary Table 24 shows the same results as Table 2 in the main paper with additional model statistics. Supplementary Table 25 shows specifications using different combinations of GDP per capita, vulnerability (given by the ND-GAIN index) and quality of institutions testing for multicollinearity issues. Supplementary Table 26 shows robustness checks for different measures of NDC ambition as well as robustness checks for updated NDCs. Supplementary Table 27 shows the results when the number of climate laws and policies that were adopted between 2016 and 2020 are included as additional explanatory variables. Supplementary Table 28 shows the results when the fossil fuel rents are included separately for coal, oil and natural gas. Supplementary Tables 29–33 show the results when alternative measures for fossil fuel dependency (total and per capita production) are used. Supplementary Tables 34–37 show the results for two alternative measures of vulnerability. Supplementary Table 38 and Supplementary Fig. 6 show the results when a dummy variable that indicates whether a respondent is a natural scientist is included in the model. Supplementary Table 39 shows the results from ordered probit models based on the original five-step Likert scale of the variable and Supplementary Table 40 for ambition as dependent variable. The additional regressions in the Supplementary Information indicate that our overall results regarding the geopolitical background, NDC ambition, vulnerability, institutional quality and respondent background are robust. Additionally, it includes descriptive statistics and non-parametric tests (for example, differences by geopolitical background) for all variables presented in Figs. 1, 2 and 3 (Supplementary Fig. 1 and Supplementary Tables 1–13, 41 and 42) and a comparison between our survey measures and measures on these concepts from other sources (Supplementary Tables 18–20 and Supplementary Figs. 2–5).
The project has been approved by the Ethics Committee at the University of Kassel, Germany, where the survey was administered to human subjects. The authors declare they have adhered to all ethical regulations.
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
The dataset necessary to reproduce the findings of this study is publicly available in anonymized form at: https://doi.org/10.7910/DVN/SHF1HU (ref. 95). Climate Action Tracker data are available at: https://climateactiontracker.org/countries/. Germanwatch data are available at: https://ccpi.org/download/the-climate-change-performance-index-2020/ and https://ccpi.org/download/the-climate-change-performance-index-2021/. LSE Climate Change Laws of the World data are available at https://climate-laws.org/. Data used by ref. 28 are available at https://www.nature.com/articles/s41467-018-07223-9 and http://paris-equity-check.org/warming-check.html. GDP per capita are available at https://data.worldbank.org/indicator/NY.GDP.PCAP.PP.KD?most_recent_value_desc=false. ND Gain Index data are available at https://gain.nd.edu/our-work/country-index/methodology/. World Competitiveness Index data are available at https://www.weforum.org/reports/the-global-competitiveness-report-2020. Polity IV data are available at https://www.systemicpeace.org/inscrdata.html. Fossil fuel rent data are available at https://data.worldbank.org/indicator/NY.GDP.COAL.RT.ZS, https://data.worldbank.org/indicator/NY.GDP.NGAS.RT.ZS and https://data.worldbank.org/indicator/NY.GDP.PETR.RT.ZS. CO2 per capita data are available at https://edgar.jrc.ec.europa.eu/dataset_ghg60#p1. Fossil fuel production data are available at https://www.eia.gov/international/data/world/coal-and-coke/coal-and-coke-production, https://www.eia.gov/international/data/world/petroleum-and-other-liquids/annual-refined-petroleum-products-production and https://www.eia.gov/international/data/world/natural-gas/dry-natural-gas-production. World Population data are available at https://data.worldbank.org/indicator/SP.POP.TOTL?view=chart. The data used from ref. 96 are available at https://web.stanford.edu/~mburke/climate/data.html.
The code to reproduce the results is available at https://doi.org/10.7910/DVN/SHF1HU.
Bodansky, D. The United Nations framework convention on climate change: a commentary. Yale J. Int. Law 18, 451–558 (1993).
Figueres, C. & Rivett-Carnac, T. The Future We Choose: Surviving the Climate Crisis (Knopf, 2020).
Keohane, R. O. & Victor, D. G. Cooperation and discord in global climate policy. Nat. Clim. Change 6, 570–575 (2016).
Barrett, S. Environment and Statecraft: The Strategy of Environmental Treaty-Making (Oxford Univ. Press, 2006).
Nordhaus, W. Climate clubs: overcoming free-riding in international climate policy. Am. Econ. Rev. 105, 1339–1370 (2015).
Victor, D. G. Global Warming Gridlock (Cambridge Univ. Press, 2011).
Bulkeley, H. et al. Governing climate change transnationally: assessing the evidence from a database of sixty initiatives. Environ. Plan. C 30, 591–612 (2012).
Victor, D. G. & Sabel, C. F. Fixing the Climate: Strategies for an Uncertain World (Princeton Univ. Press, 2022).
Abbott, K. W. The transnational regime complex for climate change. Environ. Plan. C 30, 571–590 (2012).
Ollivier-Mrejen, R., Michel, P. & Pham, M.-H. Chronicles of a science diplomacy initiative on climate change. Sci. Dipl. 7, 2 (2018).
Victor, D. G. Why Paris worked: a different approach to climate diplomacy. Yale Environment 360 (15 December 2015).
Grubb, M. From Lima to Paris, part 1: the Lima hangover. Clim. Policy 15, 299–301 (2015).
Falkner, R., Stephan, H. & Vogler, J. International climate policy after Copenhagen: towards a ‘building blocks’ approach. Glob. Policy 1, 252–262 (2010).
Oppenheimer, M., Stewart, R. B. & Rudyk, B. Building blocks for global climate protection. Stanf. Environ. Law J. 32, 341–392 (2013).
Raustiala, K. Form and substance in international agreements. Am. J. Int. Law 99, 581–614 (2005).
Barrett, S. Climate treaties and the imperative of enforcement. Oxf. Rev. Econ. Policy 24, 239–258 (2008).
Downs, G. W., Rocke, D. M. & Barsoom, P. N. Is the good news about compliance good news about cooperation? Int. Organ. 50, 379–406 (1996).
Hafner-Burton, E. M., Victor, D. G. & Lupu, Y. Political science research on international law: the state of the field. Am. J. Int. Law 106, 47–97 (2012).
Barrett, S. & Dannenberg, A. An experimental investigation into ‘pledge and review’ in climate negotiations. Clim. Change 138, 339–351 (2016).
Update of Norway’s Nationally Determined Contribution (UNFCCC, 2020); https://unfccc.int/NDCREG
Rajamani, L. & Bodansky, D. The Paris rulebook: balancing prescriptiveness with flexibility. Int. Compar. Law Q. 68, 1023–1040 (2019).
Werksman, J. Remarks on the international legal character of the Paris Agreement. Md J. Int. Law 34, 343 (2019).
Abbott, K. W. & Snidal, D. Hard and soft law in international governance. Int. Organ. 54, 421–456 (2000).
World Energy Outlook 2020 (IEA, 2020); https://www.iea.org/reports/world-energy-outlook-2020
Ou, Y. et al. Can updated climate pledges limit warming well below 2 °C? Science https://doi.org/10.1126/science.abl8976 (2021).
Hale, T. et al. Net Zero Tracker (Energy and Climate Intelligence Unit, Data-Driven EnviroLab, NewClimate Institute, Oxford Net Zero, 2021); https://zerotracker.net/
CAT Climate Target Update Tracker (Climate Analytics and NewClimate Institute, 2021); https://climateactiontracker.org/climate-target-update-tracker/
Robiou du Pont, Y. & Meinshausen, M. Warming assessment of the bottom-up Paris Agreement emissions pledges. Nat. Commun. 9, 4810 (2018).
Tørstad, V., Sælen, H. & Bøyum, L. S. The domestic politics of international climate commitments: which factors explain cross-country variation in NDC ambition? Environ. Res. Lett. 15, 024021 (2020).
Averchenkova, A. & Bassi, S. Beyond the Targets: Assessing the Political Credibility of Pledges for the Paris Agreement (The Centre for Climate Change Economics and Policy, The Grantham Research Institute on Climate Change andthe Environment, 2016); https://www.lse.ac.uk/granthaminstitute/publication/beyond-the_targets/
Morgan, M. G. Use (and abuse) of expert elicitation in support of decision making for public policy. Proc. Natl Acad. Sci. USA 111, 7176–7184 (2014).
Abdulla, A., Hanna, R., Schell, K. R., Babacan, O. & Victor, D. G. Explaining successful and failed investments in U.S. carbon capture and storage using empirical and expert assessments. Environ. Res. Lett. 16, 014036 (2020).
Dannenberg, A., Zitzelsberger, S. & Tavoni, A. Climate negotiators’ and scientists’ assessments of the climate negotiations. Nat. Clim. Change 7, 437–442 (2017).
Dannenberg, A. & Zitzelsberger, S. Climate experts’ views on geoengineering depend on their beliefs about climate change impacts. Nat. Clim. Change 9, 769–775 (2019).
Dannenberg, A., Sturm, B. & Vogt, C. Do equity preferences matter for climate negotiators? An experimental investigation. Environ. Resour. Econ. 47, 91–109 (2010).
Kornek, U., Flachsland, C., Kardish, C., Levi, S. & Edenhofer, O. What is important for achieving 2 °C? UNFCCC and IPCC expert perceptions on obstacles and response options for climate change mitigation. Environ. Res. Lett. 15, 024005 (2020).
Hafner-Burton, E. M., Hughes, D. A. & Victor, D. G. The cognitive revolution and the political psychology of elite decision making. Perspect. Politics 11, 368–386 (2013).
Adelson, B. When novices surpass experts: the difficulty of a task may increase with expertise. J. Exp. Psychol. 10, 483–495 (1984).
Larkin, J., McDermott, J., Simon, D. P. & Simon, H. A. Expert and novice performance in solving physics problems. Science https://doi.org/10.1126/science.208.4450.1335 (1980).
Lin, S.-W. & Bier, V. M. A study of expert overconfidence. Reliab. Eng. Syst. Saf. 93, 711–721 (2008).
List, J. A. Does market experience eliminate market anomalies? Q. J. Econ. 118, 41–71 (2003).
Tetlock, P. E. Expert Political Judgment: How Good is it? How Can We Know? (Princeton Univ. Press, 2005).
Hedinger, B. & Goette, L. Cooperation in the Cockpit: Evidence of Reciprocity and Trust Among Swiss Air Force Pilots (Univ. Zurich, 2006).
Voss, J. F., Vesonder, G. T. & Spilich, G. J. Text generation and recall by high-knowledge and low-knowledge individuals. J. Verbal Learn. Verbal Behav. 19, 651–667 (1980).
Chiesi, H. L., Spilich, G. J. & Voss, J. F. Acquisition of domain-related information in relation to high and low domain knowledge. J. Verbal Learn. Verbal Behav. 18, 257–273 (1979).
Feltovich, P. J., Prietula, M. J. & Ericsson, K. A. in The Cambridge Handbook of Expertise and Expert Performance (eds Ericsson, K. A. et al.) 41–68 (Cambridge Univ. Press, 2006).
Eisenstadt, M. & Kareev, Y. Aspects of human problem solving: the use of internal representations. in Explorations in Cognition (eds Norman, D. A. and Rumelhart, D. E.) 308–346 (Freeman, San Francisco, 1975).
Barrett, S. & Dannenberg, A. Climate negotiations under scientific uncertainty. Proc. Natl Acad. Sci. USA 109, 17372–17376 (2012).
Barrett, S., Dannenberg, A. Sensitivity of collective action to uncertainty about climate tipping points. Nat. Clim. Change 4, 36–39 (2014). https://doi.org/10.1038/nclimate2059
Tavoni, A., Dannenberg, A., Kallis, G. & Loschel, A. Inequality, communication, and the avoidance of disastrous climate change in a public goods game. Proc. Natl Acad. Sci. USA 108, 11825–11829 (2011).
Leiserowitz, A. Climate change risk perception and policy preferences: the role of affect, imagery, and values. Clim. Change 77, 45–72 (2006).
Lee, T. M., Markowitz, E. M., Howe, P. D., Ko, C.-Y. & Leiserowitz, A. A. Predictors of public climate change awareness and risk perception around the world. Nat. Clim. Change 5, 1014–1020 (2015).
Tingley, D. & Tomz, M. The effects of naming and shaming on public support for compliance with international agreements: an experimental analysis of the Paris Agreement. Int. Organ. https://doi.org/10.1017/S0020818321000394 (2021).
Mahajan, A., Tingley, D. & Wagner, G. Fast, cheap, and imperfect? U.S. public opinion about solar geoengineering. Environ. Politics 28, 523–543 (2019).
Victor, D. G., Raustiala, K. & Skolnikoff, E. B. (eds.) The Implementation and Effectiveness of International Environmental Commitments: Theory and Practice (International Institute for Applied Systems Analysis, MIT Press, 1998).
Chen, C. et al. Notre Dame Global Adaptation Index (University of Notre Dame,Notre Dame Global Adaptation Initiative, 2021); https://gain.nd.edu/our-work/country-index/methodology/
Ricke, K., Drouet, L., Caldeira, K. & Tavoni, M. Country-level social cost of carbon. Nat. Clim. Change 8, 895–900 (2018).
Tubi, A., Fischhendler, I. & Feitelson, E. The effect of vulnerability on climate change mitigation policies. Glob. Environ. Change 22, 472–482 (2012).
Marshall, M. G., Gurr, T. R. & Jaggers, K. Polity5 Project, Political Regime Characteristics and Transitions, 1800-2018 (Center for Systemic Peace, 2019); https://www.systemicpeace.org/inscrdata.html
Keohane, R. O. Governance in a partially globalized world. Am. Political Sci. Rev. 95, 1–13 (2001).
Brunnée, J. & Toope, S. J. Legitimacy and Legality in International Law: An Interactional Account (Cambridge Univ. Press, 2010).
Davidson, M., Karplus, V. J., Zhang, D. & Zhang, X. Policies and institutions to support carbon neutrality in China by 2060. Econ. Energy Environ. Policy 10, 2 (2021).
Finnegan, J. J. Institutions, climate change, and the foundations of long-term policymaking. Comp. Polit. Stud. https://doi.org/10.1177/00104140211047416 (2022).
Genovese, F. & Tvinnereim, E. Who opposes climate regulation? Business preferences for the European emission trading scheme. Rev. Int. Organ. 14, 511–542 (2019).
Colgan, J. D., Green, J. F. & Hale, T. N. Asset revaluation and the existential politics of climate change. Int. Organ. 75, 586–610 (2021).
Davidson, M. R. China’s power outage. Foreign Affairs (18 Nov 2021); https://www.foreignaffairs.com/articles/china/2021-11-18/chinas-power-outage
Dolphin, G., Pollitt, M. G. & Newbery, D. M. The political economy of carbon pricing: a panel analysis. Oxf. Econ. Pap. https://doi.org/10.1093/oep/gpz042 (2019).
Wettestad, J. & Jevnaker T. EU emissions trading: frontrunner – and ‘Black Sheep’? in The Evolution of Carbon Markets: Design and Diffusion (eds Wettestad, J. & Gulbrandsen, L. H.) Ch. 3 (Routledge, 2017).
Lamb, W. F. & Minx, J. C. The political economy of national climate policy: architectures of constraint and a typology of countries. Energy Res. Soc. Sci. 64, 101429 (2020).
Battaglini, M. & Harstad, B. The political economy of weak treaties. J. Political Econ. 128, 544–590 (2020).
LeVeck, B. L., Hughes, D. A., Fowler, J. H., Hafner-Burton, E. & Victor, D. G. The role of self-interest in elite bargaining. Proc. Natl Acad. Sci. USA 111, 18536–18541 (2014).
Fleurbaey M., S. et al. Sustainable development and equity. In Climate Change 2014: Mitigation of Climate Change.Contribution of Working Group III to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change(Eds. Edenhofer, O. et al.) (Cambridge University Press, Cambridge, 2014).
Skjærseth, J. B. North Sea Cooperation: Linking International and Domestic Pollution Control (Manchester Univ. Press, 2000).
Hall, P. A. & Taylor, R. C. R. Political science and the three new institutionalisms. Political Stud. 44, 936–957 (1996).
Mildenberger, M. Carbon Captured: How Business and Labor Control Climate Politics (MIT Press, 2020).
Ho, D. E. Does peer review work? An experiment of experimentalism. Stanf. Law Rev. 69, 1–119 (2017).
Karkkainen, B., Fung, A. & Sabel, C. F. Beyond backyard environmentalism. Boston Review (29 July 2014); http://bostonreview.net/forum/charles-sabel-archon-fung-bradley-karkkainen-beyond-backyard-environmentalism
Chayes, A. & Chayes, A. H. The New Sovereignty: Compliance with International Regulatory Agreements (Harvard Univ. Press, 1998).
Mansbridge, J. J. Beyond Adversary Democracy (Univ. Chicago Press, 1983).
Pollin, R. et al. Industrial policy, employment, and just transition. in America’s Zero Carbon Action Plan (eds Crete, E. et al.) Ch. 3 (SustainableDevelopment Solutions Network, 2020); https://www.unsdsn.org/Zero-Carbon-Action-Plan
Powell, D., Balata, F., Lerven, F. V. & Welsh, M. Trust in Transition: Climate Breakdown and High Carbon Workers (New Economics Foundation, 2019); https://neweconomics.org/2019/11/trust-in-transition
Gazmararian, A. & Tingley, D. Embedded Liberalism From the Ground Up: Credibility and Climate Transitions Working Paper.
Bueno de Mesquita, B., Smith, A., Siverson, R. M. & Morrow, J. D. The Logic of Political Survival (MIT Press, 2003).
Stender, F., Moslener, U. & Pauw, W. P. More than money: does climate finance support capacity building? Appl. Econ. Lett. 27, 1247–1251 (2020).
Implementation of the Framework for Capacity-Building in Developing Countries (UNFCCC Secretariat, 2020); https://unfccc.int/documents/210528
Parson, E. Protecting the Ozone Layer: Science and Strategy (Oxford Univ. Press, 2003).
Verleger, P. & Victor, D. G. Why it’s time to start caring much more about clean hydrogen. The New York Times (18 April 2022).
CAT Rating Methodology: Detailed Description of the Climate Action Tracker Rating Methods (Climate Analytics and NewClimate Institute, 2021); https://climateactiontracker.org/documents/874/CAT_2021-09_RatingMethodology_FullDescriptionNewSystem.pdf
Burck, J. et al. Climate Change Performance Index: Background and Methodology (Germanwatch, 2022); https://ccpi.org/wp-content/uploads/CCPI-2022-Background-and-Methodology_neu.pdf
Climate Change Laws of the World Database (Grantham Research Institute on Climate Change and the Environment and Sabin Center for Climate Change Law, accessed 18 July 2022); https://climate-laws.org/
Oil Rents (% of GDP) (World Bank, 2021); https://data.worldbank.org/indicator/NY.GDP.PETR.RT.ZS
Coal Rents (% of GDP) (World Bank, 2021); https://data.worldbank.org/indicator/NY.GDP.COAL.RT.ZS
Natural Gas Rents (% of GDP) (World Bank, 2021); https://data.worldbank.org/indicator/NY.GDP.NGAS.RT.ZS
Schwab, K. & Zahidi, S. The Global Competitiveness Report: How Countries are Performing on the Road to Recovery (World Economic Forum, 2020); https://www.weforum.org/reports/the-global-competitiveness-report-2020
Victor, D.G., Lumkowsky, M. & Dannenberg, A. Replication data for: Determining the credibility of commitments in international climate policy. Harvard Dataverse https://doi.org/10.7910/DVN/SHF1HU (2022).
Burke, M., Hsiang, S. M., & Miguel, E. Global non-linear effect of temperature on economic production. Nature 527, 235–239 (2015).
We thank our partners and the advisory board of the COMPLIANCE project for valuable comments and suggestions during the workshops. Thanks to J.P. Brice Affana, E. Hafner-Burton and B. LeVeck for preliminary conversations about survey design, and to J. Fowler and G. Morgan, eons ago, for sparking interest in what could be learned from expert surveys. Thanks to J. Colgan, J. Finnegan, F. Genovese, J. Green, T. Hale, B. Keohane, M. Mildenberger and D. Tingley for insights into possible explanatory factors around European ‘exceptionalism’ and to B. Keohane, again, for help with the writing. A big thanks to E. Carlton for exceptional research assistance. A.D. and M.L. acknowledge funding from the German Federal Ministry of Education and Research (BMBF) in the Funding Priority ‘Economics of Climate Change’ (COMPLIANCE: 01LA1806B). D.G.V. is funded by the Electric Power Research Institute, UC San Diego and the Scripps Institution of Oceanography.
The authors declare no competing interests.
Peer review information
Nature Climate Change thanks Cynthia Elliot, Fei Teng and Katja Biedenkopf for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Victor, D.G., Lumkowsky, M. & Dannenberg, A. Determining the credibility of commitments in international climate policy. Nat. Clim. Chang. 12, 793–800 (2022). https://doi.org/10.1038/s41558-022-01454-x
This article is cited by
Indigenous Peoples’ rights in national climate governance: An analysis of Nationally Determined Contributions (NDCs)
npj Climate Action (2023)
Nature Climate Change (2023)
Communications Earth & Environment (2023)
Nature Climate Change (2023)