A computational approach to analyzing climate strategies of cities pledging net zero

Abstract

Cities have become primary actors on climate change and are increasingly setting goals aimed at net-zero emissions, which warrants closer examination to understand how they intend to meet these goals. The incomplete and heterogeneous nature of city climate policy documents, however, has made systemic analysis challenging. We analyze 318 climate action documents from cities with net-zero targets using machine learning-based natural language processing (NLP) techniques. We aim to accomplish two goals: (1) determine text patterns that predict ‘ambitious’ net-zero targets; and (2) perform a sectoral analysis to identify patterns and trade-offs in climate action themes. We find that cities with ambitious climate actions tend to emphasize quantitative metrics and specific high-emitting sectors in their plans. Cities predominantly emphasize energy-related actions in their plans, but often at the expense of other sectors, including land-use and climate impacts. The method presented in this paper provides a replicable, scalable approach to analyzing climate action plans and a first step towards facilitating cross-city learning.

Introduction

Since the Intergovernmental Panel on Climate Change’s (IPCC) Special Report on Global Warming of 1.5 °C in 2018, more than 1,000 cities have set their own long-term goals to decarbonize and achieve net-zero emissions. Many set net-zero targets through networks that collate individual pledges, such as the UNFCCC Race to Zero campaign¹ and the Carbon Neutral Cities Alliance². These targets represent a wide range of geographic, socio-political and economic contexts, from megacities with populations larger than 10 million, such as Chengdu and Mumbai, to hundreds of smaller cities and regions on six continents. Public attention on net-zero commitments has increased since 2018, coinciding with key global climate action events, such as the 2018 Global Climate Action Summit, and net-zero announcements by major governments and businesses, such as the UK’s net-zero target announcement in July 2019 and Microsoft’s carbon negative target announcement in January 2020³. Several reports have documented the landscape of net-zero target setting. The Data-Driven EnviroLab and NewClimate conducted one analysis that found at least 826 cities and 103 regional governments have set some form of net-zero target, all of which focus on a particular sector or scope of emissions (i.e., direct emissions)³. In their 2021 analysis, the Oxford Net Zero and the Energy and Climate Intelligence Unit found that 9 percent of all regions in the top twenty-five emitting countries, as well as 13 percent of all major cities with a population over 500,000, have set a net-zero target of some kind⁴. The report notes, however, that a much smaller number of actors have set definitively robust net-zero targets - those aligning with the UNFCCC Race to Zero Campaign’s ‘starting line’ criteria of ‘Pledge, Plan, Proceed, and Publish’⁴. A close evaluation of 25 major corporations’ net-zero climate pledges and action strategies found that most of their actions bordered on greenwashing, falling far short of the promised 100 percent emissions reductions and only collectively achieving around 40 percent emissions reductions by mid-century⁵. These findings fuel ongoing debates amongst diverse stakeholders concerning the credibility of non-state and subnational net-zero pledges and whether ambitiously-declared targets will actually translate into real emissions reductions⁶.

The number, scale and scope of subnational net-zero pledges have grown rapidly in the years since the Paris Agreement officially recognized the contributions of ‘all levels of government’ to climate change efforts⁷, necessitating further examination to understand these efforts’ credibility and impact globally. Studies such as Reckien et al.⁸ and Salvia et al.⁹ have conducted large-scale (n = 885 and n = 327 respectively) content analysis of European cities’ climate action plans, examining their mitigation and adaptation strategies, but limited global analysis exists outside of Europe. Some studies analyze the net-zero targets of individual cities¹⁰ or cities within a single country¹¹ or specific sectors, such as buildings¹², yet limited data availability and inconsistent documentation have hindered broader assessment. Subnational governments may prioritize local adaptation or mitigation approaches⁸ over mainstream climate strategies in sectoral and development planning and policy¹³, making it difficult to analyze and compare strategies. A systematic global analysis of climate strategy documents from cities declaring net-zero targets would shed light on patterns and themes in subnational governments’ climate strategies and possible gaps in action.

In this paper, we collected climate policy strategy documents from cities (full list in Table S5) that have either declared a net-zero emissions reduction target or have signed on to an initiative that commits them to an equivalent target, as determined in one of the first efforts to document city, regional, and corporate net-zero target setting¹⁴. Many subnational governments have published planning climate action and policy documents, which typically provide information on the quantifiable emission reduction target (i.e., percentage emissions reduction from a baseline to a target year), specific sectors a city plans to prioritize in their decarbonization effort, policies and strategies to reach their goals, and the timeline of the implementation of policies. We are limited to cities that have published climate action policy and strategy documents, in total 318 cities out of 823 identified by Data-Driven EnviroLab and NewClimate Institute¹⁴, meaning that this sample is not comprehensive. Since the time of data collection, additional cities have committed to achieving net-zero emissions¹. Despite these limitations, our approach provides key insights into how city governments articulate plans to implement their emission reduction targets. We utilize automated, machine-learning natural language processing (NLP) techniques¹⁵ to examine patterns and trends in cities’ climate strategies and language determinants of ambitious climate actions (e.g. economy-wide net-zero targets). In doing so, we can compare different strategies cities adopt to tackle climate change, as well as specific language patterns and themes that are predictive of cities that have pledged long-term decarbonization goals towards net-zero emissions.

As more attention is given to the concept of decarbonization¹⁶, a debate has ensued over what ‘net-zero emissions’ means in practice and at various scales³. More than 130 national governments have adopted the goal of ‘net-zero emissions’¹⁷ and a growing number of cities, regions, and corporate actors have made net-zero pledges, yet the concept does not yet have real-world definition (Race To Zero Campaign | UNFCCC, n.d.)¹. Article 4 of the Paris Agreement outlines the concept at a global scale, stating that to limit global warming to 1.5 °C, ‘a balance between anthropogenic emissions by sources and removals by sinks of greenhouse gases (GHGs) must be achieved in the second half of this century¹⁸. The IPCC’s Special Report on Global Warming of 1.5 °C explains that all pathways resulting in less than 1.5 °C of warming would require global net-zero carbon dioxide emissions by around 2050, and net-zero GHGs around 2070.

There is no standard universal definition, however, for what net-zero means when applied to individual actors. A survey of the nomenclature applied to efforts to align targets towards the 1.5 °C goal revealed varied terms, including ‘carbon neutral,’ ‘climate neutral,’ ‘zero emissions,’ ‘climate positive,’ among others (Refer to Table 1 in NewClimate Institute & Data-Driven EnviroLab³). Cities and other subnational governments are left to determine their own definitions of net-zero targets, and the resulting climate pledges are heterogeneous, often incomparable, and inscrutable to a public audience attempting to make sense of them. Some organizations suggest that actors should focus on reducing their own emissions first and foremost¹⁹, while others allow for the use of offsets, such as renewable energy credits generated outside of an actor’s jurisdiction or land-based reforestation credits²⁰. The Carbon Neutral Cities Alliance, for example, asks its members to achieve net-zero emissions by reducing total GHG emissions by at least 80 percent by 2050². Likewise, some net-zero targets focus exclusively on CO₂, while others include other GHGs such as methane or nitrous oxide.

Table 1 Summary statistics of cities included in the analysis.

Targets often differ in the breadth of their scopes—some apply to specific sectors, such as buildings, while others apply only to emissions directly produced within a city’s jurisdiction and fail to consider out of boundary or downstream emissions. Considering consumption-based emissions—‘indirect’ GHG emissions generated from city residents’ consumption of goods and services like food, clothing, or electronics—is increasingly considered a necessary component of a credible net-zero target²¹. Some cities, such as Seattle, Vancouver, and London, have tried to account for consumption-based emissions, but they are the exception rather than the norm. Wiedmann et al.²² found that when accounting for consumption-based GHG emissions, 79 of the C40 Cities would overshoot their mitigation targets by 41 percent. Given the wide variation amongst cities’ net-targets, their boundaries, and emissions coverage, projecting the long-term emissions reduction impact of existing carbon neutrality and net-zero efforts is challenging.

Further complicating broad, systematic analysis of cities’ climate policy efforts more generally is the wide diversity in how cities define and prioritize climate action. Some cities, particularly those in the Global South, prioritize adaptation rather than mitigation^23,24, since adaptation efforts typically require activities tailored to particular local contexts and mitigation is frequently viewed as a global, collective-effort problem²⁵. Reckien et al.⁸, in their analysis of 885 European cities’ climate strategies, found far more cities had mitigation plans (66 percent) compared to adaptation plans (26 percent), but that a mitigation plan was not necessarily a prerequisite for an adaptation plan. However, evidence of cities’ interlinking adaptation and mitigation priorities within climate actions also exists. Lee et al.²⁵ examined 261 European cities and discovered a positive relationship between mitigation and adaptation policies, suggesting that at least in Europe cities aim to link global and local climate risks and action, enabling ‘cities to piggyback on their prior mitigation efforts when adopting adaptation policies.’ While our research questions primarily aim to evaluate cities’ net-zero mitigation efforts, these studies suggest it may be challenging and perhaps futile to attempt to disentangle climate mitigation from adaptation. Our analysis, therefore, does not assume the climate strategy documents collected refer to either mitigation or adaptation in isolation.

Results

Predicting economy-wide net-zero targets

Our logistic regression yielded predictors that help us identify traits common among ambitious net-zero plans. We find that cities that have set economy-wide net-zero targets have significantly different (p < 0.01) language use patterns than cities that do not. The top predictive terms of whether a city pledges a net-zero target are identified in Fig. 1, which shows the log(coefficients) for terms that are most highly associated with cities that pledge economy-wide net-zero targets. We group these terms into four descriptive categories or themes to facilitate interpretation: (1) specific, quantitative metrics; (2) identification of emission reduction sources; (3) governance, and (4) human-centric approaches. Mining cities’ climate policy documents for specific mentions of the predictive terms allowed us to better understand the context surrounding each term and common patterns in their usage across cities. Since the terms themselves lack the full context within which they are used in the climate action plans, Supplementary Table S4 provides specific examples from cities’ climate action plans where these terms are mentioned as a way of demonstrating their relationship to climate action. For instance, the term ‘wood’ may not have a clear connection to climate mitigation without its context, such as in Paris’s climate strategy documents, which state a desire ‘to promote the use of materials that emit fewer carbon emissions throughout their life cycles, such as locally produced wood’ (Paris City Government. 2020). We discuss these predictive terms for cities pledging economy-wide net-zero targets (i.e., referred to in this section as ‘ambitious cities’) and their context within our climate action plan corpus.

Fig. 1: Terms predictive of economy-wide net-zero targets.

P-values less than 0.01 indicate significant terms.

Theme 1. Specific, quantitative metrics

• Opportunity reduce

• Percent

• Year use

• Square

• GHG reduction

• emissions generate

• Life cycle

Ambitious cities’ climate plans tend to include specific quantitative metrics in their strategies. Mentions of targets specifying the percentage emissions reduction or increases in budgets, space-based measures like square feet in the context of building footprint or greenspace added, or mentions of a reference year used to understand changes in numbers over time. These cities also identify specific GHG reduction goals, discussing the lifecycle costs and emissions of specific projects, identifying of opportunities to reduce emissions, and defining the scope of emissions generated within city limits.

Theme 2. Emissions sources

• Wood

• Chp (combined heat and power)

• Passive

• Boiler

We find that mentions of actions specifically focused on emission sources tend to be predictive of cities setting economy-wide net-zero targets. Building-related emissions frequently appear as a primary focus sector. In heating, as cities focus on transitioning away from gas boilers, increasing the use of combined heat and power systems is frequently mentioned, as is encouraging passive design of buildings to use solar heating. Wood is mentioned as a construction material, a carbon sink in forests, and a heat source.

Theme 3. Governance

• Mayor

• Fees

• energy plan

Ambitious cities’ reduction targets tend to include or suggest specific governance mechanisms to implement their climate actions. They often include a message from the mayor indicating direct buy-in from the leadership of the city. Many mention either reducing fees for renewable energy projects or introducing fees for fossil fuels, and they contextualize how the city’s energy use fits into a regional or national energy plan.

Theme 4. Human-centered

• Inclusive

• Advocate

Ambitious cities tend to discuss the role that local communities can play in implementing their climate plans. Their plans also discuss quality of life improvements for residents through inclusive public spaces and inclusive economic opportunities. These more ambitious plans often outline strategies for engaging directly with their communities, advocating for needed changes rather than enacting mandates. These results indicate that community consideration and engagement might be an important aspect of climate action plans.

Identifying key themes in cities’ climate action plans

The key term-based topic analysis revealed themes in subnational climate action plans. Overall, we find that cities most frequently emphasize climate mitigation measures focused on energy-related actions (Fig. 2) compared to other topics we analyzed. The frequency of common themes parallels the largest climate emission sources from cities - energy consumption from buildings and the transportation sectors. Nearly all cities tend to discuss strategies aimed at specific energy sources or renewable technologies, as well as actions for the electricity sector. The ‘buildings’ topic was also identified commonly in cities’ plans, including residential building actions and retrofits. Transportation actions focus on increasing pedestrian walkability, as well as mixed modal options, from public transit to cycling. Less commonly mentioned in cities’ action plans were the topics representing land-use, heating, pollution/waste, industry, impacts and offsets.

Fig. 2: Median topic count for cities’ climate action strategies.

In our sample of 318 cities, electricity, building, and transportation were the most prevalent themes mentioned.

We also observe trade-offs in these topics (Fig. 3). Cities emphasize energy often at the expense of discussing land-use and climate impacts, as the two are negatively-correlated (r = −0.44 and r = 0.4, respectively). Similarly, cities that highlight climate impacts are less inclined to focus on buildings (r = −0.36), although the negative association is rather weak. Those that place emphasis on pollution/waste also mention land-use related efforts (r = 0.54), and actions on buildings and energy (r = 0.38) are also complementary.

Fig. 3: Correlation plot for key-term based topics.

Deeper red colors indicate a stronger positive relationship between topics, while darker blue shades indicate stronger negative associations.

From these nine commonly-emphasized topics in cities’ climate action plans, we further reduced the complexity and heterogeneity of cities’ strategies into two dominant factors, which we labeled “Ecology” and “Infrastructure” after inspecting which of the nine topics were most strongly associated with each theme. As Fig. 4 illustrates, the Ecology factor is associated with topics focused on pollution/waste, land-use, and impacts, while the Infrastructure factor is associated with the heating, building, energy and transportation topics. Between these two factors, we observe trade-offs; topics that correspond to high values in one factor tend to correspond to low values in the other factor, and vice versa. Overall, we find that the median city is more focused on urban infrastructure than ecological factors. We also found outliers for both factors and two kinds of outliers: cities that emphasize Ecological considerations more than most cities, and cities that tend to not describe urban Infrastructure-related efforts as much as most cities (Supplementary Fig. 2).

Fig. 4: Ecology and Infrastructure factors.

We identified two factors corresponding to Ecology and Infrastructure themes and observed trade-offs in topics emphasized.

As expected, the emphasis on certain topics and themes over others varies by city. Individual city wordclouds are provided in Supplementary Table S5. To illustrate two cities that contrast diametrically along the two factors of Ecology and Infrastructure, we show representative wordclouds for Munster and New Bedford, with the size of words corresponding to their tf-idf score (i.e., those with higher tf-idf scores appear larger as more relevant words for the text), and the color of words corresponding to their topic class in Fig. 5. From the word clouds we observe Munster’s focus on Infrastructure issues such as parking and building standards, while New Bedford is much more focused on Ecological Considerations like ocean stewardship and watersheds. Munster is highly focused on transportation-related emissions while New Bedford dedicates a significant portion of its plan to climate resilience, such as addressing climate impacts through habitat restoration and green infrastructure.

Fig. 5: Topic word clouds for representative cities.

The cities of A Munster and B New Bedford show differences in the topics and prevalent words for their climate strategies. The size of each word corresponds to its tf-idf score or relative frequency of that term in a cities’ climate plan. The color corresponds to the topic associated with each term.

By plotting the factor scores of these cities across the “Infrastructure” and “Ecology” factors in four quadrants (Fig. 6), we can identify common themes in how certain types of cities plan to tackle climate action. For example, we see that cities in Quadrant II (the top right quadrant) tend to focus somewhat equally on Ecology and Infrastructure approaches to emissions reductions. Cities in this quadrant include a large number of North American cities, such as San Francisco, Austin, Los Angeles, and Chicago, as well as others, such as Singapore and Tokyo. Examining the plans of these cities can help us better understand why they fit into this quadrant. San Francisco, for example, has high values on both the Ecology and Infrastructure factors. Its plan provides detailed discussion of waste and pollution issues, specifically citing commitments to reduce waste by 15% and disposal through landfill by 50% by 2030 as part of its larger goal of achieving net-zero emissions by 2050. At the same time, the plan also demonstrates consideration for transportation and industry issues, including its goal of achieving complete electrification of all vehicles by 2050²⁶.

Fig. 6: Quadrant plot of Ecology vs. Infrastructure.

Cities’ climate strategies fall into one of four quadrants indicating their approaches and possible trade-offs in addressing climate change.

Cities in the top left quadrant (Quadrant I) tend to have a strong focus on Infrastructure issues, but less consideration for Ecology-related themes. London, for example, falls neatly into this quadrant because its plan extensively focuses on building and energy strategies, but has a low score for considering climate impacts or land use. Indeed, the city’s plan highlights that it aims to achieve net-zero emissions mainly through improved efficiency in its transport, building, and energy networks. These topics are also important in determining Philadelphia’s positioning—this city’s plan describes energy and building strategies, but performs relatively poorly in considering land use issues or climate impacts. We see the city’s industrial focus even in the title of its plan: “Powering Our Future: A Clean Energy Vision for Philadelphia.” This document by no means ignores Ecological considerations and reports that achieving 100% clean energy is critical for meeting the city’s goal of 80% reduction in greenhouse gasses by 2050—yet it focuses heavily on showing how clean energy will benefit the economy by “creating local jobs, lowering utility bills, and improving air quality for Philadelphians”²⁷. Other cities in Quadrant I with similar economic emphasis in their plans include Munster (highlighted in Fig. 3) and Fontanafredda.

Cities in the bottom left quadrant (Quadrant III) tend to show little consideration for either Infrastructure issues or Ecology as we have classified them. Of note is that cities in this quadrant tend to share less detail overall in their plans, typically due to the length or format of their reporting documents. The Belgian municipality of Izegem²⁸, for example, delivers merely a one page summary report, while the Chilean capital of Santiago²⁹ and Toronto share an eight page report and a 13-slide summary presentation, respectively³⁰. Some capital cities are among those without detailed plans for achieving net-zero emissions, particularly in 2020 - the year that countries are expected to ratchet up their commitments.

Cities in the bottom right quadrant (Quadrant IV) tend to have a strong focus on Ecology and a less prominent focus on Infrastructure. This quadrant includes cities such as New Bedford (highlighted in Fig. 3), Tainan, Vancouver, Durban, Buenos Aires, and Sydney. Looking at Tainan and Vancouver, we can find that cities in this quadrant tend to have plans that feature extensive discussion of land use and pollution or waste issues, but neglect the topics of heating and energy. For example, Vancouver’s plan is a comprehensive ninety-three page document with ten chapters corresponding to ten environmental goals, but only six pages involve discussion about developing a new “green economy” within the city - most of the other chapters focus on topics such as zero waste or improving access to nature³¹. Not all of these cities have high values for their plans’ representation of climate issues specifically - in many cases, a strong focus on land use is what explains their position in this quadrant.

A few cities, including Adelaide, New York City, Cape Town score similarly on the two factor dimensions, suggesting that their climate action plans are more balanced in their approaches. Examining these cities’ wordclouds and boxplots (Supplementary Table S5), Adelaide’s climate action plan’s keyterms emphasize an even balance of both Infrastructure and Ecological terms, including ‘greening,’ ‘carbon sequestration,’ and ‘offset’ reflecting ecological terms and ‘tram’, ‘metro’ and ‘vehicle’ and ‘charging’ reflecting infrastructure terms. Their factor scores are nearly identical for both components (0.608 for ecology; 0.670 for infrastructure), which would place the city near the center of Fig. 6. New York City (0.622 and 0.645) is another city that also showed fairly equal representation in our two factors. Its plan reflects a number of the topical themes, with an emphasis on building retrofits and heating and cooling systems, while mentioning ‘vulnerable’, ‘preparedness’, ‘flooding,’ and ‘risk.’

Through this system of four quadrants, we are able to better understand the variety in climate action strategies that cites are adopting globally. To investigate why individual cities might be prioritizing certain strategies over others, we examined relationships between cities’ location in these quadrants and city characteristics (area and population), climate zone³², baseline emissions per capita, and regional location³³. While no significant differences were identified across population or area measures, as shown in Supplementary Fig. 3, the greatest statistical differences in the Ecology factor scores appear to be amongst climate zones and geographic region (p < 0.001). Cities located in the Tropical zone tend to more strongly emphasize the Ecology factor, while cities located in Arid climate regions on average reflect Infrastructure approaches more strongly in their climate strategies. Cold and Temperate cities appeared to be more similar along these factors. Tentatively, we suggest that these patterns may result from differing climate-based capacities to engage with the nine commonly-identified topics identified across plans. For instance, in Tropical cities such as Buenos Aires where there is abundant greenery, there is high potential for climate action associated with ‘land use’ - a topic categorized under Ecology that encompasses activities such as reforestation, biodiversity preservation, and sustainable agriculture. By contrast, cities in more Arid climates, such as Somoniyon, Tajikistan, may find that it is more relevant and urgent to focus on topics such as ‘heating’ and ‘energy,’ since substantial emissions are likely associated with constructing buildings that offer protection from extreme heat or cold.

Geographic region was another possible determinant of cities’ positionality in the four quadrants. While cities located in different geographies tended to be similar in the ecological dimension, cities in Europe on average tended to de-emphasize it. The degree to which cities in different geographic regions reflected infrastructure-based approaches varied significantly (p < 0.001), with cities in Eastern Europe and Central Asia exhibiting the greatest diversity (see Data Availability for cities’ metadata file that details which climate zone and region each city is classified within).

Discussion

More than 11,000 subnational governments³⁴ are pledging voluntary climate action, and more cities are expected to follow. Scientific and political calls for global decarbonization and individual actors to ‘race’ (to establish their own plans and strategies to align climate efforts to meet these goals) are further driving more participation. Alongside these efforts are increasing demands for greater accountability of non-state and subnational actors to demonstrate credible actions and progress to meet their climate goals³⁵. A critical component of substantiating these pledges will be more scrutiny of cities’ climate policies and strategies, which is a challenging task given the heterogeneity of their reported formats, languages, and availability³⁶. Some actors make net-zero pledges with absent or vague emission reduction targets, or neglect key emission sectors or sources relevant to their overall climate footprints. Systematically evaluating the plans of cities pledging net-zero targets to uncover which themes and terms may be predictive or suggestive of credible action is a first step in distinguishing cities that will deliver and those that may be greenwashing.

Successful climate action plans require integrated approaches to climate change mitigation, although urban climate researchers have identified challenges in doing so due to a range of knowledge gaps. For instance, Mi et al.³⁷ point to limited understanding of the roles of urban sectors in climate change mitigation, as well as gaps in knowledge of how cities select climate mitigation strategies amidst local challenges. Although our study does not delve into the motivations and decision making behind cities’ climate action planning, it does provide a landscape analysis of common themes and patterns in existing climate action strategies for cities to begin to identify potential opportunities for learning (see³⁸). A primary advantage of the machine-learning based natural language processing (NLP) approach is achieving a replicable, scalable method for broad pattern and theme analysis of heterogeneous, diverse climate action plans as a starting point for further, detailed examination. Undoubtedly, our approach has its own limitations in providing a broad overview rather than a deep dive into particular motivations or drivers of cities’ climate action plans (see limitations discussed below). Nonetheless, this high-level analysis has yielded key findings with implications for the broader study of subnational climate actions below.

Common characteristics in ‘ambitious’ climate action plans point to specificity and governance. ‘Ambitious’ climate action plans - quantified, economy-wide net-zero emissions target of 80 percent or higher - tend to emphasize specific metrics and targeted sectoral actions. While this conclusion may seem obvious, historically the connection between urban emissions inventories and climate action plans has been less certain. Boswell et al.³⁹ note that only about 80 cities in the United States at the time had developed climate action plans based on greenhouse gas emissions inventories, echoing a gap Yalçın et al.⁴⁰, highlighted in their analysis of European municipal or ‘territorial’ climate action plans. Other studies have highlighted the difficulty of directly causally linking target setting, climate policy responses, and emissions reductions^8,10,41. One of the main challenges for cities in developing action plans is determining long-term quantified mitigation targets, which leads municipal authorities to frequently adopt national, regional or international targets for their climate plans⁴⁰. Two challenges emerge with respect to this phenomena - targets are frequently set without any understanding of whether a city has the capacity to meet them; or secondly and conversely, cities underestimate their true emissions reduction potential.

Although we found that mentions of specific quantitative indicators, such as specific emission reduction targets, reference years and target years, among other metrics, were predictive of cities setting net-zero or equivalent emission reduction targets, this result does not necessarily imply that the cities will take the necessary implementation measures to achieve these goals. Hsu et al.⁴² found in an evaluation of just over 1,000 cities participating in the European Covenant of Mayors for Climate and Energy that setting more ambitious, longer-term emission reduction targets were weakly negatively or barely associated with performance. Therefore, it is significant that governance-related language, such as the involvement of citizens in addition to top-level leadership from mayors, was also predictive of ambitious, net-zero equivalent targets. In fact, we found language and terminology related to the word ‘mayor’ to be the strongest predictor of whether a city has pledged a net-zero or equivalent emission reduction goal. Through manual inspection of the individual strategies themselves, we find that some of the ambitious cities’ plans feature forwards or introductory remarks from the city mayors themselves or specific mention of mayoral support and buy-in. This trend is consistent with previous studies that have found that ‘active local politicians’ and ‘a supportive local community’ are key enabling factors of ambitious climate policies. It also echoes other research highlighting that municipal climate action plans tend to emphasize self-governance, as opposed to top-down or more state-centralized modes, as a means of enacting climate policy^43,44, although there are questions surrounding the effectiveness of this approach⁴⁵.

Dominant emphasis on energy-related emission sources and actions could obscure key mitigation opportunities. Our results found that cities are predominantly focused on the energy sector in their climate action plans. Since energy-related emissions from stationary energy sources (e.g., fuel combustion and electricity use in residential and municipal or commercial buildings) comprise the majority or more of cities’ greenhouse gas emissions⁴⁶, it is unsurprising that we found energy-related climate actions to be the most commonly discussed sectors across our sample. The emphasis on these energy-related emissions, however, could distract from some important opportunities for emissions reductions. As Yalçın et al.⁴⁰ identify, ‘local climate mitigation initiatives often focus on win-win measures in the field of energy saving,’ and may overlook some key sectors responsible for local emissions, such as consumption-based or Scope 3 emissions. Harris et al.⁴⁷ found that when accounting for consumption-based emissions in 10 European cities, these emission sources were twice as large as production-based emissions that cities typically report and base their climate action plans on. Fewer than 10 percent of the cities we analyzed specifically mentioned ‘Scope 3’ emissions in their climate action plans. Cities falling under Quadrant III of our factor analysis - articulating climate strategies that do not heavily emphasize language in the Ecology or Infrastructure themes - may warrant further investigation to understand whether they may be focusing on some of these critical gaps that other cities are not.

Trade-offs in topical foci limit opportunities for cross-sectoral synergies. Our observation of trade-offs in certain topics in cities’ climate actions further reinforces cities’ tendency to silo climate actions rather than opt for integrated approaches. Cities’ climate action plans predominantly emphasize language and strategies related to Infrastructure often at the expense of ‘Ecology’-related strategies, although these trade-offs are not as pronounced for cities in some climate zones or geographies compared to others (Supplementary Table S3). Our finding that energy and building-related activities are discussed at the cost of climate impacts suggests cities are missing key opportunities to articulate mitigation actions in tandem with adaptation. Lee at al.²⁵ suggest that cities addressing both mitigation and adaptation together can capitalize on shared capacity building, more efficiently utilizing limited resources and taking advantage of synergies between the two, arguably equally important, climate concerns for cities. As Otto et al.⁴⁸ note, city officials often treat mitigation and adaptation separately, developing distinct strategies and seldomly considering their integration.

The trade-off observed between the land-use and energy and transport topics suggests cities are also failing to integrate spatial planning as a critical tool for climate mitigation⁴⁹. For example, incorporating spatial considerations into climate planning can result in deeper greenhouse gas emissions reductions since it can affect transportation demand and overall energy demand³⁸. The opportunity for cities to integrate spatial planning into their climate action plans is an opportunity urban climate scholars point to as a key mode for accelerating innovation⁵⁰. Kilkis (2019) found in a benchmarking analysis of 120 cities that cross-sectoral approaches integrating efforts across energy, water and environmental systems have helped cities successfully achieve multiple sustainable development and climate goals.

Finally, there are several limitations to our study. In terms of regional diversity, most of the cities that have published climate action plans are located in the Global North. This gap is one that has been echoed in previous studies of subnational climate action⁵¹, given the longer experience of cities in Europe and North America in participating in transnational climate initiatives and developing climate action plans^39,40. We are also limited to analyzing cities that have published a climate action strategy or net-zero climate action plan. Since this is a rapidly developing landscape, with more than 733 cities signed on to the UNFCCC’s Race to Zero campaign as of November 2021, we know that more cities have developed climate action plans, with specific net-zero strategies. For example, between 2021 and 2022, the Net Zero Tracker, a dashboard that assesses net-zero commitments of cities with a population greater than 500,000, shows that the number of cities with net-zero targets found and articulated in policy documents increased from 63 to 88¹⁷. Besides the rapidly evolving landscape of subnational climate action, another limitation of our analysis is limited information regarding the outcomes or implementation of these strategies. Indeed, simply adopting a emission reduction target or articulating a climate strategy does not necessarily imply a city has taken necessary action to implement them. Our study, therefore, represents an analysis of content rather than outcomes and does not predict whether ambitiously-articulated climate plans result in achievement of results. That being said, Hsu et al.⁴² found a significant, positive relationship between cities articulating energy efficiency actions in their 2020 climate plans and mitigation performance, suggesting that in some cases a correlation exists between short-term mitigation achievement and plans.

Despite these limitations, our study demonstrates the utility of machine-learning aided text analysis techniques in systematically analyzing climate action plans and provides a scalable approach that can be replicated as more cities join global efforts to set their own net-zero and decarbonization goals.

Methods

Social scientists have, over the past decade, increasingly used automated Natural Language Processing (NLP) methods that rely on machine learning algorithms to systematically analyze content of text data^15,52,53. By sourcing quantitative data from text documents, researchers are able to apply automated techniques for both prediction and causal inference. These techniques can be grouped into two categories: supervised approaches, where a researcher specifies a priori a conceptual framework to analyze a text corpus; and unsupervised approaches, where an algorithm identifies patterns and trends in the data⁵⁴. Examples of the latter include topic modeling, which has been used to examine linkages between national Paris Agreement pledges and non-state and subnational actors Hsu et al.⁵⁵ as well as similarities among climate actions of cities, regional governments, companies and national actors³⁶.

For this study, we apply NLP approaches to accomplish two primary goals: 1) determine text patterns that predict ‘ambitious’ net-zero targets, where we define an ambitious target as one that encompasses a subnational government’s economy-wide emissions (i.e., do certain language or textual patterns distinguish between cities that set economy-wide targets and those that do not); and 2) perform a sectoral analysis to identify patterns and trade-offs in climate action themes (i.e., land-use, industry, buildings, etc.). This analysis helps us compare and contrast the ways cities address different emissions sources and examine patterns among subnational governments’ climate strategies.

Preprocessing

The Data-Driven EnviroLab and NewClimate Institute⁵⁶ developed a first analysis of city, regional and corporate net-zero targets and recorded 823 cities by May 2020 that had some form of net-zero pledge, whether through a transnational climate initiative or through an individual policy action. Between May and July 2020 we conducted Internet searches to determine whether these 823 cities had publicly-available climate strategy and policy documents. These documents were collected by conducting Internet searches of each city’s name plus keywords including, ‘climate action plan,’ ‘net-zero plan,’ and ‘carbon neutral plan.’ In the time since the collection of these documents, some cities have updated their climate plans or revised their net-zero targets. In total, we collected documents that represent 318 cities, which are predominantly located in North America (n = 78) and Europe (n = 202) (see Fig. S1). This sample is not geographically balanced or representative of cities worldwide, but since the majority of cities pledging net-zero targets are disproportionately located in the Global North⁵⁶, our sample is similarly biased. Because the number of cities making net-zero pledges has been increasing exponentially since efforts like the UNFCCC’s Race to Zero campaign launched in June 2020, we acknowledge that our analysis does not include a comprehensive sample of every city pledging net zero’s climate action plan. We further recognize that there are some cities that have, since the time of data collection, updated their climate action strategies and these changes are not captured in our analysis (see our discussion of limitations).

The format of these documents varies substantially, with lengths ranging from a few pages, such as the city of Fremont’s Resolution No. 2019-03, to Barcelona’s 164-page climate action plan that lays out a roadmap for climate action to 2030⁵⁷. The content of these documents also varies substantially, exhibiting the aforementioned diversity generally observed among net-zero targets. The plans use a wide variety of terms to describe their targets including, for instance, Cape Town’s commitment to ‘carbon neutrality’ by 2050 and Tokyo’s promise of a ‘zero emission’ city⁵⁸. The timelines laid out in these documents also vary from plan to plan. Copenhagen, for example, pledges to reach net-zero emissions by as early as 2025, and has updated its plan with renewed ambition since document collection⁵⁹. Meanwhile, the majority of other actors aim to reach net-zero by 2050. Some plans focus on a subset of years within larger pathways, such as Portland’s Climate Action Plan Progress Report, which charts the city’s proposed actions through 2030 along its larger journey to net-zero emissions by 2050⁶⁰. Some cities had more than one climate policy or strategy document, and so we concatenated these documents together so that every city had one document corpus for the analysis.

Prior to analysis, the raw documents were first converted into a text file through optical character recognition (OCR) using the pytesseract (version 0.3.8) implemented in python⁶¹, and non-English documents were translated to English using the Google Translate API. Any document that was less than 50 characters in length was excluded from analysis, leaving a total of 318 cities. To pre-process the text data corpus, the text was tokenized (split into words) and lemmatized (converted to root form), and stopwords were removed, including punctuation and numbers, proper nouns, pronouns. Next, the text was converted into a numerical representation suitable for analysis, referred to in NLP methods as text featurization or tf-idf (term-frequency-inverse document frequency) featurization. Tf-idf is a commonly-used NLP method to convert text to a numeric representation that ‘assumes the importance of a term relative to a document is inversely proportional to the frequency of occurrence of this term in all the documents’^62,63. In other words, a commonly appearing word in one text would have a low tf-idf score if it appears frequently in other documents and conversely, a word that appears frequently in one document that does not tend to be common in all documents would have a tf-idf score closer to 1. To calculate tf-idf scores for the text featurization process, we computed word-count vectors of 1-grams (word) and 2-grams (word pairs), filtering terms that are in less than 10 percent of documents, and then normalized the counts by dividing by each term’s frequency in the whole corpus. Using this method, we produced a numerical representation that allows us to understand how important a term is to a document corpus.

Predicting ‘ambitious’ economy-wide net-zero targets

In addition to collecting climate policy and strategy documents, we collected publicly-available information detailing cities’ quantitative emission reduction targets (see Supplementary Table S1) to identify whether a city has made a pledge to decarbonize or reach net-zero emissions. We coded each city in our database as pledging a net-zero target according to whether the city had signed on to the UNFCCC’s Race to Zero¹ campaign, which is a coalition of initiatives that commit members to net-zero targets, or if they have specifically mentioned ‘carbon neutral’ or ‘net-zero’ as part of their commitment. We also classified subnational actors that commit to at least an 80 percent emissions reduction target as those with an economy-wide net-zero target. Subnational actors that only pledge to achieve net-zero emissions for government operations are not considered to have an economy-wide target. In total, 242 out of 318 cities in our database were identified as pledging an economy-wide net-zero target.

We used this binary identifier (i.e., whether a city has pledged an economy-wide net-zero target) to identify patterns by training a logistic regression-based model applied to the tf-idf text features of the climate action plans using the scikit-learn package⁶⁴. Logistic regression is a common statistical model used in NLP studies to predict the probability of a document that belongs to one of two binary classes⁶⁵. By evaluating the text the model uses to make these predictions, we can uncover statistically significant patterns associated with subnational governments that are ambitious (i.e., have pledged economy-wide net-zero targets) in their climate actions.

Our model specification is in Eqs. (1) and (2), and our objective function used to fit parameters to the model is represented in Eqs. (3) and (4).

$$\log \left( {\frac{{p_i}}{{1 - p_i}}} \right) = \beta _0 + \mathop {\sum}\nolimits_j {\beta _jX_{ij}}$$

(1)

$$p_i = P\left( {Y_i = 1} \right)$$

(2)

$$\mathop {{\min }}\limits_{\beta _{0,\beta }} \lambda \left\| \beta \right\|_1 + \mathop {\sum}\limits_{i = 1}^n {c_i{{{\mathrm{log}}}}\left( {e^{\left( { - Y_i\left( {X_i^T\beta + \beta _0} \right)} \right)} + 1} \right)}$$

(3)

$$c_i = \mathop {\sum}\limits_{j = 1}^n {\frac{{{{{\mathrm{II}}}}_j}}{n},\,{{{\mathrm{II}}}}_j = \left\{ {\begin{array}{*{20}{c}} {1\,Y_j = Y_i} \\ {0\,Y_j \,\ne\, Y_i} \end{array}} \right.}$$

(4)

• X_ij is a matrix of the tf-idf bi-gram term frequency features j for each city i;

• p_i is the probability that a given city has an economy wide net- zero target.

• β₀ is the intercept, representing the log of the odds for the average city;

• β_j are coefficients assigned to each term representing the contribution of that term to the prediction of whether a city has an ambitious climate plan or not.

• Y_i for each city i is a binary outcome variable whether a city pledges an economy-wide, net-zero target;

• λ is the L1 regularization parameter (explained in more detail below);

• n is the total number of cities

• c_i is a reweighting factor we apply to correct for an imbalance in the distribution in the dataset (i.e., 76 percent of the cities in the dataset have an economy-wide net-zero target).

Since our document corpus has significantly more features (i.e., terms in our vocabulary) than datapoints (i.e., cities), there is risk that the logistic regression model overfits the data. To address potential overfitting due to the model complexity, we apply L1 regularization⁶⁶, a technique that is frequently applied in machine learning and neural networks that reduces a model’s complexity to avoid overfitting⁶⁷. L1 regularization forces the model to use a small set of terms to predict net-zero targets. It achieves this by applying a penalty to the parameters in this regression to force the model to use only the most predictive terms and ignore the vast majority of the 8,000-word vocabulary features. This step makes the model interpretable, allowing us to identify a small set of terms that best captures what predicts certain plans to be more ambitious than others.

To test the accuracy of the model, we refit the model on 50 different train-test splits (i.e. randomly sampled subsets of the data), calculated out-of-sample accuracy using leave-one-out cross-validation, and averaged the out-of-sample performance metrics from each of these fits. We then average the coefficients from these different fits to ensure that the resulting predictors are robust to sampling error. Finally, we generated p-values for each of the terms used by our model using a chi-square test of independence. The results of the model performance evaluation are detailed in Supplementary Table S2. We report accuracy metrics ranging from 0 to 1, including binary f1-score, precision, and recall averaged over 50 out-of-sample test sets. Because out of sample binary classification accuracy is the primary metric by which we evaluate logistic regression, these results indicate that the model we learn is robust to unseen data and therefore useful for interpretive analyses of the coefficients.

Key term-based topic analysis

To examine common themes among these cities’ climate strategies, we developed keyword lexicons (Supplementary Table S3) associated with primary greenhouse gas-emitting sectors where cities are likely to be pursuing emissions reductions. We started by defining this set of nine sectors based on the most common emission sources for cities⁶⁸ as well as the most probable topics and sectors for cities identified in Hsu and Rauber’s³⁶ text analysis of city, region, company, and country climate actions, such as climate impacts and offsets. These topics include:

• Land use

• Industry

• Buildings

• Transportation

• Electricity

• Heating

• Waste/pollution

• Climate Impacts

• Offsets

We then gave each of these nine categories seed terms (e.g., a seed term for the electricity sector might be ‘renewable’), which we used to find other similar terms for each category. We accomplished this using using word2vec similarity⁶⁹ (implemented using the spacy package in python), which is a computational measure of similarity between words based on how they are used in a large corpus⁷⁰. Combining a high quality set of seed terms inspired by the prior work on cities’ biggest emissions sources with this word2vec similarity search allows us to manually curate a large set of representative key terms for each topic (Supplementary Table S3). This approach was inspired by dictionary-based methods (as described in Gentzkow et al.¹⁵), where a researcher manually curates terms based on domain knowledge and uses those terms to form numerical representations of text that highlight features the social scientist cares about. In our case, this process produces manually curated lexicons aided by word vector models that can detect discussion of different emissions sources, allowing us to determine the topics of interest. We then construct a topic vector for every climate plan with nine topics each, where every element of the vector is the number of terms in a plan associated with a certain topic, thereby producing a topic-based representation of the climate plans.

We used these vectors to delineate patterns associated with how these topics are discussed in the net-zero plans. We explore these associations in three ways. To first understand how much the average city is talking about each topic, we measure the median topic count (i.e., how many words belonging to each topic is mentioned) over all cities. Second, we use factor analysis⁷¹, a statistical dimension reduction technique used to compress variability among observations with many variables into a smaller number of unobserved variables called latent factors. These latent factors are based on groupings of words (i.e., word frequencies) that can reveal underlying themes explaining what is driving a city’s focus on certain scopes of climate issues. Finally, we plot the distribution of cities along these latent factors to understand the level of variability in the strategies cities are taking. We used Koppen-Geiger climate zone classifications³² emissions and population data from the EU Covenant of Mayors for Climate and Energy and CDP (see Supplementary Table S1), area data³⁴, and regional classification data from the World Bank³³ to conduct analyses to determine relationships between city characteristics and their positionality along the latent factors identified. We applied one-sided kruskal-wallis tests of analysis of variance to compare differences between group means.

Software

Statistical analyses were conducted using the R statistical programming environment (Version 3.6.2) and in python using the libraries identified in the Methods section.