Introduction

Misinformation about climate change causes a number of negative impacts. It reduces public support for mitigation policies1 and thwarts efforts to communicate accurate information2. Misconceptions about the prevalence of contrarian views have a self-silencing effect3. While misinformation has an overall impact of reducing climate literacy1, this effect varies across the political spectrum, resulting in exacerbated polarisation4.

Social media platforms have become an active site for the spread of misinformation on a wide range of topics and have received increased scrutiny for their role in undermining trust in scientific and journalistic expertise5. At the same time, these platforms are becoming an increasingly influential source of news and information that have an important role in shaping public awareness and discussion of issues of social importance6. The decentralized and networked character of the internet lowers the barriers to posting and sharing misinformation, which ends up being further amplified by engagement-maximizing commercial algorithms7. Regulatory regimes that protect social media platforms from editorial responsibility contribute to the “wild west” information environment in which contrarian claims circulate alongside and often more widely than traditional forms of journalistic and scientific consensus6. Social media also serves as a conduit for mainstreaming contrarian claims when their prevalence online results in their being taken up by news outlets and political actors8. The problems caused by the spread of misinformation online are likely to be exacerbated by recent advances in generative artificial intelligence. As an executive of a company that tracks misinformation online put it, “Crafting a new false narrative can now be done at dramatic scale, and much more frequently—it’s like having A.I. agents contributing to disinformation”9.

Climate change has long been a key target of misinformation on social media platforms. Analysis of tweets around COP climate summits found that contrarian tweets and polarization have substantially grown since 200910. Geographically, hoax-themed tweets that question the reality of climate change are more prominent in conservative U.S. states, relative to liberal states or tweets from the UK, Canada, or Australia11. An analysis of tweets about a 2013 Intergovernmental Panel on Climate Change (IPCC) report found that Twitter users unsupportive of climate science or policies were most active in sending tweets about the IPCC, with uncivil tweets being the most viral contrarian tweets12. Similarly, tweets that are skeptical about climate change have been found to show tones of incivility13, and climate change deniers on Twitter use aggressive language and negative sentiment14.

As automated systems contribute to the generation and circulation of contrarian claims, there will be an increased need for automated detection, tracking, and response. One result will be increased pressure on journalists, platforms, watchdogs, and regulators to find ways of keeping pace with the spread of such claims. For the purposes of addressing the challenges posed by contrarian information, it is useful to be able to determine the nature of false claims. Doing so makes it possible to provide a response that addresses the substance of the claim. The ability to identify and categorize claims also makes it possible to determine the prevalence of different types of misinformation in order to shape “pre-bunking” strategies for inoculating the public against particular categories of false claims2.

It is imperative that interventions are developed and deployed to counter these negative impacts. However, this is made challenging by the fact that misinformation spreads through social media faster than factual information15. Further, once misinformation has taken hold, it is difficult to dislodge—a phenomenon known as the continued influence effect16. Consequently, solutions that can detect and respond to misinformation in a rapid fashion are required.

However, automatic detection and correction of misinformation are technically challenging, earning the label ”the holy grail of fact-checking”17. There have been efforts to automatically detect and fact-check misinformation across various domains18,19. On climate misinformation, there have been few efforts to detect misinformation. Unsupervised topic analysis has been employed to identify the major themes in conservative think-tank (CTT) texts20, link corporate funding to polarizing climate text21, and identify climate framings in newspaper articles22. There have also been efforts to detect logical fallacies in climate misinformation as well as across general topics23,24,25.

The CARDS (Computer Assisted Recognition of Denial & Skepticism) model used supervised machine learning to detect and categorize contrarian claims about climate change26. The model has been shown to be effective in categorizing a wide range of contrarian claims about climate change. It was trained to classify contrarian claims based on the taxonomy illustrated in Fig. 1.

Fig. 1: CARDS taxonomy of contrarian climate claims.
figure 1

This taxonomy provides a comprehensive overview of the frequently employed main claim and its corresponding subarguments utilized to bolster contrarian perspectives on climate change26.

Full size image

However, the CARDS model was only trained using text from contrarian blogs and conservative think-tank websites—prolific sources of climate skepticism—and its performance in classifying content from other datasets (e.g., from social media platforms) has yet to be assessed. This study evaluates and enhances the CARDS model’s performance in classifying contrarian climate claims in Twitter data. We apply the Augmented CARDS model to a dataset of climate tweets, in order to examine the various arguments that are characteristic of different types of contrarian peaks. We trained a model with a superior performance to classify the taxonomy in Fig. 1. Nevertheless, the content of contrarian claims on Twitter primarily consists of five main categories: (1) global warming isn’t happening, (2) humans aren’t causing global warming, (3) climate impacts aren’t bad, (4) climate solutions won’t work, and (5) the climate movement/science is unreliable. At the second level of this taxonomy are sub-categories of contrarian claims, such as 5.2 (climate actors are unreliable) and 5.3 (conspiracy theories).

Methods

The original CARDS model was trained using a dataset comprising paragraphs extracted from sources known for their wealth of climate contrarian content, such as conservative think-tank articles and contrarian blog posts. This training approach showed strong performance when tested on similar content sources. Nevertheless, the model’s ability to effectively differentiate between contrarian and convinced text (reflecting the scientific consensus on climate change) within the context of Twitter remained uncertain. To mitigate this uncertainty, we present an enhanced CARDS model introducing an initial binary classifier. This classifier’s primary function is to distinguish between convinced and contrarian claims, aided by the inclusion of supplementary Twitter data. Subsequently, we include an additional layer responsible for classifying contrarian claims into their respective typology.

Model architecture

Augmented CARDS enhances the performance of the original CARDS model on Twitter by utilizing additional data from the platform and rectifying category imbalances through a two-stage hierarchical architecture. In Fig. 2, the general model architecture is illustrated. It consists of an initial layer trained on a binary detection task to differentiate between convinced and contrarian tweets, coupled with an additional layer trained on a multilabel task to classify the taxonomy illustrated in Fig. 1.

Fig. 2: Model architecture.
figure 2

The two-step hierarchical model consists of a binary classifier that determines the main stance of a climate-related claim, followed by a taxonomy classifier that details the types of arguments used if the claim is identified as contrarian.

Full size image

Both classifiers incorporate the DeBERTa language model, structured based on the auto-encoding transformer architecture introduced in BERT27. The innovation includes disentangled attention and a more extensive pretraining process28,29. Specifically, we utilized the large version of DeBERTa, consisting of 24 transformer blocks with a hidden size of 1024 and 16 attention heads. In addition, an extra dense layer was employed for the classification task, bringing the total number of parameters to ~355 million.

We aim to specialize the classifiers in their respective tasks, undergoing training tailored to their specific contexts. The implementation of a hierarchical architecture responds to the necessity of modularizing both tasks to effectively handle the fine-tuning process of the pipeline and improve its performance. In addition, it mitigates the issue of unbalanced data distribution. Given that the datasets are overly dominated by convinced claims, the challenge lies in effectively detecting the remaining 17 classes of the taxonomy.

Moreover, DeBERTa’s transfer learning capabilities are mainly attributed to its pretraining on web-sourced texts. Nevertheless, since Twitter was not incorporated into the pretraining procedure29, fine-tuning is necessary to capture the linguistic features specific to the platform.

Training details

To enhance the model’s performance, we incorporated the Climate Change Twitter Dataset labelled by the University of Waterloo, featuring a 90/10 ratio of verified and misleading tweets (https://www.kaggle.com/datasets/edqian/twitter-climate-change-sentiment-dataset), to the binary classifier training set. Furthermore, the taxonomy classifier underwent training using the CARDS dataset, incorporating the 5.3 category (”climate change is a conspiracy theory”), which differed from original CARDS which merged category 5.3 with category 5.2. Separating these two categories was deemed appropriate due to the substantial prevalence of conspiracy theories in climate change tweets.

The models were fine-tuned over 3 epochs with a learning rate of 1e−5 in a v100 GPU with a batch size of 6. The input was constrained to sequences of 256 tokens with a padding method. These parameters, along with the seed were kept constant for comparison with the original CARDS method.

To assess the model’s capabilities, climate change experts labelled a testing set of tweets following the26 taxonomy. This dataset, denoted as Expert Annotated Climate Tweets in Table 1, was composed of 2607 tweets related to climate change, sampled from the platform in the second half of 2022.

Data analysis

The analysis was carried out on a large dataset of climate change-related tweets. This dataset was compiled between July and December of 2022 by the Online Media Monitor (OMM) at the University of Hamburg30. We examined the temporal frequency of the data and identified intervals of interest based on distinct patterns discerned by the model. Within these intervals, a word frequency analysis was conducted and compared against the overall word frequency of the entire dataset. This comparison enabled us to highlight specific shifts in word usage during those periods and establish a connection between this information and relevant events that took place.

The word frequency was calculated by comparing the log-fold change and the p-value derived from the distribution differences for various words. Subsequently, a filter was applied to keep only those words with a significance level greater than 0.05, and they were ranked based on their log-fold change in descending order. Finally, the top 10 most relevant words were used to characterize the event.

Results

Assessing the Augmented CARDS model

Table 1 compares the performance of the original CARDS and Augmented CARDS models in identifying contrarian claims in the original CARDS testing set (comprised of contrarian blogs and CTTs) and in our new Twitter dataset. We subdivide this task into two stages: binary detection (distinguishing between contrarian and convinced claims) and taxonomy detection (identifying claims from the CARDS taxonomy).

The original CARDS model performed exceptionally well in datasets sharing linguistic features with its original training data, including CTT articles and contrarian blog posts. This is demonstrated by the F1-score achieved in CARDS for binary detection (89.9), slightly outperforming Augmented CARDS.

However, in the taxonomy detection task, the original model showed a 5% performance decrease relative to the CARDS metrics26. This decline is attributed to the inclusion of the 5.3 category (contrarian claims involving conspiracy theories) in our analysis. This category is highly relevant in the Twitter context but was excluded from the original model in ref. 26. In addition, the Augmented CARDS architecture demonstrated better adaptability, achieving a 76.6 F1-score with additional data from Twitter, where climate change conspiracy arguments hold more significance among contrarians.

Our results indicate that Twitter is a challenging task due to the remarkable disparities in language and writing style observed between the original sources and the platform. On the other hand, the Augmented CARDS model achieves a notable improvement in the F1-Score for both the Twitter Climate Change and the Expert Annotated Climate Tweets datasets for both tasks.

The technical contributions of Augmented CARDS included leveraging additional data from the Twitter context and addressing category imbalances through a hierarchical architecture. Based on these two factors, as shown in Table 1, the Augmented CARDS model demonstrated a relative 16% performance improvement for binary detection and 14.3% for taxonomy detection on our Expert Annotated Climate Tweets dataset. This translates to an F1-score of 81.6 for binary detection and 53.4 for taxonomy detection, while maintaining a similar level of performance in the original domain. Although there is still room for improvement, especially in taxonomy detection, it would require collecting a larger Twitter-based dataset for the less common categories in this context. Most of the categories with low F1-scores are infrequent on Twitter as illustrated in Table 2.

The most prominent contrarian categories were 5.2 (climate actors are unreliable), 5.3 (conspiracy theories), 4.1 (policies are harmful), 2.1 (global warming is naturally caused), and 1.7 (extreme weather isn’t linked to climate change). Table 2 shows that our model exhibits the most substantial improvements within these categories. The F1-scores achieved by Augmented CARDS demonstrate an overall enhancement across most categories, with major improvement in the more relevant ones. Compared to blogs and CTT articles, the distribution of contrarian arguments on Twitter shows a different emphasis, with ad hominem fallacies (category 5.2) directed at climate actors being the most common type of argument. The second most common type of contrarian argument is conspiracy theories about climate change (category 5.3).

Table 1 Assessment of F1-scores achieved, comparing the original CARDS model with the Augmented CARDS model
Full size table
Table 2 F1-scores per category obtained from the Augmented CARDS model on the Expert Annotated Climate Tweets dataset
Full size table

Applying Augmented CARDS to 2022 climate tweets

We applied the Augmented Cards model to over 5 million climate-related tweets in a 6-month period in 2022, providing insight into the proliferation of climate-contrarian claims on Twitter. This novel investigation, to the best of our knowledge, uncovered the triggers that related to an upsurge in contrarian claims on the platform and the most common types of contrarian claims.

The tweets used in our analyses were collected by the University of Hamburg by filtering for terms related to ”climate change”30. Figure 3 illustrates the daily frequency of climate change-related tweets, revealing notable fluctuations, including a substantial peak in late July. On average, the dataset contains 27,464 tweets per day about climate change. However, during the peaks in late July and mid-November, the number of tweets surged to 65,196 and 43,647, respectively.

Fig. 3: Climate change Tweeter trends in 2022.
figure 3

Number of tweets related to climate change published during the second semester of 2022.

Full size image

To further understand the observed peaks, we performed statistical analyses to identify words with major variations and establish correlations between these shifts and remarkable events that occurred during the corresponding periods. The word frequency analysis involved comparing changes in word distributions during specific periods in relation to the entire dataset. We computed the log-fold change and p-value to assess differences in these distributions.

Between July 18 and 21, marking the period with the largest peak in climate tweets, the terms ”climate emergency” and ”Biden” showed the greatest shifts. Based on news reports from that time, these discussions occurred when it became apparent that President Joe Biden was considering declaring a climate emergency in response to the heatwave affecting both the United States and Europe31.

The second-largest peak of overall climate tweets was associated with COP27, as indicated by the changes in word distribution. This event was associated with a doubling of the number of tweets between September 7th and 9th, 2022. The third highest peak of overall climate tweets in our dataset coincided with Hurricane Ian32. Tweets relating to the Hurricane became the major topic of discussion related to climate change between September 28 and October 1, although they generated only half the number of tweets compared to Biden’s declaration event.

Turning now to the analysis of contrarian tweets, Fig. 4 displays the percentage per day of contrarian tweets detected by the Augmented CARDS model through a binary inference process. This analysis indicates that the average proportion of contrarian tweets per day is 15.5%, yet there is clearly a number of peaks of contrarian tweets throughout the 6-month period.

Fig. 4: Detection of contrarian tweets in 2022.
figure 4

Percentage of contrarian tweets detected by the CARDS and Augmented CARDS models.

Full size image

Overall, we identified four distinct categories of events that coincided with an upsurge in the publication of contrarian tweets, as outlined in Table 3. The triggering events can be broadly classified into three primary groups: Natural Events, Political Events, and Influencer Posts.

Table 3 Categorization of events that induced spikes in climate contrarianism on Twitter
Full size table

Natural Events, such as Hurricane IAN, and Political Events like COP27, were external occurrences originating outside the platform33,34. They coincided with a general increase in public discourse surrounding the climate change topic and occasionally prompted shifts in contrarian positions.

For example, the Biden declaration was seized upon by climate change contrarians, triggering major peaks in the percentage of contrarian tweets. The primary concern revolved around the possibility that climate warming might be used as a political pretext to declare an emergency, potentially granting expanded powers to President Biden, which could disrupt the existing state equilibrium. The percentage of contrarian claims reached a peak of 24.7% around the time of President Biden’s declaration.

Similarly, the Natural Event of Hurricane Ian triggered an increase in all tweets related to climate change and the percentage of contrarian claims. Despite generating only half the number of tweets compared to Biden’s declaration, the proportion of contrarian tweets related to Hurricane Ian reached similarly high levels. Discussions were centred around the impact of climate change on extreme weather conditions. For instance, FoxNews tweeted one of their articles, titled ”Democrats blaming climate change for Hurricane Ian at odds with science, experts say”.

The COP27 event stands in contrast to the Biden emergency declaration and Hurricane Ian. While the latter events led to important increases in both the volume of tweets and the proportion of contrarian claims, COP27 resulted in only a 1% rise in the proportion of contrarian claims among climate-related tweets.

Moreover, aside from the increase in tweet volume associated with Political and Natural Events, other cases showed a rise linked to tweets published by influential accounts. In addition, contrarian claims increased in response to influencers regardless of whether they expressed a convinced or contrarian view. These influencers, including politicians, comedians, film directors, and media figures, all share the characteristic of being public figures with a substantial number of followers. It’s important to note that our categorization of influencers is based on the positions adopted in their 2022 publications, not necessarily their current personal viewpoints.

Our final analysis is the categorisation of contrarian tweets by the typology of ref. 26 as inferred by the Augmented CARDS model. Figure 5 represents the distribution of the tweets by the most common categories identified in the climate-related tweets. The distribution of contrarian categories remains relatively stable even on dates with significant deviations. The most common form of climate contrarianism involves criticisms of climate actors such as climate scientists and environmentalists (category 5.2), comprising 40% of the total number of misleading tweets. This is followed by category 5.3, which includes tweets categorizing climate change as a conspiracy, making up ~20% of the segment. Categories 4.1 (climate policies are harmful) and 2.1 (natural cycles are causing global warming, not humans) make up the next two most relevant categories. The fifth most common category, 1.7 (extreme events are not increasing) receives an important share of the distribution during the Hurricane Ian period.

Fig. 5: Distribution of the Top 5 contrarian arguments.
figure 5

Breakdown of the five most relevant categories detected by the Augmented CARDS model on Twitter in 2022. The percentages of these categories are relative to the total proportion of contrarian claims identified by the binary model.

Full size image

Generating a time evolution of contrarian claims allows us to identify which categories dominate based on the different types of triggers. Table 4 shows that Natural Events and Political Events shifted the distribution towards topics related to categories 1.7 and 4.1, respectively. This is expected given that 1.7 relates specifically to extreme events and increased during the Hurricane Ian period. Moreover, increases in category 4.1 were associated with political events, which is to be expected given that the category involves criticisms of climate policies.

Table 4 Preferred arguments based on the type of source
Full size table

The fluctuations generated by influencers lean substantially towards categories 5.3 (conspiracy theories) and 2.1 (natural cycles/variation), irrespective of whether the influencer holds a contrarian or convinced stance. Notably, when the influencer supported a contrarian viewpoint, there was a discernible increase in the prevalence of conspiracy theories. Conversely, in instances where the influencer expressed a convinced stance, the distribution shifted slightly more toward posts asserting that climate change is a natural cycle, accompanied by a concurrent rise in conspiracy theories.

Finally, a considerable number of accounts exhibited an unusually high volume of contrarian tweets across all primary categories. On average, each user shared between 1 and 2 contrarian tweets. Notably, category 5.3 had the highest average of 1.9 tweets per user, closely followed by category 5.2 with an average of 1.8. The remaining three categories had averages of 1.57 contrarian tweets or fewer per user. This suggests that discussions revolving around conspiracy theories were generating a meaningful amount of content from a relatively closed pool of users. However, users with hundreds of published tweets could be found across all principal categories. For example, the user with the maximum number of posts related to conspiracies (5.3) reached 921 tweets just in this category. Similar cases could be found in other categories as well: 5.2 (881), 4.1 (627), 2.1 (424), and 1.7 (108). Some of these outlier accounts were evidently automated. For instance, one user posted 1977 tweets within our data period, with only 206 of them featuring unique textual content, while the rest repetitively rephrased the same idea with minor grammatical changes. Other accounts were evidently managed by individuals engaging in discussions and expressing opinions. Overall, the proportion of unique content in contrarian detections is ~93.3%, with categories 5.3 and 5.2 containing around 91.3% and 94.1% unique content, respectively. In summary, the existence of accounts managed by automated systems and users generating a substantial volume of contrarian tweets is indisputable, with their content consistently reflecting the trends identified in our taxonomy analysis. In addition, we observed that ~6% of the analyzed tweets could be categorized as spam. However, content generated by modern AI models could potentially circumvent our current analytical approach and exacerbate this issue.

Discussion

Our study showed that a classifier trained only on climate contrarian text (e.g., the original CARDS model) struggles at the binary classification task of distinguishing between convinced and contrarian text. We found that adding training data that includes annotations of both convinced and contrarian examples from improved performance in binary classification. This addressed a limitation of the original CARDS model, which performed well with known misinformation sources but struggled with general climate text that could have originated from both convinced and unconvinced sources.

Our analysis of climate tweets through a 6-month period in 2022 also revealed the dominant categories of contrarian claims and climate misinformation on social media relative to other information sources, such as contrarian blogs and CTT websites. While CTTs focused on policy claims and contrarian blogs focused on attacking climate science, more than half of the relevant climate tweets focused on either attacking climate actors or conspiracy theories. This result is consistent with other research finding that attacks on climate supporters are a dominant theme among tweets by climate contrarians, as are climate-themed conspiracy theories35. These findings are also consistent with recent scholarship on the personalisation of politics on social media36,37 and the observation that personal attacks tend to drive engagement on platforms such as Facebook and Twitter/X38, suggesting that the nature of misinformation—i.e., personalised versus non-personalised—is at least particularly dependent on the platform of interest. Overall, the dominance of personal attacks in climate misinformation on social media underscores the importance of better understanding the impact of climate misinformation in the form of ad hominem attacks and conspiracy theories, as well as exploring the efficacy of interventions that neutralise their negative impact.

We also identified the different types of misinformation peaks on Twitter, associated with external events (political or natural) or influencer posts (contrarian or convinced). External events coincided with a spike in the total number of climate tweets. while influencer-associated peaks were associated with an increase in the proportion of misinformation tweets but not an increase in overall climate tweets.

There were predictable patterns in the types of arguments in response to different peak types. The clearest signal was in response to natural events, which showed a 680% increase in category 1.7 claims making the argument that weather events weren’t linked to climate change. Political events coincided with category 4.1 claims, arguing that climate policy was harmful.

A limitation of this study is that its scope was restricted to climate misinformation on Twitter. It is yet to be seen whether the Augmented CARDS model performs at similar levels on other data sources. Future research could focus on a wider range of information sources, such as other social media platforms, congressional testimonies, public speeches, online video transcripts, and newspaper articles. Such an analysis could also yield which misinformation categories are dominant across these different information sources. Within a single information source, cross-country analysis could also interrogate different emphases in climate misinformation across different cultures. Similarly, analysis of output from different mainstream media publishers could identify the relative proportion of climate misinformation among different outlets.

Future research may also apply supervised learning models to a broader period of time to draw out long-term trends on Twitter and other platforms. For example, there has been much discussion surrounding the recent changes to Twitter’s ownership39 and the impact this has had on climate misinformation40. Our data does not span a large enough time period to make any concrete conclusions about these recent changes.

Another limitation of the CARDS model is that, to date, it has been trained on English text only. Future research could apply our methodology with training sets of non-English text, to facilitate detection of climate misinformation in other languages across different countries.

Conclusion

This study has taken a step closer to the goal of automatically detecting and correcting climate misinformation in real-time. We have shown an improvement in classifying misinformation in climate tweets, with considerable reductions in the ”false positive problem” associated with training on contrarian actors alone.

These findings have practical implications. Adopting our model could help Twitter/X to augment and enhance ongoing manual fact-checking procedures by offering a computer-assisted procedure for finding the tweets most likely to contain climate misinformation. This adoption could make finding and responding to climate-related misinformation more efficient and help Twitter/X enforce policies to reduce false or misleading claims on the platform. Yet environmental groups have shown that Twitter/X ranks dead last among major social media platforms in its policies and procedures for responding to climate misinformation and there is little evidence that X will improve these procedures in the near term41. Alternatively, our model could provide the basis for an API that Twitter/X users could employ to assess climate-related claims they are seeing in their feeds. Overall, the potential practical applications of our model underscores the need for continued academic work to monitor misinformation on Twitter/X and raises important questions on the data needed to hold social media platforms accountable for the spread of false claims.

However, there are still numerous hurdles to overcome before the goal of automated debunking is achieved. An effective debunking requires both explanation of the relevant facts and exposing the misleading fallacies employed by the misinformation. Contrarian climate claims can contain a range of different fallacies, so automatic detection of logical fallacies is another necessary task that, used in concert with the CARDS model, could bring us closer to the ”holy grail of fact-checking”17.

Regardless, this research has already provided greater understanding of climate misinformation on social media, identifying four types of misinformation spikes. Knowing the types of arguments that are likely to be posted on social media in response to external events such as climate legislation or natural events can inform interventions that seek to pre-emptively neutralize anticipated misinformation narratives.