Introduction

There is no accepted scientific understanding of how online hate manages to thrive at scale. Yet its societal consequences are widespread and occur daily globally, e.g., personal traumas1,2,3; gender, race, and religion-based abuse; child sex abuse4,5; and violent mass attacks6. Making matters more complex from a scientific viewpoint, online hate is dynamic, adaptive7,8,9,10,11 and now looks set to surge12,13,14 armed with new AI/GPT tools15,16. Overcoming it will require establishing the science of how it operates at scale17.

In addition to the obvious consequences for the direct victims of hate attacks and abuse, there is a huge secondary impact. Nearly 50% of all Americans now compromise aspects of their and their children’s daily lives in order to lower the risk of experiencing some hate-driven mass shooting, e.g., 6 May 2023 Allen, Texas shooting which appears to be one of an increasing number inspired by social media hate content18,19. Separately, 2024 will see more than 60 elections across 54 countries including the U.S. and India, where the scope for online hate to cause voter intimidation is huge20,21. Such mass-scale threats, now supercharged by AI/GPT weaponry, are accelerating efforts to win the war against online hate and other harms22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37. We refer to refs. 38,39,40 for unifying perspectives on this huge and still growing body of research, while ref. 41 provides daily updates on new studies.

The current war against online hate of all forms is being led on the regulatory side by the EU’s “Digital Services Act” (DSA) and “A.I. Act”42,43. Social media platforms (referred to hereafter as “platforms”) on the list of “Very Large Online Platforms” such as Facebook and Twitter, must carry out a risk assessment which includes an analysis of how harmful content might be disseminated through their service44. At face value, this appears to make perfect sense since the largest platforms (e.g., Facebook, Twitter) have the largest share of users. Hateful content is thought to occupy the fringes of the Internet45,46,47,48,49,50. However, winning any war requires an accurate picture of the battlefield.

Here we adopt an engineering approach and show how it leads to an understanding of the machinery and link dynamics that manage to sustain online hate at scale. We start by mapping out the dynamical structure of the online hate network across platforms. Instead of the large platforms being the key drivers, we find that waves of adaptive links connect the hate user base over time across a large ecosystem of smaller platforms, allowing hate networks to steadily strengthen, bypass mitigations, and increase their direct influence on the massive neighboring mainstream. The data suggests hundreds of thousands of individuals globally have recently been exposed, including children. We then establish governing dynamical equations derived from first principles. A tipping-point condition predicts more frequent future surges in content transmission.

Using the 2021 U.S. Capitol attack and a 2023 mass shooting as illustrations, we show our findings offer abiding insights and quantitative predictions down to the hourly scale. This dynamical focus yields general information about online hate networks, their connection to the mainstream population, the question of when information will/will not be transmitted at a global scale, and the effects of mitigation strategies.

To improve the flow of the paper for a general audience, we have put the full technical details of the data collection as well as the mathematical derivations and calculations, into the Supplementary Information (SI) which accompanies the paper online. Figures 1–3 show the key results.

Fig. 1: Hate’s highly adaptive link dynamics.
figure 1

a–c Number of new links created on each day t from hate communities (nodes). Time-series show different aggregations across the 4 link types (schematic in panel a left inset): hate to hate inter-platform; hate to hate intra-platform; hate to hate-vulnerable mainstream inter-platform; hate to hate-vulnerable mainstream intra-platform. Suppl. Fig. 4 gives explicit examples of these links. d Sankey diagram shows intra-day flows of new links from hate nodes on a given platform (source) into hate nodes on a target platform. SI Sec. 3 explains Sankey diagram construction.

Full size image
Fig. 2: Empirical data (symbols) vs. mathematical solutions of the deterministic governing equations (curves, derivation SI Sec. 4.1).
figure 2

a Relative number of links created at time t from hate communities on a given platform (source). b Relative number of links created at time t to other communities on a given platform (target). Approximately 80% of targets are hate-vulnerable mainstream communities. Only the largest curves are shown, the rest are aggregated as ‘Other’ (black curves). c, d Same as plots a and b but applied to Jan 4-5 data using different time points.

Full size image
Fig. 3: Simulations with context of a network map.
figure 3

a Dynamical network aggregated over 2.5 years’ worth of social media data. Each colored node is a hate community. Each white node is a hate-vulnerable mainstream community to which a hate node has a direct link. Edges are colored the same color as their source node. Generally, the areas of color visible in this network are space between nodes filled by dense edges. Side panels compare platforms’ involvement by only highlighting edges that originate from hate nodes on a given platform. 2023 Texas shooter’s YouTube community is shown, so too is a major Wagner mercenary community on Telegram (Reverse Side of the Medal). See SI for others and an example zoom-in (dotted box, Suppl. Fig. 15). Network layout is generated by the ForceAtlas2 algorithm68: sets of communities appear closer together when they share more links. b Comparing mitigation schemes’ effects on network topology, modeled after robustness tests carried out in ref. 55 This simulation neutralizes (removes) a random node at each timestep. The blue line shows this process carried out for hate-vulnerable nodes, while the red line shows this for nodes in the hate core. The main plot shows the simulations’ effect on the average size of non-largest connected components, while the inset plot shows the effects on the largest connected component. X-axis is scaled by the total time to neutralize all nodes. Inset uses schematic of the multi-platform network to explain these different curve behaviors: neutralizing nodes in clusters like A (disconnected from the largest connected component) has little effect on S but decreases 〈s〉; neutralizing nodes in clusters like B can slightly decrease S but increase 〈s〉; neutralizing nodes in clusters like C can strongly decrease S and strongly increase 〈s〉.

Full size image

In this paper, we frequently refer to “links,” which can be understood as edges in the hate network. A link is formed between a source community and target community when the source community shares a URL leading directly to the target or to content posted by the target. Source communities are always hate communities and target communities can be either hate or “hate-vulnerable” communities. Please see “Methods” and SI Sec. 1 for full details of our data collection and community classifications.

Results

Key Features

We map the online hate ecosystem using communities as nodes and links as edges. The resulting network (schematic Fig. 1a left inset; results Figs. 1–3; data collection methodology in SI) is a directed dynamical network with link wiring that can change quickly over time within and across platforms, and strong direct linkage from the hate networks to the massive hate-vulnerable mainstream. It contains 1542 hate communities (nodes) that in 2.5 years have created 285,378 links to each other and 2,832,128 links into 385,719 hate-vulnerable mainstream communities. The membership size for 907 of these hate communities is publicly available and averages to approximately 28,000 per community, so we can estimate there are roughly 43 M individuals involved in the hate core.

While the nodes (communities) are fairly constant over time, the link number increases massively every day and hence steadily strengthens the hate networks and their potential mainstream influence. Each new link (e.g., Figs. S1 and S5) means members of the source hate community (node) can immediately engage with the target community (node), pass hate content to it, and influence it.

Three key features emerge (illustrated in Figs. 1–3) that hate mitigations must account for in order to be effective. These insights can be directly incorporated into new legislation and/or content moderation policy:

  1. (1)

    They must focus on activity (links) between platforms, particularly including the many smaller platforms as shown explicitly in Fig. 1d—and not just activity within the largest platforms. To keep their users safe, it is ineffective for individual platforms to only focus on themselves.

  2. (2)

    They must be nimble enough to outpace rapid link creation dynamics (e.g. those shown in Figs. 1–2) down to the scale of minutes within a day—in particular, the huge waves of links which appear suddenly around notable events (Fig. 1), and which could further enflame hate, anger, and distrust during these events or their aftermath, possibly inciting new violent acts.

  3. (3)

    They must avoid the existing “brute force” strategy of chasing down a queue of reported links. This is akin to a game of whack-a-mole, i.e., existing links can get buried below newer content and hence become less relevant while fresh, unreported links become the new focus. Old links can also get removed on purpose by the community member(s), or the piece of content they are in gets removed. This link loss also means that the most active pathways that hate content spreads though are changing all the time, hence mitigations to prevent system-wide spreading need to account for this.

Governing equations and analysis

Current approaches to mitigation and legislation do not account for the key empirical features (1)–(3) that emerged. However, by deriving a set of governing dynamical equations that explicitly capture (1)–(3), we can systematically incorporate them into improved strategies and policies. Just as in engineering systems, analyzing solutions to a system’s governing equations provides insights into the underlying dynamics, enabling design of optimal interventions51.

At their core, the governing equations approximate the dynamics of interacting shockwaves, representing spikes or waves in community linking activity over time. This minimal framework of coupled linear differential equations provides an analytically tractable model that matches empirical data.

The SI sec. 4 derives these governing dynamical equations (Suppl. Eq. 102) starting from a realistic online grouping mechanism52,53. Notably, the equations reproduce the shockwave patterns observed in the link creation data (Figs. 1d and 2). This is because they are mathematically equivalent to shockwave equations—even in their minimal form (Fig. 2):

$$\dot{{S}_{i}}=H\left(t-{t}_{i}\right)\left[{a}_{i}\left({S}_{i,0}-{S}_{i}\right)+\mathop{\sum}\limits_{j}{b}_{i,j}({S}_{j}-{S}_{i})\right]$$
(1)

Here, H(…) represents the Heaviside function, and \({t}_{i}\) denotes the onset time of a new wave of link creation (SI sec. 4). The term \({a}_{i}\left({S}_{i,0}-{S}_{i}\right)\) characterizes the intrinsic growth dynamics of community i where \({a}_{i}\) \({b}_{i,j}{S}_{i,0}\) is the carrying capacity. \({\Sigma }_{j}{b}_{i,j}({S}_{j}-{S}_{i})\) describes the coupling between communities, with \({b}_{i,j}\) quantifying the functional dependence of the trajectory of community \(i\) on that of community \(j\). As analyzed in SI Sec. 4.1.5.1, setting most \({b}_{i,j}\) terms to zero provides a balance of model simplicity and good fit to the data according to quantitative goodness-of-fit assessments. \({S}_{i}\) and \({S}_{j}\) capture the time evolution for communities \(i\) and \(j\), respectively. Critically, these equations are exactly piecewise solvable in their approximate form (SI Sec. 4.1.5) which means there are no computational errors or instabilities in their solutions and hence predictions.

Figure 2 demonstrates, contrary to common assumptions, Twitter does not act as a central driving platform. Rather, the collective activity of smaller platforms emerges as the key driver, while Twitter serves more as an event reporter. Further, Fig. 2c, d demonstrate the flexibility of those same governing equations by showing that they can reproduce the link-dynamics on a more normal date range, January 4–5, with appropriately lower amplitudes. This suggests the interpretation that the January 6 events represented an amplified version of normal background hate network activity and dynamics, rather than an entirely unpredictable ‘Black Swan’ event.

The deterministic nature of the governing equations, operating at the many-link-node level, allows for quantitative forecasting at the more general meso- and macro-scales. For example, forward iteration can predict the hourly effects of a proposed mitigation strategy on future ecosystem dynamics. Conversely, the process can be reversed to identify the best mitigation that achieves a desired impact. These findings thus demonstrate how our analysis provides abiding insights and quantitative predictions.

Furthermore, these dynamical link equations provide a tipping point condition for whether hate content can spread system-wide, and how to prevent it. SI Sec. 4.2 derives this: here for simplicity, assume a community digests new hate content received via a link after time \({T}_{{\rm{digest}}-{\rm{hate}}}\) on average; subsequently forgets it after time \({T}_{{\rm{forget}}-{\rm{hate}}}\) on average; links between community clusters arise after time \({T}_{{\rm{create}}-{\rm{links}}}\) on average; and links disappear after time \({T}_{{\rm{lose}}-{\rm{links}}}\) on average. Then Suppl. Eq. 131 predicts system-wide propagation of hate content will be impeded if

$${T}_{{\rm{lose}}-{\rm{links}}}{T}_{{\rm{forget}}-{\rm{hate}}}{({T}_{{\rm{create}}-{\rm{links}}}{T}_{{\rm{digest}}-{\rm{hate}}})}^{-1} \,<\,1\,({\rm{Suppl}}.{\rm{Eq}}.131)$$

which agrees with simulations. Therefore, system-wide propagation is impeded by prolonging the time to create links or digest hate content, or alternatively, by shortening the time to eliminate links or forget hate content, so that the inequality condition holds.

Mathematically, this confirms why criteria (1)–(3) are crucial, and why mitigation or legislation that does not satisfy them will prove ineffective.

Hate Network Landscape

Aggregating the link dynamics over time (Fig. 3a) shows even more clearly that online hate does not live at the fringe, and that large platforms are not the key. The many smaller platforms in Fig. 3a act like dynamical glue that binds the hate networks together and attaches them directly to the mainstream.

Real-world events involving hate are mirrored—and may increasingly be pre-empted—by hate activity within this dynamical network. In addition to the events in Figs. 1c and 3a shows the 6 May 2023 Texas shooter’s YouTube community (now banned) which had attracted 4 separate links into it from other hate communities prior to his attack. Members of these hate communities had been alerted to his YouTube channel and could have easily posted comments and/or content that fueled extreme views and hence influence among his channel’s members—including him.

Mentions of RWDS (‘Right Wing Death Squad’) are also prevalent across Fig. 3a; so too are Wagner mercenary communities (see Suppl. Figs. 7–9). “RWDS” has appeared as insignia worn by recent mass shooters and members of neo-Nazi units in the Ukraine-Russia conflict. Figure 3a also reveals how some hate-vulnerable communities have far higher exposure risk than others – not necessarily because of their views, but because they are more appealing prey. Regardless of the reason for their popularity in hate communities, they attract more links and so sit closer to the hate core in Fig. 3a because of its node-repulsion-link-attraction ForceAtlas2 layout.

This also explains its ordered circles akin to solar system orbital structure: successive subsets of hate-vulnerable nodes have 1, 2, 3, etc. links from 4Chan (blue) and hence a net spring force pulling them toward the hate core that is 1, 2, 3, etc. times as strong. These successively smaller radius stripes hence contain hate-vulnerable communities that are roughly 1, 2, 3, etc. times more likely to receive hate content and influence.

This suggests tailoring pre-emptive action first on the inner rings closest to the hate core in Fig. 3a, then working in order of increasing radius and hence decreasing risk of exposure.

Discussion

We have shown how our approach has led to a deeper understanding of the machinery and mechanisms that manage to sustain online hate at scale. By mapping out the dynamical structure of the online hate network across platforms, we found that its dynamical features contradict current thinking. Instead of the large platforms being the key drivers, hate networks are bound together over time by waves of adaptive links across numerous smaller platforms. This allows the hate networks to progressively strengthen, sidestep mitigations, and exert greater direct influence on the massive mainstream neighbor.

We then derived deterministic governing equations from first principles to describe the link dynamics sustaining online hate. The equations are built on models of human grouping behavior, providing a framework akin to conventional engineering systems. This enables quantitative predictions about recent and future events at hourly resolution, offering enduring insights.

Such knowledge of the dynamical network and its underlying equations allows rigorously calculating and comparing expected impacts of different mitigation strategies. Formal control theory can now be leveraged to systematically design interventions54.

Figure 3b uses simulations to compare mitigation variants. These simulations randomly remove nodes either found in the set of hate communities (red) or the set of hate-vulnerable communities (blue). In such mitigation strategies, posts are removed from a community if they link to extreme content (e.g., a hate manifesto or footage from a mass shooting) posted in other communities. Iterating this continually with slow link dynamics mimics neutralizing communities across the network shown in Fig. 3a.

The simulated mitigation impacts over time (Fig. 3b) are unlike prior estimates and again challenge current thinking. Figure 3b inset shows the impact on the relative size of the largest cluster (S) which, in a slow link-dynamic limit, represents the maximum spread that any piece of hateful content can have. The main panel shows the impacts on the average size 〈s〉 of the other clusters which quantifies how linked the remaining communities are on average, and hence quantifies the threat they pose as nucleation sites for further activities.

These curves differ markedly from exponential and scale-free network models associated with the World Wide Web or Internet as a whole55. No universally optimal mitigation emerges from our simulations: removing hate nodes irrespective of platform reduces S quicker than removing hate-vulnerable nodes (inset: red curve is lower than blue), but it has the disadvantage that it generates larger nucleation clusters (larger 〈s〉) for future harms (main: red curve is higher than blue). In terms of the three criteria outlined previously, this means that while such a strategy addresses (1) and (3) by focusing on the system level link-spreading infrastructure rather than individual platforms or links, it may not adequately address (2) – the necessity for fast-paced measures around large “waves” of links – depending on the composition of the smaller nucleation clusters which factor into 〈s〉. In this area, further study of the interactions between large and small platforms within the context of the hate ecosystem is necessary.

While AI’s future weaponization remains uncertain56, our framework clarifies the dynamics of this online machinery. Future improvements will include: (1) subclassifying hate by type (e.g., anti-Semitic); (2) exploring other hate definitions; (3) analyzing blended hate types; (4) sub-classifying hate-vulnerable communities; (5) incorporating private platforms; (6) adding link weights according their use; (7) adding mainstream-to-hate communities links; (8) adding links to government sources (e.g. Fig. S3); (9) include general harm types; (10) subclassifying each community by location or scale (e.g. local vs. global57,58).

Although our data is technically a large sample of the unknown true online population, the billions of estimated users suggest it qualifies as a crude population map. Moreover, we charted this hate ecosystem not by isolated sampling but by algorithmically tracing links node-to-node. This process tended to eventually return to the same nodes and hence, like circumnavigating the globe, it hints that we have charted out—albeit crudely—the skeleton of the true online hate ecosystem.

Methods

Data collection and link tracking

Our methodology follows refs. 59,60 (SI Sec. 1) but goes beyond prior studies by (i) including the mainstream communities that hate communities link to over time (referred to in this paper as “hate-vulnerable”), (ii) tracking this data down to second-scale resolution across 13 platforms, (iii) including new decentralized61 and blockchain platforms (e.g. Minds, Steemit) for which blame cannot be pinned on single servers and cryptocurrency can incentivize users, and (iv) including gaming-related platforms62 such as Discord which played a key role in recent security leaks.

Our focus is on platform-provided, built-in communities because people join these to develop their shared interests63,64,65,66 including hate. Terminology for such communities is platform-specific. Examples are a VKontakte Club (VKontakte is a social media platform controlled by Russian state-owned bank Gazprombank and insurance company Sogaz67); a Facebook Page; a Telegram Channel; a Gab Group. Each community contains anywhere from a few to a few million users and is unrelated to “community” as used in network community detection.

A “hate” community is one in which 2 or more of its 20 most recent posts include U.S. Department of Justice-defined hate speech. Together, these hate communities make up the “hate core”. A “hate-vulnerable” community is one that is outside this hate community core but was linked to directly by a hate community (Fig. 1a inset). Hate-vulnerable communities’ views can vary significantly (we do not attempt to categorize them for this study), but mostly represent a benign mainstream that have become targets of the hate core (SI Sec. 1.2). Concretely, any link in this network appears because a post included a URL directly to another community or to content featured therein. For example, one edge in the network in Fig. 3a represents the URL to a Gab post posted in a comment thread on 4chan. See this and other examples in Fig. S5.

A link to community B can appear in community A at some time t if B’s content is of interest to A’s members (Suppl. Figs. 1 and 4 show examples). The link directs attention of A’s members to B, which may be on a different platform and another language. A’s members can then add comments and content on B without B’s members knowing this incoming link exists. Hence B’s members can unwittingly experience direct exposure to, and influence from, A’s hateful narratives. Since our focus is on hate networks, we do not include links originating in hate-vulnerable nodes.

No individual information is required. Only public communities are accessed, but the ecosystem of open communities provides a skeleton on which private communities sit (Suppl. Fig. 3).

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.