Abstract
The social media platform Twitter platform has played a crucial role in the Black Lives Matter (BLM) movement. The immediate, flexible nature of tweets plays a crucial role both in spreading information about the movement’s aims and in organizing individual protests. Twitter has also played an important role in the right-wing reaction to BLM, providing a means to reframe and recontextualize activists’ claims in a more sinister light. The ability to bring about social change depends on the balance of these two forces, and in particular which side can capture and maintain sustained attention. The present study examines 2 years worth of tweets about BLM (about 118 million in total). Timeseries analysis reveals that activists are better at mobilizing rapid attention, whereas right-wing accounts show a pattern of moderate but more sustained activity driven by reaction to political opponents. Topic modeling reveals differences in how different political groups talk about BLM. Most notably, the murder of George Floyd appears to have solidified a right-wing counter-framing of protests as arising from dangerous “terrorist” actors. The study thus sheds light on the complex network and rhetorical effects that drive the struggle for online attention to the BLM movement.
Introduction
The online platform Twitter has played a central role in the Black Lives Matter (BLM) movement (Tufekci, 2017). Twitter has been used both to draw attention to the movement’s core ideas and to mobilize supporters for protests (Ince et al., 2017). In a recent review, Dunivin et al. (2022) have suggested that increased online attention has made the movement widely visible. Although the #BlackLivesMatter hashtag was first coined in 2013 as a response to the acquittal of George Zimmerman in the shooting death of Trayvon Martin (Lebron, 2017), the murder of George Floyd by police in May 2020 catalyzed the movement both online and on the streets. Twitter played a vital role in that increased engagement.
Wu (2017) notes that “…in the battle for our attention, there is a particular importance in who gets there first or most often.” Yet speed and endurance can dissociate. Rapidly mobilized attention can rapidly fade, while a suitable framing can prolong moderate engagement. Tufekci (2017) argues that left-wing activism has effectively used Twitter to gain rapid attention, but struggles to maintain that message across time. More generally, Twitter is part of the “economy of attention” (Hendricks and Vestergaard, 2019; Simon, 1971). Particular tweets and general issues compete for scarce attentional resources.
Downs (1972) argued that some topics go through a characteristic “Issue-Attention Cycle.” Fervent attention is kicked off by a high-profile event, which opens a window for change before being followed by pessimistic counter-reactions and a fade from public view. The ability to bring about lasting change depends on how much can be done during the period of heightened awareness—which in turn depends on how long that period can be maintained. Downs identified three conditions that make an issue particularly likely to go through this cycle (Downs, 1972, p41ff): it involves a minority population, solving the issue requires dealing with structural considerations that advantage a majority, and the peaks of attention tend to be tied to shocking but transient events. All three conditions are met by BLM. Hence attentional dynamics are key to understanding BLM’s impact: change is more likely the longer it can dominate the awareness portion of the issue-attention cycle.
Yet attention alone is not sufficient for change, as awareness of a movement is not necessarily agreement with its aims. Twitter contains a substantial right-wing presence (Freelon et al., 2020), many of whom are unsympathetic or downright hostile toward the grievances expressed by BLM activists (Keib et al., 2018; Ray et al., 2017). The ability of Twitter to recontextualize and repurpose information makes it a powerful tool for framing BLM—that is, to influence "the context within which an issue, opinion, or claim is made” which in turn "influences our understanding of and attitudes towards it” (Benkler et al., 2018, p. 10). Previous work suggests that even though frames enclose and affect virtually all aspects of our political lives (Scheufele, 2000; Shah et al., 2001), their implications are especially pronounced in the context of social movements (Snow et al., 1988, 1986). One reason for this is that the success of such movements is largely contingent on public perception, which is in turn sensitive to how a movement presents itself and is presented by its opponents (Edwards and Arnon, 2021). By opting for a particular frame, a movement articulates its identity, grievances, strategies, and ambitions in ways that are more or less amenable to public opinion (Ince et al., 2017; Winstead, 2017). As public opinion is fickle, even the most carefully chosen frames are contestable (Riker, 1986).
The present study examines attentional and framing dynamics on Twitter in the context of the Black Lives Matter movement. Insofar as competing frames vie for both attention and approval, we suspect that online social movements will not only be responsive to attention-grabbing events such as police shootings and protests, but also attempt to incorporate those events within an enduring and compelling narrative. In the context of BLM, and the killing of George Floyd in particular, this leads to two research questions (RQ1 and RQ2):
-
RQ1.
How sensitive is the online attention of different groups to protests (and vice versa), and how quickly does that attention decay?
-
RQ2.
How do different groups talk about BLM, and how did that change in response to the murder of George Floyd?
We hypothesized, based in part on the work of Gunn et al. (2018) on responsiveness of politically relevant searches after mass shootings, that RQ1 would reveal a pattern whereby politically left-leaning accounts would show a rapid but more transient response to protests, while politically right-leaning accounts would show a smaller but more sustained response partly driven by reaction to the left. While Gunn et al. (2018) don’t explicitly touch on political affiliation, previous work finds that both gun-ownership and attitudes toward gun-control are strongly sorted along partisan lines, with Republicans generally opposing stricter legislation (Wozniak, 2017) and owning more firearms (Parker et al., 2017). Inasmuch as attitudes toward BLM are polarized along similar ideological cleavages (Alfano et al., 2022; Drakulich and Denver, 2022), and given that both mass- and police-shootings are violent, shocking events, it stands to reason that patterns in responsiveness to police killings will coincide with partisan identification.
For RQ2, we hypothesized that there would be differential framing of BLM narratives, particularly post-Floyd when a pro-police narrative would be more difficult to sustain, but our goals were exploratory with respect to the content of those framing narratives and how they changed.
We approached these questions by using using a large corpus of tweets about Black Lives Matter collected over the whole of 2020 and 2021. We use a mixed-methods, data-driven design to examine different groups in online discourse and the topics with which they engage. Our work builds on a number of recent analyses that employ similar methods, including a comprehensive set of topic models published by Giorgi et al. (2022). In so doing, the current paper contributes to the growing body of work on Black Lives Matter (Crenshaw et al., 2015; Gallagher et al., 2018; Lebron, 2017), the broader literature on online social movements (Hara and Huang, 2011; Harlow, 2012; Tufekci, 2017), and the still nascent study of attentional dynamics in virtual environments (Freelon et al., 2020; Gunn et al., 2018; Hendricks and Vestergaard, 2019).
Methods
Data collection
Based on an initial snowball sampling conducted in 2015 of words, phrases, and hashtags associated with the Black Lives Matter movement, we queried the Twitter Streaming API with a series of Black Lives Matter-related keywords, hashtags, and short expressions (see SI Section 1 for more detail, as well as information about code and data availability).
Tweets can fall into one of three categories: original tweets (which can either be de novo or comments on other tweets); quote tweets, which reference and comment on a tweet; and retweets of either original or quote tweets. We examined original tweets and retweets that occurred within 2020 and 2021. Note that a retweet can occur an arbitrarily long time after its parent tweet, though in practice the vast majority of retweets are to very recent tweets. We omitted quote tweets: it is often difficult to tell whether they represent endorsement or criticism, and the text itself can be difficult to interpret without the context of the original tweet.
Over 2 years we collected ~118.7 million tweets of all types. Our access to the Twitter API was disrupted from July 24th through August 8th 2020. We include tweets from that time period if they were later retweeted. These missing days are included in qualitative analyses below, but omitted from quantitative ones. Additionally, we are missing data on pure retweets from May 31, 2020, which we suspect (but cannot confirm) was part of an effort by Twitter to mitigate certain kinds of network effects in the wake of Floyd’s murder. As retweets were only used to build the group network but not as part of the subsequent analysis, the missing retweets should have little effect.
We made no attempt to filter real users from bots. It matters little for our hypotheses whether online dynamics are driven by humans alone or by a combination of humans and bots, so long as a sufficient amount of content is generated by humans. Both humans and bots amplify attention, and so both are appropriate analytic targets.
To examine the real-world causes and consequence of BLM, we used data on BLM protests. Protests are useful both as a proxy for real-world events (such as instances of police brutality) and as a consequence of online social organizing. We examined data on the number of protests per day in the US. Protest data was sourced from the Armed Conflict Location & Event Data Project (ACLED) (Raleigh et al., 2010; see also SI Section 1). Note that the ACLED database only reports the number of protests that occurred during the period in question, not the size. While it would be informative to know exactly how large these protests were, data about protest attendance is both hard to come by and notoriously unreliable. That said, polls conducted by several reputable institutions during June 2020 indicate that anywhere between 15,000,000 (Parker et al., 2020) and 28,000,000 (Hamel et al., 2020) people attended BLM protests in the span of just a few weeks. More important with respect to the current study is that, within certain bounds, the frequency at which protests take place may be more informative than the exact number of people in attendance. For frequency tells us something about diachronic engagement, which is, at any rate, unlikely to be sustained without relatively high and consistent levels of turnout and support.
Network creation and clustering
To examine social connections, we first generated a retweet network (Alfano et al., 2022; Sullivan et al., 2020): a weighted undirected network in which nodes represent authors and the weight of an edge represents the number of times that one user retweeted the other. Bare retweets (rather than quote tweets) almost always indicate endorsement of content (Metaxas et al., 2014), so looking at patterns of retweeting is a good way to reveal affiliative networks. Self-retweeting was discarded. The original retweet network consisted of ~18 million authors with ~98 million directed edges representing ~118 million total retweets. The vast majority of these authors were connected to each other by a single retweet. We further culled the network by eliminating edges with a weight ≤3 and then taking the largest connected component ( ~655 k nodes, ~1.7 M edges) for further analysis.
To find users who clustered together in their retweet behavior we used the Leiden community detection algorithm (Traag et al., 2019) as implemented in igraph (Csardi and Nepusz, 2005) and the Python leidenalg package (Traag, 2020; Traag et al., 2019). We kept clusters containing at least 5% of the original nodes. The remainder of the analyses were conducted on authors who fell into one of these clusters.
Autoregression
To examine the first research question, we conducted a timeseries analysis of the number of tweets per day by each group. An autoregression (AR) predicts the value of a variable on a day as a function of time-lagged values of both that variable and other variables. Autoregression was used successfully by Gunn et al. (2018) to examine online timecourse data about shootings and gun control as a proxy for political engagement. We looked at two autoregressions:
-
Model I: “Naive” AR The first autoregression model considered only the number of tweets per day for a group, and predicted these as a function of their own lagged tweets plus lagged protests per day as an exogenous variable. This estimated how responsive groups were to external events (using protests as a proxy) and how quickly that responsiveness decayed.
-
Model II: “Full” Vector Autoregression (VAR) The second analysis used vector autoregression (VAR) and considered the responsiveness of both group tweets and of protests as a function of lagged versions of each. The full VAR model differs from Naive AR in two respects: it treats protests as an endogenous variable (and so protests might be influenced by tweets as well as influencing them), and allows for the possibility that groups might influence one another.
The number of tweets per day varies by orders of magnitude over the course of the study, and the number of protests per day is on a smaller scale (and occasionally 0). To normalize the data to a standard scale we used the z-score of tweets and protests per day (calculated separately for each). When interpreting the results, therefore, the coefficients should be read as showing the effect of a one standard deviation (SD) variation in the corresponding variable in terms of SD units. The transformed scores were stationary using the AD Fuller test (see SI Section 3 for details). Both simple autoregression and VAR were calculated using the Python statsmodels package (Seabold and Perktold, 2010). Transformed protests were treated as an exogenous variable for the first analysis, and as an endogenous variable for the second. Data for the missing period was dropped from the analysis.
Topic modeling
To address the second research question, we turned to topic modeling. Topic modeling provides a useful way to give a high-level summary both of the themes in a corpus and of the particular content of individual documents. Allen and Murdock (2022) suggest that topics can be fruitfully thought of as indicating the contexts in which individual authors present and understand their own work. In topic modeling, documents \(d\in {{{\mathcal{D}}}}\) are represented as distributions over topics and topics \(t\in {{{\mathcal{T}}}}\) are distributions over words \(w\in {{{\mathcal{W}}}}\). In Latent Dirchlet Analysis (LDA), the weights are obtained by Bayesian updating, taking as priors Dirichlet distributions (the conjugate prior of the multinomial distribution). The matrix of frequencies of words per document with dimension D × W is decomposed into a matrix of dimension D × T representing the distributions of topics per document, and a matrix of dimension T × W representing the distributions of words per topic. As other unsupervised methods, topics allow for a lower dimensional summary of the data.
We examined original tweets made in 2020 and 2021 by each author who had been assigned to one of the main clusters. Note that while the clusters were made on the basis of retweet behavior, the tweets used need not have been retweeted to be included. Tweets were preprocessed and aggregated by author before fitting the model (see SI Section 1 for details about preprocessing). Each document in our corpus consisted of a single author’s aggregated preprocessed tweets. Aggregating author tweets for the initial topic model gives a better sense of the interests of individual authors (Alfano et al., 2022), as well as avoiding problems caused by generating topic models on very short fragments of text.
Latent Dirchlet Analysis was performed using the default parameters in scikit-learn 1.02. Our research question preferred a relatively small number of topics. To choose the specific topic number k we first built models for k = {3, 6, 9. . . 45}. For each model, we then tested the ability of a linear discriminant analysis to classify authors into clusters, taking the mean performance over 10 75/25 random train-test splits. Each model performed well above chance, with a clear improvement at k = 24 and diminishing returns thereafter (see SI Section 4). We thus used k = 24 for further analysis.
To give a qualitative picture of each group’s concerns and how they changed over time, the fitted topic model was then used to transform each individual tweet from each author. Each tweet was assigned to the topic corresponding to its maximum loading, and the proportion of tweets per topic per day was calculated for each group.
Results
Descriptive results
The clustering of the retweet network produced three clusters that were at least 5% of the original network, jointly covering 50% of authors in the the graph. To characterize the groups, we manually examined the top posters in each group, ordered by PageRank. Table 1 shows some of the top posters for each group, organized by our proposed descriptive label.
The groupings we found were in line with previous work, reflecting a general division between pro- and anti-BLM communities (Araque et al., 2020; Gallagher et al., 2018; Ince et al., 2017; Ray et al., 2017), as well as more fine-grained distinctions within these two camps using different timeslices and clustering methods (Alfano et al., 2022; Stewart et al., 2017). Note, for example, that the Right group contains the account of then-president Donald Trump and right-wing standards such as Breitbart news. Similarly, the Center-Left group contains mainstream Center-Left news outlets, the ACLU, and prominent civil rights lawyers. The Activist group is more mixed, but includes numerous accounts that self-describe as activist as well as a number of online left-wing news outlets.
Figure 1 visualizes the subgraph of the retweet network containing the top authors in the three largest clusters. It shows a familiar picture of political polarization: by and large the Right retweets the Right and the Left the Left, with a few bridge accounts connecting both. The Activist group is, unsurprisingly, closely aligned with the Center-Left, but forms a distinct cluster including smaller pockets with unusual alignments.
Nodes represent authors, edges weighted by number of retweets of one author by the other. Only the top 25% of retweeting nodes in each cluster are shown; apparently solitary pockets are connected to the others in the group on the strength of weak ties here omitted. Layout by Gephi’s forceatlas2 algorithm.
As Fig. 2 shows, both Center-Left and Right authors were engaged in BLM discourse from the beginning of data collection, with a substantial number of tweeters on both sides of the political divide. As one would expect, there was a significant spike in activity after the murder of George Floyd, with daily activity jumping roughly two orders of magnitude (see SI Section 2 for similar figures on rewteets and unique authors per day). Previous work shows that the killing of Michael Brown on August 9th 2014 generated a similar spike, albeit at a lower order of magnitude (Giorgi et al., 2022; Ince et al., 2017).
Autoregression
The naive autoregression analysis showed differences between groups both in responsiveness to protests and in the rate at which tweet activity would be expected to decay over time after an initial shock (either a 1SD increase in tweets or a 1SD increase in protests). Letting s be the decay coefficient and r the protest coefficient, Activists were most responsive to protests but with a faster decay (s = 0.76; r = 0.14), while the Right were less reactive to tweets but had a much slower decay (s = 0.92; r = 0.06). Center-Left accounts were somewhere in the middle (s = 0.88; r = 0.10).
To visualize our results, we plot the impulse response functions (IRFs) for both Models I and II. IRFs analyze “interactions between variables in a vector autoregressive model...” by representing “reactions of the variables to shocks hitting the system” (Lütkepohl, 2016). As the left half of Fig. 3 shows, the effect of a 1 SD-deviation in tweets lasts substantially longer for Right posters than others. Conversely, the right half of Fig. 3 shows that a 1SD deviation in protests predicts a much higher peak from Activists, but this effect decays more quickly, with the effect on the Center-Left larger by six days after, and that on the Right by day 12. Practically speaking, this means that Activists tend to show much more activity in response to protests, but that this activity dies off relatively quickly; the Right, by contrast, doesn’t respond as strongly to protests but their activity is sustained for a much longer time.
The full VAR model (Model II) paints a more nuanced picture. Figure 4 depicts the statistically significant coefficients for each variable (the full table can be found Section 3 of the supplemental material while Supplemental Fig. S5 shows the impulse response functions for each pair of variables).
Several results are worth noting. First, the decay coefficient for Activists is now highest of all. Indeed, as the full IRF figure shows, a 1-SD increase in tweets for Activists would have a statistically significant effect as far out as day 20, while the same change in other groups would drop below significance by day 10. The sensitivity results to protests remain the same (though the coefficient for the Right is not statistically significant).
Model II shows more complex interactions between groups and protests. Notably, the Activists appear to have a relatively strong effect on protests (whereas neither Right nor the Center-Left tweets make an appreciable difference). This fits with previous work showing that activity on Twitter can be predictive of later protests (Bahrami et al., 2018; Cadena et al., 2015), and that the predictive value of Twitter for protests often depends on the specific situation of authors (Korolov et al., 2016; Mooijman et al., 2018). The effect we see might be due to direct influence (calling for protests, calling attention to injustices, and so on), or to an indirect sensitization effect.
On the other hand, tweets by the Center-Left have a reasonably strong influence on the Right, while there is a weaker influence on Center-Left tweets from both Activists and the Right. This means that Activists drive activity both in the Center-Left and, after a delay due to their direct dampening effect, the Right. There is no converse effect of note on Activists, however.
The net effect of this influence, as seen in left column of figure of the full IRF figure, is to reproduce the pattern seen in the naive autoregression. A 1-SD change in protests causes a rapid peak in the Activists, and a smaller, slower-to-peak and slower-to-decay response in the Right. The latter response is predicted not just by the initial protests, but also by accumulated influence from the Center-Left.
In sum, the strongest predictor for each group, on either model, is its own activity on the previous day. There is relatively limited interaction between Activists and the Center-Left, and a slightly negative influence between Activists and the Right. System shocks appear to have a rapid and strong influence on left-wing political positions, followed by a slower but more sustained right-wing response that appears to be partly driven by a reaction to the initial left-wing response.
Topic modeling
The full 24-topic model is presented as Table 2. There are a number of topics that express distinctly pro- and anti-BLM viewpoints, as well as neutral topics about protests and more general “discursive” topics. Some topics (such as #3) represent features of BLM discourse that have been present from early on, while others (such as the George Floyd and Breonna Taylor focused #1 and the QAnon-inflected #18) reflect developments arising in 2020 and beyond.
The topics identified in our model are broadly consistent with both general theories of social movement frames (Benford and Snow, 2000; Snow et al., 1986) and several recent studies that focus specifically on Black Lives Matter (Gallagher et al., 2018; Giorgi et al., 2022; Ince et al., 2017; Ray et al., 2017). In line with general theory, we find that topics cluster into three broad categories: serving diagnostic, prognostic, and motivational functions respectively. For instance, in emphasizing police brutality and racial injustice, several topics (e.g., 2, 14, 15) take on a distinctly diagnostic role: articulating grievances and stressing the need for change. Other topics (e.g., 6, 10, 11) transform these grievances into action plans: calling for protests and other forms of political resistance. Previous work suggests that these prognostic topics played an important role in organizing BLM activism (Keib et al., 2018). Finally, to compel the community into action, BLM, alike other movements, enlists various motivational frames (e.g., 0, 1, 12): mobilizing activists by naming victims, naming perpetrators, and foregrounding the group’s collective history and identity (Brown et al., 2017; Winstead, 2017).
Several of the themes that we identify are also comparable to those found in previous studies, including studies that focus specifically on counter-frames. Topics 20, 22, and 23 for instance corroborate the results of Ince et al. (2017), Gallagher et al. (2018), and Giorgi et al. (2022), all of whom find that #AllLivesMatter and #BlueLiveMatter are dominant frames for articulating anti-BLM attitudes. Worth noting is that our method for identifying topics differs somewhat from previous work. For instance, in contrast to Giorgi et al. (2022), we include both retweets and original tweets rather than original tweets only. Relatedly, our topic models reflect themes that are dominant within particular retweet networks, as opposed to themes that emerged around specific hashtags. A notable upshot of our approach is that it sheds lights on several themes missed by other models—e.g., topic 2, which portrays BLM activists as “terrorists”. To the best of our knowledge, Ray et al. (2017) are the only other authors to pick up on this topic, ablbeit in the context of the murder of Micheal Brown. We will return to how and why this frame (re)emerged in the context of George Floyd’s murder.
Figure 5 visualizes the top 12 most common topics by tweet for each cluster, along with wordclouds representing the corresponding topics. The dashed vertical line indicates the murder of George Floyd, which made a notable difference in both quantitative and qualitative aspects of online BLM discourse.
Colors correspond to wordclouds representing each topic; topic number is next to each word cloud. Vertical line is date of George Floyd’s murder. Timeseries for each topic smoothed using a 15-day linear Savitzky-Golay filter (Savitzky and Golay, 1964).
We note several patterns. First, both Center-Left and Activist posters had relatively consistent levels of engagement across topics. The notable difference, coinciding with the murder of George Floyd, was an increase among Activists in the proportion of tweets that loaded on topic #0 (gray), which contains a number of Floyd-related terms. This was at the expense of more generic topics—notably the discursive topic #16 (blue) and the general topic #1 (orange), suggesting a more focused discourse.
There are further differences between the groups. The Center-Left appeals more consistently to topic #17 (green), which appears to focus on systemic racism. The activist loadings are more variable over time even with smoothing, consistent with a greater responsiveness to external events. (Note the sudden dip just shy of 400 days that coincides with the Jan 6 2021 Capitol Building insurrection; there various minor, mostly Trump-related, topics briefly take over.) The other notable difference between the Center-Left and other groups is a greater willingness to discuss both protests in the context of Trump (topic #8, gold) and the election more generally (topic #7, dark purple) than either Activist or Right accounts, suggesting a belief that it might be a winning election issue.
The Right tweets differ from the others in several notable ways. On the one hand, the murder of George Floyd increased engagement with BLM issues generally (c.f. also Fig. 2) This is reflected in the topic modeling, where the proportion of tweets devoted to topic #1 (orange) increases dramatically. Before Floyd there was a larger concentration of BLM-themed tweets on other tangential topics—especially the immigration- and military- focused topic #21 and the COVID-themed topic #5. These become almost entirely absent afterwards. (Not pictured in the graph as they are not part of the top 12 topics, but see SI Section 4 for details). This is consistent with the results of Dunivin et al. (2022), and indeed suggests that the BLM movement has garnered increased attention across the political spectrum. The shift away from apparent distractor topics (such as immigration and COVID) and towards general discussion among the Right suggests more consistent and coherent engagement.
On the other hand, the manner in which the Right talks about BLM suggests a more complex narrative. Most notable is the rapid increase in topic #2 (purple), which contains words connecting the BLM movement to Marxism, antifa, terrorism, and other bugbears of US right-wing discourse. This topic is barely present in other groups. Moreover, it plays only a minor role in the Right before Floyd but becomes one of the standard framings after. Pro-police rhetoric (topic #23, dark blue) is slightly diminished immediately following Floyd’s murder, reappears in roughly the same proportions shortly thereafter, but loses ground relative to topic # 2.
Hence, after Floyd’s murder, we see what appears to be a permanent shift in the dominant framing for Right-wing tweets. Before Floyd, pro-police rhetoric plays an important role; after, the pro-police tweets are swamped by the “antifa terrorist” frame.
The frame shift for the Right authors is so pronounced that we performed a further ad-hoc analysis to examine its causes. To rule out the possibility that this shift was driven by an influx of new participants with pre-existing “anti-antifa” sentiments, we split the Right into authors who made at least one tweet before Floyd’s murder (n = 27,176) and those who didn’t appear in our dataset until after that date (n = 82,043) and re-ran the same proportional analysis as depicted in Fig. 5. The results in Fig. 6 suggest that the two groups of Right authors are basically indistinguishable. The “antifa terrorist” framing of protesters was equally popular with both pre-existing authors and newcomers alike, suggesting a global shift in framing by the Right rather than discursive infiltration by a specifically “anti-antifa” contingent.
A final point is worth noting. There are three topics that focus on protests, but which appear to have different valence. The Right favors topic #19 (pink), which is concerned with protests but appears to be focused more on the police responses to protestors and on specific cities. Activists favor topic #10 (brown), which has a distinct focus on police brutality. The Center-Left appears to incorporate both, as well a topic #8 (gold), which incorporates a heavy focus on Trump as well. This suggests that our model is sensitive not just to discussion of protests, but to differences between the ways groups frame protests.
Discussion
General remarks
Our work suggests that Activists and right-wing tweeters represent two different poles of a speed/endurance tradeoff. The former favors focused, agile, and responsive attention to immediate events; the latter favors slow and sustained attention to a consistent alternative framing.
These results are consistent with previous work showing differential dynamics of online attention among different political groups. Online searches about gun control for instance spike after mass shootings in the US, which drives a reactive, slower, but longer-lasting interest in searches about gun purchasing among right-leaning individuals (Gunn et al., 2018). By the same token, Lee et al. (2022) recently showed that left-leaning politically controversial videos on YouTube (including videos about BLM) tended to get more views, but that right-leaning videos overall garnered more tweets and attention over a longer period of time.
Why do the two political poles differ? Multiple mechanisms may be at work; our results shed light on several of them. First, our VAR analysis showed an asymmetric influence between Center-Left and Right tweets. This suggests that the asymmetry in attention is partly driven by the Right reacting to political opponents more than the reverse. This is consistent with the recent finding of Wu and Resnick (2021), who found that conservatives are much more likely to engage with left-leaning videos on YouTube than the reverse. Indeed, one conclusion suggested by both the VAR and the topic models is that the Right doesn’t seem to respond directly to protests. Instead, they respond indirectly to protesters (Activists) and directly to their (Center-Left) advocates, both of whom are portrayed as threatening American values and institutions. Given that perceived threat is among the strongest predictors of social support for continued repression of aggrieved groups (Edwards and Arnon, 2021), this frame clearly reflects the Right’s strategic interests.
Second, differences in offline structural forces may play a role. Tufekci (2017) has argued that online platforms have made protests at once both powerful and fragile: Twitter is good at mobilizing supporters, but less useful for building the organizational structures that support long-lasting change. Conversely, Schradie (2019) has argued that conservatives have had an advantage on digital media because they are ideologically more hierarchical, and therefore better-poised to promulgate consistent messaging. This argument is consonant with Ray et al. (2017), who note that the online counter-narrative to BLM is controlled by a tightly organized group of conservative ideologues, some of whom have close links with the Republican party. Relatedly, Freelon et al. (2020) note that right-wing tweeters are more willing to work with legacy partisan media to achieve their ends. Finally, Benkler et al. (2018) argue that right-leaning mass media tend to be more willing to disseminate outlandish claims and continue to recycle those claims even after they have been debunked, therein keeping attention fixed on certain issues that are consistent with a chosen ideological frame. Benkler et al. (2018) outline a “propaganda pipeline” through which fringe stories on online message boards are picked up by mainstream outlets only to be pumped back into the internet with greater reach and longevity, regardless of veracity. In sum, Right-leaning media may be more willing to beat the drum on particular talking-points, further extending the expected “life-span” of a given news item. Hence these organizational structures might lend inertia to online activity.
A third likely mechanism is the framing effects identified by RQ2. Although the effects of competing frames are familiar from traditional media (Entman, 2007; Scheufele, 1999; Tankard et al., 2001), the decentralized, many-to-many architecture of platforms such as Twitter engenders a kind of “distributed framing” (Ince et al., 2017), where virtually any user can contribute to the dominant narrative. As the theorist Jacques Ellul noted, the work of propaganda is to provide a steady reinforcement or undermining of stereotypes. This process works best if it can latch onto "the fundamental currents of the society it seeks to influence” (Ellul et al., 1973, p. 38). Both framing and counterframing in the Black Lives Matter movement seek to tap into powerful social currents. US society has been responsive to matters of racial justice, particularly when injustice is framed as conflicting with other basic beliefs around personal dignity and individual freedom from government interference (Schuman et al., 1997). By contrast, the right-wing counternarrative we find on Twitter taps into long-established fears of the “enemy within” who seeks to undermine a way of life by undermining legitimate authority. The idea that US society is being undermined by a Marxist elite has long played a role in right-wing conspiracy theorizing (Stormer, 1964). More recently, Crimston et al. (2021) found that moral polarization combined with perceived breakdown of the social order, both of which are salient in the context of BLM, increases support for conservative/authoritarian leaders—and, presumably, the hierarchal organizations that Schradie (2019) identifies as key to sustained conservative attention on an issue.
Fourth, attentional dynamics and framing are likely to interact. Social media such as Twitter are especially sensitive to what Ridolfo and DeVoss (2009, 2017) term the “rhetorical velocity” of a piece of content—that is, to the speed at which different groups might promote or repurpose material. The particular affordances of Twitter give content an especially high rhetorical velocity. Retweets allow for rapid spread of information, while hashtags and quote tweets open up the possibility of appropriation (Ince et al., 2017; Ray et al., 2017). Ridolfo and DeVoss (2017) note, for example, how a Trump Hotels promotional tweet became repurposed years later as a vehicle for expressing outrage at Trump’s executive orders. The same dynamic is evident for BLM. For example, previous research has shown the role of the #AllLivesMatter hashtag in initially setting up a counternarrative to BLM, before being partly reclaimed by activists (Gallagher et al., 2018). Indeed, attempts among right-leaning users to draw attention away from “Black” and toward “All” or even specifically “Blue” lives are emblematic of the wider framing contest between these two camps, as well as the attentional implications of successful frame appropriation (Atkins, 2019; Carney, 2016; Gallagher et al., 2018).
Fifth and finally, the algorithms underlying Twitter’s promotion and recommendation of tweets is likely itself part of the explanation (Byrd et al., 2017; Cox, 2017). Huszár et al. (2022) provide evidence that the algorithm consistently amplifies right-leaning voices more than left-leaning ones. Algorithmic personalization systems on social media platforms such as Twitter are also tuned to promote engagement (Alfano et al., 2018, 2021; Burr et al., 2018). Hence which tweets get there “first or most often” is driven in part by what gets recommended, which is driven in turn by what others in a network find engaging. A good framing narrative begets recommendations, which begets more attention, which further entrenches a frame.
This frame-entrenchment effect is potentially exacerbated by the fact that algorithms do not promote the same content equally to everybody, but instead classify individual users as differentially more likely to engage with particular kinds of content based on their social ties. Inasmuch as clusters of socially connected users are thus likely to be presented with broadly similar content (Barberá et al., 2015; Cinelli et al., 2021), and that content exhibits particular frames, algorithmic recommendation potentially sets up what, to repurpose a familiar phrase, can be aptly referred to as “frame bubbles”. In much the same way that traditional filter bubbles lead to ignorance of alternative viewpoints (Nguyen, 2020; Sunstein, 2018), users trapped inside frame bubbles are liable to have their attention algorithmically fixed on only those frames that are consistent with their side of the narrative, leading that narrative to become entrenched further still. Worth adding to this is that algorithmic effects can interact with cognitive phenomena such as the “what you see is all there is” mindset (Kahneman, 2011). As and when this happens, users who are exposed to only one frame—say, the “Antifa terrorist” frame—may come to believe that this is the only available or reasonable interpretation of current events. To us, this suggests that the attentional dynamics of BLM discourse on Twitter are embedded within a complex, multi-layered, and potentially iterative framing sequence, where ideological frames are algorithmically filtered in ways that trigger lower-level cognitive framing effects, which in turn feedback into the algorithmic prioritization of particular ideological frames and, in so doing, keep attention fixed on one narrative rather than another.
Limitations and future directions
Our work is subject to several limitations. The roughly 15-day gap in data collection was unfortunate, though missing data should only be expected to reduce statistical power, and doesn’t appear to have affected the topic modeling in meaningful ways. Our reliance on API sampling rates, however, means that resampling the gap is not viable. Future work might consider verifying our findings using Twitter’s academic API (unavailable when the study started), giving an independent confirmation of these processes.
The groups we examined are relatively large and coarse-grained. They likely consist of multiple subgroups, but our pilot work on retweet networks has found it difficult to reliably identify stable, interesting subgroups. This is likely because broad groups tend to consist of more fluid, shifting coalitions, and so analysis of overall retweet networks either cannot find subgroups or cannot find useful ones. Further work on identifying temporally fluid subgroups would be helpful.
Our use of discrete econometric methods with a one-day grain throws away considerable information, especially in the context of rapid engagement with tweets. We chose this in part to match the timescale of our protest data. Previous work linking tweets to protest behavior has suggested that even hour-level data can be predictive (Mooijman et al., 2018). Future work on engagement might consider more sophisticated continuous models, such as the Hawkes point process approach used by (Lee et al., 2022).
Our study was limited in the sense that it did not collect all tweets by included authors, only those on BLM, and did not look at interactions other than retweeting. Given the asymmetry in online commenting noted by Wu and Resnick (2021), future work might study the extent to which alternative frames emerge in the course of interaction between different groups. Our methods were also not fine-grained enough to identify tweeters who changed their group affiliation over the course of the study. Given the general entrenchment of political attitudes in the US, we suspect that there are relatively few such authors, but it would be useful to have confirmation of this pessimistic conclusion.
Finally, as in most studies, our approach is fundamentally limited in its ability to collect surrounding context or detailed demographic information about tweet authors.
Conclusion
Our study shows that proponents and opponents of the Black Lives Matter movement show systematically different patterns of attention in the service of different ways of framing the core issues. Yet while it this may appear to be a familiar story of political polarization, we find two less familiar takeaway lessons.
We paid most attention to the two poles of the debate, but there is an important role played by the Center-Left group of tweeters as well. These authors are structurally and ideologically closely allied with Activists, but they have an intermediate attentional dynamic. The media landscape is constantly evolving in response to new technology (Garcia et al., 2019), and it may be that the current partisan divide in technological uptake has yet to reach an equilibrium point. Our work would suggest that traditional media can play an effective stabilizing role, smoothing out rapid fluctuations—and that it can do so regardless of where it sits on the political spectrum.
Second, we note that attentional dynamics might themselves represent an interesting point of intervention on social networks. Most proposed interventions on social networking concern either the content of messages (e.g., filtering misinformation) or the connectivity of the network (e.g., banning users). Both can be difficult to implement and politically controversial. Our work suggests that there is considerable variability in the attentional dynamics of online media as well. As a rough proposal, Twitter could ensure that hashtags that are rapidly gaining attention are presented alongside (less popular) hashtags that offer an alternative interpretation of events. Such a policy might prevent frame entrenchment by ensuring that users become aware of competing frames at an early stage. Alternatively, and perhaps better yet, Twitter could pair clearly partisan or racist hashtags with hashtags that activate superordinate identities, exposure to which has been shown to reduce out-group (Nordlinger, 1972) and cross-partisan animus (Brady et al., 2020; Van Bavel and Pereira, 2018).
In any case, it is important to emphasize that algorithms don’t simply promote tweets: they promote tweets of a certain age, with a certain speed, and with a certain history of visibility. It may be too much to hope for an uncontroversial set of neutral criteria that can moderate attentional dynamics for the overall good, but careful study of the ways in which attentional framing and dynamics interact will be a key to any solution.
Data availability
Python notebooks containing the entire processing chain for the paper along with instructions for adapting other datasets are available at https://osf.io/amv3r/. Also available are a collection of ‘dehydrated’ tweet IDs for both retweets and tweets, suitable for reconstruction using tools such as TWARC.
References
Alfano M, Carter JA, Cheong M (2018) Technological seduction and self-radicalization. J Am Philos Assoc 4(3):298–322
Alfano M, Fard AE, Carter JA, Clutton P, Klein C (2021) Technologically scaffolded atypical cognition: the case of YouTube’s recommender system. Synthese 199(1):835–858
Alfano M, Reimann R, Quintana I, Cheong M, Klein C (2022) The affiliative use of emoji and hashtags in the black lives matter movement: a twitter case study. Soc Sci Comput Rev
Allen C, Murdock J (2022) LDA topic modeling: Contexts for the history & philosophy of science. In: Ramsey G, De Block A (eds.) The Dynamics of Science: Computational Frontiers in History and Philosophy of Science. Pittsburgh University Press, Pittsburgh
Araque O, Gatti L, Kalimeri K (2020) MoralStrength: exploiting a moral lexicon and embedding similarity for moral foundations prediction. Knowl Based Syst 191:105184
Atkins A (2019) Black Lives Matter or All Lives Matter? Color-blindness and epistemic injustice. Soc Epistemol 33(1):1–22
Bahrami M, Findik Y, Bozkaya B, Balcisoy S (2018) Twitter reveals: using Twitter analytics to predict public protests. https://doi.org/10.48550/arXiv.1805.00358
Barberá P, Jost JT, Nagler J, Tucker JA, Bonneau R (2015) Tweeting from left to right: Is online political communication more than an echo chamber? Psychol Sci 26(10):1531–1542
Benford RD, Snow DA (2000) Framing processes and social movements: an overview and assessment. Ann Rev Sociol 26:611–639
Benkler Y, Faris R, Roberts H (2018) Network propaganda: manipulation, disinformation, and radicalization in American politics. Oxford University Press
Brady WJ, Crockett MJ, Van Bavel JJ (2020) The MAD model of moral contagion: The role of motivation, attention, and design in the spread of moralized content online. Perspect Psychol Sci 15(4):978–1010
Brown M, Ray R, Summers E, Fraistat N (2017) # sayhername: a case study of intersectional social media activism. Ethn Racial Stud 40(11):1831–1846
Burr C, Cristianini N, Ladyman J (2018) An analysis of the interaction between intelligent software agents and human users. Mind Mach 28(4):735–774
Byrd WC, Gilbert KL, Richardson Jr JB (2017) The vitality of social media for establishing a research agenda on black lives and the movement. Ethn Racial Stud 40(11):1872–1881
Cadena J (2015) Forecasting social unrest using activity cascades. PLoS ONE 10(6):e0128879
Carney N (2016) All lives matter, but so does race: Black Lives Matter and the evolving role of social media. Humanity Soc 40(2):180–199
Cinelli M, Morales GDF, Galeazzi A, Quattrociocchi W, Starnini M (2021) The echo chamber effect on social media. Proc Natl Acad Sci USA 118:e2023301118
Cox JM (2017) The source of a movement: making the case for social media as an informational source using black lives matter. Ethn Racial Stud 40(11):1847–1854
Crenshaw KW, Ritchie AJ, Anspach R, Gilmer R, Harris L (2015) Say her name: Resisting police brutality against black women. Technical report, Justice Research and Statistics Association
Crimston CR, Selvanathan HP, Jetten J (2021) Moral polarization predicts support for authoritarian and progressive strong leaders via the perceived breakdown of society. Polit Psychol 43(4):671–691
Csardi G, Nepusz T (2005) The igraph software package for complex network research. InterJ Complex Syst 1695: 1–9
Downs A (1972) Up and down with ecology: the issue-attention cycle. Public 28:38–50
Drakulich K, Denve M (2022) The partisans and the persuadables: public views of black lives matter and the 2020 protests. Perspect Polit 1–18
Dunivin ZO, Yan HY, Ince J, Rojas F (2022) Black lives matter protests shift public discourse. Proc Natl Acad Sci USA 119(10):e2117320119
Edwards P, Arnon D (2021) Violence on many sides: framing effects on protest and support for repression. Br J Polit Sci 51(2):488–506
Ellul J, Merton RK, Kellen K, Lerner J (1973) Propaganda: the formation of men’s attitudes. Vintage books, New York
Entman RM (2007) Framing bias: media in the distribution of power. J Commun 57(1):163–173
Freelon D, Marwick A, Kreiss D (2020) False equivalencies: online activism from left to right. Science 369(6508):1197–1201
Gallagher R, Reagan A, Danforth C, Dodds P (2018) Divergent discourse between protests and counter-protests: #blacklivesmatter and #alllivesmatter. PLoS ONE 13(4):e0195644
Garcia D, Galaz V, Daume S (2019) Eatlancet vs yes2meat: the digital backlash to the planetary health diet. Lancet 394(10215):2153–2154
Giorgi S (2022) Twitter corpus of the #blacklivesmatter movement and counter protests: 2013 to 2021. Proc Int AAAI Conf Web Soc Media 16(1):1228–1235
Gunn LH, Ter Horst E, Markossian TW, Molina G (2018) Online interest regarding violent attacks, gun control, and gun purchase: a causal analysis. PLoS ONE 13(11):e0207924
Hamel L et al. (2020) KFF health tracking poll—june 2020. Technical report, Kaiser Family Foundation, San Francisco, California
Hara N, Huang B-Y (2011) Online social movements. Ann Rev Inf Sci Technol 45:489–522
Harlow S (2012) Social media and social movements: Facebook and an online Guatemalan justice movement that moved offline. New Media Soc 14(2):225–243
Hendricks V, Vestergaard M (2019) Reality lost: markets of attention, misinformation and manipulation. Springer Nature
Huszár F et al. (2022) Algorithmic amplification of politics on twitter. Proc Natl Acad Sci USA 119(1):e2025334119
Ince J, Rojas F, Davis C (2017) The social media response to Black Lives Matter: how Twitter users interact with Black Lives Matter through hashtag use. Ethn Racial Stud 40(11):1814–1830
Kahneman D (2011) Thinking, fast and slow. Macmillan
Keib K, Himelboim I, Han J-Y (2018) Important tweets matter: predicting retweets in the# blacklivesmatter talk on twitter. Comput Hum Behav 85:106–115
Korolov R et al. (2016) On predicting social unrest using social media. In: 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM). IEEE. pp. 89–95
Lebron C (2017) The making of black lives matter: a brief history of an idea. Oxford University Press
Lee J, Wu S, Ertugrul AM, Lin Y-R, Xie L (2022) Whose advantage? Measuring attention dynamics across YouTube and Twitter on controversial topics. https://doi.org/10.48550/arXiv.2204.00988
Lütkepohl H (2016) Impulse response function. In: Macmillan P (ed.), The New Palgrave Dictionary of Economics, pages 1–5. Palgrave Macmillan UK, London. pp. 1–5
Metaxas, P. T. et al. (2014) Do retweets indicate interest, trust, agreement?https://doi.org/10.48550/arXiv.1411.3555
Mooijman M, Hoover J, Lin Y, Ji H, Dehghani M (2018) Moralization in social networks and the emergence of violence during protests. Nat Hum Behav 2(6):389–396
Nguyen CT (2020) Echo chambers and epistemic bubbles. Episteme 17(2):141–161
Nordlinger EA (1972) Conflict regulation in divided societies. Center for International Affairs, Harvard University
Parker K, Horowitz JM, Anderson M (2020) Amid protests, majorities across racial and ethnic groups express support for the black lives matter movement. Technical report, Pew Research Center, Washington, D.C.
Parker K, Horowitz JM, Igielnik R, Oliphant JB, Brown A (2017) The demographics of gun ownership. Technical report, Pew Research Center, Washington, D.C.
Raleigh C, Linke A, Hegre H, Karlsen J (2010) Introducing ACLED: an armed conflict location and event dataset: special data feature. J Peace Res 47(5):651–660
Ray R, Brown M, Fraistat N, Summers E (2017) Ferguson and the death of Michael Brown on twitter:# blacklivesmatter,# tcot, and the evolution of collective identities. Ethn Racial Stud 40(11):1797–1813
Ridolfo J, DeVoss DN (2009) Composing for recomposition: rhetorical velocity and delivery. Kairos J Rhetor Technol Pedagogy 13(2):n2
Ridolfo J, DeVoss DN (2017) Remixing and reconsidering rhetorical velocity. J Contemp Rhetor 7(2/3):59–67
Riker WH (1986) The art of political manipulation. Yale University Press
Savitzky A, Golay MJE (1964) Smoothing and differentiation of data by simplified least squares procedures. Anal Chem 36(8):1627–1639
Scheufele DA (1999) Framing as a theory of media effects. J Commu 49(1):103–122
Scheufele DA (2000) Agenda-setting, priming, and framing revisited: another look at cognitive effects of political communication. Mass Commun Soc 3(2-3):297–316
Schradie J (2019) The revolution that wasn’t. Harvard University Press, Cambridge
Schuman H, Steeh C, Bobo L, Krysan M (1997) Racial attitudes in America: trends and interpretations, vol. 2. Harvard University Press
Seabold S, Perktold J (2010) Statsmodels: econometric and statistical modeling with python. In: 9th Python in Science Conference. SCIPY
Shah DV, Domke D, Wackman DB (2001) The effects of value-framing on political judgment and reasoning. In: Framing public life: perspectives on media and our understanding of the social world. Lawrence Erlbaum Associates Publishers. pp. 227–243
Simon HA (1971) Designing organizations for an information rich world. In: Greenberger M (ed.) Computers, communications, and the public interest. Johns Hopkins Press, Baltimore, MD. pp. 37–72
Snow DA (1988) Ideology, frame resonance, and participant mobilization. Int Soc Mov Res 1(1):197–217
Snow DA, Rochford Jr. EB, Worden SK, Benford RD (1986) Frame alignment processes, micromobilization, and movement participation. Am Sociol Rev 51(4):464–481
Stewart LG, Arif A, Nied AC, Spiro ES, Starbird K (2017) Drawing the lines of contention: networked frame contests within #blacklivesmatter discourse. Proc ACM Hum-Comput Interact 1(CSCW):1–23
Stormer JA (1964) None dare call it treason. Liberty Bell Press, Florissant, Missouri
Sullivan E et al. (2020) Vulnerability in Social Epistemic Networks. Int J Philos Stud 28(5):731–753
Sunstein CR (2018) #Republic. Princeton university press, Princeton
Tankard JW et al. (2001) The empirical approach to the study of media framing. In: Reese SD, Gandy O, Grant AE (eds.) In: Framing public life: Perspectives on media and our understanding of the social world. Taylor and Francis. pp. 95–106
Traag V (2020) vtraag/leidenalg: Implementation of the Leiden algorithm for various quality functions to be used with igraph in python. Available athttps://github.com/vtraag/leidenalg
Traag V, Waltman L, van Eck NJ (2019) From Louvain to Leiden: guaranteeing well-connected communities. Sci Rep 9:5233
Tufekci Z (2017) Twitter and tear gas. Yale University Press
Van Bavel JJ, Pereira A (2018) The partisan brain: an identity-based model of political belief. Trend Cogn Sci 22(3):213–224
Winstead KC (2017) Black Catholicism and Black Lives Matter: the process towards joining a movement. Ethn Racial Stud 40(11):1855–1863
Wozniak KH (2017) Public opinion about gun control post–Sandy Hook. Crim Justice Policy Rev 28(3):255–278
Wu S, Resnick P (2021) Cross-partisan discussions on YouTube: conservatives talk to liberals but liberals don’t talk to conservatives. In: Proceedings of the Fifteenth International AAAI Conference on Web and Social Media, vol 15. AAAI Press, Palo Alto, California, USA
Wu T (2017) The attention merchants: The epic scramble to get inside our heads. Vintage
Acknowledgements
Work on this paper was supported by grants from The Australian Research Council (DP190101507 to C.K. and M.A.) and the John Templeton Foundation (#61378 to M.A.). This research was supported by use of the Nectar Research Cloud, a collaborative Australian research platform supported by the NCRIS-funded Australian Research Data Commons.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Ethical approval
This article does not contain any experimental studies with human participants performed by any of the authors.
Informed consent
In line with standard practice for analysis of online social media, informed consent from individual authors was deemed neither necessary nor practicable. Authors make tweets available under terms of service, which permit re-use by researchers. Tweets were collected using the Twitter API and used in line with the terms of service. Data presentation has been anonymized to avoid linking particular authors to particular tweets, and only verified authors on Twitter are identified by name.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Klein, C., Reimann, R., Quintana, I.O. et al. Attention and counter-framing in the Black Lives Matter movement on Twitter. Humanit Soc Sci Commun 9, 367 (2022). https://doi.org/10.1057/s41599-022-01384-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1057/s41599-022-01384-1