Inductive detection of influence operations via graph learning

Influence operations are large-scale efforts to manipulate public opinion. The rapid detection and disruption of these operations is critical for healthy public discourse. Emergent AI technologies may enable novel operations that evade detection and influence public discourse on social media with greater scale, reach, and specificity. New methods of detection with inductive learning capacity will be needed to identify novel operations before they indelibly alter public opinion and events. To this end, we develop an inductive learning framework that: (1) determines content- and graph-based indicators that are not specific to any operation; (2) uses graph learning to encode abstract signatures of coordinated manipulation; and (3) evaluates generalization capacity by training and testing models across operations originating from Russia, China, and Iran. We find that this framework enables strong cross-operation generalization while also revealing salient indicators-illustrating a generic approach which directly complements transductive methodologies, thereby enhancing detection coverage.


Introduction
Manipulation of public opinion by state-backed entities is an ongoing concern.Several influence operations (IO) campaigns intended to shape geopolitical discourse have been identified on various platforms-and particularly on social media [1][2][3][4][5][6][7][8][9][10][11][12].For example, IO campaigns designed to promote fake news, advance nationalistic narratives, and exacerbate political tensions have been detected across social media platforms including Twitter [1,5,11,12], Facebook [6][7][8][9][10]13], Reddit [2,14], and Gab [3,15], among others.Identifying and disrupting such campaigns is an ongoing challenge, in large part because positive attribution of foreign influence is time consuming and does not easily scale within or across platforms.Additionally, the rapid development and adoption of generative AI may enable IO to automate behaviours previously achievable only by human actors, disguising activity and enabling novel strategies which have greater efficacy, scale, reach, and specificity.Most methods of detecting IO to this point have relied on identifying and indexing specific indicators of previous campaigns, making these methods inherently transductive.While such methods will continue to play an important role in detecting and constraining IO activity, identifying increasingly novel and sophisticated IO campaigns will require inductive methods which can generalize from previous observations.We present an inductive learning framework, depicted in Figure 5, that addresses this challenge by combining data censorship, graph learning, and feature attribution to identify models and indicators that can generalize across operations and across time.
Previous work in detecting influence operations using machine learning has successfully identified a variety of IO campaigns and activity.Broadly speaking, there have been two main approaches: content-based and graph-based.Some examples of content-based approaches include: Smith et al. [16], who used narratives derived from topic models to classify Twitter IO accounts in French and English speaking networks; and Alizadeh et al. [17] who used post text and URL information to classify Twitter posts as belonging to IO or not.Examples of graph-based approaches include: Monti et al. [18], who used graph networks (GNs) to classify URLs as fake news or not; Vargas et al. [19], who used graph data to classify IO accounts on Twitter which display coordinated behaviour; and Smith et al. [16], who used a network discovery algorithm followed by causal impact estimation to understand the role of individual accounts in propagating IO narratives.
A previously distinct line of research in cybersecurity, kill chain analysis [20][21][22], focuses on identifying and disrupting threat actors at each phase of their operation.This approach formalizes various sequences of tactics, techniques, and procedures (TTPs) which IO and other cybercrime operations use to achieve their objectives.In particular, online operations kill chains enable the development of technical indicators which are signatures of cybercrime operations at various phases.These indicators can be used to detect future operations, identify abstract themes across campaigns, analyze trends, and compare TTPs across different operations and time periods.
These lines of research, as well as reports directly from social media companies, have elucidated a wide range of IO targets, objectives, strategies, and tactics.Many tactics involve the spread of malicious URLs [7], state-backed media, mis/disinformation [23], and particular narratives (e.g., pro-Russian narratives surrounding the Ukrainian war [6,8,9]); other tactics include near-simultaneous link sharing [24], troll farming [7], mass promotion of particular narratives [6,7,16], mass reporting of accounts and content [7,9], and mass spamming or "brigading" of specific pages, posts, and users [7].Identifying these tactics has enabled well-resourced social media companies such as Twitter, Meta, and Google to automate the detection of new campaigns that reuse TTPs on their respective platforms.This automation has in turn enabled rapid detection and response to coordinated IO activity.
Automated detection has greatly constrained the preferred tactics available to IO on relatively well-regulated platforms such as Facebook and Twitter.For example, networks of coordinated and near-simultaneous link sharing (<1 min.apart) are now quickly and routinely removed from these platforms [7][8][9][10][11]25].However, this conspicuous behaviour persists as an IO tactic due to the fact that social media ranking algorithms up-rank content with higher engagement, with immediate engagement having an outsized effect on relative ranking and ultimate reach of content [26].Hence, to artificially amplify specific narratives during critical periods, IO preferentially coordinate on very short timescales, even at risk of being detected.So while near-simultaneous coordination may be largely curtailed by platforms or even abandoned by IO in the future, coordination on short timescales is expected to continue.For particularly sophisticated IO networks, one would expect that future coordination patterns would mimic that of authentic users.Current landscape of automated detection on mainstream social platforms.Advances in automated detection will push influence operations towards less effective methods of coordination and more costly approaches to fake account creation.In turn, influence operations may be able to compensate by augmenting existing capabilities with emergent AI systems.Fake account detection [7,[9][10][11] has also greatly improved.In response, IO have tried to obviate detection by crafting realistic profiles that mimic authentic users in a process called persona building.The process of persona building has presumably been a manual effort to this point, as smaller numbers of these meticulously crafted fake accounts are observed as part of any IO compared to the much larger numbers of less sophisticated "spam" accounts (though part of this discrepancy may be a survivorship bias).A common approach to persona building is to mimic existing accounts that promote narratives favorable to the IO objective, such as inflammatory political content promoted by Russian and Iranian IO campaigns leading up to the 2016 and 2020 U.S. presidential elections [11,25].This process of mimicry requires significant investment of human effort, as this approach requires the generation of novel content such as text and images.However, it is not difficult to imagine that in the near future a single IO operative could automate the persona building process using novel AI tools to farm a large number of fake accounts.Indeed the use of GAN produced profile pictures [10,20] and deep fakes [27] has been reported.While the automated detection of near-simultaneous coordination and fake accounts will push campaigns towards less efficacious and more costly approaches (Figure 1), they may be able to compensate with greater scalability, novelty, and specificity enabled by AI.
While mainstream platforms have the resources and desire to improve regulation, alternative platforms are less equipped, and possibly unwilling, to follow suit.Exploiting this situation, the Russian origin Secondary Infektion campaign from 2014-2020 made use of over 300 platforms including WordPress, BlogSpot, Quora, Reddit, and LiveJournal to circulate fake news and seed fabricated primary sources [28].A subsequent Russian campaign from 2020-2022 (likely a continuation of the same operation) targeted 35 alternative platforms that intentionally have little or no regulation such as Gab, Gettr, Parler, and Truth Social [23].While all of these platforms combined have a smaller audience than most mainstream platforms, they demonstrate continued trends of IO in microtargeting specific audiences and diversifying channels of influence.Countering these trends will require methods of detection that can identify operations across platforms, as well as generalize previous observations on mainstream platforms to newly targeted platforms.Additionally, while mainstream platforms have thus far been proactive in identifying and removing inauthentic actors, it is unclear to what extent this will continue to be true.
Even in light of these trends, continuing to identify and index TTPs for transductive detection will still be paramount to constrain future IO.In other words, the foundation of IO detection will continue to be transductive-or based on specific, previously observed indicators.Transductive approaches will continue to be effective in constraining IO for two main reasons: 1) operations can only develop new TTPs so quickly; 2) previously indexed TTPs often represent the preferred tactics of IO, which they may be slow to abandon.In order to continue shaping public discourse in the near term, one can then expect continued reuse of TTPs, even if these are largely ineffective on mainstream platforms.In the long term, however, one can expect IO to develop novel tactics that avoid detection and reach larger segments of online users.On one hand, this means that future IO will likely have less impact per action (post, like, share, etc.) since they cannot maximally exploit the platforms in which they are embedded.On the other hand, AI systems such as StyleGAN2 [29], DeepFaceLab [30], GPT [31], and DALL-E [32] may allow IO to more easily craft realistic profiles and content, thereby enabling novel campaigns that employ previously costly tactics at greater scale.In such cases, it is unclear how effective transductive methods will be, if at all.Hence, developing inductive methods of detection will be necessary to proactively identify and disrupt novel campaigns which can consequentially alter public opinion in a matter of days (e.g., in the days leading up to an election [11,25]).To this end, we observe two fundamental techniques that IO use when manipulating public opinion: (I) linking to off-platform websites that are considered credible by the target audience and/or possessing decreased regulation; (II) coordinated promotion of content supporting specific narratives.
Arguably, IO can have little impact on public opinion without employing these techniques in some form.We use this observation to design (I) content-based and (II) graph-based indicators which are general enough to identify novel campaigns from previous campaigns, and use graph representation learning to encode abstract signatures of coordination from these indicators.In particular, we determine indicators that are not specific to any particular IO campaign by explicitly censoring previously identified content-and graph-based technical indicators.We call indicators resulting from this type of censorship generalized indicators, since they will be common across both IO campaigns and authentic users, and also across platforms.
We investigate how specific choices of generalized indicators and graph learning techniques can identify inauthentic actors across IO campaigns, thereby developing a framework that directly complements the transductive methodologies established in previous work.In doing so, we note the correspondence between the generalized indicators used here and previously used technical indicators: Furthermore, we investigate three specific advances of previous approaches: (1) Identification of content-based and graph-based indicators which enable cross-operation generalization; (2) Utilization of graph learning to encode abstract signatures of coordination, thereby automating graph-based feature engineering and inference; (3) Investigation of a broad coordination window, moving from near-simultaneous (<1 min.) to quasi-authentic (<100 min.)interarrival times.

Results
Following the framework presented in Figure 5, we assess the extent to which specific machine learning models and generalized indicators can identify IO accounts across campaigns, both intra-operation and inter-operation (results shown in Table 2).For this purpose, we select six IO campaigns (Figure 2) belonging to three coordinated operations: Russia, China, and Iran; and a comprehensive baseline described in the next section.We analyze intraand inter-campaign co-URL statistics in Figure 4, demonstrating operation specific trends and the independence of the three operations chosen.In Table 3, we determine which indicators enable cross-campaign generalization using an axiomatic attribution method, integrated gradients [33].In order to assess the generalization capacity of particular model and indicator choices, we formulate two tasks.The first is intra-operation classification (A in Table 2), where we train on a campaign of a particular operation (Russia, China, or Iran) and test on a later identified campaign of same operation.The second task is inter-operation classification (B in Table 2), where we train on all operations except the test operation.For the three campaigns in the training/validation set and the test set, this implies three subtasks for A and B. To enable comparison between the two sets of subtasks, we sample the validation and test sets for each respective subtask identically (e.g., the results of task A1 and B1 can be compared directly).This allows us to assess how well each model can generalize from independent operations based on any changes in performance from tasks A1, A2, and A3 to tasks B1, B2, and B3, respectively.
We evaluate the effect of varying the content-based feature set both in terms of stringency (γ max ) and minimum prevalence (k top ) on model performance in Figure 3.We choose MLP with all graph encodings (G.E.= † † † †) as a representative model since it consistently performed well across all subtasks.The effect of increasing k top improves model performance on all metrics in a nearly monotonic manner, ostensibly reaching saturation around k top = 2000.The effect of varying the maximum frequency ratio (γ max ) has a more nuanced effect on performance, but a fairly stringent value of γ max ∈ {0.43, 0.67} appears to produce greater generalization than smaller or larger values, with increases beyond γ max = 0.67 producing a nearly monotonic decrease in performance for F1(val/test), but not for AUC. (

Coordination analysis of composite dataset
In order to understand intra-and inter-operation patterns of coordination, we report co-URL counts between all campaigns in Figure 4.The 2x2 block pattern of co-URL counts along the diagonal of 4a and 4b suggests that each campaign of a particular origin is actually a continuation of the same underlying operation.Observing how these co-URLs are distributed as a function of interarrival time, we see that the earlier identified training campaigns in 4c have similar distributions of co-URLs as the later identified test campaigns in 4c in some cases.In particular, the campaigns of Chinese and Iranian origin appear to adopt near-simultaneous link sharing later than the Russian campaigns.This can be quantified by calculating the distance between CDFs for each campaign (4e).From 4f, we see that the accounts in the Chn.(19) and Iran (19) campaigns appear to use little to no coordination at short timescales compared to the baseline, but the Chn.(20) and Iran( 21) campaigns begin to display levels of coordination comparable to the Rus.( 18) campaign.The Rus.( 20) campaign displays greater levels of coordination than any campaign in our dataset, particularly at short timescales.

Feature Importance
For each model and subtask, we report the best performing graph encoding in each case (G.E. in Table 2).However, it is not clear from these results what the relative importance of each graph-based feature is on model predictions, or the relative importance of content-based to graph-based features.To quantify the relative importance of each feature set, we calculate the mean absolute integrated gradients (IG) for each feature over validation, test, and baseline sets for each subtask (Table 4 in the Appendix).In Table 3 we report the aggregated (arithmetic mean) IG values of each feature over all subtasks.We again use MLP as a representative model since it consistently performs well on all subtasks.
Overall, the net attribution of all graph-based features (node2vec, LE, RWPE, NF) appears to be substantially larger than that of all content-based features (domains), greater by roughly an order of magnitude.Notably, several quantities widely used in network analysis such as Laplacian Eigenmaps (LE), clustering coefficient, and betweenness centrality had a marginal impact on predictions, indicating that they provided little useful information for predictions and, since dropout was employed, that this information was not even redundant with other features.Meanwhile, graph embedding techniques such as node2vec and RWPE enjoy a relatively high utility, having a substantial impact on predictions.This result is not necessarily surprising since node2vec and RWPE essentially act as deep encoders, which can be decoded with high fidelity by deep neural networks (i.e.MLP and GNs, but not by LR and RF).What is surprising, on the other hand, is that several simple network quantities-degree, pagerank, and HITS-were as important to predictions as any other single feature.This implies that these quantities encode some information which is complementary to graph embedding techniques, and do so with only a single scalar value.

Discussion
Constraining influence operations is an ongoing challenge that will require continued advancement of detection capabilities in order to counter novel operations-particularly as they adopt powerful AI technologies.In particular, detection methods which go beyond established transductive methodologies and can identify novel campaigns in an inductive manner will be critical.Here we have examined the systematic application of generalized indicators and graph learning techniques, demonstrating a framework in Figure 5 which enhances detection coverage.Furthermore, this framework is broadly applicable to detecting manipulation on social media, and naturally complements detection using technical indicators identified in transductive methodologies.Overall, the most effective approaches utilized: (1) a fairly large content-based feature set (approximately 2000-2500 domains) with fairly stringent removal threshold applied (γ max ≈ 0.5); (2) a broad range of graph-encoding features, particularly node2vec, RWPE, degree, pagerank, and HITS; (3) a deep neural architecture.In other words, MLP and the three GNs outperformed LR and RF on every out-of-sample subtask.On the in-sample prediction tasks, as quantified by F1(val) of tasks A1, A2, and A3, RF actually outperforms all of the deep models.It appears that in this case RF was simply able to memorize patterns specific to the training set, as it fails to generalize to the test set.For task A, this failure is corresponds ∼5 point decrement on F1(test) and AUC(test) relative to the deep models.On task B, for which predictions had to be made by generalizing across campaigns, an even larger decrement of 10-15 points is observed across all metrics.

I. Data Collection
Among deep models, MLP consistently achieved high performance across all subtasks, achieving the highest AUC(test) in all cases but one.However, on all out-of-sample F1 scores one or more GNs outperformed MLP in all cases.This indicates that while GNs perform well at the decision boundary for classification (the näive boundary α=0.5 in all cases), misclassifications were by a greater margin than for MLP.This is possibly an indication that while the increased expressive power of GNs was beneficial for classifications on average, this could lead to even further errors on accounts which were the most difficult to classify.
While we have outlined several specific approaches for selecting feature sets and models which can generalize across campaigns, there are a number of improvements which would likely further this work.First, there are presumably other sets of generalized indicators which may be in aid in out-of-sample identification of IO accounts.The first is text data, which we did not investigate.On one hand, text embeddings which encode specific narratives or ideologies could presumably provide useful information which is not necessarily specific to a particular IO campaign.But on the other hand, the length of text shared and the prevalence of text varies greatly among accounts even with a single platform, and even more so from platform to platform.For content-based features overall, a unified approach for encoding the content and semantics of text, images, audio and video would be ideal, since focusing on a specific form of content could lead to blind spots.For example, the multi-modal encoders used by the multi-task agent Gato [34] or the GPT-4 system [31] could allow for training on text posts and making inferences on video posts, and so on.Though the co-URL is a effective and robust tool for quantifying coordination, there are many other edge-wise features e ij which quantify pair-wise relationships in a graph.In particular, several graph-based measures which quantify similarity could add useful information: node-similarity measures based on nearest neighbors such as common neighbors, Jaccard Index, Adamic Adar, and preferential attachment coefficients; as well as path based measures such as shortest path lengths, Katz measure, and hitting time.Other similarity measures can be derived from graph learning measures by applying various distance metrics such as L p , cosine, and Sørensen-Dice distances to pairs of graph encodings.Fortunately, message passing graph networks provide a natural way to incorporate similarity measures (or any edge-wise features) into predictions, making this type of extension straight forward.
Although we attempted to present a range of graph networks-convolutional, shallow message-passing, and deep message-passing graph networks-there are myriad design dimensions of graph networks which we did not explore.Among these are more advanced sampling strategies, attention mechanisms, and various message passing architectures.However, the results in this study are adequate to suggest that both graph learning and graph networks will be an indispensable tool for detecting IO into the future.
Finally, we examined a "hard" measure of coordination, the co-URL, in this paper.There could in the future, however, be softer forms of coordination which evade detection.For example, different URLs could lead to semantically or literally identical content, which would not be measured as coordination by our current approach.To hedge against this possibility, one could encode the content of URLs as embeddings and define coordination as a function of the distance between embeddings.This would generalize the current co-URL approach in which we implicitly assign identical URLs distance 0 and distinct URLs distance ∞.This is yet another indication of the utility of multi-modal content encoders in future influence detection efforts.
In summary, we have demonstrated an inductive approach to detecting IO which allow for continued utility into the future and generalization capacity across campaigns, enabling identification beyond technical indicators identified by transductive methodologies.We have illustrated how specific content-and graph-based features realize these objectives, as well as how one can systematically identify these features.Finally, we have identified several refinements of the current approach, enabling continued advancement in the automated detection of IO even as these campaigns continue to evolve.

Data Collection and Inclusion Criteria
For the purpose of evaluating intra-and inter-operation generalization of machine learning models, we selected IO of several origins (Russia, China, and Iran) for which there were significant campaign sub-networks identified at different times.To this end, six Russian, Chinese, and Iranian origin campaigns identified by Twitter between 2018 and 2021 were suitable.Another important aspect considered was the availability of baseline accounts which both interacted with the IO (to provide adequate coordination measures) and remained independent of IO (to reduce bias in frequency measures).To this end the baselines of [17,19] were used (88.5% of baseline accounts), in addition to 1129 accounts (11.5% of baseline accounts) highly interacted with by IO, as measured by co-URLs.Additionally, the 1129 high-interaction accounts were sampled at various maximum follower thresholds (n = 10 2 , 10 3 , and 10 4 ) since accounts with many followers (n > 10 4 ) were disproportionately interacted with by IO accounts.This resulted in an aggregate baseline which was highly connected to the IO and yet provided broad coverage of various types and sizes of accounts.
For this study, we focus specifically on IO accounts which displayed reasonably organic patterns of sharing, which evaded detection for some amount of time, and which could have reasonably had some impact on public discourse.We therefore selected from the initial data set accounts which: (1) were active for at least 3 months; (2) had at least 300 Tweets; (3) had at least 200 URL shares; (4) shared at least 5 unique domains; (5) had at least 10 co-URLs with at least 2 neighbors.

Extracting raw indicators
To obtain features from the raw tweet data, we expand all shortened URLs contained in tweets using the URLExpander library [35].Then, to obtain domain counts for each account, we use the tldextract library to extract the domain of each URL tweeted by an account.We then generate node-wise and edge-wise features from the raw URLs and extracted domains: Edge features: As a measure of coordination between accounts, we compute the interarrival time between shares of the same URL (co-URLs) and bin the results into 1 minute intervals to obtain a vector of co-URL frequencies between all pairs of accounts.Denoting the interarrival time as τ , the co-URL count between the ith and jth account in the interarrival window τ − 1 < t ≤ τ is then denoted e ij τ .
Node features: We use the raw counts of the most frequently shared top-level domains (e.g." cnn.com, youtube.com,nytimes.com)from each account in the composite dataset.To avoid having the models simply memorize domains which are specific to a particular IO, we censor domains where more than γ max of occurrences of the domain originate from the IO training set relative to the baseline set.For example, riafan.ru,histantv.com,and tel-avivtimes.com are censored by this method for any γ max < 10 4 since each of these domains originate at least 10 4 more frequently from IO accounts than from the baseline.See Appendix E for extended examples of censored domains.

Content-based generalized indicators
Following transductive methodologies, previous work has enabled the rapid detection of IO which attempt to propagate specific domains containing fake news, propaganda, and malware [7,10,27].Accounts sharing these domains, particularly in a coordinated manner, are now routinely identified and removed by mainstream platforms.
To identify influence efforts beyond these more flagrant indicators, we investigate domains which are commonly shared and yet may be useful indicators of IO activity.In order to quantify the extent to which a particular domain is either common to some baseline users or specific to an IO, we define the relative frequency for the ith domain in the IO training set (y = 1) relative to a baseline set (y = 0) as where the domain term frequencies for the ith domain are and f (y=1,0) i are the raw counts of the ith domain in the IO and baseline training sets.We then censor any domains which exceed a threshold γ max such that we remove domains which are specific to the IO training set with variable stringency.Particular choices of γ max can censor domains which only appear in the baseline set (γ max = 0), appear in the IO set no more than parity (γ max = 1), or which appear only in the IO set (γ max = ∞).Additionally, we retain only a select number of the censored domains, k top , the top-k domains when sorted in descending order by absolute frequency.We can then vary the stringency and minimum prevalence of our content-based feature set with γ max and k top , respectively, in order to investigate the effect of content censorship on generalization.Moreover, at less stringent thresholds (γ max > 1), we can observe the effect of directly including technical indicators of previous campaigns used in transductive methodologies.

Graph-based generalized indicators
Coordination by IO on social platforms has taken many forms, including mass spamming, mass reporting, and coordinated content sharing.Among these tactics, coordinated content sharing has perhaps been the most widely observed, and is the chief tactic employed by many campaigns.In particular, many takedown efforts have used nearsimultaneous co-URL sharing as the primary means of both identifying and substantiating coordinated inauthentic behaviour.In addition to the ubiquity of co-URLs across a variety of campaigns and platforms, they also have the appealing properties that they are agnostic to the specific content shared, are easily defined across platforms, and automatically imply a graph structure between accounts.Furthermore, each co-URL has an associated time between shares, the interarrival time, whose distribution can provide further insight into coordinated activity between accounts (see Figure 4 for examples).
While near-simultaneous co-URLs are a useful indicator for automated detection of IO activity, this behaviour is not guaranteed to persist, particularly for campaigns with high operational security (i.e., those which closely mimic authentic users).Therefore, generalized indicators of coordination should incorporate a broader time frame within which future campaigns are likely to operate.In particular, we utilize co-URLs with interarrival times from τ = 0 to 100 minutes.This time frame includes near-simultaneous sharing (<1 min.), the majority of retweets (<20 min.[36]), and the median half-life of tweet views (∼80 mins.[37]).We denote the number of co-URLs with interarrival times τ − 1 < t ≤ τ as e ij τ , and the composite co-URL vector as e ij = {e ij 1 , . . ., e ij 100 }.The graph structure implied by co-URLs, however, cannot be used directly by machine learning models to make node-wise inferences.Two approaches for utilizing graph structured data in machine learning applications are to: (i) learn unsupervised feature vectors for each node in the graph (graph embedding); and/or (ii) define graph operators which systematically aggregate data over the graph at each layer in a neural network.Both of these techniques can be referred to collectively as graph learning [38], and neural networks utilizing graph operators as graph networks.
Graph networks can utilize co-URL data in ways which may or may not directly make use of near-simultaneous link sharing behaviour, thereby offering varying degrees of generalization capacity.For example, one can define a graph which censors near-simultaneous link sharing as follows: assign A ij = 1 if and only if two accounts share at least n URLs with interarrival times less than T , or in mathematical notation where H(•) is the Heaviside step function.This definition equally counts the contribution of all co-URLs with interarrival times less than T , thereby censoring any near-simultaneous behavior while still allowing a rigorous threshold for coordination.One can then define graph operators, such as GCN [39], in terms of this censored graph.A standard way of directly using vector-valued edge features such as co-URLs, on the other hand, is within a message passing [40,41] framework.Message passing defines graph operators directly as a function ϕ(e ij ) of the edge-wise feature vectors (i.e., allowing predictions to be made directly using near-simultaneous co-URLs e ij 1 ).In order to understand the generalization capacity of graph networks with varying degrees of graph censorship, we implement three graph networks as follows: GCN, which utilizes only the censored graph; MP-GCN(s), a message passing variant of the base GCN architecture with a shallow message passing function ϕ = L, where L is a linear operator; and MP-GCN, which uses the more common deep message passing function ϕ = f , where f is a neural network.Comparing the performance of these three architectures allows us to examine the effect of graph censorship, as well as compare different graph network architectures in identifying IO.

Graph Encoding
In order to understand the utility of different types of graph-based features (from network analysis to graph learning) as well as the utility of specific features, we incorporate several candidate quantities in a node-wise feature vector which we call the graph encoding.Due to the asymmetric nature of our dataset (co-shares of content by IO accounts are visible in the dataset, but co-shares of IO account content have been removed by Twitter) we treat all graph quantities in an undirected manner by setting e ij ← e ij + e ji .All graph-based features are derived from an undirected graph computed from the co-URL vectors as in Equation 3where we select thresholds of n = 10 and T = 15 to censor the graph.While more stringent n would produce a more robust graph, we find that further reducing the number of edges rapidly disjoints the graph, making graph learning techniques infeasible.The temporal threshold T = 15 minutes represents a window in which IO could coordinate effectively and yet avoid detection, while also censoring near-simultaneous link sharing.
Graph representation learning, including graph embedding algorithms such as node2vec [42], originated as an effort to automate the feature engineering process for graph prediction tasks such as node classification and link prediction.From a modern perspective, graph embedding techniques are unsupervised methods which allow one to systematically assign relational, functional, and structural information to each node in a graph.This information can greatly improve the performance of deep learning models, with or without graph operators, on graph prediction tasks.We choose three graph embedding algorithms for our purpose here: (1) node2vec, which encodes neighborhood information of nodes into dense embeddings; (2) Laplacian Eigenmaps, a non-linear spectral embedding technique which provides a local coordinate system on graphs and effectively encodes clustering within the graph; (3) Random Walk Positional Encoding, which is based on the graph diffusion operator and uniquely assigns node embeddings based on the k-hop topological neighborhood of each node.Each of these approaches, in principle, encode different aspects of graph topology and therefore can provide predictive utility independent of one another.In each case, the dimensions of the embeddings are chosen such that further increases yield no benefit to performance across models.
While graph embeddings are a sensible method of encoding topological information for predictive tasks, they do not necessarily preclude the utility of conceptually similar network analysis quantities.To this end we include several quantities which encode relational, functional, and structural information of graphs in our graph encoding: (1) degree, which for undirected graphs is simply the number of directly adjacent neighbors of each node; (2) clustering-coefficient, which quantifies the local clustering of each node as the amount of closure between the neighbors of each node; (3) betweenness centrality, a centrality measure quantifying the extent to which a node facilitates connection within the graph via shortest paths; (4) pagerank, a centrality measure which ranks nodes according to their relative importance within a network; (5) HITS, which also ranks nodes according to relative importance but assigns two scores quantifying the extent to which a node connects the graph (hub score) and is of relative importance within the graph (authority score).For undirected graphs, the hub and authority scores of HITS are identical.

Graph Networks
Given the node-wise and edge-wise data in our feature set, there are several GN architectures which are possible choices for the predictive task at hand.We sample several architectures of increasing expressive power such that we can compare the utility of different GN design choices and degrees of graph censorship.In particular, we perform ablation on message-passing rules for encoding the co-URL vectors e ij τ in order to compare various uses of this feature set.
In general, a graph network can be written as the series of operations where the set N (i) indicates the neighborhood of the ith node where A ij = 1.The simplest graph network that we employ is a spectral GN, the popular Graph Convolutional Network (GCN), with layers defined by the feature aggregation function [39] AGG where d i is the degree of the ith node.
There are a number of ways in which message passing rules can be defined, but for graph networks one typically defines the message passing rule as where the message passing function ϕ(•) can take as input both nodewise features h (l) i and edgewise features e ij .We then incorporate these messages into the aggregation step as We employ two message passing rules to encode the co-URL vector, the first of which is a shallow message passing rule which defines our MP-GCN(s): The second rule utilizes a neural message passing function, implemented as an L layer perceptron which defines our MP-GCN: where W (l) and b (l) are the weights and biases of the lth layer and a To compare with the base GCN implementation, we insert each message passing rule into the base GCN aggregation function as Thus we have performed ablation on the message passing rule over the three GCN architectures.

Model Training
In the LR and RF implementations we tune all hyperparameters to achieve the best model performance via gridsearch.
In the MLP and the three GCN variants we use the same hyperparameters: two hidden layers of 64 units, and a dropout probability p = 0.5 applied to all units in the hidden layers.In all message passing layers we apply a dropout probability of p = 0.2.For all MLP and GCN training we use a binary cross entropy loss and the Adam optimizer with a learning rate of 10 −4 .

Integrated Gradients
Integrated gradients [33] is an axiomatic attribution method for deep neural networks.Mathematically, the IG of a function F (x) with respect to the ith component of an input x and a baseline x ′ is where α parameterizes a straight line path from x ′ to x.This method provides a more robust attribution of predictions to specific features than directly evaluating the product of the gradient and feature value which has historically been a popular attribution method.When using IG, one selects a baseline where the model prediction is neutral.Calculating the IGs of each feature for an MLP, there is not an obvious baseline which yields neutral predictions, i.e., where F (x ′ ) = 0.5.For example, simply choosing the mean or minimum value of each feature over various subsets of the data produces predictions close to 0 or 1.We therefore construct an empirical baseline comprising the subset of all nodes such that 0.4 ≤ F (x j ) ≤ 0.6, or within ±0.1 of a neutral prediction.Setting x ′ = ⟨x j ⟩ then yields F (x ′ ) = 0.534 ± 0.018 over all six subtasks, which is approximately neutral while ensuring that no particular feature in the baseline takes on an extreme value (which might be the case if we simply chose j to be the single most neutral prediction).

Error propagation of aggregated metrics
In Figure 3, several performance metrics are aggregated over subtasks by computing their harmonic mean.For each subtask and choice of parameters (γ max and k top ), there is an associated uncertainty for each metric due to their dependence on a random samples of train/validate/test splits in the data.In order to compare aggregated results for different parameter values, we propagate the uncertainties associated with each metric as follows.In general, the harmonic mean can be written and the propagated uncertainty (neglecting correlations between x i ) The partial derivatives of x with respect to each x i are and the propagated uncertainty is then Using this result we can better understand different choices of γ max and k top shown in Figure 3.

A Additional CDF distance metrics
Here we show additional metrics for the CDF distances shown in Figure 4. We see that these demonstrate a similar pattern as the mean absolute distance shown in 4f, though Mean Squared Distance demonstrates stronger clustering.

B Subtask F1(val/test) and AUC(test) for varying γ max and k top
We show here individual results for F1(val), F1(test), and AUC(test).In the top 2 panels of each figure, we show the aggregated results of Figure 3, and in the lower 6 panels we show the subtask results from which these are computed (as the harmonic mean).

D Integrated gradients by domain
Here we show the most significant domain level integrated gradients.We leave these values signed since they are largely consistent across data splits and subtasks.The signs do not necessarily indicate with certainty whether a feature was used to make a positive (y = 1) or negative (y = 0) prediction, though one expects a reasonable correspondence. (

E Frequent and Removed domains
Here we show the most frequent domains which are retained or censored in each dataset.We specifically show the result at the most stringent threshold γ max = 0.4 where MLP still demonstrates strong performance.

Figure 1 .
Figure 1.Current landscape of automated detection on mainstream social platforms.Advances in automated detection will push influence operations towards less effective methods of coordination and more costly approaches to fake account creation.In turn, influence operations may be able to compensate by augmenting existing capabilities with emergent AI systems.
Frequency of co-URLs between each subset.Counts are normalized by the size of the leading and lagging subsets.Frequency near-simultaneous co-URLs (τ < 1 min.)between subsets.Counts are normalized by the size of the leading and lagging subsets.

( c )
Training set co-URL counts as a function of interarrival time τ .Each series corresponds to a diagonal element in Figure 4a and 4b above.
Test set co-URL counts as a function of interarrival time τ .Each series corresponds to a diagonal element in Figures 4a and 4b above.
Cumulative distribution function of co-URL counts for all campaigns.Mean absolute distance between CDFs.An increase in near-simultaneous link sharing was observed across operations even as automated detection improved.

Figure 4 .
Figure 4. Summary of co-URL statistics for the 6 IO subsets and baseline.9

Figure 5 .
Figure 5. Illustration of the proposed inductive learning framework for the detection of information operations: I.) Collect IO data spanning various operations and time periods, as well as a baseline that interacts with the IO to varying degrees; II.) Extract and censor raw content-based and graph-based indicators and encode signatures of coordination via graph learning; III.) Evaluate model performance on tasks requiring generalization and determine the most important indicators using feature attribution.H(•) is the Heaviside step function.

Table 1 .
Correspondence between previously effective technical indicators and generalized alternatives.

and China 2.1 Model and Indicator Evaluation
/Validate: Figure 2. Composition of training, validation, and test and baseline sets.‡targetaudiences includes the U.S., Latin America, Saudia Arabia, Israel, Indonesia.¶target audiences includes the U.S., China, and Russia.§ account origins include U.S., Russia,

, Laplacian Eigenmaps, Random Walk Positional Encoding, and Network Features to
include).

Table 2 .
Top: Aggregated intra-operation (A) and inter-operation (B) results; F1 and ROC-AUC scores are the harmonic mean of the individual subtasks shown in the bottom six tables; G.E. is the median value of subtasks.
Bottom: Individual subtask results for intra-operation (A1, A2, and A3) and cross-operation (B1, B2, and B3) classification.We note that the validation and test sets in the intra-operation and cross-operation subtasks are sampled identically, and hence can be compared.Each graph encoding (G.E.) denotes the absence ( * ) or presence ( †) of node2vec, Laplacian Eigenmaps, Random Walk Positional Encoding, and Network Features determined from a censored graph.Each model is trained with γ max = 0.54 and k top = 2500 per Figure3.‡In sample.(B):MLP

Table 3 .
Mean absolute integrated gradients (IG) of trained MLPs over features for validation, test, and baseline subsets.For all features, we report the sum of absolute values to avoid cancellation due to conflicting signs.Additionally, we report the IG of each of the five quantities comprising (NF).For IG values of individual subtasks and domains, see Tables4 and 5in the Appendix.

Subtask level integrated gradients
In Table3, we show the mean IG values over subtasks.Here we show explicit results for each of these subtasks.The subtask level results are largely consistent with the aggregated results, although attributions for individual features can vary by up to ±20% across subtasks.

Table 4 .
IG values of individual subtasks.

Table 5 .
IG values for individual domains in each trial and val, test, and baseline set.For long domain names, [*] denotes truncation.