Abstract
In evolving complex systems such as air traffic and social organisations, collective effects emerge from their many components’ dynamic interactions. While the dynamic interactions can be represented by temporal networks with nodes and links that change over time, they remain highly complex. It is therefore often necessary to use methods that extract the temporal networks’ largescale dynamic community structure. However, such methods are subject to overfitting or suffer from effects of arbitrary, a prioriimposed timescales, which should instead be extracted from data. Here we simultaneously address both problems and develop a principled datadriven method that determines relevant timescales and identifies patterns of dynamics that take place on networks, as well as shape the networks themselves. We base our method on an arbitraryorder Markov chain model with community structure, and develop a nonparametric Bayesian inference framework that identifies the simplest such model that can explain temporal interaction data.
Introduction
To reveal the mechanisms of complex systems, researchers identify largescale patterns in their networks of interactions with communitydetection methods^{1}. Traditionally, these methods describe only static network structures without taking into account the dynamics that take place on the networks, such as people travelling by air, or the dynamics of the networks themselves, such as new routes in air traffic networks. While the dynamics on and of networks contain crucial information about the systems they represent, only recently have researchers showed how to incorporate higherorder Markov chains to describe dynamics on networks^{2,3,4,5} and higherorder temporal structures to describe dynamics of networks^{6,7,8,9,10,11,12,13,14,15,16,17,18}. However, both avenues of research have encountered central limitations: first, methods that use higherorder memory to describe dynamics on networks rely on extrinsic methods to detect the appropriate memory order^{2, 19}. Second, methods that attempt to describe dynamics of networks adapt static descriptions by aggregating time windows into discrete layers^{13,14,15, 17, 20,21,22}, and ignore dynamics within the time windows. Thus, both methods for dynamics on and of networks require or impose ad hoc timescales that can obscure essential dynamic community structure.
Furthermore, when trying to determine the timescales solely from data, the curse of dimensionality strikes: the large number of degrees of freedom makes the higherorder descriptions prone to overfitting, when random fluctuations in highdimensional data are mistaken for actual structure^{23}. Without a principled method with effective model selection to counteract this increasing complexity, it becomes difficult to separate meaningful dynamic community structure from artefacts.
To overcome these model selection and arbitrary timescale problems, we present a general and principled datadriven method by simultaneously tackling dynamics on and of networks (Fig. 1). In contrast to approaches that incorporate temporal layers in methods for static network descriptions, we build our approach on describing the actual dynamics. We first formulate a generative model of discrete temporal processes based on arbitraryorder Markov chains with community structure^{24,25,26,27}. Since our model generates event sequences, it does not aggregate data in time windows^{13,14,15, 17}, and, other than the Markov model assumption, needs no a priori imposed timescales. This model can be used to describe dynamics taking place on network systems that take into account higherorder memory effects^{2, 3} of arbitrary order. We then use the model to describe temporal networks, where the event sequence represents the occurrence of edges in the network^{10}.
In both cases, we employ a nonparametric Bayesian inference framework that allows us to select, according to the statistical evidence available, the most parsimonious model among all its variations. Hence we can, for example, identify the most appropriate Markov order and the number of communities without overfitting. In particular, if the dynamics on or of a network are random, our method will not identify any spurious patterns from noise but conclude that the data lack structure. As we also show, the model can be used to predict future network dynamics and evolution from past observations. Moreover, we provide publicly available and scalable code with loglinear complexity in the number of nodes independent of the number of groups.
Results
Inference of markov chains
Here we consider general timeseries composed of a sequence of discrete observations {x _{ t }}, where x _{ t } is a single token from an alphabet of size N observed at discrete time t, and x _{ t−1} = (x _{ t−1}, …, x _{ t−n }) is the memory of the previous n tokens at time t (Fig. 1). An nthorder Markov chain with transition probabilities p(x _{ t }x _{ t−1}) generates such a sequence with probability
where a _{ x,x } is the number of transitions x → x in {x _{ t }}. Given a specific sequence {x _{ t }}, we want to infer the transitions probabilities p(xx). The simplest approach is to compute the maximumlikelihood estimate, that is
where a _{ x } = ∑_{ x } a _{ x,x }, which amounts simply to the frequency of observed transitions. Putting this back into the likelihood of eq. (1), we have
This can be expressed through the total number of observed transitions \(E = \mathop {\sum}\nolimits_{x,{\boldsymbol{x}}} {{a_{x,{\boldsymbol{x}}}}} \) and the conditional entropy \(H\left( {\left. X \right{\bf{X}}} \right) =  \mathop {\sum}\nolimits_{\boldsymbol{x}} {\hat p({\bf{x}})} \mathop {\sum}\nolimits_{\boldsymbol{x}} {\hat p\left( {x\left {\boldsymbol{x}} \right.} \right){\rm{ln}}\,\hat p\left( {x\left {\boldsymbol{x}} \right.} \right)} \) as \({\rm{ln}}\,P\left( {\left\{ {{x_t}} \right\}\left {\hat p} \right.} \right) =  EH\left( {X\left {{X}} \right.} \right)\). Hence, the maximisation of the likelihood in eq. (1) yields the transition probabilities that most compress the sequence. There is, however, an important caveat with this approach. It cannot be used when we are interested in determining the most appropriate Markov order n of the model, because the maximum likelihood in eq. (3) increases with n. In general, increasing number of memories at fixed number of transitions leads to decreased conditional entropy. Hence, for some large enough value of n there will be only one observed transition conditioned on every memory, yielding a zero conditional entropy and a maximum likelihood of 1. This would be an extreme case of overfitting, where by increasing the number of degrees of freedom of the model it is impossible to distinguish actual structure from stochastic fluctuations. Also, this approach does not yield true compression of the data, since it does not describe the increasing model complexity for larger values of n, and thus is crucially incomplete. To address this problem, we use a Bayesian formulation, and compute instead the complete evidence
which is the sum of all possible models weighted according to prior probabilities P(p) that encode our a priori assumptions. This approach gives the correct model order for data sampled from Markov chains as long as there are enough statistics that balances the structure present in the data with its statistical weight, as well as meaningful values when this is not the case^{28}.
Although this Bayesian approach satisfactorily addresses the overfitting problem, it misses opportunities of detecting largescale structures in data. As we show below, it is possible to extend this model in such a way as to make a direct connection to the problem of finding communities in networks, yielding a stronger explanatory power when modelling sequences, and serving as a basis for a model where the sequence itself represents a temporal network.
Markov chains with communities
Instead of directly inferring the transition probabilities of eq. (1), we propose an alternative formulation: We assume that both memories and tokens are distributed in disjoint groups (Fig. 2). That is, b _{ x } ∈ [1, 2, …, B _{N}] and b _{ x } ∈ [B _{N} + 1, B _{N} + 2, …, B _{N} + B _{M}] are the group memberships of the tokens and memories uniquely assigned in B _{N} and B _{M} groups, respectively, such that the transition probabilities can be parametrised as
Here θ _{ x } is the relative probability at which token x is selected among those that belong to the same group, and λ _{ rs } is the overall transition probability from memory group s = b _{ x } to token group r = b _{ x }. The parameter θ _{ x } plays an analogous role to degreecorrection in the SBM^{29}, and is together with the Bayesian description the main difference from the sparse Markov chains developed in refs. ^{26, 27}. In the case n = 1, for example, each token appears twice in the model, both as token and memory. An alternative and often useful approach for n = 1 is to consider a single unified partition for both tokens and memories, as shown in Fig. 2b and described in detail in the Methods section ‘The unified firstorder model’. In any case, the maximum likelihood parameter estimates are
where e _{ rs } is the number of observed transitions from group s to r, \({e_s} = \mathop {\sum}\nolimits_t {{e_{ts}}} \) is the total outgoing transitions from group s if s is a memory group, or the total incoming transition if it is a token group. The labels r and s are used indistinguishably to denote memory and token groups, since it is only their numerical value that determines their kind. Finally, k _{ x } is the total number of occurrences of token x. Putting this back in the likelihood, we have
This is almost the same as the maximum likelihood of the degreecorrected stochastic block model (DCSBM)^{29}, where a _{ x,x } plays the role of the adjacency matrix of a bipartite multigraph connecting tokens and memories. The only differences are constant terms that do not alter the position of the maximum with respect to the node partition. This implies that for undirected networks without higherorder memory, there is no difference between inferring the structure directly from its topology or from dynamical processes taking place on it, as we show in detail in the Methods section Equivalence between structure and dynamics.
As before, this maximum likelihood approach cannot be used if we do not know the order of the Markov chain, otherwise it will overfit. In fact, this problem is now aggravated by the larger number of model parameters. Therefore, we employ a Bayesian formulation and construct a generative process for the model parameters themselves. We do this by introducing prior probability densities for the parameters \({{\cal D}_r}\left( {\left\{ {{\theta _x}} \right\}\left \alpha \right.} \right)\) and \({{\cal D}_s}\left( {\left\{ {{\lambda _{rs}}} \right\}\left \beta \right.} \right)\) for tokens and memories, respectively, with hyperparameter sets α and β, and computing the integrated likelihood
where we used b as a shorthand for {b _{ x }} and {b _{ x }}. Now, instead of inferring the hyperparameters, we can make a noninformative choice for α and β that reflects our a priori lack of preference towards any particular model^{30}. Doing so in this case yields a likelihood (for details, see Methods section Bayesian Markov chains with communities),
where P({x _{ t }}b, {e _{ rs }}, {k _{ x }}) corresponds to the likelihood of the sequence {x _{ t }} conditioned on the transitions counts {e _{ rs }} and token frequencies {k _{ x }}, and the remaining terms are the prior probabilities on the discrete parameters {e _{ rs }} and {k _{ x }}. Since the likelihood above still is conditioned on the partitions {b _{ x }} and {b _{ x }}, as well as the memory group counts {e _{ s }}, we need to include prior probabilities on these as well to make the approach fully nonparametric. Doing so yields a joint likelihood for both the sequence and the model parameters,
It is now possible to understand why maximising this joint likelihood will prevent overfitting the data. If we take its negative logarithm, it can be written as
The quantity Σ is called the description length of the data^{31, 32}. It corresponds to the amount of information necessary to describe both the data and the model simultaneously, corresponding to the first and second terms in eq. (12), respectively. As the model becomes more complex—either by increasing the number of groups or the order of the Markov chain—this will decrease the first term as the data likelihood increases, but it will simultaneously increase the second term, as the model likelihood decreases. The second term then acts as a penalty to the model likelihood, forcing a balance between model complexity and quality of fit. Unlike approximative penalty approaches based solely on the number of free parameters such as BIC^{33} and AIC^{34}, which are not to valid for network models^{35}, the description length of the model is exact and fully captures its flexibility. Because of the complete character of the description length, minimising it indeed amounts to achieving true compression of data, differently from the parametric maximum likelihood approach mentioned earlier. Because the whole process is functionally equivalent to inferring the SBM for networks, we can use the same algorithms^{36} (for a details about the inference method, see Methods section ‘Bayesian Markov chains with communities’).
Before we continue, we point out that the selection of priors in eq. (9) needs to be done carefully to avoid underfitting the data. This happens when strong prior assumptions obscure structures in the data^{37}. We tackle this by using hierarchical priors, where the parameter themselves are modelled by parametric distributions, which in turn contain more parameters, and so on^{38, 39}. Besides alleviating the underfitting problem, this allows us to represent the data in multiple scales by a hierarchical partition of the token and memories. We describe this in more detail in the Methods section Bayesian Markov chains with communities.
This Markov chain model with communities succeeds in providing a better description for a variety of empirical sequences when compared with the common Markov chain parametrisation (see Table 1). Not only do we systematically observe a smaller description length, but we also find evidence for higherorder memory in all examples. We emphasise that we are protected against overfitting: If we randomly shuffle the order of the tokens in each data set, with dominating probability we infer a fully random model with n = 1 and B _{N} = B _{M} = 1, which is equivalent to an n = 0 memoryless model. We have verified that we infer this model for all analysed data sets. Accordingly, we are not susceptible to the spurious results of nonstatistical methods^{23}.
To illustrate the effects of community structure on the Markov dynamics, we use the US flight itineraries as an example. In this data set, the itineraries of 1,272,696 passengers were recorded, and we treat each airport stop as a token in a sequence (for more details, see Methods section Data sets). When we infer our model, the itinerary memories are grouped together if their destination probabilities are similar. As a result, it becomes possible, for example, to distinguish transit hubs from destination hubs^{2}. We use Atlanta and Las Vegas to illustrate: Many roundtrip routes transit through Atlanta from the origin to the final destination and return to it two legs later on the way back to the origin. On the other hand, Las Vegas often is the final destination of a roundtrip such that the stop two legs later represents a more diverse set of origins (Fig. 3). Resembling the results of the map equation for network flows with memory^{2}, this pattern is captured in our model by the larger number of memory groups that involve Las Vegas than those that involve Atlanta. Moreover, the division between transit and destinations propagates all the way to the upper hierarchical levels of the memory partition.
In addition to this itinerary memory clustering, the coclustering with airport tokens also divides the airports into hierarchical categories. For example, Atlanta is grouped with nearby Charlotte at the first hierarchy level, and with Detroit, Minneapolis, Dallas and Chicago at the third level. This extra information tells us that these airports serve as alternative destinations to itineraries that are similar to those that go through Atlanta. Likewise, Las Vegas is grouped together with alternative destinations Phoenix and Denver. This type of similarity between airports—which is not merely a reflection of the memory patterns—is not expressed with the assortative approach of the map equation, which solely clusters densely connected memories with long flow persistence times^{2}. A more direct comparison between our Bayesian inference framework and the map equation is not meaningful, since these two approaches represent the network divisions differently (for a detailed discussion, see Methods section ‘Comparison with the map equation for network flows with memory’). Indeed, it is the simultaneous division of memories and tokens that effectively reduce the overall complexity of the data, and provide better compression at higher memory order. Consequently, the communitybased Markov model can capture patterns of higherorder memory that conventional methods obscure.
Temporal networks
A general model for temporal networks treats the edge sequence as a time series^{6, 40, 41}. We can in principle use the present model without any modification by considering the observed edges as tokens in the Markov chain, that is, x _{ t } = (i, j)_{ t }, where i and j are the endpoints of the edge at time t (see Fig. 1b). However, this can be suboptimal if the networks are sparse, that is, if only a relatively small subset of all possible edges occur, and thus there are insufficient data to reliably fit the model. Therefore, we adapt the model above by including an additional generative layer between the Markov chain and the observed edges. We do so by partitioning the nodes of the network into groups, that is, c _{ i } ∈ [1, C] determines the membership of node i in one of C groups, such that each edge (i, j) is associated with a label (c _{ i }, c _{ j }). Then we define a Markov chain for the sequence of edge labels and sample the actual edges conditioned only on the labels. Since this reduces the number of possible tokens from O(N ^{2}) to O(C ^{2}), it has a more controllable number of parameters that can better match the sparsity of the data. We further assume that, given the node partitions, the edges themselves are sampled in a degreecorrected manner, conditioned on the edge labels,
where κ _{ i } is the probability of a node being selected inside a group, with \(\mathop {\sum}\nolimits_{i \in r} {{\kappa _i} = 1} \). The total likelihood conditioned on the label sequence becomes
Since we want to avoid overfitting the model, we once more use noninformative priors, but this time on {κ _{ i }}, and integrate over them,
Combining this result with eq. (9), we have the complete likelihood of the temporal network,
conditioned only on the partitions. As we show in detail in the Methods section Temporal networks, this model is a direct generalisation of the static DCSBM, with a likelihood composed of two separate static and dynamic terms. One recovers the static DCSBM exactly by choosing B _{N} = B _{M} = 1, making the state transitions memoryless.
Finally, to make the model nonparametric, we again include the same prior as before for the node partition c, in addition to token and memory partition b, such that the total nonparametric joint likelihood is maximised,
In this way, we again protect against overfitting, and we can infer not only the number of memory groups B _{N} and token groups B _{M}, as before, but also the number of groups in the temporal network itself, C. If, for example, the temporal network is completely random—that is, the edges are placed randomly both in the aggregated network as well as in time—we again infer B _{N} = B _{M} = C = 1 with the largest probability. We refer to the Methods section Temporal networks for a complete derivation of the final likelihood.
We employ this model in a variety of dynamic network data sets from different domains (for details, see Table 2 and Methods section Data sets). In all cases, we infer models with n > 0 that identify many groups for the tokens and memories, meaning that the model succeeds in capturing temporal structures. In most cases, models with n = 1 best describe the data, implying that there is not sufficient evidence for higherorder memory, with exception of the network of chess moves, which is best described by a model with n = 2. This result is different from the results for the comparably long nonnetwork sequences in Table 1, where we identified higherorder Markov chains. Again, this is because the alphabet size is much larger for temporal networks—corresponding to all possible edges that can be encountered. Hence, for the data sets in Table 2 the size of the alphabet is often comparable with the length of the sequence. In view of this, it is remarkable that the method can detect any structure at all. The intermediary layer where the Markov chain generates edge types instead of the edges directly is crucial. If we fit the original model without this modification, we indeed get much larger description lengths and we often fail to detect any Markov structure (not shown).
To illustrate how the model characterises the temporal structure of these systems, we focus on the proximity network of high school students, which corresponds to the voluntary tracking of 327 students for a period of 5 days^{42}. Whenever the distance between two students fell below a threshold, an edge between them was recorded at that time. In the bestfitting model for these data, the inferred groups for the aggregated network correspond exactly to the known division into 9 classes, except for the PC class, which was divided into two groups (Fig. 4). The groups show a clear assortative structure, where most connections occur within each class. The clustering of the edge labels in the second part of the model reveals the temporal dynamics. We observe that the edges connecting nodes of the same group cluster either in singlenode or small groups, with a high incidence of selfloops. This means that if an edge that connects two students of the same class appears in the sequence, the next edge is most likely also inside the same class, indicating that the students of the same class are clustered in space and time. The remaining edges between students of different classes are separated into two large groups. This division indicates that the different classes meet each other at different times. Indeed, the classes are located in different parts of the school building and they typically go to lunch separately^{42}. Accordingly, our method can uncover the associated dynamical pattern from the data alone.
Temporal prediction
Using generative models to extract patterns in data yields more than a mere description, since they generalise observations and predict future events. For our Bayesian approach, the models can even be used to predict tokens and memories not previously observed. This ability is in strong contrast to more heuristic methods that are only designed to find partitions in networks or time series, and cannot be used for prediction. Furthermore, the predictive performance of a model is often used on its own to evaluate it, and serves as an alternative approach to model selection: since an overly complicated model incorporates noise in its description, it yields less accurate predictions. Thus, maximising the predictive performance also amounts to a balance between quality of fit and model complexity, similarly to the minimum description length approach we have used so far. It is important, therefore, to determine if these two different criteria yield consistent results, which would serve as an additional verification of the overall approach.
We show this consistency by considering a scenario where a sequence is divided into two equalsized contiguous parts, \(\left\{ {{x_t}} \right\} = \left\{ {x_t^*} \right\} \cup \left\{ {x_t^\prime } \right\}\). That is, a training set \(\left\{ {x_t^*} \right\}\) and a validation set \(\left\{ {x_t^\prime } \right\}\). We then evaluate the model by fitting it to the training set and use it to predict the validation set. If we observe only the training set, the likelihood of the validation set is bounded below by \(P\left( {\left\{ {x_t^\prime } \right\}\left {\left\{ {x_t^*} \right\}} \right.,{b^*}} \right) \ge {\rm{exp}}(  \Delta \Sigma )\), where \({b^*} = {\rm{argma}}{{\rm{x}}_b}\,P\left( {b\left {\left\{ {x_t^*} \right\}} \right.} \right)\) is the best partition given the training set and ΔΣ is the description length difference between the training set and the entire data (for a proof, see Methods section Predictive heldout likelihood). This lower bound will become tight when both the validation and training sets become large, and hence can be used as an asymptotic approximation of the predictive likelihood. Table 2 shows empirical values for the same data sets as considered before, where n = 0 corresponds to using only the static DCSBM to predict the edges, ignoring any time structure. The temporal network model provides better prediction in all cases, and the best Markov order always coincides with the one that yields the minimum description length, thus confirming a full agreement between both criteria in these cases.
Discussion
We presented a dynamical variation of the degreecorrected stochastic block model that can capture long pathways or largescale structures in sequences and temporal networks. The model does not require the optimal Markov order or number of groups as inputs, but infers them from data because the underlying arbitraryorder Markov chain model is nonparametric. Its nonparametric nature also evades a priori imposed timescales. We showed that the model successfully finds meaningful largescale temporal structures in realworld systems and that it predicts their temporal evolution. Moreover, in the Methods section we extend the model to situations where the dynamics take place in continuous time or is nonstationary. In contrast to approaches that force networkformation dynamics into discrete time windows, and require a priori knowledge about the appropriate amount of dynamical memory, our approach provides a principled and versatile alternative.
Methods
Bayesian Markov chains with communities
As described in the main text, a Bayesian formulation of the Markov model consists in specifying prior probabilities for the model parameters, and integrating over them. In doing so, we convert the problem from one of parametric inference where the model parameters need to be specified before inference, to a nonparametric one where no parameters need to be specified before inference. In this way, the approach possesses intrinsic regularisation, where the order of the model can be inferred from data alone, without overfitting^{30, 43}.
To accomplish this, we rewrite the model likelihood, using eqs. (1) and (5), as
and observe the normalisation constraints \(\mathop {\sum}\nolimits_{x \in r} {{\theta _x} = 1} \), and \(\mathop {\sum}\nolimits_r {{\lambda _{rs}} = 1} \). Since this is just a product of multinomials, we can choose conjugate Dirichlet priors probability densities \({{\cal D}_r}\left( {\left\{ {{\theta _x}} \right\}\left {\left\{ {{\alpha _x}} \right\}} \right.} \right)\) and \({{\cal D}_s}\left( {\left. {\left\{ {{\lambda _{rs}}} \right\}} \right\left\{ {{\beta _{rs}}} \right\}} \right)\), which allows us to exactly compute the integrated likelihood,
where \({A_r} = \mathop {\sum}\nolimits_{x \in r} {{\alpha _x}} \) and \({B_s} = \mathop {\sum}\nolimits_r {{\beta _{rs}}} \). We recover the Bayesian version of the common Markov chain formulation (see ref. ^{28}) if we put each memory and token in their own groups. This remains a parametric distribution, since we need to specify the hyperparameters. However, in the absence of prior information it is more appropriate to make a noninformative choice that encodes our a priori lack of knowledge or preference towards any particular model, which amounts to choosing α _{ x } = β _{ rs } = 1, making the prior distributions flat. If we substitute these values in eq. (19), and rearrange the terms, we can show that it can be written as the following combination of conditional likelihoods,
where
with \(\left( {\left( {\begin{array}{*{20}{c}} m \\ n \end{array}} \right)} \right) = \left( {\begin{array}{*{20}{c}} {m + n  1} \\ n \end{array}} \right)\) being the multiset coefficient, that counts the number of mcombinations with repetitions from a set of size n. The expression above has the following combinatorial interpretation: P({x _{ t }}b, {e _{ rs }}, {k _{ x }}) corresponds to the likelihood of a microcanonical model^{39} where a random sequence {x _{ t }} is produced with exactly e _{ rs } total transitions between groups r and s, and with each token x occurring exactly k _{ x } times. In order to see this, consider a chain where there are only e _{ rs } transitions in total between token group r and memory group s, and each token x occurs exactly k _{ x } times. For the first transition in the chain, from a memory x _{0} in group s to a token x _{1} in group r, we have the probability
Now, for the second transition from memory x _{1} in group t to a token x _{2} in group u, we have the probability
Proceeding recursively, the final likelihood for the entire chain is
which is identical to eq. (21).
The remaining terms in eqs. (22) and (23) are the prior probabilities on the discrete parameters {k _{ x }} and {e _{ rs }}, respectively, which are uniform distributions of the type 1/Ω, where Ω is the total number of possibilities given the imposed constraints. We refer to ref. ^{39} for a more detailed discussion on those priors.
Since the integrated likelihood above gives P({x _{ t }}b, {e _{ s }}), we still need to include priors for the node partitions {b _{ x }} and {b _{ x }}, as well as memory group counts, {e _{ s }}, to make the above model fully nonparametric. This is exactly the same situation encountered with the SBM^{37,38,39}. Following refs. ^{38, 39}, we use a nonparametric twolevel Bayesian hierarchy for the partitions, P({b _{ i }}) = P({b _{ i }}{n _{ r }})P({n _{ r }}), with uniform distributions
where n _{ r } is the number of nodes in group r, \(M = \mathop {\sum}\nolimits_r {{n_r}} \), which we use for both {b _{ x }} and {b _{ x }}, that is, P(b) = P({b _{ x }})P({b _{ x }}). Analogously, for {e _{ s }} we can use a uniform distribution
The above priors make the model fully nonparametric with a joint and marginal probability P({x _{ t }}, b) = P({x _{ t }}, b, {e _{ s }}) = P({x _{ t }}b, {e _{ s }})P(b)P({e _{ s }}). This approach successfully finds the most appropriate number of groups according to statistical evidence, without overfitting^{32, 37, 38, 44}. This nonparametric method can also detect the most appropriate order of the Markov chain, again without overfitting^{28}. However, in some ways it is still suboptimal. The use of conjugate Dirichlet priors above was primarily for mathematical convenience, not because they closely represent the actual mechanisms believed to generate the data. Although the noninformative choice of the Dirichlet distribution (which yields flat priors for {e _{ rs }} and {e _{ s }}) can be well justified by maximum entropy arguments (see ref. ^{30}), and are unbiased, it can in fact be shown that it can lead to underfitting of the data, where the maximum number of detectable groups scales suboptimally as \(\sqrt N \) ^{37}. As shown in ref. ^{38}, this limitation can be overcome by departing from the model with Dirichlet priors, and replacing directly the priors P({e _{ rs }}{e _{ s }}) and P({e _{ s }}) of the microcanonical model by a single prior P({e _{ rs }}), and noticing that {e _{ rs }} corresponds to the adjacency matrix of bipartite multigraph with E edges and B _{N} + B _{M} nodes. With this insight, we can write P({e _{ rs }}) as a Bayesian hierarchy of nested SBMs, which replaces the resolution limit above by N/ln N, and provides a multilevel description of the data, while remaining unbiased. Furthermore, the uniform prior in eq. (8) for the token frequencies P({k _{ x }}{e _{ rs }}, (b) intrinsically favours concentrated distributions of k _{ x } values. This distribution is often skewed. We therefore replace it by a twolevel Bayesian hierarchy \(P\left( {\left. {\left\{ {{k_x}} \right\}} \right\left\{ {{e_{rs}}} \right\},b} \right) = \mathop {\prod}\nolimits_r {P\left( {\left. {\left\{ {{k_x}} \right\}} \right\left\{ {n_k^r} \right\}} \right)P\left( {\left. {\left\{ {n_k^r} \right\}} \right{e_r}} \right)} \), with
and \(P\left( {\left. {\left\{ {n_k^r} \right\}} \right{e_r}} \right) = q{\left( {{e_r},{n_r}} \right)^{  1}}\), where q(m, n) is the number of restricted partitions of integer m into at most n parts (see ref. 39 for details).
As mentioned in the main text, in order to fit the model above we need to find the partitions {b _{ x }} and {b _{ x }} that maximise P({x _{ t }}, b), or fully equivalently, minimise the description length \(\Sigma =  {\rm{ln}}\,P\left( {\left\{ {{x_t}} \right\},b} \right)\) ^{31}. Since this is functionally equivalent to inferring the DCSBM in networks, we can use the same algorithms. In this work we employed the fast multilevel MCMC method of ref. ^{36}, which has loglinear complexity \(O\left( {N\,{\rm{lo}}{{\rm{g}}^2}N} \right)\), where N is the number of nodes (in our case, memories and tokens), independent of the number of groups.
The unified firstorder model
The model defined in the main text is based on a coclustering of memory and tokens. In the n = 1 case, each memory corresponds to a single token. In this situation, we consider a slight variation of the model where we force the number of groups of each type to be the same, that is, B _{N} = B _{M} = B, and both partitions to be identical. Instead of clustering the original bipartite graph, this is analogous to clustering its projection into a directed transition graph with each node representing a specific memory and token simultaneously. When considering this model, the likelihoods computed in the main text and above remain exactly the same, with the only difference that we implicitly force both memory and token partitions to be identical, and omit the partition likelihood of eq. (27) for one of them. We find that for many data sets this variation provides a slightly better description than the coclustering version, although there are also exceptions to this.
We used this variation of the model in Fig. 4 because it yielded a smaller description length for that dataset, and simplified the visualisation and interpretation of the results in that particular case.
Temporal networks
Here we show in more detail how the likelihood for the temporal network model is obtained. As we discuss in the Results section Temporal networks, the total likelihood of the network conditioned on the label sequence is
where d _{ i } is the degree of node i, and m _{ rs } is the total number of edges between groups r and s. Maximum likelihood estimation gives \({\hat \kappa _i} = {d_i}{\rm{/}}{e_{{c_i}}}\). But since we want to avoid overfitting the model, we once more use noninformative priors, this time on {κ _{ i }}, integrate over them, henceforth omitting the trivial Kronecker delta term above and obtain
with \(P\left( {\left\{ {{d_i}} \right\}} \right) = \mathop {\prod}\nolimits_r {{{\left( {\left( {\begin{array}{*{20}{c}} {{n_r}} \\ {{e_r}} \end{array}} \right)} \right)}^{  1}}} \). Combining this with eq. (9) as P({(i, j)_{ t }}c, b) = P({(i, j)_{ t }}{(r,s)_{ t }}, c)P({(r, s)_{ t }}b), we have the complete likelihood of the temporal network
This likelihood can be rewritten in such a way that makes clear that it is composed of one purely static and one purely dynamic part,
The first term of eq. (34) is precisely the nonparametric likelihood of the static DCSBM that generates the aggregated graph with adjacency matrix A _{ ij } = k _{ x } = (i,j) given the node partition {c _{ i }}, which itself is given by
if Stirling’s approximation is used. The second term in eq. (34) is the likelihood of the Markov chain of edge labels given by eq. (9) (with {x _{ t }} = {(r,s)_{ t }}, and {k _{ x }} = {m _{ rs }}). This model, therefore, is a direct generalisation of the static DCSBM, with a likelihood composed of two separate static and dynamic terms. One recovers the static DCSBM exactly by choosing B _{N} = B _{M} = 1—making the state transitions memoryless—so that the second term in eq. (34) above contributes only with a trivial constant 1/E! to the overall likelihood. Equivalently, we can view the DCSBM as a special case with n = 0 of this temporal network model.
Predictive heldout likelihood
Given a sequence divided in two contiguous parts, \(\left\{ {{x_t}} \right\} = \left\{ {x_t^*} \right\} \cup \left\{ {x_t^\prime } \right\}\), that is, a training set \(\{ x:t^*\} \) and a validation set \(\left\{ {x_t^\prime } \right\}\), and if we observe only the training set, the predictive likelihood of the validation set is
where \({b^*} = {\rm{argma}}{{\rm{x}}_b}\,P\left( {b\left {\left\{ {x_t^*} \right\}} \right.} \right)\) is the best partition given the training set. Moreover, we have
where b′ corresponds to the partition of the newly observed memories (or even tokens) in \(\left\{ {x_t^\prime } \right\}\). Generally we have P(b′b ^{*}) = P(b′, b ^{*})/P(b ^{*}), so that
where \(\hat b' = {\rm{argma}}{{\rm{x}}_{b'}}P\left( {\left\{ {x_t^\prime } \right\} \cup \left. {\left\{ {x_t^*} \right\}} \right{b^*},b'} \right)P\left( {{b^*},b'} \right)\) and ΔΣ is the difference in the description length between the training set and the entire data. Hence, computing the minimum description length of the remaining data by maximising the posterior likelihood relative to the partition of the previously unobserved memories or tokens, yields a lower bound on the predictive likelihood. This lower bound will become tight when both the validation and training sets become large, because then the posterior distributions concentrate around the maximum, and hence can be used as an asymptotic approximation of the predictive likelihood.
Continuous time
So far, we have considered sequences and temporal networks that evolve discretely in time. Although this is the appropriate description for many types of data, such as text, flight itineraries and chess moves, in many other cases events happen instead in real time. In this case, the time series can be represented—without any loss of generality—by an embedded sequence of tokens {x _{ t }} placed in discrete time, together with an additional sequence of waiting times {Δ_{ t }}, where Δ_{ t } ≥ 0 is the real time difference between tokens x _{ t } and x _{ t−1}. Employing a continuoustime Markov chain description, the data likelihood can be written as
with P({x _{ t }}p) given by eq. (1), and
where
is a maximumentropy distribution governing the waiting times, according to the frequency λ. Substituting this in eq. (40), we have
where \({\Delta _{\bf{x}}} = \mathop {\sum}\nolimits_t {{\Delta _t}{\delta _{{{\boldsymbol{x}}_t},{\boldsymbol{x}}}}} \). To compute the nonparametric Bayesian evidence, we need a conjugate prior for the frequencies λ _{ x },
where α and β are hyperparameters, interpreted, respectively, as the number and sum of prior observations. A fully noninformative choice would entail α → 0 and β → 0, which would yield the socalled Jeffreys prior^{46}, P(λ) ∝ 1/λ. Unfortunately, this prior is improper because it is not a normalised distribution. In order to avoid this, we use instead α = 1 and \(\beta = \mathop {\sum}\nolimits_{\boldsymbol{x}} {{\lambda _{\boldsymbol{x}}}{\rm{/}}M} \), taking into account the global average. While this is not the only possible choice, the results should not be sensitive to this prior since the data will eventually override any reasonable assumption we make. Using this prior, we obtain the Bayesian evidence for the waiting times as
Hence, if we employ the Bayesian parametrisation with communities for the discrete embedded model as we did previously, we have
with P({x _{ t }}, b) given by eq. (10).
Since the partition of memories and tokens only influences the first term of eq. (46), corresponding to the embedded discretetime Markov chain, P({x _{ t }}, b), the outcome of the inference for any particular Markov order will not take into account the distribution of waiting times—although the preferred Markov order might be influenced by it. We can change this by modifying the model above, assuming that the waiting times are conditioned on the group membership of the memories,
where η _{ r } is a frequency associated with memory group r. The Bayesian evidence is computed in the same manner, integrating over η _{ r } with the noninformative prior of eq. (43), yielding
where \({\Delta _r} = \mathop {\sum}\nolimits_t {{\Delta _t}{\delta _{{b_{{{\boldsymbol{x}}_t}}},r}}} \). Since this assumes that the waiting times will be sampled from the same distribution inside each group, the inference procedure will take the waiting time into account, and will place memories with significantly different delays into different groups.
As an example of the use of this model variation, we consider a piano reduction of Beethoven’s fifth symphony (extracted in MIDI format from the Mutopia project at http://www.mutopiaproject.org), represented as a sequence of E = 4223 notes of an alphabet of size N = 63. We consider both model variants, where the timings between notes are discarded, and where they are included. If individual notes are played simultaneously as part of a chord, we order them lexicographically and separate them by Δt = 10^{−6} s. The results of the inference can be seen in Table 3. The discretetime model favours an n = 1 Markov chain, whereas the continuoustime model favours n = 2. This is an interesting result that shows that the timings alone can influence the most appropriate Markov order. We can see in more detail why by inspecting the typical waiting times conditioned on the memory groups, as shown in Fig. 5. For the discretetime model, the actual continuous waiting times (which are not used during inference) are only weakly correlated with the memory groups. On the other hand, for the continuoustime model we find that the memories are divided in such a way that they are strongly correlated with the waiting times: there is a group of memories for which the ensuing waiting times are always Δt = 10^{−6}, corresponding to node combinations that are always associated with chords. The remaining memories are divided into further groups that display at least two distinct timescales, that is, short and long pauses between notes. These statistically significant patterns are only visible for the higher order n = 2 model.
In the above model the waiting times are distributed according to the exponential distribution of eq. (41), which has a typical timescale given by 1/λ. However, one often encounters processes where the dynamics are bursty, that is, the waiting times between events lack any characteristic scale, and are thus distributed according to a powerlaw
for Δ > Δ_{ m }, otherwise P(Δβ) = 0. One could in principle repeat the above calculations with the above distribution to obtain the inference procedure for this alternative model. However, this is in fact not necessary, since by making the transformation of variables
we obtain for eq. (49)
which is the same exponential distribution of eq. (41). Hence, we need only to perform the transformation of eq. (50) for the waiting times prior to inference, to use the bursty model variant, while maintaining the exact same algorithm.
Nonstationarity and hidden contexts
An underlying assumption of the Markov model proposed is that the same transition probabilities are used for the whole duration of the sequence, that is, the Markov chain is stationary. Generalisations of the model can be considered where these probabilities change over time. Perhaps the simplest generalisation is to assume that the dynamics is divided into T discrete epochs, such that one replaces tokens x _{ t } by a pair (x, τ)_{ t }, where τ ∈ [1, T] represents the epoch where token x was observed. In fact, τ does not need to be associated with a temporal variable—it could be any arbitrary covariate that describes additional aspects of the data. By incorporating this type of annotation into the tokens, one can use a stationary Markov chain describing the augmented tokens that in fact corresponds to a nonstationary one if one omits the variable τ from the token descriptors—effectively allowing for arbitrary extensions of the model by simply incorporating appropriate covariates, and without requiring any modification to the inference algorithm.
Another consequence of this extension is that the same token x can belong to different groups if it is associated with two or more different covariates, (x, τ _{1}) and (x, τ _{2}). Therefore, this inherently models a situation where the group membership of tokens and memory vary in time.
As an illustration of this application of the model, we consider two literary texts: an English translation of ‘War and peace’, by Leo Tolstoy, and the French original of ‘À la recherche du temps perdu’, by Marcel Proust. First, we concatenate both novels together, treating it as a single text. If we fit our Markov model to it, we obtain the n = 3 model shown in Fig. 6a. In that figure, we have highlighted tokens and memories that involve letters that are exclusive to the French language, and thus most of them belong to the second novel. We observe that the model essentially finds a mixture between English and French. If, however, we indicate in each token to which novel it belongs, for example (x, wp)_{ t } and (x, temps)_{ t }, we obtain the model of Fig. 6b. In this case, the model is forced to separate between the two novels, and one indeed learns the French patterns differently from English. Since this nonstationary model possesses a larger number of memory and tokens, one would expect a larger description length. However, in this cases it has a smaller description length than the mixed alternative, indicating indeed that both patterns are sufficiently different to warrant a separate description. Therefore, this approach is capable of uncovering change points^{47}, where the rules governing the dynamics change significantly from one period to another.
The above extension can also be used to uncover other types of hidden contexts. For example, in a temporal network of student proximity, we know that pairs of individuals that are far away are unlikely to be conditionally dependent on each other. If this spatial information is not available in the data, it may be inferred in same way it was done for language above. If the information is available, it can be annotated on the transitions, yielding a multilayer version of the model, similar to the layered SBM^{15}.
Equivalence between structure and dynamics
The likelihood of eq. (4) in the main text is almost the same as the DCSBM^{29}. The only exceptions are trivial additive and multiplicative constants, as well as the fact that the degrees of the memories do not appear in it. These differences, however, do not alter the position of the maximum with the respect to the node partition. This allows us to establish an equivalence between inferring the community structure of networks and modelling the dynamics taking place on it. Namely, for a random walk on a connected undirected graph, a transition i → j is observed with probability A _{ ij } p _{ i }(t)/k _{ i }, with p _{ i }(t) being the occupation probability of node i at time t. Thus, after equilibration with p _{ i }(∞) = k _{ i }/2E, the probability of observing any edge (i,j) is a constant: p _{ i }(∞)/k _{ i } + p _{ j }(∞)/k _{ j } = 1/E. Hence, the expected edge counts e _{ rs } between two groups in the Markov chain will be proportional to the actual edge counts in the underlying graph given the same node partition. This means that the likelihood of eq. (4) in the main text (for the n = 1 projected model described above) and of the DCSBM will differ only in trivial multiplicative and additive constants, such that the node partition that maximises them will be identical. This is similar to the equivalence between network modularity and random walks^{48}, but here the equivalence is stronger and we are not constrained to purely assortative modules. However, this equivalence breaks down for directed graphs, higherorder memory with n > 1 and when model selection chooses the number of groups.
Comparison with the map equation for network flows with memory
Both the communitybased Markov model introduced here and the map equation for network flows with memory^{2} identify communities in higherorder Markov chains based on maximum compression. However, the two approaches differ from each other in some central aspects. The approach presented here is based on the Bayesian formulation of a generative model, whereas the map equation finds a minimal modular entropy encoding of the observed dynamics projected on a node partition. Thus, both approaches seek compression, but of different aspects of the data.
The map equation operates on the internal and external transitions within and between possibly nested groups of memory states and describes the transitions between physical nodes [x _{ t } is the physical node or token in memory states of the form x = (x _{ t }, x _{ t−1}, x _{ t−2}, …)]. The description length of these transitions is minimised for the optimal division of the network into communities. By construction, this approach identifies assortative modules of memory states with long flow persistence times. Moreover, for inferring the most appropriate Markov order, this dynamics approach requires supervised approaches to model selection that uses random subsets of the data such as bootstrapping or cross validation^{49}.
On the other hand, the model presented here yields a nonparametric loglikelihood for the entire sequence as well as the model parameters, with its negative value corresponding to a description length for the entire data, not only its projection into groups. Minimising this description length yields the optimal coclustering of memories and tokens, and hence assumes no inherent assortativity. Therefore it can be used also when the underlying Markov chain is disassortative. Moreover, the description length can also be used for unsupervised model selection, where the Markov order and number of groups are determined from the entire data, obviating the need for bootstrapping or cross validation. Furthermore, the present approach can be used to generate new data and make predictions based on past observations.
These distinctions mean that the two different approaches can give different results and that the problem at hand should decide which method to use.
Data sets
Below we give a description of the data sets used in this work.
US flight itineraries: This data set corresponds to a sample of flight itineraries in the US during 2011 collected by Bureau of Transportation Statistics of the United States Department of Transportation (http://www.transtats.bts.gov/). The data set contains 1,272,696 itineraries of varied lengths (airport stops). We aggregate all itineraries into a single sequence by concatenating the individual itineraries with a special separator token that marks the end of a single itinerary. There are 464 airports in the data set, and hence we have an alphabet of N = 465 tokens, and a single sequence with a total length of 83,653,994 tokens.
War and peace: This data set corresponds to the entire text of the english translation of the novel War and Peace by Leo Tolstoy, made available by the Project Gutenberg (extracted verbatim from https://www.gutenberg.org/cache/epub/2600/pg2600.txt). This corresponds to a sequence with an alphabet of size N = 84 (including letters, space, punctuation and special symbols) and a total length of 3,226,652 tokens.
Taxi movements: This data set contains GPS logs from 25,000 taxi pickups in San Francisco, collected by the company Uber (retrieved from http://www.infochimps.com/datasets/uberanonymizedgpslogs, also available at https://github.com/dima42/ubergpsanalysis). The geographical locations were discretised into 416 hexagonal cells (see ref. ^{2} for details), and the taxi rides were concatenated together in a single sequence with a special separator token indicating the termination of a ride. In total, the sequence has an alphabet of N = 417 and a length of 819,172 tokens.
RockYou password list: This data set corresponds to a widely distributed list of 32,603,388 passwords from the RockYou video game company (retrieved from http://downloads.skullsecurity.org/passwords/rockyouwithcount.txt.bz2). The passwords were concatenated in a single sequence, with a special separator token between passwords. This yields a sequence with an alphabet of size N = 215 (letters, numbers and symbols) and a total length of 289,836,299 tokens.
High school proximity: This data set corresponds to a temporal proximity measurement of students in a french high school^{42}. A total of N = 327 students were voluntarily tracked for a period of 5 days, generating E = 5818 proximity events between pairs of students.
Enron email: This data set corresponds to timestamped collection of E = 1,148,072 emails between directed pairs of N = 87,273, senders and recipients of the former Enron corporation, disclosed as part of a fraud investigation^{50}.
Internet AS: This data set contains connections between autonomous systems (AS) collected by the CAIDA project (retrieved from http://www.caida.org). It corresponds to a timestamped sequence of E = 500,106 directed connections between AS pairs, with a total of N = 53,387 recorded AS nodes. The timestamps correspond to the first time the connection was seen.
APS citations: This dataset contains E = 4,262,443 timestamped citations between N = 425,760 scientific articles published by the American Physical Society for a period of over 100 years (retrieved from http://journals.aps.org/datasets).
prosper.com loans: This data set corresponds to E = 3,394,979 timestamped directed loan relationships between N = 89,269 users of the prosper.com website, which provides a peertopeer lending system (retrieved from http://konect.unikoblenz.de/networks/prosperloans).
Chess moves: This data set contains 38,365 online chess games collected over the month of July 2015 (retrieved from http://ficsgames.org/download.html). The games were converted into a bipartite temporal network where each piece and position correspond to different nodes, and a movement in the game corresponds to a timestamped edge of the type piece → position. The resulting temporal network consists of N = 76 nodes and E = 3,130,166 edges.
Hospital contacts: This data set corresponds to a temporal proximity measurement of patients and health care workers in the geriatric unit of an university hospital^{51}. A total of N = 75 individuals were voluntarily tracked for a period of 4 days, generating E = 32,424 proximity events between pairs of individuals.
Infectious sociopatterns: This data set corresponds to a temporal proximity measurement of attendants at a museum exhibition^{52}. A total of N = 10,972 participants were voluntarily tracked for a period of 3 months, generating E = 415,912 proximity events between pairs of individuals.
Reality mining: This data set corresponds to a temporal proximity measurement of university students and faculty^{53}. A total of N = 96 people were voluntarily tracked for a period of an entire academic year, generating E = 1,086,404 proximity events between pairs of individuals.
Beethoven’s fifth symphony: A piano reduction of Beethoven’s fifth symphony, extracted in MIDI format from the Mutopia project at http://www.mutopiaproject.org, represented as a sequence of E = 4223 notes of an alphabet of size N = 63.
Code availability
A free C++ implementation of the inference algorithm is available as part of the graphtool Python library^{45}, available at http://graphtool.skewed.de.
Data availability
The data sets analysed during the current study are available from their sources listed in the ‘Data sets’ section above, of from the corresponding author on reasonable request.
References
Fortunato, S. Community detection in graphs. Phys. Rep. 486, 75–174 (2010).
Rosvall, M., Esquivel, A. V., Lancichinetti, A., West, J. D. & Lambiotte, R. Memory in network flows and its effects on spreading dynamics and community detection. Nat. Commun. 5, 4630 (2014).
Lambiotte, R., Salnikov, V. & Rosvall, M. Effect of memory on the dynamics of random walks on networks. J. Complex Netw. 3, 177–188 (2015).
De Domenico, M., Lancichinetti, A., Arenas, A. & Rosvall, M. Identifying modular flows on multilayer networks reveals highly overlapping organization in interconnected systems. Phys. Rev. 5, 011027 (2015).
Scholtes, I. When is a network a network? Multiorder graphical model selection in pathways and temporal networks. Preprint at http://arxiv.org/abs/1702.05499 (2017).
Holme, P. Modern temporal network theory: a colloquium. Eur. Phys. J. B. 88, 234 (2015).
Mucha, P. J., Richardson, T., Macon, K., Porter, M. A. & Onnela, J.P. Community structure in timedependent, multiscale, and multiplex networks. Science 328, 876–878 (2010).
Rosvall, M. & Bergstrom, C. T. Mapping change in large networks. PLoS ONE 5, e8694 (2010).
Bassett, D. S. et al. Robust detection of dynamic community structure in networks. Chaos 23, 013142 (2013).
Pfitzner, R., Scholtes, I., Garas, A., Tessone, C. J. & Schweitzer, F. Betweenness preference: quantifying correlations in the topological dynamics of temporal networks. Phys. Rev. Lett. 110, 198701 (2013).
Bazzi, M. et al. Community detection in temporal multilayer networks, with an application to correlation networks. Multiscale Model. Simul. 14, 1–41 (2016).
Sarzynska, M., Leicht, E. A., Chowell, G. & Porter, M. A. Null models for community detection in spatially embedded, temporal networks. J. Complex Netw 4, 363–406 (2016).
Gauvin, L., Panisson, A. & Cattuto, C. Detecting the community structure and activity patterns of temporal networks: a nonnegative tensor factorization approach. PLoS ONE 9, e86028 (2014).
Xu, K. S. & Iii, A. O. H. in Social Computing, BehavioralCultural Modeling and Prediction (eds Greenberg, A. M., Kennedy, W. G. & Bos, N. D.) 201–210 (Springer, 2013).
Peixoto, T. P. Inferring the mesoscale structure of layered, edgevalued, and timevarying networks. Phys. Rev. E 92, 042807 (2015).
MacMahon, M. & Garlaschelli, D. Community detection for correlation matrices. Phys. Rev. 5, 021006 (2015).
Ghasemian, A., Zhang, P., Clauset, A., Moore, C. & Peel, L. Detectability thresholds and optimal algorithms for community structure in dynamic networks. Phys. Rev. 6, 031005 (2016).
Zhang, X., Moore, C. & Newman, M. E. J. Random graph models for dynamic networks. Preprint at http://arxiv.org/abs/1607.07570 (2016).
Xu, J., Wickramarathne, T. L. & Chawla, N. V. Representing higherorder dependencies in networks. Sci. Adv. 2, e1600028 (2016).
Kivelä, M. et al. Multilayer networks. J. Complex Netw. 2, 203–271 (2014).
Xu, K. & Hero, A. Dynamic stochastic blockmodels for timeevolving social networks. IEEE J. Sel. Top. Signal Process. 8, 552–562 (2014).
Xu, K. S. Stochastic block transition models for dynamic networks. In Proc. 18th International Conference on Artificial Intelligence and Statistics (Eds Lebanon, G. & Vishwanathan, S. V. N.) 1079–1087 (2015).
Guimerà, R., SalesPardo, M. & Amaral, L. A. N. Modularity from fluctuations in random graphs and complex networks. Phys. Rev. E 70, 025101 (2004).
Baum, L. E., Petrie, T., Soules, G. & Weiss, N. A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statist 41, 164–171 (1970).
Rabiner, L. & Juang, B. An introduction to hidden Markov models. IEEE ASSP Mag. 3, 4–16 (1986).
Jääskinen, V., Xiong, J., Corander, J. & Koski, T. Sparse Markov chains for sequence data. Scand. J. Statist. 41, (639–655 (2014).
Xiong, J., Jääskinen, V. & Corander, J. Recursive learning for sparse Markov models. Bayesian. Anal. 11, 247–263 (2016).
Strelioff, C. C., Crutchfield, J. P. & Hübler, A. W. Inferring Markov chains: Bayesian estimation, model comparison, entropy rate, and outofclass modeling. Phys. Rev. E 76, 011106 (2007).
Karrer, B. & Newman, M. E. J. Stochastic blockmodels and community structure in networks. Phys. Rev. E 83, 016107 (2011).
Jaynes, E. T. Probability Theory: The Logic of Science (Cambridge University Press, 2003).
Grünwald, P. D. The Minimum Description Length Principle (The MIT Press, 2007).
Rosvall, M. & Bergstrom, C. T. An informationtheoretic framework for resolving community structure in complex networks. Proc. Natl Acad. Sci. USA 104, 7327–7331 (2007).
Schwarz, G. Estimating the dimension of a model. Ann. Statist. 6, 461–464 (1978).
Akaike, H. A new look at the statistical model identification. IEEE Trans. Autom. Control 19, 716–723 (1974).
Yan, X. et al. Model selection for degreecorrected block models. J. Stat. Mech. 2014, P05007 (2014).
Peixoto, T. P. Efficient Monte Carlo and greedy heuristic for the inference of stochastic block models. Phys. Rev. E 89, 012804 (2014).
Peixoto, T. P. Parsimonious module inference in large networks. Phys. Rev. Lett. 110, 148701 (2013).
Peixoto, T. P. Hierarchical block structures and highresolution model selection in large networks. Phys. Rev. 4, 011047 (2014).
Peixoto, T. P. Nonparametric Bayesian inference of the microcanonical stochastic block model. Phys. Rev. E 95, 012317 (2017).
Holme, P. & Saramäki, J. Temporal networks. Phys. Rep. 519, 97–125 (2012).
Scholtes, I. et al. Causalitydriven slowdown and speedup of diffusion in nonMarkovian temporal networks. Nat. Commun. 5, 5024 (2014).
Mastrandrea, R., Fournet, J. & Barrat, A. Contact patterns in a high school: a comparison between data collected using wearable sensors, contact diaries and friendship surveys. PLoS ONE 10, e0136497 (2015).
MacKay, D. J. C. Information Theory, Inference and Learning Algorithms (Cambridge University Press, 2003).
Peixoto, T. P. Model selection and hypothesis testing for largescale network models with overlapping groups. Phys. Rev. 5, 011033 (2015).
Peixoto, T. P. The graphtool python library. Available at http://graphtool.skewed.de (2014).
Jeffreys, H. An invariant form for the prior probability in estimation problems. Proc. R. Soc. London A 186, 453–461 (1946).
Peel, L. & Clauset, A. Detecting change points in the largescale structure of evolving networks. In Proc. 29th AAAI Conference on Artificial Intelligence (AAAI, 2015).
Delvenne, J.C., Yaliraki, S. N. & Barahona, M. Stability of graph communities across time scales. Proc. Natl Acad. Sci. USA 107, 12755–12760 (2010).
Persson, C., Bohlin, L., Edler, D. & Rosvall, M. Maps of sparse Markov chains efficiently reveal community structure in network flows with memory. Preprint at http://arxiv.org/abs/1606.08328 (2016).
Klimt, B. & Yang, Y. in Machine Learning: ECML 2004 (eds. Boulicaut, J.F., Esposito, F., Giannotti, F. & Pedreschi, D.) 217–226 (Springer, 2004).
Vanhems, P. et al. Estimating potential infection transmission routes in hospital wards using wearable proximity sensors. PLoS ONE 8, e73970 (2013).
Isella, L. et al. What’s in a crowd? Analysis of facetoface behavioral networks. J. Theor. Biol. 271, 166–180 (2011).
Eagle, N. & (Sandy) Pentland, A. Reality mining: sensing complex social systems. Pers. Ubiq. Comput. 10, 255–268 (2006).
Ziv, J. & Lempel, A. A universal algorithm for sequential data compression. IEEE Trans. Inf. Theory 23, 337–343 (1977).
Ziv, J. & Lempel, A. Compression of individual sequences via variablerate coding. IEEE Trans. Inf. Theory 24, 530–536 (1978).
Acknowledgements
We thank M. Buongermino Pereira and L. Bohlin for comments on the manuscript. M.R. was supported by the Swedish Research Council grant 201600796.
Author information
Affiliations
Contributions
Both authors conceived the project, performed the research, and wrote the paper. T.P.P. performed the analytical and numerical calculations.
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing financial interests.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Peixoto, T.P., Rosvall, M. Modelling sequences and temporal networks with dynamic community structures. Nat Commun 8, 582 (2017). https://doi.org/10.1038/s41467017001489
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467017001489
Further reading

The shape of memory in temporal networks
Nature Communications (2022)

Quantifying agent impacts on contact sequences in social interactions
Scientific Reports (2022)

Twostage anomaly detection algorithm via dynamic community evolution in temporal graph
Applied Intelligence (2022)

Trajectories through temporal networks
Applied Network Science (2021)

Theory of preference modelling for communities in scalefree networks
Applied Network Science (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.