Evolution of cooperation on temporal networks

Population structure is a key determinant in fostering cooperation among naturally self-interested individuals in microbial populations, social insect groups, and human societies. Traditional research has focused on static structures, and yet most real interactions are finite in duration and changing in time, forming a temporal network. This raises the question of whether cooperation can emerge and persist despite an intrinsically fragmented population structure. Here we develop a framework to study the evolution of cooperation on temporal networks. Surprisingly, we find that network temporality actually enhances the evolution of cooperation relative to comparable static networks, despite the fact that bursty interaction patterns generally impede cooperation. We resolve this tension by proposing a measure to quantify the amount of temporality in a network, revealing an intermediate level that maximally boosts cooperation. Our results open a new avenue for investigating the evolution of cooperation and other emergent behaviours in more realistic structured populations.


I. INTRODUCTION
Understanding and sustaining the evolution of cooperation in human and animal societies have long been a challenge since Darwin [1][2][3][4][5]. Evolutionary game theory offers a prominent paradigm to explain the emergence and persistence of cooperation among egoists, and many results have been obtained from analytical calculations [4,5], numerical simulations [5,6], and behavioral experiments [7][8][9][10][11][12][13][14]. Traditionally, researchers have been focusing on the well-mixed or homogeneous population scenarios [2][3][4]15]. Yet, both spatial population structures and social networks suggest that real populations are typically not well-mixed. Indeed, in a population some individuals may interact more likely than others do. In both theory and experiments, the ideally well-mixed scenario has been extended to heterogeneous structured populations represented by complex networks, where nodes represent individuals and links capture who interacts with whom [11,13,[15][16][17]. And a unifying framework coined as network reciprocity is proposed for emergence of cooperation in structured populations [18], especially for the networks with the degree heterogeneity which is typically observed in scale-free networks [17,19].
Despite their deep insights, those works all rely on a key assumption that the contact graph or the interaction network of individuals is time invariant. In reality, this assumption is often violated, especially in social networks, where contacts between individuals are typically short-lived. Emails and text messages for example represent near-instantaneous and hence ephemeral links in a network. Even in cases where the contacts have non-negligible durations -such as phone calls, or the face-to-face interactions between inpatients in the same hospital ward -their finite nature means that the network structure is in constant flux. It has been shown that the temporality of edge activations can noticeably affect various dynamical processes, ranging from the information or epidemics spreading [20][21][22][23] to network accessibility [24] to controllability [25].
It is natural to expect that temporality will have a similarly profound effect in social systems, particularly in situations when individuals engage in interactive behavior. Indeed, if Alice interacts with Bob who only later betrays Charlie, Alice's behavior toward Bob could not have been influenced by his later treachery. Yet the links A-B-C would be ever-present in a static representation of this social network. Despite some existing efforts [26], up to our knowledge, the impact of temporal networks on the evolution of cooperation has not been systematically explored. It is still unclear whether the temporality will enhance the cooperation or not.
Here for the first time, we explore the impacts of temporality of human interactions on the evolution of cooperation over both empirical and synthetic networks. Moreover, the impacts of the bursty behavior rooted in human activity on the evolution of cooperation are also investigated.

II. MODEL
We conduct our investigation in the setting of classic evolutionary game theory, in which each of two players may choose a strategy of cooperation (C) or defection (D). Each receives a payoff R for mutual cooperation, and an amount P for mutual defection. When the players' strategies disagree, the defector receives a payoff T while the cooperator receives S. These outcomes can be neatly encoded in the payoff matrix where the entries give the payoff each player receives under different combinations of strategy. For simplicity, we focus on the case where R = 1, T = b and S = P = 0, leaving the sole parameter b > 1, which represents the temptation to defect and hence the system's tendency toward selfish behavior. This parameter choice corresponds to the classic Prisoner's Dilemma, wherein the optimal strategy for any single individual is to defect, while mutual cooperation is the best choice for the alliance [15,17,27,28]. Figure 1 illustrates the essence of our framework. We consider a temporal network to be a sequence of separate networks on the same set of N nodes, which we call snapshots. These snapshots are constructed from empirical data by aggregating social contacts over successive windows of ∆t ( Fig. 1a and 1b), yielding the links active in that snapshot. To capture the interactions occurring on these networks, we initially set an equal fraction of cooperators and defectors (network nodes) in the population on the first snapshot. At the beginning of each generation (round of games), every individual i plays the above game with each of its k i neighbors, accumulating a total payoff P i according to the matrix above. At the end of each generation each player i may change his or her strategy, by randomly picking a neighbor j with payoff P j from its k i neighbors, and then imitating j's strategy with probability (P j − P i )/(Dk d ) if P j > P i . Here D = T − S and k d is the larger of k i and k j [17,29]. We repeat this procedure g times within each snapshot before changing the network structure (Fig. 1c). In this way, g controls the timescale difference between the dynamics on the network and the dynamics of the network. We continue running the game, changing the network structure every g generations, until the system reaches a stable fraction of cooperators, f c .

A. Temporal networks facilitate the evolution of cooperation
Our principal result is the temporal networks generally enhance cooperation relative to their static counterparts, and allow it to persist at higher levels of temptation b. Figure 2 shows the equilibrium fraction of cooperators f c for temporal networks formed from empirical data of four social systems [30]: attendees at a scientific conference (ACM conference) [31], students at a high school in Marseilles, France in two different years (Student 2012 [32], Student 2013 [33]), and workers in an office building (Office 2013) [34]. In each of these systems there exists a broad range of g over which f c is greater in the temporal network than in its static equivalent, at almost all values of b. This is true even for small ∆t. Here the network's links are distributed over a large number of rarefied snapshots, leaving little network "scaffolding" on which to build a stable cooperation. Nonetheless, there exists a range of g that can compensate for this sparsity, again leaving temporal networks the victor. Indeed, the only situation in which temporal networks are less amenable to cooperation than static networks is when g is small. In this limit, the evolutionary timescale is comparable to the dynamical timescale, and patterns of cooperation have no time to stabilize before being disrupted by the next change in network structure.
To test whether these results arise from the specific temporal patterns in real social systems, we have also simulated games on temporal versions of synthetic scale-free (SF) [35] and Erdős-Rényi (ER) [36] random networks (see Methods). We again find that with almost level of temporality (i.e., g < ∞), cooperators have an easier time gaining footholds in the population (Fig. 3). Interestingly, the temporal scale free networks yield a higher f c , all other things being equal, than the temporal ER networks ( Fig. 3 and Fig. S1). As such, temporality preserves the cooperative advantage of heterogenous populations, previous observed in static networks [17].

B. Effects of burstiness on the evolution of cooperation
Analyses of the temporal patterns of human interactions in email [37], phone calls [37], and written correspondence [38] have revealed a high degree of burstiness -periods of intense activity followed by "lulls" of relative silence. Such correlations embedded in temporal interactions have been shown to have effects on network dynamics above and beyond those of temporality alone, for instance accelerating the spread of contagions [22,39]. We have verified that burstiness is present to varying degrees in the four data sets we study, in the form of a power law distribution of inter-event times between the node activations ( Fig. S2). But to what extent do these patterns help or hinder the evolution of cooperation?
We have studied this question by randomizing the contacts in each of the datasets we study, both their source and target (i, j) and their timestamps t. We stress that this randomization has the effect of erasing bursty behavior at the level of individual node. In every temporal network, we find that cooperation is improved when the natural burstiness is removed in this way, suggesting that bursty behavior impedes the evolution of cooperation (Fig. 4). For the effects of other null models that permute only the structure or the time stamps of the contacts, please refer to Figs. S3 to S6, where we also show that the above results are robust after the data is randomized with various methods. Furthermore, this is true for nearly all choices of parameters ∆t, g, and b. But how do we make sense of these findings in relation to the observation above, namely that real temporal networks generically promote cooperation?

C. Temporality determines the fate of cooperators
The parameters g and ∆t, and the burstiness represent three different facets of temporality. Specifically, the relationship between the dynamical/structural timescales, the amount the network structure is spread over time, and the correlations between the associated snapshots, respectively. To understand the effects of these parameters in a unified way, we introduce the following measure of the temporality T of a temporal network with M snapshots as Here a ij (m) is the connectivity between nodes i and j in snapshot m, being 1 if the nodes have a contact in the associated time window and 0 otherwise, and the above fraction equals to 0 for any two nearby empty networks without links. This measure captures the likelihood that any currently inactive link will become active in the next snapshot (or conversely, that an active link becomes inactive). If we need to replay the temporal network M is T /∆t , and T /∆t − 1 if we do not. For a temporal network, generally 0 < T ≤ 1, and T = 0 for static network where network topology does not change with time. Figure 5 shows the value of T computed for the original and randomized versions of each of the four data sets we study. We see that the original data displays high temporality, which decreases following the randomization procedure (RPTRE) described above. Considering that f c for the randomized temporal networks is typically higher than in the originals (Fig. 4), this suggests that too high temporality is inimical to the spread of cooperation, instead fostering egoistic behavior. On the other hand, too low of a T is also associated with diminished cooperation. For example, f c is not maximal in Fig. 2 for ∆t = 24, which corresponds to snapshots that are relatively dense and slowly changing, paving the way for defectors to extort cooperators. Altogether, the picture that emerges is one of an intermediate regime -a "sweet spot" of temporality in which cooperation is enhanced relative to static systems.

IV. CONCLUSION AND DISCUSSION
Considering the real characteristics of human interactions where the underlying networks are temporal and possess the underlying interactive patterns, we have addressed the evolution of cooperation on temporal networks. After finding that temporal networks from empirical datasets favor the evolution of cooperation more than their static counterparts, we also validate our results on synthetic networks. This central finding holds even after the empirical data is randomized, thereby destroying specific temporal patterns (such as bursts) characterizing real human interactions. Altogether, this suggests that temporality -and temporality alone -is sufficient to improve cooperation. Interestingly, after randomizations, we find that the level of cooperation is further improved suggesting that the bursty nature of human interactions hinders the maintenance of cooperation to some degree. At last, we demonstrate that the temporality of a temporal network determines the fate of cooperators, with cooperators flourishing at intermediate values of the network temporality. By virtue of both empirical and traditional synthetic data, our explorations systematically illustrate the effects of temporality on the evolution of cooperation.
Note that the intrinsic temporal nature of the contact graph or interaction network is fundamentally different from the slight change of population structure due to individuals' migration [40][41][42]. The latter is usually restricted to the elaborate rules or strategies based on a presumed synthetic static network [27,29,[40][41][42][43][44][45][46][47][48]. The coevolution of the network and strategy has been studied in the case where the network changes passively and with small temporality under constant average degree and population size [49][50][51][52][53][54]. These coevolutionary dynamics arise from players' strategic switch of partners, a process typically governed by pre-determined mechanisms. However, it is unlikely that the natural temporality observed in real human social dynamics is driven exclusively (or even primarily) by strategic switching in pursuit of a given objective.
Another natural extension of the current work on temporal networks is to consider the group interactions, which involve the interactions between individuals who are not directly connected with one another [55][56][57][58]. These interactions generate much more dynamical complexity, which cannot be captured by pairwise interactions [59,60]. This is also true in microbial populations, where even pairwise outcomes could predict the survival of threespecies competitions with high accuracy, yet information from the outcomes of three-species competitions is still needed as we want to predict the scenario over more number of species [61]. Moreover, the menu of strategies can be expanded beyond the simple dichotomy of cooperation versus defection. For example, the canonical three strategies game rock-paperscissors, which may serve as a model to study the biological diversity in microbial populations and communities [62][63][64].
Finally, our results have implications for other dynamical processes occurring on temporal networks. If we regard the evolution of cooperation on temporal networks as a spreading dynamics of different strategies, it may serve as a new angle to investigate other related dynamics. For example, consider epidemic spreading, where the temporal network characteristics of networks had been shown to either speed up [22,65] or slow down [37,66] the spreading, and the shuffle of time stamps was shown to enhance the spreading in a network of sex buyers and prostitutes [39]. After evaluating the payoffs (benefits and costs) of susceptible and infected individuals as they encounter one another, our framework of the evolution of cooperation may help us understand more phenomena including the epidemic spreading on temporal networks.

METHODS
Empirical temporal networks. We construct temporal networks from empirical datasets [30] by aggregating contacts into undirected network links over time windows of ∆t (Fig. 1a). Here, a contact is a triplet (t, i, j) representing the fact that individuals i and j interacted during the time interval (t, t + 20s]. In this way, we obtain a temporal network with T /∆t snapshots, where T is the total time span of the dataset and z is the smallest integer greater than or equal to z. Thus the active time interval for the snapshot m is from (m − 1)∆t to m∆t, and a link between i and j exists if players i and j interact at least once in that time period (Fig. 1b). We obtain a static network in the limit where ∆t = T .
Synthetic temporal networks. We generate temporal versions of scale-free and random networks with size N and average degree k by first generating a base static network, using static model [67] and the Erdős-Rényi model [36], respectively. We then form M snapshots by randomly and independently choosing a fraction p of edges to be active in each one. We have verified that our results hold under more sophisticated methods for building temporal networks from a static network backbone, such as the activity-drive model [68] Randomizations of empirical datasets. We consider four widely-used null models [69] to randomize the empirical data: Randomized Edges (RE) where we randomly choose pairs of edges (i, j) and (i , j ), and replace them with (i, i ) and (j, j ) or (i, j ) and (j, i ) with equal probability provided this results in no self loops; Randomly Permuted Times (RPT), where we shuffle the timestamps of the contacts, leaving their sources and targets unaltered; Randomly Permuted Times + Randomized Edges (RPTRE) which consists first of RPT followed by RE; and Time Reversal (TR), where the temporal order of the contacts is reversed.    in each snapshot before changing the network structure to the next one, and totally we run G generations until the composition of the population is stable. If T /∆t * g < G, we repeat the sequence of snapshots from the beginning until convergence.  Table I.  We see that the frequency of cooperators generally increases after the bursty behavior is destroyed, suggesting that correlations in activity within a social network is antagonistic toward the formation of cooperation. Results on each dataset after randomizations with different null models [69] are given in Figs. S3 to S6 in the SI. Other parameters are the same as those in Fig. 2.    fraction of cooperators f c as a function of the dilemma parameter b for different g. RE and TR have no effect on the correlations in temporal activity by construction, and hence have no effects on network temporality apparently. RPT and RPTRE, on the other hand, destroy the temporal correlations between edges, thereby lowering the (too high) temporality of the system. Actually for the temporal network where we run g generation on each snapshot, the temporality of the underlying population structure is about T /g. Thus for small g under RPT and RPTRE, f c is increased markedly relative to the original dataset, while for large g the gains are more modest.
The above findings are also true for other datasets (see Figs. S4 to S6). Overall, our results showing that temporal networks could facilitate the evolution of cooperation are robust even after the data is randomized. Other parameters are the same as those in Fig. 2.