Optimal interdependence between networks for the evolution of cooperation

Recent research has identified interactions between networks as crucial for the outcome of evolutionary games taking place on them. While the consensus is that interdependence does promote cooperation by means of organizational complexity and enhanced reciprocity that is out of reach on isolated networks, we here address the question just how much interdependence there should be. Intuitively, one might assume the more the better. However, we show that in fact only an intermediate density of sufficiently strong interactions between networks warrants an optimal resolution of social dilemmas. This is due to an intricate interplay between the heterogeneity that causes an asymmetric strategy flow because of the additional links between the networks, and the independent formation of cooperative patterns on each individual network. Presented results are robust to variations of the strategy updating rule, the topology of interdependent networks, and the governing social dilemma, thus suggesting a high degree of universality.

N etwork reciprocity is amongst the most well-known mechanisms that may sustain cooperation in evolutionary games that constitute a social dilemma 1 . It was discovered by Nowak and May 2 , who observed that on structured populations cooperators can aggregate into compact clusters and so avoid being wiped out by defectors. Although the mechanism may not work equally well for all social dilemmas 3 , and recent empirical evidence based on large-scale economic experiments indicate that it may be compromised or fail altogether 4,5 , there is still ample interest in understanding how and why networks influence the evolution of cooperation. Recent reviews are a testament to the continued liveliness of this field of research [6][7][8][9] .
Previous research concerning evolutionary games on interdependent networks has revealed, for example, that biased utility functions suppress the feedback of individual success, which leads to a spontaneous separation of characteristic time scales on the two interdependent networks 38 . Consequently, cooperation is promoted because the aggressive invasion of defectors is more sensitive to the deceleration. Even if the utilities are not biased, cooperation can still be promoted by means of interdependent network reciprocity 39 , which however requires simultaneous formation of correlated cooperative clusters on both networks. It has also been shown that the coupling of the evolutionary dynamics in each of the two networks enhances the resilience of cooperation, and that this is intrinsically related to the non-trivial organization of cooperators across the interdependent layers 40 . Perhaps most closely related to the setup of the present work is that by Wang et al. 42 , who showed that probabilistic interconnections between interdependent networks can very much promote the evolution of cooperation. In our model, however, the strategy transfer between networks is prohibited. The interdependence is thus due solely to coupling together the payoffs of select players that reside on different networks.
Here, we wish to determine how strong the interdependence between the networks really ought to be for the optimal promotion of cooperation. Since existing works unequivocally declare that interdependence works in favor of the resolution of social dilemmas, one might intuitively assume that the stronger the interdependence the better. As we will show, however, this assumption is not necessarily true. To address the problem, we consider primarily the prisoner's dilemma game on two square lattices, where a certain fraction of randomly selected players is allowed to connect with the corresponding players in the other lattice. While strategy transfers between the two networks are not allowed, the additional connections between the corresponding players do influence their utility, and thus their ability to retain and possibly spread their strategies on the home network. This introduces two new parameters, namely the fraction of players that is allowed to form links with the corresponding players in the other network r, and the strength of this links a. Together, these two parameters determine the strength of interdependence, and also the success of resolving social dilemmas. For further details with regards to the studied evolutionary games, the applied dynamical rule, and the topology of interaction networks, we refer to the Methods section. Independent of the strategy updating rule, the topology of interdependent networks, and the governing social dilemma, we will show that cooperation is promoted best if only an intermediate fraction of players is allowed to have external links to the other network, but also that those links should be sufficiently strong. We will also reveal mechanisms that lead to the emergence of the optimal interdependence.

Results
To begin with, we show in Fig. 1 the impact of parameters r and a on the outcome of the prisoner's dilemma game. It can be observed that there exists an intermediate range of the fraction of players that are allowed to form an external link at which cooperators fare best. Irrespective of the temptation to defect b, values around r < 0.5 yield an optimal outcome of the social dilemma. Yet the coupling strength is important too. Only if the value of a is sufficiently large are the players able to utilize the advantage of being linked to their corresponding players in the other network. Although the level of cooperation appears to fade slightly beyond a 5 0.7 if the temptation to defect is low or moderate [panels (a) and (b)], the prevailing conclusion is that the coupling strength needs to be sufficiently strong.
To clarify the mechanism that is responsible for the promotion of cooperation, we first monitor the evolution of cooperation by measuring not just the overall average cooperation level (f C ), but separately also the average cooperation level for players with (f C d ) and without (f C o ) external links to the other network in dependence on the number of full Monte Carlo steps (MCS), as defined in the Methods section. For easier reference, we will refer to individuals with external links to the other network as ''distinguished'' and to those who have no such links as ''ordinary'' players (note that the subscripts in f C d and f Co are chosen accordingly). Figure 2 reveals that the cooperation level amongst the distinguished players who do have external links to the other network is significantly higher than the cooperation level amongst the players who are not externally linked. Expectedly, the overall cooperation level is in-between f C d and f Co . The identified difference between f C d and f Co is crucial, because it indicates that players who have the ability to collect an additional payoff from the other network are more likely to cooperate. Indeed, the natural selection of the cooperative strategy among distinguished players is higher than in the whole population, which is a consequence of an asymmetric strategy flow that emerges between the distinguished and other players. Because of generally higher payoffs, the distinguished players are followed by the others, which results in the selection of cooperation around them. In other words, distinguished players with an external link to the other network play the role of leaders in the community, similarly as was reported many times before for hubs on heterogeneous isolated (individual) networks 6 .
To test this explanation further, we directly adjust the teaching activity of players, i.e., the ability to pass their strategy to a neighbor 59,60 , which can be done effectively by introducing a multiplication factor w to Eq. 1. We consider two options. First, we depreciate all distinguished players by using w 5 0.05 while keeping w 5 1 for those who do not have an external link, and second, we reverse these values. The expectation is that the first option will nullify the advantage of interdependence between the two networks, while the second   For comparison, we show in panel (a) the results obtained with the basic model where all players have w 5 1. There the change from a 5 0 to a 5 0.5 (note that for a 5 0 the difference between distinguished and ordinary players vanishes) introduces a noticeable increase in the critical temptation to defect where cooperators die out and an overall increase in the level of cooperation. But if w 5 0.05 is applied to distinguished players, then they become unable to lead the others despite of the fact that their utilities are higher than those of their neighbors. In the absence of hierarchy that was previously warranted by the interdependence between networks, there will form no groups with homogeneous strategies, and hence the advantage of cooperation will not be revealed. As panel (b) shows quite interestingly, a 5 0 (absence of interdependence between networks) even slightly outperforms a 5 0.5, because a weak hierarchy is then restored due to the unequal teaching activity of players. Note that at a 5 0.5 the unequal teaching activity is counteracted by the additional payoff distinguished players receive from the other network. At a 5 0 this counterbalance is lost, and the ordinary players become the leaders due to their higher value of w. The positive effect on the level of cooperation is, however, rather marginal.
In the opposite case, when distinguished players are endowed with the full teaching activity w 5 1 while ordinary players without an external link are depreciated with w 5 0.05, the change from a 5 0 to a 5 0.5 is the strongest [panel (c)]. Here the inequality of w and the additional payoffs stemming from the other network are able to strengthen each other and fortify the leaders to yield the highest cooperation level within the framework of this model. The enforced role of distinguished players will cause an immediate reaction from the followers, and this prompt reaction will select the more successful cooperative strategy as described before. From the technical point of view, it is interesting to note that the curves for a 5 0 in Figs. 3(b) and (c) coincide, because in the absence of extra payoff from the other network it does not matter whether distinguished or ordinary players are endowed with w 5 1 (or w 5 0.05). Since the fraction of distinguished players was set to r 5 0.5, both fractions are equally large, thus resulting in the same cooperation level.
Results presented in Fig. 3 and the pertaining interpretation can be corroborated by comparing the evolution of cooperation within different groups of players, similarly as done in Fig. 2 above. As Fig. 4(b) shows, when the teaching activity is reduced for distinguished players the cooperation among them is not favored by natural selection. Consequently, only a very low value of b can ensure a reasonably high cooperation level in the whole population because the latter is dragged down considerably by the low f C d . There is a slight improvement among other players, as evidenced by the higher f Co , which is due to a higher teaching activity. However, despite their higher teaching activity, ordinary players cannot lead the whole population efficiently because distinguished players are reluctant to follow them due to their higher individual utility. If either the teaching activity of all players is left intact [panel (a)] or the higher teaching activity is awarded to distinguished players [panel (c)], then f Co vf C d , as observed initially in Fig. 2. Thus, the basic mechanism of cooperation promotion is restored or even additional fortified.
Furthermore, it is important to emphasize that the optimal interdependence between the networks can work in favor of cooperation only if there are distinguished players in both graphs. On the other hand, if the additional utilities flow only in one direction, the mechanism will fail or yield only a marginally better outcome. Results supporting this argumentation are presented in Fig. 5. For reference, the outcome on an isolated network is depicted as well. It can be observed that if distinguished players in the first network can collect additional payoffs from the second network, but the evolution in the second network is independent from the first network because players there are unable to collect additional payoffs (the interdependence is unilateral), then the critical temptation to defect b at which cooperators die out does not increase at all. In fact, only the level of cooperation increases slightly in the mixed C 1 D phase. If the interdependence is bilateral, i.e., players on both networks can collect an additional payoff from the other network, however, the positive impact on the evolution of cooperation is much stronger. This highlights the importance of mutual interdependence and the independent formation of cooperative patterns on each individual network. Players connected to their corresponding partners in the other network can support each other effectively only if homogenous cooperative domains emerge on both networks. Notably, the importance of correlated growth and formation of cooperative domains has been emphasized already in 39 , although here they need not overlap geographically. We could observe practically the same cooperation level when distinguished players collected additional payoffs from an ordinary player in the other network. Namely, there is no need to link distinguished players from different networks with one another. The  In panel (b) the teaching activity of all distinguished players who have an external link to the other network is reduced to w 5 0.05. It can be observed that the promotion of cooperation due to the interdependence vanishes. In panel (c) the teaching activity of all ordinary players is reduced to w 5 0.05, which further strengthens the leading role of the distinguished players and leads to the strongest promotion of cooperation. All panels feature results for a 5 0 and a 5 0.5 at r 5 0.5, where a 5 0 means that effectively the two networks are isolated, i.e., there is no interdependence because the links between the two networks yield no additional utility to either player. www.nature.com/scientificreports crucial condition is the chance of heterogeneity on both networks 61 , the positive effect of which can then be mutually amplified through the interdependence.
Our argument for the spreading of cooperative behavior among distinguished players can also be supported by how the border of the full D phase behaves in dependence of r and a at a high temptation to defect. As results presented in Fig. 1(c) suggest, a smaller density of distinguished players can be compensated by a higher value of a, but only up to a certain point. At such a high temptation value cooperators cannot survive even for large values of a if the density of distinguished players r is below a threshold value. Naturally, the critical r depends slightly on the temptation to defect [compare panels (b) and (c)], but the smallest value of the phase transition point is close to the critical r c 5 0.1869(1) value of jamming coverage of particles during a random sequential adsorption when nearest and next nearest neighbor interactions are excluded on a square lattice 62 . Shortly, if distinguished players are too rare, then to support them via a high a will not yield the desired impact because their influence cannot percolate. The latter, however, is an essential condition to maintain cooperation, which was already pointed out in previous works 63,64 .
It is lastly of interest to verify the robustness of these observations, first with regards to the strategy updating rule. As results presented in Fig. 6 demonstrate, our conclusions are not restricted solely to the Fermi-type strategy updating 6,65 , but remain valid also under the best-takes-over rule 2 and proportional imitation 66 . In both cases an optimal intermediate value of r is clearly inferable, and the positive effect on the evolution of cooperation is the stronger the larger the value of the coupling strength a. This is qualitatively identical as observed in Fig. 1 with the Fermi rule.
Since previous works revealed that the clustering coefficient could be a decisive factor affecting the evolution of cooperation in games that are governed by pairwise interactions 67,68 (note that this is not the case for games governed by group interactions 69 ), it is also instructive to examine the relevance of network interdependence under this condition. Unlike the square lattice that has a zero clustering coefficient, the triangular lattice has a high clustering coefficient, and thus serves the purpose very well. Results presented in Fig. 7(a) attest to the fact that the existence of optimal network interdependence does not depend on structural properties of each individual network, as indeed the a -r dependence of f C is the same as observed before for   the square lattice in Fig. 1. To conclude, we extend our exploration also to other social dilemmas, more precisely the snowdrift game, and as shown in Fig. 7(b), the main conclusions remain intact. To reiterate, the exists and intermediate level of interdependence between networks that is optimal for the resolution of social dilemmas.

Discussion
Summarizing, we have studied the evolution of cooperation in the spatial prisoner's dilemma and the snowdrift game subject to interconnectedness by means of a different fraction of differently strong links between the corresponding players residing on the two interdependent networks. We have found that for cooperation to be optimally promoted, the interdependence should stem only from an intermediate fraction of links connecting the two networks, and that those links should affect the utility of players significantly. The existence of optimal interdependence has been attributed to the heterogeneity that is brought about by the enhanced utility of those players that do have external links to the other network, as opposed to those who have not. This introduces asymmetric strategy flow, which in turn leads to the emergence of influential leaders that can act as strong cooperative hubs in their respective networks. Importantly, the compact cooperative patterns that appear independently on both networks support each other mutually through the links that constitute the interdependence. Indeed, we have shown that the mechanism works best only if the interdependence is bilateral, and if the asymmetric strategy flow is not counteracted by artificial low weights assigned to the reproducibility of interconnected players. In case of unidirectional interdependence or if the reproducibility of players is altered, however, some marginal benefits for cooperators may still exist, but these are then far removed from the full potential of interdependent networks to aid the resolution of social dilemmas. We have tested the robustness of these conclusions by replacing the Fermi strategy adoption rule with the best-takesover and the proportional imitation rule, as well as by replacing the square lattice having zero clustering coefficient with the triangle lattice that has a much higher clustering coefficient, as well as finally by replacing the prisoner's dilemma game with the snowdrift game. Quite remarkably, we have found that the optimal interdependence persist across all these different setups, thus leading to the conclusion that it ought to be to a large degree a universally valid phenomenon.
While the games on interdependent networks studied here are not meant to model a particular real-life situation, they nevertheless do capture the essence of some situations that are viable in reality. For example, it is generally accepted that not all individuals are equally fond of making connections outside of their natural environment. Similarly, some would very much wish to do so, but may not have a chance. These and similar considerations may all affect the level of interdependence between two or more networks, and it is within the realm of these possibilities that our study predicts the existence of an optimal level of interconnectedness. Future studies could address the coupling of more complex (small world or scale-free, for example) interaction topologies, the outcome of other games on interdependent networks, such as for example the traveler's dilemma game that has recently been studied in a spatial setting 70 , as well as the impact of coevolution and growth, both of which have recently been the subject of much interest [71][72][73][74][75] . Overall, it seems safe to conclude that the interdependence of interaction networks offers several exciting possibilities for further research related to evolutionary games, and it ought to bring the models a step closer to actual conditions, given that networks indeed rarely exist in an isolated state.

Methods
The evolutionary game is staged on two square lattices, each of size L 3 L, where initially each player x is designated either as a cooperator (s x 5 C) or defector (s x 5 D) with equal probability. Likewise randomly, a fraction r of players on each lattice is selected and allowed to form an external link with a corresponding player in the other lattice. Although we predominantly use the square lattice, we will also resort to using the triangle lattice, given that the difference in the clustering coefficient has been determined as a potentially key factor for the outcome of games that are governed by pairwise interactions 67,68 .
The accumulation of payoffs p x on both networks follows the same standard procedure, depending on the type of the governing social dilemma. In both games two cooperators facing one another acquire R, two defectors get P, whereas a cooperator receives S if facing a defector who then gains T. The prisoner's dilemma game is characterized by the temptation to defect T 5 b, reward for mutual cooperation R 5 1, and punishment P as well as the sucker's payoff S equaling 0, whereby 1 , b # 2 ensures a proper payoff ranking 2 . We note that qualitatively similar results are obtained also for other values of S. The snowdrift game, on the other hand, has T 5 b, R 5 b 2 1/2, S 5 b 2 1 and P 5 0, where the temptation to defect can be expressed in terms of the cost-to-benefit ratio r 5 1/(2b 2 1) with 0 # r # 1. Due to the interdependence (external links between corresponding players), however, the utilities used to determine fitness are not simply payoffs obtained from the interactions with the nearest neighbors on each individual network, but rather U x 5 p x 1 ap x9 for players that have an external link, and U x 5 p x otherwise. The parameter 0 # a # 1 determines the strength of external links, i.e., the larger its value the higher the potential increase of utility of two players that are connected by the external link.
Importantly, while the interdependence affects the utility of players, it does not allow strategies to be transferred between the two networks. Thus, the evolution of the two strategies proceeds in accordance with the standard Monte Carlo simulation procedure comprising the following elementary steps for each network. First, a randomly selected player x acquires its utility U x by playing the game with all its nearest neighbors and taking into account also the potential addition to the utility stemming from the possible external link, as described above. Next, one randomly chosen neighbor of x within the same network, denoted by y, also acquires its utility U y in the same way. Lastly, player x attempts to adopt the strategy s y from player y with a probability determined by the Fermi function where K 5 0.1 quantifies the uncertainty related to the strategy adoption process 6,65 . The latter is usually associated with errors in decision making and imperfect information transfer between the players. Notably, to test the robustness of our findings, we will also use the best-takes-over 2 and the proportional imitation 66 strategy updating rule. Regardless of which type of interaction network, evolutionary game, or the strategy updating rule is used, in accordance with the random asynchronous update, each player on both networks is selected once on average during a full Monte Carlo step. Moreover, sufficiently large system sizes (from L 5 200 to L 5 800) and relaxation times need to be used to avoid finite size effects and to ensure a stationary state has been reached. Presented results were averaged over up to 30 independent runs to further improve accuracy.