Abstract
Strategic interactions arise in all domains of life. This form of competition often plays out in dynamically changing environments. The strategies employed in a population may alter the state of the environment, which may in turn feedback to change the incentive structure of strategic interactions. Feedbacks between strategies and the environment are common in socialecological systems, evolutionaryecological systems, and even psychologicaleconomic systems. Here we develop a framework of ‘ecoevolutionary game theory’ that enables the study of strategic and environmental dynamics with feedbacks. We consider environments governed either by intrinsic growth, decay, or tipping points. We show how the joint dynamics of strategies and the environment depend on the incentives for individuals to lead or follow behavioral changes, and on the relative speed of environmental versus strategic change. Our analysis unites dynamical phenomena that occur in settings as diverse as human decisionmaking, plant nutrient acquisition, and resource harvesting. We discuss implications in fields ranging from ecology to economics.
Introduction
In many settings an individual’s payoff depends on both her own strategy, or type, as well as the strategic composition in the entire population. Such interactions arise across a range of disciplines, from microeconomics to animal behavior, and they have been analyzed using game theory^{1,2}. A gametheoretic analysis of competing types typically assumes that the nature of the strategic interaction is fixed in time, or that it depends on the state of an independent, exogenous environment. Realworld systems, however, often feature bidirectional feedbacks between the environment and the incentives in strategic interactions: an individual’s payoff depends not only on her actions relative to the population, but also on the state of the environment, and the state of the environment is influenced by the actions adopted by individuals in the population.
Reciprocal feedbacks between strategies and the environment play out in many complex systems with broad biological and societal relevance^{3,4,5,6,7,8,9,10,11}. In fisheries, for example, the relative reward of a highintensity versus a lowintensity harvesting strategy depends upon the current biomass of the fish stock; and, conversely, stock dynamics depend on the frequencies of these two harvesting strategies^{12}. In ecology, species strategies and interactions determine their competitive balance while also changing the abiotic environment and, in turn, the nature of competition. For example, symbiotic nitrogen fixation in plants increases local nitrogen availability over time, altering the competitive environment^{13}. Conversely, nutrient availability will change incentives for nutrient exchange in such symbioses^{14}. Likewise, global climate change is caused by the strategic decisions of individuals, corporations, and nations, which have longterm environmental repercussions that will, in turn, alter the strategic landscape those parties face. Feedbacks even occur in psychology, where deliberative decisionmaking can beneficially shape the shared environment of social norms and institutions, but that environment then sets the stage for the success of less costly, nondeliberative decisionmaking^{15}. Examples from these diverse fields share the common feature that feedbacks between strategies and the environment fundamentally alter dynamical predictions^{16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31}. Understanding such systems requires a framework for studying gametheoretic dynamics when the nature of competitive interactions influences the environment, and conversely.
Here we develop a general framework for ecoevolutionary games. We analyze the class of twostrategy, linear games linked to a renewing or decaying resource. We derive stability criteria for all strategic/environmental equilibria, and we derive the conditions that permit cyclic dynamics, bistability, or a stable equilibrium that supports multiple coexisting types. We show that environmental feedbacks alter longterm outcomes and expand the suite of dynamical possibilities. Importantly, we can characterize the large range of possible dynamical behavior in terms of a few easily interpretable quantities: incentives to lead or follow strategic changes in the population. We find that cyclical dynamics arise in a restricted subset of game structures, and that in these regions, the dynamical behavior depends critically on the relative timescale of strategic versus environmental change.
Results
Ecoevolutionary games
First we describe a general framework for modeling ecoevolutionary feedbacks, and then we analyze linear twostrategy ecoevolutionary games. We characterize the range of possible dynamical behaviors in these systems, and we apply this analysis to several case studies drawn from a range of disciplines.
Ecoevolutionary games occur when evolutionary game dynamics are environmentally coupled. First, consider a set S of different strategies that can be employed in a system of interest. Depending on the system being studied, the strategies may be defined as a set of alternative resource extraction technologies, or various nutrient acquisition strategies of plants, or different physiologies or morphologies of organisms, or different cognitive types that excel in different environments. We assume that an individual’s fitness from adopting strategy s_{i} ∈ S is π_{i}(x, n). The individual’s fitness is a function of her strategy, the frequency of all K strategies in the population, \({\bf{x}}=\left[{x}_{1},{x}_{2},\ldots ,{x}_{K}\right]\), which we call the population strategy profile, and also the state of the environment, n. Depending upon the context, the environmental state might correspond to the concentration of a vital nutrient, the biomass of species in a community, the concentration of greenhouse gasses (or other pollutants) in the atmosphere, or the quality of social norms. Since the fitness of an individual depends on the strategies employed by other individuals, the setting we have described is gametheoretic in nature.
We study the dynamics of the frequencies of strategies with the replicator equation^{32}, writing the rate of change of the frequency of strategy s_{i} as
where \(\phi ({\bf{x}})={\sum }_{j=1}^{K}{x}_{j}{\pi }_{j}({\bf{x}},n)\) is the mean fitness of the population and ϵ_{3} is a parameter that describes the speed of strategy dynamics. This equation implies that the frequency of a strategy increases when the fitness of those who adopt it is greater than the average fitness of the population.
We have described a gametheoretic interaction that is environmentally dependent. But we are interested in systems which, furthermore, contain environmental feedbacks. Such feedbacks arise when strategies have an impact on the environment. The impact on the environment is channeled through a function, h(x, n), which aggregates the influence of the current population strategy profile on changes to the environmental variable n. The environmental factor n may also have its own intrinsic dynamic governed by f(n), which describes the intrinsic rate of change of the environmental variable as a function of the current environmental state. Depending on the system of study, these intrinsic dynamics could describe food webs (if modeling a higherdimensional environmental state), soil weathering, or earth systems processes. This results in environmental change governed by
where ϵ_{1} and ϵ_{2} determine the speeds of the intrinsic environmental dynamics (independent of strategies played) and of the extrinsic impact of strategies on the environment, respectively. In total, this describes a system where evolutionary dynamics are reciprocally linked with the environment in a dynamic ecoevolutionary game. The model is described by a system of K differential equations (1 environmental equation and K − 1 strategy equations, since x lies on a simplex).
For a twostrategy game with an environmental feedback we can write the ecoevolutionary system as
because in this case x_{1} = 1 − x_{2}. This framework features three different timescales: the timescale of intrinsic dynamics of the environment, ϵ_{1}, the timescale of the environmental impact of the strategies currently employed in the population, ϵ_{2}, and the timescale of strategy update dynamics (strategy evolution) in the population, ϵ_{3}. We can normalize the first two timescales relative to the third so that we drop ϵ_{3}, without loss of generality. This framework allows for nonlinearity in the payoff structures as well as in environmental impact and intrinsic dynamics, so that the space of models and potential dynamics is vast. We focus on models in which payoffs are linear in the state of the environment and the strategy frequencies, but we also consider nonlinear payoffs (see Supplementary Note 6).
Linear ecoevolutionary games
In this section we describe a class of twostrategy ecoevolutionary games where the payoffs to individuals are linear in both the population strategy profile, x, and in the environmental state, n. We will show that several important models from disparate fields are instances of linear ecoevolutionary games, in which we can write the payoffs in terms of a matrix
Here the state of the environment, n, is normalized to fall between 0 and 1, and the entries of the two matrices correspond to the payoffs of the game under rich (n = 1) and poor (n = 0) environmental states. Using Π(n), we write the payoffs for using strategy 1 and strategy 2 as π_{1}(x, n) and π_{2}(x, n), respectively, where x denotes the fraction of the population that plays strategy 1.
In the remainder of the results section we systematically analyze linear ecoevolutionary games coupled to environments (resources) with either renewing or decaying intrinsic dynamics. We also analyze a class of models coupled to complex environments governed by tipping points.
Ecoevolutionary games with selfrenewing and decaying resources
Intrinsic resource dynamics can take many forms, but are broadly categorized as renewing or decaying. We consider two different environmental dynamics: (i) a renewable resource where each strategy exerts degradation (or harvesting) pressure on the resource stock, and (ii) a decaying resource that is produced as a byproduct of each strategy.
We first suppose there is a resource stock, m, that grows logistically in the absence of consumption or harvesting, and is diminished by harvesting or consumption associated with the strategies in a game. Let e_{L} and e_{H} be the resource harvest effort of strategies low and high, respectively, with e_{L} < e_{H}. The resource dynamics are then governed by
where x is the fraction of the population playing strategy L, r is the intrinsic rate of resource growth, and k is the resource carrying capacity. Here q is a parameter that maps resource degradation pressures (or harvesting efforts) (e_{L}, e_{H}) into the rate of reduction in the resource. We assume that environmental impact rates are restricted so that m will be positive at equilibrium. This implies that e_{H} ∈ (0, r∕q) and e_{L} ∈ [0, e_{H}), which spans the effort the leads to maximum sustainable yield as well as the bioeconomic effort level associated with open access and the tragedy of the commons. And so, such models are well suited to address questions of environmental stewardship or overexploitation.
The resource stock m can assume any nonnegative value. Regardless of its initial value, though, m will eventually fall between its equilibrium values when either high or loweffort strategies dominate the population. Therefore, we can transform m to an environmental state n in our framework, which is bounded between 0 and 1, using a simple linear transformation between these two extreme equilibrial values (see Supplementary Note 1).
Using the payoff matrix from Eq. 5 we can write our ecoevolutionary system as
In terms of our general framework (Eq. 2) we have \(f(n)=\left(rq({e}_{{\mathrm{L}}}n + {e}_{{\mathrm{H}}}(1n))\right)n\) and \(h(x,n)=\left(rq({e}_{{\mathrm{L}}}n+{e}_{{\mathrm{H}}}(1n))\right)x\). Here we have assumed that the timescales of intrinsic resource dynamics and resource harvesting are the same (i.e., ϵ_{1} = ϵ_{2} = ϵ), so that we have only one relative timescale remaining. In other words, ϵ quantifies the rate of environmental dynamics (both intrinsic and extrinsic) compared to the rate of strategy dynamics.
We can make a similar mapping for a model with a decaying resource, such as a pollutant. Let m denote the pollutant level in the environment. Each strategy played in the evolutionary game produces the pollutant as a byproduct at some rate. Let e_{L} and e_{H} be the emissions rates of the low emissions and high emissions strategies, respectively. Then we can model the stock of m with the differential equation
where x is the fraction of the population that plays the strategy with low emissions (strategy L), (1 − x) is the fraction the plays the high emission strategy and α is the decay rate of the resource stock. We can again define n, bounded between 0 and 1, as a linear transformation of m (see Supplementary Note 2), yielding dynamics governed by
Although decaying and renewing resources arise in different biological or social contexts, they both yield the same qualitative results and our analysis of dynamical outcomes applies to both cases.
Ecoevolutionary games with environmental tipping points
An alternative model of environmental dynamics was introduced by Weitz et al.^{24}, in which the environment is degraded by one strategy and enhanced by the other: \(\frac{{\mathrm{d}}n}{{\mathrm{d}}t}=\tilde{\epsilon }n(1n)(\theta x(1x))\). The last term in this equation denotes environmental enhancement by one strategy at rate θ and degradation at relative rate one. Compared to the models above with intrinsic dynamics, the environment described by Weitz et al.^{24} has no intrinsic dynamic: it changes only as a direct result of the strategies in the population. We contextualize the preceding work of Weitz et al.^{24} within a more general framework that includes intrinsic environmental dynamics governed by tipping points; and we show that the qualitative conclusions of Weitz et al.^{24} occur as a limiting special case (see case study 4, below).
Incentives for behavioral change in ecoevolutionary games
The dynamics of the general linear ecoevolutionary system above, with a renewing or decaying resource, can be understood in terms of four parameter combinations. These four parameters have intuitive interpretations as incentives to change behavior:
These four parameters will allow us express dynamical outcomes in terms of the incentive to lead or follow strategy change under either a rich or a poor environmental state (Fig. 1).
The parameter \({\Delta }_{{\mathrm{L}}}^{1}\) quantifies the incentive to switch to the strategy with high environmental impact (denoted by Δ) given that all other individuals follow the lowimpact strategy (denoted by the L subscript) and the system is currently in a rich environmental state (denoted by the superscript 1). In the context of socioecological systems, \({\Delta }_{{\mathrm{L}}}^{1}\) can be interpreted as the incentive to “lead a gold rush”—that is, be the first player to switch to a highimpact strategy and reap the rewards of an abundant resource. By contrast, the parameter \({\Delta }_{{\mathrm{H}}}^{1}\) quantifies the incentive to switch to the highimpact strategy under a rich environmental state, given that every other player has already switched. In other words, \({\Delta }_{{\mathrm{H}}}^{1}\) is the incentive to “follow a gold rush”.
The parameter \({\delta }_{{\mathrm{H}}}^{0}\) quantifies incentive to switch to the lowimpact strategy (denoted by δ) when in a poor environmental state and when all other players are following the highimpact strategy. And so we can think of \({\delta }_{{\mathrm{H}}}^{0}\) as the incentive to “lead an environmental movement” by reducing harvesting of a depleted resource. Finally, the parameter \({\delta }_{{\mathrm{L}}}^{0}\) quantifies the incentive to switch to the lowimpact strategy given that all other individuals are following the lowimpact strategy and the environment is in a poor state. Thus \({\delta }_{{\mathrm{L}}}^{0}\) can be seen as the incentive to “follow an environmental movement”.
The verbal descriptions of these four critical parameters apply in the context of a socioecological system, such a fishery. But there are natural alternative interpretations of these parameters for a range of related phenomena across diverse fields.
The general linear evoevolutionary system has up to four equilibria, by which we mean fixed points. There are two equilibria which support a single strategy in the population, at \(({x}^{* },{n}^{* })\in \left\{(0,0),(1,1)\right\}\); and there are also up to two equilibria that support multiple coexisting strategies in the population, denoted by \(({x}_{+}^{* },{n}_{+}^{* })\) and \(({x}_{}^{* },{n}_{}^{* })\). The equilibrium (x^{*}, n^{*}) = (0, 0) is stable only if \({\delta }_{{\mathrm{H}}}^{0}\ <\ 0\), which is intuitively clear from Fig. 1. Similarly, the equilibrium (x^{*}, n^{*}) = (1, 1) is stable only if \({\Delta }_{{\mathrm{L}}}^{1}\ <\ 0\). The equilibrium at \({x}_{+}^{* }\) is always a saddle, and thus never stable. Whereas the equilibrium \({x}_{}^{* }\) can be either stable or unstable.
We only find persistent cycles in the ecoevolutionary system when the interior equilibrium \({x}_{}^{* }\) is unstable. Conditions for this equilibrium to be unstable first require
and
Instability of \({x}_{}^{* }\) also requires
where ϵ is the speed of environmental feedback relative to speed of strategy updating. The value of ϵ_{crit} can be expressed analytically in terms of the parameters of the system, and it differs slightly for renewing versus decaying resource feedbacks (see Supplementary Eqs. 25 and 47 in Supplementary Notes 1 and 2).
Dynamical regimes in ecoevolutionary games
Strategyenvironment dynamics exhibit several different qualitative regimes, depending on the incentives to switch strategies (δ’s and Δ’s) and on the timescale separation, ϵ, between strategy evolution and environmental impacts.
When there is no incentive to lead either the environmental movement or the gold rush (\({\Delta }_{{\mathrm{L}}}^{1},{\delta }_{{\mathrm{H}}}^{0}\ <\ 0\)), as in Fig. 2a, then both edge equilibria are stable, and only the saddle equilibrium falls within the state space. This means that the dynamics in this regime will exhibit bistability—with attraction to a population composed entirely of one or another strategy, depending upon the initial conditions. This result is intuitive because, in this regime, there is no incentive for individuals to be leaders of change in either the poor or rich environmental state. Therefore the system will eventually be dominated by one or the other strategic type, with the corresponding environmental equilibrium.
When there are positive incentives for individuals to lead both the gold rush and the environmental movement (\({\Delta }_{{\mathrm{L}}}^{1},{\delta }_{{\mathrm{H}}}^{0}\,> \,0\), Fig. 2b), then neither edge equilibrium is stable. In this regime, because individuals are always incentivized to lead change, environmental quality can possibly cycle over time. However, positive incentives to lead change are not sufficient to induce cycles. Cycles in this regime require that the incentives to be a follower of change are also positive and stronger, in aggregate, than the incentives to lead change. Furthermore, cycles also require that the environmental feedback is sufficiently slow compared to strategy evolution (Fig. 3). In sum, when there are positive incentives to lead both movements, a stable limit cycle will occur when \({\Delta }_{{\mathrm{H}}}^{1}\,> \, 0\), \({\delta }_{{\mathrm{L}}}^{0}\,> \, 0\), \({\Delta }_{{\mathrm{H}}}^{1}{\delta }_{{\mathrm{L}}}^{0}\,> \,{\Delta }_{{\mathrm{L}}}^{1}{\delta }_{{\mathrm{H}}}^{0}\), and ϵ < ϵ_{crit}. We find no evidence of cycles outside this region (see Supplementary Note 3).
When there are positive incentives to lead the environmental movement but not to lead the gold rush (\({\Delta }_{{\mathrm{L}}}^{1}\, <\,0\), \({\delta }_{{\mathrm{H}}}^{0}\,> \, 0\); Fig. 2c), then a population composed of lowimpact strategists will always be stable, whereas the highimpact strategic state will always be unstable. In this region we find parameter regimes that lead to a single monomorphic equilibrium, bistability, or stable limit cycles. In the blueshaded region of Fig. 2c there are no interior equilibria, so the system will settle on lowimpact strategists alone. In the yellow and green regions, however, there are two interior equilibria. One of these is always a saddle while the other one can be stable or unstable. In the green region, with a stable interior equilibrium the system is bistable: it will approach either a population monomorphic for lowimpact strategists, or a stable mix of both strategies. In the yellow region, the stable interior equilibrium becomes unstable under slow environmental feedbacks, leading to a limit cycle or a monomorphic population depending upon the rate of environmental feedback (Fig. 4).
When there are positive incentives to lead the gold rush but not the environmental movement (\({\Delta }_{{\mathrm{L}}}^{1}\,> \, 0\), \({\delta }_{{\mathrm{H}}}^{0}\,<\,0\); Fig. 2d), the highimpact state will always be stable and the lowimpact state unstable. Here we find that, analogous to the regime in Fig. 2c, bistability, limit cycles, and dominance by one strategy can all occur, depending on the relative incentives to follow change and the speed of environmental feedbacks.
The analysis above gives a description of the possible outcomes for a linear twostrategy ecoevolutionary system (for a detailed analysis see “Renewable resource systemlevel analysis” in Supplementary Note 1). The most striking result is that five easily interpretable parameters determine the qualitative properties of the system: four parameters that describe the incentives at the corners of state space, and one parameter describing the relative speed of environmental feedbacks. These parameters give immediate insight into when bistability, cyclic dynamics, mixed equilibria, or dominance by a single strategy can arise in ecoevolutionary games.
The speed of environmental feedbacks plays an important role in determining both the stability characteristics of equilibria and the basins of attraction of equilibria. Figure 4 shows the approximate basins of attraction for two equilibria under environmental dynamics of different speeds. These figures correspond to the region of potential cycles in Fig. 2c.
Our analysis in this section has assumed that payoffs are linear in the state of the environment and the frequencies of the strategies in the population. These assumptions allowed us to characterize all possible outcomes in terms of a few parameters. While these simplifying assumptions may seem to limit the range of applicability, in the remainder of the paper we highlight scientifically and societally relevant cases that fall within this model, as well as examples that extend beyond the linear framework.
Case studies
A large collection of prior studies on environmentstrategy feedbacks, across a range of disciplines, can be be understood as linear ecoevolutionary games. The dynamical properties of these models are all predicted by our analysis of these systems. In this section, we briefly review these models to highlight the broad applicability of our framework and to showcase the diversity of dynamical phenomena that occur in ecoevolutionary games.
Case study 1: coevolution of the environment and decisionmaking
Rand et al.^{15} developed a psychological model of decisionmaking where individuals can either make automatic hardwired decisions, or can make controlled decisions that are flexible and can shape a beneficial state of the environment. Although the motivation of their model is far from ecology, their formulation is a special case of the decaying resource model. Rand et al.^{15} found that under certain circumstances these feedbacks can lead to cyclical dynamics: automatic and controlled agents cycle in abundance as the environment fluctuates in its favorability towards these the two cognitive styles. Rand et al.^{15} motivate their study by noting that controlled decision making is likely to be costly but will allow individuals to choose optimal behavior. Further, they assume that when a population is dominated by controlled agents, then the fitness difference between optimal and suboptimal decisions will decrease because institutions or public goods created by controlled agents will stabilize the environment. This then favors automated agents who choose an option quickly without paying the cost of controlled processing. Rand et al.^{15} introduce a parameter that makes the cost of control frequencydependent, so that when control agents are rare, it is more costly for them to ensure a stable environment.
The minimal model by Rand et al.^{15} is a special case of our decaying resource framework, as we prove in Supplementary Note 4. By mapping their model onto our framework, the dynamical properties are immediately understood in terms of the incentive parameters of our analysis. In particular, we find that the Rand et al.^{15} model falls within Fig. 2b—i.e., there are positive incentives to lead change. Embedding their model as special case of linear ecoevolutionary games allows us to show, for example, that cycles arise due to the frequencydependent cost of being a controlled agent which assures that the model falls within the yellow region of Fig. 2b. Further, we can compute the critical time lag that produces cycles between cognitive styles, coupled with environmental cycles (see “Application to Rand et al. model” in Supplementary Note 4).
Case study 2: grasslegume competition
Evoevolutionary games provide a natural framework for studying competition between grasses and legumes. Many legumes form symbioses with nitrogenfixing bacteria, allowing them to thrive in nitrogenlimited environments. Through time, however, some of the fixed nitrogen becomes available in the soil to nearby plants. In effect, the plant strategy of nitrogen fixation both frees plants from nitrogen limitation, and generates an environmental feedback that increases the availability of nitrogen in the soil. Grasses, on the other hand, do not fix nitrogen. Competition between obligate nitrogenfixing legumes and grasses can be modeled as a special case of our decaying resource framework. Here the environmental state corresponds to the degree to which nitrogen is limiting, and the two strategies correspond to the species present in the system. We can thus write the relative abundance of grasses (the low nitrogen emission strategy) as x and legumes (the high nitrogen emissions strategy) as 1 − x.
We can determine the qualitative dynamics that will arise in grasslegume competition, based on the δ and Δ parameters and the relative timescale of environmental versus strategy dynamics, ϵ.
First, consider the grassdominated state (x = 1), with nitrogen limitation at its greatest (n = 1). In this state, we expect legumes to be able to invade because of the advantage of nitrogen fixation in a lownitrogen environment. Thus we expect \({\Delta }_{{\mathrm{L}}}^{1}\ > \ 0\). Similarly, we expect \({\delta }_{{\mathrm{H}}}^{0}\ > \ 0\) because in an environment where nitrogen is not limiting (n = 0) that is dominated by nitrogen fixing legumes (x = 0), nonfixing grasses will be able to invade since they do not pay the cost of nitrogen fixation, but can reap the benefit of a nutrientrich environment.
Therefore the dynamics will fall somewhere in Fig. 2b. Legumes are likely to have a competitive advantage in a low nitrogen environment regardless of their relative abundance, thus we expect \({\Delta }_{{\mathrm{H}}}^{1}\ > \ 0\). The same holds for grasses, given nitrogen is not limiting, so that \({\delta }_{{\mathrm{L}}}^{0}\ > \ 0\). Finally, in species competition, it is typically more difficult for the first individual to successfully invade and establish than it is for an established species to spread and increase in abundance^{33}. This implies that we expect \({\Delta }_{{\mathrm{L}}}^{1}\ <\ {\Delta }_{{\mathrm{H}}}^{1}\) and \({\delta }_{{\mathrm{H}}}^{0}\ <\ {\delta }_{{\mathrm{L}}}^{0}\).
Because \({\Delta }_{{\mathrm{L}}}^{1}{\delta }_{{\mathrm{H}}}^{0}\ <\ {\Delta }_{{\mathrm{H}}}^{1}{\delta }_{{\mathrm{L}}}^{0}\) holds we expect grasslegume systems to be susceptible to cyclic dynamics. However, cyclic dynamics still require that the timescale of the feedback between the abundance of legumes and nitrogen availability is sufficiently slow. This too is reasonable in nature, because nitrogen is a valuable resource and a legume will tend to limit the rate at which fixed nitrogen leaks into the environment.
More broadly, feedback between plants and soil microbial communities, including through nitrogen fixation, can generate similar ecoevolutionary dynamics, with consequences for the maintenance of diversity^{34,35}. A related feedback can occur between available soil nitrogen and nitrogenfixation strategies of the rhizobium bacteria. Theoretical^{36} and empirical^{37} findings indicate that high nitrogen availability favors rhizobia that fix less nitrogen, while low nitrogen favors strains that fix more. This suggests that cycling may also occur in coupled nitrogenstrain frequency dynamics.
Case study 3: commonpool resource harvesting
Next, we consider a classic example of linked strategy and environmental dynamics: commonpool resource harvesting. There is extensive literature on commonpool harvesting, which forms the basis for bioeconomics^{38}. Ecoevolutionary game theory provides a natural framework to situate commonpool resource models—because strategic interactions depend upon, and conversely influence, the abundance of the commonpool resource.
It seems plausible that even the simplest form of commonpool resource harvesting will lead to cyclic dynamics: as the biomass of resource stock collapses and overshoots, harvesters respond by reducing effort, until the resource rebounds and higheffort strategies are again profitable. Nevertheless, our analysis shows that such cycles will never occur without additional complications. We formulate commonpool resource dynamics by assuming individuals can harvest with either high, e_{H}, or low, e_{L}, effort. We let the evolutionary process on strategy frequencies be governed by a profit function,
where q is the harvest efficiency and w is the marginal cost of harvest effort. This profit function maps the resource level, η, and harvest effort, e_{i}, into fitness. As in the general renewable resource model, we assume that η is governed by logistic growth, and that the harvest rate is proportional to η and effort (e_{L}, e_{H}). Since π(e_{i}, η) is linear, this model is a special case of the renewable resource model we have exhaustively analyzed.
Transforming the resource level into a normalized environmental metric, we can construct a payoff matrix, Π (n) that maps the commonpool resource harvesting model onto our framework of ecoevolutionary games. The resulting parameter values satisfy \({\Delta }_{{\mathrm{L}}}^{1}={\Delta }_{{\mathrm{H}}}^{1}\ > \ 0\) and \({\delta }_{{\mathrm{H}}}^{0}\,=\,{\delta }_{{\mathrm{L}}}^{0}\ > \ 0\) when there are positive profits at the environmental state resulting from pure lowimpact strategists and negative profits at the environment resulting from pure highimpact strategists. Under these profit assumptions, the commonpool resource system falls at the boundary of the blue and yellow regions of Fig. 2b—i.e., incentives to lead and follow change are all positive, but the there is no (positive) value of ϵ that produces cycles. And so the only possible outcome of this commonpool system is a stable mix of lowimpact and highimpact strategists (see Supplementary Note 5 for detailed analysis and for other possible scenarios). However, since the system falls on the boundary of a parameter region that permits cycles, small changes to the system may induce cyclic dynamics.
We considered two extensions, introducing market pricing (where p decreases as harvest quantity increases) or introducing harvesting efficiency gains (where q increases as a harvest strategy becomes more common). Both extensions fall outside the scope of the linear ecoevolutionary games analyzed in this paper. Market pricing induces nonlinearity in the payoffs that harvesters receive, and frequency dependent harvest efficiency alters environmental dynamics outside of the renewing and decaying resource models considered above.
Under commonpool resource harvesting with market pricing, while the range of dynamical outcomes increases (see Supplementary Fig. 2), we do not find cyclic dynamics (see “Market pricing analysis” in Supplementary Note 6). This result occurs because market pricing effects all harvesters in the same way, and thus does not provide the extra incentive for being a follower of strategy change that can cause cycles.
Harvest efficiency may depend on strategy frequency if each strategy requires specialized skills and labor. As a strategy increases in frequency, increased opportunities for social learning may lead to increased proficiency and efficiency gains^{39,40}. This effect alters both payoffs to individuals and the dynamics of the resource (see Supplementary Note 7 for analysis). As a result of these intertwined consequences, we find instances of nonmonotonicity—where increasing the growth rate of the resource, for example, can first destabilize and then stabilize an interior equilibrium (see Supplementary Fig. 4). Despite this added complexity, the intuition developed from our general framework still applies. In particular, our analysis of commonpool resource harvesting as a linear ecoevolution game showed that increasing the values of either \({\Delta }_{{\mathrm{H}}}^{1}\) or \({\delta }_{{\mathrm{L}}}^{0}\) could cause cycles, by moving the system into the yellow region in Fig. 2b. Frequencydependent harvest efficiency plays a similar role to increasing \({\Delta }_{{\mathrm{H}}}^{1}\) or \({\delta }_{{\mathrm{L}}}^{0}\), by making it more profitable to switch to a highfrequency strategy due to increased efficiency, and helps explain the cyclical dynamics that arise in this case.
Case study 4: relationship to Weitz et al.^{24}
Weitz et al.^{24} developed a model of ecoevolutionary games where the environment is governed by a tipping point: it is driven to one of two extreme states in direct response to the strategies employed. Weitz et al.^{24} found that the existence of persistent oscillations (specifically in the region of Fig. 2b) does not depend on the relative timescale of strategic versus environmental dynamics; whereas by contrast we find that the dynamical behavior depends critically on the relative timescale, ϵ. To understand this discrepancy between models with intrinsic environmental dynamics, which are the focus of our paper, and the model of Weitz et al.^{24}, we analyzed a generalization of their model.
There is a simple biological motivation for environmental dynamics governed by tipping points. Imagine that the environment consists of particles (e.g., individual fish) that change state depending on the frequency of the strategies being employed. For example, individual fish might switch between being active versus hiding, depending on the feeding behavior of their predators. Weitz et al.^{24} model the case when all particles have the same threshold value. More generally, one can imagine that the particles are heterogeneous, with a distribution of thresholds. Denoting the cumulative distribution function of thresholds by F(x), we can write the dynamical system as
where n is the fraction of the environment that is in the state associated with strategy 1, x is the frequency of strategy 1, and F^{−1}(n) is the inverse of the cumulative distribution function of environmental tipping points.
For example, with a uniform distribution of tipping points centered at μ with length of a, the environmental dynamics are expressed as
In the limit a → 0 this model coincides precisely with the model of Weitz et al.^{24} (by setting the tipping point μ = 1/(1 + θ), and the timescale \(\epsilon =(1+\theta )\tilde{\epsilon }\)), and it contains no intrinsic environmental dynamics. In this limit Weitz et al.^{24} found that the existence of persistent oscillations does not depend on the relative timescales of strategic versus environmental changes, ϵ.
In general, when the variability of tipping point goes to zero, Eq. 20 coincides with the model of Weitz et al.^{24}. But aside from this limiting case, these environmental dynamics include an intrinsic component and, in all such cases, the existence of oscillatory solutions depends on the timescale separation between strategic and environmental change. These results hold for both the uniform distribution of tipping points (Eq. 21), as well as for distributions that produce complex nonlinear dynamics. For example, under a truncated normal distribution of environmental tipping points, as the variance decreases the model again approximates Weitz et al.^{24} and, despite this nonlinearity, the general patterns of stability remain strikingly similar to our analysis of environments with intrinsic dynamics (Supplementary Note 8).
Discussion
We have developed a framework to study linked environmental and evolutionary game dynamics. Our analysis provides a systematic account of dynamical outcomes for an arbitrary game with linear payoff structures, with environments that either intrinsically grow or decay. We have also analyzed examples with nonlinear payoffs, or where the environment exhibits tipping points. This framework applies to any gametheoretic interaction where the strategies that individuals employ impact the environment through time, and the state of the environment conversely influences the payoffs of the game. Such feedbacks are common in socialecological systems, evolutionary ecology, and psychology.
Despite the presence of environmental feedbacks in many systems across disciplines, analysis of feedback has previously been treated on casebycase basis. We propose ecoevolutionary games as a synthetic framework for systematic analysis of strategic interactions with environmental feedback, allowing insight into dynamical commonalities across disparate systems. An explicit account of environmental feedbacks reveals added complexity and nuance^{11}. For example, Sigdel et al.^{25} have shown social norms can either cause or prevent cyclic dynamics depending on the strength of environmental feedback. By relating their model to the space of models we analyze, we can provide a synthetic understanding for why these dynamical features arise (see Supplementary Note 9). We have shown that many prior studies of strategic interactions with environmental feedbacks occur as special cases of our general framework, with an explicit mapping to the parameters of our framework. Furthermore, we have shown that the intuition developed in the simple models considered here can extend to more complex, nonlinear models as well.
Perhaps the most striking result is that the rich dynamical outcomes that arise in ecoevolutionary games can be understood in terms of five intuitive parameters (see Fig. 2): the incentives to lead or follow strategy change under rich or poor environmental conditions, and the relative timescale of strategic versus environmental dynamics.
We have also generalized one of the first models of ecoevolutionary games introduced by Weitz et al.^{24}. The main difference compared to our analysis is that Weitz et al.^{24} studied a feedback caused by a single environmental tipping point, in the absence of any intrinsic environmental dynamics. In many systems of ecological and social relevance the environmental variables that affect payoffs have intrinsic dynamics, meaning that they regenerate (e.g., a population) or decay (e.g., pollution) when left by themselves. Whereas Weitz et al.^{24} found heteroclinic cycles independent of the timescale separation between environmental and strategy dynamics, we find that the existence of limit cycles depends critically on the degree of timescale separation, for renewing and decaying environments as well as for environments with a distribution of tipping points.
Prior work has shown that coupling strategies and the environment can induce persistent cycles even when neither the intrinsic environmental or intrinsic strategic dynamics exhibit cycles on their own^{25}, and our paper reaffirms this. At the same time, it is well known that many ecological systems can produce complex environmental dynamics even without feedback from individual actions^{41}. Intrinsic complexity can result from multiple interacting environmental factors or a single environmental factor that is subject to agestructured or stagestructured population dynamics. Sigdel et al.^{30} analyze a case where the environment has Allee effects. Allee effects can create hysteresis and critical transitions from which environmental recovery is unlikely.
Structured interactions arising from environmental heterogeneity, network structure, and spatial structure will add further complexity. Spatial models of environmental feedbacks indicate that feedbacks can lead to correlations between strategies and their environment, with consequences for the stability of cooperation^{42,43}. Spatially structured interactions, and more generally, network structured interactions among individuals can fundamentally alter predictions about the strategies that will be successful in a system, but it is not well known what effects such structured interactions will have on a system where strategies and the environment are coupled. Recent work suggests that it could lead to environmental heterogeneity and the simultaneous support of many strategies with differing environmental and strategic consequences^{44}.
We focused most of our analysis on linear twostrategy ecoevolutionary games. And yet nonlinearities in the payoff structure of strategic interactions have important effects. We have shown that introducing nonlinear payoffs, through market pricing, in a commonpool resource harvesting model generates a broader class of qualitative outcomes, and the possibilities under a general nonlinear payoff functions are likely even broader. Further, as the size of the strategy space increases, so does the dimensionality of the dynamical system. In these cases, a complete description of the system is not possible with only the five parameters we have considered. Nonetheless, the incentives to lead and follow strategy change may still provide valuable insights into higherdimensional and nonlinear systems. If new strategies emerge then the game itself can evolve^{17}, leading to novel interactions with the environment. These extensions make it clear that ecoevolutionary games can be seen as complex adaptive systems with emergent properties that are not easily predicted from the environmental dynamics or the evolutionary game structure alone.
In human societies the institutions structuring social interactions can be seen as part of the environment that coevolve with strategic behaviors^{12,45,46,47}. We can represent the institutional environment in our framework by constructing an institutional metric that modulates the game being played. Institutions may also interact with an explicit resource and affect the incentives individuals face in a given environment, or how strategies affect environmental dynamics. For example, the success of international environmental agreements to achieve environmental stewardship depends on how individuals and nations respond to their incentives under a changing environment. In the context of pollution control and climate change mitigation, action is an intertemporal public good, with benefits of individual action accruing to a large population and in the future^{48}. Policies of individual nations are likely to have feedbacks on the international institutional setting, the environment, and the choices that individuals make.
Environmental feedbacks in strategic interactions are the norm, not the exception. An explicit account of these feedbacks reveals commonalities among many societally relevant systems, ranging from the psychology of decisionmaking to species interactions and climatechange action, and alters predictions about expected outcomes in such systems. Incorporating strategyenvironment feedbacks into evolutionary game theory is paramount in future studies.
Methods
Analysis
Detailed analyses supporting our key findings, illustrated in Fig. 2 and described in the section “Dynamical regimes in ecoevolutionary games”, can be found in Supplementary Notes 1–3. Details of the application of our analysis to the case studies explored in the main text can be found in Supplementary Notes 4–8.
Simulations
Figures 3 and 4 show simulations that illustrate the importance of the relative timescale of strategic versus environmental dynamics on the longrun outcomes of ecoevolutionary games. These simulations can be replicated using the code referenced in the “Code availability” section.
Reporting summary
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
Data availability
All data shown in figures can be produced from the simulation code which is made freely available at https://github.com/atilman/EcoEvoGamesCode.
Code availability
Python code used to generate figures is available at https://github.com/atilman/EcoEvoGamesCode.
References
 1.
Nash, J. F. Equilibrium points in nperson games. Proc. Natl Acad. Sci. 36, 48–49 (1950).
 2.
Smith, J. M. Evolution and the Theory of Games (Cambridge University Press, Cambridge, 1982).
 3.
Sethi, R. & Somanathan, E. The evolution of social norms in common property resource use. Am. Econ. Rev. 86, 766–788 (1996).
 4.
Lacitignola, D., Petrosillo, I., Cataldi, M. & Zurlini, G. Modelling socioecological tourismbased systems for sustainability. Ecol. Model. 206, 191–204 (2007).
 5.
Tavoni, A., Schlüter, M. & Levin, S. The survival of the conformist: social pressure and renewable resource management. J. Theor. Biol. 299, 152–161 (2012).
 6.
Levin, S. et al. Socialecological systems as complex adaptive systems: modeling and policy implications. Environ. Dev. Econ. 18, 111–132 (2013).
 7.
Innes, C., Anand, M. & Bauch, C. T. The impact of human–environment interactions on the stability of forestgrassland mosaic ecosystems. Sci. Rep. 3, 2689 (2013).
 8.
Richter, A. & Dakos, V. Profit fluctuations signal eroding resilience of natural resources. Ecol. Econ. 117, 12–21 (2015).
 9.
Tilman, A. R., Levin, S. & Watson, J. R. Revenuesharing clubs provide economic insurance and incentives for sustainability in commonpool resource systems. J. Theor. Biol. 454, 205–214 (2018).
 10.
Mullon, C., Keller, L. & Lehmann, L. Social polymorphism is favoured by the coevolution of dispersal with social behaviour. Nat. Ecol. Evol. 2, 132 (2018).
 11.
Estrela, S. et al. Environmentally mediated social dilemmas. Trends Ecol. Evol. 34, 6–18 (2018).
 12.
Tilman, A. R., Watson, J. R. & Levin, S. Maintaining cooperation in socialecological systems: effective bottomup management often requires suboptimal resource use. Theor. Ecol. 10, 155–165 (2017).
 13.
Menge, D. N. L., Levin, S. A. & Hedin, L. O. Evolutionary tradeoffs can select against nitrogen fixation and thereby maintain nitrogen limitation. Proc. Natl Acad. Sci. 105, 1573–1578 (2008).
 14.
Grman, E., Robinson, T. M. & Klausmeier, C. A. Ecological specialization and trade affect the outcome of negotiations in mutualism. Am. Nat. 179, 567–581 (2012).
 15.
Rand, D. G., Tomlin, D., Bear, A., Ludvig, E. A. & Cohen, J. D. Cyclical population dynamics of automatic versus controlled processing: an evolutionary pendulum. Psychol. Rev. 124, 626 (2017).
 16.
Peck, J. R. & Feldman, M. W. The evolution of helping behavior in large, randomly mixed populations. Am. Nat. 127, 209–221 (1986).
 17.
Worden, L. & Levin, S. A. Evolutionary escape from the prisoner’s dilemma. J. Theor. Biol. 245, 411–422 (2007).
 18.
Cortez, M. H. & Ellner, S. P. Understanding rapid evolution in predatorprey interactions using the theory of fastslow dynamical systems. Am. Nat. 176, E109–E127 (2010).
 19.
Akçay, E. & Roughgarden, J. The evolution of payoff matrices: providing incentives to cooperate. Proc. R. Soc. B 278, 2198–2206 (2011).
 20.
Huang, W., Haubold, B., Hauert, C. & Traulsen, A. Emergence of stable polymorphisms driven by evolutionary games between mutants. Nat. Commun. 3, 919 (2012).
 21.
Lade, S. J., Tavoni, A., Levin, S. A. & Schlüter, M. Regime shifts in a socialecological system. Theor. Ecol. 6, 359–372 (2013).
 22.
Stewart, A. J. & Plotkin, J. B. Collapse of cooperation in evolving games. Proc. Natl Acad. Sci. 111, 17558–17563 (2014).
 23.
Lee, J. H., Kakinuma, K., Okuro, T. & Iwasa, Y. Coupled social and ecological dynamics of herders in mongolian rangelands. Ecol. Econ. 114, 208–217 (2015).
 24.
Weitz, J. S., Eksin, C., Paarporn, K., Brown, S. P. & Ratcliff, W. C. An oscillating tragedy of the commons in replicator dynamics with gameenvironment feedback. Proc. Natl Acad. Sci. 113, E7518–E7525 (2016).
 25.
Sigdel, R. P., Anand, M. & Bauch, C. T. Competition between injunctive social norms and conservation priorities gives rise to complex dynamics in a model of forest growth and opinion dynamics. J. Theor. Biol. 432, 132–140 (2017).
 26.
Akçay, E. Collapse and rescue of cooperation in evolving dynamic networks. Nat. Commun. 9, 2692 (2018).
 27.
Cortez, M., Patel, S. & Schreiber, S. Destabilizing evolutionary and ecoevolutionary feedbacks drive ecoevo cycles in empirical systems. Preprint at https://doi.org/10.1101/488759v3 (2018).
 28.
Hilbe, C., Šimsa, Š., Chatterjee, K. & Nowak, M. A. Evolution of cooperation in stochastic games. Nature 559, 246 (2018).
 29.
Patel, S., Cortez, M. H. & Schreiber, S. J. Partitioning the effects of ecoevolutionary feedbacks on community stability. Am. Nat. 191, 381–394 (2018).
 30.
Sigdel, R., Anand, M. & Bauch, C. T. Convergence of socioecological dynamics in disparate ecological systems under strong coupling to human social systems. Theor. Ecol. 12, 285–296 (2019).
 31.
Shao, Y., Wang, X. & Fu, F. Evolutionary dynamics of group cooperation with asymmetrical environmental feedback. EPL 126, 40005 (2019).
 32.
Taylor, P. D. & Jonker, L. B. Evolutionary stable strategies and game dynamics. Math. Biosci. 40, 145–156 (1978).
 33.
Tilman, D. Niche tradeoffs, neutrality, and community structure: a stochastic theory of resource competition, invasion, and community assembly. Proc. Natl Acad. Sci. 101, 10854–10861 (2004).
 34.
Bever, J. D., Westover, K. M. & Antonovics, J. Incorporating the soil community into plant population dynamics: The utility of the feedback approach. J. Ecol. 85, 561–573 (1997).
 35.
Mangan, S. A. et al. Negative plantsoil feedback predicts treespecies relative abundance in a tropical forest. Nature 466, 752 (2010).
 36.
Akçay, E. & Simms, E. L. Negotiation, sanctions, and context dependency in the legumeRhizobium mutualism. Am. Nat. 178, 1–14 (2011).
 37.
Weese, D. J., Heath, K. D., Dentinger, B. T. & Lau, J. A. Longterm nitrogen addition causes the evolution of lesscooperative mutualists. Evolution 69, 631–642 (2015).
 38.
Clark, C. Mathematical Bioeconomics (Wiley, New York, 1990).
 39.
Henrich, J. Demography and cultural evolution: how adaptive cultural processes can produce maladaptive lossesthe tasmanian case. Am. Antiq. 69, 197–214 (2004).
 40.
Smolla, M. & Akçay, E. Cultural selection shapes network structure. Sci. Adv. 5, eaaw0609 (2019).
 41.
Turchin, P. Complex Population Dynamics: A Theoretical/Empirical Synthesis, vol. 35 (Princeton University Press, Princeton, 2003).
 42.
Pepper, J. W. & Smuts, B. B. A mechanism for the evolution of altruism among nonkin: positive assortment through environmental feedback. Am. Nat. 160, 205–213 (2002).
 43.
Hauert, C., Saade, C. & McAvoy, A. Asymmetric evolutionary games with environmental feedback. J. Theor. Biol. 462, 347–360 (2018).
 44.
Lin, Y.H. & Weitz, J. S. Spatial interactions and oscillatory tragedies of the commons. Phys. Rev. Lett. 122, 148102 (2019).
 45.
Ostrom, E. Governing the Commons (Cambridge University Press, Cambridge, 1990).
 46.
North, D. C. Institutions, Institutional Change and Economic Performance (Cambridge University Press, Cambridge, 1990).
 47.
Bowles, S., Choi, J.k. & Hopfensitz, A. The coevolution of individual behaviors and social institutions. J. Theor. Biol. 223, 135–147 (2003).
 48.
Hauser, O. P., Rand, D. G., Peysakhovich, A. & Nowak, M. A. Cooperating with the future. Nature 511, 220 (2014).
Author information
Affiliations
Contributions
A.R.T., J.B.P., and E.A. designed the research, performed the research and wrote the paper.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Communications thanks the anonymous reviewers for their contribution to the peer review of this work. Peer reviewer reports are available.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Tilman, A.R., Plotkin, J.B. & Akçay, E. Evolutionary games with environmental feedbacks. Nat Commun 11, 915 (2020). https://doi.org/10.1038/s41467020145316
Received:
Accepted:
Published:
Further reading

Cyclical behavior of evolutionary dynamics in coordination games with changing payoffs
International Journal of Game Theory (2021)

Stochastic Evolutionary Game Analysis Between Special Committees and CEO: Incentive and Supervision
Dynamic Games and Applications (2021)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.