Evolutionary branching—resident-mutant coexistence under disruptive selection—is one of the main contributions of Adaptive Dynamics (AD), the mathematical framework introduced by S.A.H. Geritz, J.A.J. Metz and coauthors to model the long-term evolution of coevolving multi-species communities. It has been shown to be the basic mechanism for sympatric and parapatric speciation, despite the essential asexual nature of AD. After 20 years from its introduction, we unfold the transition from evolutionary stability (ESS) to branching, along with gradual change in environmental, control, or exploitation parameters. The transition is a catastrophic evolutionary shift, the branching dynamics driving the system to a nonlocal evolutionary attractor that is viable before the transition, but unreachable from the ESS. Weak evolutionary stability hence qualifies as an early-warning signal for branching and a testable measure of the community’s resilience against biodiversity. We clarify a controversial theoretical question about the smoothness of the mutant invasion fitness at incipient branching. While a supposed nonsmoothness at third order long prevented the analysis of the ESS-branching transition, we argue that smoothness is generally expected and derive a local canonical model in terms of the geometry of the invasion fitness before branching. Any generic AD model undergoing the transition qualitatively behaves like our canonical model.
Twenty years ago, S.A.H. Geritz, J.A.J. Metz and coauthors1,2,3 introduced a mathematical framework—referred to as Adaptive Dynamics (AD)—for modeling long-term evolutionary dynamics, with special emphasis on the generation of diversity through evolutionary branching. AD has been successfully developed and applied in and outside biology (see ref. 4 and refs therein) as well as debated, mainly due to its asexual nature (see vol. 18 of the J. Evol. Bio.).
Evolutionary branching takes off when a resident and a similar mutant type coexist in the same environment and natural selection is disruptive, i.e., favors outer rather than intermediate phenotypes. In the restricted (but classical) formulation in which resident individuals are characterized by the same value x of a one-dimensional strategy, Geritz, Metz et al.1,2,3 derived explicit conditions for evolutionary branching in terms of the invasion fitness5—the exponential rate of growth sx(y) initially shown by a mutant strategy y appeared when the resident is at its ecological regime.
Subsequently, analogous conditions have been derived for models including sexual characters6,7, spatial distribution8 and multi-dimensional phenotypes9, establishing evolutionary branching as the prerequisite step for sympatric and parapatric speciation10.
Evolutionary branching requires, first, the resident strategy x to be in the vicinity of a convergence stable singular strategy x*—a strategy making neutral the selection pressure measured by the fitness gradient ∂ysx(y)|y=x, i.e.,
Second, there should be pairs (x1, x2) close to the singular point (x*, x*) for which a resident x = x1 and a mutant y = x2 can coexist. For this, a sufficient condition is a negative fitness cross-derivative, i.e.,
that implies the existence, locally to (x*, x*), of a conical region of coexistence in the strategy plane (Fig. 1). Geritz, Metz et al.1,2,3 showed that mutual invasibility occurs in the region, i.e., that (2) implies the invasion fitness of strategy x2 against the resident x1, and vice versa, , to be both positive (and vanishing each on one of the region boundaries). The existence of a unique ecological attractor of coexistence was shown more recently13,14,15 for the class of unstructured ecological models—no individual distinction w.r.t. age, size, location, etc.—under stationary coexistence. (See Fig. 1c, where is the equilibrium density of the resident before mutant invasion and and are the equilibrium densities of coexistence).
Third, selection must be disruptive, a sufficient condition for which being
Condition (3) was obtained1,2,3 based on the twice differentiability of the dimorphic fitness —the invasion fitness of a mutant y invading an environment set by the two residents x1 and x2 at their ecological regime—and by expanding it around (x1, x2, y) = (x*, x*, x*) (see Appx. 1 in ref. 3 for details). Exploiting the consistency relations
C1: , the link between the dimorphic and monomorphic fitness functions at the singular point,
C2: , the order irrelevance of the two residents, and
C3: (a) and (b), the selective neutrality of the two residents,
It says that the selection pressures on strategies x1 and x2, as measured by the fitness gradients
are opposite for (x1, x2) in the region of coexistence locally to (x*, x*), meaning that dimorphic evolution points away or toward (x*, x*) if (see Fig. 1f,e, respectively).
The smoothness of the dimorphic fitness at the singular point (x1, x2) = (x*, x*) is a controversial open problem of AD. In ref. 1 (Sect. 6.3.2), the twice differentiability is shown geometrically (see Methods for an algebraic proof) and the authors argue that higher-order derivatives may fail to exist at the singular point. The source of the expected nonsmoothness is the ecological attractor of coexistence being undefined when x1 = x2. Think, e.g., of the equilibrium densities and in Fig. 1. They are evidently discontinuous at (x*, x*), respectively being equal to zero and along the extinction boundary 1 (blue) and to and zero along the extinction boundary 2 (red). The nonsmoothness of the dimorphic fitness arising at 3rd order remained a mystery of AD that long prevented the analysis of the ESS-branching transition18.
The nonsmoothness conjecture was recently disproved (exploiting a new structural ecological property19) for the class of unstructured ecological models under stationary coexistence. We have indeed shown20 the smoothness of the dimorphic fitness up to 3rd order. Moreover, our methodology is general up to any order and there is thus no reason to expect nonsmoothness to arise at some higher order.
Here we lift the above result to the generality of any class of ecological models, based on the newly discovered smoothness of the dimorphic fitness. Only assuming a three-times differentiable dimorphic fitness, we show that the consistency relations C1–C3 (with C1 considered along the extinction boundaries delimiting the coexistence region, see Methods) determine the expansion of the dimorphic fitness up to 3rd order and the result is the same as obtained in ref. 20. That is, the 1st order is null, the 2nd coincides with (4) and the 3rd is
Note that the expansion is given in terms of the monomorphic fitness derivatives ( and are the involved third derivatives)—that are determined by the ecological dynamics before branching5—in contrast to what preliminarily expected based on the analysis of Lotka-Volterra models21. Unfortunately, the consistency arguments offered by relations C1–C3 do not determine the fitness expansion at higher orders, so whether the local approximation of the dimorphic fitness is fully controlled by the geometry of the monomorphic fitness remains an open question. Any answer necessarily requires specifying the class of underlying ecological models, so that one can proceed with the direct computation of the fitness expansion, as in ref. 20.
Our analysis allows to go beyond 2nd order in the analysis of the branching dynamics, thus solving the 20-yrs-lasting impasse. In particular, the analysis at 3rd order is of fundamental importance, because necessary to unfold the mechanisms underlying the ESS-branching transition ( under the coexistence condition (2), see Fig. 2), close to which the branching dynamics is dominated by the 3rd-order terms in the approximation of the dimorphic fitness. Note that close to the transition is the leading 3rd-order coefficient in (6), so we perform our analysis assuming the genericity condition
The analyses at 4th and higher orders will be possible only under specific ecological assumptions and are interesting for the understanding of degenerate branching scenarios22, where some of the fitness derivatives vanish at the singularity due to model-specific properties, like symmetries in the phenotypic dependence of demographic parameters around optimal values.
Based on the 3rd-order expansion (4, 6), we derive the following canonical model for the ESS-branching transition:
The evolutionary dynamics of model (8) are depicted (under ) in Fig. 2 in the coexistence region delimited by the extinction boundaries η1(Δx1, Δx2) = 0 (blue) and η2(Δx1, Δx2) = 0 (red). Within the coexistence region, represents the birth output of populations i at the ecological regime and si(Δx1, Δx2)(Δxi − Δxj), i ≠ j, approximates the fitness gradient (5) (see Methods). The (black) trajectories describe the continuous dynamics obtained in the limit of infinitesimally small mutational steps11,12 ( standing for the time-derivative of xi on a suitable evolutionary time scale), but the fitness approximation and the direction of dimorphic evolution describe as well the evolution driven by finitely small mutations23,24. The dashed (blue and red) lines solve s1(Δx1, Δx2) = 0 and s2(Δx1, Δx2) = 0 and therefore approximate the (x1 and x2) nullclines of the evolutionary dynamics (the lines on which selection is neutral in strategy x1 and x2, respectively, so that evolution points vertically and horizontally). The nullclines and the extinction boundaries (the latter also behaving as nullclines with same color code) are known to connect at special points1,25—the boundary equilibria with one strategy at the singularity x*, the other being absent (connections with different colors, half-filled points) and the points where the absent strategy is (potentially) singular (same color connections, occurring at the horizontal and vertical extremal points of the extinction boundaries). Model (8) shows such connections while approaching the ESS-branching transition.
The unfolding of Fig. 2 has been observed in previous analyses of specific models24,25,26,27 and conjectured to be general based on the above described properties of the evolutionary nullclines28,29. Our contribution at last gives the full theoretical support. We have shown that any single or multi-species community undergoing the ESS-branching transition due to gradual changes in, e.g., climate, nutrients, habitat fragmentation, or biotic control and exploitation qualitatively behaves like model (8) near the singular point (x*, x*).
Figure 2 clearly shows the mechanisms underlying the ESS-branching transition. Restricting the attention to the coexistence region above the diagonal (since relation C2 implies the symmetry w.r.t. the diagonal), the boundary equilibria (half-filled) at x1 = x* (left) and at x2 = x* (right) behave as saddles for the dimorphic evolutionary dynamics, attracting along one trajectory—the stable manifold of the saddle—and repelling for all other nearby initial conditions. In the ESS case (left), the saddle’s stable manifold separates the initial conditions leading to the ESS—a dimorphic phase up to the extinction of one of the two populations followed by the monomorphic convergence (represented on the diagonal) to the ESS—from those leading away, typically to a nonlocal evolutionary attractor not involved in the transition25. In the branching case (right), the initial conditions near the singular point (x*, x*) all lead away. Approaching the transition (left-to-right) along with gradual changes in the model parameters, the boundary saddle approaches the ESS, so do the initial conditions leading away. Note that branching is possible at the transition too (for mutant strategy x2 larger than x*, central panel; branching at the transition symmetrically requires x2 < x* under , not shown, see Methods).
In real systems, where mutational steps are small but finite23,24, the monomorphic dynamics converging to the singular strategy x* eventually jump into the dimorphic region. In an ESS case far from the transition (see Fig. 2, left, at a scale at which mutations are sufficiently small), the dimorphic dynamics jump back monomorphic a few steps later, while the distance to the ESS keeps contracting (see ref. 1, Sect. 3.2.4). However, close to the transition, the initial conditions that trigger the branching dynamics become feasible (see again Fig. 2, left, at a scale at which mutations are large enough to reach initial conditions above the saddle’s stable manifold). Branching is therefore possible before the transition actually occurs and is properly discussed in term of finite mutations—since infinitesimal mutations cannot reach the coexistence region away from the singular strategy x* (see Methods for further detail). In the branching case (Fig. 2, right), the dimorphic dynamics eventually lead away.
The new evolutionary attractor reached after branching is already viable before the transition25 and this shows that the environment is ready to host populations 1 and 2 before branching becomes feasible. Invasion by alien phenotypes could therefore anticipate the endogenous diversity that branching brings about. Whether the new attractor is dimorphic or evolution steps back monomorphic discriminates long-term from temporary diversity. Repeated branching and extinction30,31 could also be triggered by the ESS-branching transition, showing a peculiar case in which the new evolutionary attractor is actually involved in the transition.
The ESS-branching transition is irreversible—a catastrophic evolutionary shift32—in the sense that a small change in a model parameter causes an evolutionary transient toward a nonlocal attractor and once the transient is triggered it can not be reversed by counteracting the parameter perturbation. Weak evolutionary stability—quantified by a small negative and possibly testable33 by estimating the invasion fitness of artificially introduced similar strategies—hence qualifies as an early-warning signal for branching and a measure of the community’s resilience against biodiversity.
As a catastrophic evolutionary shift, the ESS-branching transition bridges the meso and macro scales of evolution28. The short periods of rapid morphological change observed by paleontologists in the fossil record, along with gradual environmental changes on the macro scale—the so-called punctuated equilibria34—could correspond to the branching dynamics triggered by the transition and developing on the meso scale.
This work fully explains one of the most important “phase transition” in Nature.
We consider two similar populations, with densities n1 and n2 and one-dimensional strategies x1 and x2 ≈ x1. In the monomorphic phase populations 1 and 2 are resident and mutant, respectively, whereas they are both residents in the dimorphic phase. We do not specify a particular class of ecological models, but only assume that the resulting monomorphic and dimorphic invasion fitnesses are smooth (only differentiability up to 3rd order is actually required). In particular for the dimorphic fitness , smoothness at (x1, x2, y) = (x*, x*, x*) means that a polynomial local expansion in , i = 1, 2, Δy := y − x* holds good in the resident-mutant coexistence region—the error of a truncated expansion of order k is when (Δx1, Δx2) vanish along a path in the coexistence region.
The above smoothness assumption is currently proved for the class of unstructured ecological models under stationary coexistence20, though we expect it to hold with wide generality. Specifically, smoothness is generically expected radially from the singular point (x*, x*) in the strategy plane (x1, x2)1. Then, showing the smoothness of the dimorphic fitness at (x1, x2, y) = (x*, x*, x*) reduces to showing that the directional expansion along rays (x1, x2) = (x* + ε cos θ, x* + ε sin θ) at given θ and around y = x* is polynomial in the direction components (cos θ, sin θ). However, without specific assumptions on the underlying ecological model, this is possible only up to order 2 (see SI, Sect. 1).
Using a 3rd-order expansion of the dimorphic fitness and imposing the consistency relations C1–C3, with C1 replaced by
along the extinction boundary 2 on which only strategy x1 is present, we derive the expansion (4, 6) under the coexistence condition (2) (note that imposing the monomorphic-dimorphic link on the extinction boundary 1 is redundant due to the diagonal symmetry between the two boundaries and property C2, see SI, Sects. 2 and 3).
Specifically, to impose C1′, we use the polar coordinates (ε, θ) and ε-parameterize the extinction boundary 2 as θ = θ2(ε), θ2(ε) being the function that gives the angle θ of the boundary point at distance ε from the singular point (x*, x*). Then, C1′ becomes
to be imposed together with its (ε, Δy)-derivatives at (ε, Δy) = (0, 0) up to order 3. This involves the angle θ2(0)—the tangent direction to the extinction boundary 2 at (x*, x*)—and the first derivative θ2′(0)—the local curvature of the boundary (whether θ increases or decreases while moving away from (x*, x*), see Fig. 1e,f). The two quantities are determined by the second and third derivatives of the monomorphic fitness. They are obtained by imposing the second and third ε-derivatives of the boundary definition
(the first derivative being uninformative).
Unfortunately, the above procedure does not apply at order 4, as the conditions imposed by relations C1′–C3 are less than the number of coefficients in the expansion of the dimorphic fitness. The missing conditions must then come from specific assumptions to be made at the ecological level.
The derivation and the analysis of the canonical model (8) are detailed in the Supplementary Information (Sects. 4 and 5).
We illustrate the developed body of theory on a well-known example: a single species AD Lotka-Volterra model of asymmetric competition26. The resident-mutant ecological model reads:
where here stands for the time-derivative of ni on the ecological time scale, i = 1, 2 and the strategy x scales (from −∞ to +∞) with competitive ability. The intrinsic growth rate ρ and the competition function α are strategy-dependent, with Gaussian , σ > 0, that identifies an optimal strategy for a single population and sigmoidal , v > 0, that gives a competitive advantage to larger strategies.
The model is simple enough that we can solve analytically for all the relevant quantities: the monomorphic and dimorphic resident equilibrium densities (at which in the absence of population 2 and in the presence of both populations)
the monomorphic and dimorphic fitnesses
the monomorphic and dimorphic fitness gradients
the singular strategy making the monomorphic fitness gradient zero
the fitness second derivatives ruling branching at x*
and the third derivatives entering our approximations
It is easy to verify that the singular strategy x* is attracting the monomorphic evolutionary dynamics for any positive (σ, v) (the eigenvalue of the continuous dynamics is , that coexistence in its vicinity is always possible ( for v > 0) and that branching () occurs if σ2 > (1 + v)2/v. At σ2 = (1 + v)2/v the system undergoes the ESS-branching transition. Increasing the value of σ, the ESS turns into a branching point.
Figure 3 compares our (approximating) canonical model (8) and extinction boundaries , i = 1, 2, with their fully nonlinear versions. As in Fig. 2, the extinction boundary 1 and the x1-nullcline of the dimorphic evolutionary dynamics are plotted in blue (solid and dashed); red for boundary 2 and the x2-nullcline. Lighter colors are used for the fully nonlinear versions (recall that the approximation is quadratic for the extinction boundaries and linear for the xi-nullclines, locally to the singular point (x*, x*)).
How to cite this article: Dercole, F. et al. The transition from evolutionary stability to branching: A catastrophic evolutionary shift. Sci. Rep. 6, 26310; doi: 10.1038/srep26310 (2016).
Metz, J. A. J., Geritz, S. A. H., Meszéna, G., Jacobs, F. J. A. & van Heerwaarden, J. S. Adaptive dynamics: A geometrical study of the consequences of nearly faithful reproduction. In van Strien, S. J. & Verduyn Lunel, S. M. (eds.) Stochastic and Spatial Structures of Dynamical Systems 183–231 (Elsevier Science, North-Holland, 1996).
Geritz, S. A. H., Metz, J. A. J., Kisdi, E. & Meszéna, G. The dynamics of adaptation and evolutionary branching. Phys. Rev. Lett. 78, 2024–2027 (1997).
Geritz, S. A. H., Kisdi, E., Meszéna, G. & Metz, J. A. J. Evolutionarily singular strategies and the adaptive growth and branching of the evolutionary tree. Evol. Ecol. 12, 35–57 (1998).
Dercole, F. & Rinaldi, S. Analysis of Evolutionary Processes: The Adaptive Dynamics Approach and its Applications (Princeton University Press, Princeton, NJ, 2008).
Metz, J. A. J., Nisbet, R. M. & Geritz, S. A. H. How should we define fitness for general ecological scenarios? Trends Ecol. Evol. 7, 198–202 (1992).
Dieckmann, U. & Doebeli, M. On the origin of species by sympatric speciation. Nature 400, 354–357 (1999).
Kisdi, E. & Geritz, S. A. H. Adaptive dynamics in allele space: Evolution of genetic polymorphism by small mutations in a heterogeneous environment. Evolution 53, 993–1008 (1999).
Doebeli, M. & Dieckmann, U. Speciation along environmental gradients. Nature 421, 259–264 (2003).
Leimar, O. The evolution of phenotypic polymorphism: randomized strategies versus evolutionary branching. Am. Nat. 165, 669–681 (2005).
Ridley, M. Evolution. (Blackwell, Oxford, UK, 2004), 3rd edn.
Dieckmann, U. & Law, R. The dynamical theory of coevolution: A derivation from stochastic ecological processes. J. Math. Biol. 34, 579–612 (1996).
Champagnat, N., Ferrière, R. & Méléard, S. Unifying evolutionary dynamics: From individual stochastic processes to macroscopic models. Theor. Popul. Biol. 69, 297–321 (2006).
Geritz, S. A. H. Resident-invader dynamics and the coexistence of similar strategies. J. Math. Biol. 50, 67–82 (2005).
Meszéna, G., Gyllenberg, M., Jacobs, F. J. & Metz, J. A. J. Link between population dynamics and dynamics of Darwinian evolution. Phys. Rev. Lett. 95, 078105 (2005).
Dercole, F. & Geritz, S. Unfolding the resident-invader dynamics of similar strategies. J. Theor. Biol. 394, 231–254 (2016).
Maynard Smith, J. & Price, J. The logic of animal conflicts. Nature 246, 15–18 (1973).
Maynard Smith, J. Evolution and the Theory of Games (Cambridge University Press, Cambridge, UK, 1982).
Durinx, M., Metz, J. A. J. & Meszéna, G. Adaptive dynamics for physiologically structured population models. J. Math. Biol. 56, 673–742 (2008).
Dercole, F. The ecology of asexual pairwise interactions: The generalized law of mass action. Theor. Ecol. (2016), 10.1007/s12080-015-0287-3.
Della Rossa, F., Dercole, F. & Landi, P. The branching bifurcation of adaptive dynamics. Int. J. Bifurcat. Chaos 25, 1540001 (2015).
Durinx, M. Life amidst Singularities. Doctoral thesis, Institute of Biology, Leiden University, The Netherlands (2008).
Doebeli, M. & Ispolatov, I. Continuously stable strategies as evolutionary branching points. J. Theor. Biol. 266, 529–535 (2010).
Doebeli, M. & Ruxton, G. D. Evolution of dispersal rates in metapopulation models: Branching and cyclic dynamics in phenotype space. Evolution 51, 1730–1741 (1997).
Doebeli, M. & Dieckmann, U. Evolutionary branching and sympatric speciation caused by different types of ecological interactions. Am. Nat. 156, 77–101 (2000).
Geritz, S. A. H., Kisdi, E., van der Meijden, E. & Metz, J. A. J. Evolutionarily dynamics of seed size and seedling competitive ability. Theor. Popul. Biol. 55, 324–343 (1999).
Kisdi, E. Evolutionary branching under asymmetric competition. J. Theoret. Biol. 197, 149–162 (1999).
Landi, P., Dercole, F. & Rinaldi, S. Branching scenarios in eco-evolutionary prey-predator models. SIAM J. Appl. Math. 73, 1634–1658 (2013).
Metz, J. A. J. Thoughts on the geometry of meso-evolution: collecting mathematical elements for a post-modern synthesis. In Chalub, F. A. C. C. & Rodrigues, J. F. (eds.) The Mathematics of Darwin’s Legacy 197–234 (Birkhauser, Basel, 2011).
Metz, J. A. J. Adaptive dynamics. In Hastings, A. & Gross, L. (eds.) Encyclopedia of Theoretical Ecology 7–17 (University of California Press, Berkeley, 2012).
Kisdi, E., Jacobs, F. J. A. & Geritz, S. A. H. Red Queen evolution by cycles of evolutionary branching and extinction. Selection 2, 161–176 (2001).
Dercole, F. Remarks on branching-extinction evolutionary cycles. J. Math. Biol. 47, 569–580 (2003).
Scheffer, M., Carpenter, S., Foley, J. A., Folke, C. & Walker, B. Catastrophic shifts in ecosystems. Nature 413, 591–596 (2001).
Hui, C., Richardson, D. M., Landi, P., Minoarivelo, H. O., Garnas, J. & Roy, H. E. Defining invasiveness and invasibility in ecological networks. Biological Invasions, 18(4), 971–983 (2016).
Gould, S. J. & Eldredge, N. Punctuated equilibria: the tempo and mode of evolution reconsidered. Paleobiology 3, 115–151 (1977).
We thank Stefan Geritz (Helsinki University) and Hans Metz (Leiden University) for many fruitful discussions, Hans in particular for suggesting to exploit the fitness’ monomorphic-dimorphic link on the extinction boundaries of dimorphism. We also acknowledge the contribution of the reviewers. Financial support was provided by MIUR-FIRB under contract RBFR08TIA4.
The authors declare no competing financial interests.
Electronic supplementary material
About this article
Cite this article
Dercole, F., Della Rossa, F. & Landi, P. The transition from evolutionary stability to branching: A catastrophic evolutionary shift. Sci Rep 6, 26310 (2016). https://doi.org/10.1038/srep26310
This article is cited by
Adaptive Dynamics of a Stoichiometric Phosphorus–Algae–Zooplankton Model with Environmental Fluctuations
Journal of Nonlinear Science (2022)
Coffee Berry Borer (Hypothenemus hampei) and its role in the evolutionary diversification of the coffee market
Journal of Evolutionary Economics (2021)
Biological Invasions (2021)