Abstract
Understanding the relationship between complexity and stability in large dynamical systems—such as ecosystems—remains a key open question in complexity theory which has inspired a rich body of work developed over more than fifty years. The vast majority of this theory addresses asymptotic linear stability around equilibrium points, but the idea of ‘stability’ in fact has other uses in the empirical ecological literature. The important notion of ‘temporal stability’ describes the character of fluctuations in population dynamics, driven by intrinsic or extrinsic noise. Here we apply tools from random matrix theory to the problem of temporal stability, deriving analytical predictions for the fluctuation spectra of complex ecological networks. We show that different network structures leave distinct signatures in the spectrum of fluctuations, and demonstrate the application of our theory to the analysis of ecological time-series data of plankton abundances.
Similar content being viewed by others
Introduction
‘Will a large complex system be stable?’ asks the title of Robert May’s seminal 1972 paper1 that threw fuel on the fire of the complexity-stability debate and popularised the use of random matrix theory (RMT) in theoretical ecology. At first sight, answering this question with mathematics seems impossible. The huge number of interactions in real-world ecosystems hampers any attempt to create a precisely calibrated model, as the challenge of measuring all necessary parameters seems insurmountable. What May pointed out was that it might in fact not be necessary to know exact parameter values; knowledge of their statistical distribution could be sufficient. Combining the random model ecosystems proposed by Gardner and Ashby2, with results of Ginibre3 in RMT, May showed how complexity—measured in terms of the number of species and the connectance of their interaction network—could decrease ecosystem stability.
Although modelling ecosystems using random community matrices has been criticised4,5—with some arguing that these serve best as a null model for ecosystem structure6—this growing field has continued to provide insights into the mechanisms that promote ecosystem stability. For instance, Allesina and Tang7,8 generalised the community matrix model to account for different interaction types, elucidating the important stabilising role of predator-prey interactions. We now have quite a detailed view on the extent to which high-level ecosystem information (such as trophic9 or community10 structures) can be incorporated into the RMT framework to give more accurate predictions of the stability boundary.
The notion of stability referred to by May and these later works is that of asymptotic linear stability of an equilibrium point. While this definition is a natural mathematical choice, it belies the rich array of interpretations of ‘ecological stability’ present in the empirical ecological literature11. In order to make clear the differences between these interpretations, Grimm and Wissel12 created an inventory for different types of stability measures used in ecology. Some of these in particular are more attuned to the measures favoured by empirical ecologists. One such measure is ‘temporal stability’, often described as the constancy of ecological variables relative to their mean, which is commonly used as an indicator for ecological stability13,14,15,16,17,18,19,20. In ref. 21, Suweis et al. proposed to study the attenuation of perturbations as they propagate through ecological networks, introducing measures of reactivity and localization. Taking a different approach, recently Arnoldi et al.22 employed the term ‘variability’ to describe the inverse of temporal stability in a random community matrix model. In that work, they consider the scale of response to persistent external (environmental) noise applied to an ecosystem. While this is an important and useful measure, it does not capture anything of the temporal characteristics of fluctuations in ecosystems, which can drive a system away from equilibrium, and thus are important precursors to linear and nonlinear instabilities23.
In this paper, we seek to bridge empirical and theoretical measures of stability by developing a theoretical framework for the analysis of temporal stability of ecosystems. Our key object of study is the ‘power spectral density’, a statistical measure that captures the frequency and amplitude of noisy fluctuations in time series (see Fig. 1 for examples). The relationship between such power spectra and temporal notions of ecological stability is multifaceted. Particular points of interest are the height of spectrum, which gives information about the magnitude of stochastic fluctuations, the locations of non-zero peaks corresponding to quasi-cyclic signals, or a peak at zero indicating baseline wander. Moreover, the Fourier transform of the spectrum yields the autocorrelation structure of the stochastic trajectories. We provide a brief guide to these concepts in the Methods section 'Interpreting the power spectral density in the context of temporal stability', though in the main text we will refrain from ascribing overly simplified interpretations to power spectra.
Left: a mixed community of species with randomly assigned predation relationships. Right: a model ecosystem with two trophic levels—200 predators and 800 prey species. In both cases fluctuations are illustrated via a typical single-species time series (ξ(t), upper) and the mean power spectra (ϕ(ω), lower), where circles are simulation results, and solid lines our theory. Parameters are: Nx = 200, Ny = 800, cx = 20, cy = 5, α = 10, b = 1, d = 1; full details of all simulations are found in Methods section.
Beyond providing a more detailed view of temporal stability, an investigation of power spectra yields a number of further advantages. For instance, power spectra are readily computed from empirical data and provide detailed information about intrinsic fluctuations and (via the fluctuation dissipation theorem24) response to external perturbations. Previous theoretical studies of power spectra in low-dimensional systems have yielded important and sometimes surprising results in fields including epidemiology, game theory, and ecology25,26,27. The method is particularly powerful in explaining the emergence of persistent quasi-cyclic oscillations driven by noise. Until now, however, a major limitation of this theory has been its restriction to models with very small numbers of interacting elements for which the approach is analytically tractable with existing methods, while the applicability of the theory to larger systems is limited by comparatively slow numerical schemes, and difficulty parameterising large models. Here, by applying techniques from the statistical physics of complex systems, we demonstrate the possibility of deriving exact analytic formulae for the power spectra of large random and noisy dynamical systems.
We apply our method to characterise the stochastic fluctuations of species abundances in random Lotka–Volterra type ecosystem models. As a result, we find that their temporal stability is universally characterised by a few key parameters, including the proportion of predator-prey interactions and the rate of population turnover. This result is a temporal analogue of the famous Winger semi-circle law for random matrix eigenvalue distributions28 and points to the wide applicability of the theory we develop. Importantly, the universal character of the power spectrum we derive is independent of the choice of random variables in the model, and only depends on the aggregate properties we identify.
Just as May’s RMT calculations are open to generalisations and refinements, so too is our approach to temporal stability. We illustrate this flexibility of the theory by incorporating trophic structure to our ecosystem models. Subsequently, we discover a distinct signature of this type of structure: the confinement of fluctuations to a fixed band of frequencies. Taken together, these results raise the exciting prospect of being able to draw conclusions about the internal structure of an ecosystem through the analysis of its fluctuations.
The paper is structured as follows. First, we demonstrate how to compute the mean power spectral density of a large random Lotka–Volterra system in the section ‘Interaction types determine fluctuation spectra’, showing how different dominant interaction types result in distinct fluctuation power spectra. We then show how to compute the spectrum for an individual species within the large random ecosystem system in the section ‘Species fluctuations exhibit strong heterogeneity’, and further generalise the method in the section ‘Trophic structure induces fluctuation frequency gap’ to consider bipartite interaction networks, showing how a two-level trophic system can leave a distinct fingerprint in the power spectrum of an ecosystem. Readers interested in the potential of our results as a tool for analysis of real-time series data may wish to jump to the section ‘Confronting RMT in theoretical ecology with time-series data’ which provides a proof of concept in this direction. Here we explore an ecological time-series dataset of plankton abundances, showing how our results provide a technique to infer the structural details of real ecosystems. Full derivations of our analytic results are provided in the Methods section, along with detailed descriptions of the models that we use for demonstration throughout this paper.
Results
Interaction types determine fluctuation spectra
Our approach enables the computation of the power spectral density of fluctuations in large random systems of a very general class; a full and detailed derivation is given in the Methods. In the case of ecosystem stability, the dynamical system in question is that describing the interactions of different species. Many modelling choices are possible in this context. For clarity, we will focus here on an established modelling paradigm—large Lotka-Volterra type ecosystems—and explore the extent to which the nature of the species interactions affects the shape of the fluctuation spectrum.
Following classic models of ecosystem dynamics, we consider N species occupying a domain of size V, writing xi(t) for the density of individuals of species i at time t. For large but finite V, standard techniques (see Methods) allow us to describe the change of the species densities by a set of stochastic differential equations (SDEs) :
Here, the coefficients αij for i ≠ j describe the interaction between species i and j, and the ηi(t) are Gaussian noise terms with correlations \(\langle {\eta }_{i}(t){\eta }_{j}(t^{\prime} )\rangle =\delta (t-t^{\prime} ){B}_{ij}({\boldsymbol{x}})\).
We parameterise the model as follows. For simplicity (and to isolate the effect of interaction types) we model each species as having the same birth rate bi ≡ b and density-dependent mortality rate αii ≡ −b. The other interaction coefficients αij are chosen at random so that (i) each species interacts with an average of c others (for each possible interaction we include it with probability c/N, independent of all others), (ii) interactions have mean strength \({\mathbb{E}}| {\alpha }_{ij}| =\mu\) and second moment \({\mathbb{E}}{\alpha }_{ij}^{2}={\sigma }^{2}\), (iii) the correlation is controlled by the symmetry parameter \(\gamma ={\mathbb{E}}[{\alpha }_{ij}{\alpha }_{ji}]/{\sigma }^{2}\in [-1,1]\). Crucially, the full details of the distribution of the parameters αij are not required, thanks to the universality property of large random matrices29,30.
In the methods, we show how these rates can be derived from a simple model of pairwise species interactions which can be mutualistic, competitive, or predatory. The proportion of predator-prey type interactions is tied to the symmetry parameter γ. At γ = −1 all interactions are of the predator-prey type (αij = −αji), at γ = 1 only purely mutualistic (αij = αji > 0) or competitive (αij = αji < 0) are present, and between these extremes, there is a random mix of interaction types.
With this choice of (random) parameters, each species density will fluctuate around the scaled carrying capacity \({x}_{i}^{* }=1\), which, following8, is stable provided \(b\,> \,\sqrt{c{\sigma }^{2}}(1+\gamma )\) (we refer to31,32 for stability and feasibility of equilibrium states with heterogeneous species abundance distributions). Around this fixed point, species in the stochastic system in Eq. (1) will exhibit approximately linear fluctuations ξi, described by an Ornstein-Uhlenbeck process of the form
Here A is the Jacobian of Eq. (1), known as the community matrix in the context of theoretical ecology, and ζ is an N-vector of Gaussian white noise with correlation matrix B = B(x*). We assume that the equilibrium point at x = x* is linearly asymptotically stable (i.e. stable in the mathematical sense described by May1) and now proceed to investigate its temporal stability as characterised by the power spectra (see Methods section Interpreting the Power Spectral Density in the Context of Temporal Stability).
The power spectral density of fluctuations Φ(ω) is defined as the Fourier transform of the covariance \({\mathbb{E}}[{\boldsymbol{\xi }}(t){\boldsymbol{\xi }}{(t+\tau )}^{T}]\). For multivariate Ornstein-Uhlenbeck processes one can show (see e.g.33) that
In the Methods we show how to apply random matrix theory techniques to compute the power spectral density, via a complex Gaussian integral representation of the above matrix equation. The approach provides a general framework for computing the fluctuations in large systems specified by random matrices A and B. We derive an expression for the mean-field power spectral density \(\phi (\omega )={\mathbb{E}}[{{\mathbf{\Phi }}}_{ii}]\) in terms of the resolvent function r. Specifically,
where expectation is taken only over the non-zero entries of A and B, and \(r\in {\mathbb{C}}\) solves the self-consistent equation
This result holds for general random matrix models in which interaction parameters are drawn from the same distribution for all species pairs (and we later show how the method can be extended for other model types with species-specific parameters using a single-defect approximation or partitioned networks). For the present case of our Lotka-Volterra model, the community matrix coincides with the interaction matrix (that is Aij = αij). In the methods, we derive the rules \({\mathbb{E}}[{B}_{ii}]=2b+c\mu\), \({\mathbb{E}}[{A}_{ij}{B}_{ij}]=0\) for the statistics of the noise correlation matrix. To get a sense for the information contained in Eq. (4), we explore the result for several cases with varying interaction structures.
First consider a weak interaction limit where the difference between species is rather small, so that σ2 ≪ 1. In this case we find a simple Lorentzian spectral density:
Fluctuations of this type are indicative of a highly stable system in which the balance of interaction types γ has no influence. Next let us consider a limit where the power spectral density shows significant differences depending on the proportion of predator-prey interactions within the community, in particular, focusing on ecosystems that are near the stability boundary.
In the case of a system with predator-prey interactions only, we have γ = −1 and the ecosystem is stable for all positive birth rates b. Expanding in small b we find that fluctuations are of order 1/b, but are almost completely confined to a low-frequency window. If ω2 < 4cσ2 then
with an order 1/ω tail outside this range. Note that Eq. (7) has the shape of a quarter-circle, to be viewed as a natural counterpart to the Wigner semi-circle law in classical random matrix theory28. The result is illustrated in Fig. 1, left panel.
For a random mixture of interaction types with γ = 0 no approximations are necessary as Eq. (4) simplifies to
The stability boundary here is given by b2 = cσ2. The above result therefore implies the emergence of a 1/ω2 divergence in the power spectrum at low frequencies when such a system is close to instability (see Fig. 2).
Left: stable systems with γ = 0 (purple diamonds) and γ = 1 (green circles) have similar fluctuation spectra, with a higher proportion of predator–prey interactions in the γ = 0 case associated with higher overall excitation. Right upper: near instability a pole emerges at ω = 0, corresponding to baseline drift in marginally stable systems. Right lower: In log–log axes we see the different nature of the pole for mixed (γ = 0, purple diamonds) communities compared with those with only symmetric interactions (γ = 1, green circles). For γ = −1, see Fig. 1, left panel. Common parameters are N = 1000, c = 50, σ2 = 0.5.
When only mutualistic or competitive interactions are present (i.e. γ = +1), the full solution to Eq. (7) in this case is complicated, but for stable systems appears qualitatively similar to the result Eq. (8) above. Near the stability boundary, however, we find another behaviour. When b2 = 4cσ2, we find
In contrast to the previous case, this power spectrum exhibits a pole of order \(1/\sqrt{\omega }\) at low-frequency, followed by a 1/ω2 tail at high frequency (see Fig. 2).
Between these results, we are able to see how the proportion of predator-prey interactions in an ecosystem leaves a signature in the fluctuation spectrum. When predator-prey interactions are dominant, the shape of the spectrum is pulled towards a quarter circle law (Fig. 1, left panel); when they are rare, the low-frequency pole near instability changes its character (Fig. 2).
Species fluctuations exhibit strong heterogeneity
So far, we have considered only the mean power spectral density of fluctuations. The cavity method technology employed in the derivation of Eq. (4) can also yield detailed information about the fluctuation spectra of individual species in an ecosystem model. Suppose one is interested in a focal species i, and has data on the type and strength of interactions this species has with others in its ecosystem, as well as an estimate of the large scale ecosystem parameters appearing in Eq. (4). It is possible to make use of this data in a ‘single defect approximation’ (SDA) scheme in which one considers the fluctuations of species i when embedded in a large unknown ecosystem.
In the Methods, we show how to derive an SDA approximation \({\phi }_{i}^{{\rm{SDA}}}\) to the spectral density of fluctuations for species i, given by the expression
where ϕMF and rMF are the mean-field power spectrum and resolvent obeying Eq. (4).
In Fig. 3 we compare the average power spectral density of all species with the spectra of individual species as computed directly and via the SDA approximation. We immediately notice that the mean-field power spectral density is often not representative of individual species, which show surprisingly strong heterogeneity in their fluctuation spectra. Another interesting feature of these results is the presence of peaks in the power spectral density away from zero for some species — this implies quasi-periodic fluctuations in these populations that are not observed in the ecosystem as a whole.
Thin lines show power spectral densities for individual species in a predator-prey ecosystem model, computed using the single-defect approximation of Eq. (10). Thick lines show the power spectral density for comparison the two species with extremal fluctuations at low frequency, computed directly via Eq. (17); the dashed line shows the mean power spectral density. Parameters are: N = 500, c = 20, γ = −1, σ2 = 1/4/c, b = 0.2.
Finally, we observed the curious feature that (for this model at least) the total power of fluctuations appears approximately conserved, meaning that those species which do not have large fluctuations at low frequencies are the same as those with unusually large fluctuations at higher frequencies. At present we do not have an intuitive explanation for this behaviour, highlighting the richness of non-obvious information present in these complex power spectral densities.
Trophic structure induces fluctuation frequency gap
In the above investigations, we have employed a simple ecosystem model in which species interactions are assigned completely at random. In the past fifty years of research into random matrix ecosystem models, far more sophisticated and realistic models have been developed. Let us now illustrate how our methods may be applied to more detailed models using the example of ecosystems with explicit trophic structure. Here we focus on a bipartite predator-prey network as an example.
Consider a large model ecosystem composed of Nx predator species and Ny prey species, writing xi and yj for the density of predator species i and prey species j, respectively. With no prey-prey or predator-predator interactions, the interaction structure is bipartite. Each predator species has an extrinsic death rate d and depends upon the consumption of prey for reproduction. This consumption may come from a selection of cx different prey species for each predator, with Rij > 0 giving the predation rate of predator i on prey j. Conversely, each prey has birth rate b, but is hunted by cy predators, where Nxcx = Nycy. The SDEs for the predator and prey densities are given by
where ηi,j(t) are Gaussian noise with \(\langle {\eta }_{i}(t),{\eta }_{j}(t^{\prime} )\rangle ={\delta }_{ij}(t-t^{\prime} ){B}_{ij}({\boldsymbol{x}},{\boldsymbol{y}})\). In the Methods we show how these equations (and the specific form of Bij) are derived from an individual-based model.
This model has an equilibrium state (x*, y*), around which linear-order fluctuations will occur, analogously to Eq. (2) above. We compute a community matrix of the form
where the first i = 1, …, Nx rows and columns represent the predator species, and the remaining j = Nx + 1, …, Nx + Ny rows and columns correspond to the prey species. The noise matrix is derived from the underlying individual-based model (see Methods) and given by
In the Methods we develop a general approach to computing the power spectral density of large random systems with bipartite structure such as this. The method requires explicitly keeping track of the contributions associated to each species group and their interactions. In the mean-field, this approach delivers a set of equations (61) to be solved for the mean contributions to the resolvent rx, ry, and to the power spectrum, ϕx, ϕy. Figure 4 shows the shape of the power spectrum for predator and prey species in this bipartite ecosystem. Surprisingly, we find that fluctuations are mainly confined to a narrow window of frequencies, with a gap in excited frequencies around zero. Examination of the system in Eq. (61) allows us to determine the window of excited frequencies to be bounded by the critical frequencies
The exact power spectral densities (dots) for predator species in a two trophic level model are computed numerically from Eq. (17), and the corresponding mean power spectral density (solid line) are obtained by solving Eq. (61) (inset shows the prey species). Excited frequencies are confined to a band (shaded) between critical frequences given in Eq. (14). The order 1/N peak at ω = 1 in the simulation result relates to the high-level bipartite structure. Its location is predicted by a corresponding 2D system, also shown here for comparison (dashed line). Parameters: Nx = 100, Ny = 200, cx = 20, cy = 10, α = 5, b = 1, d = 1.
The contrast between the power spectral density of this two-trophic-level model to that of a mixed ecosystem with predator-prey interactions was illustrated in Fig. 1. In Fig. 4 we show the spectrum in more detail, highlighting the band of excited frequencies predicted by Eq. (14). In the present context, it means that observed time series will not exhibit baseline wander and can therefore be considered to have a higher long-term temporal stability than the mixed ecosystems explored above (see section Interpreting the power spectral density in the context of temporal stability).
Comparisons between simulations and our analytical results shows another interesting feature: an order 1/N disagreement at frequency ω = 1, which is outside of the excited range. This can be explained by considering an effective two-species model in which we consider only a single ‘average’ predator and prey pair. This 2D system has an eigenvalue pair with unit imaginary part, giving rise to quasi-cycle behaviour as documented in ref. 27. It is important to note that this contribution is small relative to the rest of the spectral density, meaning that the bulk of fluctuations of a structured ecosystem cannot be inferred from considering a low-dimensional representative model.
Confronting RMT in theoretical ecology with time series data
Although hugely influential in the field of theoretical ecology over the last 50 years, traditional work on RMT has so far led to rather limited empirically testable insights. The central issue is that while many ecological considerations can be incorporated in a random matrix model, each leads to a binary outcome; the system is either stable or unstable to small perturbations. Thus testing the predictions of these models demands the time-intensive task of measuring real species interaction networks (which are assumed to be stable) and asking whether they indeed tend to be weakly connected (as suggested by May1), have a dominance of predator-prey interactions (as suggested by Allesina and Tang7), or satisfy some other prediction of the theory. In contrast, the approach presented in this paper offers the tantalizing prospect of directly linking the ecological RMT framework with comparatively easy-to-obtain time series data.
To trial the use of our methods in the analysis of real ecological data, we have investigated a high-resolution time series dataset for the abundance of coastal plankton species, taken over a period of 88 consecutive days34. In Fig. 5 we show the estimated empirical mean power spectrum from the data (circles), compared to that of the best fit Lotka-Volterra random ecosystem model according to our theory. Full details of the data analysis and fitting are given in the Methods. Examination of this fit reveals several qualitative features of the implied ecological interactions.
We plot the power spectral density of data taken from34 (blue circles) alongside that of a fitted Lotka-Volterra random ecosystem model (solid line). Fitted parameters are b = 0.6643, cσ2 = 0.1316, γ = 0.8078, data averaged over n = 3 samples per day. Also shown is the best fit under the restriction γ < 0 (dashed line). The lower right panel shows the spectral boundary inferred from the fit (black ellipse), along with the eigenvalues of a sample random community matrix for illustration.
First, we note that the best fit value for the interaction symmetry parameter is γ = 0.81, implying an ecosystem in which predator-prey interactions are scarce, and is more likely dominated by competition. Trust in this finding is strengthened by the fact that the fit is quite sensitive to this parameter; the dashed line in Fig. 5 gives the best fit under the constraint γ < 0, which performs poorly, especially for low frequency.
Interestingly, when viewed in logarithmic axes (Fig. 5 main panel), the plankton abundance power spectrum appears to exhibit a similar change of scaling between high and low frequency ranges to that seen in Fig. 2 for the case of symmetric interactions near instability. We can assess the closeness to instability by considering the spectrum inferred from the best fit model, as shown in the lower right panel of Fig. 5. The rightmost edge (\({\lambda }_{\max }=\sqrt{c{\sigma }^{2}}(1+\gamma )-b\approx -0.0086\)) is very close to zero, implying ecosystem dynamics which are close to instability. This feature corresponds to the large peak at zero in the power spectral density, which suggests that low frequency perturbations to the overall species abundances are very slow to relax.
One feature of the spectrum not reproduced by the simple models considered thus far is the smaller additional peak around ω ≈ 1.5. This peak has a few possible explanations: an external effect of some sort; possible secondary structure in the ecological interaction network, which could manifest on a system wide scale such as the trophic structure analysed in the previous section; or a feature isolated to a smaller number of more dominant species. A further limitation of the model used here is the assumption of uniform species abundance; in reality, species abundances tend to be distributed log-normally, with few species contributing to the majority of ecosystem biomass. Incorporating such model refinements are well-within the bounds of analytical tractability for our approach (see, for instance32 and31); we hope and expect the theoretical groundwork we have developed here will pave the way for the investigation of such features in future studies.
In the above, we have illustrated the use of our methods to infer details of the structure and stability of real ecosystems from time series data, as well as to identify departures from the unstructured assumptions of standard RMT models. Indeed, such departures are present in many real world ecosystems, with important consequences for the validity of any predictions made within the standard RMT framework4. In contrast, RMT has recently found renewed attention in the field of microbiome research, where it is believed that the key conceit of standard RMT models (that communities are unstructured) holds35. However in this field, the spectre of model parameterization again raises its head.
In ref. 35, a species-interaction network presented in ref. 36 was used to parameterize an RMT model and show that ecological interactions in the microbiome tended to be weak and non-cooperative. The species interaction network determined in ref. 36 was itself the result of fitting mouse intestinal microbiome abundances to a deterministic generalized Lotka-Volterra model. However, fully fitting this model required disturbing the mouse microbiota away from its equilibrium state using antibiotics (S fixed point species abundances are insufficient to parameterize an S × S species interaction network, so data on non-equilibrium transient trajectories were required). While such experimental manipulation may be permissible for studying the microbiota of model organisms such as mice, the ethical issues of such experimentation in humans have raised questions about the informativeness of temporal data for understanding microbial communities such as the human gut microbiota37.
In contrast to the approach taken in ref. 35, our methodology requires no external perturbation to a host’s microbiome, relying as it does solely on the natural demographic fluctuations present in any finite population. In addition, our approach allows the RMT model itself to be directly parameterized through data, rather than requiring the fitting of an intermediate model.
Discussion
In this study we have revisited the complexity–stability question in theoretical ecology with a fresh perspective that develops a random matrix theory approach to temporal stability as captured by the power spectrum of fluctuations. We have applied our techniques to calculate analytic formulae describing the mean power spectra of large Lotka-Volterra ecosystems. We find the fluctuations are described by just a few key parameters: the mean, variance and correlation of entries of the community matrix and noise correlator. We further expanded the method to investigate the role of trophic structures in determining temporal stability, demonstrating the flexibility of the method and usage across a broader range of models. Finally we fitted our model to existing time series data, that suggest a majority of competitive or mutualistic interactions within plankton ecosystems. In short, our approach allows us to link the large scale statistical properties of interaction parameters with the emergent fluctuations in species dynamics.
Amongst the many results that this promising technique grants access to, several findings from our investigation are worth recapping here. Part of the power of random matrix theory is that it uncovers universal properties of large classes of systems of a certain type. In the present case we find that, in analogy to the famous Wigner semi-circle law, the details of the distributions of matrix elements are unimportant beyond the handful of key parameters identified. Our parameter γ, which controls the proportion of predator-prey interactions (and hence the correlation of off-diagonal elements in the interaction matrix) is found to be of crucial importance. At one extreme, we find a semi-circular spectral profile, at the other we find a pole at zero frequency which has either 1/ω2 or \(1/\sqrt{\omega }\) divergence, depending on the symmetry of interactions. When an explicit trophic structure is incorporated into the model, it was necessary to adapt our method to general bipartite networks. Here, we found a gap in the power spectral density, implying that this high-level structure leads to greater long-term temporal stability. Finally, going beyond these universal results for the mean spectrum, we find a huge variability in fluctuations at the individual species level. These are not visible within the bulk but are captured by a single defect approximation, showing that some species may exhibit quasi-cyclic oscillations even when no such signal is present in the larger system.
In each model investigated in this paper, we have characterised stochastic behaviour emerging on a macroscopic scale from the statistical properties of the underlying microscopic interactions. We emphasise that, as illustrated in our section on trophic levels, the fluctuations observed in large scale systems with a certain structure are likely to be substantially different from those of small scale models previously investigated. The models presented here have been chosen for simplicity and clarity, and they only scratch the surface of what can be achieved with this method. More realistic models might include a consideration of e.g. the dynamical assembling process of ecosystems38, heterogeneous turnover rates39, or explicitly spatial models where spatio-temporal patterning may persist40.
From an ecological perspective, it is desirable to connect our theoretical work to empirical investigations into ecosystem stability. In contrast to the traditional viewpoint of asymptotic linear stability, our methods directly address a fundamental empirical quantity—timeseries of species abundance. Beyond simply providing more detail as to the temporal dynamics of an ecosystem around an equilibrium point, our method has also opened up the exciting possibility of identifying the signature of a certain interaction structures in the power spectrum of oscillations in data gathered in the field. We fitted our model to a highly resolved time series data set on a plankton ecosystem. We found that the empirical data is indicative of an ecology dominated by competitive and mutalistic interactions, with far fewer predator-prey interactions. This insight is consistent with recent results that suggest that self-regulation (competition) and facilitation (mutualism) are widespread in phytoplankton communities41.
In order to realise this vision, some important further work is needed. Real ecosystems do not exist in a vacuum—we must consider the role of the surrounding environment, including interactions with external factors such as seasonal variation or changing climate. Our theoretical approach encourages further work focused on the application to data sets gathered in field studies with modifications more suitable for the method we presented.
Finally, we wish to emphasise that—despite the ecological focus in this paper—the models of the kind we analysed are ubiquitous in many different fields, and the methods we use throughout the paper offer a general framework for large dynamical systems with random variables. Models of large interaction networks are also used in fields as varied as deep learning42, finance43, biochemistry44 and neuroscience45. All of these systems depend on a high number of parameters that are often difficult to measure empirically. Our method provides a possibility to compute the power spectral density and gain insight into the model, which relies only on statistical meta parameters.
Methods
Power spectral density for a general Ornstein–Uhlenbeck process
In the following we develop a method to compute the power spectral density of N-dimensional Ornstein–Uhlenbeck processes,
where ζ(t) is an N-vector of Gaussian white noise with correlations \({\mathbb{E}}[{\boldsymbol{\zeta }}(t){\boldsymbol{\zeta }}{(t^{\prime} )}^{T}]=\delta (t-t^{\prime} ){\boldsymbol{B}}\). The matrix A determines the mean behaviour of ξ and is considered to be locally stable, i.e. all eigenvalues of A have negative real part. Using the matrices A and B one can fully determine the power spectral density of fluctuations for the Ornstein-Uhlenbeck process.
We are interested in the case that the coefficients Aij and Bij are derived from a complex network of interactions with weights drawn at random, possibly with correlations. This framework encompasses a very general class of models with a wealth of real-world applications including but not limited to the ecological focus we have here. The method we describe exploits the underlying network structure of A and B to deduce a self-consistent scheme of equations whose solution contains information on the power spectral density.
We start with the definition of the power spectral density Φ(ω) as the Fourier transform of the covariance \({\mathbb{E}}[{\boldsymbol{\xi }}(t){\boldsymbol{\xi }}{(t+\tau )}^{T}]\) at equilibrium,
From ref. 33 on multivariate Ornstein–Uhlenbeck processes, we know that the power spectral density can also be written in the form of the matrix equation,
In practice, this equation is difficult to use for large systems as large matrix inversion is analytically intractable and numerical schemes are slow and sometimes unstable. We take an alternative route by recasting Eq. (17) as a complex Gaussian integral reminiscent of problems appearing in the statistical physics of disordered systems. Our approach in the following is to treat ω as a fixed parameter and drop the explicit dependence from our notation. We begin by writing
Simplification of the integrand is achieved by unpicking the matrix inversion in the exponent via a Hubbard-Stratonovich transformation46,47. To this end we recast the system in the language of statistical mechanics by introducing N complex-valued ‘spins’ ui and N auxiliary variables vi, with the ‘Hamiltonian’
Introducing a bracket operator
we can obtain succinct expressions for the power spectral density Φ = 〈uu†〉 as well as the resolvent matrix \({\boldsymbol{{\mathcal{R}}}}={({\rm{i}}\omega -{\boldsymbol{A}})}^{-1}=\langle {\boldsymbol{u}}{{\boldsymbol{v}}}^{\dagger }\rangle\). Thus we may write,
where \({\mathcal{Z}}=| {\boldsymbol{A}}-i\omega {\boldsymbol{I}}{| }^{2}/{\pi }^{2N}\).
This construction may seem laborious at first, but it unlocks a powerful collection of statistical mechanics tools, including the ‘cavity method’. Originally, the cavity method has been introduced in order to analyse a model for spin glass systems48,49. Further applications of the method include the analysis of the eigenvalue distribution in sparse matrices50,51,52. We will exploit the network structure in a similar fashion in order to compute the power spectral density.
In our analysis, we find that it is convenient to split the Hamiltonian in Eq. (19) into the sum of its local contributions at sites i, \({{\mathcal{H}}}_{i}\), and contributions from interactions between i and j, \({{\mathcal{H}}}_{ij}\),
These terms can be decomposed as \({{\mathcal{H}}}_{i}={{\boldsymbol{w}}}_{i}^{\dagger }{{\boldsymbol{\chi }}}_{i}{{\boldsymbol{w}}}_{i}\) and \({{\mathcal{H}}}_{ij}={{\boldsymbol{w}}}_{i}^{\dagger }{{\boldsymbol{\chi }}}_{ij}{{\boldsymbol{w}}}_{j}\), where we introduce the compound spins \({{\boldsymbol{w}}}_{i}={({u}_{i},{v}_{i})}^{T}\) and transfer matrices,
Let us focus on the power spectral density of a particular variable ξi, obtained from the diagonal element ϕi = Φii. For this we compute the single-site marginal fi by integrating over all other variables,
Alternatively, ϕi can be obtained as the top left entry of the covariance matrix \({{\mathbf{\Psi }}}_{i}=\langle {{\boldsymbol{w}}}_{i}{{\boldsymbol{w}}}_{i}^{\dagger }\rangle\). We write the covariance matrix as the integral,
which could also be expressed in terms of a Gaussian integral,
By comparing Eqs. (25) and (26) we find that
We now insert Eq. (22) into Eq. (24) and obtain,
where we write \({f}_{j}^{(i)}\) for the ‘cavity marginals’,
In essence, the above discussion amounts to organising the 2N integrals in Eq. (21) in a convenient way, with the advantage of providing a simple intuition for the role of the underlying network. The superscript (i) is used to indicate that the quantity corresponds to the cavity network where node i has been removed. We will further use this notation for the ‘cavity covariance matrix’ \({{\mathbf{\Psi }}}_{jl}^{(i)}\) introduced in the following.
Next we perform the integration in Eq. (28) and compare to the form in Eq. (27). We thus obtain a recursion formula for the covariance matrix Ψi and the cavity covariance matrices \({{\mathbf{\Psi }}}_{jl}^{(i)}\),
where the notation i ~ j indicates that we sum over nodes j connected to node i. Unless there is some specific structure underlying the network, we assume that most real world cases have a ‘tree-like’ structure from the local view point of a single node i. Hence, it is highly unlikely that the nodes j and l are nearby in the cavity network where node i is removed, and thus \({{\mathbf{\Psi }}}_{jl}^{(i)}\) only gives non-zero contributions if j = l. We therefore reduce Eq. (30) and obtain for the covariance matrix,
Similarly, the cavity covariance matrix obeys the equation,
Here we use that Ψ(i, j) = Ψ(j) when the nodes i and k are not connected. In other words, removing node j from the cavity network where node i is missing, has the same effect as removing it from the full network. The system in Eq. (31) describes a collection of nonlinear matrix equations that must be solved self-consistently.
For networks with high enough connectivity (and to good approximation even with modest connectivity), the removal of a single node does not affect the rest of the network, as its contribution is negligible compared to the full system. Hence the system in Eq. (31) can be reduced to a smaller set of equations approximately satisfied by the matrices Ψi:
The power spectral density ϕi can be obtained as the top left entry of Ψi.
In order to progress further, we now consider specific approximations that help us compute the power spectral density. First, we take a mean-field approach in order to obtain the mean power spectral density for all nodes part of the network; we then use the result for the mean-field in order to compute a close approximation to the local power spectral density of a single node. Later, we adapt the method to partitioned networks where nodes belong to different types of connected groups.
Mean field
For the following, we assume that all agents in the system behave the same on average. In practice, the terms governed by self-interactions Aii are drawn from the same distribution for all agents. Similarly, the terms including Bii are governed by one distribution. Interaction strengths and connections with other nodes in the network are also sampled equally for all agents (we have explored a large Lotka-Volterra ecosystem as an example of such a network). In the mean-field (MF) formulation we assume that the mean degree and excess degree are approximately equal, and replace all quantities in Eqs. (31) and (32) with their average. Ψi = ΨMF ∀ i. We then obtain the following recursion equation,
In order to solve this equation, we parameterise,
where the top left entry ϕ corresponds to the mean power spectral density, and we introduce r as the mean diagonal element of the resolvent matrix \({\boldsymbol{{\mathcal{R}}}}\). Finally by inserting the ansatz of Eq. (35) into Eq. (34) we obtain,
where c is the average degree (i.e. number of connections) per node. Moreover, the expectations in the second term are to be taken over connected nodes i ~ j (i.e. non-zero matrix entries).
From Eq. (36) above, we obtain the equations,
We solve the second equation in Eq. (37) for r and write the mean power spectral density in terms of r,
This equation informs the first part of the results presented in the main text.
Single defect approximation
The single defect approximation (SDA) makes use of the mean-field approximation for the cavity fields, but retains local information about individual nodes. We parameterise similarly to Eq. (35) for a single individual. Moreover, we replace all other quantities with the respective mean-field approximation. Specifically, we obtain
We solve this equation for \({\phi }_{i}^{{\rm{SDA}}},{r}_{i}^{{\rm{SDA}}}\), which delivers
Partitioned network
Previously we assumed that all nodes in a network are interchangeable in distribution. However, many real-world applications feature agents with different properties, imposing a high-level structure on the network. We realise this by partitioning nodes into distinct groups that interact with each other (see the section Trophic structure model for a simple example).
In order to handle different connected groups we make use of the cavity method as in Eqs. (31) and (32). In particular, we split the sum in the second term on the right-hand side of these equations into contributions from each group in the partitioned network. Let M denote the number of subgroups Vm in a partitioned network then we write,
Similar to the previous sections we replace all quantities with a mean-field average \({{\mathbf{\Psi }}}_{m}^{{\rm{MF}}}\), but for each group separately. Hence we obtain M equations of the form
In order to compute the mean power spectral density for different groups separately, we use a parameterisation as in Eq. (35) for each group. Therefore we have,
for all m = 1, …, M. This delivers 2M equations to solve for all rm and ϕm. Numerically this is straightforward, although algebraically long-winded for the general case. However, the equations simplify for special cases. In the section Trophic structure model we demonstrate this method for a bipartite network where a lack of intra-group interactions simplifies the analysis.
Large Lotka-Volterra ecosystem
Model description
First, we define the framework for a general Lotka-Volterra ecosystem with N species and a large but finite system size V ≫ 1. Note that this parameter can be interpreted as a scaling factor for the fluctuation amplitude and thus, larger systems exhibit higher stability and quantitative reliability for our analytic results. Let Xi denote the number of individuals and xi = Xi/V the density of species i = 1, …, N. We start from the following set of reactions that define the underlying stochastic dynamics of the system:
The self-interactions are governed by the birth rate bi > 0 and density-dependent mortality rate Rii > 0. Furthermore, we define three interaction types between species i and j, namely mutualism, competition and predation. In the case of mutualistic interactions, both species benefit from each other, whereas competition means that both species have a higher mortality rate, depending on the density of the other species. For predator-prey pairs, one predator species benefits from the death of a prey species. The predator and prey species are chosen randomly, such that species i is equally likely to be a predator or prey of species j.
With probability Pc we assign an interaction rate Rij > 0 to the species pair (i, j), and with probability 1 − Pc there is no interaction between species i and j (i.e. Rij = 0). In other words, each species has on average c = NPc interaction partners. The reaction rates are considered to be i.i.d. random variables drawn from a half-normal distribution \(| {\mathcal{N}}(0,{\sigma }^{2})|\), where we write for the mean reaction rate \(\mu ={\mathbb{E}}[{R}_{ij}]=\sigma \sqrt{2/\pi }\) and raw second moment \({\sigma }^{2}={\mathbb{E}}[{R}_{ij}^{2}]\). For each interaction pair, the interaction type is chosen such that the proportion of predator-prey pairs is p ∈ [0, 1], and all non-predator-prey interactions are equally distributed between mutualistic and competitive interactions (i.e. the overall proportion of mutualistic/competitive interactions is 1/2(1 − p)). Lastly, we define the symmetry parameter γ = 1 − 2p, where γ = −1 if all interactions are of predator-prey type (p = 1), and similarly γ = +1 if there are no predator-prey interactions (p = 0). In a mixed case where predator-prey and mutualistic/competitive interactions have equal proportion (p = 1/2), we have γ = 0. Later we will see that γ is equivalent to the correlation of signed interaction strengths.
In the limit V → ∞, the dynamics of the species density xi obey the ordinary differential equations,
where αij are the interaction coefficients with ∣αij∣ = ∣αji∣ = Rij. The signs of the interaction coefficients are determined by the type of interaction between species i and j. For mutualistic interactions we have αij = αji > 0, and αij = αji < 0 for competitive interactions. In the case of predator–prey interactions the coefficients have opposite sign αij = −αji. Hence the symmetry parameter as described above is given by the correlation of interaction coefficients \(\gamma ={\mathbb{E}}[{\alpha }_{ij}{\alpha }_{ji}]\). Furthermore, in order to ensure bounded species densities, we require negative self-interactions αii = −Rii < 0.
If species live in isolation (i.e. when αij = 0 ∀ i ≠ j), we see that the densities approach the ‘effective’ carrying capacity Ki = −bi/αii. For the following computations we consider a large Lotka-Volterra system. Since we are only interested in the effects of interactions between species, we assume that all self-interactions are approximately equal. Thus we write for the birth rate bi = b and mortality rate αii = −b. This gives the effective carrying capacity K = 1 for all species.
The fixed point x* at the deterministic equilibrium state is given by,
We assume a random mixture of mutualistic and competitive interactions with equal proportions, and therefore the interaction coefficients αij have zero mean ( ∀ i ≠ j). Furthermore, we postulate that for large ecosystems where N → ∞, the equilibrium state \({x}_{i}^{* }={\mathbb{E}}[{x}_{i}^{* }]\equiv {x}^{* }\). Hence we obtain the expected equilibrium density x* = 1 for all species i. Note that the following computations are valid for any known fixed point x*, and our assumptions are for mathematical simplification only. The results are independent of the particular equilibrium configuration, as long as a stable equilibrium can be measured and extracted from data (we discuss a few caveats where we apply our method to time series data from a plankton ecosystem). This assumption allows us to write the Jacobian matrix for a linearisation around the equilibrium state, with elements,
In other words, the community matrix of a large Lotka-Volterra system as described above has the same form as the interaction matrix, i.e. Aij = αij. The local stability of such community matrix A is given by the elliptic law7,8. It states that with high probability all eigenvalues of the random matrix A are distributed on an ellipse in the complex plane, centered at (−b, 0) on the real axis. Thus for a stable matrix we require all eigenvalues to be negative, and hence the horizontal semi-axis of the ellipse determines the allowed range for the centre. It follows the stability criterion,
with the average number of connections c per species, and the correlation \(\gamma ={\mathbb{E}}[{A}_{ij}{A}_{ji}]\). For a random community matrix (i.e. γ = 0), we recover the stability criterion that has been proven by May1, \(\sqrt{c{\sigma }^{2}}\ <\ b\). If γ < 0, where the proportion of predator-prey type interactions is larger, the horizontal semi-axis of the ellipse becomes smaller. In other words, the stability criterion relaxes for predator-prey interactions. For γ = −1 (i.e. Aij = −Aji ∀ i, j), all interactions are of predator-prey type and all eigenvalues become purely imaginary. Therefore the stability criterion becomes 0 < b, as the ellipse stretches vertically into the imaginary plane. The opposite is true for mutualistic/competitive interactions (i.e. γ = +1), where eigenvalues are distributed on an ellipse with large horizontal radius along the real axis. Thus it is more likely that some eigenvalues have positive real part and the system destabilises. We choose the parameter b for each case, such that the stability criteria are fulfilled.
For a large but finite system size V, we write the stochastic differential equations,
where ηi(t) are Gaussian random variables with \(\langle {\eta }_{i}(t),{\eta }_{j}(t^{\prime} )\rangle =\delta (t-t^{\prime} ){B}_{ij}\). The noise matrix B can be obtained from the reactions that determine the process. The diagonal elements are given by the self-interactions and total interaction from all other species, and the off-diagonal elements depend on the type of interaction between species i and j. We assume that only predator-prey type interactions contribute to the covariance of species fluctuations (i.e. that only predator-prey interactions involve the simultaneous change in abundance of a species pair). Therefore, we write
We next linearise around the fixed point to obtain a new equation for the fluctuations, \({\boldsymbol{\xi }}=\sqrt{V}({\boldsymbol{x}}-{{\boldsymbol{x}}}^{* })\), which has the form of an Ornstein-Uhlenbeck process as defined in Eq. (15). Recall that in our simplified model the equilibrium abundance x* = 1 (note however, that in general the entries of the noise matrix B depend on the particular fixed point of a given system). Therefore we write for the noise matrix evaluated at the fixed point
where μ is given as the mean (over non-zero entries) reaction rate \(\mu ={\mathbb{E}}[{R}_{ij}]=\sigma \sqrt{2/\pi }\).
Computing the power spectral density
Let us now compute the mean power spectral density ϕ of the process described above using Eq. (38) as starting point. We replace the necessary quantities that we obtain from the community matrix A and noise matrix B as defined in the previous section. In particular, we have the expected diagonal elements of the community matrix \({\mathbb{E}}[{A}_{ii}]=-b\), and the noise matrix \({\mathbb{E}}[{B}_{ii}]=2b+c\mu\). Moreover the raw second moment of the non-zero interactions is given by \({\mathbb{E}}[{A}_{ij}]={\sigma }^{2}\) and the correlation \({\mathbb{E}}[{A}_{ij}{A}_{ji}]=\gamma {\sigma }^{2}\). We use that \({\mathbb{E}}[{A}_{ij}{B}_{ij}]=0\ \forall i,j\) since the off-diagonal elements of the noise matrix are only non-zero if there is a predator-prey interaction between species i and j. However, the elements of Aij have opposite signs in the case of predator-prey pairs and thus sum to zero.
Plugging in these quantities into Eq. (38) we obtain,
In the main text, we explore the theoretical ecological consequences of this result.
Trophic structure model
Model description
In the following we define a model analogous to the one described in the previous section. For a large but finite system size V we write the model in terms of a stochastic process. Previously we allowed for different types of interactions, however, in this model we only focus on predator-prey interactions. More specifically, the interaction network is partitioned into Nx predator species and Ny prey species, where N = Nx + Ny is the total number of species. We assume that predators only interact with prey and vice versa (i.e. we assume no inter-species interactions within the groups of predators or prey) as illustrated in Fig. 6. Moreover, each predator and prey species interacts with themselves (density-dependent mortality).
Here we have nx = 4 predators with cx = 3 prey each, and ny = 6 prey with cy = 2 predators each. The bold lines illustrate the connections to the focal node before and after removing a node from the network. Predator nodes (black) only have local contributions from prey nodes (white) and vice versa. In the mean-field approximation for the power spectral density, these contributions are replaced by the average of each group.
In the previous model we assigned the same birth rate to all species in the ecosystem. Here we assume that predators decline at rate d in absence of prey, and b is the birth rate of prey species. For simplicity, we assume that d, b are fixed quantities, equal for all predators and prey respectively. Furthermore, Rij is the interaction rate between predator i and prey j. Each predator species has a fixed number of prey cx and each prey species has a fixed number of predators cy, such that Nxcx = Nycy. The parameters cx, cy can be interpreted as outgoing degrees of predator and prey nodes respectively. Connections between predators and prey are then wired randomly. The interaction strength is set to Rij = α, and considered equal for all predator-prey interactions (analogous to a mean reaction rate). Where there is no interaction between species, the interaction rate is simply set to zero. Note that this means that the total sum of interaction strength is constant αcx and αcy for all predator and prey species respectively. In contrast to the previous model, now only the network structure contributes to the randomness of the system.
Let xi denote the density of predator species i = 1, …, Nx, and yj the density of prey species j = 1, …, Ny. In the deterministic limit where V → ∞ we then write the following ODEs,
Given the fixed number of connections cx, cy and interaction strength α, we can simplify the ODEs to two equations for the average predator and prey densities,
In the limit of large N, the equilibrium state of the system converges to the average quantities obtained from this reduced form. The biologically relevant equilibrium states for this system are given by the trivial fixed points (x*, y*) = (0, 0), (0, b), and the non-trivial fixed point,
Next, we write the Jacobian matrix for a linearisation around the non-trivial fixed point. The community matrix takes the form,
where the first i = 1, …, Nx rows and columns represent the predator species, and the remaining j = Nx + 1, …, Nx + Ny rows and columns correspond to the prey species.
For a large but finite system size V we write the corresponding stochastic differential equations,
where ηi,j(t) are Gaussian noise with \(\langle {\eta }_{i}(t),{\eta }_{j}(t^{\prime} )\rangle ={\delta }_{ij}(t-t^{\prime} ){B}_{ij}\). The noise matrix is given by the self- and total interactions on the diagonal, and the interactions between predators and prey on the off-diagonal. We therefore write,
Again, this allows us to write the dynamics in form of an Ornstein-Uhlenbeck process as defined in Eq. (15).
Computing the power spectral density
In the following, we use features of the bipartite interaction network. For instance, all nodes that are connected to e.g. node xi, will be prey nodes yj, and thus are not connected with each other (see Fig. 6). This allows us to write the following recursion formulas for the mean power spectral densities according to Eq. (42),
Recall that the top left entries of Ψx and Ψy deliver the mean power spectral densities for predators ϕx and prey ϕy respectively. For the bipartite model, the helping matrices χi, χij (as defined in Eq. (23)) are given by,
Inserting and writing out Eq. (59) gives,
where cx, cy are the number of connections per predator and prey species respectively. Analogous to Eq. (38) we now derive a system of equations and solve for rx, ry and ϕx, ϕy. In the main text we describe the features of the power spectral density deduced from this system of equations.
Interpreting the power spectral density in the context of temporal stability
For orientation, we here provide some interpretation of the power spectral density in the context of temporal stability. Essentially when we talk about temporal stability, we can be referring to one of two measures. The first is how far stochastic trajectories tend to stray from their equilibrium value over long time horizons. We refer to this as ‘variability’20. The second is how quickly population abundances tend to change over finite time horizons. We will characterise this by the ‘temporal autocorrelation’.
The variability can be characterised by the variance in time-averaged trajectories around the mean22. For a system such as Eq. (2), which we recall can be a linear approximation for a nonlinear system such as Eq. (1), we find that ξ is normally distributed with zero mean and a covariance matrix, Σ, that solves the following Lyapunov equation53;
The stationary distribution of ξ is then \({P}_{{\rm{st}}}({\boldsymbol{\xi }})={\mathcal{N}}({\boldsymbol{0}},{\mathbf{\Sigma }})\). For instance, in the left panels of Fig. 7, we show stochastic trajectories for two different systems, with standard deviations marked by black dashed lines. Meanwhile, the marginal normal distribution for these trajectories is plotted in the inset of the right panels of Fig. 7. A system can then be said to be ‘less stable’ (in a temporal sense) if it has a greater variability. A consideration of the solutions to Eq. (62) shows that this measure of temporal stability is highly correlated with asymptotic stability; less stable deterministic systems tend to have stochastic counterparts with higher variance around equilibrium states.
Left panels: Examples of stochastic trajectories with 0 means indicated by solid lines and the standard deviations indicated by dashed lines. Insets show the same trajectories over a smaller time-window. Right panels: Corresponding power-spectral densities of the trajectories in the left hand panels. Insets show histograms of the corresponding stochastic trajectories, overlaid by the theoretical stationary distribution (black dashed line). Bottom panel: The temporal autocorrelation of the stochastic trajectories in the left hand panels can be obtained as the Fourier transform of the corresponding power spectra in the right hand panels.
Despite the fact that the trajectories in Fig. 7 have the same variance (see black dashed lines in left panels and inset plots in right panels) it is clear that they have very different temporal structure. While these differences are entirely masked by the measure of variability (which time-averages out the temporal structure), such differences are captured by the power-spectral density (see Fig. 7, right panels). For instance, the peak at ω ≈ 0.3 in the power spectrum in the upper right hand panel indicates that the trajectories in the upper left panel exhibit quasi-cycles (i.e. have a typical frequency, see inset), while the peak at ω = 0 at following decay of the power spectrum in the middle right hand panel indicates that the trajectories in the middle left panel do not exhibit quasi-cycles (i.e. do not have a typical frequency, see inset).
In the context of temporal stability, the relationship between the power spectra and the autocorrelation function 〈ξ(t)ξ(t − τ)〉 is of particular importance. By the Wiener-Khinchin theorem, we know that the autocorrelation function is given by the Fourier transform of the power spectrum. This is shown in the bottom panel of Fig. 7. The autocorrelation of the trajectory in the upper panels decays rapidly with time. In contrast, the autocorrelation of the trajectory in the middle panel decays more slowly. This can be clearly seen in the inset trajectory plots (left hand panels, top and middle). Thus we see that a distinct measure of temporal stability exists that is more appropriate over shorter time horizons; a system can be said to be ‘less stable’ over finite times if it has a more rapidly decaying autocorrelation function. This measure of temporal stability is more weakly correlated with asymptotic stability than its counterpart, variability, as it is affected by the magnitude imaginary parts of the system’s eigenvalues (rather than their real parts, as in asymptotic stability).
Figure details
Figure 1, left panels
A large random Lotka-Volterra ecosystem of the type described above was generated. Parameters used were: N = 1000, c = 50, γ = −1, σ2 = 1/4/c, b = 0.05. The solid line is the result of Eq. (6), noting that \(\mu =\sqrt{2{\sigma }^{2}/\pi }\). For the empirical power spectrum, we used an Euler-Maryama time-stepping method to simulate a time series of length \({t}_{\max }={2}^{10}\) and time step h = 2−7. The power spectrum for each species was calculated with a Fast Fourier Transform, and the result averaged over all species. The top panel shows part of the time series generated for the first species.
Figure 1, right panel
A two-trophic level model ecosystem was generated as described above. Parameters in this case were: Nx = 200, Ny = 800, cx = 20, cy = 5, α = 10, b = 1, d = 1. Time series and power spectra were computed similarly to the left panels.
Figure 2
For the left panel, we generated Lotka-Volterra ecosystems with parameters N = 1000, c = 50, σ2 = 0.5, using \(b=2+(1+\gamma )\sqrt{\left(\right.}c* {\sigma }^{2}\left.\right)\) for the simulations with γ = 0 and γ = 1. Time series and spectra were computed similarly to Fig. 1. For the right panels more care is needed. Finite random matrices typically have a small number of eigenvalues that are order \(1/\sqrt{N}\) larger than predicted by the stability boundary in the limit N → ∞. To achieve the near-instability results in this figure, we first generated the off-diagonal entries of the community matrices, then chose the birth rate b to put the rightmost eigenvalue of A exactly at zero.
Figure 3
Parameters here are: N = 500, c = 20, γ = −1, σ2 = 1/4/c, b = 0.2. For the ‘direct’ results we numerically computed the power spectral density according to the matrix formula in Eq. (17). This was preferable to simulations of the time series, as a long time horizon is required to achieve good resolution of the individual contributions to the power spectral density.
Figure 4
Parameters here are: Nx = 100, Ny = 200, cx = 20, cy = 10, α = 5, b = 1, d = 1.
Figure 5
The dataset 41467_2017_2571_MOESM6_ESM.xlsx was imported into Matlab and processed as follows: We took the average of the three reported daily measurements to construct an 88-day time series for each species. To limit boundary effects we discarded all species with at least one with zero measured abundance, in doing so retaining 100 species. The mean was subtracted and then the power spectrum fitted using the covariance method with 8th order autoregression. The model fitting was achieved with a non-linear least squares method applied to our Eq. (38), with parameters b, cσ2 (a composite parameter), γ, and an additional scale parameter for overall noise strength.
Reporting summary
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
Data availability
Plankton abundance data used in Fig. 5 are taken from ref. 34 available at: nature.com/articles/s41467-017-02571-4#Sec24. All simulation data can be reproduced using the code available at https://doi.org/10.5281/zenodo.4720998.
Code availability
Code to reproduce all Figures is available at https://doi.org/10.5281/zenodo.4720998.
References
May, R. M. Will a large complex system be stable? Nature 238, 413 (1972).
Gardner, M. R. & Ashby, W. R. Connectance of large dynamic (Cybernetic) systems: critical values for stability. Nature 228, 784–784 (1970).
Ginibre, J. Statistical ensembles of complex, quaternion, and real matrices. J. Math. Phys. 6, 440–449 (1965).
James, A. et al. Constructing random matrices to represent real ecosystems. Am. Naturalist 185, 680–692 (2015).
Jacquet, C. et al. No complexity–stability relationship in empirical ecosystems. Nat. Commun. 7, 12573 (2016).
Leigh Jr, E. G., Rosindell, J. & Etienne, R. S. Unified neutral theory of biodiversity and biogeography. Scholarpedia 5, 8822 (2010).
Allesina, S. & Tang, S. Stability criteria for complex ecosystems. Nature 483, 205–208 (2012).
Allesina, S. & Tang, S. The stability-complexity relationship at age 40: a random matrix perspective. Popul. Ecol. 57, 63–75 (2015).
Allesina, S. et al. Predicting the stability of large structured food webs. Nat. Commun. 6, 1–6 (2015).
Grilli, J., Rogers, T. & Allesina, S. Modularity and stability in ecological communities. Nat. Commun. 7, 1–10 (2016).
Donohue, I. et al. Navigating the complexity of ecological stability. Ecol. Lett. 19, 1172–1185 (2016).
Grimm, V. & Wissel, C. Babel, or the ecological stability discussions: an inventory and analysis of terminology and a guide for avoiding confusion. Oecologia 109, 323–334 (1997).
Levins, R. Coexistence in a variable environment. Am. Naturalist 114, 765–783 (1979).
Ives, A. R., Gross, K. & Klug, J. L. Stability and variability in competitive communities. Science 286, 542–544 (1999).
Lehman, C. L. & Tilman, D. Biodiversity, stability, and productivity in competitive communities. Am. Naturalist 156, 534–552 (2000).
Tilman, D., Reich, P. B. & Knops, J. M. H. Biodiversity and ecosystem stability in a decade-long grassland experiment. Nature 441, 629–632 (2006).
Jiang, L. & Pu, Z. Different effects of species diversity on temporal stability in single trophic and multitrophic communities. Am. Naturalist 174, 651–659 (2009).
Loreau, M. & de Mazancourt, C. Species synchrony and its drivers: neutral and nonneutral community dynamics in fluctuating environments. Am. Naturalist 172, E48–E66 (2008).
Campbell, V., Murphy, G. & Romanuk, T. N. Experimental design and the outcome and interpretation of diversity-stability relations. Oikos 120, 399–408 (2011).
Donohue, I. et al. On the dimensionality of ecological stability. Ecol. Lett. 16, 421–429 (2013).
Suweis, S., Grilli, J., Banavar, J. R., Allesina, S. & Maritan, A. Effect of localization on the stability of mutualistic ecological networks. Nat. Commun. 6, 1–7 (2015).
Arnoldi, J.-F., Loreau, M. & Haegeman, B. Resilience, reactivity and variability: a mathematical comparison of ecological stability measures. J. Theor. Biol. 389, 47–59 (2016).
Wiesenfeld, K. Noisy precursors of nonlinear instabilities. J. Stat. Phys. 38, 1071–1097 (1985).
Kubo, R. The fluctuation-dissipation theorem. Rep. Prog. Phys. 29, 255 (1996).
Alonso, D., McKane, A. J. & Pascual, M. Stochastic amplification in epidemics. J. R. Soc. Interface 4, 575–582 (2007).
Galla, T. Intrinsic noise in game dynamical learning. Phys. Rev. Lett. 103, 198702 (2009).
McKane, A. J. & Newman, T. J. Predator-prey cycles from resonant amplification of demographic stochasticity. Phys. Rev. Lett. 94, 218102 (2005).
Wigner, E. P. On the distribution of the roots of certain symmetric matrices. Ann. Math. 67, 325–327 (1958).
Tao, T. & Vu, V. Random matrices: the circular law. Commun. Contemp. Math. 10, 261–307 (2008).
Tao, T. et al. Random matrices: Universality of esds and the circular law. Ann. Probab. 38, 2023–2065 (2010).
Stone, L. The feasibility and stability of large complex biological networks: a random matrix approach. Sci. Rep. 8, 8246 (2018).
Gibbs, T., Grilli, J., Rogers, T. & Allesina, S. Effect of population abundances on the stability of large random ecosystems. Phys. Rev. E 98, 022410 (2018).
Gardiner, C. Stochastic Methods: A Handbook for the Natural and Social Sciences. Springer Series in Synergetics. Springer-Verlag, Berlin Heidelberg, 4 edition, 2009.
Martin-Platero, A. M. et al. High resolution time series reveals cohesive but short-lived communities in coastal plankton. Nat. Commun. 9, 266 (2018).
Coyte, K. Z., Schluter, J. & Foster, K. R. The ecology of the microbiome: Networks, competition, and stability. Science 350, 663–666 (2015).
Stein, R. R. et al. Ecological modeling from time-series inference: Insight into dynamics and stability of intestinal microbiota. PLOS Comput. Biol. 9, 1–11 (2013).
Xiao, Y. et al. Mapping the ecological networks of microbial communities. Nat. Commun. 8, 2042 (2017).
Galla, T. Dynamically evolved community size and stability of random lotka-volterra ecosystems (a). Europhys. Lett. 123, 48004 (2018).
Gibbs, T., Grilli, J., Rogers, T. & Allesina, S. Effect of population abundances on the stability of large random ecosystems. Phys. Rev. E 98, 022410 (2018).
McKane, A. J., Biancalani, T. & Rogers, T. Stochastic pattern formation and spontaneous polarisation: the linear noise approximation and beyond. Bull. Math. Biol. 76, 895–921 (2014).
Picoche, C. & Barraquand, F. Strong self-regulation and widespread facilitative interactions in phytoplankton communities. J. Ecol. 108, 2232–2242 (2020).
Pennington, J. & Worah, P. Nonlinear random matrix theory for deep learning. In Advances in Neural Information Processing Systems, 2637–2646, (2017).
Moran, J. & Bouchaud, J.-P. Will a large economy be stable, Available at SSRN (2019).
Luo, F. et al. Constructing gene co-expression networks and predicting functions of unknown genes by random matrix theory. BMC Bioinforma. 8, 299 (2007).
Almog, A. et al. Uncovering functional signature in neural systems via random matrix theory. PLoS Comput. Biol. 15, e1006934 (2019).
Stratonovich, R. L. On a method of calculating quantum distribution functions. Sov. Phys. Dokl. 2, 416 (1957).
Hubbard, J. Calculation of partition functions. Phys. Rev. Lett. 3, 77–78 (1959).
Mézard, M., Parisi, G. & Virasoro, M. A. SK Model: the replica solution without replicas. Europhys. Lett. 1, 77–82 (1986).
Mezard, M., Parisi, G. and Virasoro, M. A. Spin glass theory and beyond. Number v. 9 in World Scientific lecture notes in physics. World Scientific, Singapore; New Jersey, 1987. OCLC: ocm14929802.
Rogers, T., Castillo, I. P., Kühn, R. & Takeda, K. Cavity approach to the spectral density of sparse symmetric random matrices. Phys. Rev. E 78, 031116 (2008).
Rogers, T. & Castillo, I. P. Cavity approach to the spectral density of non-Hermitian sparse matrices. Phys. Rev. E 79, 012101 (2009).
Metz, F. L., Neri, I. & Rogers, T. Spectral theory of sparse non-hermitian random matrices. J. Phys. A: Math. Theor. 52, 434003 (2019).
Horn, R. A. & Johnson, C. R. Topics in Matrix Analysis (Cambridge University Press, Cambridge, 1991).
Acknowledgements
T.R. and Y.K. gratefully acknowledge the support of the Royal Society. G.W.A.C. thanks Leverhulme Trust for support through the Leverhulme Early Career Fellowship.
Author information
Authors and Affiliations
Contributions
Y.K., Q.Y., G.W.A.C. and T.R. developed the theory and performed the computations. Y.K. wrote the manuscript with support from T.R. and G.W.A.C. Y.K., Q.Y., G.W.A.C. and T.R. discussed and approved the final manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Peer review information Nature Communications thanks Francesca Di Patti, Ihor Vakulchyk and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Krumbeck, Y., Yang, Q., Constable, G.W.A. et al. Fluctuation spectra of large random dynamical systems reveal hidden structure in ecological networks. Nat Commun 12, 3625 (2021). https://doi.org/10.1038/s41467-021-23757-x
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467-021-23757-x
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
