Abstract
We present a machine learning framework bridging manifold learning, neural networks, Gaussian processes, and EquationFree multiscale approach, for the construction of different types of effective reduced order models from detailed agentbased simulators and the systematic multiscale numerical analysis of their emergent dynamics. The specific tasks of interest here include the detection of tipping points, and the uncertainty quantification of rare events near them. Our illustrative examples are an eventdriven, stochastic financial market model describing the mimetic behavior of traders, and a compartmental stochastic epidemic model on an ErdösRényi network. We contrast the pros and cons of the different types of surrogate models and the effort involved in learning them. Importantly, the proposed framework reveals that, around the tipping points, the emergent dynamics of both benchmark examples can be effectively described by a onedimensional stochastic differential equation, thus revealing the intrinsic dimensionality of the normal form of the specific type of the tipping point. This allows a significant reduction in the computational cost of the tasks of interest.
Similar content being viewed by others
Introduction
Complex systems are typically characterized by multiscale phenomena giving rise to unexpected emergent behavior^{1,2} including catastrophic shifts/major irreversible changes in the dominant mesoscopic/macroscopic spatiotemporal behavioral pattern. Such sudden major changes occur with higher probability near socalled tipping points^{3,4,5,6}, which are most often associated with bifurcation points in nonlinear dynamics terminology. The computation of the frequency/probability of occurrence of such transitions, and the detection of the corresponding tipping points that underpin them, is of critical importance in many realworld systems. Our need for understanding (and controlling) such phenomena has made AgentBased Models (ABMs) a key modeling tool for building digital twins in domains ranging from ecology^{7,8,9,10} and epidemics^{11,12,13,14,15,16}, to finance and economy^{17,18,19,20}. Examples include the models of infectious disease agent study (MIDAS) research network, initiated in 2004 by the US National Institutes of Health (NIH) with the mission to develop largescale ABMs to understand infectious disease dynamics and assist policymakers to detect, and respond to flu pandemics; the SantaFe Artificial Stock Market; and, the Eurace ABM of the European economy^{19,21}. When such detailed highfidelity ABMs are available, systemslevel analysis practice often involves performing extensive, bruteforce temporal simulations to estimate the frequency distribution of abrupt transitions^{17,19,20,22,23}. However, such an approach confronts the “curse of dimensionality”: the computational cost rises exponentially with the number of degrees of freedom^{17,20}. Such a direct simulation scenarios approach is therefore neither systematic nor computationally efficient for highdimensional ABMs. Furthermore, it often does not provide physical insight regarding the mechanisms that drive the transitions. A systematic analysis of such mechanisms requires two essential tasks. First comes the discovery of an appropriate lowdimensional set of collective variables (observables) that can be used to describe the evolution of the emergent dynamics^{24,25}. Such coarsescale variables may, or may not, be a priori available, depending on how much physical insight we have about the problem. For this task, various manifold/machine learning methods have been proposed, including diffusion maps (DMAPs)^{26,27,28,29,30}, ISOMAP^{31,32}, and local linear embedding (LLE)^{33,34}, but also autoencoders (AE)^{35,36,37,38}.
Based on this initial analysis, the second task pertains to the construction of appropriate reducedorder models (ROMs), in order to parsimoniously perform useful numerical tasks and—hopefully—obtain additional physical insight. One option is the construction of ROMs “by paper and pencil”, using the tools of statistical mechanics^{24,25}. However, restrictive assumptions, that are made in order to obtain explicit closures bias the estimation of the actual location of tipping points, as well as the statistics and the uncertainty quantification (UQ) of the associated catastrophic shifts^{24}.
Another option is the direct, datadriven identification of surrogate models in the form of ordinary, stochastic, or partial differential equations via machine learning. Such approaches include, to name a few, sparse identification of nonlinear dynamical systems (SINDy)^{39}, Gaussian process regression (GPR)^{29,40,41}, feedforward neural networks (FNNs)^{29,30,38,42,43,44,45,46}, random projection neural networks (RPNNs)^{30,47}, recursive neural networks (RvNN)^{37}, reservoir computing (RC)^{48}, autoencoders^{37,38,49,50}, as well as DeepOnet^{51}. However, their approximation accuracy clearly depends very strongly on the available training data, especially around the tipping points, where the dynamics can even blow up in finite time.
If the coarsevariables are known, the Equationfree (EF) approach^{1} offers an efficient alternative for learning “on demand” local blackbox coarsegrained maps for the emergent dynamics on an embedded lowdimensional subspace; this bypasses the need to construct (global, generalizable) surrogate models. This approach can be particularly useful when conducting numerical bifurcation analysis, or designing controllers for ABMs^{52}. However, even with a knowledge of good coarsescale variables, constructing the necessary lifting operator (going from coarse scale descriptions to consistent fine scale ones) is far from trivial^{52,53}.
Here, based on our previous efforts on the construction of latent spaces^{52,54,55} and ROM surrogates via machine learning (ML)^{29,30,34,45,46} from microscopic detailed spatiotemporal simulations, we present an integrated ML framework for the construction of two types of surrogate models: global as well as local. In particular, we learn (a) mesoscopic Integro Partial Differential Equations (IPDEs), and (b)—guided by the EF framework^{1,56}—local embedded lowdimensional meanfield Stochastic Differential Equations (SDEs), for the detection of tipping points and the construction of the probability distribution of the catastrophic transitions that occur in their neighborhood.
Our main methodological point is that, given a macroscopic task, it is the task itself that determines the type of surrogate model required to perform it. Here, the tasks are the identification of tipping points as well as the uncertainty quantification of escape times in their neighborhood. For such tasks, a first option—common in practice—is the construction of a datadriven, MLidentified IPDEs for macroscopic fields, such as the agent density. Such equations provide some physical insight for the emergent dynamics, yet this insight does not come without its problems: for example, collecting the training data and designing the sampling process in the relatively high dimensional parameter and state space is not an easy task, especially in the unstable regimes where the dynamics may blow up in finite time. The second option, assuming an approximate knowledge of the tipping point location, is to identify a less detailed, meanfieldlevel, effective SDE. This offers the capability of more easily estimating escape time statistics through either: (a) bruteforce bursts of SDE simulations, or through (b) numerically solving a boundaryvalue problem given the identified lowdimensional drift and diffusivity functions. For such approach, a challenging issue is the discovery of a convenient/interpretable lowdimensional latentspace.
Thus, in the spirit of the Wykehamist “manners makyth man”, and of Rutherford Aris’ “manners makyth modellers”^{57}, in our case, we argue for a “tasks makyth models” consideration in selecting the right approach for MLassisted model selection^{57}.
Our illustrative case studies are (i) an eventdriven stochastic agentbased model describing the interactions, under mimesis of traders in a simple financial market^{58}. In other words, the traders in this ABM tend to imitate the behavior of other traders, because of social conformity or subtle psychological pressure to align their behavior with that of other agents (their peers). This model exhibits a tipping point, marking the onset of a financial “bubble”^{23,56}; (ii) a stochastic ABM of a hosthost interaction epidemic evolving on an ErdösRényi social network^{59}. This ABM is characterized by a tipping point marking the onset of outbreaks and regions of hysteresis, where transitions between “endemic disease” and “global infection” states can occur.
The proposed ML framework reveals that the emergent dynamics of both ABMs around the tipping points can be effectively described on a onedimensional manifold. In other words, it discovers the intrinsic dimensionality of the normal form of the specific type of tipping point, which for both problems is a saddlenode bifurcation. This allows for a significant reduction of the computational cost required for numerical analysis and simulations.
Results
Case study 1: Tipping points in a financial market with mimesis
ABMs enable the creation of digital twins for financial markets, thus offering a valuable tool in our arsenal for explaining outofequilibrium phenomena such as “bubbles” and crashes^{17} that emerge mainly due to positive feedback mechanisms of imitation and herding of investors that lead to an escalating increase of the demand^{60} (see for example the Santa Fe artificial stock market^{19,61}, and the EURACE ABM for modeling the European economy^{21}). While the practical application of ABMs for providing predictions about realworld financial instabilities remains an ongoing area of research, they can be used to shed light on the mechanisms that lead to such crises^{17,60}. Towards to this aim, our first illustrative example is an eventdriven agentbased model approximating the dynamics of a simple financial market with mimesis proposed by Omurtag and Sirovich^{58}. The ABM describes the interactions of a large population of, say N, financial traders. Each agent is described by a realvalued state variable X_{i}(t) ∈ (−1, 1) associated to their tendency to buy (positive values) or sell (negative values) stocks in the financial market according to constantly updated financial news, as well as to their interactions with the other traders^{58}. The ith agent acts, i.e., buys or sells, only when its state X_{i} crosses one of the decision boundaries/thresholds X = ±1. As soon as an agent i buys or sells, the agent’s state is forthwith reset to zero.
In the absence of any incoming good news \({I}_{i}^{\,+}\) or bad news \({I}_{i}^{\,}\), the preference state exponentially decays to zero with a constant rate γ. Thus, each agent is governed by the following SDE:
The effect of information arrivals \({I}_{i}^{\,\pm }(t)\) is represented by a series of instantaneous positive/negative “discrete jumps” of size ϵ^{±}, arriving randomly at Poisson distributed times \({t}_{{k}^{+}},\,{k}^{+}=1,2,\ldots \,\) and \({t}_{{k}^{}},\,{k}^{}=1,2,\ldots \,\), with average rates of arrival ν^{+}(t) and ν^{−}(t), respectively. Furthermore, the dynamics of each agent are driven by arrivals of two types of information: exogenous (ex) (e.g., publicly available financial news), as well as an endogenous (en) stream of information arising from the social connections of the agents, so that
A tunable parameter g embodies the strength of mimesis: the extent to which arriving information affects the willingness or apprehension of the agent to buy or sell. For this model, the term \({\nu }_{en}^{\,\pm }\) is set to be the same for all agents and is influenced by the perceived overall buying R^{+}(t) and selling R^{−}(t) rates:
where R^{±}(t) are defined as the fraction of agents buying or selling per unit of time Δt:
where \({T}_{i}^{\,\pm }\) are the instants at which the ith agent crosses the decision boundary ± 1.
In Fig. 1a, we depict the mean preference state for N = 50,000 agents for values of the mimesis strength g = 35, 40, 42, 45, 46, 47. In Fig. 1b, c, we depict representative trajectories of the time evolution of the agent probability density distribution (pdf) for g = 45 and g = 47, respectively. We see that the simulations exhibit a tipping point that arises at a parameter value g ≈ 45.5. At the neighborhood of this tipping point, due to the inherent stochasticity of the mimetic trading process, emanate “financial bubbles”, where all agents hurry to buy assets (see Fig. 1a). The ABM model also predicts financial crashes in regimes of the phasespace where the mean value of the mesoscopic density field is negative, and the agents rush to sell (for more details see ref. ^{56}).
A concise analytical mesoscopic description of the population dynamics was derived by Omurtag and Sirovich in ref. ^{58}. The model, reported here, is a FokkerPlancktype (FP) IPDE for the agent pdf ρ(x, t), given by:
where μ and σ are drift and diffusivity timedependent parameters, respectively, δ is the Dirac delta and J^{±} are integral operators accounting for the agents crossing the decision boundaries.
Further details about the derivation of the FP equation (5) are presented briefly in the section A of the Supplementary Information (SI).
ML mesoscopic IPDE surrogate for the financial ABM
In ref. ^{56}, we showed that the analytical ROM IPDE in Eq. (5) nontrivially underestimates the location of the tipping point with respect to the parameter g, defined in Eq. (3). Here we show how one can achieve a better approximation through datadriven blackbox surrogates. Based on data generated as in section E1 of the Supplementary Information (SI), for learning the righthandside operator of the IPDE, we have considered the relevant features that we found with Automatic Relevance Determination (see the section “Methods” and section E2 of the Supplementary Information). We used the following (blackbox) mesoscopic model for the dynamic evolution of the density ρ:
where I^{+}, I^{−} are integrals in a small neighborhood of the boundaries (see section A of the Supplementary information (SI)). Here, for learning the RHS of the blackbox IPDE (6), we implemented two different structures, namely (a) a feedforward neural network (FNN)^{62,63,64}; and (b) a Random Projection Neural Network^{47,65,66,67,68} in the form of Random Fourier Feature network (RFF)^{69}. The two network structures and their training protocols are described in more detail in the sections D2 and D3 of the Supplementary information (SI). The two alternative machine learning schemes, on the test set, obtain similar performance in terms of accuracy. In terms of the mean absolute error (MAE) the FNN got 1.10E−04 and the RFF got 1.09E−04. For mean squared error (MSE), the FNN got 2.60E−08 and the RFF got 2.53E−08. For the Regression Pearson correlation R, FNN got 0.9866 and RFF 0.9873. The main notable difference between the two schemes is in the computational time needed to perform the training, since remarkably, the training of the RFF, which required 26.06 (s), turned out to be at least 50 times faster than the one required for the deeplearning scheme, which required 1488.33 (s).
Bifurcation analysis of the mesoscopic IPDE for the financial ABM
To locate the tipping point, we have performed bifurcation analysis, using both MLidentified IPDE surrogates, as discussed in section “Methods”. Furthermore, we compared the derived bifurcation diagram(s) and tipping point(s) with what was obtained in refs. ^{52,56} using the EF approach (see in the section C of the Supplementary Information (SI) for a very brief description of the EF approach). As shown in Fig. 2a, the two ML schemes approximate visually accurately the location of the tipping point in parameter space. However, the FNN scheme fails to trace accurately the actual coarsescale unstable branch, near which simulations blow up extremely fast. More precisely, the analytical FP predicts the tipping point at g* = 41.90 with corresponding steadystate \({\bar{X}}^{*}=0.1607\) and the EF at g* = 45.60 and \({\bar{X}}^{*}=0.1627\); our FNN predictions are at g* = 45.77 and \({\bar{X}}^{*}=0.1644\), the RFF ones at g* = 45.34 and \({\bar{X}}^{*}=0.1684\).
Macroscopic physical observables and latent datadriven observables via DMAPs
An immediate physically meaningful candidate observable is the first moment \(\bar{X}\) of the agent distribution function (as also shown in ref. ^{23}).
As simulations of the ABM show (see the inset in Fig. 2b), the mean preference state \(\bar{X}\), is onetoone with another physically meaningful observable, the buying rate R^{+}. We also used the DMAPs algorithm, to discover datadriven macroscopic observables. In our case, DMAPs applied to collected data (see section F1 of the Supplementary Information (SI) for a detailed description of how the data were collected), discovers a 1D latent variable ψ_{1} that is itself onetoone with \(\bar{X}\), see Fig. 2b. The locallinear regression algorithm proposed in^{70} was applied to make sure that all the higher eigenvectors can be expressed as locallinear combinations of ψ_{1} and thus they do not span independent directions. Figure 2c illustrates that the normalized leaveoneout error, denoted as r_{k}, is small for ψ_{2}, …, ψ_{10} suggesting they are all dependent/harmonics of ψ_{1}.
Therefore, any of the three macroscopic observables (two physical and one datadriven) can be interchangeably used to study the collective behavior of the model.
Learning the meanfield SDE and performing bifurcation analysis for the financial ABM
Here, for our illustrations, we learned parameterdependent SDEs for all of the three coarse variables we mentioned, namely the physically meaningful variables \(\bar{X}\), R^{+}, and the DMAPs coordinate ψ_{1}. In the main text, we report the results for the identified SDEs in terms of the DMAPs coordinate ψ_{1} and in section F3 of Supplementary Information (SI) we report the results for the identified SDEs with respect to the \(\bar{X}\) and R^{+} (see Supplementary Fig. 1) .
Given this trained macroscopic SDE surrogate, the drift term (deterministic component) of the identified dynamics was used to construct the bifurcation diagram with AUTO^{71} (see Fig. 2d). A saddlenode bifurcation was identified for g* = 45.77 where \({\psi }_{1}^{\star }=0.021\). The estimated critical parameter value from the SDE is in agreement with our previous work (g ≈ 45.60^{56}). Details pertaining to the neural networks’ architectures used to identify the SDE are provided in section F5 of Supplementary Information (SI).
Rareevent analysis/UQ of catastrophic shifts (financial “bubbles”) via the identified meanfield SDE
Given the identified steady states at a fixed value of g we performed escape time computations. For g = 45.25, we estimated the average escape time needed for a trajectory initiated at the stable steady state to reach \(\bar{X}=0.3\), i.e., sufficiently above the unstable branch. As shown in Fig. 2b, ψ_{1} and R^{+} are effectively onetoone with \(\bar{X}\), and we can easily find the corresponding critical values for ψ_{1} = −0.01 (flipped) and R^{+} = 0.16. We now report a comparison between the escape times of an SDE identified based on the DMAPs coordinate ψ_{1} and those of the full ABM. In section F3 of Supplementary Information (SI) we also report the escape times of the SDE for \(\bar{X}\) and R^{+} observables. To estimate these escape times we sampled a large number (10,000 in our case) of trajectories. In section F4 of Supplementary Information (SI) we also include the escape time computation by using the closedform formula for the 1D case. The computation there was performed numerically by using quadrature and the milestoning approach^{22}.
In Fig. 2e, the histograms of the escape times for the identified SDE trained on ψ_{1} for g = 45.25 and g = 43.75 are shown. In Fig. 2f, we also illustrate the empirical histogram of escape times of the full ABM for g = 45.25. The estimated values for the mean and standard deviation, as computed with temporal simulations from the SDE trained on the DMAPs variable ψ_{1} for g = 45.25, g = 43.75 and the full ABM at g = 45.25 are here reported in Table 1.
As shown, the mean escape time of the full ABM is a factor of five larger than that estimated by the simplified SDE model in ψ_{1} for g = 45.25 (still within an order of magnitude!). The SDE model for g = 43.75 gives an escape time comparable to the one of the ABM for g = 45.25. Given that the escape times change exponentially with respect to the parameter distance from the actual tipping point, a small error in the identified tipping point easily leads to large (exponential) discrepancies in the estimated escape times.
Computational cost for the financial ABM
We compared the computational cost required to estimate escape times with many stochastic temporal simulations, through the full ABM and the identified meanfield SDE. To fairly compare the computational costs, we computed the escape times with the ABM for g = 45.25, and that of the SDE for g = 43.75, since the two distributions of the escape times are more comparable. The estimation in both cases was conducted on Rockfish (a communityshared cluster at Johns Hopkins University) by using a single core with 4GB RAM. For the 10,000 sampled stochastic trajectories, the total computational for the identified coarse SDE in ψ_{1} was \(33.56\min\) and the average time per trajectory, \(3.36\times 1{0}^{3}\min\). The mean time per function evaluation was approximated as the ratio of mean time per trajectory over mean number of iterations.
For the ABM, the total computational time needed was 18.56 days and the mean time per trajectory was \(2.67\min\). Therefore, the total computational time for computing the escape time with the SDE model in ψ_{1} was around 800 times faster than the ABM. This highlights the computational benefits of using the reduced surrogate models in lieu of the full ABM for escape time computations.
Case study 2: Tipping points in a compartmental epidemic model on a complex network
In our second illustrative example, a compartmental epidemic ABM on a social network^{59}, individuals are characterized by three discrete states: Susceptible (S), Infected (I) and Recovered (R). The model is implemented through a "caricature of a social network" approximated by an ErdosRényi network with N = 10,000 nodes. The probability of having a connection between two nodes picked randomly is p = 0.0008 (as proposed in^{59}). The evolution rules are the following:

Rule 1 (S → I): Susceptible individuals may become infected upon contact with infected individuals, with probability P_{S→I} = λ. This tunable parameter is “tracked” for studying the outcomes of abrupt changes in the macroscopic behavior.

Rule 2 (I → R): The transition between I and R happens with a probability P_{I→R} = μ([I]). The probability of recovery depends, at each time step, on the overall density of infected individuals [I], according to the function^{59}):
$$\mu ([I])=0.3\left(1\frac{1}{1+\exp (9([I]0.5))}\right).$$(7)Such a nonlinear function for the probability of recovery has also been used in other works to express the heterogeneity in the “environment” around each individual (see also the discussion in ref. ^{59}).

Rule 3 (I → R): A recovered individual (R) loses its immunity and becomes susceptible (S) with a fixed probability \({P}_{R\to S}=\epsilon=\frac{1}{5}\). This condition expresses the case of temporal immunity.
For further details about the construction of the ErdösRényi network, see section B of the Supplementary Information (SI).
The above rules establish a complex stochastic microscopic model that change the state of each individual over time. In order to describe the model at a macroscopic (emergent) level, let us represent the overall density of susceptible, infected, and recovered individuals as [S], [I], and [R], respectively. In Fig. 1d–f, we depict the stochastic trajectories of the overall densities [S], [I], [R] of the N = 10,000 agents for values of the rate of infection λ = 0.27, 0.28, 0.29, 0.30, 0.31, 0.32. For such macroscopic observables, a simple analytical closure can be found assuming that a uniform and homogeneous network is a good approximation. This means assuming that (a) the degree of each node practically coincides with the mean degree of the network, denoted as \(z={\mathbb{E}}(k)\); (b) that the probabilities of two connected nodes being in a susceptible and infected state, respectively, are independent of each other. The resulting mean field model reads^{59}:
For other higherorder analytical macroscopic pairwise closures, such as the Bethe Ansatz, the Kirkwood approximation and Ursell expansion, the interested reader can consult^{59}.
ML macroscopic meanfield surrogates for the epidemic ABM
It is well known that for dynamics evolving on complex networks, a closedform, analytically derived meanfield approximation in Eq. (8) is usually not accurate^{59}. Here, we show how one can achieve a better approximation through the construction of effective mean fieldlevel ML surrogate models. We will start with the identification, from data, of a mean fieldlevel effective SIR model. Then, following the proposed approach, we identify—again from data—an effective onedimensional SDE to model the stochastic dynamics close to the tipping point and quantify the probability of occurrence of an outbreak where all the population becomes infected.
Given the highfidelity data collected from the epidemic ABM, as described in section G1 of the Supplementary Information (SI), to learn the ML mean field SIR surrogate we used two coupled feedforward neural networks (FNN), labeled F_{S} and F_{I}, each with two hidden layers with 10 neurons for each layer, for learning a blackbox evolution for the effective dynamics of the two macroscopic densities [S] and [I], that reads:
with the constraint [S] + [I] + [R] = 1. We remind the reader that the parameter λ, representing the probability of a susceptible individual to get infected, is tracked for bifurcation analysis purposes. The training process (see sections D2 and D3 of Supplementary Information (SI)) results to a MAE of 7.27e−04 and a MSE of 1.27e−06 on the test set. The regression error for the two networks was R(F_{S}) = 0.9996 and R(F_{I}) = 0.9992.
Bifurcation analysis of the ML meanfield SIR surrogates
To locate the tipping point, we have performed bifurcation analysis using the ML meanfield SIR surrogates. For our illustration, we also compare the derived bifurcation diagram(s) and tipping point(s) with those obtained in^{59} using the EF approach and the analytically derived meanfield SIR model given by Eq. (8). As shown in Fig. 3a, the ML SIRS surrogate approximates adequately the location of the tipping point in parameter space, as well as the entire bifurcation diagram as constructed with the EF approach. Additionally, it outperforms the statisticalmechanicsderived meanfield approximation given by Eq. (8). More precisely, the statisticalmechanicsderived meanfield SIRS model predicts the tipping point at λ* ≈ 0.166 with corresponding steadystate ([S]*, [I]*) = (0.138, 0.449) and the EF at λ* = 0.289 with corresponding steadystates ([S]*, [I]*) = (0.138, 0.451); our ML meanfield SIR surrogate predictions are at λ* = 0.304 with corresponding steadystates ([S]*, [I]*) = (0.135, 0.456).
Macroscopic physical observables and datadriven observables via DMAPs for the epidemic ABM
Two immediate physically meaningful candidate observables are the densities [S], [I]. However, close to the saddle node bifurcation the system is effectively onedimensional and [S] and [I] are effectively onetoone with each other in the longterm dynamics, eventually taking place on the slow eigenvector of the stable steady state.
We also demonstrate this via the DMAPs algorithm. In our case, DMAPs applied to the collected data (see section G2 of the Supplementary Information (SI) for more details), discovers a onedimensional latent variable parameterized by ϕ_{1} that is also onetoone with [I] (see Fig. 3b). The locallinear regression algorithm proposed in ref. ^{70} was also applied to confirm that the remaining eigenvectors are harmonics of ϕ_{1} and thus they do not span independent directions, see Fig. 3c. This confirms that the emergent ABM dynamics close to the tipping point lie on onedimensional manifold.
Learning the effective meanfieldlevel SDE and performing bifurcation analysis for the epidemic ABM
Here, for our illustration, we learned a onedimensional parameterdependent SDE for the DMAPs coordinate ϕ_{1}.
Given this trained macroscopic SDE surrogate, the drift term (deterministic component) of the identified dynamics was used to construct the bifurcation diagram with AUTO (see Fig. 3d). A saddlenode bifurcation was identified for λ* = 0.294 where \({\phi }_{1}^{\star }=0.006\). The estimated critical parameter value from the SDE is in agreement with our previous work (λ ≈ 0.289^{59}).
Rareevent analysis/UQ of catastrophic shifts via the identified SDE
Given the identified steady states at a fixed value of λ we performed escape time computations. For λ = 0.29, we estimated the average escape time needed for a trajectory initiated at the stable steady state to reach [I] = 0.662, a value sufficiently above the unstable branch. The corresponding value for the DMAPs coordinate was ϕ_{1} = −0.007. We now report a comparison between the escape times of the SDE identified based on the DMAPs coordinate ϕ_{1} and those of the epidemic ABM. For this model we also sampled 10,000 stochastic trajectories.
In Fig. 3e, the histograms of the escape times for the SDE trained on ϕ_{1} are shown for λ = 0.29 and λ = 0.285. For the full epidemic ABM in Fig. 3f, we depict the histogram of escape times for λ = 0.29.
The estimated values for the mean and standard deviation, as computed with temporal simulations from the SDE trained on the Diffusion Map variable ϕ_{1} for λ = 0.29, are reported in Table 2.
As shown, the mean escape time of the full ABM is four times larger than that estimated by the simplified SDE model for λ = 0.29 (still within an order of magnitude). The SDE model for λ = 0.285 gives an escape time comparable to the one of the ABM for λ = 0.29.
As we mentioned earlier, the escape times change exponentially depending on the parameter value. Therefore, a small error in the estimated location of the tipping point can lead to large discrepancies in the estimated escape times.
Computational cost for the epidemic ABM
We compared the computational time required to estimate escape times with the full ABM and the identified meanfield SDE. To compare the computational times, we computed the escape times with the ABM for λ = 0.29, and that of the SDE for λ = 0.285, since the mean escape times there are more similar. The estimation of the computational cost for both models (SDE and ABM) was conducted in Matlab. For the 10,000 stochastic trajectories, the total computational time for the identified coarse SDE was ~1 min and the one for the full epidemic ABM was ~16 h.
Discussion
Performing uncertainty quantification of catastrophic shifts, designing control policies for them^{52,56} and thus eventually preventing them, is one of the biggest challenges of our times. Climate change and extinction of ecosystems, outbreak of pandemics, economical crises, all can be attributed to both systematic changes and stochastic perturbations that, close to tipping points, can drive the system abruptly towards another regime that might be catastrophic. Such tipping points are most often associated with underlying bifurcations. Hence, the systematic identification of the mechanisms—types of bifurcations—that govern such shifts, and the quantification of their occurrence probability, is of utmost importance. Towards this aim, as real experiments in the large scale can be difficult, or impossible to perform (not least due to ethical reasons) mathematical models and especially highfidelity largescale agentbased models are a powerful tool in our arsenal to build informative “digital twins” (see also the discussion in refs. ^{3,4}). However, due to the “curse of dimensionality” that pertains to the dynamics of such largescale “digital twins”, the above task remains computationally demanding and challenging.
Here, we proposed a machinelearningbased framework to infer tipping points in the emergent dynamics of largescale agentbased simulators. In particular, we proposed and critically discussed the construction of mesoscopic and coarser/meanfieldlevel ML surrogates from highfidelity spatiotemporal data for: (a) the location of bifurcation points and their type, and (b) the quantification of the probability distribution of the occurrence of the catastrophic shifts. As our illustration, we used two largescale ABMS: (1) an eventdriven stochastic agentbased model describing the mimetic behavior of traders in a simple financial market, and (2) an epidemic ABM on a complex social network. In both ABMs tipping points arise, which give rise to financial bubbles and epidemic outbreaks, respectively. While analytical surrogates may provide some physical insight for the emergent dynamics, they introduce biases in the accurate numerical bifurcation analysis, and thus also in the accurate detection of tipping points, especially when dealing with IPDEs. On the other hand, discovering sets of variables that span the lowdimensional latent space on which the emergent dynamics emerge—via manifold learning—and then learning surrogate meanfieldlevel SDEs in these variables, offers an attractive and computationally “cheap” alternative. Such an approach for the construction of tipping pointtargeted ROMs can provide an accurate approximation of the tipping point. However, this physicsagnostic approach, may result in macroscopic observables that are not, at least not explicitly, physically interpretable.
Clearly, different modeling tasks are best served by different coarsescale surrogates. This underpins the importance of selecting the right ML modeling approach for different systemlevel tasks, being conscious of the pros and cons of the different scale surrogates.
Further extensions of our framework may include the learning of a more general class of SDEs (e.g., based on Lévy process)^{72}; and possibly moving towards learning effective SPDEs or even fractional evolution operators that could lead to more informative surrogate models^{25}.
There is a point in noting that our work here does not focus on constructing (or validating) early warning systems based on realworld data. It is, however, conceivable that some of our datadriven, DMAPs identified collective variables may serve as candidate coordinates for such earlywarning systems^{73}. Our main target has been to show how one can systematically construct reducedorder models via machine learning for understanding, analyzing the mechanisms governing the emergence of tipping points, and quantifying the probability of occurrence of rare events around them, from detailed ABM simulations—a problem that can suffer from the curse of dimensionality.
Methods
Given high dimensional spatiotemporal trajectory data acquired through ABM simulations, the main steps of the framework are summarized as follows (see also Fig. 4, for a schematic):

a.
Discover lowdimensional latent spaces, on which the emergent dynamics can be described at the mesoscopic or the macroscopic scale.

b.
Identify, via machinelearning, blackbox mesoscopic IPDEs, ODEs, or (after further dimensional reduction), macroscopic meanfield SDEs.

c.
Locate tipping points by exploiting numerical bifurcation analysis of the different surrogate models.

d.
Use the identified (NNbased) surrogate meanfield SDEs to perform rareevent analysis (uncertainty quantification) for the catastrophic transitions. This is done here in two ways: (i) performing repeated bruteforce simulations around the tipping points, (ii) for this effectively 1D problem, using explicit statistical mechanical (FeynmanKac) formulas for escape time distributions.
In what follows, we present the elements of the methodology. For further details about the methodology and implementation, see in the Supplementary information (SI).
Discovering lowdimensional latent spaces
The computational modeling of complex systems featuring a multitude of interacting agents poses a significant challenge due to the enormous number of potential states that such systems can have. Thus, a fundamental step, for the development of ROMs that are capable of effectively capturing the collective behavior of ensembles of agents is the discovery of an embedded, in the highdimensional space, lowdimensional manifold and an appropriate set of variables that can usefully parametrize it.
Let’s denote, by \({{{{{{{{\bf{X}}}}}}}}}_{k}\in {{\mathbb{R}}}^{D}\), k = 1, 2, … the highdimensional state of the ABM at time t. The goal is then to project/map the highdimensional data onto lowerdimensional latent manifolds \({{{{{{{\mathcal{M}}}}}}}}\subset {{\mathbb{R}}}^{D}\), that can be defined by a set of coarsescale variables. The hypothesis of the existence of this manifold is related to the existence of useful ROMs and vice versa.
Here, to discover such a set of coarsegrained coordinates for the latent space, we used DMAPs^{26,28,54} (see section D1 of the Supplementary Information (SI) for a brief description of the DMAPs algorithm).
For both ABMs, we have some a priori physical insight for the mesoscopic description. For the financial ABM, one can for example use the probability density function (pdf) \(\rho (X)dx={\mathbb{P}}(X(t)\in [x,x+dx])\) across the possible states X_{k} in space. Thus, the continuum pdf constitutes a spatially dependent mesoscopic field that can be modeled by a FP IPDE as explained above. For the epidemic ABM, there is a physical insight on the macroscopic meanfield description, which is the wellknown meanfield SIRS model. Multiscale macroscopic descriptions can also be constructed including higherorder closures^{59}. Alternatively, one can also collect “enough” statistical moments of the underlying distribution such as the expected value, variance, skewness, kurtosis, etc. Nevertheless, the collected statistics may not automatically provide insight into their relevance in the effective dynamics and a further feature selection/sensitivity analysis may be needed.
Focusing on a reduced set of coarsescale variable is particularly relevant when there exists a significant separation of time scales in the system’s dynamics. By selecting only a few dominant statistics, one can effectively summarize the behavior of the system at a coarser level.
The choice of the scale and details of coarsegrained description, leads to different modeling approaches. For example, focusing at the mesoscale for the population density dynamics, we aim at constructing a FPlevel IPDE for the financial ABM, and a meanfield SIR surrogate for the epidemic ABM. At an even coarser scale, e.g., for the first moment of the distribution, and taking into account the underlying stochasticity, a natural first choice is the construction of a meanfield macroscopic SDE. Here, we construct surrogate models via machine learning at both these distinct coarsegrained scales.
Learning mesoscopic IPDEs via neural networks
As we have discussed in the introduction, the identification of evolution operators of spatiotemporal dynamics using machine learning tools, including deep learning and Gaussian processes, represents a wellestablished field of research. The main assumption here is that the emergent dynamics of the complex system under study on a domain \({{\Omega }}\times [{t}_{0},{t}_{end}]\subseteq {{\mathbb{R}}}^{d}\times {\mathbb{R}}\) can be modeled by a system of say m IPDEs in the form of:
where F^{(i)}, i = 1, 2, …m are m nonlinear integrodifferential operators; u(x, t) = [u^{(1)}(x, t), …, u^{(m)}(x, t)] is the vector containing the spatiotemporal fields, \({{{{{{{{\mathcal{D}}}}}}}}}^{{{{{{{{\boldsymbol{\nu }}}}}}}}}{{{{{{{\bf{u}}}}}}}}({{{{{{{\bf{x}}}}}}}},t)\) is the generic multiindex νth order spatial derivative at time t:
\({I}_{1}^{(i)},{I}_{2}^{(i)},\ldots \,\) are a collection of integral features on subdomains \({{{\Omega }}}_{1}^{(i)},{{{\Omega }}}_{2}^{(i)},\cdots \subseteq {{\Omega }}\):
\({K}_{j}^{(i)}:{{\mathbb{R}}}^{d}\times {{\mathbb{R}}}^{m}\mapsto {{\mathbb{R}}}^{d}\) are nonlinear maps and \({{{{{{{\boldsymbol{\varepsilon }}}}}}}}\in {{\mathbb{R}}}^{p}\) denotes the (bifurcation) parameters of the system. The righthandside of the ith IPDE depends on say, a number of γ^{(i)} variables and on bifurcation parameters from the set of features:
At each spatial point x_{q}, q = 1, 2, …, M and time instant t_{s}, s = 1, 2, …, N, a single sample point (an observation) in the set \({{{{{{{{\mathcal{S}}}}}}}}}^{(i)}\) for the ith IPDE can be described by a vector \({Z}_{j}^{(i)}\equiv {Z}_{(q,s)}^{(i)}\in {{\mathbb{R}}}^{{\gamma }^{(i)}}\), with j = q + (s − 1)M. Here, we assume that such mesoscopic IPDEs in principle exist, but they are not available in closedform. Henceforth, we aim to learn the macroscopic laws by employing a Feedforward Neural Network (FNN), in which the effective input layer is constructed by a finite stencil (sliding over the computational domain), mimicking convolutional operations where the applied “filter” involves values of our field variable(s) u^{(i)} on the stencil, and returns features \({Z}_{j}^{(i)}\in {{{{{{{{\mathcal{S}}}}}}}}}^{(i)}\) of these variables at the stencil centerpoint, i.e., spatial derivatives as well as (local or global) integrals (see Fig. 5a for a schematic).
The FNN is fed with the sample points \({Z}_{j}^{(i)}\in {{\mathbb{R}}}^{{\gamma }^{(i)}}\) and its output is an approximation of the time derivative u_{t}. In sections D2 and D3 of the Supplementary Information (SI) we describe two different approaches (gradient descent and random projections) to train such a FNN. For alternative operatorlearning approximation methods, see e.g., ref. ^{74} and ref. ^{51}.
Remark on learning meanfield ODEs
The proposed framework can be also applied for the simpler task of learning a system of m ODEs in terms of the m variables u = (u^{(1)}, u^{(2)}, …, u^{(m)}), thus learning i = 1, …, m functions:
where \({{{{{{{\bf{p}}}}}}}}\in {{\mathbb{R}}}^{k}\) denotes the parameter vector.
Feature selection with automatic relevance determination
Training the FNN with the “full” set of inputs \({{{{{{{{\mathcal{S}}}}}}}}}^{(i)}\subseteq {{\mathbb{R}}}^{{\gamma }^{(i)}}\), described in Eq. (13), consisting of all local mean field values as well as all their coarsescale spatial derivatives (up to some order ν) is prohibitive due to the curse of dimensionality. Therefore, one important task for the training of the FNN is to extract a few “relevant”/dominant variable combinations. Towards this aim, we used Automatic Relevance Determination (ARD) in the framework of Gaussian processes regression (GPR)^{29}. The approach assumes that the collection of all observations \({{{{{{{{\bf{Z}}}}}}}}}^{(i)}=({Z}_{1}^{(i)},{Z}_{2}^{(i)},\ldots,{Z}_{MN}^{(i)})\), of the features \({z}_{l}\in {{{{{{{{\mathcal{S}}}}}}}}}^{(i)}\), are a set of random variables whose finite collections have a multivariate Gaussian distribution with an unknown mean (usually set to zero) and an unknown covariance matrix K. This covariance matrix is commonly formulated by a Euclidean distancebased kernel function k in the input space, whose hyperparameters are optimized based on the training data. Here, we employ a radial basis kernel function (RBF), which is the default kernel function in Gaussian process regression, with ARD:
\({{{{{{{{\boldsymbol{\theta }}}}}}}}}^{(i)}=[{\theta }_{0}^{(i)},{\theta }_{1}^{(i)},\ldots,{\theta }_{{\gamma }^{(i)}}^{(i)}]\) are a (γ^{(i)} + 1)dimensional vector of hyperparameters. The optimal hyperparameter set \({\tilde{{{{{{{{\boldsymbol{\theta }}}}}}}}}}^{(i)}\) can be obtained by minimizing a negative log marginal likelihood over the training data set (Z^{(i)}, Y^{(i)}), with inputs the observation Z^{(i)} of the set \({{{{{{{{\mathcal{S}}}}}}}}}^{(i)}\) and corresponding desired output given by the observation Y^{(i)} of the time derivative \({u}_{t}^{(i)}\):
As can be seen in Eq. (15), a large value of θ_{l} nullifies the difference between target function values along the lth dimension, allowing us to designate the corresponding z_{l} feature as “insignificant”. Practically, in order to build a reduced input data domain, we define the normalized effective relevance weights \({W}_{r}^{(i)}(\cdot )\) of each feature input \({z}_{l}\in {{{{{{{{\mathcal{S}}}}}}}}}^{(i)}\), by taking:
Thus, we define a small tolerance tol in order to disregard the components such that \({W}_{r}^{(i)}({z}_{l}) \, < \, tol\). The remaining selected features (\({W}_{r}^{(i)}({z}_{l})\ge tol\)) can still successfully (for all practical purposes) parametrize the approximation of the righthandside of the underlying IPDE.
Macroscopic meanfield SDEs via neural networks
Here, we present our approach for the construction of embedded surrogate models in the form of meanfield SDEs. Under the assumption that we are close (in phase and parameter space) to a previously located tipping point, we can reasonably assume that the effective dimensionality of the dynamics can be reduced to the corresponding normal form. We already have some qualitative insight on the type of the tipping point, based for example on the numerical bifurcation calculations that located it (e.g., for the financial ABM from the analytical FP IPDE, from our surrogate IPDE, or from the EF analysis^{52,56}), while for the epidemic ABM from the EF analysis in ref. ^{59}. For both these two particular problem, we have found that the tipping point corresponds to a saddlenode bifurcation.
Given the nature of the bifurcation (and the single variable corresponding normal form) we identify a onedimensional SDE, driven by a Wiener process, from data. We note that learning higherorder such SDEs, or SDEs based on the more general Lévy process and the OrnsteinUhlenbeck process^{75}, is straightforward.
For a diffusion process with drift, say X_{t} = {x_{t}, t > 0}, the drift, μ(x_{t}) and diffusivity σ^{2}(x_{t}) coefficients over an infinitesimally smalltime interval dt, are given by:
where, δx_{t} = x_{t+δt} − x_{t}.
The 1D SDE driven by a Wiener process W_{t} reads:
Here, for simplicity, we assume that the onedimensional parameter ε, enters into the dynamics, via the drift and diffusivity coefficients. Note that the parameter ε can be either the parameter g introduced in Eq. (3) or the parameter λ for the epidemic ABM (representing the probability that a susceptible individual may get infected). Our goal is to identify the functional form of the drift μ(x, ε) and the diffusivity σ(x, ε) given noisy data close to the tipping point via machine learning. For the training, the data might be collected from either longtime trajectories or short bursts initialized at scattered snapshots, as in the EF framework. These trajectories form our data set of inputoutput pairs of discretetime maps. A data point in the collected data set can be written as \(({x}_{0}^{(k)},{h}^{(k)},{x}_{1}^{(k)}{\varepsilon }^{(k)})\), where \({x}_{0}^{(k)}\) and \({x}_{1}^{(k)}\) measures two consecutive states at \({t}_{0}^{(k)}\) and \({t}_{1}^{(k)}\) with (small enough) time step \({h}^{(k)}={t}_{1}^{(k)}{t}_{0}^{(k)}\) and ε^{(k)} is the parameter value for this pair. Based on the above formulation, going to \({x}_{1}^{(k)}\) from \({x}_{0}^{(k)}\) by:
Here, for the numerical integration of the above equation to get stochastic realizations of \({x}_{1}^{(k)}\), we assume that the EulerMaruyama numerical scheme can be used, reading:
where \({{\Delta }}{W}^{(k)}={W}_{{t}_{1}^{(k)}}{W}_{{t}_{0}^{(k)}}\in {\mathbb{R}}\) is a onedimensional random variable, normally distributed with expected value zero and variance h^{(k)}.
Considering the point \({x}_{1}^{(k)}\) as a realization of a random variable X_{1}, conditioned on \({x}_{0}^{(k)}\) and h^{(k)}, drawn by a Gaussian distribution of the form:
we approximate the drift \(\mu ({x}_{0}^{(k)};{\varepsilon }^{(k)})\) and diffusivity \(\sigma ({x}_{0}^{(k)};{\varepsilon }^{(k)})\) functions by simultaneously training two neural networks, denoted as μ_{θ} and σ_{θ}, respectively. This training process involves minimizing the loss function:
which is derived in order to maximize the loglikelihood of the data and where θ denotes the trainable parameters (e.g., weights and biases of the neural networks μ_{θ} and σ_{θ}). A schematic of the Neural Network, based on EulerMaruyama, is shown in Fig. 5b.
Locating tipping points via our surrogate models
In order to locate the tipping point, based on either the mesoscopic IPDE or the embedded meanfield 1D SDE model, we construct the corresponding bifurcation diagram in its neighborhood, using pseudoarclength continuation as implemented in numerical bifurcation packages. For the identified SDE, we used its deterministic part, i.e., the drift term, to perform continuation. The required Jacobian of the activation functions of the neural network is computed by symbolic differentiation. Note that this is just a validation step (we already know the location and nature of the tipping point).
Rareevent analysis/UQ of catastrophic shifts
Given a sample space Ω, an index set of times T = {0, 1, 2, … } and a state space S, the first passage time, also known as mean exit time or mean escape time, of a stochastic process x_{t}: Ω × T ↦ Son a measurable subset A ⊆ S is a random variable which can be defined as
where ω is a sample out of the space Ω. One can define the mean escape time from A, which works as the expectation of τ(ω):
For a ndimensional stochastic process, as it is the ABM under study, S is typically set to be \({{\mathbb{R}}}^{n}\), and A is usually a bounded subset of \({{\mathbb{R}}}^{n}\). In the case of our local 1D SDE model, the subset A reduces to an open interval (a, b), with the initial condition of this stochastic process x_{0} also chosen in this interval.
We discuss two ways for quantifying the uncertainty of the occurrence of those rareevents. The first, presented in the main text, involves direct computational “cheap” temporal simulations of the 1D SDE, where one gets an empirical probability distribution; the second, presented only in section F4 of the Supplementary Information (SI), is a closedform expression, based on statistical mechanics^{76}, for the mean escape time (assuming an exponential distribution of escape times).
Computation of escape times based on temporal simulations of the 1D SDE
We perform numerical integration of multiple stochastic trajectories of the SDE to obtain an estimation of the desired mean escape time. The algorithm for estimating the escape times of a onedimensional stochastic process with initial condition x_{0} on the interval (a, b) can be described as follows:

1.
Given a fixed time step h > 0, we perform i + 1 numerical integration steps of the SDE until a stopping condition, i.e., x_{t} exits, for the first time t_{i+1}, the interval at a or b.

2.
Record the time τ = t_{i}, which corresponds to a realization of the escape time.

3.
Repeat the above steps 1–2 for K iterations, and each time collect the observed escape time.

4.
Compute the statistical mean value of the collected escape times that corresponds to an approximation of the mean escape time.
In our case, the initial condition x_{0} was set equal to the stable steady state, and the termination condition a or b at which we consider the dynamics escaped/exploded.
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.
Data availability
Data used in this work are publicly available in the GitLab repository https://gitlab.com/nicolasevangelou/agent_based.
Code availability
The code used to produce the findings of this study are publicly available in the GitLab repository https://gitlab.com/nicolasevangelou/agent_based.
References
Kevrekidis, I. G. et al. Equationfree, coarsegrained multiscale computation: enabling microscopic simulators to perform systemlevel analysis. Commun. Math. Sci. 1, 715–762 (2003).
Karniadakis, G. E. et al. Physicsinformed machine learning. Nat. Rev. Phys. 3, 422–440 (2021).
Scheffer, M. & Carpenter, S. R. Catastrophic regime shifts in ecosystems: linking theory to observation. Trends Ecol. Evol. 18, 648–656 (2003).
Scheffer, M. Foreseeing tipping points. Nature 467, 411–412 (2010).
Dakos, V. et al. Ecosystem tipping points in an evolving world. Nat. Ecol. Evol. 3, 355–362 (2019).
Armstrong McKay, D. I. et al. Exceeding 1.5 C global warming could trigger multiple climate tipping points. Science 377, eabn7950 (2022).
Grimm, V. et al. Patternoriented modeling of agentbased complex systems: lessons from ecology. Science 310, 987–991 (2005).
McLane, A. J., Semeniuk, C., McDermid, G. J. & Marceau, D. J. The role of agentbased models in wildlife ecology and management. Ecol. Model. 222, 1544–1556 (2011).
An, L. Modeling human decisions in coupled human and natural systems: review of agentbased models. Ecol. Model. 229, 25–36 (2012).
Russo, L., Russo, P., Vakalis, D. & Siettos, C. Detecting weak points of wildland fire spread: a cellular automata model risk assessment simulation approach. Chem. Eng. Trans. 36, 253–258 (2014).
Iozzi, F. et al. Little Italy: an agentbased approach to the estimation of contact patternsfitting predicted matrices to serological data. PLoS Computat. Biol. 6, e1001021 (2010).
Grefenstette, J. J. et al. Fred (a framework for reconstructing epidemic dynamics): an opensource software system for modeling infectious diseases and control strategies using censusbased populations. BMC Public Health 13, 1–14 (2013).
Kumar, S., Grefenstette, J. J., Galloway, D., Albert, S. M. & Burke, D. S. Policies to reduce influenza in the workplace: impact assessments using an agentbased model. Am. J. Public Health 103, 1406–1411 (2013).
Siettos, C., Anastassopoulou, C., Russo, L., Grigoras, C. & Mylonakis, E. Modeling the 2014 ebola virus epidemic–agentbased simulations, temporal analysis and future predictions for Liberia and Sierra Leone. PLoS Curr. 7, 1–18 (2015).
Kerr, C. C. et al. Covasim: an agentbased model of COVID19 dynamics and interventions. PLOS Comput. Biol. 17, e1009149 (2021).
Faucher, B. et al. Agentbased modelling of reactive vaccination of workplaces and schools against COVID19. Nat. Commun. 13, 1414 (2022).
Farmer, J. D. & Foley, D. The economy needs agentbased modelling. Nature 460, 685–686 (2009).
Buchanan, M. Economics: Meltdown modelling. Nature 460, 680–683 (2009).
LeBaron, B. Agentbased computational finance. Handb. Comput. Econ. 2, 1187–1233 (2006).
Axtell, R. L. & Farmer, J. D. Agentbased modeling in economics and finance: past, present, and future. J. Econ. Lit. https://www.aeaweb.org/articles?id=10.1257/jel.20221319&&from=f (2022).
Deissenberg, C., Van Der Hoog, S. & Dawid, H. Eurace: a massively parallel agentbased model of the European economy. Appl. Math. Comput. 204, 541–552 (2008).
BelloRivas, J. M. & Elber, R. Simulations of thermodynamics and kinetics on rough energy landscapes with milestoning. J. Comput. Chem. 37, 602–613 (2015).
Liu, P., Siettos, C., Gear, C. W. & Kevrekidis, I. Equationfree model reduction in agentbased computations: coarsegrained bifurcation and variablefree rare event analysis. Math. Model. Nat. Phenom. 10, 71–90 (2015).
Zagli, N., Pavliotis, G. A., Lucarini, V. & Alecio, A. Dimension reduction of noisy interacting systems. Phys. Rev. Res. 5, 013078 (2023).
Helfmann, L., Djurdjevac Conrad, N., Djurdjevac, A., Winkelmann, S. & Schütte, C. From interacting agents to densitybased modeling with stochastic PDEs. Commun. Appl. Math. Comput. Sci. 16, 1–32 (2021).
Coifman, R. R. et al. Geometric diffusions as a tool for harmonic analysis and structure definition of data: diffusion maps. Proc. Natl Acad. Sci. USA 102, 7426–7431 (2005).
Nadler, B., Lafon, S., Coifman, R. R. & Kevrekidis, I. G. Diffusion maps, spectral clustering and reaction coordinates of dynamical systems. Appl. Comput. Harmon. Anal. 21, 113–127 (2006).
Coifman, R. R., Kevrekidis, I. G., Lafon, S., Maggioni, M. & Nadler, B. Diffusion maps, reduction coordinates, and low dimensional representation of stochastic systems. Multiscale Model. Sim. 7, 842–864 (2008).
Lee, S., Kooshkbaghi, M., Spiliotis, K., Siettos, C. I. & Kevrekidis, I. G. Coarsescale PDEs from finescale observations via machine learning. Chaos: Interdiscip. J. Nonlinear Sci. 30, 013141 (2020).
Galaris, E., Fabiani, G., Gallos, I., Kevrekidis, I. G. & Siettos, C. Numerical bifurcation analysis of PDEs from lattice Boltzmann model simulations: a parsimonious machine learning approach. J. Sci. Comput. 92, 1–30 (2022).
Balasubramanian, M. & Schwartz, E. L. The isomap algorithm and topological stability. Science 295, 7–7 (2002).
Bollt, E. Attractor modeling and empirical nonlinear model reduction of dissipative dynamical systems. Int. J. Bifurc. Chaos 17, 1199–1219 (2007).
Roweis, S. T. & Saul, L. K. Nonlinear dimensionality reduction by locally linear embedding. science 290, 2323–2326 (2000).
Papaioannou, P. G., Talmon, R., Kevrekidis, I. G. & Siettos, C. Timeseries forecasting using manifold learning, radial basis function interpolation, and geometric harmonics. Chaos: Interdiscip. J. Nonlinear Sci. 32, 083113 (2022).
Kramer, M. A. Nonlinear principal component analysis using autoassociative neural networks. AIChE J. 37, 233–243 (1991).
Chen, W. & Ferguson, A. L. Molecular enhanced sampling with autoencoders: onthefly collective variable discovery and accelerated free energy landscape exploration. J. Comput. Chem. 39, 2079–2102 (2018).
Vlachas, P. R., Arampatzis, G., Uhler, C. & Koumoutsakos, P. Multiscale simulations of complex systems by learning their effective dynamics. Nat. Mach. Intell. 4, 359–366 (2022).
Floryan, D. & Graham, M. D. Datadriven discovery of intrinsic dynamics. Nat. Mach. Intell. 4, 1113–1120 (2022).
Brunton, S. L., Proctor, J. L. & Kutz, J. N. Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proc. Natl Acad. Sci. USA 113, 3932–3937 (2016).
Raissi, M., Perdikaris, P. & Karniadakis, G. E. Machine learning of linear differential equations using Gaussian processes. J. Comput. Phys. 348, 683–693 (2017).
Chen, Y., Hosseini, B., Owhadi, H. & Stuart, A. M. Solving and learning nonlinear PDEs with Gaussian processes. J. Comput. Phys. 447, 110668 (2021).
RicoMartinez, R., Krischer, K., Kevrekidis, I. G., Kube, M. C. & Hudson, J. L. Discretevs. continuoustime nonlinear signal processing of Cu electrodissolution data. Chem. Eng. Commun. 118, 25–48 (1992).
Alexandridis, A., Siettos, C., Sarimveis, H., Boudouvis, A. & Bafas, G. Modelling of nonlinear process dynamics using Kohonen’s neural networks, fuzzy systems and Chebyshev series. Comput. Chem. Eng. 26, 479–486 (2002).
Arbabi, H., Bunder, J. E., Samaey, G., Roberts, A. J. & Kevrekidis, I. G. Linking machine learning with multiscale numerics: datadriven discovery of homogenized equations. Jom 72, 4444–4457 (2020).
Lee, S., Psarellis, Y. M., Siettos, C. I. & Kevrekidis, I. G. Learning black and graybox chemotactic PDEs/closures from agent based Monte Carlo simulation data. J. Math. Biol. 87, 15 (2023).
Dietrich, F. et al. Learning effective stochastic differential equations from microscopic simulations: linking stochastic numerics to deep learning. Chaos: Interdiscip. J. Nonlinear Sci. 33, 023121 (2023).
Fabiani, G., Galaris, E., Russo, L. & Siettos, C. Parsimonious physicsinformed random projection neural networks for initial value problems of odes and index1 daes. Chaos: Interdiscip. J. Nonlinear Sci. 33, 043128 (2023).
Vlachas, P. R. et al. Backpropagation algorithms and reservoir computing in recurrent neural networks for the forecasting of complex spatiotemporal dynamics. Neural Netw. 126, 191–217 (2020).
Bertalan, T., Dietrich, F., Mezić, I. & Kevrekidis, I. G. On learning Hamiltonian systems from data. Chaos: Interdiscip. J. Nonlinear Sci. 29, 121107 (2019).
Li, X., Wong, T.K. L., Chen, R. T. Q. & Duvenaud, D. K. in Proceedings of The 2nd Symposium on Advances in Approximate Bayesian Inference (eds Zhang, C., Ruiz, F., Bui, T., Dieng, A. B. & Liang, D.), Vol. 118 of Proceedings of Machine Learning Research, 1–28 (PMLR, 2020).
Lu, L., Jin, P., Pang, G., Zhang, Z. & Karniadakis, G. E. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nat. Mach. Intell. 3, 218–229 (2021).
Patsatzis, D. G., Russo, L., Kevrekidis, I. G. & Siettos, C. Datadriven control of agentbased models: an equation/variablefree machine learning approach. J. Comput. Phys. 478, 111953 (2023).
Chin, T. et al. Enabling equationfree modeling via diffusion maps. J. Dyn. Differ. Equa. https://doi.org/10.1007/s1088402110127w (2022).
Singer, A., Erban, R., Kevrekidis, I. G. & Coifman, R. R. Detecting intrinsic slow variables in stochastic dynamical systems by anisotropic diffusion maps. Proc. Natl Acad. Sci. USA 106, 16090–16095 (2009).
Evangelou, N. et al. Double diffusion maps and their latent harmonics for scientific computations in latent space. J. Comput. Phys. 485, 112072 (2023).
Siettos, C., Gear, C. W. & Kevrekidis, I. G. An equationfree approach to agentbased computation: bifurcation analysis and control of stationary states. EPL 99, 48007 (2012).
Aris, R. Manners makyth modellers. Chem. Eng. Sci. 46, 1535–1544 (1991).
Omurtag, A. & Sirovich, L. Modeling a large population of traders: mimesis and stability. J. Econ. Behav. Organ. 61, 562–576 (2006).
Reppas, A., De Decker, Y. & Siettos, C. On the efficiency of the equationfree closure of statistical moments: dynamical properties of a stochastic epidemic model on Erdős–Rényi networks. J. Stat. Mech.: Theory Exp. 2012, P08020 (2012).
Sornette, D. Nurturing breakthroughs: lessons from complexity theory. J. Econ. Interact. Coord. 3, 165–181 (2008).
LeBaron, B., Arthur, W. B. & Palmer, R. Time series properties of an artificial stock market. J. Econ. Dyn. control 23, 1487–1516 (1999).
Cybenko, G. Approximation by superpositions of a sigmoidal function. Math. Control, Signals Syst. 2, 303–314 (1989).
Hornik, K., Stinchcombe, M. & White, H. Multilayer feedforward networks are universal approximators. Neural Netw. 2, 359–366 (1989).
Hagan, M. T. & Menhaj, M. B. Training feedforward networks with the marquardt algorithm. IEEE Trans. Neural Netw. 5, 989–993 (1994).
Pao, Y.H. & Takefuji, Y. Functionallink net computing: theory, system architecture, and functionalities. Computer 25, 76–79 (1992).
Jaeger, H. The “echo state” approach to analysing and training recurrent neural networkswith an erratum note. Bonn., Ger.: Ger. Natl Res. Cent. Inf. Technol. GMD Tech. Rep. 148, 13 (2001).
Huang, G.B., Zhu, Q.Y. & Siew, C.K. Extreme learning machine: theory and applications. Neurocomputing 70, 489–501 (2006).
Gauthier, D. J., Bollt, E., Griffith, A. & Barbosa, W. A. Next generation reservoir computing. Nat. Commun. 12, 1–8 (2021).
Rahimi, A. & Recht, B. Random features for largescale kernel machines. In Advances in Neural Information Processing Systems Vol. 20. (2007).
Dsilva, C. J., Talmon, R., Coifman, R. R. & Kevrekidis, I. G. Parsimonious representation of nonlinear dynamical systems through manifold learning: a chemotaxis case study. Appl. Comput. Harmon. Anal. 44, 759–773 (2018).
Doedel, E. J. Auto: a program for the automatic bifurcation analysis of autonomous systems. Proc. Tenth Manit. Conf. Numer. Math. Comput. 30, 265–284 (1981).
Fang, C., Lu, Y., Gao, T. & Duan, J. An endtoend deep learning approach for extracting stochastic dynamical systems with αstable lévy noise. Chaos: Interdiscip. J. Nonlinear Sci. 32, 063112 (2022).
Scheffer, M. et al. Earlywarning signals for critical transitions. Nature 461, 53–59 (2009).
Li, Z. et al. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895 (2020).
Karatzas, I., Karatzas, I., Shreve, S. & Shreve, S. E. Brownian Motion and Stochastic Calculus, Vol. 113 (Springer Science & Business Media, 1991).
Frewen, T. A., Hummer, G. & Kevrekidis, I. G. Exploration of effective potential landscapes using coarse reverse integration. J. Chem. Phys. 131, 10B603 (2009).
Acknowledgements
The US AFOSR FA9550210317 and the US Department of Energy SA220052S001 partially supported I.G.K. and N.E. The PNRR MUR, projects PE0000013Future Artificial Intelligence ResearchFAIR & CN0000013 CN HPC  National Centre for HPC, Big Data and Quantum Computing, Gruppo Nazionale Calcolo ScientificoIstituto Nazionale di Alta Matematica (GNCSINdAM) partially supported C.S. “la Caixa” Foundation Fellowship (ID100010434), code LCF/BQ/AA19/11720048 supported C.P.M.L.
Author information
Authors and Affiliations
Contributions
G.F.: Data curation, formal analysis, investigation, methodology, software, validation, visualization, writing—original draft, writing—review & editing. N.E.: Data curation, formal analysis, investigation, methodology, software, validation, visualization, writing—original draft, writing—review & editing. T.C.: Data curation, methodology, software, writing—original draft. J.M.B.R.: Data curation, formal analysis, investigation, methodology, software, writing—review & editing. C.P.M.L.: Data curation, investigation, methodology, software. C.S.: Conceptualization, formal analysis, supervision, investigation, methodology, software, validation, writing—review & editing. I.G.K.: Conceptualization, formal analysis, supervision, investigation, methodology, validation, writing—review & editing.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Communications thanks Bjorn Sandstede, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. A peer review file is available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Fabiani, G., Evangelou, N., Cui, T. et al. Taskoriented machine learning surrogates for tipping points of agentbased models. Nat Commun 15, 4117 (2024). https://doi.org/10.1038/s41467024480247
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41467024480247
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.