## Introduction

Biological regulation consists of complex networks of dynamical interactions1,2,3,4,5. Transcription factors control production of other transcription factors6,7, kinases the activation of other kinases1,3,4, cells the growth of other cells8,9, and species the population size of other species10. “Network motifs” describe particularly important substructures in biological networks, such as negative feedback, feedforward regulation and cascades2,6,11. The function of network motifs often depends on emergent properties of many parameters, notably delays7,12,13, and previous work showed that delays can lead to dramatic biological changes, such as sustained oscillations12,14,15,16,17. Delays are crucial to behavior in natural18,19,20 and synthetic21,22 genetic oscillators, in development and disease2,7,14,16,17,23,24,25,26,27,28,29,30,31, and in ecological booms and busts8. Despite such examples, there has been no thorough treatment of network motifs with delays included explicitly.

Here we provide a theoretical and practical basis for analyzing network motifs with explicit delays, and we demonstrate the utility of this approach in a variety of biological contexts. Whereas network motifs are typically modeled using ordinary differential equations (ODEs) with variables reacting to one another instantaneously (timed by rate constants), we explore network motif modeling using delay differential equations (DDEs), which have derivatives depending explicitly on the value of variables at times in the past. For example,

$$\dot{x}(t)=\alpha x(t)$$
(1)

is an ODE, and

$$\dot{x}(t)=\alpha x(t-\tau )$$
(2)

is a corresponding DDE, where x(t − τ) represents the value of x at a time τ units in the past, making the effect of x on the current rate of change of x delayed by a time τ. The time scales and delays are thus explicit (Fig. 1), better capturing dynamics where intermediate steps are not fast, as compared to ODE models, which may fail to capture real delays, hide their effects by oversimplification, or require additional variables and parameters to predict complex phenomena17,23,32. Such delay models have been productive in a variety of biological scenarios, such as processes with many intermediate but unimportant steps33,34 and age-structured populations34,35. Network models with delays have also been explored in various contexts36,37,38,39. Intercellular signal transduction, for example, requires multiple time-consuming steps that are not negligible23,32.

With DDEs, multiple steps (“cascades”) within a network can be rigorously simplified into a single step with delay (see below), an approach which has been explored in a variety of biological contexts18,33,34,40,41,42. This makes interpretation of the phenomenology simpler than with ODEs and reduces the number of equations and parameters in the model18,23. This idea can be visualized by depicting regulatory networks as directed graphs, with nodes representing biological species, and pointed vs. blunt arrows indicating activation vs. repression, respectively (Fig. 1). DDE models allow a single arrow to faithfully capture many biochemical steps18,33,43, expanding the available dynamics in a model with a reduced set of equations and parameters. DDE regulatory models have in fact been used widely to model a range of biological phenomena such as development17,44 and hematopoiesis30,31. For instance, a 1-variable ODE such as Eq. (1) can only produce exponential growth or decay45,46. A corresponding DDE, such as Eq. (2), can also oscillate (with amplitude approaching zero or infinity) and, if nonlinear, lead to stable oscillations, bistability and chaos13,26,47,48,49.

A key challenge in using DDEs is their mathematical complexity relative to ODEs50,51. For example, while Eq. (1) is 1-dimensional because one initial condition (x at t = 0) determines all future behavior, Eq. (2) is infinite-dimensional, because x must be specified at all times − τt ≤ 0 to predict a unique solution50,51. Concretely, the solution to Eq. (1) can be written as a single exponential, while Eq. (2) generally must be specified as a sum of an infinite number of exponential terms to satisfy initial conditions.

Despite the challenges, much progress has been made in analytical understanding of DDEs46,52,53, and numerical methods exist for simulation54,55. We thus see an opportunity to use DDEs to recapitulate dynamics found in ODE solutions of network motif behavior with fewer genes and thus fewer modeling parameters and equations (Fig. 1), a type of “modeling simplicity.”

In this work, we thoroughly examine the most common network motifs2,6 with explicit delays and present an approachable, step-by-step view of the mathematical analysis (applying established DDE methods34,51) in order to make such delay equations easy to use for biologists and others. We show the essential role of delays in autoregulation, feedforward loops, feedback loops, multiple feedback, and complex networks, as well as instances where delays may be ignored. A reference summary is provided at the end in Table 2. Finally, we discuss how these results can be applied to understanding fundamental design principles of various natural biological systems.

## Results

Our results are divided up into 7 sections corresponding to 7 different regulatory networks of increasing complexity. An overall reference table (Table 2) is included in the discussion.

### Motif 0: direct Hill regulation

We first describe how we use DDEs to model simple regulation of a gene Y by a gene X as the most basic motif, and then provide a simple yet unified mathematical framework for both activation and inhibition with delay.

#### Activation and inhibition can both be modeled with a single unified function

We consider a transcription factor x regulating the production of a protein y (Fig. 2). If x activates y, the production rate of y increases with increasing x, generally saturating at a maximum rate α. Often there is an additional cooperativity parameter n which determines how steeply y increases with x in the vicinity of the half-maximal x input value k. In this framework, n = 0 is constitutive production, n = 1 is a Michaelis-Menten regulation, 1 < n < 5 is a typical biological range of cooperativity, and n →  is a step-function regulation. If x represses y, the same conditions hold except that the production rate of y then decreases with increasing x. A standard quantitative model for this behavior is called the Hill function, and serves as a good approximation for many regulatory phenotypes in biology, including transcription and translation rates, phosphorylation, enzymatic activity, neuronal firing, and so forth56,57. In general there may also be a leakage rate α0 that yields constant y production in the absence of x. The concentration of y is also generally removed at a rate β proportional to its concentration. This removal term can represent many biophysical processes, such as degradation, dilution, compartmentalization, or sequestration58,59; for simplicity we mainly use the term “degradation.”

Together these can be written as:

$$\dot{y}(t)={\alpha }_{0}+\frac{\alpha {x}^{n}(t)}{{k}^{n}+{x}^{n}(t)}-\beta y(t)\qquad {\rm{(activator)}}.$$
(3)

If x instead “represses” y instead of activating it, the form is similar:

$$\dot{y}(t)={\alpha }_{0}+\frac{\alpha {k}^{n}}{{k}^{n}+{x}^{n}(t)}-\beta y(t)\qquad {\rm{(repressor)}}.$$
(4)

For biologically meaningful results, all variables and parameters in these equations should be real and non-negative. Note, however, that the activator case is equivalent to the repressor case with n < 0. In this paper we will therefore allow n to be negative, which is simply a notational modification used in order to combine the two cases. With this notation, the effective cooperativity is n. Thus, we have:

$$\dot{y}(t)={\alpha }_{0}+\frac{\alpha {k}^{n}}{{k}^{n}+{x}^{n}(t)}-\beta y(t)\quad \left\{\begin{array}{ll}-\infty \,<\,n\,<\,0\hfill&\!\!\!\!\!\!\!({\rm{activator}})\\ n=0\hfill&({\rm{constitutive}})\\ 0\,<\,n\,<\,\infty\hfill &\!\!\!\!\!\!({\rm{repressor}})\end{array}\right..$$
(5)

which provides a unified, single-function description for both activators and inhibitors (and constitutive expression), providing a powerful mechanism to analyze both cases simultaneously as we will do throughout this work.

The negative value of n allowed here should not be confused with the term “negative cooperativity” used in the binding kinetics literature60, which in our notation would refer strictly to a negative value of n with magnitude less than one (i.e., − 1 < n < 0). The Hill function has an inflection point for all n > 1, which allows there to be 3 fixed points for n < 0 when feedback is introduced below (Supplementary Fig. 1).

#### Delays may be present in regulation, but are not modeled in removal

Converting Eq. (5) from ODE to DDE to account for explicit delay τ in regulation, we replace x(t) with x(t − τ) in the regulation (Hill) term, but not in the removal term, which is directly dependent on y itself, and thus not expected to have any delay. A delayed removal term can also in general lead to negative values, limiting its use in a biological context. This form of regulation is quite general, but for concreteness we will generally refer to quantities like x (or y) as the concentrations of some protein x. Thus we arrive at an explicit-delay DDE model of activating or repressive biological regulation as follows:

$$\dot{y}(t)={\alpha }_{0}+\frac{\alpha {k}^{n}}{{k}^{n}+{x}^{n}(t-\tau )}-\beta y(t)\quad \left\{\begin{array}{ll}-\infty \,<\,n\,<\,0\hfill&({\rm{activator}})\\ n=0\hfill&\,\,\,\,\,\,\,({\rm{constitutive}})\\ 0\,<\,n\,<\,\infty \hfill&\,({\rm{repressor}})\end{array}\right..$$
(6)

#### Nondimensionalizing yields 4 key parameters for any regulation

We can non-dimensionalize Eq. (6) by dividing all concentrations by the half-maximal input concentration k and dividing times by the degradation time 1/β, which has the effect of measuring concentrations in units of k and times in units of 1/β. This yields:

$$\dot{Y}(T)=\epsilon +\frac{\eta }{1+{X}^{n}(T-\gamma )}-Y(T)\quad \left\{\begin{array}{ll}-\infty \,<\,n\,<\,0\hfill&({\rm{activator}})\\ n=0\hfill&\,\,\,\,\,\,\,({\rm{constitutive}})\\ 0\,<\,n\,<\,\infty \hfill&\,({\rm{repressor}})\end{array}\right.,$$
(7)

with dimensionless variables X = x/k, Y = y/k, T = tβ, and dimensionless parameters ϵ = α0/kβ, η = α/kβ, γ = τβ, and n. We thus reduce the number of parameters from 6 to 4, and as discussed below, primarily η and γ are important. In this form, η is a normalized “regulation strength” and γ is a normalized delay. These dimensionless parameters can be considered “big” or “small” by comparing their value relative to unity. For example, γ = τβ ~ 1 implies that the delay (τ) and degradation (1/β) times are approximately equal. In subsequent sections we generally explore parameter values between 0.1 and 10. The dynamics of direct activation and repression via Eq. (7) are demonstrated by numerical simulation in Fig. 2c. For simplicity, here and in all other simulations, we use constant initial conditions (variables constant in time for T ≤ 0) unless otherwise specified.

### Motif I: cascade (sequential regulation)

A common network motif in many biological networks is the cascade (Fig. 3), a series of regulatory steps (i.e., x regulates y, which regulates z, etc.)2,6,61. Since each regulator must reach the corresponding half-maximal input value k before significantly affecting the next item in the cascade, each step adds an effective delay. The following analysis falls within a range of methods used to replace intermediate steps with a delay18,41,42.

#### Delayed direct regulation approximates cascades of non-delayed regulation

For a non-delayed cascade motif in which X regulates Y, which in turn regulates Z (Fig. 3), one can write down a nondimensional set of governing equations as follows:

$$\frac{{\beta }_{z}}{{\beta }_{x}}\dot{X}(T) = \;{\eta }_{X}-X(T)\\ \frac{{\beta }_{z}}{{\beta }_{y}}\dot{Y}(T) = \;\frac{{\eta }_{Y}}{1+{X}^{{n}_{X}}(T)}-Y(T)\\ \dot{Z}(T) = \;\frac{{\eta }_{Z}}{1+{Y}^{{n}_{Y}}(T)}-Z(T),$$
(8)

in which βx, βy, and βz are the dimensional degradation rates of X, Y, and Z, respectively (Supplementary Note 1). We have included here an explicit input function X for concreteness, but more generally we can consider any input X(T) with characteristic timescale 1/βx.

If Y changes quickly compared to X (i.e., βx/βy 1), a first approximation to arriving at a simplified model that ignores Y is to replace the appearance in $$\dot{Z}(T)$$ of Y(T) with the X-dependent pseudo-steady state Ypss(X(T)) found by setting $$\dot{Y}(T)=0$$. A more precise estimate with the looser demand that $${({\beta }_{x}/{\beta }_{y})}^{2}\ll 1$$ can be found by using the same pseudo-steady state, but delayed in time as Ypss(X(T − γ)), with γ = βz/βy (Supplementary Note 1). Intuitively, this delay takes into account the fact that on the slower timescale of X, Y effectively follows the dynamics of X, but the delay accounts for the non-zero time βz/βy that it takes for Y to respond to such changes in X (the βz deriving merely from normalizing time T = tβz). By removing Y we lose certain information, such as the ratio of Z or X to Y, but the inclusion of a delay provides for a reasonably quantitative approximation of the output Z’s dynamics as a function of the input X.

After plugging the delayed pseudo-steady state of Y into $$\dot{Z}(T)$$, one can match the values of the composite Hill-within-Hill function at X = 0, X = 1, and X →  as well its the slope at X = 1 compared to a single Hill function with leakage, and thereby approximate the cascade as a single-step regulation of Z by X (Supplementary Note 1):

$$\dot{Z}(T)\approx {\alpha }_{0}+\frac{\alpha {k}^{h}}{{k}^{h}+{X}^{h}(T-\gamma )}-Z(T).$$
(9)

where the combined regulation parameters are (Supplementary Note 1)

$$\begin{array}{lll}{\alpha }_{0}=\frac{{\eta }_{Z}}{1+{\eta }_{Y}^{{n}_{Y}}}\left(\frac{{\mathrm{sgn}}\,{n}_{Y}+1}{2}\right)\hfill& \alpha =\frac{{\eta }_{Z}{\eta }_{Y}^{| {n}_{Y}| }}{1+{\eta }_{Y}^{| {n}_{Y}| }}\hfill\\ {k}^{h}={\left(\frac{{2}^{| {n}_{Y}| }-1}{1+{\eta }_{Y}^{| {n}_{Y}| }}\right)}^{{\mathrm{sgn}}\,{n}_{Y}}& h=-\frac{{n}_{X}{n}_{Y}}{2}\left(\frac{{2}^{| {n}_{Y}| }}{{2}^{| {n}_{Y}| }-1}\right)\end{array}\quad .$$
(10)

The k can be removed by renormalization of X.

Equation (10) has several biological consequences. First, the overall Hill coefficient h is negatively proportional to the product of individual cooperativities nX, nY. This makes the cascade activating for either two activators or two repressors, and repressive otherwise. It also makes the cascade more cooperative than its components, in line with past analyses62 and as found experimentally61. Third, the leakage is zero for nY < 0 and negligible (i.e., α0α) for nY > 0 if ηY 1. This means Y must be produced in sufficient amount during the delay to repress Z. We generally ignore leakage in the main text; for a more detailed discussion see Supplementary Note 3 and Supplementary Fig. 2.

#### The total delay in a cascade equals the sum of individual delays

For cascades with delays at each step, the same analysis implies that total delay equals the sum of individual delays (βz/βy + γy + γz, with γy the intrinsic delay between X and Y and γz the intrinsic delay between Y and Z). Figure 3 shows simulations of ODE, equivalent DDE, and multi-delay DDE cascade models, matching the above analytical results.

### Motif II: autoregulation

Autoregulation, one of the most common biological motifs2,63, describes a single biological species that regulates its own production (Figs. 4a–c).

#### The complete phase space for autoregulation demonstrates the quantitative and qualitative importance of delays

Based on Eq. (5), the governing equation for such a system with delayed regulation is given by setting Y = X (output equals input) in Eq. (7):

$$\dot{X}(T)=\epsilon +\frac{\eta }{1+{X}^{n}(T-\gamma )}-X(T).$$
(11)

This equation has four parameters (η, γ, ϵ, and n).

Since leakage must be small relative to regulation (i.e., ϵ/η 1) for regulation to be strongly effective (“activated” rate much greater than “non-activated” rate), we will focus here on the case with no leakage (ϵ = 0). We treat non-negligible leakage in the supplements (Supplementary Note 3). Note that in general Eq. (11) has no closed-form solution.

#### Fixed points do not depend on delays

Fixed points X(T) = X* do not change with time, implying $$\dot{X}(T)=0$$ and X(T) = X(T − γ) = X*. In Eq. (11), this implies that production must balance degradation (Fig. 4d). For repressors (n > 0) there is only a single fixed point for biologically relevant parameter values. For activators (n < 0), there can be 1, 2, or 3 fixed points (2 fixed points is a border case for −n > 1). Explicitly, the fixed point values are given by

$${X}^{* }\left(1+{{X}^{* }}^{n}\right)=\eta$$
(12)

or X* = 0 (for activators only). Again, we assumed ϵ = 0 for simplicity. Note that these fixed points only depend on 1/η = βk/α and ϵ/η = α0/α, and have no dependence on the delay time τ.

#### Linearization is sufficient to determine bifurcations in qualitative behavior

Small disturbances from fixed points δX(T) = X(T) − X* 1 linearize Eq. (11) (Supplementary Note 2). Assuming a solution $$\delta X(T)=A\exp (\lambda T)$$ yields a transcendental “characteristic equation” for the eigenvalues λ of Eq. (11):

$$\lambda +\eta M{e}^{-\gamma \lambda }+1=0.$$
(13)

The constant function of the fixed point value M(X*) is defined as

$$M({X}^{* })=\frac{n{{X}^{* }}^{n-1}}{{\left(1+{{X}^{* }}^{n}\right)}^{2}},$$
(14)

whose sign is given by n (positive for repressors and negative for activators).

To determine stability of the fixed points, we solve for conditions in which λ crosses the imaginary axis ($$\mathrm{Re}\,\lambda =0$$), occuring at either λ = 0 (a saddle-node bifurcation) or λ = iω (a Hopf bifurcation).

#### Saddle-node bifurcations determine bistability in autoactivators

For the saddle-node case (λ = 0), Eqs. (13) and (12) reduce to (Supplementary Note 2):

$$\eta =-n{(-n-1)}^{-\frac{-n-1}{-n}}.$$
(15)

which is biologically meaningful ($$\eta \,> \, 0,\mathrm{Im}\,\eta =0$$) only for autoactivation (n < 0), where − n is the biologically relevant quantity. We can see that Eq. (15) separates monostability from bistability by noting that the production and removal curves (Fig. 4d) are tangent when Eq. (15) holds (Supplementary Note 2), yielding two stable fixed points when η is above this boundary and only one otherwise. For −1 < n ≤ 0, the origin (X* = 0) is unstable (since M(X* = 0) → −), and it ceases to be a fixed point for n > 0. Note that this boundary does not depend on delay; however, that does not preclude other features such as basins of attraction from depending on delay64.

Viewed as a function of n, Eq. (15) limits bistability to −n > 1 and the boundary to 1 ≤ η ≤ 2 (Supplementary Fig. 1b). Since this function is non-monotonic, it is also possible (although biologically perhaps unrealistic except on an evolutionary timescale), to hold a value of 1 < η < 2 and decrease n to a point where the bistability is lost, and then decrease n still further until bistability is gained again.

#### Hopf bifurcations determine oscillatory behavior in autorepressors

For the Hopf bifurcation (λ = iω), Eqs. (13) and (12) result in two equations, one for the real parts of Eq. (13) and one for the imaginary parts (Supplementary Note 2):

$$\begin{array}{ll}\gamma =\frac{1}{\omega }\left(-{\tan }^{-1}\omega +\pi k\right),\qquad k=0,1,\ldots \\ \eta =\frac{| n| }{\sqrt{1+{\omega }^{2}}}{\left(\frac{| n| }{\sqrt{1+{\omega }^{2}}}-1\right)}^{-\frac{n+1}{n}}\; .\hfill\end{array}$$
(16)

Equation (16) represent a series of curves parameterized by ω, and is biologically meaningful (γ > 0, η > 0) for k ≥ 1. For autoactivation (n < 0), these curves all lie above the bistability boundary (Eq. (15)), and thus have no qualitative effect on behavior. For autorepression (n > 0), however, the outermost curve (k = 1) dictates the onset of oscillations for any n > 1.

This boundary has both horizontal (η) and vertical (γ) asymptotes, given as:

$$\lim_{\omega \to 0}{\eta }_{{\rm{Hopf}}}= \;| n| {(| n| -1)}^{-\frac{n+1}{n}}\\ \lim_{\omega \to \sqrt{{n}^{2}-1}}{\gamma }_{{\rm{Hopf}}}= \;\frac{{\cos }^{-1}\left(-| n{| }^{-1}\right)}{\sqrt{{n}^{2}-1}}.$$
(17)

Thus oscillations require both a minimal regulation strength and a minimum delay. The vertical (γ) asymptote approaches zero as n → , so large regulatory strength can achieve oscillations even with minuscule delay if cooperativity is extremely steep.

The oscillation period can be approximated by linearization near consecutive maxima and minima of the oscillation (Supplementary Note 2). This method yields a period of approximately 2(γ + 1), or 2(τ + 1/β) in dimensionful terms (compare also to17). Biologically, this means that the concentration X is pushed from high to low (or vice versa) after the delay time (τ) has elapsed and the concentration has equilibrated to its new value via degradation (1/β).

Damped oscillations are expected when the largest eigenvalue satisfying Eq. (13) has non-zero imaginary part. This is never true for n < 0, and is true for n > 0 above the curve denoted “spiral boundary” in Fig. 1e and Supplementary Fig. 3. This curve is given as follows:

$$\eta =n\gamma {e}^{\gamma +1}{\left(n\gamma {e}^{\gamma +1}-1\right)}^{-\frac{n+1}{n}},$$
(18)

which is derived in Supplementary Note 4. It approaches η = 0 for large delay and has a vertical asymptote at nγeγ+1 = 1, so there is some non-zero delay that has non-oscillating decay for any strength η and finite n.

Putting together the derived boundary curves yields an η-γ phase space for each n, displayed in Fig. 4e and Supplementary Fig. 3a-b along with simulations showing that this analytical treatment matches the dynamical behavior.

#### Leakage prevents oscillations and bistability

Autoregulation with leakage (Supplementary Note 3) adds the extra parameter ϵ ≠ 0. Small leakage ϵη does not change the qualitative results while large leakage effectively overwhelms the regulation term, preventing both oscillations (for autorepression) and bistability (for autoactivation). The full treatment with leakage and the corresponding phase portraits are presented in the supplements (Supplementary Note 3, Supplementary Fig. 2).

### Motif III: logic

A general class of functions used to describe natural2,58,65,66,67 and synthetic68,69 biological networks are logic gates, which have two inputs regulating a single output (Fig. 5). Gates exhibit either high (“on”) or low (“off”) output depending on whether inputs are on or off. For example, the AND gate specifies high output only if both inputs are on. In this section we provide a specific DDE-based framework that covers 14 out of 16 possible 2-input logic operations, and show that these operations form a continuous 2D parameter space.

#### A two-parameter summing function reproduces all 2-input monotonic logic gates

We first write out nondimensionalized equations corresponding to the logic gate motif as depicted in Fig. 5a. We assume that the degradation constants (β) for Z and R are equal for simplicity, and that there is no leakage.

$$\begin{array}{lll}\dot{Z}(T)=\frac{{\eta }_{Z1}}{1+{X}^{{n}_{1}}(T-{\gamma }_{1})}+\frac{{\eta }_{Z2}}{1+{Y}^{{n}_{2}}(T-{\gamma }_{2})}-Z(T)\\ \dot{R}(T)=\frac{{\eta }_{R}}{1+{Z}^{{n}_{3}}(T-{\gamma }_{Z})}-R(T),\hfill\end{array}$$
(19)

We describe the regulation of Z by X and Y using a sum of Hill terms. Logic gate behavior can be captured with other approaches such as a product of Hill terms, or summation within a single Hill term58,66,67,71, each representing subtly different biology. We choose the separate Hill term approach as it describes many logic functions simply by tuning regulatory strengths, and can be extended to include multiplicative terms (Supplementary Note 5). Caveats include multiple states for Z, requiring additional binarization via R (Fig. 5c), as well as poor response to ratiometric inputs, discussed in the next section on feedforward motifs.

Using Eq. (19), we can characterize the motif logic based on the idea of dynamic range matching72. Every regulator in Eq. (19) is effectively compared against unity in the denominator of the Hill function for its corresponding output. For instance, Z provides an “on” or “off” signal to R if Z > 1 or Z < 1 respectively. Z can take on values below 1 as long as $${\eta }_{Z1}\ll \max ({X}^{| {n}_{1}| })$$ and $${\eta }_{Z2}\ll \max ({Y}^{| {n}_{2}| })$$, otherwise Z will always activate R, as indicated by the areas marked TRUE in Fig. 5.

Let us say that X and Y settle on steady-state values X* ≡ ηX, Y* ≡ ηY, as inputs to our logic gate. A value of ηX or ηY significantly greater than 1 is then “high” (true, 1), and “low” (false, 0) if much less than 1. We then want to determine whether Z* is greater than (true) or less than (false) 1. For example, if n1,2 > 0 (two repressors), Table 1 gives the possible steady states of Z. If ηZ1 and ηZ2 are greater than 1, these steady states approximate a NAND gate. If they are less than 1, but sum to greater than 1, the steady states instead approximate a NOR gate.

Equivalent calculations for all variations of n1, n2, ηX, ηY, ηZ1, and ηZ2 (Supplementary Table 2) show that logic is dependent on only two parameters, $${\mathrm{sgn}}\,({n}_{1}){\eta }_{Z1}$$ and $${\mathrm{sgn}}\,({n}_{2}){\eta }_{Z2}$$. These form a 2D parameter space that summarize the logic into a convenient chart (Fig. 5b). This implies evolution or other tuning can smoothly convert between logic gates (see also66,67). The chart is symmetric on interchange of the two inputs (η1 ↔ η2, n1 ↔ n2), and anti-symmetric (that is, application of logical NOT) by conversion of an activator to a repressor or vice versa (ni → −ni). Fourteen of the 16 possible two-input logic functions are represented. All fourteen are monotonic in that Z(X, Y = 1) > Z(X, Y = 0) and Z(X = 1, Y) > Z(X = 0, Y). The two non-monotonic gates, XOR and XNOR, are not represented by this simple logic motif, because summation (addition of Hill terms) is monotonic. However, they can be constructed by connecting several of these gates70,72.

#### The logic parameter space can be divided into AND-type, OR-type, and single-input functions

Only the 8 regulatory functions on the diagonals of Fig. 5 (excluding FALSE and TRUE) make use of both inputs. These eight can be further divided into positive or negative regulation on each arm (4 possibilities) in conjunction with an AND or OR gate (8 possibilities total). Specifically, “OR-type” logic applies when both ηZ1 > 1 and ηZ2 > 1 (OR, NAND, and both IMPLY gates), and “AND-type” logic applies when both ηZ1 < 1 and ηZ2 < 1 (AND, NOR, and both NIMPLY gates). This can be seen mathematically by using Boolean logic simplification. For example, X NOR Y = NOT AND NOT Y. Similarly, X NAND Y = NOT X OR NOT Y.

It is important to note that this logic scheme is an approximation, and in reality the sum of two Hill terms provides a form of fuzzy logic73. That is, if the inputs X or Y are close to 1, or if the regulatory strengths ηZ1,2 are close to 1, then Z will also be close to 1 for some input combinations (Supplementary Note 5).

Note that if X(T) and Y(T) are independent, they can be time-shifted in Eq. (19) by γ1 and γ2, respectively, showing that the dynamics do not depend on delays. This is not true if X and Y are dependent (e.g., X = Y), which leads to interesting dynamics that we examine below in feedforward loops and double feedback.

### Motif IV: feedforward loop

Equipped with a multivariable generalization of 1-variable Hill regulation (Eq. (19)), we turn our attention to the feedforward motif (Fig. 6), a non-cyclic regulation network in which an input, X, regulates an output, Z, via two separate regulation arms. One arm is “direct”, in which X regulates Z in a single step, while the second arm is “indirect”, with X regulating an intermediate Y, which in turn regulates Z. In this way, the first arm “feeds forward” past the cascade (Fig. 6a–b). The motif is found commonly in biological networks6,74, comprising about 30% of three-gene regulatory interactions in transcriptional circuits75. Feedforward loops are conventionally described as “incoherent” if X activates Z through one arm but represses through the other, and “coherent” otherwise (Supplementary Table 3). In this section, we show that the essential behaviors of feedforward loops are due to a difference in delays between the two inputs to Z and the logic function between the two inputs to the output Z.

#### Feedforward dynamics depend only on the difference in time delay between two regulatory arms and their 2-input logic

With a DDE model, we can drop the cascaded intermediate Y, focusing solely on Z as regulated by X via two arms that have unequal delays (Fig. 6a, b). Looking at Fig. 6a, we can then write down a simple equation for the regulation of Z, by setting Y = X in Eq. (19) (Supplementary Note 6):

$$\dot{Z}(T)=\frac{{\eta }_{1}}{1+{X}^{{n}_{1}}(T-{\gamma }_{1})}+\frac{{\eta }_{2}}{1+{K}^{{n}_{2}}{X}^{{n}_{2}}(T-{\gamma }_{2})}-Z(T).$$
(20)

Because each regulation term is controlled by the same input (X), we cannot in general normalize the half-maximal input to 1 in both terms as we did for logic (Eq. (19)); K = k1/k2 represents the ratio of the two input scales. Likewise, we cannot entirely time-shift out the delays; instead, shifting X backwards in time by γ1 ($$\hat{X}(T)=X(T+{\gamma }_{1})$$), we see that the behavior of the feedforward equation actually depends only on the difference in regulatory delays, Δγ = γ2 − γ1 rather than each delay independently:

$$\dot{Z}(T)=\frac{{\eta }_{1}}{1+{\hat{X}}^{{n}_{1}}(T)}+\frac{{\eta }_{2}}{1+{K}^{{n}_{2}}{\hat{X}}^{{n}_{2}}(T-{{\Delta }}\gamma )}-Z(T).$$
(21)

For clarity we will from now on refer to $$\hat{X}$$ as X without the hat unless specified.

To understand the behavior of Eq. (21), we will analyze the output of Z to two types of input X: a step input, and a continuous oscillation. Termed the step and frequency responses, respectively, these are often used in control theory to characterize input-output relations76.

#### Step-pulse response of feedforward motifs can be solved analytically and demonstrates filtering and pulse-generation behaviors

We considered a step input (Fig. 6c) to include both a step on (from low to high X) as well as a step off (from high to low X):

$$X(T)={X}_{0}+({\eta }_{X}-{X}_{0})\left({{\Theta }}(T)-{{\Theta }}(T-\omega )\right)$$
(22)

where the Heaviside function is defined such that Θ(T) = 0 for T < 0 and Θ(T) = 1 for T > 0. Eq. (22) represents a square input pulse of width ω starting at X0 and reaching to ηX. This is an on-pulse if ηX > X0 and an off-pulse if ηX < X0.

The lack of feedback in feedforward loops and the fact that the square-pulse input of Eq. (22) takes on only two values simplify the form of Eq. (20) significantly, allowing it to be solved explicitly (Supplementary Note 6). The solution is:

$$Z(T)={Z}_{ss}+{\mathrm{sgn}}\,({n}_{1}){\eta }_{1}{Z}_{1}^{{n}_{1}}f(T)+{\mathrm{sgn}}\,({n}_{2}){\eta }_{2}{Z}_{K}^{{n}_{2}}f(T-{{\Delta }}\gamma )$$
(23)

in which

$$f(T) =\left(1-{e}^{-T}\right){{\Theta }}(T)-\left(1-{e}^{-(T-\omega )}\right){{\Theta }}(T-\omega )\\ {Z}_{ss} =\frac{{\eta }_{1}}{1+{X}_{0}^{{n}_{1}}}+\frac{{\eta }_{2}}{1+{\left(K{X}_{0}\right)}^{{n}_{2}}}\\ {Z}_{\kappa }^{n} =\frac{1}{1+{\left(\kappa {\eta }_{X}\right)}^{| n| }}-\frac{1}{1+{\left(\kappa {X}_{0}\right)}^{| n| }}.$$
(24)

Here, Zss is the steady-state value of Z, $${Z}_{\kappa }^{n}$$ are the magnitudes of deviation from steady state due to each arm, and f(T) specifies where the responses from the two regulation arms are active. In terms of the original X (as opposed to the time-shifted $$\hat{X}$$), the output in Eq. (24) is shifted to the right in time by γ1 relative to the original input X(T).

There are several interesting results to note in this equation. First, the cooperativities (n1, n2) and thresholds (K) affect the response magnitude, but not the dynamics (f), because the input signal is already infinitely steep.

Second, the η values only affect the steady-state Zss and the response logic (as $${\mathrm{sgn}}\,({n}_{i}){\eta }_{i}$$, see logic discussion). Only 8 logic gates correspond to feedforward loops, in that they respond to both inputs: the four AND-type functions and four OR-type functions.

Third, f indicates where responses are active, varying from 0 far outside a pulse, aprroaching 1 inside for wide pulses (ω 0). The output then has four responses (pulse start and pulse end for each arm), in which Z moves away from Zss by as much as $${Z}_{\kappa }^{n}$$ (pulse on) before returning (pulse off). Responses must be driven externally, unlike autoregulation.

For coherent feedforward motifs, input pulses must be sufficiently wide (ω 1) to fully activate the output R. This implies that coherent feedforward motifs act as filters against short signals, either short on signals (AND and NAND) or short off signals (OR and NOR). The delay difference Δγ determines how quickly the response starts (AND and NOR logic) or returns to baseline once the pulse is over (for OR and NAND logic), so that large Δγ maintains a response long after the input pulse has subsided.

For incoherent feedforward motifs, the output R is only fully activated if the pulse is wide (ω 1) and the delay difference is also wide (Δγ 1), in which case a pulse of output is formed. The output pulse is formed on either the on-step (slow arm IMPLY fast arm, and slow arm NIMPLY fast arm) or the off-step (fast arm IMPLY slow arm, and fast arm NIMPLY slow arm). This means that incoherent feedforward motifs act as pulse generators. The delay difference Δγ determines the minimum width of input pulse that will produce a response in R.

Many of the feedforward behaviors described in this section corroborate earlier results from ODE models2,2,58,74,75,77,77,78, but are derived here with just a single equation. In particular, it is well known that coherent feedforward loops have filter capabilities and that incoherent feedforward loops can act as pulse generators (temporal2,58,77 or spatial79). We did not demonstrate fold-change detection (FCD)78 because of our use of sum-like logic; an alternative logic formulation does show FCD, implying that FCD has certain mechanistic requirements (Supplementary Note 6). Furthermore, we clearly demonstrated that the feedforward loop dynamics depend entirely on the difference of delays in the two regulation arms, a fact that can only be seen indirectly with ODE models58.

#### Frequency response of feedforward motifs can be solved analytically and demonstrates low- and band-pass filtering capabilities

Biological regulatory networks often encode information as the change in frequency of an oscillating input80, which has been suggested to be more robust to noise than encoding information in absolute concentration81,82,83. That feedforward loops filter short square pulses suggests more general frequency-filtering capabilities. We therefore analyze feedforward frequency response to sinusoidal input (Fig. 6d–g). We show that the response follows the outline of a universal transfer function curve, independent of the logic or delay difference.

Instead of the step input analyzed above, here we consider a sinusoidal input

$$X(T)=A(1+\cos (2\pi fT))$$
(25)

which oscillates between zero and 2A (twice the amplitude) at a frequency f > 0. Taking the Fourier decomposition of each Hill-regulated term 1/[1 + Xn(T)] and plugging into the governing Eq. (21) (Supplementary Note 6) provides the output Z(T) in terms of its magnitude Ik and phase ϕk as a function of frequency:

$$Z(T) =\frac{{\eta }_{1}{a}_{0}^{x;{n}_{1}}+{\eta }_{2}{a}_{0}^{x;{n}_{2}}}{2}+\mathop{\sum }\limits_{k=1}^{\infty }{I}_{k}\cos (2\pi kfT-{\phi }_{k})\\ {I}_{k} =\sqrt{\frac{{({\eta }_{1}{a}_{k}^{x;{n}_{1}})}^{2}+2{\eta }_{1}{\eta }_{2}\cos (2\pi kf{{\Delta }}\gamma ){a}_{k}^{x;{n}_{1}}{a}_{k}^{x;{n}_{2}}+{({\eta }_{2}{a}_{k}^{x;{n}_{2}})}^{2}}{1+{(2\pi kf)}^{2}}}\\ {\phi }_{k} ={\rm{atan2}}\left(2\pi kf{\eta }_{1}{a}_{k}^{x;{n}_{1}}+{\eta }_{2}(2\pi kf\cos (2\pi kf{{\Delta }}\gamma )+\sin (2\pi kf{{\Delta }}\gamma )){a}_{k}^{x;{n}_{2}},\right.\\ \qquad\qquad \left.{\eta }_{1}{a}_{k}^{x;{n}_{1}}+{\eta }_{2}(\cos (2\pi kf{{\Delta }}\gamma )-2\pi kf\sin (2\pi kf{{\Delta }}\gamma )){a}_{k}^{x;{n}_{2}}\right),$$
(26)

where $${a}_{k}^{x;n}$$ are Fourier coefficients of Hill-regulated terms 1/[1 + Xn(T)] at integer multiples k of the fundamental frequency f:

$${a}_{k}^{x;n}=\frac{1}{\pi }\mathop{\int}\nolimits_{0}^{2\pi }\ \frac{\cos (kT)}{1+{A}^{n}{(1+\cos (T))}^{n}}\ {\rm{d}}T,$$
(27)

for all f > 0. Note that the magnitudes are symmetric to interchange of the two regulation arms (η1 ↔ η2, n1 ↔ n2), while the phases are not.

Looking at frequencies that are integer multiples of 1/Δγ, we can see that both the magnitudes and phases follow a universal envelope not dependent on delays or logic (i.e., no dependence on γ or η values):

$$\begin{array}{lll}{I}_{k,{\rm{env}}}={I}_{0}/\sqrt{1+{(2\pi kf)}^{2}}\\ {\phi }_{k,{\rm{env}}}={\tan }^{-1}\left(2\pi kf\right)-{\phi }_{0}\end{array}$$
(28)

where (from Eq. (26), at k = 0 or f → 0) $${I}_{0}={\eta }_{1}{a}_{k}^{x;{n}_{1}}+{\eta }_{2}{a}_{k}^{x;{n}_{2}}$$ and $${\phi }_{0}=\pi ({\mathrm{sgn}}\,{I}_{0}+1)/2$$. The actual magnitudes and phases are then modulated from these envelopes with period 1/kΔγ. The fundamental-frequency (k = 1) response (the largest magnitude, see Fig. 6d, e) are shown compared to the envelopes in Fig. 6f (known as Bode plots76).

At frequencies f that are integer multiples of 1/kΔγ, the cosine in Ik equals ±1. Because $${a}_{k}^{x;-n}=-{a}_{k}^{x;n}$$ (see Eq. (27), Supplementary Note 6), the output magnitude Ik then increases to a local maximum for coherent feedforward motifs ($${\mathrm{sgn}}\,({n}_{1})={\mathrm{sgn}}\,({n}_{2})$$) and decreases to a local minimum for incoherent feedforward motifs ($${\mathrm{sgn}}\,({n}_{1})=-{\mathrm{sgn}}\,({n}_{2})$$). The opposite holds for frequencies that are half-integer multiples of 1/kΔγ. For the special case of perfectly balanced incoherent feedforward motifs (η1 = η2, n1 = −n2, K = 1), the magnitudes Ik decrease to zero for frequencies that are half-integer multiple of 1/Δγ; otherwise, the maxima (for coherent) and minima (for incoherent) equal Ik(f = 0).

Incredibly, the envelopes (relative to f = 0) have zero parameters, making them universal to all feedforward loops. The frequency at which the magnitude decreases to one-half its value at f = 0 occurs at $$f=\sqrt{3}/2\pi k$$, with a phase shift of −π/3.

Despite the non-dependence on Δγ, the absolute maximum magnitude obtained for incoherent feedforward motifs does depend on Δγ, as does the frequency at which this maximum response occurs. The dependence on delay difference is particularly strong near Δγ = 1 (Fig. 6g). This is because for Δγ 1, the envelope decreases by half when f is many multiples of Δγ, and thus past many local maxima. The local maximum then occurs at 0 < f 1 with a maximum above Ik(f = 0). For Δγ 1, on the other hand, the envelope decreases by half before a single sinusoid is complete, so the maximum is at f → 0.

At Δγ ≈ 1, the first sinusoid reaches its maximum while the envelope is descending most steeply, and the maximum occurs at f ≈ 1 with a value highly dependent on Δγ. To activate R, X must go above 1 for at least part of the output oscillation, meaning that I1 must go above 1 − 〈I〉 = 1 − I0. Looking at Fig. 6g, we see that for large Δγ 1, this occurs for small frequencies, and the feedforward loop acts as a low-pass filter. However, there is a window of Δγ ≈ 1 in which 1 − I0 is large, at modest frequencies (0.1 < f < 10), corresponding to the hump in magnitude, e.g., around f = 0.5 for Δγ = 1 in Fig. 6f. The incoherent feedforward motif thus acts as a bandpass filter for Δγ ≈ 1, with ineffective regulation outside of modest frequencies.

For coherent feedforward motifs, the frequency at which the first minimum in output response occurs follows the same logic, but the maximum response is always found at f → 0. Since the maximum response is at low frequencies, coherent feedforward motifs act as low-pass filters, as has been shown to be a basic expected behavior of chemical networks84.

### Motif V: multi-component feedback

The common two-component feedback motif58 is the equivalent of a cascade with feedback (Fig. 7a). We can write down the governing equations for such a system, using the same normalizations as before (see Supplementary Note 7), assuming for simplicity zero leakage and that x and y have the same degradation rates β. This is equivalent to the 3-step cascade Eq. (8) with equal degradation rates and setting Z = X to form the loop.

$$\begin{array}{lll}\dot{X}(T)=\frac{{\eta }_{1}}{1+{Y}^{{n}_{1}}(T-{\gamma }_{1})}-X(T)\\ \dot{Y}(T)=\frac{{\eta }_{2}}{1+{X}^{{n}_{2}}(T-{\gamma }_{2})}-Y(T).\end{array},$$
(29)

The fixed points are given by $${\eta }_{1}=X(1+{Y}^{{n}_{1}})$$, $${\eta }_{2}=Y(1+{X}^{{n}_{2}})$$ or (for two activators) X = Y = 0. Linearizing around these fixed points and assuming ansatz solutions of the form $$\delta X(T)=A\exp ({\lambda }_{1}T),\delta Y(T)=B\exp ({\lambda }_{2}T)$$, we find the set of characteristic equations

$$\begin{array}{lll}A(\lambda +1)+{\eta }_{1}{M}_{1}B{e}^{-\lambda {\gamma }_{1}}=0\,\\ B(\lambda +1)+{\eta }_{2}{M}_{2}A{e}^{-\lambda {\gamma }_{2}}=0.\end{array}$$
(30)

where λ1 = λ2 ≡ λ must hold for the ansatz to be true for all time. This can be rewritten in matrix form, with a matrix J times the vector (A, B).

Diagonalizing J results in two characteristic equations (Supplementary Note 7)

$${{{\Lambda }}}_{\pm }\equiv \lambda \pm \sqrt{{\eta }_{1}{M}_{1}}\sqrt{{\eta }_{2}{M}_{2}}{e}^{-\lambda ({\gamma }_{1}+{\gamma }_{2})/2}+1=0$$
(31)

for two corresponding eigenmodes $${v}_{+}={(1,R{e}^{i\phi })}^{\top }$$ and $${v}_{-}={(-1,R{e}^{i\phi })}^{\top }$$, respectively, where B/A = Reiϕ is the ratio of δX to δY components in magnitude-phase notation (Supplementary Note 7). Overall, the eigenvectors indicate a phase difference between X and Y of ϕ for v+ and ϕ + π for v. When the Λ+ version of Eq. (31) is satisfied for $$\mathrm{Re}\,\lambda =0$$, v+ bifurcates and when the Λ version is satisfied for $$\mathrm{Re}\,\lambda =0$$, v bifurcates.

#### Saddle-node bifurcations determine bistability modes

For saddle-node bifurcations, at which bistability begin, we set λ = 0. This can only be satisfied by the Λ+ version of Eq. (31) if n1 < 0, n2 < 0 (two activators), and by the Λ version if n1 > 0, n2 > 0 (two repressors). A feedback loop of an odd number of repressors cannot have a saddle-node bifurcation. As for linear cascades (Fig. 3), the overall autoregulation is repressive only if it contains an odd number of repressors. The bistability boundary for both cases is then given by (Supplementary Note 7):

$${\eta }_{1} ={n}_{1}{n}_{2}\frac{{X}^{{n}_{2}+1}}{({n}_{1}{n}_{2}-1){X}^{{n}_{2}}-1}\\ {\eta }_{2} ={n}_{1}{n}_{2}\frac{{Y}^{{n}_{1}+1}}{({n}_{1}{n}_{2}-1){Y}^{{n}_{1}}-1}\\ f(X,Y) \equiv \frac{{n}_{1}{n}_{2}{X}^{{n}_{2}}{Y}^{{n}_{1}}}{\left(1+{X}^{{n}_{2}}\right)\left(1+{Y}^{{n}_{1}}\right)}=+1$$
(32)

where f = 1 provides an implicit monotonic mapping between X and Y that restricts η1(X) with respect to η2(Y).

The phase difference ϕ = 0, implying that two-activator loops show bistability with X and Y either both high or both low, because the components of v+ are of like sign, whereas two-repressor loops (v) show bistability with X high and Y low or vice versa (Fig. 7b, d).

#### Hopf bifurcations determine transient oscillations in modes restricted by bistability

For Hopf bifurcations, at which oscillations begin, we set λ = iω. The boundaries are given for both eigenmodes by (Supplementary Note 7)

$${\gamma }_{1} = \frac{1}{\omega }\left[-{\tan }^{-1}\omega +\phi +\frac{\pi }{2}(1-{\mathrm{sgn}}\,{n}_{1})+\pi k\right],\ k=0,1,\ldots \\ {\gamma }_{2} = \frac{1}{\omega }\left[-{\tan }^{-1}\omega -\phi +\frac{\pi }{2}(1-{\mathrm{sgn}}\,{n}_{2})+\pi k\right],\ k=0,1,\ldots \\ {\eta }_{1} = \frac{|{n}_{1}{n}_{2}| }{1+{\omega }^{2}}\cdot \frac{{X}^{{n}_{2}+1}}{\left(\frac{| {n}_{1}{n}_{2}| }{1+{\omega }^{2}}-1\right){X}^{{n}_{2}}-1}\\ {\eta }_{2} = \frac{| {n}_{1}{n}_{2}| }{1+{\omega }^{2}}\cdot \frac{{Y}^{{n}_{1}+1}}{\left(\frac{| {n}_{1}{n}_{2}| }{1+{\omega }^{2}}-1\right){Y}^{{n}_{1}}-1}\\ f(X,Y) \equiv \frac{| {n}_{1}{n}_{2}| }{1+{\omega }^{2}}\cdot \frac{{X}^{{n}_{2}}{Y}^{{n}_{1}}}{\left(1+{X}^{{n}_{2}}\right)\left(1+{Y}^{{n}_{1}}\right)}=1$$
(33)

where again f = 1 provides an implicit monotonic mapping η1(X) and η2(Y).

These curves lie above the bistability boundary, so oscillations in v+ are not observed for two activators nor in v for two repressors. The alternate eigenmode does show oscillations in each case (k = 0, v for activators and k = 1, v+ for repressors), but these oscillations are transient due to existence of bistability-induced, distant stable fixed points. For one repressor and one activator, sustained oscillations begin at k = 0 for v, with ϕ = −π/2 if n1 > 0, n2 < 0 (Y leads X) and ϕ = π/2 if n1 < 0, n2 > 0 (Y lags X).

Based on Eq. (33), we can see that the primary Hopf bifurcations are equivalent for differing values of the delays as long as the average delay is equal, because the oscillation frequency ω of the eigenmodes only depends (implicitly) on the average delay:

$$\langle \gamma \rangle \equiv \frac{1}{2}\left({\gamma }_{1}+{\gamma }_{2}\right)=\frac{1}{\omega }\left(-{\tan }^{-1}\omega +\frac{\pi }{4}(2-{\mathrm{sgn}}\,{n}_{1}-{\mathrm{sgn}}\,{n}_{2})+\pi k\right).$$
(34)

In particular, oscillations are possible for activator-repressor loops even when one delay is zero, as long as the sum of delays is greater than zero. On the other hand, the phase difference between the oscillations depends only on the difference in delays given the value ω:

$$\phi =\frac{\omega }{2}({\gamma }_{1}-{\gamma }_{2})+\frac{\pi }{4}\left({\mathrm{sgn}}\,{n}_{1}-{\mathrm{sgn}}\,{n}_{2}\right)$$
(35)

Together with the additional phase difference of π for v, this implies that for equal delays, there are synchronous oscillations for two-repressor loops, anti-synchronous oscillations for two-activator loops, and π/2-shifted oscillations for activator-repressor loops. This phase relation also holds off the bifurcation boundary, where λ = μ + iω, as ϕ does not depend on μ (Supplementary Note 7). Note also that the overall phase in the expressions for γ are 0 for two repressors, π for two activators, and π/2 for activator-repressor loops.

Finally, one can also conclude from Eq. (30) that (Supplementary Note 7)

$$\lambda =\frac{1}{{\gamma }_{2}-{\gamma }_{1}}\mathrm{ln}\,\left(\frac{{\eta }_{1}{M}_{1}{B}^{2}}{{\eta }_{2}{M}_{2}{A}^{2}}\right),$$
(36)

which holds for both the real and imaginary parts of λ. This implies that the oscillations grow most quickly and have the highest frequency when the delays are nearly equal (γ1 = γ2). This consequently makes the transients most noticeable for equal delays. All these results can be seen in the simulated data (Fig. 7b,d).

### Motif VI: double feedback

Multiple feedback (Fig. 8) has been described to lead to interesting dynamics, such as excitablity85, quasiperiodicity71, and chaos71. A full analysis of this motif is beyond the scope of this work, for reasons described below.

We focus on chaos, a dynamical behavior characterized by irregular, unpredictable oscillations45. It has been suggested that biological systems avoid chaos86,87, but also that chaos can describe pathological dynamics such as irregular breathing26 and epilepsy27. There may also be cases where chaos is important to biological function88,89,90,91. Mathematical models with71,92,93 and without86,94 delays can be chaotic.

#### Chaotic behaviors are prevented in monotonic regulation with linear or cyclic network topology

Due to cyclic network topology (no more than one loop) and monotonic regulation, none of the motifs so far can yield chaotic dynamics46. Either non-monotonic feedback (as in the Mackey-Glass equation26) or non-cyclic topology71 is required for chaos. Double feedback (Fig. 8a) is a minimal motif fulfilling both requirements, although the non-monotonicity is sufficient (Supplementary Note 8).

The governing equation for such a double feedback motif is thus given by setting the output and both inputs of the logic equation (Eq. (19)) to X (and letting K = 1 for simplicity):

$$\dot{X}(T)=\frac{{\eta }_{1}}{1+{X}^{{n}_{1}}(T-{\gamma }_{1})}+\frac{{\eta }_{2}}{1+{X}^{{n}_{2}}(T-{\gamma }_{2})}-X(T).$$
(37)

#### Chaotic behavior is possible for non-monotonic feedback with multiple feedback and disparate delays

Simulation of Eq. (37) demonstrates chaos for some parameters of positive/negative mixed feedback, as evidenced by sustained oscillations with variable maxima (Fig. 8b), occupying an apparently fractal region in phase space (Fig. 8c), and a large number of peaks in frequency space (Fig. 8d). To confirm that the dynamics are in fact chaotic, we calculated the dominant Lyapunov exponent using the Wolf method95 (see “Methods”), finding a positive (i.e., chaotic) value of 0.0040 ± 0.00055 bits (mean ± standard deviation). We also calculated the box dimension of a reconstructed phase space with coordinates (X(T), X(T − 10), X(T − 20)) used by the Wolf method, yielding fractional dimension ~1.81, with 95% confidence interval (1.76, 1.87). These results indicate chaos (see Supplementary Fig. 4).

Varying γ2 relative to γ1 = 1 (Fig. 8e) shows regions of presumably chaotic dynamics interspersed with simple oscillation, as found in many chaotic systems45. In particular, we only find chaos if the delays in the two arms differ substantially. Both delays appear to be important, not only their ratio or difference, since no oscillations exist when both delays are less than one. A full exploration of the two-delay parameter space is left to future work.

In constrast, we found quasiperiodic dynamics for dual negative feedback (Supplementary Fig. 5). Qualitatively, the time dynamics appear less pulsatile (Supplementary Fig. 5b) than dual positive/negative feedback, in line with reports that dual positive/negative feedback demonstrates excitability85. The attractor also has a somewhat different profile (Supplementary Fig. 5c) that in reconstructed space (X(T), X(T − 10), X(T − 20)) traces out a torus with box dimension ~2 and near-zero Lyapunov exponent (see Supplementary Fig. 4), indicating quasiperiodic rather than chaotic dynamics. The Fourier spectrum also exhibits a less dense set of peaks (Supplementary Fig. 5d).

Many parameter choices can prevent chaos in dual feedback. Double autoactivation (n1 < 0, n2 < 0) prevents chaos via a stable fixed point (X = 0)45. Identical regulation strength and cooperativity (η1 = η2, n1 = n2) behaves like autoregulation close to fixed points, with γ = γ1 + γ2, as do identical delays (γ1 = γ2), with ηM = η1M1 + η2M2 (Supplementary Note 8). Thus, for chaotic solutions to occur via double feedback, there must be at least one negative feedback arm, the delays must differ, and either the cooperativities or strengths must differ. If biological systems evolved to avoid chaos86,87, it may be that these conditions are selected against, even if the double-feedback motif is not.

The complete parameter space for double feedback is likely to be highly complex, with boundaries possibly formed via an infinite number of bifurcation curves48. In particular, while we demonstrated chaotic and quasiperiodic dynamics for specific parameter values, a more in-depth exploration is warranted to fully understand the dynamics for all parameters. For further exploration of double negative feedback and more complex delayed network motifs that exhibit chaotic dynamics, see Suzuki et al.71.

## Discussion

In this work, we systematically studied the most common network motifs (Table 2) by use of DDEs. Compared to the more widely used mechanistic ODE descriptions, we demonstrated numerous instances where inclusion of explicit delays simplifies complex systems into smaller motifs while continuing to capture their key dynamics that are otherwise lost in simplifying complex systems as small motifs. To further modeling simplicity, we also identified unifying descriptions for activators and repressors (Eq. (7)) and for boolean Hill function logic (Fig. 5) that are useful for both DDE and ODE models.

There are multiple findings to highlight. First, multistep cascades with and without feedback reduce effectively to direct, delayed regulation, providing an intuitive interpretation of biological delays as resulting from multi-step processes, and establishing analytical relationships for converting between parameters of the corresponding ODE and DDE models. Second, delays determine many key motif properties, such as the oscillation period in negative autoregulation, and pulse width and frequency cutoff in feedforward motifs. Interplay of multiple delays plays a similar role in multi-component feedback for determining absolute and relative oscillation periods, and in multiple feedback for determining chaotic dynamics. Third, we showed quantitatively not only when delays are crucial to behavior, as in oscillations, but also when they may be safely ignored. Delays do not affect steady states or logic of independent inputs, only a difference of delays between regulation arms is important in determining feedforward behavior, and only the sum of delays determines the onset of transient oscillations in feedback loops.

Using DDE models to contract a network into a much smaller network with equivalent topology (replacing all cascades with delayed, direct regulation) may aid in discovery of large-scale motifs that have important functions or that might otherwise be perceived as statistically insignificant. The motifs in this paper were originally discovered by scanning biological networks for subnetworks of N nodes that occur more frequently than expected by chance6,11 (this in fact being the original definition of network motifs). However, this discovery method becomes increasingly difficult for N 5 due to the combinatorial scaling of the number of possible motifs96. Performing a similar search on contracted networks would be equivalent to searching the original network for larger motifs, and places a stronger emphasis on topology than on number of components involved (e.g., X → Z being equivalent to X → Y → Z). While this can be done without DDEs, the introduction of delays allows one to perform the contraction while maintaining key information about the original dynamics of retained components.

While our work has focused on the most basic network motif topologies (Table 2), the same techniques can be applied to other more complex networks. Expanding on the eigenmode analysis we performed for multi-component feedback loops, one can describe the linear behavior of an arbitrary network near its fixed points by a matrix form of the characteristic equation $$J{\mathbf{a}}=0$$ with $${J}_{ij}=(\lambda +1){\delta }_{ij}+{\eta }_{j}{M}_{j}{e}^{-{\gamma }_{ij}\lambda }$$ (Supplementary note 9, Supplementary Fig. 6). This approach allows simple models to provide detailed predictions of complex biological phenomena such as pattern formation23.

To clarify the basic role of delays in network motifs, we ignored several phenomena which we suggest that future work explore. These include: (1) Noise, which is intrinsic to many biological systems97 and known to affect dynamics of delay equations15,92,98,99. (2) Time-varying and stochastic delays42,47,98,100. (3) Non-constant initial conditions (“histories”) for T < 0; for example, autorepression shows in-phase and anti-phase locking with constant histories, while sine-wave histories with randomized phase do not (Supplementary Fig. 7). (4) Non-Hill regulation and complex degradation functions26,78, such as zeroth-order101, nonlinear102, or delayed removal, as well as diffusion or spatial effects103. (5) Feedback with multiple delays, which we only briefly analyzed in the double feedback motif, and likely have very complicated parameter spaces48,49.

Overall, we believe that our work may help resolve fundamental biological and engineering questions regarding a variety of phenomena, including transcription factor networks6,17,77, cell cycles87, other biological clocks14,18,25, and pattern formation23,44. Multiple feedback analysis may determine whether chaos affects the cell cycle87, or whether biology avoids possibly chaotic motifs. Delays in multicellular signaling23,32 may distinguish among models of pattern formation16,24,32,103,104. Finally, exploring “contracted” networks with delays may uncover entirely new functional network motifs that are larger and more complex than currently known96.

## Methods

### Analytics and numerical simulation

Analytics were in general performed by hand, and checked for validity using Mathematica. Numerical simulations were run in Matlab using the dde23 delay differential equation solver for DDEs and ode45 for ODEs. Simulating activators as repressors with n < 0 technically fails when x is identically zero (Eq. (5)), since that would imply division by zero, but the limit as x goes to zero causes the regulation term to be zero, which is the same result as assumed by our notation. An initial value of exactly zero for x can thus lead to a divide-by-zero error in simulations, and so initial conditions of exactly zero were not used, as that case is an uninteresting fixed point for activators in any case. Note also that the consitutive case for Eq. (5) is degenerate, in that n = 0, α ≠ 0 is equivalent to n ≠ 0, α = 0 with α0 → α0 + α/2.

### Phase plot simulations and analysis

For autoregulation phase plots, simulations were run with 100 constant-history initial conditions spread logarithmically between 10−4 and 2η and run from T = 0 to T = 100(γ + 1). Solutions were considered stable if for all 100 simulations the maximum absolute value of the discrete derivative in the last three-quarters of the simulation time was less than 0.1. Stable solutions were sub-categorized as bistable if a histogram of final values over all 100 solutions had more than 1 peak. Solutions were considered oscillatory if the average Fourier transform of the last three-quarters of the simulation time for all 100 solutions had more than zero peaks with amplitude (square root of power) greater than 100. Solutions were considered spiral if this oscillation condition held for the first one-quarter of the simulation time only. For two-component loops, initial conditions were used that ranged between 0 and $$\max ({\eta }_{1},{\eta }_{2})$$, for equal X and Y and for apposing X and Y. Bistability was determined as for autoregulation, and a cutoff of 0.05 was used to determine “low” values. All simulation histories were constant except where indicated in Supplementary Fig. 7. Specific parameter values and simulation details are given in the figures and/or made explicit in the MATLAB code in Supplementary Data 1.

### Lyapunov exponents and box dimensions

For calculating dominant Lyapunov exponents, we used the Wolf method95 with recommended parameters from the MATLAB script “Wolf Lyapunov exponent estimation from a time series” (version 1.2.0.1) provided by the authors on the Mathworks FileExchange (https://www.mathworks.com/matlabcentral/fileexchange/48084-wolf-lyapunov-exponent-estimation-from-a-time-series). In particular, we used an embedding dimension of 3 and a phase space reconstruction delay of 10 for a dataset with ~350 orbits of ~50–60 data points per orbit. The last 50 iterations of the algorithm were used to generate a mean and standard deviation of the estimated dominant Lyapunov exponent. A full set of parameters can be found in the included code. For calculating attractor dimensions, we used the box-counting method, whose code is also provided in Supplementary Data 1, on the reconstructed phase space (X(T), X(T − 10), X(T − 20)) used for calculating Lyapunov exponents. A linear regression was performed using MATLAB’s fit function on half the linear domain to generate a mean and confidence interval (using MATLAB’s confint) for the slope (i.e., box dimension) between numbers of boxes covering the attractor versus length of each box.

### Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.