Network motifs emerge from interconnections that favour stability

Abstract

The microscopic principles organizing dynamic units in complex networks—from proteins to power generators—can be understood in terms of network ‘motifs’: small interconnection patterns that appear much more frequently in real networks than expected in random networks^1,2. When considered as small subgraphs isolated from a large network, these motifs are more robust to parameter variations, easier to synchronize than other possible subgraphs, and can provide specific functionalities^{3,4,5,6,7,8,9,10,11,12,13,14,15}. But one can isolate these subgraphs only by assuming, for example, a significant separation of timescales, and the origin of network motifs and their functionalities when embedded in larger networks remain unclear. Here we show that most motifs emerge from interconnection patterns that best exploit the intrinsic stability characteristics at different scales of interconnection, from simple nodes to whole modules. This functionality suggests an efficient mechanism to stably build complex systems by recursively interconnecting nodes and modules as motifs. We present direct evidence of this mechanism in several biological networks.

Main

In complex natural systems such as biological networks, the particular topologies of network motifs have been shaped by evolution. Evolution and natural selection accumulate stable intermediate components (that is, nodes), which are interconnected to form more complex systems. This modular design principle is observed at many scales, from the motion control architecture of vertebrates to the emotional response of human beings^16,17,18,19. Yet, in general, the interconnection of stable components may result in an unstable system²⁰. Thus, it is natural to consider that nature favours interconnections that make it easier to obtain a stable networked system. In this letter, we show that most network motifs in real networks emerge precisely from such consideration. This property can be used in building larger systems by applying it at different scales of interconnection.

To start, we consider a set of N nodes with scalar dynamics of the form

with initial condition x_i(t₀) = x_i0, i = 1, …, N. Here the scalars x_i, u_i and y_i are the state, input and output of node i, respectively. The state of a node may represent the expression level of a gene, the concentration of a metabolite, the charge of a capacitor, and so on. Vector dynamics are discussed later in the context of modules. The functions f_i(x_i, t), which are typically nonlinear, determine the nodal dynamics.

Nodes interact with each other by interconnecting their inputs with their outputs . Analysing the stability of networked systems with nonlinear interconnections

requires knowledge of the functional form and parameters of the interconnection g(y) and nodal dynamics f_i(x_i, t), which is hard to obtain in most systems (Supplementary Information 1.7). Linear interconnections do not require such knowledge, enabling us to quantify the contribution of the interconnection to the stability of the networked system without the need of detailed knowledge of the nodal dynamics. More precisely, by considering

where is the weighted adjacency matrix of the interconnection network, our analysis requires a single constant per node—its contraction rate, defined later on—characterizing its intrinsic stability properties. Here a_ij ≠ 0 represents a directed edge from node j to node i. In general, the linear interconnection (3) can be used to approximate (2) in some working range^5,7,21. Furthermore, diffusive coupling of oscillators and several models of neural networks actually use linear interconnection networks^22,23,24.

Our standing assumption on the isolated nodes is that they are stable, and we aim to quantify for which interconnections it is easier to get a stable networked system. The separate contribution of the isolated nodes and the interconnection to the stability of the networked system is made transparent by using contraction theory. Contraction theory is a tool to analyse the stability of dynamic systems based on a differential-geometric viewpoint inspired by fluid mechanics²⁵, in contrast to Lyapunov stability theory, which is based on analogues of mechanical energy. A system is contracting if the trajectories associated with any two initial conditions exponentially converge towards each other. More precisely, a dynamic system of the form

with state is contracting with rate α > 0 if there exists a vector norm |⋅| and constant β > 0 such that for any two initial conditions their corresponding trajectories x(x_a, ⋅), x(x_b, ⋅) satisfy

Denote by J(x, t) = ∂_xf(x, t) the Jacobian of system (4). Then contraction is equivalent to the existence of a matrix measure μ such that μ(J(x, t)) ≤ −α, for all and t ≥ t₀ (refs 25, 26). Any vector norm |⋅| induces a matrix norm ∥⋅∥ and a matrix measure μ by

both of which are well defined for any matrix . In particular, we prove that

where is the set of all matrix measures in and λ_i(A) are the eigenvalues of A (Theorem 1 in Supplementary Information 1). We use the notation μ_A to emphasize that the matrix measure achieving the minimum in (5) depends on the matrix A itself. Recall also that matrix measures are subadditive: μ(A₁ + A₂) ≤ μ(A₁) + μ(A₂), for any .

In the case of scalar isolated systems, as in (1) with u_i = 0, contraction with rate α_i is equivalent to the condition J_i(x_i, t) = ${\partial }_{x_i}$ f(x_i, t) ≤ −α_i for all and t ≥ t₀. The contraction property of isolated nodes might be lost when they are interconnected, so that the networked system is no longer contracting. Indeed, owing to the subadditivity of matrix measures, the Jacobian of the networked system (1)–(3) satisfies

where J_i = J_i(x_i, t) and D_α = diag(α₁, …, α_N).

We define μ(A) as the contraction loss of the interconnection network. Then the inequality (6) indicates that the networked system remains contracting if the effective contraction of the isolated nodes μ(−D_α) < 0 dominates the contraction loss μ(A) due to the interconnection. Consequently, interconnections with small contraction loss best favour stability, as they require smaller contraction rates from the isolated nodes to keep the whole network contracting.

The choice of matrix measure in (6) is a degree of freedom that should be optimized to make μ(−D_α) + μ(A) as negative as possible. Solving this optimization problem is not trivial, as the matrix measure minimizing the contraction loss μ(A) might also decrease the effective contraction of the isolated nodes (that is, making the term μ(−D_α) less negative). In Proposition 1 of Supplementary Information 1, we prove that the optimal matrix measure is given by μ_A defined in (5), provided that the contraction rates of all nodes are equal, or that the off-diagonal entries of A are non-negative (that is, interactions between nodes are only positive). Choosing the matrix measure (5), the contraction loss of some classes of networks depends only on their topology. For example, Proposition 2a of Supplementary Information 1.4 shows that acyclic networks have zero contraction loss regardless of their edge-weights (positive or not). This implies that feedforward and bifan interconnections, and their generalizations²⁷, always have zero contraction loss (Supplementary Information 1.5). Moreover, non-positive contraction loss for interactions with arbitrary strength requires that reciprocal interactions have opposite signs, and the absence of cycles (feedback loops) of length 3 or more, see Proposition 2b in Supplementary Information 1.4. In particular, the contraction loss of negative feedback between two nodes is at most zero (Supplementary Information 1.4).

Interestingly, the presence of negative interactions always improves the stability of the networked system by decreasing the contraction loss, in the sense that for any and , see Lemma 1 in Supplementary Information 1.3. Here is defined as and if j ≠ i. Therefore, it is possible to add functionalities requiring negative interactions by replacing a positive interaction by a negative one without decreasing the stability of the networked system. This result motivated us to consider positive interactions only in the rest of the paper because they provide the worst-case analysis of the contraction loss of an interconnection with respect to all possible edge-weights, and their contraction loss can be optimally computed using μ_A.

We analysed the contraction loss of all 3- or 4-node subgraphs and identified those with the lowest contraction loss in their density class, defined as the set of all subgraphs with the same number of nodes and edges⁵. Those subgraphs with lowest contraction loss best favour the stability of the networked system. As the precise value of the interactions may change from one system to other, we randomly select them from a uniform distribution on [0, a_max] to form an ensemble of 10,000 weighted adjacency matrices with the same connectivity pattern. From this ensemble the mean contraction loss 〈μ_A〉 is computed, see Fig. 1. Supplementary Information 2 details how to compute analytically and numerically the mean contraction loss of a subgraph. The observed order of the subgraphs according to their mean contraction loss is independent of the interaction strength distribution and the value of a_max > 0 (Supplementary Information 3). See also Supplementary Information 4 for results using a different matrix measure to compute the contraction loss.

Figure 1: Mean contraction loss of all 3- or 4-node subgraphs.

a, Mean contraction loss of all subgraphs with 3 and 4 nodes (a_max = 1), the same colour indicates the same number of edges. Vertical dashed lines separate groups of subgraphs with the same number of edges (density classes) in increasing order from left to right (2 to 6 edges for 3-node subgraphs, and 3 to 12 edges for 4-node subgraphs). Grey markers show the network motifs reported in ref. 1, circles denote biological networks (gene transcription, neurons and food webs) and triangles denote man-made networks (electronic circuits and the World Wide Web). Motif M2 appears in both gene regulatory networks and some electronic circuits (forward logic chips). Error bars represent standard deviation. b, The 12 network motifs reported in ref. 1.

We find that all motifs reported in ref. 1 (except M₃, M₉ and M₁₂, which contain feedback loops of length 3 or more) emerge among the subgraphs with the minimum 〈μ_A〉 within their respective density class. In particular, all motifs found in biological networks (marked in circles in Fig. 1) have zero contraction loss. Motifs containing feedback loops with length 3 or more do not have the minimum contraction loss within their density class—they do not favour stability—and these motifs also require more accurate tuning of their edge-weights to be stable (Supplementary Information 1.4). Recall also that feedback provides functionalities associated with performance (such as robustness to external disturbances²⁵), which do not necessarily favour the stability of the networked system.

To further disentangle the relation between network motifs and subgraphs with low contraction loss, we compared the Z-score and relative contraction loss of subgraphs in several real networks. As introduced in ref. 1, the Z-score of a subgraph A in a real network quantifies its statistical significance as a motif, and is defined by

where N_real is the number of occurrences of subgraph A in the real network, 〈N_rand〉 the average number of occurrences in an ensemble of its randomizations, and σ_rand its standard deviation. A subgraph with a high (low) Z-score is over(under)-represented in the real network. The normalized Z-score of a subgraph is its Z-score divided by the maximum (in absolute value) Z-score of all subgraphs with the same number of nodes.

We define the relative contraction loss of a subgraph A as

where μ_max (or μ_min) is the maximum (or minimum) mean contraction loss among all subgraphs within the density class of A. The case r(A) = 0 (or r(A) = 1) corresponds to a subgraph with the minimal (or maximal) mean contraction loss among its density class. The relative contraction loss is undefined for all subgraphs with 3 nodes/2 edges or 4 nodes/3 edges (that is, μ_min = μ_max = 0), as they are acyclic and thus have zero contraction loss (Proposition 2a of Supplementary Information 1.4). Hence, we discard them from the discussion that follows as subgraphs with zero contraction loss maintain the stability of the networked system.

We compared the relative contraction loss and the normalized Z-score of 3- and 4-node subgraphs in several biological networks, finding that over-represented subgraphs (motifs) tend to have low relative contraction loss, see Fig. 2. The phenomenon is stronger for 3-node subgraphs, where under-represented subgraphs (anti-motifs) have high relative contraction loss. In other words, subgraphs that favour stability are over-represented, whereas 3-node subgraphs which do not favour stability are under-represented. We did not find this phenomenon in other classes of networks containing feedback motifs with high Z-score (such as the electronic circuits shown in Supplementary Information 6), suggesting that other factors apart from maintaining stability play a central role in their construction.

Figure 2: Relative contraction loss versus normalized Z-score.

Each marker represents a subgraph of the neuron network of C. elegans, the gene transcription networks of yeast and E. coli, and the food web at Saint Martin Island. Subgraphs with high Z-score tend to have small relative contraction loss. In the case of 3-node subgraphs, under-represented subgraphs (anti-motifs) tend to have high relative contraction loss.

Next we explore how the small contraction loss property of motifs can be used to build bigger networked systems. Consider a set of N modules (that is, connected subgraphs) possibly having vector dynamics

i = 1, …, N, where , and are the state, input and output vectors of module i. The matrices and determine which nodes of the module interact with other modules. The interconnection of modules is again described by equation (3), but the matrix is no longer necessarily square because some modules may have different numbers of inputs and outputs.

Each isolated module is assumed contracting with rate α_i > 0 under measure μ_i. The contraction rate of a module can be calculated using the contraction rate of its internal nodes and their respective interconnection topology A_i. To each module, we associate a condensed node, with scalar state and linear dynamics, that inherits the module’s contraction rate

In addition, we use the interconnection network of the full system to define a condensed weighted adjacency matrix as follows:

where M_ij = B_iA_ijC_j, and is the (i, j) block of the original A interconnection network (3), see Supplementary Fig. 1. Above, ∥⋅∥ _{i, j} stands for the induced matrix norm

with |x| _i = |P_i^1/2x| ₂ a weighted Euclidean norm with metric found as the solution to the linear matrix inequality of Theorem 1 in Supplementary Information 1. When the off-diagonal elements of A_i are non-negative, a diagonal solution to such a matrix inequality exists and the metric P_i just assigns different units to different modules (Supplementary Information 1). Also, in the case when each module has a single input and a single output, A_cond takes a particular simple form in which its (i, j) entry is |γ_ijA_ij| if i ≠ j and γ_ijA_ij if i = j, with .

In Theorem 2 of Supplementary Information 1 we prove that if the condensed networked system (8)–(9) is contracting, then the original system (7)–(3) is also contracting. This result also holds when instantaneous contraction rates are used for the modules, making the condensed dynamics (8) dependent on the states of the original dynamics (Supplementary Information 1.6). Hence, the interconnections between modules have minimal contraction loss if they are also network motifs. This suggests a modular design principle to build complex systems, starting by building modules interconnecting nodes as network motifs, and then interconnecting those modules again as network motifs.

To better illustrate this point, consider the feedback interconnection of three 3-node motifs shown in Fig. 3a. Each isolated motif will be contracting if

where α_{j, min} is the minimum contraction rate of the nodes inside the jth motif, and A_j is its internal interconnection. Indeed, α_j is just the contraction rate of motif j. The smaller is the contraction loss of the internal topology, the larger is the contraction inherited by the module. As the contraction loss of feedforward motifs is zero, in this example each condensed node inherits the minimal contraction rate of its nodes, that is, α_j = α_{j, min}. To construct the condensed interconnection network, we first note that

The interconnection of the modules is described by the adjacency matrix of the 3-node feedback interconnection motif , whose only non-zero values are A₁₂, A₂₃ and A₃₁ (see Fig. 3a). Then, it is not surprising that the corresponding A_cond obtained using (9) is again the adjacency matrix of a 3-node feedback interconnection. The condensed interconnected system will be contracting if α_min = min{α₁, α₂, α₃} > μ_cond(A_cond) = μ_A(A). In addition, Theorem 2 of Supplementary Information 1 implies that under such a condition the original interconnected system is also contracting. As the constraints were imposed in the contraction rates only, the details of the node dynamics were not used in the analysis.

Figure 3: Relative contraction loss of motifs at different scales of interconnection.

a, Interconnection of motifs as motifs. The original network is condensed using (8) and (9) into the condensed network. Contraction of the condensed network ensures contraction of the original network. b, Original and two consecutive condensations for the yeast transcription network. Network motifs are recursively found and condensed into a single node, see Supplementary Information 5 for details. c, Number of motifs versus relative contraction loss for the original and condensed networks. The Saint Martin food-web network is not shown as it does not contain motifs after a single condensation.

The contraction loss of the interconnection plays an important role in the stability of the whole network, because modules inherit a larger contraction rate when their internal interconnection has a smaller contraction loss. And this applies at different scales of interconnection: if the system is recursively condensed, the resulting condensed modules at each step inherit larger contraction rates when the interconnections between the modules in the previous step have smaller contraction loss. In this form, the interconnection of ‘motifs of motifs’ is a recursive and modular network design procedure in which the contraction loss remains minimal at each step of construction of the network. Both humans and nature seem to favour this design principle by recursively interconnecting already designed modules^28,29.

The idea of ‘motifs of motifs’ was used in ref. 30 to reverse-engineer electronic circuits and coarse-grain a signal-transduction protein network. In contrast, here we aim to check if motifs at different scales still have low relative contraction loss, thus providing direct evidence of a design principle found in real-world complex networks. We used a collection of real networks to test our hypothesis by recursively searching and condensing motifs (details of the method and used networks are found in Supplementary Information 5 and 7, respectively). We found that most motifs in the original and condensed regulatory networks of Escherichia coli and yeast have low relative contraction loss, Fig. 3. For the neuron network of Caenorhabditis elegans this only happens for 3-node motifs. A closer analysis reveals that most 4-node motifs in the C. elegans with high relative contraction loss also have small Z-score, see Fig. 2. In other words, they are not strongly over-represented.