Abstract
The microscopic principles organizing dynamic units in complex networks—from proteins to power generators—can be understood in terms of network ‘motifs’: small interconnection patterns that appear much more frequently in real networks than expected in random networks1,2. When considered as small subgraphs isolated from a large network, these motifs are more robust to parameter variations, easier to synchronize than other possible subgraphs, and can provide specific functionalities3,4,5,6,7,8,9,10,11,12,13,14,15. But one can isolate these subgraphs only by assuming, for example, a significant separation of timescales, and the origin of network motifs and their functionalities when embedded in larger networks remain unclear. Here we show that most motifs emerge from interconnection patterns that best exploit the intrinsic stability characteristics at different scales of interconnection, from simple nodes to whole modules. This functionality suggests an efficient mechanism to stably build complex systems by recursively interconnecting nodes and modules as motifs. We present direct evidence of this mechanism in several biological networks.
Similar content being viewed by others
Main
In complex natural systems such as biological networks, the particular topologies of network motifs have been shaped by evolution. Evolution and natural selection accumulate stable intermediate components (that is, nodes), which are interconnected to form more complex systems. This modular design principle is observed at many scales, from the motion control architecture of vertebrates to the emotional response of human beings16,17,18,19. Yet, in general, the interconnection of stable components may result in an unstable system20. Thus, it is natural to consider that nature favours interconnections that make it easier to obtain a stable networked system. In this letter, we show that most network motifs in real networks emerge precisely from such consideration. This property can be used in building larger systems by applying it at different scales of interconnection.
To start, we consider a set of N nodes with scalar dynamics of the form
with initial condition xi(t0) = xi0, i = 1, …, N. Here the scalars xi, ui and yi are the state, input and output of node i, respectively. The state of a node may represent the expression level of a gene, the concentration of a metabolite, the charge of a capacitor, and so on. Vector dynamics are discussed later in the context of modules. The functions fi(xi, t), which are typically nonlinear, determine the nodal dynamics.
Nodes interact with each other by interconnecting their inputs with their outputs . Analysing the stability of networked systems with nonlinear interconnections
requires knowledge of the functional form and parameters of the interconnection g(y) and nodal dynamics fi(xi, t), which is hard to obtain in most systems (Supplementary Information 1.7). Linear interconnections do not require such knowledge, enabling us to quantify the contribution of the interconnection to the stability of the networked system without the need of detailed knowledge of the nodal dynamics. More precisely, by considering
where is the weighted adjacency matrix of the interconnection network, our analysis requires a single constant per node—its contraction rate, defined later on—characterizing its intrinsic stability properties. Here aij ≠ 0 represents a directed edge from node j to node i. In general, the linear interconnection (3) can be used to approximate (2) in some working range5,7,21. Furthermore, diffusive coupling of oscillators and several models of neural networks actually use linear interconnection networks22,23,24.
Our standing assumption on the isolated nodes is that they are stable, and we aim to quantify for which interconnections it is easier to get a stable networked system. The separate contribution of the isolated nodes and the interconnection to the stability of the networked system is made transparent by using contraction theory. Contraction theory is a tool to analyse the stability of dynamic systems based on a differential-geometric viewpoint inspired by fluid mechanics25, in contrast to Lyapunov stability theory, which is based on analogues of mechanical energy. A system is contracting if the trajectories associated with any two initial conditions exponentially converge towards each other. More precisely, a dynamic system of the form
with state is contracting with rate α > 0 if there exists a vector norm |⋅| and constant β > 0 such that for any two initial conditions their corresponding trajectories x(xa, ⋅), x(xb, ⋅) satisfy
Denote by J(x, t) = ∂xf(x, t) the Jacobian of system (4). Then contraction is equivalent to the existence of a matrix measure μ such that μ(J(x, t)) ≤ −α, for all and t ≥ t0 (refs 25, 26). Any vector norm |⋅| induces a matrix norm ∥⋅∥ and a matrix measure μ by
both of which are well defined for any matrix . In particular, we prove that
where is the set of all matrix measures in and λi(A) are the eigenvalues of A (Theorem 1 in Supplementary Information 1). We use the notation μA to emphasize that the matrix measure achieving the minimum in (5) depends on the matrix A itself. Recall also that matrix measures are subadditive: μ(A1 + A2) ≤ μ(A1) + μ(A2), for any .
In the case of scalar isolated systems, as in (1) with ui = 0, contraction with rate αi is equivalent to the condition Ji(xi, t) = f(xi, t) ≤ −αi for all and t ≥ t0. The contraction property of isolated nodes might be lost when they are interconnected, so that the networked system is no longer contracting. Indeed, owing to the subadditivity of matrix measures, the Jacobian of the networked system (1)–(3) satisfies
where Ji = Ji(xi, t) and Dα = diag(α1, …, αN).
We define μ(A) as the contraction loss of the interconnection network. Then the inequality (6) indicates that the networked system remains contracting if the effective contraction of the isolated nodes μ(−Dα) < 0 dominates the contraction loss μ(A) due to the interconnection. Consequently, interconnections with small contraction loss best favour stability, as they require smaller contraction rates from the isolated nodes to keep the whole network contracting.
The choice of matrix measure in (6) is a degree of freedom that should be optimized to make μ(−Dα) + μ(A) as negative as possible. Solving this optimization problem is not trivial, as the matrix measure minimizing the contraction loss μ(A) might also decrease the effective contraction of the isolated nodes (that is, making the term μ(−Dα) less negative). In Proposition 1 of Supplementary Information 1, we prove that the optimal matrix measure is given by μA defined in (5), provided that the contraction rates of all nodes are equal, or that the off-diagonal entries of A are non-negative (that is, interactions between nodes are only positive). Choosing the matrix measure (5), the contraction loss of some classes of networks depends only on their topology. For example, Proposition 2a of Supplementary Information 1.4 shows that acyclic networks have zero contraction loss regardless of their edge-weights (positive or not). This implies that feedforward and bifan interconnections, and their generalizations27, always have zero contraction loss (Supplementary Information 1.5). Moreover, non-positive contraction loss for interactions with arbitrary strength requires that reciprocal interactions have opposite signs, and the absence of cycles (feedback loops) of length 3 or more, see Proposition 2b in Supplementary Information 1.4. In particular, the contraction loss of negative feedback between two nodes is at most zero (Supplementary Information 1.4).
Interestingly, the presence of negative interactions always improves the stability of the networked system by decreasing the contraction loss, in the sense that for any and , see Lemma 1 in Supplementary Information 1.3. Here is defined as and if j ≠ i. Therefore, it is possible to add functionalities requiring negative interactions by replacing a positive interaction by a negative one without decreasing the stability of the networked system. This result motivated us to consider positive interactions only in the rest of the paper because they provide the worst-case analysis of the contraction loss of an interconnection with respect to all possible edge-weights, and their contraction loss can be optimally computed using μA.
We analysed the contraction loss of all 3- or 4-node subgraphs and identified those with the lowest contraction loss in their density class, defined as the set of all subgraphs with the same number of nodes and edges5. Those subgraphs with lowest contraction loss best favour the stability of the networked system. As the precise value of the interactions may change from one system to other, we randomly select them from a uniform distribution on [0, amax] to form an ensemble of 10,000 weighted adjacency matrices with the same connectivity pattern. From this ensemble the mean contraction loss 〈μA〉 is computed, see Fig. 1. Supplementary Information 2 details how to compute analytically and numerically the mean contraction loss of a subgraph. The observed order of the subgraphs according to their mean contraction loss is independent of the interaction strength distribution and the value of amax > 0 (Supplementary Information 3). See also Supplementary Information 4 for results using a different matrix measure to compute the contraction loss.
We find that all motifs reported in ref. 1 (except M3, M9 and M12, which contain feedback loops of length 3 or more) emerge among the subgraphs with the minimum 〈μA〉 within their respective density class. In particular, all motifs found in biological networks (marked in circles in Fig. 1) have zero contraction loss. Motifs containing feedback loops with length 3 or more do not have the minimum contraction loss within their density class—they do not favour stability—and these motifs also require more accurate tuning of their edge-weights to be stable (Supplementary Information 1.4). Recall also that feedback provides functionalities associated with performance (such as robustness to external disturbances25), which do not necessarily favour the stability of the networked system.
To further disentangle the relation between network motifs and subgraphs with low contraction loss, we compared the Z-score and relative contraction loss of subgraphs in several real networks. As introduced in ref. 1, the Z-score of a subgraph A in a real network quantifies its statistical significance as a motif, and is defined by
where Nreal is the number of occurrences of subgraph A in the real network, 〈Nrand〉 the average number of occurrences in an ensemble of its randomizations, and σrand its standard deviation. A subgraph with a high (low) Z-score is over(under)-represented in the real network. The normalized Z-score of a subgraph is its Z-score divided by the maximum (in absolute value) Z-score of all subgraphs with the same number of nodes.
We define the relative contraction loss of a subgraph A as
where μmax (or μmin) is the maximum (or minimum) mean contraction loss among all subgraphs within the density class of A. The case r(A) = 0 (or r(A) = 1) corresponds to a subgraph with the minimal (or maximal) mean contraction loss among its density class. The relative contraction loss is undefined for all subgraphs with 3 nodes/2 edges or 4 nodes/3 edges (that is, μmin = μmax = 0), as they are acyclic and thus have zero contraction loss (Proposition 2a of Supplementary Information 1.4). Hence, we discard them from the discussion that follows as subgraphs with zero contraction loss maintain the stability of the networked system.
We compared the relative contraction loss and the normalized Z-score of 3- and 4-node subgraphs in several biological networks, finding that over-represented subgraphs (motifs) tend to have low relative contraction loss, see Fig. 2. The phenomenon is stronger for 3-node subgraphs, where under-represented subgraphs (anti-motifs) have high relative contraction loss. In other words, subgraphs that favour stability are over-represented, whereas 3-node subgraphs which do not favour stability are under-represented. We did not find this phenomenon in other classes of networks containing feedback motifs with high Z-score (such as the electronic circuits shown in Supplementary Information 6), suggesting that other factors apart from maintaining stability play a central role in their construction.
Next we explore how the small contraction loss property of motifs can be used to build bigger networked systems. Consider a set of N modules (that is, connected subgraphs) possibly having vector dynamics
i = 1, …, N, where , and are the state, input and output vectors of module i. The matrices and determine which nodes of the module interact with other modules. The interconnection of modules is again described by equation (3), but the matrix is no longer necessarily square because some modules may have different numbers of inputs and outputs.
Each isolated module is assumed contracting with rate αi > 0 under measure μi. The contraction rate of a module can be calculated using the contraction rate of its internal nodes and their respective interconnection topology Ai. To each module, we associate a condensed node, with scalar state and linear dynamics, that inherits the module’s contraction rate
In addition, we use the interconnection network of the full system to define a condensed weighted adjacency matrix as follows:
where Mij = BiAijCj, and is the (i, j) block of the original A interconnection network (3), see Supplementary Fig. 1. Above, ∥⋅∥ i, j stands for the induced matrix norm
with |x| i = |Pi1/2x| 2 a weighted Euclidean norm with metric found as the solution to the linear matrix inequality of Theorem 1 in Supplementary Information 1. When the off-diagonal elements of Ai are non-negative, a diagonal solution to such a matrix inequality exists and the metric Pi just assigns different units to different modules (Supplementary Information 1). Also, in the case when each module has a single input and a single output, Acond takes a particular simple form in which its (i, j) entry is |γijAij| if i ≠ j and γijAij if i = j, with .
In Theorem 2 of Supplementary Information 1 we prove that if the condensed networked system (8)–(9) is contracting, then the original system (7)–(3) is also contracting. This result also holds when instantaneous contraction rates are used for the modules, making the condensed dynamics (8) dependent on the states of the original dynamics (Supplementary Information 1.6). Hence, the interconnections between modules have minimal contraction loss if they are also network motifs. This suggests a modular design principle to build complex systems, starting by building modules interconnecting nodes as network motifs, and then interconnecting those modules again as network motifs.
To better illustrate this point, consider the feedback interconnection of three 3-node motifs shown in Fig. 3a. Each isolated motif will be contracting if
where αj, min is the minimum contraction rate of the nodes inside the jth motif, and Aj is its internal interconnection. Indeed, αj is just the contraction rate of motif j. The smaller is the contraction loss of the internal topology, the larger is the contraction inherited by the module. As the contraction loss of feedforward motifs is zero, in this example each condensed node inherits the minimal contraction rate of its nodes, that is, αj = αj, min. To construct the condensed interconnection network, we first note that
The interconnection of the modules is described by the adjacency matrix of the 3-node feedback interconnection motif , whose only non-zero values are A12, A23 and A31 (see Fig. 3a). Then, it is not surprising that the corresponding Acond obtained using (9) is again the adjacency matrix of a 3-node feedback interconnection. The condensed interconnected system will be contracting if αmin = min{α1, α2, α3} > μcond(Acond) = μA(A). In addition, Theorem 2 of Supplementary Information 1 implies that under such a condition the original interconnected system is also contracting. As the constraints were imposed in the contraction rates only, the details of the node dynamics were not used in the analysis.
The contraction loss of the interconnection plays an important role in the stability of the whole network, because modules inherit a larger contraction rate when their internal interconnection has a smaller contraction loss. And this applies at different scales of interconnection: if the system is recursively condensed, the resulting condensed modules at each step inherit larger contraction rates when the interconnections between the modules in the previous step have smaller contraction loss. In this form, the interconnection of ‘motifs of motifs’ is a recursive and modular network design procedure in which the contraction loss remains minimal at each step of construction of the network. Both humans and nature seem to favour this design principle by recursively interconnecting already designed modules28,29.
The idea of ‘motifs of motifs’ was used in ref. 30 to reverse-engineer electronic circuits and coarse-grain a signal-transduction protein network. In contrast, here we aim to check if motifs at different scales still have low relative contraction loss, thus providing direct evidence of a design principle found in real-world complex networks. We used a collection of real networks to test our hypothesis by recursively searching and condensing motifs (details of the method and used networks are found in Supplementary Information 5 and 7, respectively). We found that most motifs in the original and condensed regulatory networks of Escherichia coli and yeast have low relative contraction loss, Fig. 3. For the neuron network of Caenorhabditis elegans this only happens for 3-node motifs. A closer analysis reveals that most 4-node motifs in the C. elegans with high relative contraction loss also have small Z-score, see Fig. 2. In other words, they are not strongly over-represented.
References
Milo, R. et al. Network motifs: Simple building blocks of complex networks. Science 298, 824–827 (2002).
Alon, U. Network motifs: Theory and experimental approaches. Nature Rev. Genetics 8, 450–461 (2007).
Mangan, S. & Alon, U. Structure and function of the feed-forward loop network motif. Proc. Natl Acad. Sci. USA 100, 11980–11985 (2003).
Mangan, S., Zaslaver, A. & Alon, U. The coherent feedforward loop serves as a sign-sensitive delay element in transcription networks. J. Mol. Biol. 334, 197–204 (2003).
Prill, R. J., Iglesias, P. A. & Levchenko, A. Dynamic properties of network motifs contribute to biological network organization. PLoS Biol. 3, e343 (2005).
Lodato, I., Boccaletti, S. & Latora, V. Synchronization properties of network motifs. Europhys. Lett. 78, 28001 (2007).
Ma, W., Trusina, A., El-Samad, H., Lim, W. A. & Tang, C. Defining network topologies that can achieve biochemical adaptation. Cell 138, 760–773 (2009).
Shoval, O. et al. Fold-change detection and scalar symmetry of sensory input fields. Proc. Natl Acad. Sci. USA 107, 15995–16000 (2010).
Rodrigo, G. & Elena, S. F. Structural discrimination of robustness in transcriptional feed-forward loops for pattern formation. PLoS ONE 6, e16904 (2011).
Wang, P., Lu, J. & Ogorzalek, M. J. Global relative parameter sensitivities of the feedforward loops in genetic networks. Neurocomputing 78, 155–165 (2012).
Hart, Y., Antebi, Y. E., Mayo, A. E., Friedman, N. & Alon, U. Design principles of cell circuits with paradoxical components. Proc. Natl Acad. Sci. USA 109, 8346–8351 (2012).
Genot, A. J., Fujii, T. & Rondelez, Y. Computing with competition in biochemical networks. Phys. Rev. Lett. 109, 208102 (2012).
Hart, Y. & Alon, U. The utility of paradoxical components in biological circuits. Mol. Cell 49, 213–221 (2013).
Hart, Y., Mayo, A. E., Shoval, O. & Alon, U. Comparing apples and oranges: Fold-change detection of multiple simultaneous inputs. PLoS ONE 8, e57455 (2013).
Adler, M., Mayo, A. & Alon, U. Logarithmic and power law input–output relations in sensory systems with fold-change detection. PLoS Comput. Biol. 10, e1003781 (2014).
Bernstein, N. A. The Co-ordination and Regulation of Movements (Pergamon Press, 1967).
Bizzi, E., Giszter, S. F., Loeb, E., Mussa-Ivaldi, F. A. & Saltiel, P. Modular organization of motor behavior in the frog’s spinal cord. Trends Neurosci. 18, 442–446 (1995).
LeDoux, J. The Emotional Brain: The Mysterious Underpinnings of Emotional Life (Simon and Schuster, 1998).
Kashtan, N. & Alon, U. Spontaneous evolution of modularity and network motifs. Proc. Natl Acad. Sci. USA 102, 13773–13778 (2005).
Slotine, J.-J. & Lohmiller, W. Modularity, evolution, and the binding problem: A view from stability theory. Neural Netw. 14, 137–145 (2001).
Allesina, S. & Tang, S. Stability criteria for complex ecosystems. Nature 483, 205–208 (2012).
Han, S. K., Kurrer, C. & Kuramoto, Y. Dephasing and bursting in coupled neural oscillators. Phys. Rev. Lett. 75, 3190–3193 (1995).
Campbell, S. & Wang, D. Synchronization and desynchronization in a network of locally coupled Wilson-Cowan oscillators. IEEE Trans. Neural Netw. 7, 541–554 (1996).
Nakao, H. & Mikhailov, A. S. Turing patterns in network-organized activator-inhibitor systems. Nature Phys. 6, 544–550 (2010).
Lohmiller, W. & Slotine, J.-J. E. On contraction analysis for non-linear systems. Automatica 34, 683–696 (1998).
Russo, G., di Bernardo, M. & Sontag, E. A contraction approach to the hierarchical analysis and design of networked systems. IEEE Trans. Autom. Control 58, 1328–1331 (2013).
Kashtan, N., Itzkovitz, S., Milo, R. & Alon, U. Topological generalizations of network motifs. Phys. Rev. E 70, 031909 (2004).
Simon, H. A. The architecture of complexity. Proc. Am. Phil. Soc. 106, 467–482 (1962).
Alon, U. Biological networks: The tinkerer as an engineer. Science 301, 1866–1867 (2003).
Itzkovitz, S. et al. Coarse-graining and self-dissimilarity of complex networks. Phys. Rev. E 71, 016127 (2005).
Acknowledgements
We thank A.-L. Barabási for valuable discussions. M.T.A. was supported by the CONACyT postdoctoral grant 207609, the ARL NS-CTA grant (W911NF-09-2-0053), and the DARPA grant (W911NF-12-C-002). Y.-Y.L. was supported in part by the John Templeton Foundation (award number 51977).
Author information
Authors and Affiliations
Contributions
All authors designed and did the research. M.T.A. analysed the empirical data and did the analytical and numerical calculations. M.T.A. and Y.-Y.L. wrote the manuscript. J.-J.S. edited the manuscript.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Information
Supplementary Information (PDF 10470 kb)
Rights and permissions
About this article
Cite this article
Angulo, M., Liu, YY. & Slotine, JJ. Network motifs emerge from interconnections that favour stability. Nature Phys 11, 848–852 (2015). https://doi.org/10.1038/nphys3402
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nphys3402
This article is cited by
-
Time delays modulate the stability of complex ecosystems
Nature Ecology & Evolution (2023)