Self-organization and pattern formation in network-organized systems emerges from the collective activation and interaction of many interconnected units. A striking feature of these non-equilibrium structures is that they are often localized and robust: only a small subset of the nodes, or cell assembly, is activated. Understanding the role of cell assemblies as basic functional units in neural networks and socio-technical systems emerges as a fundamental challenge in network theory. A key open question is how these elementary building blocks emerge, and how they operate, linking structure and function in complex networks. Here we show that a network analogue of the Swift-Hohenberg continuum model—a minimal-ingredients model of nodal activation and interaction within a complex network—is able to produce a complex suite of localized patterns. Hence, the spontaneous formation of robust operational cell assemblies in complex networks can be explained as the result of self-organization, even in the absence of synaptic reinforcements.
Pattern formation in reaction-diffusion systems1,2 has emerged as a mathematical paradigm to understand the connection between pattern and process in natural and sociotechnical systems3. The basic mechanisms of pattern formation by local self-activation and lateral inhibition, or short-range positive feedback and long-range negative feedback4,5 are ubiquitous in ecological and biological spatial systems, from morphogenesis and developmental biology1,6 to adaptive strategies in living organisms7,8 and spatial heterogeneity in predator-prey systems9. Heterogeneity and patchiness associated with Turing patterns in vegetation dynamics have been proposed as a connection between pattern and process in ecosystems10,11, suggesting a link between spatial vegetation patterns and vulnerability to catastrophic shifts in water-stressed ecosystems12,13,14.
The theory of non-equilibrium self-organization and Turing patterns has been recently extended to network-organized natural and socio-technical systems15,16,17,18, including complex topological structures such as multiplex19,20, directed21 and cartesian product networks22. Self-organization is rapidly emerging as a central paradigm to understand neural computation23,24,25. The dynamics of neuron activation, and the emergence of collective processing and activation in the brain, are often conceptualized as dynamical processes in network theory26,27,28. Self-organized activation has been shown to emerge spontaneously from the heterogenous interaction among neurons25, and is often described as pattern formation in two-population networks29,30,31,32. Localization of neural activation patterns is a conceptually challenging feature in neuroscience. Cell assemblies, or small subsets of neurons that fire synchronously, are the functional unit of the cerebral cortex in the Hebbian theory of mental representation and learning33,34,35,36,37. Associative learning forms the basis of our current understanding of the structure and function of neural systems38,39,40. It is also the modeling paradigm for information-processing artificial neural networks41,42,43. The emergence of cell assemblies in complex neural networks is a fascinating example of pattern formation arising from the collective dynamics of interconnected units25,44. Understanding the mechanisms leading to pattern localization remains a long-standing problem in neuroscience34,44,45,46,47,48.
Here we show that simple mechanisms of nodal interaction in heterogeneous networks allow for the emergence of robust local activation patterns through self-organization. The simplicity and robustness of the proposed single-species pattern-forming mechanisms suggest that analogous dynamics may explain localized patterns of activity emerging in many network-organized natural and socio-technical systems. We demonstrate that robust local, quantized activation structures emerge in the dynamics of network-organized systems, even for relatively simple dynamics. We propose a minimal-ingredients, phenomenological model of nodal excitation and interaction within a network with heterogeneous connectivity. Our goal is to demonstrate that a simple combination of local excitation of individual units, combined with generic excitatory/inhibitory interactions between connected units, leads to self-organization, and can explain the spontaneous formation of cell assemblies without the need for synaptic plasticity or reinforcement. Our model can be understood as a network analogue of the Swift-Hohenberg continuum model49,50,51, and is able to produce a complex suite of localized patterns. The requirements are minimal and general: simple local dynamics based on canonical activation potentials, and interactions between nodes that induce short-range anti correlation and long-range correlation in activation. Because of their robustness and localization, self-organized structures may provide an encoding mechanism for information processing and computation in neural networks.
Model of Network Dynamics and Stability Analysis
We restrict our analysis to the simplified case of symmetric networks, but our main results can be generalized to other network topologies, including directed21 and multiplex19,20 networks. A node’s state of activation, measured through a potential-like variable u, is driven by local excitation dynamics and by the interaction with other nodes in the network via exchanges through the links connecting them. In dimensionless quantities, the proposed excitation-inhibition model for the evolution of potential, ui, in each node , is given by the modelwhere f (ui, μ) is a dynamic forcing term, representing a double well potential, and μ is a bifurcation parameter that will be used to establish the conditions for stability and localization of the response patterns (Fig. 1A). The currents, Ii, represent the excitatory/inhibitory interactions among nodes in the network. The structure of these nodal interactions is one of the key pattern forming mechanisms in the present model. We consider short-range anti-correlation, and higher-order, longer-range dissipative interactions. This two-level interaction structure, which induces anti correlation in the short range (nearest-neighbors, or first-order connectivity), and long-range correlation (second-nearest neighbors, or second-order connectivity) is represented in Fig. 1A. Mathematically, we express the integration of synaptic contributions as
The structure of the above nodal interactions turns the dynamics (1)–(2) into a network anologue of the Swift-Hohenberg continuum model,which is a paradigm for pattern-forming systems49,50,51,52,53. The simplest form for the interaction matrices representing these correlation/anti-correlation effects (while ensuring that the interaction fluxes conserve mass or charge) is based on network representation of Laplacian and bi-Laplacian operators, L(2) and L(4), respectively. The network Laplacian, L(2), is a real, symmetric and negative semi-definite N × N matrix, whose elements are given by 16, where Aij is the adjacency matrix of the network, is the degree (connectivity) of node i and δij is the Kronecker delta. A diffusive, Fickian-type flux of the activation potential u to node i is expressed as (see Fig. 1A–top). Plain waves and wavenumbers on a network topology are represented by the eigenvectors and the eigenvalues Λα and of the Laplacian matrix, which are determined by the equation , with 2. All eigenvalues are real and non-positive and the eigenvectors are orthonormalized as , where . The elements of the bi-Laplacian matrix of a network can be expressed as where the matrix has information about second order nodal connectivity and takes nonzero values if node i is two jumps away from node j. The operation models negative diffusion (inhibition) from the first neighbors of node i and at the same time diffusion from its two-jump neighborhood (see Fig. 1A–bottom). The bi-Laplacian, L(4), has the same eigenvectors as L(2) (i.e. ϕα) and its eigenvalues are the square of those of L(2), .
To understand the properties and pattern-forming mechanisms in our model, we first investigate the stability of flat states of the dynamical system (2):
Flat, stationary solutions of Eq. (4) satisfy f (ui, μ) = 0, where the nodal state of activation is equal for all nodes in the network, . For , there are three uniform solution branches given by u0 = 0 and . It is well known in one and two dimensional continuum spaces that these uniform states can become unstable and a wealth of self-organized patterns can arise49,50,51,52,53. In a linear stability analysis, the stability of flat stationary solutions to small perturbations is determined by the eigenvalues of the Laplacian and bi-Laplacian matrices. Introducing small perturbations, δui, to the uniform state , , the linearized version of Eq. (4) takes the form , where . After expanding the perturbation δui over the set of the Laplacian eigenvectors, , where qα is the expansion coefficient, the linearized equation is transformed into a set of N independent linear equations for the different normal modes:where Λα are the eigenvalues of the Laplacian matrix. The α-mode is unstable when Re λα is positive. Instability occurs when one of the modes (the critical mode) begins to grow. At the instability threshold, Re λα = 0 for some αc and Re λ < 0 for all other modes. In Fig. 1B,C we summarize the linear stability analysis of the flat states of our model on a scale-free network constructed using the Barabási-Albert model (BA) of network growth and preferential attachment54. We find that, indeed, there is a large parameter range for which the resting potential is stable. As we demonstrate below, in the stable regime, input stimuli may trigger localized patterns of activation.
Localized activation patterns are possible due to the particular structure of the model, with short- and long-range nodal interactions. Mathematically, the localized states are homoclinic orbits in the network space around the base resting state, . The existence of these homoclinic orbits can be studied using the technology developed for the linear stability analysis. Since homoclinic orbits leave the flat state as we approach a small neighborhood (cluster) of the network, the fixed point must have both stable and unstable eigenvalues. We linearize Eq. (4) around u = 0 and write , , arriving to the relation . Since the Laplacian eigenvalues Λα are real and non-positive values, we can write them in the form . If μ > 0 the topological eigenvalues of u = 0 form a complex quartet . For μ = 0 they collide pairwise on the imaginary axis, and for μ < 0 they split and remain on the imaginary axis . For μ = −1 two of the topological eigenvalues collide at the origin and for μ < −1 they move onto the real axis. These results are summarized in Fig. 2A. The topological eigenvalues in the neighborhood of μ = 0 are characteristics of the reversible 1 : 1 resonance bifurcation. Theory shows that under certain conditions the hyperbolic regime contains a large variety of topologically localized states51.
To understand the onset of localized patterns for different model parameters and input stimuli, we construct the bifurcation diagram of the resting state, as a function of the total potential energy of the stimulus and bifurcation parameter μ, in the vicinity of 51,52. A single bifurcation branch—constructed using a pseudo-arclength continuation method55—has a characteristic “snaking” structure of localized states with varying activation energy (Fig. 2B). As the system jumps from one steady state branch to the next one, a new neighborhood in the network is being activated. Figure 2C visualizes the different steady localized states of the six different branches as they are spotted in the diagram of Fig. 2B. The response of the system is quantized: the transition from one pattern of activation to another one is discontinuous as we vary the activation energy , or the parameter μ (Fig. 2B). These jumps in activation energy correspond to the addition of neighbor nodes to the cluster (Fig. 2C).
The discontinuous—quantized—nature of the network response leads to robustness in the local, final equilibrium patterns with respect to the input signal amplitude. To gain insight into the robustness of the localized patterns of activation, we performed a synthetic test in which we initially stimulate a specific neighborhood in the network, where we set (i.e. a step-like function signal in network topology) and let the system evolve to equilibrium without decay. We gradually increase the amplitude of the initial signal, and record the final energy values of the equilibrium, localized states. For small amplitudes the perturbation relaxes back to the resting state, and no activation pattern is elicited. There is a threshold in the energy of the input stimulus beyond which robust quantized states are formed. The states are robust in the sense that further increments in the input signal amplitude do not change the final equilibrium pattern (Fig. 3A).
The self-organized local structures are also robust with respect to random noise in the initial stimulus. We perform Monte Carlo simulations that probe the impact of the noise-to-signal ratio on the energy of the emerging quantized state. We have confirmed that the presence of small-amplitude noise has no effect on the equilibrium states of nodal activity. As can be expected, we do observe a departure from the energy of the base equilibrium state when the noise-to-signal ratio is sufficiently large, thereby masking the base stimulus altogether (Fig. 3B).
Mean-Field Approximation of the Global Activation Patterns
Our model predicts a range of parameter values where localized states disappear, and are replaced by global activation patterns. Mathematically, global patterns are possible when the non-active stationary solution is perturbed outside the parameter region of localized patterns (μ < 0). These—global—Turing patterns1,16 can be understood and modeled using the Mean-Field Approximation (MFA), a method that segregates nodes according to their degree and has been successfully used to approximate a wide variety of dynamical processes in heterogeneous networks, like epidemic spreading56,57,58, activator-inhibitor models16 and voter models59.
This theory allow us to reduce the problem to a single equation for the membrane potential for all the nodes in the system. Since in our model both the degree and two-jump degree play important role in the formation of patterns, we use a MFA where we assume that all the nodes with the same degree and two-jump degree behave in the same way. We start by writing Eq. (3) in the formwhere the local fields felt by each node, , and are introduced. These local fields are then approximated as , and , where is the degree and is the number of secondary connections of node i (two-jump degree). The global mean fields are defined by where and , where . Here, Nk is the number of nodes with degree k, is the number of nodes with k(2) number of two-jump neighbors and is the size of the network. In the above expressions, with we denote the sum over the nodes with degree k and with the sum over the nodes with two-jump nodal connectivity k2.
With this approximation, the individual model equation on each node interacts only with the global mean fields Hu and Huu and its dynamics is described by:
Since all nodes obey the same equation, we have dropped the index i and introduced the parameters α(i) = ki and . The activation potential depends now on the global fields Hu and Huu as well as on the parameter compination (α, β), i.e. . If the global mean fields Hu and Huu are given, the combination (α, β) plays the role of a bifurcation parameter that controls the dynamics of each node in the system. The time independent version of above mean field equation can be written as a third degree algebraic equation that we solve N times for the N nodes in the system. For each node i, we get three solutions , that can be stable or unstable depending on the sign (negative or positive respectively) of the operator .
After tuning the bifurcation parameter μ to a negative value, we can compute the global Turing pattern from direct numerical simulations and determined the global mean fields Hu and Huu. Each node i in the network is characterized by its degree and second nodal connectivity, so that it possesses a certain parameter combination, (α, β). Substituting these computed global mean fields as well as the values of α and β into equation (7), bifurcation diagrams of a single node can be obtained and projected onto the Turing pattern. In Fig. 4A we show for our “toy network model” that the stable brunches of the nodal bifurcation diagrams calculated using the MFA fit very well the computed Turing pattern. We further assess the dependence of the network topology on the global pattern formation and we find that when the degree distribution is narrower compared to a scale-free network, the distribution of the (α, β) is more homogeneous and therefore the stationary Turing patterns look smoother (Fig. 4B,C). Therefore, global network Turing patterns are essentially explained by the bifurcation diagrams of individual nodes coupled to the global mean fields, with the coupling strength determined by their degree and two-jump connectivity.
Our results suggest a new mechanism for the formation of localized nodal assemblies in networks, arising from long-range—second neighbor—interactions. Rather than relying on reinforcing mechanisms—synaptic plasticity, we show that localized, robust nodal assemblies are possible due to self-organization. The emergence of localized activation patterns derived from the simple and general functional structure of our proposed conceptual model: local dynamics based on activation potentials, and interactions between nodes that induce short-range anticorrelation and long-range correlation in node-to-node exchanges. The proposed system is a network analogue of the Swift-Hohenberg continuum model, and is able to produce a complex suite of robust, localized patterns. These self-organized, local structures can provide robust functional units to understand natural and technical network-organized processes.
How to cite this article: Nicolaides, C. et al. Self-organization of network dynamics into local quantized states. Sci. Rep. 6, 21360; doi: 10.1038/srep21360 (2016).
Funding for this work was provided by an MIT Vergottis Graduate Fellowship and a McDonnell Postdoctoral (RYC-2012-11704) Fellowship in Studying Complex Systems (to CN), the US Department of Energy through a DOE CAREER Award (grant DE-SC0003907) and a DOE Mathematical Multifaceted Integrated Capability Center (grant DE-SC0009286) (to RJ), and a Ramón y Cajal Fellowship from the Spanish Ministry of Economy and Competitiveness (to LCF).
This work is licensed under a Creative Commons Attribution 4.0 International License. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in the credit line; if the material is not included under the Creative Commons license, users will need to obtain permission from the license holder to reproduce the material. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/