The Use of Hebbian Cell Assemblies for Nonlinear Computation

When learning a complex task our nervous system self-organizes large groups of neurons into coherent dynamic activity patterns. During this, a network with multiple, simultaneously active, and computationally powerful cell assemblies is created. How such ordered structures are formed while preserving a rich diversity of neural dynamics needed for computation is still unknown. Here we show that the combination of synaptic plasticity with the slower process of synaptic scaling achieves (i) the formation of cell assemblies and (ii) enhances the diversity of neural dynamics facilitating the learning of complex calculations. Due to synaptic scaling the dynamics of different cell assemblies do not interfere with each other. As a consequence, this type of self-organization allows executing a difficult, six degrees of freedom, manipulation task with a robot where assemblies need to learn computing complex non-linear transforms and – for execution – must cooperate with each other without interference. This mechanism, thus, permits the self-organization of computationally powerful sub-structures in dynamic networks for behavior control.

Suppl. Figure 2: Basic setup of the motor task system. Details see Methods section. < . > t denotes the temporal average.

Constraints on cell assembly formation
The basis of the results shown here is the capability of the network to form competitive cell assemblies. Here we show mathematically that this is a non-trivial process which only happens when certain constraints are fulfilled. Many of the plasticity rules that have been discussed in the literature cannot easily achieve this, while the here-used combination of plasticity and scaling seem very well suited for this purpose.
We assume as usual that plasticity has only access to local information such as presynaptic activity F j , postsynaptic activity F i and synaptic weight W ij [1,2]. The selforganized formation of cell assemblies, which compete with each other, implies two major constraints: (i) synaptic weights inside an assembly have to be larger than outside and (ii) stimulation of one connection has to induce depression leading to competition at other non-stimulated presynaptic connections.
We can now discuss under which conditions these constraints can be fulfilled. (i) "Cohesion": A cell assembly is formed by a strong stimulus to a group of units. Thus, the links connecting these stimulated units have to be larger than others (outside the cell assembly). In other words, if two recurrently connected units i and j receive strong inputs, their resulting synaptic weights W s have to be larger than the ones (w w ) of non-or weakly stimulated units (W s > W w ; Suppl. Fig. 3 A). This constraint implies that the fixed point of the synaptic weight W * ij has to be:

A B
outside cell assembly inside cell assembly competition w W w S Suppl. Figure 3: Two constraints have to be fulfilled to enable the formation of competitive cell assemblies. (A) The first constraint is that dependent on the input (arrows) the connections outside the cell assembly (left) have to be smaller than inside (right; thickness of lines indicate strength). (B) The second constraint implies that a strong stimulus has to induce a decay of non-stimulated connection, so-called competition.
As Equation 1 looks similar to Hebbian plasticity, it is important to stress that this equation describes the fixed point of the weight dynamics and not the dynamics itself and is therefore not related to the Hebbian rule. Interestingly, all plasticity rules which assure stable weight dynamics by the ratio of pre-and postsynaptic activity, as, for instance, the Oja-rule [3], yield W s = W w and, therefore, do not allow cell assembly formation as defined above. Furthermore, all plasticity rules controlled by a homeostatic postsynaptic activity mechanism (as, for instance, the BCM-rule [4]), so that the fixed point postsynaptic activity F * i has to be equal a constant value Λ aṡ decrease the synaptic weights if units are stimulated. Thus, W s < W w , and the constraint is not fulfilled.
(ii) "Competition": To assure that cell assemblies do not 'smear' into each other while only one is active, the learning rule has to be competitive. This means that a (strong) stimulation at one connection of postsynaptic unit i induces a decrease of other presynaptic efficacies j (Suppl. Fig. 3 B). Therefore, the fixed point of the non-stimulated connection has to be adapted by the increased postsynaptic activity F i : Combining the constraints (i) and (ii): Ultimately the system needs to fullfil both constraints for which we then generically get: with parameters Γ, κ, η and α. The offset Γ assures that the activity-dependence of competition does not counterbalance the post-synaptic activity-dependence of the cohesion term. η determines the time scale of the cohesion term and κ of the competition term, thus, the ratio η κ determines which term dominates the dynamics. The power α determines the gradient of the resulting fixed point-activity function [2]. All learning rules with Equation 5 as fixed point of their weight dynamics are able to form cell assemblies as defined above, for instance, learning rules of following form: With τw η = τ H , τw κ = τ SS , Γ = F T , and α = 1/2 this equation is equal to the interaction of synaptic plasticity and scaling (Eq. 6 Methods section) used here LT P

Controlling assembly outgrowth speed
In the previous section we showed under which conditions activity-dependent synaptic adaptation yields cell assembly formation (Eq. 5). Among others, the interaction between synaptic plasticity and scaling fulfills these constraints. The question, however, remains how fast (slow) such processes ought to be to allow for a quick enough acquisition of computational power while still guaranteeing stability. Thus, apart from the above constraints also the temporal development of such systems is important, captured by the time constant ratio τ ratio , which determines the difference in adaptation speed between synaptic plasticity and scaling. Biological experiments [5] show that synaptic scaling takes about hours to days compared to the time scale of several minutes of synaptic plasticity. Thus, the ratio τ ratio is about 60 (used here) and more. How does this parameter effect the outgrowth of cell assemblies?
Consider a simplified version of the cell assembly outgrowth: two connected units with one unit ca as part of the cell assembly and the other os outside. The outside unit becomes member of the assembly if the connection from the assembly unit to the outside unit becomes strong. Thus, the growth process of the synaptic efficacy determines the outgrowth speed of the cell assembly. Note, cell assemblies only grow out when they are active (F ca ≈ F max ), otherwise, the activities are so low that synaptic changes can be neglected. Thus,Ẇ We rewrite this term dependent on the ratio of time scales τ ratio = τ SS τ H τ HẆos,ca = F os · F max + 1 τ ratio F T − F ca · W 2 os,ca .
The fixed point of the synaptic weight W * = W max , which has to be reached, depends also on the ratio τ ratio . Thus, we have to estimate the outgrowth relative to the fixed point: During synaptic development the synaptic weight passes two extreme cases: (i) a small initial value (W os,ca W max ) and (ii) converging to the fixed point (W os,ca ≈ W max ). In each case we can linearize the activity function to F os ≈ c F max W os,ca with constant c x different for both cases x ∈ {i, ii}.
(i): For small weights we can neglect the higher weight-orders in Equation 11.
Thus, the initial outgrowth depends on the square root of the time scale ratio τ ratio . In other words, if synaptic scaling is too fast compared to plasticity, it dampens the synaptic development and, in turn, the cell assemblies cannot grow out.
(ii): For weights comparable to the maximum weight the higher weight-orders in Equation 11 dominate: So, the weight converges to the maximum depending on 1/τ ratio . Thus, if synaptic scaling is too slow, the system needs long until it will reach its fixed point. Suppl. Figure 4: The ratio of time scales τ ratio influences the outgrowth speed of cell assemblies. Plotted is the duration the system needs to reach the fixed point (W * ) when the cell assembly is all the time active.
Given case (i) and (ii) the time scale ratio τ ratio should neither be too small (case i) nor too large (case ii). Thus, τ ratio has to be in an intermediate regime. In Suppl. Figure 4 we have measured the time the synaptic weight needs to reach the fixed point (blue) given different values of τ ratio . Clearly their is an intermediate regime of τ ratio (including the here used value) with an optimal outgrowth rate.

Cell assembly growth
Given an external stimulation (purple in Suppl. Fig. 5 A) to a subset of randomly connected units, the stimulation induces units activities in large parts of the network. As expected, the activity of the directly stimulated units (red) is higher than the activity of the non-stimulated ones (blue). However, with ongoing learning the non-stimulated units become more active, too. This is due to an increase of the internal excitatory feedback from the network (Suppl. Fig. 5 B) by increasing synaptic weights (cell assembly outgrowth). Interestingly, the stimulated units receive a stronger feedback from the network than the non-stimulated units. This is due to the specificity of the synaptic changes leading to the formation of cell assemblies. For instance, Suppl. Fig. 6 shows that the first strong connections are formed between the stimulated units (red lines) before connections from the stimulated to non-stimulated (purple) are made and finally also between nonstimulated units (blue). Furthermore, the distribution of the weights develop from an x-axis y-axis x-axis x-axis x-axis t=5000 t=10000 t=55000 t=105000 stimulated non-stim. stim-stim stim-non-stim non-stim-non-stim connections: neurons: Suppl. Figure 6: Development of a cell assembly. Units are distributed randomly. Lines show connections with weights larger than W max /2.
The dynamics of the outgrowth of a cell assembly is independent of the exact value of the cell-assembly-threshold θ (Suppl. Fig. 7 A). A detailed analysis of the resulting network also shows that a cluster of strongly interconnected units emerges during learning. For this we calculate the weighted-shortest path between each pair of units given the excitatory weight matrix [7]. If two units are strongly connected with each other, the path is short and, therefore, this measure is small. Given the matrix of shortest paths, for visualization, units are now sorted into two groups: those inside the cell assembly (estimated from the threshold-analysis) and the rest. As expected, with ongoing learning, the units, which are part of the cell assembly, also formed a cluster of low-valued weighted-shortest paths (red box in Suppl. Fig. 7 B).
Remarkably, the dynamics of the units activities within the cell assembly change with ongoing learning (Suppl. Fig. 7 C). Given a pulsed input to the previously stimulated units, the cell assembly units initially respond in a highly correlated manner. However, the temporal structure of the units activities within a response becomes more complex with growing assembly size. Such complex units activations can then indeed serve as a basis for computation. The assembly responses get also longer and they exceed the input duration, but eventually die down to baseline. This decay indicates that the system is in a subcritical regime [8,9](comparable to the AI-state [10]) and does not show chaotic [11] or persistent [12] dynamics.    Suppl. Figure 8: Strong connections contribute most to the computational capacity of the cell assembly. By deleting all connections below a threshold θ, we test the contribution of the remaining, strong connections to the nonlinear calculations (here we used nonlinear task 1). The error increases dramatically if connections with weights larger than 0.7 · W max are deleted (core of the cell assembly). Please note the logarithmic error-axis.
The outgrowth of the cell assembly clearly supports the performance of the network to compute and solve non-linear tasks. However, it is not clear which part of the cell assembly contributes most to the computation. Therefore, for the final network in Figure 1 main text, we deleted all connections below a threshold θ and tested the computational power of the resulting network. We repeated this procedure for several different thresholds (Suppl. Figure 8). As expected, the strong connections (W > 0.7 · W max ) contribute most to the computational power of the system as the error increases dramatically by deleting them. Thus, the core of the cell assembly with strongly interconnected units is the most important part. The halo (with smaller synaptic weights) contributes only a little bit to the performance of the system. Suppl. Figure 9: Comparison between the adaptive network and random static networks for different weight thresholds θ. Panels are as in main text Figure 2 A with thresholds θ = 0.25 · W max (left) and θ = 0.75 · W max (right). Independent of the threshold the number of connections needed in the adaptive network is significantly smaller than for random static ones. Here only one topology is shown.