Abstract
Spiketimingdependent plasticity (STDP) adjusts synaptic strengths according to the precise timing of pre and postsynaptic spike pairs. Theoretical and computational studies have revealed that STDP may contribute to the emergence of a variety of structural and dynamical states in plastic neuronal populations. In this manuscript, we show that by incorporating dendritic and axonal propagation delays in recurrent networks of oscillatory neurons, the asymptotic connectivity displays multistability, where different structures emerge depending on the initial distribution of the synaptic strengths. In particular, we show that the standard deviation of the initial distribution of synaptic weights, besides its mean, determines the main properties of the emergent structural connectivity such as the mean final synaptic weight, the number of twoneuron loops and the symmetry of the final structure. We also show that the firing rates of the neurons affect the evolution of the network, and a more symmetric configuration of the synapses emerges at higher firing rates. We justify the network results based on a twoneuron framework and show how the results translate to large recurrent networks.
Introduction
Spiketimingdependent plasticity (STDP)^{1,2,3,4,5} modifies synaptic strengths according to the relative timing of pre and postsynaptic spike pairs. When the postsynaptic spike follows the presynaptic spike, a potentiation of the corresponding synaptic strength is induced by the STDP mechanism, whereas the synaptic strength is depressed in the opposite case^{2}. STDP is a local learning rule, where the synaptic modification only depends on the joint prepost activity of the corresponding synapse. However, STDP may significantly impact on the global dynamics of neuronal networks, so that qualitatively different connectivity patterns can emerge under the influence of STDP^{6,7,8,9,10,11,12,13,14,15,16,17}. It is commonly accepted that the temporally asymmetric learning window of the classic pairbased STDP eliminates strong bidirectional loops between neuronal connections^{18,19,20,21,22,23}. Although this interesting feature of STDP can explain the emergence of feedforward networks^{4,19,24,25}, it is in contradiction to the prevalence of recurrent connections in cortical circuits^{26,27,28,29,30}.
However, it is possible to stabilize strong bidirectional connections by employing variations of STDP^{22} or by considering independent noise^{31} in the plastic neuronal networks. On the other hand, as shown recently, by taking into account dendritic and axonal propagation delays, the conventional pairbased additive STDP may lead to the emergence of different connectivity patterns including both unidirectional and bidirectional connections, or decoupled neurons^{32}.
An asymmetric learning window of STDP profile can push the network dynamics toward a synchronized state with strongly upregulated synaptic connections, or conversely, the network can evolve into a desynchronized regime with downregulated synaptic strengths^{33,34,35,36,37}. It was shown that the initial mean synaptic coupling determines the finally evolving mean strength of the synaptic connections. Strong initial coupling leads to synchronized dynamics that can be enhanced and preserved by STDP, while weak initial connections lead to desynchronized states, i.e., there is a multistability of synchronized and desynchronized states in the plastic neuronal networks^{33,34,38}. Hence, STDPdriven neuronal populations can induce different multistable attractor states characterized by specific coupling regimes^{33,34,38}.
Dendritic and axonal propagation delays within and between brain areas may assume different values and need not be identical. Dendritic delays are typically smaller than axonal delays^{20}. However, dendritic delays may range from submillisecond to a few milliseconds^{39,40}, e.g. AgmonSnir and Segev^{39} demonstrated that the dendritic delay in octopus cells is about 0.5 ms. Axonal propagation delays may range from 0.3 ms in thalamocortical connections^{41} and 1 ms in corticotectal connections^{42} to 20 ms in corticocortical connections^{43}, and even more, up to 40 ms in corticothalamic circuits^{42}.
The presence of dendritic and axonal propagation delays can regulate the emergent structures of STDPdriven plastic neuronal populations^{20,32}. Propagation delays affect the function of STDP in two different ways. First, the total propagation delay τ = τ_{d} + τ_{a}, i.e., the sum of the dendritic τ_{d} and axonal τ_{a} propagation delays, determine the synchronization tendency of the coupled neurons and the time difference of spiking of the neurons^{44,45,46,47,48,49,50}. Second, since the effect of pre and postsynaptic neurons arrive at the synapse after axonal and dendritic delays, respectively, the difference of the two propagation delays ξ = τ_{d} − τ_{a}, changes the relative timing at the synapse. We recently showed^{32} that by incorporating propagation delays in a plastic neuronal network model, bidirectional connections can be retained and potentiated despite the previously reported simulation results with conventional STDP in the absence of delays^{33,34,38}.
In this study, we focus on the multistability of the structural dynamics of the network with respect to the distribution of the initial synaptic weights in coupled networks with delays. Multistability was previously reported with respect to the mean initial synaptic weight: The final mean synaptic strength mainly depends on the initial average weight^{34,38}. Here we show that within the present framework not only the mean, but also the standard deviation of the initial connections crucially determines the final mean strength of the synapses. Moreover, the presence of bidirectional connections and the symmetry of the final structure is determined by the initial discrepancy of the synaptic weights. We show that the multistability of the network can be explained in a modular way, by the presence of different attractors for a twoneuron motif in the twodimensional space of the initial connection strengths. We also show that the domain of attraction of each final structural state depends on the firing frequency of the neurons: At high firing rates the attraction domain for symmetric connections grows in expense of shrinking that of the unidirectional connections. This leads to a more symmetric final structure when the neurons are firing at higher rates. Intriguingly, all of these nontrivial phenomena are only seen when the propagation delays are incorporated in the formulation of the model: Without propagation delays, any initial preparation ends up with unidirectional connections regardless of the firing rate of the neurons and the initial weights of the synapses.
Results
Theoretical Framework
We consider two neuronal oscillators described in a phase reduced model. In this approach, the evolution equation for the relative phase of the two neuronal oscillators can be written as follows (for the derivation see Methods, Eq. (6)):
where χ = φ_{2} − φ_{1} is the phase lag, \({\rm{\Omega }}\,=\,2\pi {\rm{\Delta }}\nu \) that Δν = ν_{2} − ν_{1} is the frequency mismatch, and ψ = 2πντ is the rescaled delay with τ = τ_{d} + τ_{a} being the sum of dendritic τ_{d} and axonal τ_{a} propagation delays. The values for dendritic and axonal propagation delays are chosen based on the experimental observations in cortical areas^{42,43}. The quantity Γ reflects the relative difference of the synaptic strengths in two opposite directions, where g_{ij} with i, j = 1, 2 (j ≠ i) is the synaptic strength of the synapse connecting presynaptic neuron j to postsynaptic neuron i.
The synapses connecting the two neurons are subjected to classic pairbased STDP rule where the synaptic strengths g_{ij} = g_{ij}(t) are modified based on the delayed time lag between pre and postsynaptic activity at synaptic site, i.e., \({\rm{\Delta }}{t^{\prime} }_{ij}={\rm{\Delta }}{t}_{ij}+\xi \)^{32} where Δt_{ij} = t_{i} − t_{j} is the time lag in the absence of propagation delays and ξ = τ_{d} − τ_{a} is the difference between dendritic and axonal propagation delays. The synaptic strengths are updated by an additive rule at each step g_{ij} → g_{ij} + Δg_{ij}, according to the following learning function (see Methods):
where A_{+}(A_{−}) and τ_{+}(τ_{−}) are the learning rate and the effective time window of synaptic potentiation (depression), respectively, and sgn(Δt) is the sign function. The synaptic strengths are bounded in the range (g_{min}, g_{max}) = [0.05, 1] by the hard bound saturation constraint^{4,51}: The synaptic strengths are set to g_{min} (g_{max}) once they cross the lower (upper) limit of their allowed range. We consider a balanced STDP profile^{4,19} with equal potentiation and depression contribution, i.e. A_{+} = A_{−} and τ_{+} = τ_{−}, in order to extract the pure influence of propagation delay and oscillation frequency on the synaptic dynamics. We assumed that the changes in synaptic strengths are slow in comparison to the fast timescale of the system, therefore, the neurons remain phaselocked despite of eventual change in the synaptic strengths. Given the firing frequency ν and assuming that the frequency mismatch between the two oscillators is negligible, i.e. \({\rm{\Omega }}\simeq 0\), the fast timescale equation of the twoneuron motif (Eq. (1)) exhibits a stable fixed point which is given by Eq. (9), \({\chi }^{\ast }={\rm{t}}{\rm{a}}{{\rm{n}}}^{1}({\rm{\Gamma }}\,{\rm{\tan }}\,\psi )\), and hence firing time difference of two neurons at the stable state will be Δt^{*} = χ^{*}/(2πν). These values could be inserted in Eq. (3) to give the evolution of the synaptic connections^{32,52}.
Bistability in the TwoNeuron Motif
In Fig. 1 we show the synaptic dynamics (illustrated by the vector field) overlaid on the the representation of the instantaneous time lag of the spiking neurons (shown by colors), for the different values of the initial synaptic strengths (g_{21}(0), g_{12}(0)). It can be seen that the basin of attraction for the three different final connectivity patterns (bidirectional, unidirectional and decoupled) is determined by the firing frequency ν and the difference between dendritic and axonal delays ξ. The results shown in Fig. 1 denote the theoretical prediction of different stable structural states that coexist for different combinations of delay and firing frequency. The colors indicate the stable fixed point of the time lag Δt^{*} given by Eq. (9), and the vector field shows the changes of the synaptic strengths from Eq. (3). Given the initial values of the synaptic strengths, the instantaneous colorcoded fixed point of the time lag determines the synaptic changes depicted by the vector field, and the subsequent values of the synaptic strengths. The corresponding trajectories of the time evolution of the synaptic strengths resulted from numerical experiments with three different initial values are shown by yellow solid curves in Fig. 1G and I. The numerical solutions practically follow the vector field directions predicted by the analytical results. The time course of the simulated synaptic strengths and the firing time lag are presented in Fig. 2.
Figure 1, left and right columns illustrate that according to different initial arrangements of the connections, the twoneuron motif will end up with one of the bistable coupling regimes: Bidirectional/unidirectional coupling (Fig. 1, left column, dendritic delay is greater), or decoupled/unidirectional coupling (Fig. 1, right column, axonal delay is greater). The basin of attraction of the symmetric coupling regimes (i.e. bidirectional or decoupled final configuration) grows with an increase of the firing frequency (Fig. 1, left and right columns, top to bottom), making it more probable that in the highfrequency firing regime the neurons are either bidirectionally coupled (Fig. 1G) or remain decoupled (Fig. 1I).
We also examined the effect of a frequency mismatch Δν = ν_{2} − ν_{1}, i.e. the difference between the frequencies of the two oscillators introduced in Eq. (1), on the evolution of the synapses. Expectedly, the frequency mismatch shifts the border of the basins of attraction in a way that the neuron with the greater firing rate more likely wins the competition in generating a stronger outgoing synapse and suppressing the reverse one (see Fig. 3), which is in accordance with previous studies^{21,53}. However, for small frequency mismatches (e.g. Δν = 0.02 Hz in Fig. 3A–C) bistability can still be present, and all three final configurations may be achieved depending on other parameters, i.e., the difference in propagation delays and firing frequencies.
The relation between dendritic and axonal delays is crucial to the emergence of bistability. When dendritic and axonal delays are equal, ξ = 0, which is equivalent to ignoring both in the STDP implementation, no bistability emerges and the final connectivity pattern is unidirectional, regardless of the initial synaptic strengths (see Figs 1 and 3, middle columns). This is consistent with the results of Babadi and Abbott^{23}, obtained for a balanced STDP profile. Note, in our model the bistability and the possibility of the emergence of symmetric connectivity patterns are observed for symmetric STDP just because the propagation delays are explicitly considered (also see Madadi Asl et al.^{32}).
The dependence of the final stable coupling regimes on the initial values of the synaptic strengths can be explained based on Eq. (9), \({\rm{\Delta }}{t}^{\ast }={\tan }^{1}({\rm{\Gamma }}\,\tan \,\psi )\mathrm{/(2}\pi \nu )\), that gives the asymptotic time lag between pre and postsynaptic spikes. Therefore, the instantaneous fixed point of the time lag (Eq. (9)) is determined by the relative synaptic strength Γ, the firing frequency of the oscillation ν, and the total delay τ. In turn, the instantaneous value of Δt^{*} determines the evolution of the synaptic strengths according to Eq. (3). More specifically, the values of the synaptic strengths determine the relative synaptic strength Γ = (g_{12} − g_{21})/(g_{12} + g_{21}) from Eq. (1), where the interaction of the relative synaptic strength with the fixed point of the time lag \({\rm{\Delta }}{t}^{\ast }\) ultimately determines the final stable coupling regime. The right panels of Fig. 4, show that the colorcoded relative synaptic strength Γ is close to zero around the diagonal line (light color) and maximal at the corners (dark color). As shown in Fig. 4 (left panels) the basin of attraction of the twoneuron motif changes when \({\rm{\Delta }}{t}^{\ast }\) (blue curve) crosses a certain value of Γ (denoted by arrow). The bistability is between bidirectional (red) and unidirectional (orange) connections (Fig. 4A, left panel), or between decoupled (blue) and unidirectional (orange) states (Fig. 4B, left panel).
Recurrent Excitatory Networks
In the twoneuron motif the relative synaptic strength \(\Gamma \), given by Eq. (1), determines the final configuration of the connections (see Fig. 4). Small Γ favors more symmetric final structure, and bidirectional connections can emerge if the axonal delays are small. Intriguingly, in large networks the initial distribution of the connection strengths is the key factor determining the emergence of bidirectional connections: In an initially homogeneous network with narrow distribution of the synaptic strengths, bidirectional connections (i.e. twoneuron loops) are more likely to survive in the network, while heterogeneity in the initial synaptic strengths drives the network to an asymmetric final structure with a small number of bidirectional connections. In contrast, so far, the evolution of the network and the emergent structure was attributed only to the mean of the initial synaptic strengths^{33,34,38}. In order to study the generalization of our framework to the network level, we took a fixed value of the mean initial synaptic strength and varied the width of the distribution of the initial synaptic strengths by changing the standard deviation σ_{g} of the distribution. We constructed an initially fully connected network of N = 200 excitatory neurons. Synaptic strengths are modified based on the pairbased additive STDP rule given by Eq. (2). Based on the analysis for the case of a twoneuron motif, the dynamics of the network for a given relative delay (difference of axonal and dendritic delays) and firing frequency can be predicted by the initial distribution of the synaptic strengths. Figure 5 shows the results for the recurrent excitatory network with relative delay ξ = 0.2 ms, as in the case of shortrange synaptic projections, in the highfrequency firing regime. The dendritic delay τ_{d} = 0.5 ms is greater than the axonal τ_{a} = 0.3 ms. According to Fig. 1G, our analysis of the twoneuron motif predicts a multistability regarding the number of bidirectional and unidirectional connections in the emergent network. Figure 5A, left panel shows the time evolution of the mean synaptic strength, and right panel illustrates the initial and final distribution of the synaptic strengths where the network architecture is dominated by either bidirectional loops (blue distribution) or unidirectional connections (grey distribution). Figure 5B, left panel shows the synchronization degree of the neuronal oscillators and right panel represents the number of twoneuron loops. Figure 5C shows the final coupling matrices for pre and postsynaptic neurons. For smaller values of the standard deviation (e.g. σ_{g} = 0.05, 0.08 in Fig. 5C), based on our twoneuron motif analysis, the network is driven to potentiate bidirectional connections. However, for greater values of the standard deviation (e.g. σ_{g} = 0.10, 0.15 in Fig. 5C), in the emergent structure unidirectional connections have more chance to survive.
For large axonal delays, as in the case of longrange synaptic projections, a similar bistability is observed between unidirectional connections and a disconnected network. Figure 6 shows the results for ξ = −0.5 ms. The axonal delay τ_{a} = 1.0 ms is greater than the dendritic τ_{d} = 0.5 ms. Figure 6A, left panel shows the evolution of the mean synaptic strength in the network, and right panel represents the initial and final distribution of the synaptic strengths where the network structure tends to be decoupled (blue distribution) or dominated by unidirectional connections (grey distribution). Figure 6B, left panel shows the synchronization degree of the neuronal oscillators and right panel represents the number of twoneuron loops. Figure 6C shows the final coupling matrices for pre and postsynaptic neurons. In the case with small standard deviation of the synaptic strengths (e.g. σ_{g} = 0.05, 0.08 in Fig. 6C), most of the connections tend to be depressed. In contrast, for wider distributions (e.g. \({\sigma }_{g}=\mathrm{0.10,}\,0.15\) in Fig. 6C), the synapses get unidirectionally potentiated. Note that the stability of the disconnected network in our simulations relies on our choice of the lower hard bound of the synaptic strength, g_{min}. Assigning a finite value for g_{min} stabilizes the network structure at a point where all connections attain the minimum value. Setting the minimum synaptic weight at zero, destabilizes the disconnected network, and one of the synapses builds up to achieve a unidirectional link. Our choice seems reasonable based on the experimental results on STDP^{3,5}. The emergence of a stable decoupled state in the phaselocked mode can, e.g., be of importance for the development of stimulation methods for neurological disorders designed to induce longlasting, sustained therapeutic effects by shifting a neuronal network from phaselocked attractors to more favorable, decoupled attractors^{38,54,55}.
By decreasing the firing frequency and increasing the disparity of the neuronal firing rate the symmetric (bidirectional or decoupled) connections are less likely to survive as revealed by the twoneuron motif analysis (Figs 1 and 3). We extracted the probability (P_{asym}) of appearance of unidirectional connections in the twoneuron motif, by taking the ratio of the area in the (g_{21}, g_{12}) space which leads to unidirectional connections to the total area of the same parameter space. Figure 7A shows that this probability is a decreasing function of frequency. To investigate the impact of this result on the evolution of the network, we defined the network asymmetry index C_{net} which quantifies the average asymmetry of the connections between pairs of neurons (see Methods). Expectedly, the network asymmetry index shows a similar decreasing trend which indicates that the number of unidirectional connections in the network decreases with increasing frequency (see Fig. 7A). As shown in Fig. 7B, by increasing the frequency of the oscillations, network asymmetry index decreases, which is consistent with a similar decrease in P_{asym} in the twoneuron motif, indicating that at higher frequencies the probability of the appearance of symmetric connections (twoneuron loops or decoupled pairs) increases. Also, based on the results of Fig. 3, a sufficiently pronounced inhomogeneity in the firing rate of the neurons can prevent the formation of strong bidirectional connections. Figures 8 and 9 show the network results in the presence of heterogeneous firing frequencies. Left panels in Figs 8A and 9A represent the mean synaptic strength and right panels illustrate the distribution of the firing frequencies and the final synaptic strengths based on the inhomogeneity in the distribution of the firing rates. Left panels in Figs 8B and 9B show the order parameter of the oscillators, whereas right panels represent the number of twoneuron loops in the network which is suppressed by increasing the inhomogeneity in the distribution of the firing rates. Figures 8C and 9C show the final coupling matrices for pre and postsynaptic neurons. As shown in Figs 8 and 9, by increasing the standard deviation σ_{ν} of the distribution of the firing rates, the number of loops in the final structure decreases. This is also illustrated in Fig. 7C where the asymmetry index C_{net} increases with increasing σ_{ν}. The probability P_{asym} in the twoneuron motif also follows the same ascending trend. Figure 7D shows that by increasing the inhomogeneity in the initial distribution of the synaptic strengths with standard deviation σ_{g}, unidirectional connections with high values of the asymmetry index are more likely to emerge. This increasing trend was also generally predicted by the probability P_{asym} in the twoneuron motif based on our theoretical framework.
Discussion
STDP is conventionally known as a mechanism which generically breaks the structural symmetry of neuronal networks and underlies the formation of feedforward networks^{4,19,24,25,56}. This result in recurrent networks contradicts the experimental observations in cortical circuits where bidirectional connections are frequent^{26,27,28,29,30}. Bidirectional connections can be retained through alternative versions of conventional STDP: Tripletbased STDP^{57} and STDP with a shifted profile^{23}. Tripletbased STDP, which has been designed to conform the dependence of the experimentally observed synaptic changes to the firing rates^{58}, can promote bidirectional connections in the highfrequency regime^{57}. Pairbased STDP with a rightward shifted profile can also lead to bidirectional connections^{23}, if potentiation dominates depression. The imbalance of the STDP profile in favor of potentiation in short spikepairing times, may result in the retention of bidirectional connections when the neurons are loosely phaselocked^{23}. In a more realistic modeling, considering forward and backward propagation delays, we have previously shown^{32} that the widely accepted consequences of conventional STDP, mentioned above, can be challenged. Here we showed that in this framework, the evolution of the bidirectional connections and the symmetry of the emerging structure shows multistability and the degree of heterogeneity in the initial setting of the network determines the final structure of the network.
Experimentally observed pairbased STDP parameters have been shown to be unbalanced: A_{+} > A_{−} and τ_{+} < τ_{−}^{2,3,5}. However, a balanced profile of STDP, i.e. \({A}_{+}\,=\,{A}_{}\) and τ_{+} = τ_{−}, has widely been used in computational and theoretical studies^{4,19,20,21,23}, since it can provide a suitable substrate for extracting the pure effect of other parameters, e.g. propagation delays, on the dynamics of the system. For example, as noted above, in a potentiation dominated profile of STDP, bidirectional connections can emerge without incorporation of propagation delays in the model^{23,31,35}. In the present study we chose a balanced STDP profile to highlight the effects which can be directly attributed to propagation delays.
Previous studies ascribed the final stable connection regime in neuronal networks with synaptic plasticity without propagation delays to the initial mean coupling^{33,34}. Here, based on the analysis presented for the twoneuron motif, we focused on the multistability of the network structure due to the disparity of the initial synaptic strengths. Our results show that for every pair of neurons, the final configuration of the synaptic connections follows a bistable dynamics governed by the initial mismatch of the strength of the synaptic connections in both directions, where the initial asymmetry in the structure (difference in the synaptic strengths) specifies the final synaptic configuration. The emerging configurations depend on the difference between dendritic and axonal delays: For larger dendritic delays, the final structures are unidirectional and bidirectional, while for larger axonal delays the two achievable states are unidirectional or fully decoupled neurons. In particular, our results highlight the importance of propagation delays, which are usually neglected in modeling studies of STDP: Without taking into account delays, the bistability cannot emerge, and all initial settings end up with a unidirectional configuration.
Our findings demonstrate the crucial role of the presence and the range of dendritic and axonal propagation delays in regulating the emergent structures of synaptic connections in plastic excitatory neuronal networks. We show that connections with experimentally measured values of shortrange delays^{41,42} and gamma band firing rates^{58} might be favorable for strong bidirectional loops or unidirectional connections, depending on the initial distribution of the synaptic strengths (see Fig. 5). In contrast, in the case of longrange connections, a loosely connected network or unidirectional synaptic structures might emerge (see Fig. 6). Therefore, the difference of dendritic and axonal propagation delays is crucial for the selection of the network’s ultimately emerging coupling regimes. In this way, qualitatively distinct connectivity patterns may emerge.
The emergence of multistable coupling regimes can be of interest since such systems can be used in order to construct memory building blocks. Bistability of the synaptic strengths, arising from the positive feedback nature of STDP, results in the emergence of different stable attractor states of the synaptic connections^{59}. Switching between different stable attractors enables neuronal networks to maintain and retrieve memories. Bistability of the network dynamics with recurrent connections mediated by STDP has been addressed for stationary^{13} or oscillatory inputs^{60}. Bistable dynamics can be characterized by the presence of two extreme stable fixed points of the synaptic strengths at g_{min} and g_{max}, imposed by the hard bound saturation constraint.The choice of the saturation constraints can have significant effects on the final distribution of the synaptic strengths; however, it does not crucially affect the weight dynamics^{61}. Therefore, the main results presented in this study, i.e., development of different connectivity patterns can also be qualitatively obtained when soft bounds are imposed on the synaptic strengths.
The dependence of the emergent structure to the firing rate of the neurons was experimentally observed^{58}, showing that pairing of spikes with both positive and negative time differences leads to potentiation in the high firing rate regime, which implicitly indicates that bidirectional connections can be promoted in this regime. While previous attempts with pairbased STDP failed to reproduce these results, our study shows that this can be achieved by considering propagation delays without having to consider triplets of spikes in the formulation^{32}. Here, we showed that the domain of attraction for each achievable final configuration of the synapses depends on the firing frequency of the neurons. For low firing frequency, the two synapses in the pairs of bidirectional connections evolve in different directions and unidirectional connections are more likely to emerge and survive in the network, as suggested by several studies on STDP^{18,19,20,21,22,23}. However, by increasing the firing frequency, the simultaneous potentiation or depression of the bidirectional connections becomes more probable, so that the measure of the attraction domain for a symmetric configuration in the two dimensional space of the initial synaptic strengths expands. Accordingly, in recurrent neuronal networks the width of the distribution of the synaptic strengths and the firing frequencies determine the emergent structure of the network and, in particular, the number of twoneuron loops in the final structure.
Furthermore, our results show that the symmetric depression of reciprocal connections between neurons in the phaselocked state is possible when favorable combinations of initial synaptic strengths, range of firing rates, and propagation delays are met. We showed that by considering a nonzero lower hard bound g_{min}, the depression of reciprocal connections can be stabilized in order to construct a loosely connected network, as predicted by our theoretical framework. The possibility of a symmetric stable depression of reciprocal connections in the phaselocked mode may contribute to a further development of brain stimulation techniques that induce an antikindling, i.e., an unlearning of abnormally upregulated synaptic connectivity and, in turn, abnormal synchrony^{38}. Coordinated reset (CR) stimulation^{62}, a desynchronizing multisite stimulation technique was successfully tested in preclinical^{63,64} and clinical^{65,66,67} proof of concept studies. However, our approach presented here, may lead to further improvements of brain stimulation techniques causing sustained therapeutic effects.
Methods
PairBased STDP Model
The neuronal oscillators are subjected to classic pairbased STDP rule where the synaptic strengths are modified based on the learning window function introduced in Eq. (2). The evolution of the synaptic strengths by the pairbased STDP rule can be calculated by taking the average of Eq. (2) over a period of the spiking neuron and smoothing, \({\dot{g}}_{ij}\simeq {\rm{\Delta }}{g}_{ij}/T\) (see Supplementary Fig. S1; for details on the interplay between the temporal scale of the parameters with STDP models see^{68}), which is given by:
where A_{+}(A_{−}) and τ_{+}(τ_{−}) are the rate and the effective time window of synaptic potentiation (depression), respectively. T is the period of spiking and sgn(Δt) is the sign function. In the entire manuscript the profile of STDP is balanced by setting the parameters to A_{+} = A_{−} = 0.005, and τ_{+} = τ_{−} = 20 ms. Synaptic strengths are confined in the interval (g_{min}, g_{max}) = [0.05,1]. It should be noted that hard bounds are imposed on the allowed range of synaptic strengths: The synaptic strengths are set to g_{min} (g_{max}) once they cross the lower (upper) limit of their allowed range.
Phase Reduced Model
Considering that the rate of synaptic change is small, and therefore, the changes in the synaptic strengths are negligible on the fast timescale of the system, the reduced averaged phase model for weakly pulsecoupled neuronal oscillators characterized by intrinsic frequency ω_{i} = 2πν_{i} and infinitesimal phase sensitivity Z(φ) can be written as follows^{69,70}:
where the neuronal oscillators are coupled via delayed connections of strength g_{ij} with total delay τ = τ_{d} + τ_{a}. φ_{i} is the phase of the ith oscillator, ψ = ω_{i}τ is the rescaled delay, and N is the number of oscillators. The neurons fire every time their phase passes multiples of 2π. In the model, we ignore the synaptic processing time, but the results are not affected by this assumption. Eq. (4) can be written for two coupled typeII phase oscillators with analytical phase response curve (PRC) Z(φ) = −sin(φ):
where subtracting these two relations gives the evolution equation for the relative phase of the two neurons:
where χ = φ_{2} − φ_{1} is the phase lag, and Ω = ω_{2} − ω_{1}. Using the trigonometric identity sin(x ± y) = sinx cosy ± cosx siny, Eq. (6) can be rearranged to Eq. (1).
Dynamical Analysis of the Joint Phase Model
In general, the fixed point of the phase lag \({\chi }_{i}^{\ast }\) of the Eq. (6) for typeI neuronal oscillator with analytical PRC function Z(ψ ± χ) = 1 − cos(ψ ± χ), is given by:
where \({\chi }_{1}^{\ast }\) is the inphase firing solution and \({\chi }_{2}^{\ast }\) belongs to antiphase state. Given the synaptic strengths, only one of these fixed points are stable in a given Ω and delay time ψ. The fixed points of typeII neuronal oscillator with analytical PRC function \(Z(\psi \pm \chi )=\,\sin (\psi \pm \chi )\) can be derived similarly:
Eqs (7) and (8) show that the fixed points of both typeI and typeII neuronal oscillations are selfconsistent in the presence of intrinsic frequency mismatch Ω between the two oscillators. In this case, the stable fixed point \({\chi }_{i}^{\ast }\) is simply where the two \({y}_{1}={\chi }_{i}^{\ast }\) and \({y}_{2}={\tan }^{1}(f(\psi ,{\chi }_{i}^{\ast }))\) curves intersect. The other approach is to solve the equation \({\chi }_{i}^{\ast }{\tan }^{1}(f(\psi ,{\chi }_{i}^{\ast }))=0\) numerically using any rootfinding scheme. However, without loss of generality, one can assume that the intrinsic frequency mismatch between the two oscillators is negligible, i.e. \({\rm{\Omega }}\simeq 0\). Ignoring Ω, the typeI fixed point is still selfconsistent, but the typeII fixed point can be simplified to:
where \({\rm{\Gamma }}\) is a positive expression that reflects the relative synaptic strength.
Network Model
A fully connected recurrent excitatory network with N = 200 typeII neuronal phase oscillators were simulated in the highfrequency regime with ν = 80 Hz. In the network simulation, the initial mean coupling is fixed at \(\bar{g}\)(0) = 0.5. The elements of the adjacency matrix A are multiplied by the corresponding array in the synaptic strength matrix G in order to establish a weighted adjacency matrix A_{G}. Note that the main diagonal arrays of the weighted adjacency matrix are zero since there is no selfloop in the network. Phase oscillators obey Eq. (1) and the excitatory synapses are modified by the pairbased STDP profile according to Eq. (2). The phases of the oscillators are initially uniformly distributed between 0 and π. The dendritic propagation delay is fixed at τ_{d} = 0.5 ms. STDP parameters are A_{+} = A_{−} = 0.005, and τ_{+} = τ_{−} = 20 ms. We also define an order parameter r(t) for the network of phase oscillators ranging between 0 and 1 that measures the degree to which the system is synchronized^{71}:
where Ψ(t) is the average phase and N is the number of neuronal oscillators.
Counting Loops
A bidirectional connection corresponds to a closed loop of length 2 in a network of neurons. In order to measure the number of such closed loops, a directed graph is constructed. Transformation of the synaptic strength matrix G, into a directed graph is performed by considering a threshold h = 0.2. Assuming that there are no selfloops, i.e. \({g}_{ii}=0\), the corresponding value in the graph adjacency matrix M of the resultant directed graph is assigned to 1 when the synaptic strength is greater than h, and zero otherwise. The number of closed loops of length 2 in the graph adjacency matrix M is then^{21,23}:
where Tr denotes the matrix trace. In order to perform a better comparison, L_{2} is normalized to the total number of possible loops of the same length i.e. N(N − 1)/2, ignoring selfloops, where N denotes the number of the neuronal phase oscillators or nodes in the network. Therefore, the resulted L_{2} is a normalized number between 0 and 1.
Asymmetry Index
To quantify the proportion of unidirectional connections and discriminate asymmetric modification of the synapses from symmetric changes (either symmetric potentiation or symmetric depression), we used another measure which has been introduced by Bayati and Valizadeh^{17}. We first defined the synaptic cost of the network as the sum of all synaptic strengths in the weighted adjacency matrix \({{\bf{A}}}_{{\bf{G}}}\), i.e. \(S={\sum }_{i,j}{a}_{ij}{g}_{ij}\), where a_{ij} and g_{ij} are the arrays of adjacency A and synaptic strength G matrices, respectively. The asymmetry level of the connections between two neurons can be measured by calculating the quantity C_{ij} = −C_{ji} = a_{ij} g_{ij} − a_{ji} g_{ji}. The asymmetry index of the network can then be defined as \({C}_{{\rm{net}}}\mathrm{=(1/}S){\sum }_{i > j}{C}_{ij}\). By definition, the network asymmetry index C_{net} is scaled in the range [0,1]. In the case of a fully symmetric network, C_{net} = 0, whereas in a fully asymmetric network, C_{net} = 1.
Data availability
All data generated or analysed during this study are included in this published article (and its Supplementary Information files).
References
Gerstner, W., Kempter, R., van Hemmen, J. L. & Wagner, H. A neuronal learning rule for submillisecond temporal coding. Nature 383, 76 (1996).
Markram, H., Lübke, J., Frotscher, M. & Sakmann, B. Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps. Science 275, 213–215 (1997).
Bi, G. Q. & Poo, M. M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience 18, 10464–10472 (1998).
Song, S., Miller, K. D. & Abbott, L. F. Competitive hebbian learning through spiketimingdependent synaptic plasticity. Nature Neuroscience 3, 919–926 (2000).
Bi, G. Q. & Poo, M. M. Synaptic modification by correlated activity: Hebb’s postulate revisited. Annual Review of Neuroscience 24, 139–166 (2001).
Kempter, R., Gerstner, W. & van Hemmen, J. L. Intrinsic stabilization of output rates by spikebased hebbian learning. Neural Computation 13, 2709–2741 (2001).
Gütig, R., Aharonov, R., Rotter, S. & Sompolinsky, H. Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. Journal of Neuroscience 23, 3697–3714 (2003).
Izhikevich, E. M., Gally, J. A. & Edelman, G. M. Spiketiming dynamics of neuronal groups. Cerebral Cortex 14, 933–944 (2004).
Morrison, A., Aertsen, A. & Diesmann, M. Spiketimingdependent plasticity in balanced random networks. Neural Computation 19, 1437–1467 (2007).
Gilson, M., Burkitt, A. N., Grayden, D. B., Thomas, D. A. & van Hemmen, J. L. Emergence of network structure due to spiketimingdependent plasticity in recurrent neuronal networks. i. input selectivity–strengthening correlated input pathways. Biological Cybernetics 101, 81–102 (2009).
Gilson, M., Burkitt, A. N., Grayden, D. B., Thomas, D. A. & van Hemmen, J. L. Emergence of network structure due to spiketimingdependent plasticity in recurrent neuronal networks. ii. input selectivity–symmetry breaking. Biological Cybernetics 101, 103–114 (2009).
Gilson, M., Burkitt, A. N., Grayden, D. B., Thomas, D. A. & van Hemmen, J. L. Emergence of network structure due to spiketimingdependent plasticity in recurrent neuronal networks. iii. partially connected neurons driven by spontaneous activity. Biological Cybernetics 101, 411–426 (2009).
Gilson, M., Burkitt, A. N., Grayden, D. B., Thomas, D. A. & van Hemmen, J. L. Emergence of network structure due to spiketimingdependent plasticity in recurrent neuronal networks. iv. Biological Cybernetics 101, 427–444 (2009).
Gilson, M., Burkitt, A. N., Grayden, D. B., Thomas, D. A. & van Hemmen, J. L. Emergence of network structure due to spiketimingdependent plasticity in recurrent neuronal networks. v. selforganization schemes and weight dependence. Biological Cybernetics 103, 365–386 (2010).
Gilson, M., Burkitt, A. & van Hemmen, L. J. Stdp in recurrent neuronal networks. Frontiers in Computational Neuroscience 4, 1–15 (2010).
Mikkelsen, K., Imparato, A. & Torcini, A. Emergence of slow collective oscillations in neural networks with spiketiming dependent plasticity. Physical Review Letters 110, 208101 (2013).
Bayati, M. & Valizadeh, A. Effect of synaptic plasticity on the structure and dynamics of disordered networks of coupled neurons. Physical Review E 86, 011925 (2012).
Abbott, L. F. & Nelson, S. B. Synaptic plasticity: taming the beast. Nature Neuroscience 3, 1178–1183 (2000).
Song, S. & Abbott, L. F. Cortical development and remapping through spike timingdependent plasticity. Neuron 32, 339–350 (2001).
Lubenov, E. V. & Siapas, A. G. Decoupling through synchrony in neuronal circuits with propagation delays. Neuron 58, 118–131 (2008).
Kozloski, J. & Cecchi, G. A. A theory of loop formation and elimination by spike timingdependent plasticity. Frontiers in Neural Circuits 4, 1–11 (2010).
Knoblauch, A., Hauser, F., Gewaltig, M.O., Körner, E. & Palm, G. Does spiketimingdependent synaptic plasticity couple or decouple neurons firing in synchrony? Frontiers in Computational Neuroscience 6, 1–27 (2012).
Babadi, B. & Abbott, L. F. Pairwise analysis can account for network structures arising from spiketiming dependent plasticity. PLoS Computational Biology 9, e1002906 (2013).
Karbowski, J. & Ermentrout, G. B. Synchrony arising from a balanced synaptic plasticity in a network of heterogeneous neural oscillators. Physical Review E 65, 031902 (2002).
Masuda, N. & Kori, H. Formation of feedforward networks and frequency synchrony by spiketimingdependent plasticity. Journal of Computational Neuroscience 22, 327–345 (2007).
GoldmanRakic, P. S. Development of cortical circuitry and cognitive function. Child Development 58, 601–622 (1987).
Markram, H., Lübke, J., Frotscher, M., Roth, A. & Sakmann, B. Physiology and anatomy of synaptic connections between thick tufted pyramidal neurones in the developing rat neocortex. The Journal of Physiology 500, 409–440 (1997).
Song, S., Sjöström, P. J., Reigl, M., Nelson, S. & Chklovskii, D. B. Highly nonrandom features of synaptic connectivity in local cortical circuits. PLoS Biology 3, e68 (2005).
Morishima, M. & Kawaguchi, Y. Recurrent connection patterns of corticostriatal pyramidal cells in frontal cortex. Journal of Neuroscience 26, 4394–4405 (2006).
Douglas, R. J. & Martin, K. A. Recurrent neuronal circuits in the neocortex. Current Biology 17, R496–R500 (2007).
Lücken, L., Popovych, O. V., Tass, P. A. & Yanchuk, S. Noiseenhanced coupling between two oscillators with longterm plasticity. Physical Review E 93, 032210 (2016).
Madadi Asl, M., Valizadeh, A. & Tass, P. A. Dendritic and axonal propagation delays determine emergent structures of neuronal networks with plastic synapses. Scientific Reports 7, 39682 (2017).
Maistrenko, Y. L., Lysyansky, B., Hauptmann, C., Burylko, O. & Tass, P. A. Multistability in the kuramoto model with synaptic plasticity. Physical Review E 75, 066207 (2007).
Popovych, O. V. & Tass, P. A. Desynchronizing electrical and sensory coordinated reset neuromodulation. Frontiers in Human Neuroscience 6, 1–14 (2012).
Popovych, O. V., Yanchuk, S. & Tass, P. A. Selforganized noise resistance of oscillatory neural networks with spike timingdependent plasticity. Scientific Reports 3, 2926 (2013).
Popovych, O. V. & Tass, P. A. Control of abnormal synchronization in neurological disorders. Frontiers in Neurology 5, 1–12 (2014).
Tass, P. A. Vibrotactile coordinated reset stimulation for the treatment of neurological diseases: Concepts and device specifications. Cureus 9, 1–16 (2017).
Tass, P. A. & Majtanik, M. Longterm antikindling effects of desynchronizing brain stimulation: a theoretical study. Biological Cybernetics 94, 58–66 (2006).
AgmonSnir, H. & Segev, I. Signal delay and input synchronization in passive dendritic structures. Journal of Neurophysiology 70, 2066–2085 (1993).
Schierwagen, A. & Claus, C. Dendritic morphology and signal delay in superior colliculus neurons. Neurocomputing 38, 343–350 (2001).
Cleland, B., Levick, W., Morstyn, R. & Wagner, H. Lateral geniculate relay of slowly conducting retinal afferents to cat visual cortex. The Journal of Physiology 255, 299–320 (1976).
Swadlow, H. A. & Weyand, T. G. Corticogeniculate neurons, corticotectal neurons, and suspected interneurons in visual cortex of awake rabbits: receptivefield properties, axonal properties, and effects of eeg arousal. Journal of Neurophysiology 57, 977–1001 (1987).
Swadlow, H. A. Efferent neurons and suspected interneurons in s1 forelimb representation of the awake rabbit: receptive fields and axonal properties. Journal of Neurophysiology 63, 1477–1498 (1990).
Ernst, U., Pawelzik, K. & Geisel, T. Synchronization induced by temporal delays in pulsecoupled oscillators. Physical Review Letters 74, 1570–1573 (1995).
Ermentrout, B., Pascal, M. & Gutkin, B. The effects of spike frequency adaptation and negative feedback on the synchronization of neural oscillators. Neural Computation 13, 1285–1310 (2001).
Sadeghi, S. & Valizadeh, A. Synchronization of delayed coupled neurons in presence of inhomogeneity. Journal of Computational Neuroscience 36, 55–66 (2014).
Esfahani, Z. G., Gollo, L. L. & Valizadeh, A. Stimulusdependent synchronization in delayedcoupled neuronal networks. Scientific Reports 6, 23471 (2016).
Ermentrout, B. Type i membranes, phase resetting curves, and synchrony. Neural Computation 8, 979–1001 (1996).
Achuthan, S. & Canavier, C. C. Phaseresetting curves determine synchronization, phase locking, and clustering in networks of neural oscillators. Journal of Neuroscience 29, 5218–5233 (2009).
Câteau, H., Kitano, K. & Fukai, T. Interplay between a phase response curve and spiketimingdependent plasticity leading to wireless clustering. Physical Review E 77, 051909 (2008).
Kempter, R., Gerstner, W. & Van Hemmen, J. L. Hebbian learning and spiking neurons. Physical Review E 59, 4498 (1999).
Aoki, T. & Aoyagi, T. Coevolution of phases and connection strengths in a network of phase oscillators. Physical Review Letters 102, 034101 (2009).
Pariz, A. et al. High frequency neurons determine effective connectivity in neuronal networks. NeuroImage 166, 349–359 (2018).
Zeitler, M. & Tass, P. A. Augmented brain function by coordinated reset stimulation with slowly varying sequences. Frontiers in Systems Neuroscience 9, 1–11 (2015).
Zeitler, M. & Tass, P. A. Antikindling induced by twostage coordinated reset stimulation with weak onset intensity. Frontiers in Computational Neuroscience 10, 1–22 (2016).
Bayati, M., Valizadeh, A., Abbassian, A. & Cheng, S. Selforganization of synchronous activity propagation in neuronal networks driven by local excitation. Frontiers in Computational Neuroscience 9, 1–15 (2015).
Pfister, J. P. & Gerstner, W. Triplets of spikes in a model of spike timingdependent plasticity. Journal of Neuroscience 26, 9673–9682 (2006).
Sjöström, P. J., Turrigiano, G. G. & Nelson, S. B. Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32, 1149–1164 (2001).
Chaudhuri, R. & Fiete, I. Computational principles of memory. Nature Neuroscience 19, 394–403 (2016).
Pfister, J. P. & Tass, P. Stdp in oscillatory recurrent networks: theoretical conditions for desynchronization and applications to deep brain stimulation. Frontiers in Computational Neuroscience 4, 1–10 (2010).
Morrison, A., Diesmann, M. & Gerstner, W. Phenomenological models of synaptic plasticity based on spike timing. Biological Cybernetics 98, 459–478 (2008).
Tass, P. A. A model of desynchronizing deep brain stimulation with a demandcontrolled coordinated reset of neural subpopulations. Biological Cybernetics 89, 81–88 (2003).
Tass, P. A. et al. Coordinated reset has sustained aftereffects in parkinsonian monkeys. Annals of Neurology 72, 816–820 (2012).
Wang, J. et al. Coordinated reset deep brain stimulation of subthalamic nucleus produces longlasting, dosedependent motor improvements in the 1methyl4phenyl1, 2, 3, 6tetrahydropyridine nonhuman primate model of parkinsonism. Brain Stimulation: Basic, Translational, and Clinical Research in Neuromodulation 9, 609–617 (2016).
Tass, P. A., Adamchic, I., Freund, H. J., von Stackelberg, T. & Hauptmann, C. Counteracting tinnitus by acoustic coordinated reset neuromodulation. Restorative Neurology and Neuroscience 30, 137–159 (2012).
Adamchic, I. et al. Coordinated reset neuromodulation for parkinson’s disease: Proofofconcept study. Movement Disorders 29, 1679–1684 (2014).
SyrkinNikolau, J. et al. Coordinated reset vibrotactile stimulation shows prolonged improvement in parkinson’s disease. Movement Disorders 33, 179–180 (2018).
Gilson, M., Bürck, M., Burkitt, A. N. & van Hemmen, J. L. Frequency selectivity emerging from spiketimingdependent plasticity. Neural Computation 24, 2251–2279 (2012).
Izhikevich, E. M. Weakly pulsecoupled oscillators, fm interactions, synchronization, and oscillatory associative memory. IEEE Transactions on Neural Networks 10, 508–526 (1999).
Hoppensteadt, F. C. & Izhikevich, E. M. Weakly connected neural networks (Springer Science & Business Media, 1997).
Kuramoto, Y. Chemical oscillations, waves, and turbulence (Springer Science & Business Media, 1984).
Acknowledgements
The study was funded by the John A. Blume Foundation. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Author information
Authors and Affiliations
Contributions
A.V. and P.A.T. conceived the study, M.M.A. conducted the numerical simulations and theoretical approximations, M.M.A., A.V. and P.A.T. analysed the results. M.M.A., A.V. and P.A.T. wrote and reviewed the paper.
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Madadi Asl, M., Valizadeh, A. & Tass, P.A. DelayInduced Multistability and Loop Formation in Neuronal Networks with SpikeTimingDependent Plasticity. Sci Rep 8, 12068 (2018). https://doi.org/10.1038/s41598018305659
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598018305659
Further reading

Multistability in a star network of Kuramototype oscillators with synaptic plasticity
Scientific Reports (2021)

Multicoding in neural information transfer suggested by mathematical analysis of the frequencydependent synaptic plasticity in vivo
Scientific Reports (2020)

Comparing feedforward and recurrent neural network architectures with human behavior in artificial grammar learning
Scientific Reports (2020)

Permutationbased time irreversibility in epileptic electroencephalograms
Nonlinear Dynamics (2020)

Synchronization and multistability in the coupled neurons with propagation and processing delays
Nonlinear Dynamics (2020)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.