Abstract
There is an increasing need to implement neuromorphic systems that are both energetically and computationally efficient. There is also great interest in using electric elements with memory, memelements, that can implement complex neuronal functions intrinsically. A feature not widely incorporated in neuromorphic systems is history-dependent action potential time adaptation which is widely seen in real cells. Previous theoretical work shows that power-law history dependent spike time adaptation, seen in several brain areas and species, can be modeled with fractional order differential equations. Here, we show that fractional order spiking neurons can be implemented using super-capacitors. The super-capacitors have fractional order derivative and memcapacitive properties. We implemented two circuits, a leaky integrate and fire and a Hodgkin–Huxley. Both circuits show power-law spiking time adaptation and optimal coding properties. The spiking dynamics reproduced previously published computer simulations. However, the fractional order Hodgkin–Huxley circuit showed novel dynamics consistent with criticality. We compared the responses of this circuit to recordings from neurons in the weakly-electric fish that have previously been shown to perform fractional order differentiation of their sensory input. The criticality seen in the circuit was confirmed in spontaneous recordings in the live fish. Furthermore, the circuit also predicted long-lasting stimulation that was also corroborated experimentally. Our work shows that fractional order memcapacitors provide intrinsic memory dependence that could allow implementation of computationally efficient neuromorphic devices. Memcapacitors are static elements that consume less energy than the most widely studied memristors, thus allowing the realization of energetically efficient neuromorphic devices.
Similar content being viewed by others
Introduction
Neuronal and artificial intelligence benefit from history-dependent self-adapting systems1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16. History-dependence explains neuronal dynamics from the movement of synaptic receptors to the generation of complex patterns of action potentials or spikes2,4,17,18,19,20,21,22. Our fractional order leaky integrate-and-fire (LIF) and Hodgkin–Huxley (HH) computer models reproduced a wide range of results of power-law history dependent firing rate and spike timing adaptation21. A fractional order differential equation is a non-local operator that involves intrinsic memory from previous activity23. Since power-law history-dependence in neurons show optimal coding properties5,7, it would be of importance to build neuromorphic circuits that implement such non-linear self-adapting mechanism. However, implementing a fractional order derivative in an electric circuit has been elusive, primarily due to the lack of electric elements with such properties24.
Electrical elements with memory, or memelements, are theoretic electric components whose intrinsic characteristics change with previous activity25. The most studied memelement for neuromorphic applications is the memristor, which has been realized with different materials26,27,28,29. A less studied element is the memcapacitor30. Memcapacitors could theoretically consume lower static power than memristors, see analysis in31 and other results32,33,34. Theoretical work shows that memelements can be described with fractional order integro-differential equations35. Within the physical foundations of the LIF and HH, the fractional order derivative corresponds to a fractional order capacitor. Thus, we propose that neuromorphic systems could be implemented using memcapacitors with fractional order differentiation properties.
In this work, we show that a particular electric element known as a super-capacitor36 has both, fractional differentiation and memcapacitive properties. Using super-capacitors we implemented a fractional LIF circuit37. The fractional LIF circuit replicates all the properties of our computer model that are also found in multiple brain areas5,38,39,40. We also implemented a fractional order HH circuit. This circuit showed distinct dynamics that were not previously described with our models, but that we were able to replicate. Under some input current conditions the circuit generated spiking dynamics consistent with a system in criticality. Both, the fractional LIF and HH circuits showed optimal coding properties. We found out that the critical dynamics of the fractional HH circuit reproduces the spontaneous and evoked activity of pyramidal neurons from the electro-sensory line lobe (ELL) in live immobilized weakly-electric fish (Apteronotus leptorhynchus). We used the fractional HH circuit to predict changes in spiking activity after prolonged stimulation, which we replicated experimentally. We also reproduced our results using a CMOS implementation of a fractional order capacitor41. Taken all together, our work shows that we can use fractional order memcapacitors to implement neuromorphic elements with intrinsic firing rate adaptation and optimal coding properties. This firing rate adaptation is the result of intrinsic memory and not a simple relaxation of the spiking mechanism. The realization of memcapacitive based neuromorphic elements could provide a platform for the implementation of systems that are energy and computationally efficient.
Results
A super-capacitor with fractional order differentiator and memcapacitive properties
The fractional order LIF model is21:
where \(v\) is voltage; \({G}_{m}\) is the membrane conductance; \(i\) is current; \(t\) is time; and \({D}^{\eta }\) is the Caputo definition of a fractional order derivative of order \(\eta\):
The \({\left(t-u\right)}^{-\eta }\) term is the intrinsic memory trace. The voltage resets at threshold (\({v}_{Th}\)) and has a refractory period (\({t}_{r}\)). The memory trace continuously integrates past activity.
The classic LIF is a model of current through a resistor and an ideal capacitor in parallel (Eq. 1 with \(\eta =1)\)42. The current, \({i}_{c}\), through the capacitor with capacitance \({C}_{o}\) is:
By analogy, in the fractional LIF model this corresponds to a fractional order capacitor:
Furthermore, real capacitors have an associated conductance (\({G}_{c}\))43:
Using a Laplace transform on both sides, re-arranging and applying an inverse Laplace transform the response of this circuit to a constant input current, \({i}_{c}\left(t\right)=I\) , is:
With \({\tau }_{C}={C}_{o}/{G}_{c}\), and \({E}_{\eta }\left(z\right)\) the Mittag–Leffler function. At asymptotic times, Eq. 6 converges to a power-law:
When \(\eta =1\) and \({{R}_{c}=1/G}_{c}=0\) the equation reduces to the ideal capacitor:
The charging and discharging properties of some super-capacitors follow fractional order dynamics44. Super-capacitors have capacitances orders of magnitude larger than traditional electrolytic capacitors because these elements are mainly designed for energy storage applications45. We connected in series four \(22 \text{ }mF\) super-capacitors to reduce their capacitance (\(5.5 \text{ }mF\) equivalent). Using Eq. 7 we determined that the super-capacitor stack had a fractional derivative order, \(\eta = 0.82 \pm 2.20 \times 10^{ - 3} \;{ }95\%\;{\text{CI}}\) (Fig. 1A)43.
A memcapacitor is defined as25,30:
where \(q\) is electric charge; \(C\) is capacitance with units of \(F/x\cdot t\); and \(x\) is an internal variable (such as the flux, \(\varphi\)). From the definition of flux:
If we assume that \(C\left(\varphi ,v,t\right)={C}_{0}\varphi\), then
Equation 11 results in a hysteresis curve pinched at the origin. Indeed, the measured \(q\) vs.\(V\) curve of the super-capacitors (Fig. 1B) shows hysteresis pinched at the origin and a frequency dependent amplitude, both properties of memcapacitors25. Thus, the super-capacitors have fractional order derivative and memcapacitive properties.
A fractional order leaky integrate-and-fire analog circuit
We implemented an analog circuit of a LIF neuron37 (Fig. 2A). The circuit generates spikes of less than \(0.1 \text{ } ms\) with \({v}_{th}=2 \text{ }V\). We substituted \({C}_{1}=0.1 \text{ } \mu F\) with the super-capacitor stack. When using an electrolytic \(5 \text{ } mF\) capacitor the LIF circuit failed to generate action potentials (\(0.1-10 \text{ }mA\) input), not shown. However, the circuit with the super-capacitor stack generated spikes with the same \({v}_{th}\) but with higher amplitude (Fig. 2B). This could be because the charging of super-capacitors is much slower than electrolytic capacitors thus allowing the circuit to function.
The classic LIF generates action potentials at a constant rate when stimulated with constant current (not shown). By contrast, the fractional LIF shows a delay in generating the first spike, followed by a power-law decrease in the inter-spike intervals (ISIs, Fig. 2C,D). The combination of slowed onset in firing and acceleration of the ISI affects the calculation of the firing rate from the time of onset of the first spike (Fig. 2E)21. However, at longer stimulation times, the firing rate of the fractional LIF circuit becomes faster than for the classical case. We have described this process with our computer model in which the intrinsic memory trace acts as a feedback mechanism that feeds into increasing firing rates21.
We wanted to test how previous activity affected the firing rate adaptation of the fractional LIF circuit5. We injected square-wave inputs of different frequencies into the fractional LIF circuit (Fig. 3A). The DC and amplitude of input signal were adjusted to always generate action potentials. We calculated the time constant of instantaneous firing rate adaptation for the positive and negative parts of the cycle as a function of period length by fitting a single exponential to each phase of the cycle (Fig. 3A black). This resulted in a time constant of adaptation that depended on the previous period length, a non-linear phenomenon observed in real neurons5,38 (Fig. 3B black). The fractional LIF model reproduced this behavior (\(\eta =0.2\), Fig. 3B gray). Thus, the firing rate of the fractional LIF shows history dependence.
We wanted to determine if the fractional order properties of the super-capacitors were reflected in the firing rate activity of the neuron. If the firing rate, \(f\), of a neuron encodes the input, \({I}_{n}={\text{Asin}}(2\pi \omega t),\) as a fractional order derivative, then46:
We injected a sinusoidal input to the fractional LIF neuronal circuit. The amplitude and DC components were set to generate spikes at all input values. We varied the input wavelength (\(\lambda =1/\omega\)) from \(0.1 \text{ }s\) to \(40 \text{ }s\) and fitted a sine function to the instantaneous firing rate (Fig. 3C). This shows that the firing rate amplitude decays as a power-law, \({\eta }_{G}= 0.39\pm 0.07 \text{ } 95 \% \text{ } CI\), and the phase shift is fixed \({\eta }_{\phi }= 0.32\pm 0.04 \text{ }SEM\), Fig. 3D. Thus, the spiking of the fractional LIF circuit has fractional order derivative properties.
The power-spectrum of multiple natural signals have a power-law structure (\(1/{f}^{\beta }\)) also known as pink-noise7,47,48,49,50,51,52,53,54,55,56. From Eq. 12 if \(\beta =\eta\) and \({\text{A}}{=(2\mathrm{\pi \omega })}^{-\beta }\) then the power spectrum of the firing rate should be flat, \({\eta }_{LIF}\sim 0\), also known as spectrum whitening which is a tale-tale of optimal coding computations57,58. The same additive properties predict a constant shift in the exponent of the firing rate power spectrum, \({\Delta \eta }_{LIF}={\eta }_{LIF}-\beta\). We calculated \({\Delta \eta }_{LIF}\) to inputs with pink-noise (range 0.01 to 1.00 Hz) and varying \(\beta\) values. (Fig. 4A–D). This analysis shows that for \(\beta =[-0.2, -0.4, -0.6]\) the value \({\Delta \eta }_{LIF}\) is constant. For values of \(\beta <-0.6\) the system becomes less sensitive to the input. The same analysis shows that the firing rate spectrum is flat when \(\beta =-0.2\) (\({\eta }_{LIF}=-0.09 \pm 0.14 \text{ } 95 \% \text{ } CI\)).
The fractional differentiation properties of the leaky integrate-and-fire circuit depend on the fractional order of the super-capacitors
We used an analogue circuit simulation platform to study the effect of fractional oder super-capacitors of different orders on the responses of the fractional LIF circuit (see Methods). To do this we used an available super-capacitor model59,60 (Fig. 5A). We systematically varied the values of the resistances in the model and characterized the resulting fractional order (Fig. 5B). We then built an analog circuit simulation of the fractional LIF. The model showed spike latencies (Fig. 5C), firing rate adaptation to square inputs (Fig. 5D), and fractional derivative properties (Fig. 5E) as a function of the fractional order of the super-capacitor (Fig. 5F).
A fractional order Hodgkin–Huxley circuit shows criticality and optimal coding
Criticality in the fractional order Hodgkin–Huxley circuit
Voltage activated membrane conductances show history-dependence61. To study history-dependence in a biophysical model of neuronal activity we implemented a circuit of the classic HH model62 (Fig. 6A). The circuit generated spikes that closely matched the HH model (Sup. Mat. Fig. S1). Following on our previous studies of a fractional HH computer model20 we added to the \(n\)-gate part of the circuit a stack of five super-capacitors (\(47 \text{ }mF\) each) in series (\(9.40 \text{ }mF\) equivalent) with fractional order of \({\eta }_{2} = 0.93 \pm 0.01 \text{ }95\% \text{ } CI\) (Sup. Mat. Fig. S2). The fractional and classic circuits had the same \({v}_{th}\). At high currents (\(>0.25 \text{ }mA\)) the fractional HH circuit showed increased variability and reduction in firing rate (Fig. 6B). This was because the circuit generated a mixture of normal and complex spikes, spikes containing a plateau potential of varying periods (Fig. 6C) that lasted for as long as the stimulus was on (not shown).
The fractional HH computer model replicated the complex spikes (Sup. Mat. Fig. S3) using \(\eta =0.2-0.3\) for the \(n\)-gate. A phase plane plot of the measured and modeled complex spikes shows mixed mode oscillations with a smaller attractor representing the plateau (Sup. Mat. Fig. S4)20 known as a canard63,64. Thus, the complex spikes emerge from the internal dynamics of the system and not from some saturation in the circuit.
We hypothesized that the apparent random nature of the complex spikes was because the circuit was at criticality65. We defined an avalanche as the number of simple spikes between two complex spikes. The histogram of avalanches follows a power-law, consistent with a system in criticality (Fig. 6D). This was also replicated in the model with the fractional order of the \(n\)-gate linearly related to the avalanche structure (Fig. 6E,F). Thus, complex spikes emerge when the system is at criticality.
Diffusion entropy analysis characterizes stochastic processes with memory66,67,68. This analysis shows that the classical and fractional HH at low-input current have no memory. By contrast, the circuit at criticality shows history-dependence (Fig. 6G). The difference in dynamics in the fractional HH circuit is probably due that at low input current the dynamics are controlled by the electrolytic capacitor and at higher input current (and voltages) the super-capacitors dominate the dynamics.
Instead of super-capacitors we used a CMOS implementation of fractional order capacitors41, see Supplementary Material Fig. S5. For the three different values of fractional order implemented in the chip the HH circuit generated avalanches and the spiking activity showed diffusion entropy history-dependence (Fig. 6H–K).
Fractional order capacitance is also found in skin, fruits, and vegetables, due to their fractal cellular structures69,70,71. To generalize our results, we implemented the circuit using dried fruit, \(\eta =0.67\pm 0.03 \text{ } 95\% \text{ }CI\). Under such arrangement the HH circuit generated spikes and power-law ISIs (Sup. Mat. Fig. S6). Thus, collectively, our results show that fractional order capacitance, independent of its physical nature, is sufficient to generate fractional order spiking activity.
Transition from simple to complex spiking suggests a change in coding
We characterized the firing rate of the circuit to sinusoidal input with fixed amplitude (\(AC = 0.05 \text{ }mA\)) and frequency (\(1 \text{ }Hz\)) and varying DC (Fig. 7A). The amplitude of the firing rate decreases and flips signs as DC values increase while the phase switches from negative to positive (Fig. 7B,C). The flipping of the firing rate amplitude is because of widening of the spikes without becoming complex spikes (\(<0.22 \text{ }mA\)). In the presence of complex spikes, there was no phase lag (not shown). We called the regime where there is positive phase in the absence of complex spikes the sub-critical phase. The reversal in amplitude can be interpreted as a change in coding the input from the firing rate to the ISI (Eq. 12).
The fractional Hodgkin–Huxley circuit reproduces and predicts sensory processing and critical dynamics of neurons from the live weakly-electric fish
Fractional differentiation, optimal coding, and criticality in fish and circuit
Neurons in weakly-electric fish perform a fractional differentiation of modulated electrical stimulation with a gain characterized by \({\eta }_{ELLg}=0.26\pm 0.09 \text{ } 95\% \text{ } CI\), and phase advance of \({\eta }_{ELL\phi }=0.43\pm 0.02 \text{ }SEM\) (Fig. 8A) mediated by a potassium conductance48. We replicated this behavior in the fractional HH circuit in the sub-critical state with an input current with DC of \(2.4 \text{ }\mu A\) and amplitude of \(0.05 \text{ }mA\) over a range of 0.25 to 2.00 \(Hz\) (Fig. 8B). The power-law gain, \({\eta }_{HHg}=0.33\pm 0.08 \text{ } 95\% \text{ }CI\), and the phase advance \({\eta }_{HH\phi }=0.26 \pm 0.03 \text{ }SEM\) had values very close to those measured experimentally.
The same neurons that perform a fractional order differentiation show spectrum whitening of pink-noise signals. When some LS neurons from the ELL are stimulated with pink-noise with \({\beta }_{In}=-0.83\) the power-spectrum of the firing rate is flat, \({\eta }_{ELLout}=0.02 \pm 4x{10}^{-3} \text{ }95\%\text{ } CI\) , Fig. 8C 7. We stimulated the fractional HH circuit with pink-noise with \(\beta =[-0.2, -2]\). It was when we used a pink-noise with \(\beta =-0.2\) that the exponent of the firing rate was flat, \({\eta }_{HHout}= -0.07 \pm 0.21 \text{ } 95\% \text{ }CI\). We also calculated the power-spectrum to pink-noise with \(\beta =-0.75\) expecting a power-spectrum \({\eta }_{HHout}=-0.55\), the experimental results were close to this value, \({\eta }_{HHOut}=-0.41\pm 0.07 \text{ } 95\% \text{ }CI\) (Fig. 8D).
Without changing any parameters in the fractional HH circuit, we tested if we could reproduce other properties of ELL neurons. For example, neurons that show fractional order differentiation can be classified as ON or OFF cells. The ON cells follow the input, while OFF cells decrease their firing when the input is high. Our fractional HH circuit at low input current (\(DC = 1.0 \text{ }mA\), amplitude \(0.05 \text{ }mA\)) reproduces the ON behavior and without fractional order differentiation. In the sub-critical regime (\(DC = 2.4 \text{ }mA\), amplitude \(0.05 \text{ }mA\)), the circuit shows fractional differentiation and OFF behavior (Fig. 8E).
At rest, the neurons of the ELL generate spiking bursts72. We hypothesized that the burst were equivalent to complex spikes observed in the circuit. Indeed, the avalanches, number of spikes between bursts, in the weakly-electric fish have a power-law distribution with an exponent of \(-0.92\pm 0.06 \text{ } 95\% \text{ }CI\) (Fig. 8F).
The fractional HH circuit predicts long-term spike time adaptation
History-dependence should result in inputs affecting the spiking of the neuron for periods of time longer than its characteristic time constants. Using the circuit in normal, sub-critical, or critical regimes we delivered a stimulus (lower the input current for 30 s but high enough to keep firing) and monitored the spiking for 10 min. We calculated the relative cumulative spikes between the post- and pre-stimulus times (see Methods). The classical (Fig. 9A) and fractional sub-critical (not shown) circuits showed no changes in spiking activity (horizontal lines). By contrast, when we performed the measurements with the stimulus the circuit showed a prolonged and robust decrease in firing (Fig. 9B,C).
While recording from the live fish, we delivered a masking electric stimulus that inhibit ELL cells, similar to the ‘low’ input current in the HH circuit. We calculated the cumulative spikes in control and stimulus conditions. This shows that ELL neurons respond to the masking stimulus with a robust increase in the cumulative spikes (Fig. 9D). The neurons continue to respond to subsequent stimulus over multiple repetitions. After the 8th period, the neurons show no change in cumulative spiking with respect to control (Fig. 9E).
Past work shows that fractional order optimal coding properties of ELL pyramidal cells are strongly influenced by both feedback input from nucleus praeminentialis (nP) as well as serotonergic input73. We bilaterally blocked feedback signals and repeated the experiment showing no change in cumulative firing (Fig. 9F). Taken together, our results show that primary sensory neurons show long lasting intrinsic memory that is consistent with their fractional order properties.
Discussion
In this work, we showed that an off-the-shelf super-capacitor is a memcapacitor and a fractional order differentiator. We then used our theoretical framework of fractional order neurodynamics20,21,74 to implement fractional order LIF and HH circuits. These circuits replicate non-linear properties of single neurons observed in multiple brain areas and animal species. Of particular importance is that fractional order dynamics provides a natural mechanism for optimal coding of signals characterized by power-laws, which are common in natural and constructed environments53,75,76,77,78,79,80,81. Remarkably, we did not tune any of the parameters of our circuits to any particular experimental results. Thus, showing that fractional order differentiation could be a robust neuromorphic implementation that encompasses the history-dependence properties of real neurons.
Super-capacitors as fractional order memelements and a platform for efficient neuromorphic systems
Super-capacitors are distinct from electrolytic capacitors due to their large energy density82. The large majority of super-capacitors achieve high capacitances due to the use of porous materials like activated carbon obtained from natural sources82. The super-capacitors employed in this work use this technology. These super-capacitors have an electric double-layer between a solid electrode and an aqueous electrolyte (for a review refer to83). A parameter of great importance for super-capacitors is the minimum voltage necessary to activate the electrolytic reaction, known as the cell voltage. Our super-capacitors have a cell voltage of \(1.7 \text{ }V\)84. This is the voltage at which fractional order differentiation and then the appearance of complex spikes is observed in the fractional HH circuit (Fig. 7B, transforming the DC applied voltage). Thus, the emergence of history dependent properties of the circuit is explained by the activation of the super-capacitors.
The super-capacitors had a fixed fractional order intrinsic to their physical properties. While this is a current challenge in fabrication, there are promising new technologies41,44,85,86,87,88, including their miniaturization86. In any case, we analyzed the dependence of the fractional differentiation of the firing rate with respect to the fractional order of the super-capacitor using circuit models or CMOS implementations. In both cases, we linked the properties of the memcapacitor to the coding properties of the neuron. Our research supports efforts to develop fractional order super-capacitors with varying orders and miniaturized to allow the implementation of neuromorphic devices31.
We demonstrated that the fractional order properties of the circuit are not exclusive to electric implementations. Instead, by using a CMOS circuit, we showed that any fractional order dielectric could provide the dynamics necessary to implement fractional order neuromorphic systems. We consider our results using dried fruit as a further demonstration that any dielectric with fractional order properties could provide functionality to neuromorphic devices; however, we are well aware that these measurements have low precision and uniformity due to the nature of the natural product. In any case, our work supports research in the development of biomimetic memcapacitors89,90.
Memcapacitors can consume lower static power than memristors, and thus have the potential for building neuromorphic systems orders of magnitude more energy-efficient31,32,33,34. This is because capacitors are electric field based, instead of current based. Initial work with neural networks not only supports this idea, but also suggests that they could be computationally efficient91,92,93. Thus, fractional-order memcapacitros could be a platform for energetically and computational efficient neuromorphic systems.
Further development of fractional order LIF and HH systems
The fractional order LIF reproduced all the properties of our previously published computer model21. However, there were some limitations. The fractional differentiation required that the input current had a relatively small amplitude, suggesting that some other parts of the circuit dominate its dynamics at larger input amplitudes. It was not our objective to maximize the effect of any part of the circuit, but to demonstrate the possibility of implementing fractional neurodynamics in neuromorphic circuits. In any case, our work using the circuit model analysis provides the building blocks to optimize parameters for particular input ranges and tasks.
The fractional HH circuit has three states: normal, sub-critical, and critical. Our previous computational work suggested the emergence of complex patterns and complex spikes as a function of input current and time20. However, computational limitations did not allow us to investigate our computer model for periods comparable to what we reported here. Thus, showing another advantage of a neuromorphic approach.
The classical regime is consistent with low input current that does not engage the super-capacitor properties. This requires further study in relation to super-capacitor physical properties and design. Another area of study is the sub-criticality regime where we showed fractional order differentiation. In this regime we interpreted that the signal was being encoded by the ISI instead of the firing rate. This requires experimental corroboration and an understanding of how neuromorphic systems could operate in this regime. Finally, at higher input currents, the circuit generated simple and complex spikes. The complex spikes were reproduced by our models and are consistent with the emergence of a canard. This dynamics requires further study in the context of chaotic systems94,95,96 and the physical and intrinsic memory properties of the super-capacitors44. In all cases, it is necessary to determine the energy consumption of the circuits and compare to other implementations.
The rich dynamics shown by the fractional HH circuit reminded us of the spiking properties of ELL neurons in the weakly-electric fish. These neurons show fractional differentiation and optimal coding properties in the live animal receiving naturalistic stimulation. The fractional HH circuit was not capable to reproduce all properties of all the neurons in only one of its states. However, we were able to match the behavior of different real neurons with the circuit in the classical, sub-critical, or critical state. While the matches were not identical, the overall properties are captured by our neuromorphic element. The important point is that the response to the dynamical input depends on the bias constant current in which this is mounted, opening the possibility of using the same neuron to perform different computational tasks dynamically.
Instead of using the circuits to replicate published work, we used the fraction HH circuit to predict the behavior of ELL neurons in the live fish. Our results show that the spiking activity of these neurons retain the effect of a stimulus for multiple minutes. Furthermore, our experimental results show that this adaptation is over tens of minutes and reflects adaptation at much larger scales that previously thought. These results challenge current models of primary sensory neurons97,98. Traditionally, these neurons are expected to transmit input reliably and fast to the rest of the nervous system. The fact that they show long-term adaptation to previous input calls into question current ideas of coding in which these neurons are assumed, essentially, memoryless.
Recent directions in neuromorphic spiking circuit is to reduce the use of capacitor or fully eliminate them99. However, systems made on memcapacitive technology could be at least 8 × more energetically efficient that with resistive devices31 while other theoretical work suggest that memcapacitive reservoir computing could be efficient computationally efficient91. Future work should address the trade-offs between the computational properties of fractional order capacitors against their impact on area efficiency in circuit design and operation speed of complex neuromorphic hardware.
Perspective
A grand challenge of Artificial Intelligence and engineering is to build systems that are concurrently energetically and computationally efficient100. Our work further supports the use of memcapacitors by focusing on those elements that also implement a fractional order differentiation. By combining memcapacitive and fractional differentiation properties these systems could be energy efficient and computationally optimal for the types of signals commonly found in nature. Our results also show that neuromophic elements that reproduce fractional order neurodynamics can be used to predict the responses of real neuronal systems, thus providing a promising platform for brain-machine interfaces101.
Methods
Super-capacitors, circuits, and computer models
Super-capacitors
For all of our circuits we used NEC/TOKIN (TOKIN, Japan) super-capacitors. For the LIF circuit we used 22 mF super-capacitors (part # FS0H223ZF) rated at 5.5 V and 60 Ω maximum equivalent series resistance (ESR) at 1 kHz. Similarly, for the HH circuit we used 47 mF super-capacitors (part # FYH0H473ZF) also rated at 5.5 V but with 100 Ω maximum ESR at 1 kHz. Both types of super-capacitors show fractional order dynamics44.
The fractional leaky integrate-and-fire model and circuit
The fractional order LIF is21 (Eq. 5):
where \({v}_{m}\) is the membrane voltage, \({v}_{r}\) is the resting voltage, \({\tau }_{m}= {r}_{m}{c}_{m}\) is the membrane time constant, with \({r}_{m}\) the membrane resistance and \({c}_{m}\) the membrane conductance. The conductance is \({{g}_{m}=1/r}_{m}\). The input current is \(I\). The neuron generates a spike when \({v}_{m}\ge {v}_{Th}\) and the voltage is clamped to \({v}_{r}\) for a refractory period, \({t}_{r}\). The fractional order of the derivative is \(0<\eta \le 1\).
We used the Grunwald–Letnikov integration of the fractional derivative
With the integration time \(\Delta t=0.1 \text{ }ms\). The memory weights are
With \({c}_{0}^{\eta }=\eta\). The system can also be solved using the L1 scheme (see below). We determined that a memory window longer than 10 s did not change the results.
The original LIF circuit is the well-known design by Carver Mead (Fig. 2A). The circuit uses MOSFET transistors. Transistors \({T}_{1}\), \({T}_{2}\), \({T}_{3}\) and \({T}_{6}\) were p-channel; and \({T}_{4}\) and \({T}_{5}\) were n-channel. The \({T}_{1}\) and \({T}_{2}\) transistors implement a resetting of the voltage, Vm, that is due to charge accumulation of capacitor C1. The transistor circuits \({T}_{3} -{T}_{4}\) and \({T}_{5} -{T}_{6}\) implement current mirrors that generate the spike. The original circuit had the following characteristics \({C}_{1}=0.1 \text{ }\mu F\); \(C2 = 22 \text{ }\mu F\); \(Vb = 5 \text{ }V\). The source of \({T}_{4}\) and \({T}_{5}\) were at \(5 \text{ }V\). The spiking traces analyzed were measured from \(Vm\). As described in the results, \({C}_{1}\) was substituted with a stack of super-capacitors in series.
Hodgkin–Huxley model and circuit
The Hodgkin–Huxley model is:
where \(\overline{{g }_{K}}\) is the total potassium conductance; \(\overline{{g }_{Na}}\) is the total sodium conductance;\(n\)
is the activation potassium gate; \(m\) is the sodium activation gate; and \(h\) is the sodium inactivation gate. The general form of the gating variables is
where \(x = [ n, m, h ]\), the function \({\alpha }_{x}\) is the forward rate, and \({\beta }_{x}\) is the backward rate. The functional forms of \(n\), \(m\), and \(h\) are
The fractional order gating dynamics is
In this work we concentrated in studying the effects of fractional order in the potassium n-gate.
The normal parameter values were \({c}_{m}= 0.47 \text{ }\mu F\), \(\overline{{g }_{Na}}= 120\text{ } mS\), \(\overline{{g }_{k}}= 36\text{ } mS\), \(\overline{{g }_{m}}= 0.3\text{ } mS\), \({E}_{Na}= 50 \text{ }mV\), \({E}_{K}= -77 \text{ }mV\), and \({v}_{r} = -65 \text{ }mV\). However, to replicate the criticality behavior (Fig. 6E) we modified the model with the following parameters: \({c}_{m}= 1 \text{ }\mu F\),\(\overline{{g }_{Na}}= 70\text{ } mS\), \(\overline{{g }_{k}}= 5\text{ } mS\), and \({E}_{K}= -77 \text{ }mV\), \({v}_{r} = -54.4 \text{ }mV\).
For these simulations we used a Runge–Kutta level 4 for the integer derivative equations and an L1 scheme for the fractional order potassium gate:
with
We used a previously published design of a circuit that can replicate the Hodgkin–Huxley model62, Fig. 6A. This circuit has four branches where each part of the circuit is equivalent to a part of the model. The two branches in the middle correspond to the sodium and potassium conductances. The original circuit has the following parameter values \({C}_{m}= 0.1 \text{ }\mu F\), \({r}_{l}= 100 \text{ }k\Omega\), \({E}_{l}= 0 \text{ }V\),\({E}_{Na}= 5 \text{ }V\), \({R}_{1}= 100 \text{ }\Omega\), \({R}_{2}= 100 \text{ }\Omega\), \({R}_{3}= 1 \text{ }k\Omega\), \({R}_{4}= 674 \text{ }\Omega\), \({E}_{K}= 1.1\text{ } V\). The capacitor that controlled the potassium conductance had an original value of \({C}_{n} = 0.1 \text{ }mF\). For the fractional HH circuit, we connected Cn in series with five super-capacitors.
Measurements
We controlled the delivery of current or voltage and data acquisition with a National Instruments card (DAQ, PCI-6071E). We provided the current to our circuit with the DAQ connected to an Analog Stimulus Isolator (A-M Systems, model 2200). The membrane potential was acquired with an analog input of the same DAC, and the information was processed using LabVIEW (National Instruments, TX).
Weakly-electric fish experiments
Animals
All animal procedures were approved by McGill University’s animal care committee and were performed in accordance with the guidelines of the Canadian Council on Animal Care. The study is reported in accordance with ARRIVE guidelines (https://arriveguidelines.org). Specimens of either sex of the wave-type weakly electric fish Apteronotus leptorhynchus (N = 7) were used exclusively in this study. Animals were purchased from tropical fish suppliers and were housed in groups (2–10 individuals) at controlled water temperatures (26–29 °C) and conductivities (100–800 µS*cm−1) according to published guidelines102. It was not possible to determine the age of the specimens used.
Surgery
Surgical procedures have been described in detail previously103,104. Briefly, 0.1–0.5 mg of tubocurarine (Sigma) was injected intramuscularly to immobilize the animals for electrophysiology experiments. The animals were then transferred to an experimental tank (30 cm x 30 cm x 10 cm) containing water from the animal’s home tank and respired by a constant flow of oxygenated water through their mouth at a flow rate of 10 mL*min−1. Subsequently, the animal’s head was locally anesthetized with lidocaine ointment (5%; AstraZeneca), the skull was partly exposed, and a small window (~ 5 mm2) was opened over the hindbrain as well as bilaterally over both midbrains in order to access nucleus praeeminentialis (nP) for drug application as described elsewhere103,104.
Stimulation
The electric organ discharge of Apteronotus leptorhynchus is neurogenic, and therefore is not affected by injection of curare. All stimuli consisted of amplitude modulations (AMs) of the animal’s own EOD and were produced by triggering a function generator to emit one cycle of a sine wave for each zero crossing of the EOD as done previously105. The frequency of the emitted sine wave was set slightly higher (~ 30 Hz) than that of the EOD, which allowed the output of the function generator to be synchronized to the animal’s discharge. The emitted sine wave was subsequently multiplied with the desired AM waveform (MT3 multiplier; Tucker Davis Technologies), and the resulting signal was isolated from the ground (A395 linear stimulus isolator; World Precision Instruments). The isolated signal was then delivered through a pair of chloritized silver wire electrodes placed 15 cm away from the animal on either side of the recording tank perpendicular to the fish’s rostro-caudal axis. This stimulation protocol has been used by multiple previous studies (see, e.g.,106,107). We used stimuli consisting of a 5–15 Hz noise (4th order Butterworth) carrier waveform (i.e., AM) whose amplitude (i.e., envelope) was further modulated sinusoidally at 0.05, 0.1, 0.25, 0.5, 1, 2 Hz to mimic the envelope signals due to relative movement between two fish108,109,110. Alternatively, we delivered a step increase in EOD amplitude lasting 30 s and repeated 20 times at intervals of 300 s. The data in Fig. 8 was extracted from the plots in their corresponding publication7.
Recordings
The data presented in Fig. 8A,D,G were performed with simultaneous extracellular recordings from ELL pyramidal cells within the lateral segment using metal-filled micropipettes as described previously7,103,104,105,111,112. We used CED 1401-plus hardware and Spike II software (Cambridge Electronic Design, Cambridge, UK) to record the resulting signal with resolution 0.1 ms.
For the experiments in Figs. 8F and 9 simultaneous extracellular recordings from ELL pyramidal cells were made using Neuropixels probes (Imec Inc.) as done previously113,114,115,116. The probe was angled at approximately 15 deg. with respect to vertical and advanced between 1500 and 2000 µm into the ELL with reference to the probe tip. Neurons were recorded at depths between 200 and 1400 µm from the brain surface and as such most likely included neurons within the lateral and centrolateral segments117. Recordings were digitized at 30 kHz using spikeGLX (Janelia Research Campus) and stored on a hard drive for further analysis. Spikes were sorted using Kilosort118 and subsequently curated using the phy application (https://github.com/cortex-lab/phy) 118,119.
Pharmacological inactivation
To study the effects of descending pathways, we recorded ELL pyramidal cells and behavior for control in conjunction with their responses after bilateral lidocaine (1 mM) injection into nP. Drug application pipettes were made using single-barrel borosilicate capillary glass tubes (OD 1.5 mm, ID 0.86 mm, A-M Systems) and pulled by a vertical micropipette puller (Stoelting) to a fine tip that was subsequently broken to attain a tip diameter of approximately 5 μm. All pharmacological injections were performed approximately 1250–1750 μm below the surface where the nP is located using a duration of 130 ms at ~ 20 psi using a Picospritzer (General Valve) as done previously103,104,120. It is important to note that the effects of such inactivation start within at most 30 s after injection and last throughout the experiment.
Code and analysis
We developed custom made MATLAB (Natick, MA) code to simulate the spiking models, and to analyze all the simulations and experiments. We stimulated and acquired the data from the circuit with a National Instruments data acquisition card (DAQ, PCI-6071E). We provided the current to our circuit with the DAQ connected to an Analog Stimulus Isolator (A-M Systems, model 2200). The membrane potential was acquired with an analog input of the same DAC, and the information was processed using LabVIEW. We report 95% confidence intervals (95% CI) and Standard Error of the Mean (SEM).
Diffusion entropy analysis
Diffusion Entropy Analysis (DEA) was originally introduced by Allegrini et al.66. Modified versions of DEA have been applied recently67,68. We consider a diffusion process given by
This trajectory is converted into many diffusional trajectories using a moving window ‘l’ to record \(\Delta x\left( t \right) = x\left( {t + l} \right) - x\left( t \right)\). For different values of t, it is equivalent to generating many Gibbs copies from which we can get the probability distribution function (PDF) p(x,l). We can then evaluate the Shannon entropy of the process using,
If the PDF is a Gaussian function, then \(\delta = 0.5.\) If it is a Lévy function, it fits the scaling property,
where \(\delta\) is the anomalous scaling \(\delta > 0.5.\) Using Eq. 24 and 25 in the asymptotic limit \(l \to \infty\) we get,
where A is a constant. Note that when the PDF departs from the ordinary Gaussian case, it has slow tails with an inverse power-law function.
Data availability
All data collected and data analyses are available in GitHub (Github.com/SantamariaLab) or by request to Dr. Fidel Santamaria (fidel.santamaria@utsa.edu).
References
Xiao, F., Hrabe, J. & Hrabetova, S. Anomalous extracellular diffusion in rat cerebellum. Biophys. J. 108(9), 2384–2395 (2015).
Santamaria, F., Wils, S., De Schutter, E. & Augustine, G. J. Anomalous diffusion in purkinje cell dendrites caused by spines. Neuron. 52(4), 635–648 (2006).
Marinov, T. & Santamaria, F. Modeling the effects of anomalous diffusion on synaptic plasticity. BMC Neurosci. 14(1), P343 (2013).
Santamaria, F., Wils, S., De Schutter, E. & Augustine, G. J. The diffusional properties of dendrites depend on the density of dendritic spines. Eur. J. Neurosci. 34(4), 561–568 (2011).
Lundstrom, B. N., Higgs, M. H., Spain, W. J. & Fairhall, A. L. Fractional differentiation by neocortical pyramidal neurons. Nat. Neurosci. 11(11), 1335–1342 (2008).
Anastasio, T. J. The fractional-order dynamics of brainstem vestibulo-oculomotor neurons. Biol. Cybern. 72(1), 69–79 (1994).
Huang, C. G. & Chacron, M. J. Optimized parallel coding of second-order stimulus features by heterogeneous neural populations. J. Neurosci. 36(38), 9859 (2016).
Holland, J. H. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence (MIT Press, Cambridge, 1992).
Bakirov, R., Fay, D. & Gabrys, B. Automated adaptation strategies for stream learning. Mach. Learn. 110(6), 1429–1462 (2021).
Tang, J. et al. Bridging biological and artificial neural networks with emerging neuromorphic devices: Fundamentals, progress, and challenges. Adv. Mater. 31(49), 1902761 (2019).
Boybat, I. et al. Neuromorphic computing with multi-memristive synapses. Nat. Commun. 9(1), 1–12 (2018).
Li, C. et al. Efficient and self-adaptive in-situ learning in multilayer memristor neural networks. Nat. Commun. 9(1), 1–8 (2018).
de Lemos, R. & Grześ, M., (eds). Self-adaptive artificial intelligence. In: 2019 IEEE/ACM 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS) (2019). IEEE.
Couceiro, M. S., Rocha, R. P., Ferreira, N. & Machado, J. Introducing the fractional-order Darwinian PSO. Signal Image Video Process. 6(3), 343–350 (2012).
Gheibi, O., Weyns, D. & Quin, F. Applying machine learning in self-adaptive systems: A systematic literature review. ACM Trans. Auton. Adapt. Syst. (TAAS). 15(3), 1–37 (2021).
Cao, J., Lin, Z. & Huang, G.-B. Self-adaptive evolutionary extreme learning machine. Neural Process. Lett. 36(3), 285–305 (2012).
Santamaria, F., Gonzalez, J., Augustine, G. J. & Raghavachari, S. Quantifying the effects of elastic collisions and non-covalent binding on glutamate receptor trafficking in the post-synaptic density. PLOS Comput. Biol. 6(5), e1000780 (2010).
Weinberg, S. H. & Santamaria, F. History dependent neuronal activity modeled with fractional order dynamics. Comput. Models Brain Behav., 531–548 (2017).
Vázquez-Guerrero, P., Gómez-Aguilar, J. F., Santamaria, F. & Escobar-Jiménez, R. F. Synchronization patterns with strong memory adaptive control in networks of coupled neurons with chimera states dynamics. Chaos Solitons Fractals 128, 167–175 (2019).
Teka, W., Stockton, D. & Santamaria, F. Power-law dynamics of membrane conductances increase spiking diversity in a Hodgkin–Huxley model. PLOS Comput. Biol. 12(3), e1004776 (2016).
Teka, W., Marinov, T. M. & Santamaria, F. Neuronal spike timing adaptation described with a fractional leaky integrate-and-fire model. PLOS Comput. Biol. 10(3), e1003526 (2014).
Gilboa, G., Chen, R. & Brenner, N. History-dependent multiple-time-scale dynamics in a single-neuron model. J. Neurosci. 25(28), 6479–6489 (2005).
Tarasov, V. E. No nonlocality. No fractional derivative. Commun. Nonlinear Sci. Numeri. Simul. 62, 157–163 (2018).
Elwakil, A. S. Fractional-order circuits and systems: An emerging interdisciplinary research area. IEEE Circuits Syst. Mag. 10(4), 40–50 (2010).
Di Ventra, M., Pershin, Y. V. & Chua, L. O. Circuit elements with memory: memristors, memcapacitors, and meminductors. Proc. IEEE. 97(10), 1717–1724 (2009).
Chua, L. O. & Kang, S. M. Memristive devices and systems. Proc. IEEE. 64(2), 209–223 (1976).
Li, Y., Wang, Z., Midya, R., Xia, Q. & Yang, J. J. Review of memristor devices in neuromorphic computing: materials sciences and device challenges. J. Phys. D Appl. Phys. 51(50), 503002 (2018).
Upadhyay, N. K. et al. Emerging memory devices for neuromorphic computing. Adv. Mater. Technol. 4(4), 1800589 (2019).
Jeong, H. & Shi, L. Memristor devices for neural networks. J. Phys. D Appl. Phys. 52(2), 023003 (2018).
Yin, Z., Tian, H., Chen, G. & Chua, L. O. What are memristor, memcapacitor, and meminductor?. IEEE Trans. Circuits Syst. II Express Briefs. 62(4), 402–406 (2015).
Demasius, K.-U., Kirschen, A. & Parkin, S. Energy-efficient memcapacitor devices for neuromorphic computing. Nat. Electron. 4(10), 748–756 (2021).
Abbasi, A., Setoudeh, F., Tavakoli, M. B. & Horri, A. A novel design of high performance and robust ultra-low power SRAM cell based on memcapacitor. Nanotechnology. 33(16), 165202 (2022).
Wang, Z. et al. Capacitive neural network with neuro-transistors. Nat. Commun. 9(1), 3208 (2018).
Han, J.-K., Yu, J.-M., Kim, D.-W. & Choi, Y.-K. An artificial neuron with a leaky fin-shaped field-effect transistor for a highly scalable capacitive neural network. Adv. Intell. Syst. 4, 2200112 (2022).
Abdelouahab, M.-S., Lozi, R. & Chua, L. Memfractance: A mathematical paradigm for circuit elements with memory. Int. J. Bifurc. Chaos 24(09), 1430023 (2014).
Raza, W. et al. Recent advancements in supercapacitor technology. Nano Energy. 52, 441–473 (2018).
Mead, C. Analog VLSI and neural systems. Reading, Mass.: Addison-Wesley; 1989. xxii, 371 p., 8 p. of plates p.
Miller, M. N., Okaty, B. W. & Nelson, S. B. Region-specific spike-frequency acceleration in layer 5 pyramidal neurons mediated by Kv1 subunits. J. Neurosci. 28(51), 13716–13726 (2008).
Gutmaniene, N., Svirskiene, N. & Svirskis, G. Firing properties of frog tectal neurons in vitro. Brain Res. 981(1–2), 213–216 (2003).
Beierlein, M., Gibson, J. R. & Connors, B. W. Two dynamically distinct inhibitory networks in layer 4 of the neocortex. J. Neurophysiol. 90(5), 2987–3000 (2003).
Tsirimokou, G., Psychalinos, C., Elwakil, A. S. & Salama, K. N. Electronically tunable fully integrated fractional-order resonator. IEEE Trans. Circuits Syst. II Express Briefs. 65(2), 166–170 (2017).
Dayan, P. & Abbott, L. F. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, Cmabridge, 2005).
Allagui, A., Freeborn, T. J., Elwakil, A. S. & Maundy, B. J. Reevaluation of performance of electric double-layer capacitors from constant-current charge/discharge and cyclic voltammetry. Sci. Rep. 6, 38568 (2016).
Allagui, A. et al. Review of fractional-order electrical characterization of supercapacitors. J. Power Sources. 400, 457–467 (2018).
Şahin, M. E., Blaabjerg, F. & Sangwongwanich, A. A comprehensive review on supercapacitor applications and developments. Energies. 15(3), 674 (2022).
Podlubny, I. Fractional differential equations: an introduction to fractional derivatives, fractional differential equations, to methods of their solution and some of their applications: Elsevier (1998).
Pozzorini, C., Naud, R., Mensi, S. & Gerstner, W. Temporal whitening by power-law adaptation in neocortical neurons. Nat. Neurosci. 16(7), 942–948 (2013).
Huang, C. G., Zhang, Z. D. & Chacron, M. J. Temporal decorrelation by SK channels enables efficient neural coding and perception of natural stimuli. Nat. Commun. 7(1), 11353 (2016).
Metzen, M. G. & Chacron, M. J. Weakly electric fish display behavioral responses to envelopes naturally occurring during movement: implications for neural processing. J. Exp. Biol. 217(8), 1381–1391 (2014).
Atick, J. J. & Redlich, A. N. What does the retina know about natural scenes?. Neural Comput. 4(2), 196–210 (1992).
Stamper, S. A., Madhav, M. S., Cowan, N. J. & Fortune, E. S. Beyond the Jamming Avoidance Response: weakly electric fish respond to the envelope of social electrosensory signals. J. Exp. Biol. 215(23), 4196–4207 (2012).
Sproule, M. K., Metzen, M. G. & Chacron, M. J. Parallel sparse and dense information coding streams in the electrosensory midbrain. Neurosci. Lett. 607, 1–6 (2015).
Lewicki, M. S. Efficient coding of natural sounds. Nat. Neurosci. 5(4), 356–363 (2002).
Rodríguez, F. A., Chen, C., Read, H. L. & Escabí, M. A. Neural modulation tuning characteristics scale to efficiently encode natural sound statistics. J. Neurosci. 30(47), 15969–15980 (2010).
Dan, Y., Atick, J. J. & Reid, R. C. Efficient coding of natural scenes in the lateral geniculate nucleus: Experimental test of a computational theory. J. Neurosci. 16(10), 3351–3362 (1996).
Wang, X.-J., Liu, Y., Sanchez-Vives, M. V. & McCormick, D. A. Adaptation and temporal decorrelation by single neurons in the primary visual cortex. J. Neurophysiol. 89(6), 3279–3293 (2003).
Rieke, F., Warland, D., Steveninck, R. R. & Bialek, W. Spikes: Exploring the Neural Code (MIT Press, Cambridge, 1999).
Atick JJ. Could information theory provide an ecological theory of sensory processing? Netw. Comput. Neural Syst. 22(1–4):4–44 (2011).
Weddell, A. S., Merrett, G. V., Kazmierski, T. J. & Al-Hashimi, B. M. Accurate supercapacitor modeling for energy harvesting wireless sensor nodes. IEEE Trans. Circuits Syst. II Express Briefs. 58(12), 911–915 (2011).
Zubieta, L. & Bonert, R. Characterization of double-layer capacitors for power electronics applications. IEEE Trans. Ind. Appl. 36(1), 199–205 (2000).
Villalba-Galea, C. A. Hysteresis in voltage-gated channels. Channels. 11(2), 140–155 (2017).
Maeda, Y. & Makino, H. A pulse-type hardware neuron model with beating, bursting excitation and plateau potential. BioSystems. 58(1–3), 93–100 (2000).
Wechselberger, M., Mitry, J. & Rinzel, J. Canard theory and excitability. Nonauton. Dyn. Syst. Life Sci. 2102, 89–132 (2013).
Jalics, J., Krupa, M. & Rotstein, H. G. Mixed-mode oscillations in a three time-scale system of ODEs motivated by a neuronal model. Dyn. Syst. 25(4), 445–482 (2010).
O’Byrne, J. & Jerbi, K. How critical is brain criticality?. Trends Neurosci. 45, 820–837 (2022).
Allegrini, P., Grigolini, P., Hamilton, P., Palatella, L. & Raffaelli, G. Memory beyond memory in heart beating, a sign of a healthy physiological condition. Phys. Rev. E. 65(4), 041926 (2002).
Jelinek, H. F. et al. Diffusion entropy vs. multiscale and Renyi entropy to detect progression of autonomic neuropathy. Front. Physiol. 11, 607324 (2021).
Bohara, G., Lambert, D., West, B. J. & Grigolini, P. Crucial events, randomness, and multifractality in heartbeats. Phys. Rev. E 96(6), 062216 (2017).
Alimisis, V., Dimas, C., Pappas, G. & Sotiriadis, P. P. Analog realization of fractional-order skin-electrode model for tetrapolar bio-impedance measurements. Technologies 8(4), 61 (2020).
Roy, A., Mallick, A., Das, S. & Aich, A. An experimental method of bioimpedance measurement and analysis for discriminating tissues of fruit or vegetable. AIMS Biophysics. 7(1) (2020).
Mohsen, M., Said, L. A., Madian, A. H., Radwan, A. G. & Elwakil, A. S. Fractional-order bio-impedance modeling for interdisciplinary applications: A review. IEEE Access. 9, 33158–33168 (2021).
Ávila-Åkerberg, O., Krahe, R. & Chacron, M. Neural heterogeneities and stimulus properties affect burst coding in vivo. Neuroscience. 168(1), 300–313 (2010).
Berman, N. J. & Maler, L. Neural architecture of the electrosensory lateral line lobe: adaptations for coincidence detection, a sensory searchlight and frequency-dependent adaptive filtering. J. Exp. Biol. 202(10), 1243–1253 (1999).
Coronel-Escamilla, A., Tuladhar, R., Stamova, I. & Santamaria, F. Chapter Fifteen—Fractional-order dynamics to study neuronal function. In: Radwan, A. G., Khanday, F. A., Said, L. A. (eds). Fractional-Order Modeling of Dynamic Systems with Applications in Optimization, Signal Processing and Control. 2: Academic Press; 2022. p. 429–56.
Serra-Peralta, M., Serrà, J. & Corral, Á. Lognormals, power laws and double power laws in the distribution of frequencies of harmonic codewords from classical music. Sci. Rep. 12(1), 2615 (2022).
Levitin, D. J., Chordia, P. & Menon, V. Musical rhythm spectra from Bach to Joplin obey a 1/f power law. Proc. Natl. Acad. Sci. 109(10), 3716–3720 (2012).
Pinto, C. M., Lopes, A. M. & Machado, J. T. A review of power laws in real life phenomena. Commun. Nonlinear Sci. Numer. Simul. 17(9), 3558–3578 (2012).
Brown, J. H. et al. The fractal nature of nature: power laws, ecological complexity and biodiversity. Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci. 357(1421), 619–626 (2002).
Machado, J. T., Pinto, C. M. & Lopes, A. M. A review on the characterization of signals and systems by power law distributions. Signal Process. 107, 246–253 (2015).
Dong, D. W. & Atick, J. J. Statistics of natural time-varying images. Netw. Comput. Neural Syst. 6(3), 345 (1995).
Kinouchi, O., Diez-Garcia, R. W., Holanda, A. J., Zambianchi, P. & Roque, A. C. The non-equilibrium nature of culinary evolution. N. J. Phys. 10(7), 073020 (2008).
Sharma, K., Arora, A. & Tripathi, S. K. Review of supercapacitors: Materials and devices. J. Energy Storage. 21, 801–825 (2019).
González, A., Goikolea, E., Barrena, J. A. & Mysyk, R. Review on supercapacitors: Technologies and materials. Renew. Sustain. Energy Rev. 58, 1189–1206 (2016).
TOKIN. SuperCapacitor User's manual 2017 [Available from: https://www.tokin.com/english/product/pdf_dl/supercap_manual.pdf.
Chatterjee, D. P. & Nandi, A. K. A review on the recent advances in hybrid supercapacitors. J. Mater. Chem. A 9(29), 15880–15918 (2021).
Wang, J., Li, F., Zhu, F. & Schmidt, O. G. Recent progress in micro-supercapacitor design, integration, and functionalization. Small Methods. 3(8), 1800367 (2019).
Sung, J.-H., Kim, S.-J., Jeong, S.-H., Kim, E.-H. & Lee, K.-H. Flexible micro-supercapacitors. J. Power Sources. 162(2), 1467–1470 (2006).
Ahmadi, P., Maundy, B., Elwakil, A. S. & Belostotski, L. High-quality factor asymmetric-slope band-pass filters: A fractional-order capacitor approach. IET Circuits Devices Syst. 6(3), 187–197 (2012).
Najem, J. S. et al. Dynamical nonlinear memory capacitance in biomimetic membranes. Nat. Commun. 10(1), 3239 (2019).
Basham, C., Pitz, M., Najem, J., Sarles, S., Hasan, M. S. (eds). Memcapacitive devices in neuromorphic circuits via polymeric biomimetic membranes. Smart Materials, Adaptive Structures and Intelligent Systems; 2019: American Society of Mechanical Engineers.
Tran, D. & Teuscher, C. Computational capacity of complex memcapacitive networks. J. Emerg. Technol. Comput. Syst. 17(2), 17 (2021).
Tran, S. D. & Teuscher, C., editors. Memcapacitive reservoir computing. 2017 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH); 2017: IEEE.
Tran, D. & Teuscher, C. Memcapacitive devices in logic and crossbar applications. arXiv preprint arXiv:170405921. (2017).
Rajagopal, K., Akgul, A., Jafari, S. & Aricioglu, B. A chaotic memcapacitor oscillator with two unstable equilibriums and its fractional form with engineering applications. Nonlinear Dyn. 91, 957–974 (2018).
Ma, X. et al. A novel simple chaotic circuit based on memristor–memcapacitor. Nonlinear Dyn. 100, 2859–2876 (2020).
Fitch, A. L., Iu, H. H. & Yu, D. editors. Chaos in a memcapacitor based circuit. 2014 IEEE international symposium on circuits and systems (ISCAS); 2014: IEEE.
Doi, E. & Lewicki, M. S. A Simple model of optimal population coding for sensory systems. PLOS Comput. Biol. 10(8), e1003761 (2014).
Gjorgjieva, J., Meister, M. & Sompolinsky, H. Functional diversity among sensory neurons from efficient coding principles. PLOS Comput. Biol. 15(11), e1007476 (2019).
Liang, F.-X., Wang, I.-T. & Hou, T.-H. Progress and benchmark of spiking neuron devices and circuits. Adv. Intell. Syst. 3(8), 2100007 (2021).
Verdecchia, R., Sallou, J., & Cruz, L. A systematic review of Green AI. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery :e1507 (2023).
Coronel-Escamilla, A., Gomez-Aguilar, J. F., Stamova, I. & Santamaria, F. Fractional order controllers increase the robustness of closed-loop deep brain stimulation systems. Chaos Solitons Fractals 140, 110149 (2020).
Hitschfeld, E. M., Stamper, S. A., Vonderschen, K., Fortune, E. S. & Chacron, M. J. Effects of restraint and immobilization on electrosensory behaviors of weakly electric fish. ILAR J. 50(4), 361–372 (2009).
Huang, C. G., Metzen, M. G. & Chacron, M. J. Feedback optimizes neural coding and perception of natural stimuli. Elife. 7, e38935 (2018).
Metzen, M. G., Huang, C. G. & Chacron, M. J. Descending pathways generate perception of and neural responses to weak sensory input. PLoS Biol. 16(6), e2005239 (2018).
Bastian, J., Chacron, M. J. & Maler, L. Receptive field organization determines pyramidal cell stimulus-encoding capability and spatial stimulus selectivity. J. Neurosci. 22(11), 4577–4590 (2002).
Marsat, G., Proville, R. D. & Maler, L. Transient signals trigger synchronous bursts in an identified population of neurons. J. Neurophysiol. 102(2), 714–723 (2009).
Mejias, J. F., Marsat, G., Bol, K., Maler, L. & Longtin, A. Learning contrast-invariant cancellation of redundant signals in neural systems. PLoS Comput. Biol. 9(9), e1003180 (2013).
Stamper, S. A., Fortune, E. S. & Chacron, M. J. Perception and coding of envelopes in weakly electric fishes. J. Exp. Biol. 216(Pt 13), 2393–2402 (2013).
Metzen, M. G. & Chacron, M. J. Weakly electric fish display behavioral responses to envelopes naturally occurring during movement: Implications for neural processing. J. Exp. Biol. 217(Pt 8), 1381–1391 (2014).
Metzen, M. G., Chacron, M. J. Envelope coding and processing: Implications for Perception and behavior. In: Carlson, B., Sisneros, J., Popper, A., Fay, R., editors. Electroreception: Fundamental Insights from Comparative Approaches. Springer Handbook of Auditory Research. Cham: Springer; 2019. p. 251–77.
Bastian, J., Chacron, M. J. & Maler, L. Plastic and non-plastic cells perform unique roles in a network capable of adaptive redundancy reduction. Neuron. 41, 767–779 (2004).
Huang, C. G., Zhang, Z. D. & Chacron, M. J. Temporal decorrelation by SK channels enables efficient neural coding and perception of natural stimuli. Nat. Commun. 7, 11353 (2016).
Haggard, M. & Chacron, M. J. Coding of object location by heterogeneous neural populations with spatially dependent correlations in weakly electric fish. PLoS Comput. Biol. 19, e1010938 (2023).
Marquez, M. M. & Chacron, M. J. Serotonin increases population coding of behaviorally relevant stimuli by enhancing responses of ON but not OFF-type sensory neurons. Heliyon. 9, e18315 (2023).
Metzen, M. G. & Chacron, M. J. Descending pathways increase sensory neural response heterogeneity to facilitate decoding and behavior. iScience. https://doi.org/10.1016/j.isci.2023.107139 (2023).
Wang, Z. & Chacron, M. J. Synergistic population coding of natural communication stimuli by hindbrain electrosensory neurons. Sci. Rep. 11(1), 10840 (2021).
Maler, L., Sas, E., Johnston, S. & Ellis, W. An atlas of the brain of the weakly electric fish Apteronotus Leptorhynchus. J. Chem. Neuroanat. 4, 1–38 (1991).
Steinmetz, N. A. et al. Neuropixels 2.0: A miniaturized high-density probe for stable, long-term brain recordings. Science. 372(6539), eabf4588 (2021).
Metzen, M. G. & Chacron, M. J. Population coding of natural electrosensory stimuli by midbrain neurons. J. Neurosci. 41, 3822–3841 (2021).
Clarke, S. E. & Maler, L. Feedback synthesizes neural codes for motion. Curr. Biol. 27(9), 1356–1361 (2017).
Funding
National Institute of Health grant 1R01EB026939 and NSF-EFRI 2318139 to FS. CONACyT (Mexico) scholarship 625406 to PVG. Canadian Institutes of Health Research to MJC.
Author information
Authors and Affiliations
Contributions
PVG and FS wrote manuscript text and prepared figures. PVG implemented and characterized circuits. MC designed and carried out electrophysiology experiments. AE and CP provided CMOS circuit. RT provided diffusion entropy analysis. FS designed project. All authors reviewed the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Vazquez-Guerrero, P., Tuladhar, R., Psychalinos, C. et al. Fractional order memcapacitive neuromorphic elements reproduce and predict neuronal function. Sci Rep 14, 5817 (2024). https://doi.org/10.1038/s41598-024-55784-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-024-55784-1
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.