Introduction

The human brain—three pounds of matter between our ears—has not yet been understood completely because of its complexity. For many decades, researchers have focused on understanding the principles and detailed actions of the human brain and, in general, the mammalian brain1,2,3,4. The unique functionalities of the mammalian brain, such as parallel information processing, low power consumption and learning capacity, make it fascinating. These unique functionalities are of great interest to not only neuroscientists but also physicists and electrical/materials engineers. There have been many attempts to realize “artificial brains” either by hardware- or software-based techniques5,6,7,8,9,10. In particular, the latter is often termed as in silico neural network11. Here, the term ‘artificial brain’ denotes an electronic system that mimics some limited neural functionalities. In general, such efforts are often referred to as neuromorphic engineering, a phrase coined by Carver Mead5.

The basic elements in a mammalian brain—a complex neural network—are neurons and synapses; synapses define the connectivity between neighboring neurons and function as local memories12. Synapses and neurons are the basic elements in artificial neural networks (ANNs) as well. Note that in this study ANN denotes both hardware- and software-based networks. Neurons are of significant importance as they generate action potentials (also known as spikes) and work as information units in neural networks2. Various types of artificial neuron models can be employed in ANNs, such as leaky integrate-and-fire (LIF) neuron1,2,3, Hodgkin-Huxley neuron13 and Izhikevich neuron models14,15. Among these, the LIF neuron is the simplest model and can be easily implemented in ANNs1,2,3.

To date, a great deal of efforts have been made to realize the different types of artificial hardware neurons, including the LIF neuron, using conventional complementary metal-oxide-semiconductor (CMOS) technologies16,17. This CMOS-based approach has been the mainstream approach of neuromorphic (hardware) engineering. A recent emerging research trend in neuromorphic engineering is the increasing adoption of alternative approaches to realize artificial neurons and synapses. These emerging approaches differ from the mainstream approach in that neural functionalities are implemented by introducing functional materials-based elements that could partly replace CMOS-based elements in the former12. One of the advantages of these new approaches is that it may enable the circuitry of ANNs to be substantially simplified by using a less number of CMOS elements than the conventional approach. An example of such approaches is a recent breakthrough by Pickett et al., who achieved the LIF neuron by using two pairs of a Mott insulator and a capacitor18. Basically, the neuristor concept introduced by Crane19 in 1962 was employed in the LIF neuron model; hence this LIF neuron is termed as neuristor-based LIF (NLIF) neuron in this study so as to differentiate it from the standard LIF neuron.

Mott insulators are known to undergo temperature-driven insulator-to-metal transitions, so that the conductivity abruptly increases when the lattice temperature exceeds the transition temperature20. This transition is reversible, that is, the initial conductivity is recovered when the lattice temperature again falls below the threshold for the reverse transition, i.e., metal-to-insulator transition. As the lattice temperature change is due to Joule heating, the current–voltage (I–V) behavior of the Mott insulator is estimated to be volatile, i.e., threshold switching18,21,22. Threshold switching plays a key role in the functioning of the NLIF neuron18,23. Other than the Mott insulator, amorphous higher chalcogenides24,25,26,27, Si n+/p/n+ junctions28 and particular transition metal oxides such as NbOx23,29 are also known to exhibit volatile threshold switching.

Regardless of the type of threshold switch (TS) in the NLIF neuron, the variability of threshold switching behavior cannot be completely avoided24,26. It is, therefore, required to determine the effect of this variability on the neuronal behavior of the NLIF neuron that often leads to neuronal noises. The quantitative understanding of the noise is of significant importance when employing the NLIF neuron in both in silico and hardware-based ANNs because some noise properties are acceptable in ANNs insomuch as the noise does not cause serious errors during information processing30.

In this study, we conducted simulation on the NLIF neuron in order to identify its neuronal behavior – its noise characteristics and information representation in the presence of noise. Indeed, the employed analysis methods are widely used in characterization of other neuron models1,3,31,32, so that one can readily compare the characteristics of the NLIF neuron model with those of other models.

We first attempted to find optimized operational windows for the variables in the NLIF neuron model. It was indeed not an easy task to find the windows owing to the many variables involved simultaneously. We suggest a method to find successful spiking conditions by conducting static and dynamic calculations on the NLIF neuron circuit. The acquired windows should be narrowed down by taking into account the optimal selectivity of individual NLIF neurons for stimulation. The neuronal selectivity is one of the essential functions of neurons, given that they work as information encoders, in particular, in the presence of noise. Next, we allowed the variability of the TS in an individual NLIF neuron under optimal firing conditions. Such variability is most likely seen every switching cycle24,26—switching-event-driven variability. Threshold switching events repeatedly occur throughout the external stimulation duration, rendering the variability to feature a noise in the neuron’s response. By analyzing the noise property, the relationship between the distribution and the consequent noise is understood and compared with the noise present in biological neurons. A question arising from the analysis on individual NLIF neurons is “Can conveying information, such as encoding and decoding, be achieved in a reliable manner by a population of these individual NLIF neurons?” This question is related to neuronal behavior at a higher dimension, i.e., the group, rather than at the individual neuronal level. The Bayesian decoder was employed so as to examine the reliability of the information conveyed by a population of NLIF neurons. As a result, the reliability was evaluated by means of “uncertainty.”

Results

Optimal firing conditions of individual NLIF neurons

In a single NLIF neuron circuit, standard circuit elements such as resistors (R1, R2 and RL), capacitors (C1 and C2) and TSs (S1 and S2) are in use, as shown in Fig. 1a. 18 When it comes to a network of NLIF neurons, V2 in Fig. 1a is relayed to a neighboring neuron through a synapse, so that V2 works as the output voltage, corresponding to membrane potential. To generate a positive spike V2, Vdc2 and Vdc1 need to be positive and negative, respectively. For simplicity, it is assumed that Vdc2 = −Vdc1 = Vd. The two dc voltage sources (Vdc1 and Vdc2) effectively supply power – enabling active operation – and determine the spiking dynamics including the spike’s height and the undershoot level following the spike. The spiking dynamics will be explained in detail later. The key component in the NLIF neuron is the TS that performs monostable switching. The monostability of a TS can be understood from the schematic of current–voltage (I–V) hysteresis of the TS illustrated in Fig. 1b. The behavior of the TS is described by four parameters: Ron, Roff, Von and Voff, which denote the on- and off-state resistances and threshold voltages for off-to-on and on-to-off switching, respectively. The assumption of linear I–V behaviors in both the states allows constant Ron and Roff in a given operational voltage range. For simplicity, switches S1 and S2 are assumed to be identical.

Figure 1
figure 1

Circuitry of NLIF neuron and threshold switching behavior.

(a) Circuitry of the NLIF neuron. (b) A schematic of I−V behavior of a TS.

Full size image

The NLIF neuron fires spikes only when switch S2 flickers at a given input current (Iin). Note that the term “flicker” means completing a threshold switching cycle, for instance, that along the arrows in Fig. 1b. To meet this requirement, the five standard circuit components (R1, R2, RL, C1 and C2), the four operational parameters of the TS (Ron, Roff, Von and Voff) and the dc voltage (Vd), i.e., ten variables in total, should be optimized. Owing to the difficulty in optimizing such a large number of variables, it is required to rule out several variables, in particular the operational parameters of the TS, that are most likely estimated from available experimental data24,26,27,29. In this calculation, Roff and Von were set to 1 Mohm and 1 V, respectively, so that only two variables (Ron and Voff) of the TS remain. They were converted to the following normalized variables: Roff/Ron and Voff/Von. These threshold switching parameters are summarized in Table 1. Note that these ratios are often employed in characterizing resistive switching devices. A further reduction in the number of variables was made by setting Rd and Vd to 1 Gohm and 0.9 V, respectively. RL works as a voltage divider in this single neuron; it is desired to be large. Vd needs to be close to, but smaller than, Von so as to turn on switch S2 with a small input current Iin; 90 percent of Von, i.e., 0.9 V, was taken as Vd.

Table 1 Parameters used in the optimization of the operational window.
Full size table

To arrive at a condition of flickering switch S2 at a given Iin, time-independent calculations were performed with capacitors C1 and C2 ruled out (see Fig. 2a). The calculations provided Iin and R2 windows for spiking at given Roff/Ron and Voff/Von values. The condition drawn from these static calculations is a “prerequisite” for successful spiking in the time domain. This is because the capacitors only determine the rate of voltage redistribution in the NLIF neuron upon switching of S1 and S2 and the voltages across the two switches will eventually reach Von. Meeting the four requirements, shown in Fig. 2b and described below, allows S2 to flicker. Note that on-switching of switch S1 is a necessary condition for that of switch S2, but off-switching of switch S1 is unnecessary for that of switch S2. Requirement i: setting Roff for both switches in the circuit results in a voltage across switch S1 (|V1+Vd|) that is larger than Von, leading to the off-to-on switching of switch S1, given the aforementioned necessary condition for on-switching of switch S2. Requirement ii: setting Ron and Roff for switches S1 and S2, respectively, results in a voltage across switch S2 (|V2–Vd|) that is larger than Von, leading to off-to-on switching of switch S2. Requirement iii: setting Roff and Ron for switches S1 and S2, respectively, allows on-to-off switching of switch S2 by decreasing |V2–Vd| below Voff. Requirement iv: setting Ron for both switches allows on-to-off switching of switch S2 regardless of on- or off-switching of switch S1, given that off-switching of switch S1 is not a necessary condition for that of switch S2. Satisfying these requirements, R2 windows with respect to input current Iin and combinations of Roff /Ron (5, 10, 20 and 50) and Voff /Von (0.3, 0.5 and 0.7) values were obtained as indicated using the grey zones in Fig. 2c. The white zones correspond to the failure of spiking. Insomuch as a current rather than a voltage is applied, R1 and R2 are independent variables as can be seen in Fig. 2c. It is noticed that the higher Roff /Ron and the lower Voff /Von ratio are, the wider R2 window is. For the following calculations, we chose moderate parameters of the TS (Roff/Ron = 20 and Voff/Von = 0.5) and R1 and R2 of 100 kohm.

Figure 2
figure 2

Operational windows of series resistance.

(a) Circuitry of the NLIF neuron for static calculations. RL and Vd are set as 1 Gohm and 0.9 V, respectively. (b) A table of requirements for successful spiking. (c) Acquired operational windows (grey zones) of R2 and Iin for successful spiking at given ratios of Roff/Ron and Voff/Von.

Full size image

As mentioned earlier, the windows drawn from the static calculations serve as necessary, rather than sufficient, conditions for successful firing of the NLIF neuron in a time domain; therefore, capacitors C1 and C2 need be optimized as well. The only concern in spiking in due course would be the sequential on-switching events of switches S1 and S2 when both are in the off-state, i.e., aforementioned Requirements i and ii are met in consecutive order. The major role of capacitors C1 and C2 in spiking is time-dependent redistribution of V1 and V2 upon switching of S1 and S2. The capacitors determine the rate of the redistribution. That is, the higher the capacitance, the lower the rate. To satisfy the above-mentioned requirements, the time required for the evolution of V2—eventually leading to |V2–Vd|≥Von, i.e., on-switching of switch S2 upon the on-switching of switch S1—should be sufficiently short to hinder the off-switching of switch S1 in the meantime. Otherwise, an increase in |V2–Vd| in due course, owing to the on-switching of switch S1, would be abruptly diminished before Von is reached. Note that it was assumed that the switching times of S1 and S2 are sufficiently short to have a negligible impact on the time-dependent voltage redistribution. In addition, regarding the off-switching of switch S2, the static calculations basically assume the instability of the on-state of switch S2 (see Requirements iii and iv) and thus off-switching occurs regardless of capacitances of C1 and C2.

Given the above-mentioned requirements, a capacitance window for spiking at an input current of 1 μA is obtained as shown in Fig. 3a. The input current profile with respect to time is plotted in Fig. 3d. The maximum capacitance of C2 for successful spiking at a given capacitance of C1 tends to increase monotonically with that of C1. A higher capacitance of C1 allows a longer discharging time of C1; the discharging arises from the on-switching of switch S1 and continues as far as |V1+Vd| >Voff, i.e., off-switching of switch S1. A higher capacitance of C2 allows the charging time of C2 to be longer; the charging arises from an increase in |V2| (V2 < 0) occurring upon the prior on-switching of switch S1 and continues until |V2–Vd| ≥Von, i.e., on-switching of switch S2. Thus, a higher capacitance of C1 enables the capacitance range of C2 to widen, leading to the formation of the capacitance window shown in Fig. 3a. To highlight the capacitance dependence, four C1 and C2 pairs, denoted by α, β, γ and δ in Fig. 3a, were sampled and for each pair a “membrane potential,” i.e., V2, the profile with respect to time was evaluated for the given input current Iin. The results plotted are shown in Fig. 3b and 3c. In case of δ, the charging period of capacitor C2 is longer than the discharging period of capacitor C1 and thus switch S1 recovers its off-state before a transition of switch S2 into the on-state. Therefore, no spiking is observed (see Fig. 3c). To closely look at the evolution of V1, V2, RS1 (R of S1) and RS2 (R of S2) for case β, their time-dependent behaviors are zoomed in in Fig. 3e and 3f.

Figure 3
figure 3

Operational windows of capacitance.

(a) Window of C1 and C2 (grey zone) for successful spiking in due course, evaluated by time-dependent calculation. The inset shows the NLIF neuron circuit with parameters used in this calculation. Four combinations of C1 and C2, α (7 nF, 2 nF), β (3 nF, 2 nF), γ (3 nF, 4 nF) and δ (3 nF, 4 nF), are sampled and voltage-time behaviors of NLIF neurons with the capacitance combinations are plotted in (b,c). Input current Iin is shown in (d). For case β, the evolution of V1, V2, RS1 and RS2 is zoomed in in (e,f) to identify the self-consistent relation between them. The black and the red dashed line denote thresholds for on- and off-switching, respectively.

Full size image

The spiking dynamics is described by the membrane potential V2 and the auxiliary variable V1 as follows:

and

The NLIF neuron model is similar to the LIF neuron model regarding such that capacitor C2 integrates potential and fires a spike when the threshold for the on-switching of S2 is reached. However, a difference lies in the auxiliary variable V1 in Eqs. (1) and (2). Given these two variables, the spiking dynamics can be mapped onto a V2-V1 phase-plane, which is analogous to two-dimensional Hodgkin-Huxley neuron models such as FitzHugh-Nagumo model33,34. Eqs. (1) and (2) express V1- and V2-nullclines as the following linear equations:

and

respectively. RS1 and RS2 are history- and voltage-dependent and thus so are these nullclines. When RS1 = Roff and RS1 = Ron, the V1-nullcline is given by

and

respectively. Likewise, the V2-nullcline is expressed as

and

for case of RS2 = Roff and RS2 = Ron, respectively.

When Iin = 0, V1 and V2 stay at a stable fixed point (V2 = 0.044, V1 = –0.042) that is indicated in Fig. 4a. The parameters used in the phase analysis are listed in Table 2, which correspond to case β shown in Fig. 3. Upon the application of a constant current, this V1-nullcline in Eq. (5) shifts upwards by so that the fixed point moves far in the above-threshold region as shown in Fig. 4b. Consequently, the (V2, V1) trajectory moves towards the fixed point in due course as seen in Fig. 4b until the on-switching condition for S1, V1 ≥ |Von| + Vdc1 = 0.1 V, is encountered. The other V1-nullcline in Eq. (6) emerges at this moment, leading the trajectory to a new fixed point – again outside the sub-threshold region (see Fig. 4c). On its way, the off-switching condition for S1, V1 ≤ |Voff | + Vdc1 = −0.4 V, is met; the initial V1-nullcline (Eq. (5)) is thus recovered, changing the direction again (see Fig. 4d). The path undergoes another change when it reaches the on-switching condition for S2, V2 ≤ − |Von| + Vdc2 = −0.1 V, as a consequence of emergence of the above-threshold V2-nullcline given by Eq. (8) (Fig. 4e). Encountering the off-switching condition for S2, V2 ≥ − |Voff | + Vdc2 = 0.4 V, recovers the sub-threshold V2-nullcline (Eq. (7)), so that the path heads to a new fixed point as shown in Fig. 4f. The subsequent spiking dynamics follows the limit cycle that is indicated using a grey line in Fig. 4f. The spiking dynamics of the NLIF neuron differs from that of the Hodgkin-Huxley neuron mainly in the fact that a stable fixed point varies upon V1 and V2 on the phase-plane, due to the V1- and V2-nullclines conditional on V1 and V2.

Table 2 Parameters used in the neuronal response function and tuning function simulation.
Full size table
Figure 4
figure 4

Spiking dynamics on two-dimensional phase-plane.

(a) V1- and V2-nullcline and a stable fixed point (grey circle) when Iin = 0. The white area denotes the sub-threshold region. (b–f) Changes in the V1- and V2-nullcline upon threshold switching of S1 and S2 and the consequent trajectory of V1 and V2 on the phase-plane. The grey cycle in (f) shows the corresponding limit cycle. The black and red dashed lines mean thresholds for on- and off-switching, respectively.

Full size image

Notably, the limit cycle is confined in the area (−|Von| + Vdc2 ≤ V2 ≤ −|Voff | + Vdc2 and |Voff | + Vdc1 ≤ V1 ≤ |Von| + Vdc1) on the phase-plane as seen in Fig. 4. That is, the spike’s height and the level of the following undershoot are determined by Vdc1, Vdc2, Von and Voff, so that they are important parameters in spike’s shape design.

Neuronal selectivity of the NLIF neuron

The circuit parameters of the NLIF neuron are required to be further optimized by taking into account the neuronal selectivity of the NLIF neuron for stimulation. As an information encoder, the NLIF neuron should be able to represent “distinguishable” responses to different stimuli. Neuronal responses are typically parameterized by the spiking rate or the spike number – also known as activity – in a given time period. Regarding the neuronal selectivity, the NLIF neuron needs to vary its firing rate upon input current Iin. In particular, the firing rate and the input current are expected to be in a one-to-one correspondence relationship. Otherwise, one can hardly estimate the stimulus by counting the number of spikes, implying difficulty in “decoding” neuronal information. This difficulty in decoding consequently reduces the amount of information conveyed by the neuron35. In general, a neuronal encoding process is described by a = G[Iin(s)], where a and s denote activity and stimulus, respectively. In this study, we define neuronal “activity” denoting the number of spikes in a time period of 30 ms. The function G in a biological neuron is nonlinear and exhibits a threshold value for activation, i.e., firing, of the neuron. This function is often referred to as the neuronal response function in which the activity is determined by input current Iin.

For cases of aforementioned α, β and γ, the neuronal response functions were simulated in the input current Iin range (0–1.0 μA) and they are plotted in Fig. 5a. The neuronal response functions tend to increase monotonically with input current Iin as long as the current is larger than a threshold of approximately 0.25 μA. This threshold results from a threshold voltage for the on-switching of switches S1 and S2 (Von). All these functions appear to fulfill the aforementioned requirements for successful neuronal encoding. Nevertheless, a higher da/dIin value is favorable considering the fact that it reduces the uncertainty in discrimination when “noisy” neuronal responses are decoded. This issue will be revisited later when dealing with the noisy behavior of the NLIF neuron. Thus, case β appears to be most favorable and further discussion in this study will be narrowed down on this particular case. The corresponding parameters are listed in Table 2.

Figure 5
figure 5

Tuning function of ideal NLIF neuron.

(a) Neuronal response functions of the NLIF neuron with the three combinations of C1 and C2 (α, β and γ). (b) A Gaussian distribution of input current Iin, centered at a preferred orientation of 0°. (c) An ideal tuning curve of the NLIF neuron corresponding to the β case with a preferred orientation of 0°.

Full size image

In biological neurons, the input current Iin is understood to be determined by stimulus s, that is, Iin is a function of stimulus s. Each individual neuron has a preferred stimulus sp, at which Iin injected into a given neuron becomes the maximum. Note that this corresponds to the case of controlled neurophysiology experiments. The neuronal in-vivo function is, however, by and large triggered by an incident spike train(s) that provides a time-varying, rather than constant, synaptic current1. For simplicity, only one-dimensional stimuli are of concern in this study. For instance, stimulus s can be a one-dimensional visual stimulus such as the orientation angle of a light bar for the primary visual cortex36,37 and a wind direction for the cricket cercal system38. For convenience, the orientation of a light bar is considered as a one-dimensional stimulus in this study. It is assumed that the input current is described by a Gaussian function whose maximum is placed at preferred stimulus sp as follows: , where Iinmax and σs are the maximum Iin and the standard deviation, respectively. Fig. 5b shows the assumed Iin distribution with respect to stimulus, where Iinmax, sp and σs are 1 μA, 0° and 30°, respectively. Entering this Iin(s) function into the neuronal response function G[Iin] for β eventually gives the tuning curve shown in Fig. 5c. This bell-shaped tuning curve appears consistent with that obtained for typical biological neurons. The response of the NLIF neuron shows its maximum at an orientation of 0°, corresponding to its preferred orientation and tails around the preferred orientation. That is, stimuli within an orientation range of approximately −40° to 40° are able to activate the NLIF neuron although they are not exactly coincident with the preferred orientation. As a matter of fact, this imperfect-looking tuning curve enables a population of neurons with the limited number of preferred orientations to encode continuous, i.e., analog, information31. If neurons represented delta-function-like tuning curves, then an infinite number of such neurons would be required for encoding analog orientation information.

Noisy NLIF neuron

The tuning curve shown in Fig. 5c is of a perfectly working NLIF neuron. The orientation information encoded in the neuron can be decoded without uncertainty. Now, an arising question is “how large is the impact of imperfect behavior of the NLIF neuron on neuronal encoding and decoding?” Imperfect behavior is most likely caused by the variability of switching parameters of switches S1 and S2 (Ron, Roff, Von and Voff). For actual experimental NLIF neurons, variations in such parameters cannot be avoided. Thus, an attempt to determine the quantitative uncertainty in processing neuronal information, caused by such variations, was made by evaluating the neuronal encoding and decoding processes with varying switching parameters. Firing each spike in a spike train involves off → on → off switching of each S1 and S2 in a consecutive order. Given that the switching parameters in an experimental switch, in general, varies on each switching cycle24,26, it is rather natural to assign different switching parameters to each switch immediately after each switching event. That is, repeating a random update on the parameters lasts throughout the entire spiking period. In this regard, the switching-event-driven randomness leads to time-varying variability and thus “noise” rather than heterogeneity39. Such variations change the inter-spike interval (ISI) while a constant Iin is applied for a given time period. Spiking behavior involving the variation is shown in Fig. 6. For this simulation, Ron and Roff of switches S1 and S2 were randomly sampled using Gaussian PDFs – centered at 50 kohm and 1 Mohm, respectively, with various deviations. After each of on- and off-switching events, a new resistance was assigned to the switches. The switching-event-driven update therefore lets Ron and Roff fluctuate in time as shown in Fig. 6b and d, implying a noise.

Figure 6
figure 6

Poisson-like noise of NLIF neuron.

(a) Noisy response of the NLIF neuron with 10 percent resistance deviation and (b) the corresponding fluctuation of resistance of TSs in time. (c) Another set of noisy response and (d) resistance fluctuation under the same condition. The input current for both cases is plotted in (e). (f) Spiking dynamics in (a), mapped onto the phase-plane for the dynamics. (g) Variance of activity with respect to mean activity for the four different resistance deviations (5, 10, 20 and 30 percent). (h) ISI distribution for the case of 10 percent of resistance deviation at four different Iin values (1.00, 0.92, 0.74 and 0.49 μA). (i) Autocorrelation of spikes in a train at 0.5 μA input current and 10 percent of resistance deviation.

Full size image

It should be noted that the nullclines, Eqs. (5), (6), (7), (8), are determined by Ron and Roff of S1 and S2 and thus their variation essentially alters the nullclines and the corresponding fixed point. The random update, therefore, alters the trajectory of (V2, V1) on the phase-plane. The spiking dynamics on the phase-plane and in the corresponding time-domain is shown in Fig. 6f and 6a, respectively. In this regard, the noise of the NLIF neuron is distinguished from other noise models for LIF models, e.g. diffusive noise given white noise and/or noisy synaptic current40 and for conductance-based model such as Hodgkin-Huxley neuron41,42.

The observed noise characteristics were quantitatively analyzed by examining a relationship between the mean and the variance of the spike number (activity) for a time period of 30 ms (see Fig. 6g). This relationship generally explains a type of the present noise. For instance, in some biological neurons, this relationship is often given by and B≈1, where and n denote the variance and the mean of the activity, respectively43. This type of noise is referred to as the Poisson noise because the spike generation takes after the Poisson process1. The Poisson noise results in a straight line whose slope is unity as indicated with the dashed line in Fig. 6g. The relationship in Fig. 6g approximately features linearity – similar to the Poisson noise – albeit steeper than unity (ca. 1.3). The NLIF neuron therefore exhibits a Poisson-like, rather than perfect Poisson, noise and the variance is by and large larger than the Poisson noise at given mean activities. Interestingly, for the 5 percent deviation case, the variance is much smaller than that of the Poisson neuron at mean activities of approximately 30. This is attributed to the activity limit by the capacitors’ charging and discharging times that restrict the integration time for spiking (see Figure S1 in Supplementary Information).

The stochastic characteristics of this seemingly Poisson-like noise were further confirmed by analyzing the distribution of ISIs and the autocorrelation of the spikes in a given spike train; the results are shown in Fig. 6h and i, respectively. In Fig. 6h, the distribution is better fitted to a Gamma, rather than exponential, function regarding the effective refractory time caused by the finite recharging time of mainly capacitor C11. The evaluated autocorrelation data in Fig. 6i represent typical delta-function-like distribution, suggesting no correlation between the spikes. These noise analyses, therefore, identify Poisson-like noise characteristics of the observed noise.

To achieve successful spiking, the standard deviations of the switching parameters should be confined within particular ranges. Mostly, failure of spiking takes place when switch S2 becomes stuck to its on-state and high membrane potential (V2) is maintained. Typical examples of successful spiking and failure cases are seen in Fig. 7a and c, respectively. Insomuch as switch S2 keeps its on-state in case of failure, the membrane potential remains high, so that no further switching of switch S1 occurs. The current pulse duration was set to 30 ms. The number of successful spiking events was evaluated with separately varying the standard deviation of each switching parameter to determine the tolerance limit of each parameter (see Fig. 7e-h). We also varied input current Iin (0.4, 0.6, 0.8 and 1.0 μA) in order to identify its effect on success in spiking. It turned out that the tolerance limit for Ron and Roff reaches up to approximately 30 percent, whereas the limit for Voff is less than 20 percent as shown in Fig. 7e–h. It should be noted that the current pulse duration affects the probability of occurrence of an “Ron-stuck” event. This happens because the longer the pulse lasts, the more spikes are likely to be emitted and the more likely that after one of these spikes switching parameters are set to a configuration that does not support further firing.

Figure 7
figure 7

Failure of spiking.

(a) Successful spiking example and (b) the corresponding variation of resistance of TSs. (c) Evolution of membrane potential in time in case of failure and (d) the corresponding change of resistance of TSs. This failure arises from switch S2 being stuck to its on-state. The number of successful spiking events on 100 trials was evaluated at given standard deviation of each switching parameter while the other parameters are fixed: (e) Roff, (f) Von, (g) Ron and (h) Voff. The evaluation was done at four different input currents (0.4, 0.6, 0.8 and 1.0 μA). The circuit parameters in use, encompassing the mean values of Roff, Von, Ron and Voff, are listed in Table 2.

Full size image

The switching parameters were chosen in a random manner by employing a Gaussian probability density function (PDF) with particular standard deviations. The means of these distributions were placed at the values used in the calculation of the perfect tuning curve in Fig. 5c (Ron: 50 kohm, Roff: 1 Mohm, Von: 1 V and Voff: 0.5 V, the other parameters are shown in Table 2). The encoding process of a noisy individual NLIF neuron was evaluated by calculating its tuning curve based on statistics. Ron and Roff simultaneously varied at different standard deviations and the mean activity at each orientation was obtained out of 100 trials. Given the very limited tolerance for Voff variation as shown in Fig. 7h, no variation in Voff was taken into account. Variation in Von was also ruled out in light of its negligible effect on successful operation probability (see Fig. 7f). The calculated tuning curves for four different deviations (5, 10, 20 and 30 percent) are shown in Fig. 8. The maximum activity tends to decrease with increasing standard deviation. This rapid decrease in activity results from the fact that higher deviation renders switching parameters more likely to settle into a configuration that does not support further spiking. Unexpected spiking also takes place, in particular, at orientations outside the active range (−40° - 40°). Unlike the ideal tuning curve in Fig. 5c, a one-to-one correspondence relationship between orientation and activity is no longer satisfied. Therefore, a significant difficulty in decoding the neuronal information arises, consequently reducing the amount of information conveyed by the noisy neuron35.

Figure 8
figure 8

Variability effect on tuning function.

Poisson-like-noise-including tuning curves of the NLIF neuron allowing 5, 10, 20 and 30 percent of resistance deviation.

Full size image

Representation of a population of NLIF neurons

The noise in the individual NLIF neuron seems to be an obstacle to appropriate neuronal information processing because of the difficulty in decoding caused by the noise. Fortunately, neuronal information processing in the brain does not strongly rely on individual neurons; instead, the task is in general performed by a population of individual neurons2,31,35. Nevertheless, the neuronal noise can still contaminate the population response. Some types of correlations between neurons in a population are known to reduce errors to some extent35, but this does not seem to be the general case. A possible answer to the question “how do brains as groups of unreliable (noisy) neurons work reliably?” is that populations of neurons may encode and decode “probability distributions” rather than particular values44,45,46. In other words, encoding and decoding are viewed as processes retuning probability distributions over all possible values: response and stimulus distributions for encoding and decoding, respectively. Especially, decoding is most likely based on a statistical inference process, in particular, Bayesian inference32,45,47,48. In fact, some psychophysical evidence for Bayesian inference have been found in, for instance, contrast-depending velocity perception45,49. Given the role of the NLIF neuron in either hardware-based or in silico systems, it is then an important task to examine the NLIF neuron as a Bayesian decoder, quantitatively evaluating probability distributions over the orientation at given degrees of variability of the switching parameters.

According to the Bayes’ rule, a posterior PDF P[s|r] is given by the product of the likelihood function P[r|s] and the prior PDF P[s]:

where s and r denote stimulus and response, respectively. The notation P[B|A] means the conditional probability of event B, given event A; the likelihood function P[r|s] denotes the probability of observing response r, given stimulus s. This function describes the variability of the response to a particular stimulus. Likewise, the posterior PDF P[s|r] means the probability of stimulus s, given observation of response r. Insomuch as no condition is imposed on the prior and the response PDF, P[s] and P[r] are constant in ranges of stimulus s and response r, respectively. Thus, the posterior can be evaluated if the likelihood is known; the maximum of the posterior PDF corresponds to the most probable stimulus s estimated from the response observations. That is, the observed neuronal responses can be decoded in terms of probability. Note that P[s|r]/P[r] denotes a normalized likelihood PDF.

Unfortunately, the NLIF neuron representing a Poisson-like noise does not allow an analytical description of its likelihood function unlike Poisson neurons1. The only way to obtain the likelihood is collecting the responses of a population of NLIF neurons statistically, given various stimuli acting on it. Statistics were made on 20 NLIF neurons of 20 different preferred stimuli sp that are homogeneously distributed in the orientation range −180° to 180°. Now, the response r is a vector quantity of 20 components: . The tuning curves of these 20 neurons on the assumption of no noise are shown in Figure S2 in Supplementary Information. No correlations between neighboring neurons are assumed and the firing event on each neuron is regarded to be independent, allowing the following simple calculation:

The likelihood function was acquired by repeating spike number evaluation over 1000 times at each stimulus s and 300 stimuli were sampled between −180° and 180°. Given Eqs. (9) and (10) and the constant PDFs P[s] and P[r], the posterior PDF satisfies the condition . Thus, we can evaluate the probability of population representation of a particular pattern when subject to a given stimulus s. We snapshotted patterns of the population of NLIF neurons with resistance deviations of 5, 10, 20 and 30 percent at a stimulus of 0° and the patterns are plotted in Fig. 9a, b, c and d, respectively. The different preferred orientations of the population let a few neurons preferring stimuli in the vicinity of 0° be activated despite the noise complicating the patterns.

Figure 9
figure 9

Bayesian decoding of population representation of NLIF neurons.

Snapshotted activity patterns of a population including 20 NLIF neurons for (a) 5, (b) 10, (c) 20, (d) 30 percent resistance deviation cases at a stimulus of 0°. The results of the Bayesian decoding, i.e., posterior PDFs, for the patterns are shown in (e), (f), (g) and (h), respectively. The acquired posterior PDFs are compared with those of a population of 20 Poisson neurons (dashed lines). The standard deviations σ of the posterior PDFs of the Poisson-like NLIF neurons for the different resistance deviations are shown in (i) in comparison with those of the Poisson neurons.

Full size image

Finally, the aforementioned Bayesian decoding was done for the patterns, leading the posterior PDFs shown in Fig. 9e, f, g and h, respectively. The posterior PDFs are more or less noisy showing data scattering; the larger the deviation, the larger the data scattering. This data scattering is also a matter of population size, i.e., the more neurons in the population, the less decoding error. Given a large increase in calculation time with increasing the number of neurons in the population, we placed 20 neurons in the population; however, the Bayesian decoding of larger population sizes definitely enables correct estimation. The calculated posterior PDFs were fitted using Gaussian PDFs so as to evaluate the center and standard deviation of each posterior PDF. As can be seen in Fig. 9, the center of each PDF is found to be placed around 0°. This revealed that this Bayesian decoder most likely give a correct answer and a correct inference will be made if made by means of the Bayes’ rule despite the present Poisson-like noise. Nevertheless, note that statistics cannot be free from error in any cases so that the Bayesian decoding can give a wrong answer at times. Besides, it turns out that the standard deviation in the decoding becomes larger as increasing the variability of Ron and Roff of the TSs (see Fig. 9i).

The Bayesian decoding results were compared with the case of populations of Poisson neurons. The likelihood PDF as well as the posterior PDF of a population of independent Poisson neurons is given by a closed-form expression; an increase in the number of Poisson neurons in the population leads to a posterior PDF of a Gaussian form32. At a given stimulus, the likelihood PDF of each independent Poisson neuron was analytically calculated with similar activity as that of the NLIF neuron at the same stimulus. As a result, the posterior PDF of a population of Poisson neurons with the four resistance deviations could be obtained from the population response patterns shown in Fig. 9a, b, c and d; the calculated PDFs are plotted using dashed lines in Fig. 9e, f, g and h, respectively. In fact, the 20 Poisson neurons already provide a Gaussian PDF as shown in the figures. In comparison with the population of Poisson neurons, it is noticed that the Poisson-like NLIF neurons represent smaller maxima and larger deviations than the Poisson neurons under the same condition except the case of 5 percent deviation of TSs’ resistance (see Fig. 9a). A difference in the standard deviation of the posterior PDF between the NLIF neurons and the Poisson neurons is observed in Fig. 9i. The larger uncertainty deviation of the Bayesian decoding for the NLIF neurons arises from the larger deviation of activity of the NLIF neuron than that of a Poisson neuron as shown in Fig. 6g. Likewise, the larger maximum of posterior PDF of the NLIF neurons in Fig. 9e than the Poisson neurons is understood in terms of the smaller variance of activity at 5 percent deviation of TSs’ resistance at the high mean activities, shown in Fig. 6g. The smaller variance is attributed to the activity limit by charging and discharging of the capacitors.

Discussion

The NLIF neuron studied in this work can serve as a prototypical in silico neuron model exhibiting a Poisson-like noise. The circuitry is simple and perhaps easy enough to be implemented in large-scale ANNs. In particular, as a result of this study, it is understood that the variability of the TSs’ resistance leads to such a Poisson-like noise that the noise behavior of this prototypical in silico neuron needs to be under control and appropriately designed to meet the noise behavior required for ANNs built for specific purposes.

Nevertheless, when it comes to hardware realization of such NLIF neurons, there are several practical obstacles that should be overcome to realize the goal. What is of significant importance in the Poisson-like NLIF neuronal behavior is the minimum variability of Voff of the TS in the NLIF neuron. As discussed earlier, the tolerance limit of Voff is merely a few percent unlike that of the other switching parameters, i.e., Ron, Roff and Von. Thus, “reliability” of this unreliable neuron requires meeting this stringent requirement for ensuring reliable operation. Apart from this restriction, other requirements discussed earlier may be satisfied by appropriate choices of TS materials, systems and their design. Another important issue that potentially hinders practical use of this type of neuron is the long-term reliability of switches S1 and S2, which are subject to the relatively high dc-voltage stress (Vdc1 and Vdc2). The dc voltages allow active operation of the NLIF neuron, working as effective power suppliers. Regarding the limit cycle confined in the area – (−|Von|+Vdc2 ≤ V2 ≤ −|Voff | + Vdc2 and |Voff | + Vdc1 ≤ V1 ≤ |Von| + Vdc1) on the phase-plane (see Fig. 4) – dc voltages close to, but smaller than, Von need to be applied to switches S1 and S2 and thus the consequent electrical stress most likely affects the switches adversely. Eventually, it most likely leads to dielectric breakdown when a dielectric layer is in use as a TS material. This issue is also directly related to a high power consumption problem. The constant application of dc voltages during the lifetime of the NLIF neuron gives rise to severe power consumption, which is definitely against one of the inherent advantages of neuromorphic systems over standard digital systems, i.e., low power consumption. Therefore, addressing these significant problems properly accelerates practical use of such NLIF neurons in hardware-based neuromorphic systems.

The most crucial conclusion drawn from this study is that the potential variability of behavior of the TS is allowed up to a certain level as long as the Bayesian decoder is able to discriminate the encoded information correctly. In addition, the uncertainty, i.e., standard deviation, of the posterior PDF shrinks when introducing a larger number of NLIF neurons in the population. In general, the statistical accuracy of a survey increases with the number of samples. Thus, the uncertainty of posterior and likelihood of individual NLIF neurons is compensated by the increase in accuracy. An increase in the number of neurons in the population, therefore, tolerates a larger variability of switching parameters of the TSs. Nevertheless, confining the variability within a tolerance range is still of significant importance, especially confining that of Von.

Methods

Neuronal response function calculation

Eqs. (1) and (2) were solved by employing the Crank-Nicolson method. The equations are described by the following discrete forms:

and

The superscripts of V1 and V2 denote the ith node in the time domain. Eqs. (11) and (12) were numerically solved and RS1 and RS2 were updated when the evaluated potential reaches the threshold switching conditions.

Additional Information

How to cite this article: Lim, H. et al. Reliability of neuronal information conveyed by unreliable neuristor-based leaky integrate-and-fire neurons: a model study. Sci. Rep. 5, 9776; doi: 10.1038/srep09776 (2015).