A molecular neuromorphic network device consisting of single-walled carbon nanotubes complexed with polyoxometalate

In contrast to AI hardware, neuromorphic hardware is based on neuroscience, wherein constructing both spiking neurons and their dense and complex networks is essential to obtain intelligent abilities. However, the integration density of present neuromorphic devices is much less than that of human brains. In this report, we present molecular neuromorphic devices, composed of a dynamic and extremely dense network of single-walled carbon nanotubes (SWNTs) complexed with polyoxometalate (POM). We show experimentally that the SWNT/POM network generates spontaneous spikes and noise. We propose electron-cascading models of the network consisting of heterogeneous molecular junctions that yields results in good agreement with the experimental results. Rudimentary learning ability of the network is illustrated by introducing reservoir computing, which utilises spiking dynamics and a certain degree of network complexity. These results indicate the possibility that complex functional networks can be constructed using molecular devices, and contribute to the development of neuromorphic devices.


Supplementary Note 1: Impulse generation in a POM/SWNT complex device with a microscale channel
We recently confirmed that a microscale POM/SWNT complex device generated current impulses. As shown in Supplementary Figs. 1a and 1b, SWNTs served as bridges between Au electrodes with a 1-µm gap length, where the SWNTs were aligned by using the electrophoresis method. POMs were dosed into the device after fabrication of the SWNT device. The fabrication details are described in a reference paper [S1] including a study of stochastic-resonance device fabrication by using spontaneous noise generation in a POM/SWNT complex system.
Supplementary Figure 1c shows time domain current for 0.5 s with a DC bias voltage of 0.5 V across the electrodes. In our previous study for noise generation, we found that even a device whose POM density was higher than that appropriate for noise generation, generated current impulses. We consider that the current impulses to originate from the POMs inserted between the SWNTs and Au electrodes shown in Supplementary Fig. 1d. Based on the previous study, we know that POMs adsorbed onto the SWNTs generate noise rather than impulses. The conductance change of the observed impulses was three orders of magnitude greater than that of the noise. Hence, conductance switching at the POM junction should be the origin of the impulses in these small size devices.
The current impulses generated by the small-sized POM/SWNT device (having the electrodes with 1-µm gap length shown in Supplementary Fig. 1) were analogous to those generated by a large-sized POM/SWNT device (having the electrodes with 1-mm gap length shown in Fig. 2b), including the spike frequencies and amplitudes (current scale of microamperes). The electric field has almost the same order of magnitude in both devices:150 V/ 1mm = 1.5 × 10 + (V/m) for the large-sized device (generating impulses with 150 V bias voltage, as in Fig. 3b), and 0.5 V/1µm = 5 × 10 + (V/m) for the small-sized device (also generating impulses with 0.5 V bias, as in Supplementary  Fig. 1c).

Supplementary Note 2: Rudimentary reservoir computing with SWNT/POM network device model
We here demonstrate rudimentary reservoir computing on the POM/SWNT device model. Supplementary Figure 2a depicts the typical structure of a reservoir system with external feedback. A recurrent generator network (blue spheres and directed arrows in Supplementary Fig. 2a) with firing rates r generates complex nonlinear dynamics, and drives a linear readout unit Σ with output z through weights w (red) that are modified during learning. The output z is fed back to the reservoir, to maintain the complex nonlinear dynamics in the reservoir. Note that only the connections shown in red are subject to modification during learning.
Our 2D POM/SWNT network model is used as a reservoir, as shown in Supplementary Fig. 2b. The external feedback is implicitly introduced into this reservoir system, where its feedback is created by connecting the external bias voltage source B across the electrodes, which results in maintaining and generating noisy or spiking dynamics, as exhibited in Fig. 4f. Among all of the POM particles (small blue spheres in Supplementary Fig. 2b), particles are randomly selected, and the charges are read out by virtual probes (red lines in Supplementary Fig. 2b). The charges read out at time , 1 3 ( ), ⋯ , 8 ( )9 ( ≡ r( ) ) drive the readout unit Σ through weights 1 3 ( ), ⋯ , 8 ( )9 (≡ w( )), and the reservoir output is given by ( ) = w = ( )r( ).
During learning, a temporal signal ( ) is applied to the system as a supervisor. By using the FORCE learning algorithm[S 2] , the weight is updated to attain ( ) ≈ ( ).
Extensive numerical simulations were conducted to reveal the fundamental properties of our reservoir system. Here, the NARMA10 sequence[S 3] , one of the most widely used benchmarks in reservoir computing, was used as the supervisor. Supplementary Figure 2c presents one of the results when = 100 with 5,500 POM particles and 4,500 defects on a 100 × 100 rectangular grid ( A = 45%), where the readout output ( ) (purple) is plotted, while the supervisor (NARMA10 sequence; green) is superimposed, exhibiting that the reservoir system could memorize a part of the NARMA10 temporal sequences essentially through the FORCE learning, by utilizing complex dynamics generated by the POM/SWNT network.
The quality of memory function is determined by the replicability of the given data as well as the data length. To reveal the quality, we evaluated the normalized root-mean-square deviation (NRMSD) between the supervisor ( ) and generated output ( ) after learning. Supplementary Figure 2d shows NRMSD versus the signal length for two different values (50 and 100). The NRMSD increases gradually as the signal length increases, whereas larger decreases the NRMSD, indicating that the quality of memory function is improved by increasing .
In the simulations described above, particles were randomly selected from all of the particles in the device. Supplementary Figure 2e shows yet another method of the random particle selection, where particles are randomly selected from areas beside the source electrode (upper, labelled by 'source side') or the drain electrode (bottom, 'drain side'). Such topological deflection upon random particle selection may influence the performance of the reservoir system due to the network complexity difference, because the dynamics of the particles sampled near the source electrode are generated by local interactions in the source-side (shallow) network, whereas those of the particles sampled nearby the drain electrode are generated by deep interactions among particles from the source to the drain side. Supplementary Figure 2f compares the NRMSDs in the cases of source-and drain-side particle sampling as functions of the normalized sampling area (≡ ) when = 100. Both of the NRMSDs monotonically decreased as increases. A significant difference between the NRMSDs is observed when < 0.2, whereas the values are almost the same when > 0.2, which indicates that the source-side sampling is much better than the drain-side sampling when the placement of readout wires is limited around the source or drain electrode ( < 0.2); however, to minimize the NRMSD, the particles must be sampled from particles located anywhere in the device ( = 1).
Our POM/SWNT network device imitates the spiking behaviours of complex neural networks; however, does not include any synaptic function. One of the difficult challenges is to include synaptic functions in POM/SWNT networks for effective neuromorphic demonstrations. One possible method is to complex memristive molecules, such as BPDN molecules [S 4] , in the POM/SWNT network so that POMs (as spiking neurons), memristive molecules (as synapses), or both, can be localized at the SWNT junctions; however, it might take a very long time to find computational and useful functions. On the other hand, in our reservoir framework, the reservoir itself (POM/SWNT networks as complex spiking networks) and the external synaptic readout wires are separated; hence, one can get one of the useful reservoir functions, i.e. temporal coding of complex time series, by controlling the synaptic weight through FORCE learning, while fully utilizing the complex dynamics of the POM/SWNT networks.
During learning, a temporal error between the output of the readout unit and supervisor signal ( ), The learning algorithm above is referred to as the FORCE learning algorithm[S 2] . In Supplementary Fig. 2c, the NARMA10 sequence[S 3] was used as the supervisor. The sequence was generated by where ( ) consists of scalar random numbers with a uniform distribution in the intervals [0,0.5]. The reservoir system was trained, i.e. w of the linear readout unit was modified by the FORCE learning by comparing ( ) and ( ), during = 4,920~5,000 steps, at which the array had already been charged, using the given parameter sets (the same parameter sets of the results shown in Fig. 4f). After learning, w was fixed, and the simulation was restarted using the same random seed used in the learning. Then, the reservoir output was observed with the fixed w.
In Supplementary Figs. 2d and 2f, the NRMSD between the supervisor ( ) and generated output ( ) after learning, was calculated by using where represents the data length. In Supplementary Fig. 2d, the NRMSD versus (50 and 100) and the signal length swept from 160 to 400 steps, where averaged NRMSD values over 50 trials for each value with different random seeds are plotted. Two different readout-particle selection methods, where particles were randomly selected from areas beside the source electrode (upper, labelled by 'source side') or the drain electrode (bottom, 'drain side'). (f) Comparison of NRMSDs for source-and drain-side particle sampling as a function of the normalized sampling area ( ).