A distributed nanocluster based multi-agent evolutionary network

As an important approach of distributed artificial intelligence, multi-agent system provides an efficient way to solve large-scale computational problems through high-parallelism processing with nonlinear interactions between the agents. However, the huge capacity and complex distribution of the individual agents make it difficult for efficient hardware construction. Here, we propose and demonstrate a multi-agent hardware system that deploys distributed Ag nanoclusters as physical agents and their electrochemical dissolution, growth and evolution dynamics under electric field for high-parallelism exploration of the solution space. The collaboration and competition between the Ag nanoclusters allow information to be effectively expressed and processed, which therefore replaces cumbrous exhaustive operations with self-organization of Ag physical network based on the positive feedback of information interaction, leading to significantly reduced computational complexity. The proposed multi-agent network can be scaled up with parallel and serial integration structures, and demonstrates efficient solution of graph and optimization problems. An artificial potential field with superimposed attractive/repulsive components and varied ion velocity is realized, showing gradient descent route planning with self-adaptive obstacle avoidance. This multi-agent network is expected to serve as a physics-empowered parallel computing hardware.


Fabrication of Ag nanocluster based multi-agent evolutionary network system
The device fabrication started from silicon wafers with 300 nm thermally grown silicon dioxide.
Water molecules absorbed on SiO2 surface were expelled by heating wafer at 170 °C for 10 min. Afterwards, PMMA as the positive resist was covered on the wafer surface using spin coating (3000 r/min, 1 min), followed by a series of processes consisting of electron-beam lithography, development (IPA/MIBK with a volume ratio of 3:1), electron beam evaporation and lift-off to form the pattern of metal electrodes. The metal materials Ti (1 nm)/Au (40nm) were used for the fabrication of metal electrodes in this study. To get the polymer electrolyte, polyethylene oxide (PEO, 100,000 g/mol) was dissolved in acetonitrile by 5wt.‰. The homogeneous solution was then dropped above the surface of device and the solvent was removed using high-speed spin coating to form the solid polymer electrolyte with selfassembled lamellar structures, providing long-range Ag + ion transport paths (Supplementary Ref. S1). Subsequently, discrete Ag nanoclusters were incorporated into the solid polymer electrolyte by electron beam evaporation with a thickness of 3 nm below filming condition.
The fabrication process of device with two terminals is schematically shown in Supplementary heating the substrate at 170 °C for 10 min. b, PMMA (950,000 g/mol) was spin-coated onto the SiO2 surface with a spin rate of 3000 r/min for 1 min. The sample was subsequently heated at 170 °C for 3 min to harden the PMMA film. Designed pattern was formed on the PMMA film with electron-beam lithography. c, IPA/MIBK (volume ratio: 3:1) was used to develop the pattern for 1 min, followed by fixing the sample in IPA for 30 s. d, 1 nm Ti and 40 nm Au were successively deposited by electron beam evaporation and lift-off process was used to form the final electrodes. e, Dropping of the polymer electrolyte solution. f, Acetonitrile and water molecules were removed from the electrolyte by high-speed spin coating, and thus g, the solid polymer electrolyte with uniform thickness was formed. h, Electron beam evaporation was used to form Ag nanoclusters partly incorporating into the polymer electrolyte. i, Ag conductive filament is connected between the two terminals due to the self-organized evolution of Ag nanoclusters under applied voltage bias.

Supplementary Figure 2 | The micro morphology characterization of Ag nanoclusters
after device fabrication using SEM technique. Ag nanoclusters are partly incorporated into the solid PEO electrolyte material. Scale bar: 50 nm.

Characterization of the micro-morphology of devices by AFM
In this article, scanning electron microscope (SEM) was mainly used to characterize the microscopic morphology of devices in pursuit of high resolution. In fact, there are many other 4 feasible ways to achieve the same goals. Here, the surface morphology of sample was characterized by the peakforce tapping mode of atomic force microscope (AFM). Supplementary Fig. 3 gives some results of morphological characterization using AFM.

Supplementary Figure 3 | AFM characterization of the micro-topography of device
surface. SEM images of two-terminal device a, before and b, after electrical stimulation, which have been shown in Fig. 1c and 1d, respectively. Scale bar: 100 nm. c, AFM image of twoterminal device before memristive switching, corresponding to the SEM image in (a). Scale bar: 100 nm. d, AFM image of two-terminal device after applying the voltage bias V, corresponding to the SEM image in (b). The conductive filament shown in (d) is marked with dashed lines as guides for the eyes. Scale bar: 100 nm. The applied voltage V was 25 V.  Fig. 1(d). The constant voltage bias V = 25 V was applied between the two terminals of device and we can see a significant increase of current at t ≈ 12 s, indicating the conductive filament is connected between the terminals.

Monte Carlo simulation
In this work, the transport of Ag + ions on PEO surface under various electrodes and applied voltages was calculated by kinetic Monte Carlo simulation using Matlab. The simulation process is schematically illustrated in Supplementary Fig. 5. Firstly, the positions of electrodes were initialized according to experimental conditions, and the Ag clusters were randomly generated on the surface. Subsequently, the electrical field distribution was calculated by Kirchhoff's law. During the simulation, each lattice points represents a node and every node is connected with 8 neighbor nodes. When both of the neighbor lattice points were occupied by metal materials (Ag clusters or electrodes), the connected resistance would be set to a low value; otherwise, the connected resistance would be set to a high value.
The boundary condition is that when the edge is an electrode, the potential is the applied voltage, while the insulating boundary would be set to open state. Obviously, internal potentials could be calculated by node analysis, and the electric filed and current could be derived from potentials. Subsequently, the probability of Ag ionization and hopping to unoccupied lattice points could be calculated by Equation (S1) based on Ag distribution and electric fields: Here, p is the ionization-hopping possibility, f is the hoping attempt frequency, U is the hopping barrier between the lattice, nq is Ag + ion's charge, ΔV is the potential difference of the lattice, kB is Boltzmann's constant, T is the absolute temperature. According to the hopping probability, the Ag + ion transport could be calculated by Monte Carlo method, where a random number matrix can be generated and compared with the hopping possibility of 8 directions for each lattice so that the next state of Ag distribution can be determined. Finally, the potential would be calculated by the new Ag distribution, followed by simulating Ag + ion transport. These processes would be repeated until a conductive filament is formed between the electrodes.

Supplementary Figure 5 | Schematic of the kinetic Monte Carlo simulation process. a,
Flow diagram of the simulation process. b, The considered physical process and basic calculation method using the resistance network approach in the simulation.

The effect of compliance current on the morphology of conductive filaments
Voltage bias V was applied to the terminals in the two-terminal devices with ~200 nm gap distance under different conditions of compliance current (CC) to investigate the effect of compliance current on the conductive filaments. Supplementary Fig. 6 shows the SEM images of device after electrical stimulation and the corresponding time dependent current measurements (I-t) with different compliance currents (50 nA, 100 nA, 200 nA and 300 nA).
We can see that when the same voltage bias V was applied, larger compliance current led to stronger conductive filament and more obvious branching expansion effect. For one thing, the 7 conductive filament will be too weak to be characterized if the compliance current is too small.
For another thing, too large compliance current leads to severe Joule-heating effect which will bring some negative effects such as uncontrollable connection of conductive filaments and increased power consumption. Therefore, it is very important for us to select the appropriate compliance current during the electrical stimulation for obtaining the correct result. In this work, different values of compliance current are selected for different physical structures. For the basic modulation units and parallel structures, a compliance current between 10-50 nA is adopted in most cases. For the problems containing serial structures, larger compliance current of 0.3-10 μA is usually needed to provide larger driving force, due to the generally longer gap distance in serial structures. Under the same condition of applied voltage bias V (15 V), conductive filament is finally connected between the two terminals and larger compliance current led to stronger filament with more branches. Scale bar: 100 nm.

Pinched hysteresis loops and memristive effect by I-V measurements
To study the memristive switching characteristic of our devices, positive and negative voltage sweep were alternately applied to collect the electrical data. Supplementary Fig. 7 shows the forming and subsequent several I-V sweep curves of a two-terminal device with ~100 nm gap.
The repeatable I-V curves under 400 alternating sweep cycles are given in Supplementary Fig.   8 8, indicating the MAEN devices can be stably operated. It can be seen that higher threshold voltage (> 5 V) is needed to drive the initial electrochemical reaction and migration of Ag nanoclusters to form the conductive filament during the forming process. The device spontaneously relaxed back to the off state after removing the voltage bias due to the filament rupture facilitated by interfacial energy minimization with diffusion mechanism (Supplementary Ref. S2). The pinched hysteresis loops from the repeatable I-V sweep measurements confirm the memristive nature of our devices.
The MAEN device shows threshold switching characteristics due to the spontaneous dissolution of conductive filament driven by the interfacial energy minimization. Therefore, the device will relax back to the off state after removing the electrical stimulation. Although the continuous filament is broken into discrete clusters, they do not fully recover to their original positions. As a result, the threshold voltages in the subsequent switching processes are significantly reduced, and a new filament can be easily connected based on the previously incompletely ruptured filament. For example, when a small voltage bias (2 V) was applied onto a pristine MAEN device with 100 nm gap, there was no switching within 20 s ( Supplementary   Fig. 9a). Once a large voltage bias (10 V) was applied, a conductive filament can be formed between the two terminals, along with an obvious increase in current at t ≈ 2.5 s ( Supplementary   Fig. 9b). Although the device spontaneously returned to off state after removing the bias, the application of a small voltage bias of 2 V has switched the device to on state again within 1 s (Supplementary Fig. 9c), since the incompletely ruptured filament can be recovered by a smaller driving force. The device can maintain the on-state for a long time at this small voltage bias, i.e. >500 s in Supplementary Fig. 9c. The structure stability of conductive filament can be further tuned by decreasing the gap distance to limit interfacial energy driven surface diffusion. In addition, a smaller gap will be beneficial for the decrease of threshold voltage and solution time, thus saving the computational costs. corresponding electric field distributions (bottom panels). The conductive filament is finally connected along the path corresponding to the highest voltage bias.

Proof of the gap equivalence in the voltage modulation unit
In Fig. 1m, the voltage bias V representing smaller weight was applied between the terminals T1 and T3 while 1/2V representing larger weight was applied between the terminals T2 and T3.
Conductive filament is finally connected between the terminals T1 and T3, which corresponds to the selection of edge with smaller weight following the optimal solution principle. In order to prove the equivalence of gap in the voltage modulation unit for solving problems, we interchanged the potential applied to the terminals T1 and T2. Supplementary Fig. 12 shows that conductive filament establishes connection between the terminals T2 and T3 when the voltage bias V representing smaller weight is applied between the terminals T2 and T3. Combined with the result shown in Fig. 1m, we can conclude that all the gaps are initially equivalent in the voltage modulation unit, and the connection path of conductive filament is exclusively determined by the voltage bias.

Supplementary Figure 12 | Proof of the gap equivalence in the voltage modulation unit
by interchanging the potential applied to the terminals T1 and T2 of device structure in Fig. 1m. a, SEM image of device morphology after resistive switching. The distance of two gaps were ~200 nm. The voltage bias 1/2V was applied between the terminals T1 and T3 while the voltage bias V was applied between the terminals T2 and T3. Conductive filament is finally connected between the terminals T2 and T3. Scale bar: 100 nm. b, Time-dependent current 13 measurement from the terminals T1 (blue curve) and T2 (orange curve). The applied V was 30 V.

Hybrid distance-voltage modulation unit
Supplementary Fig. 13 gives the initial state of an as-fabricated device corresponding to the

Intrinsic stochastic nature of the conductive filament growth
In the hybrid distance-voltage modulation unit shown in Supplementary Fig. 13a, two solution results were observed in our experiments under the voltage bias scheme shown in Supplementary Fig. 15: the conductive filament is only formed between the terminals T4 and T5 ( Supplementary Fig. 15a, case 1); the conductive filament is both formed between the terminals T2 and T5, and between the terminals T4 and T5 ( Supplementary Fig. 15b, case 2). The different connectivity results were also observed in the integration structures of basic modulation units. Supplementary Fig. 16 demonstrates three connectivity patterns observed in the parallel structure composed of two identical voltage modulation units under a voltage bias scheme: the conductive filament is only formed between the terminals T2 and T3 ( Supplementary Fig. 16, case1); the conductive filament is only formed between the terminals T4 and T6 ( Supplementary Fig. 16, case2); the conductive filament is both formed between the terminals T2 and T3, and between the terminals T4 and T6 ( Supplementary Fig. 16, case3). The

Fabrication of the metal islands
Supplementary Fig. 17 shows the fabrication of metal islands. Same materials and lithography conditions were used to fabricate the metal islands and the metal electrodes in this article.
Therefore, the metal islands can be prepared in the same steps for the preparation of metal electrodes by following the basic process shown in Supplementary Fig. 1

Monte Carlo simulation of evolving process in the device with metal islands
In order to further prove the modulation effect of metal islands on the connectivity pattern of conductive filament, Monte Carlo simulation was carried out to study the physical process involved in the evolution of system. Supplementary Fig. 18 shows the evolution of Ag atom/cluster distributions (upper panels) and corresponding electric field distributions (bottom panels) in the device in Fig. 3f over time, obtained from a kinetic Monte Carlo simulation. We can clearly see that the metal islands between the terminals effectively enhance the electric field intensity between the metal islands with each other, and between the metal islands and the terminals, leading to the competitive conductive filament growth along the arrangement direction of upper two metal islands. The final connectivity pattern of conductive filament in the MC simulation is consistent with the experimental results in Fig. 3g.

The effect of terminal position on the connectivity pattern of conductive filament
In addition to the metal islands, the terminal position can also effectively regulate the electric field distribution to affect the final connectivity pattern of conductive filament, and thus the flexibility of graph structure mapping can be further improved. The completely different graph structure information can be expressed by simply regulating the terminal position of devices.
For the graph structure shown in Supplementary Fig. 21a, the problem to be solved is to find the shortest path between the nodes N1 and N2. This problem can be mapped into the device configuration shown in Supplementary Fig. 21b where the terminal T2 on the right side of device in Fig. 3f is moved to the upper side. The change of terminal position directly affects the electric field distribution, thus the connectivity pattern of conductive filament is accordingly modified. From the Supplementary Fig. 21c we can see that the conductive filament is connected between the terminals T1 (N1) and T2 (N2) by way of the metal island i2

The reusability of MAEN system
The spontaneous diffusion dynamics of conductive filament driven by interfacial energy minimization between the Ag nanoclusters and dielectrics (Supplementary Ref. S2 and S5) causes electrical disconnection between the electrode terminals after removing the external biasing, which provides substrate for reusing the same MAEN device for the solution of problems. Supplementary Fig. 22 shows a consecutive sequence of operations on the same MAEN system with 4 terminals and 3 metal islands. For each step, the conductive filament can establish connection along the optimal path between the corresponding terminals, and the spontaneous relaxation of particles after removing the biases allows the device to be reused and operate correctly in the next steps. It is worth mentioning that the previous connectivity patterns may have a certain impact on the subsequent solutions. In the future, material systems with large wetting contact angles, such as MgOx:Ag, SiOxNy:Ag and HfOx:Ag (Supplementary Ref. S2), may be considered. It is reported that the conductive filament may quickly shrink to original Ag nanoclusters driven by interfacial energy in these material systems when the external biasing is removed, so that the reusability of system can be further improved. The reconfigurability of MAEN system 22 A reconfigurable computing system is needed for solving different problems with the same structure. In fact, the MAEN system possesses potential for reconfigurability. Supplementary   Fig. 23 gives an example to show that different problems can be solved with the same device structure. We can see that under two different voltage bias schemes, Ag nanoclusters spontaneously align into a connective filament along the optimal path, leading to two different connectivity patterns. In the case of Supplementary Fig. 23a, V and 0 were applied to the neighboring terminals T1 and T4 respectively, while the other two terminals (i.e. T2 and T3) were biased at 1/2V. In this case, both distance modulation and voltage modulation are involved.
The voltage biases V and 1/2V were applied to the paths with equal length " ! ↔ $ ↔ " " and " $ ↔ # ↔ " ", constituting the voltage modulation mode. Since the applied voltage bias between T1 and T4 is higher, the conductive filament was formed along the path " ! ↔ $ ↔ " " where the electric field is highest, leading to a significant increase of current from the terminals T1 and T4. In another case of Supplementary Fig. 23b, V and 0 were applied to the diagonal terminals T1 and T3 respectively, while the other two terminals (i.e. T2 and T4) were assigned to 1/2V. Compared with the previous case, a different pattern connecting the terminals T1 and T3 by way of the metal islands i2 and i3 (i.e. " ! ↔ $ ↔ # ↔ $ ") was formed. In addition, the graph problem shown in Fig. 3e can also be solved by using the device structure shown in Supplementary Fig. 23, where the original two-terminal device structure (Fig. 3f) is included as a sub-graph. In future research works, we propose that the back gates can be further added to achieve gate modulation. Once the back gates are incorporated, the weight can be represented by the gate control signal, which provides another degree of freedom to effectively regulate cluster evolution and filament growth. By applying a driving bias signal while the control signals representing weight information are applied to the corresponding gate. The gate modulation scheme combines the advantages of distance modulation and voltage modulation schemes, simultaneously possessing high efficiency, high flexibility and high reconfigurability for solution of problems. Supplementary Fig. 24 gives an example of solving a 3×3 maze problem based on the gate modulation unit. Thanks to the same device structure independent of the specific maze, a system mapping the N×N maze is capable of solving arbitrary n×n maze problems (n ≤ N), suggesting high reconfigurability of the system.

The improvement of system generality with unified node shape
In order to demonstrate the potential of system generality, the sharp tip of electrode terminals was replaced by more rounded shape to share a similar geometry with the circular metal islands.
From Supplementary Fig. 25 one can see that the connectivity pattern of conductive filaments after changing the terminal shape ( Supplementary Fig. 25b) is exactly same as before ( Supplementary Fig. 25a) under the same voltage scheme. Two shortest paths were simultaneously founded relying on the self-organized evolution of Ag nanoclusters. The result indicates that the optimal route between each two nodes in the MAEN system may be computed 25 because they share similar geometry, providing more general and promising application scenarios.

Monte Carlo simulation of artificial potential field based on the MAEN
The physical process of artificial potential field evolution in Fig. 4c under the comprehensive control of electric field and ion mobility was further studied using kinetic Monte Carlo simulation. Supplementary Fig. 27 shows the evolution of Ag atom/cluster distributions over time, obtained from the kinetic Monte Carlo simulation. The experimentally observed connection path of conductive filament shown in Fig. 4d was well reproduced.

MAEN circuit modules for efficient writing/reading
In this work, the solution result represented by the connectivity pattern of the filament was mainly obtained by SEM observations, and the time-dependent current measurement was used as an auxiliary readout method, since the detailed connectivity pattern may not be completely reflected through the limited number of probes in the testing probe stations. To achieve efficient writing/reading when the problems to be solved contain multiple inputs/outputs, a dedicated circuit platform for electrical measurements can be developed for the MAEN system.
Supplementary Fig. 30 depicts a schematic diagram of the reading and writing periphery circuits. The module "CTRL" is the controller that sends control signals to decide corresponding operations, such as the terminals to be addressed and the voltage bias to be applied. The multiplexer "MUX" receives control signal from "CTRL" and selects the voltage bias that corresponds to the terminals. The signal converter "ADC & DAC" is used to input and read the analog electrical signal. We should point out that the SEM observation in this work is to demonstrate the correct solution of the problems using MAEN, and in the meantime 29 the gap distance and compliance current were still relatively large to ensure sufficiently long and thick filament(s) for clear observation. Once the SEM observation is replaced by reading and writing periphery, both the gap distance and compliance current can be further reduced, and these will also contribute to the reduction of the solution time and therefore further enhancing the computational efficiency.

Supplementary Figure 30 | Schematic diagram of circuit platform including writing and
reading periphery for MAEN. The module "CTRL" is the controller which sends control signals to decide corresponding operations, such as the terminals to be addressed and the voltage bias to be applied. The multiplexer "MUX" receives control signal from "CTRL" and selects the voltage bias that corresponds to the terminals. The signal converter "ADC & DAC" is used to input and read the analog electrical signal.

Discussion on the energy consumption of MAEN system
In this work, relatively large gap distance (hundreds of nanometers) and high compliance current (hundreds of nanoamps to several microamps) were adopted to ensure obvious filament morphology under SEM observations. A rough estimation by directly multiplying the compliance current, applied voltage and forming time leads to relatively high energy consumption from micro to milli Joule. However, it is worth mentioning that during the forming process, the current is always at a low level (from femtoamp to picoamp) before the conductive filament establishes a connection between the terminals, and only reaches the 30 compliance current after filament formation. Therefore, the actual energy consumption should be calculated by the time integral and should be much lower than the estimated value above, namely, In the future, a dedicated circuit platform including reading and writing periphery can be developed to probe the MAEN result. Once the SEM observation is replaced by reading and writing periphery, both the gap distance (d) and compliance current (I) can be further reduced, and this will also significantly reduce the applied voltage (V) and switching timescale (t). The reduction in compliance current (I), applied voltage (V) and switching time (t) will significantly reduce the power consumption of the MAEN device. Furthermore, the device fabrication processes can also be optimized to further reduce the power consumption. For example, dielectric materials with higher ion mobility can be used to effectively promote Ag movement, which therefore is able to further reduce the applied voltage (V) and switching time (t). The above optimizations in gap distance, compliance current and ion transport properties etc. are expected to be capable of dramatically reducing the power consumption of the MAEN devices.