Reconfigurable Stochastic neurons based on tin oxide/MoS2 hetero-memristors for simulated annealing and the Boltzmann machine

Neuromorphic hardware implementation of Boltzmann Machine using a network of stochastic neurons can allow non-deterministic polynomial-time (NP) hard combinatorial optimization problems to be efficiently solved. Efficient implementation of such Boltzmann Machine with simulated annealing desires the statistical parameters of the stochastic neurons to be dynamically tunable, however, there has been limited research on stochastic semiconductor devices with controllable statistical distributions. Here, we demonstrate a reconfigurable tin oxide (SnOx)/molybdenum disulfide (MoS2) heterogeneous memristive device that can realize tunable stochastic dynamics in its output sampling characteristics. The device can sample exponential-class sigmoidal distributions analogous to the Fermi-Dirac distribution of physical systems with quantitatively defined tunable “temperature” effect. A BM composed of these tunable stochastic neuron devices, which can enable simulated annealing with designed “cooling” strategies, is conducted to solve the MAX-SAT, a representative in NP-hard combinatorial optimization problems. Quantitative insights into the effect of different “cooling” strategies on improving the BM optimization process efficiency are also provided.

SET/RESET operation is required for the flip of Boolean value, the endurance seems to be important parameter. Any comment about this? (5) Can the authors comment any advantage and benefit of the exponential-class sigmoidal for the BM applications? (6) The authors described the BM operation as a function of run or iteration number. How long the one run or iteration is taken? Because of the slow switching process of the fabricated device, the BM system can be further slow. The author should address this comment. (7) The baseline obtained from the theoretical calculation or simulation need to be provided in the BM result to compare with the experimental results. (8) The authors insisted that an appropriate cooling strategy can lead to more efficient performance in the BM. Can the author suggest any guideline for the appropriate cooling strategy?
Reviewer #2: Remarks to the Author: This work reports simulated annealing and Boltzmann machine based on three-terminal heteromemristors. The work are novel and will be of interest to others in the field. It can be published after addressing a few comments below: 1, In the device structure, the authors used a thin SnSe oxidized SnOx as the filament layer. How does the oxidation process affect the underlying MoS2? Will other oxide such as HfO2 function similarly? 2, Fig.2d, the Teff approaches 50% when Vg=-20V, shall we expect the rapid increase when Vg goes more negative as the simulation shows? Is there any limit of the Teff? 3, Fig.3, does it require each DUT to be exactly the same? Or not, will the randomness of DUT itself affect the machine implementation? 4, How does this approach compare with other methods using pure RRAM, MRAM or CMOS? What are the main merits?
that the extended defects in the oxide material, e.g., dislocations and grain boundaries, can facilitate filament formation and lower the required electric field. Refs. [2,3] also mentioned that in unipolar device, the filament formation operation is still due to a breakdown-like process with random creation of voltagestress-induced vacancy or defect sites, which is electric-field driven. Hence, we believe both the joule heating and the electric-field driven effect can be playing roles in the device operation.
There are several experimental studies on SnO x based memristive devices [4][5][6] , which have demonstrated the existence of oxygen-vacancy filament in SnO x layer. Considering that only a few one-dimensional filaments will be formed inside a three-dimensional material, preparing a cross-sectional TEM sample right at the position of a filament would be experimentally challenging. On the other hand, there are other techniques that can more easily allow the conductive filament to be directly observed in such devices. We carried out conductive atomic force microscopy (c-AFM) measurement to image the conductive filament in SnO x . In this test, bias voltage is applied through the AFM tip to form the conductive filament in SnO x . c-AFM measurement then maps the current in the active region after the filament is set. Figure R1 shows the test structure (same structure as the actual device with the material layers in reverse order to allow c-AFM probing) used in the c-AFM measurement. The tip acts as a conductive probe to apply the voltage at the top of MoS 2 layer, thereby setting the device into either the ON state or OFF state. A voltage of -5V is applied at the AFM tip (20 nm diameter) to set the device and the characteristic I-V curve shown in Figure R3 demonstrates the successful set process. After set, a small read voltage (-1V) is applied to read the current levels. In Figure R2, the green area indicates the region of higher current density (~105 pA) which is the location of the conductive filament. The filament diameter is about 20 nm. Besides, an obvious bubbleshape feature is also observed in the experimental process, when applying the voltage through the tip. This observation was also mentioned by other papers and could be explained by the oxygen accumulation in the filament region near the tip 6,7 , which supports the oxygen-vacancy type filament mechanism.

Comments 3:
In addition to this, they just mentioned that both the conductivity of the MoS 2 sheet itself and the interfacial barrier at MoS 2 /SnO x can be modulated by V g , even without any energy band diagrams and references. And there is no optical/SEM image for top-view of the device. How can make the filament inside the SnO x when the voltage is applied to semiconducting (non-conducting) MoS 2 and top metal electrode? The probing Figure R2 c-AFM current mapping read at -1V after device is set. The green region indicates the location of the conductive filament region with higher current density. ways and forming process are missing. This reviewer suggest that the authors must supplement the switching mechanism and the origin of the gate-tunable characteristics by including various experiments and discussions. Taken all together, the current manuscript is required to be significantly improved overall before it is published in Nature Communications. There are additional comments and suggestions that might help to improve the manuscript as below: Response: We thank the reviewer for this comment. We included an optical image in Figure R4b for the top-view of a typical device below. As shown in this figure, the top electrode (TE) is deposited on top of SnO x , and V TE is applied to this electrode. The other metal electrode (BE) contact is MoS 2 and is grounded. The gate bias V g is applied to the Si back gate. Since MoS 2 is a semiconducting material, V g applied on the Si back gate can modulate its Fermi-level and charge density, such that its resistance can be modulated accordingly.
The device schematic and bias condition is shown in Figure R4a. The current path between the top electrode and the GND is a vertical flow through the filament in serial combination with the horizontal flow through the back-gated MoS 2 layer. The resistance of the MoS 2 layer can be modulated by the applied back-gate voltage, which in turn modulates the potential drop across the SnO x filament switching layer. This potential drop across (and hence current flowing through) the SnO x layer can still lead to filament formation and rupture in it. As the gate bias becomes more positive (or leas positive/more negative), the MoS 2 layer will be more conductive (or less conductive), which will in turn tune the effective potential drop across the SnO x filament switching layer. A c-AFM mapping of conductive filament in an equivalent test structure is shown in Figure R2 in our response to the reviewer's previous comment.
The schematic band diagram along the current path, which is formed in the MoS 2 layer in series with the filament in the SnO x layer, is shown in Figure R5, where E F =0 is the Fermi level of GND, E F , TE is the Fermi level of the top electrode, and F n,MoS2 is the electron quasi-Fermi level in MoS 2 at the position of the vertical filament contact. The short lines schematically represent the oxygen vacancy levels in the SnO x layer. At low V G , the semiconducting MoS 2 layer has a low carrier density and thicker interface barrier, and as the gate voltage increases, the carrier density in MoS 2 increases and the contact barrier thickness can be reduced by the gate modulation, which results in lower resistance. Hence, the gate bias modulates both the carrier density in the MoS 2 layer as well as the properties of the MoS 2 /SnO x interfacial region, both of which in turn can influence the current and electric field inside the SnO x layer in this device structure to affect the filament dynamics. Similarly, other researchers have reported using voltage that is applied between semiconducting Si contact and top electrode to form filament inside SiO 2 filament switching layer 8 .
In addition, following the reviewer's suggestion, we have included the data about the forming process below in Figure R6 (same figures as in Figure R3 above) obtained from the c-AFM measurement of equivalent test structure as shown in Figure R1. During the first switching cycle, a higher applied voltage is needed to form the initial filament as measured in the c-AFM characterization (shown in Figure R6 a). The forming voltage is around -6.5 V (the voltage is negative due to the reverse device configuration used in the c-AFM test ( Figure R1)). After the initial forming process, the set voltage reduces to around -4.9 V for this test structure, lower than the forming voltage (Figure R6 b). The results confirm the forming process of the device. Figure R4b in Supplementary S1, Figure R5 in Supplementary S4, and Figure R6 in Supplementary S3 with relevant discussions. Figure R6 a, first (forming) switching cycle. b, subsequent switching cycle. Figure R5 Band diagram of SnOx/MoS2 hetero-memristive device under low and high VG.

Comments 4:
(1) The detailed probing methods and measurements in Fig. 1a should be clearly provided (e.g., an optical image). Also, the authors should identify electrical characteristics of the SnO x memristor without the insertion of MoS 2 by applying V g .

Response:
We thank the reviewer for this comment. As discussed in our response to Comments 3 above, we have included an optical image in Figure R4b for the top-view of the device. As shown in this figure, the top electrode (TE) is deposited on top of SnO x , and V TE is applied to this electrode. The other metal electrode (BE) contact is MoS 2 and is grounded. The gate bias V g is applied to the Si back gate. Since MoS 2 is a semiconducting material, V g applied on the Si back gate can modulate its Fermi-level and charge density, such that its resistance can be modulated accordingly.
If the MoS 2 under SnO x is completely removed, then the grounded electrode (contacted on the MoS 2 sheet) will be disconnected, and the device would not have a conducting current path because now the SnO x layer would be connected with only a top electrode, an insulating SiO 2 gate dielectric, and a Si gate.
If we do not insert MoS 2 and replace it with a metal contact under SnO x , the device would lose its gate tunability because now V g is completely screened by this metal film.
Moreover, if we do not consider using the gate, the intrinsic SnO x memristor with two metallic terminals has already been studied in the literature and numerous work has demonstrated its electrical characteristics including the unipolar memristive behavior 4-6,9-12 . These device, however, would not have the gate tunability.

Comments 5:
(2) In the sampling of exponential-class sigmoidal distribution, a statistical study was performed by switching from the high resistance state to the low resistance state for up to 2 second. This seem to be very slow. The authors should address the slow switching speed and discuss the effect of the slow sampling characteristics on the BM operation.

Response:
We thank the reviewer for bringing up this point. Switching speed in our device mainly depends on the effective voltage that drops across the SnO x . Higher voltage can produce higher electric field and higher current, which can facilitate oxygen ion movement and speed up the filament dynamics and increase the switching speed.
Theoretically, the filament formation is controlled by microscopic ionic hopping and ionization process, whose rates can be expressed as, = 0 exp (− − ), where 0 is a rate constant, is the barrier height, d is the effective hopping distance, is the electric field along the hopping direction 13 . This equation shows that the speed of filament dynamics is approximately exponentially dependent on the electric field. This has been demonstrated both analytically and experimentally 14,15 .
As the applied voltage decreases, the electric field decreases, which results in exponential decrease of the hopping and ionization rates. In the sampling of exponential-class sigmoidal distribution that is demonstrated in this work, the applied voltage V TE is kept relatively low to allow for more robust longer term device operation, which leads to relatively slow switching in this work. To shorten the switching time, we could optimize the material and device sizes. Using thinner SnO x in this device would make the switching time shorter. Another approach is to select oxide material that may provide higher hopping and ionization rate than SnO x .
The sampling speed of the BM operation is mainly limited by the switching speed of the filament formation process in the SnO x /MoS 2 structure. By optimization the device designs mentioned above, and together with scaling down the device size to reduce the parasitic, the sampling speed of the BM can be improved.
In this work, we focus more on the demonstration of the new device concept, and further optimization of the device and BM speed performance will be our future work.
Changes to supplementary information: We have included the discussion on the switching speed in Supplementary S5.

Comments 6:
(3) V g = -20 V -30 V is very high, which can lead to the high power dissipation by V g due to gate leakage current and gate charging process. Any comment about this issue?
Response: We thank the reviewer for this comment. In our device, the back gate has a 285 nm thermally grown SiO 2 dielectric layer, which is also commonly used in the initial demonstration of many new two dimensional material based electronic device concepts 16,17 . The effective electric field introduced by V g (in the range of -20 V to 30 V) is in fact quite low considering the thickness of the SiO 2 dielectric layer. In our device, the gate leakage current is around a few pA as shown in Figure R7 (limited by noise level of the current measurement setup), which is very low if compared to the device currents. This low gate leakage current is not expected to lead to strong gate charging and high power dissipation.
The V g can be scaled down by reducing the gate dielectric thickness and/or the use of high-dielectric. We would like to mention that the main focus of this work is on the initial demonstration of the new device concepts and its application, and it will be our future work to further optimize the performance metrics of the device.

Comments 7:
(4) This reviewer did not find the endurance parameter in this manuscript. Because the multiple SET/RESET operation is required for the flip of Boolean value, the endurance seems to be important parameter. Any comment about this? Figure R7 Drain current and gate current of a hetero-memristive device during set process.
Response: Since this work focuses the initial demonstration of new device concept, we did not optimize the device in terms of the endurance metrics. On the other hand, the device endurance is still reasonable even without too much optimization. The endurance of the device switching is shown in Figure R8. Due to the limitations in our endurance characterization capabilities, the device is measured up to 5000 switching cycles. The data is also extrapolated to the 10 6 cycles mark. The set pulse is 5V and the read pulse is 1V. the reset pulse is -5 V and the read pulse is 1V. Figure R8 in Supplementary S2.

Comments 8:
(5) Can the authors comment any advantage and benefit of the exponential-class sigmoidal for the BM applications?
Response: We thank the reviewer for this comment. The advantages and benefits of the exponential-class sigmoidal distribution for the BM applications could be summarized as following: 1. Sigmoid function is one of the most popular activation functions for the neural networks. The main reasons are (a) it is monotonic and continuous (b) it is differentiable and easy to compute the gradient which is always required for training.
2. The exponential-class sigmoidal sampling of the device fully captures the behavior of stochastic neuron, which is the basic composite of Boltzmann machine construction by definition. Its sampling distribution arising from the intrinsic randomness and energy distribution in the ionic motions can emulate the thermodynamic effect on neural behavior. The stochastic neurons may fire or not (flip the state or not) in response to the input signals and the firing rate is equivalent to the probability of successful SET occurrence following exponential-class sigmoidal distribution 18,19 .
3. Boltzmann machine is named after Boltzmann distribution, which is fundamental in many real-world physical phenomenon [20][21][22] . The exponential-class sigmoidal also resembles the Fermi-Dirac distribution that Figure R8 Endurance measured up to 5000 consecutive switching cycles.
determines the random ion movements in physical systems. Also, it is suitable for this framework to describe and solve the other problems that also follow the Fermi-Dirac distribution, which is general in real physics and device simulation 23 . 22 . The device demonstrated here that is capable of generating exponential-class sigmoidal statistics provides the possibility of dynamically tuning the effective temperatures in the BM process. Thus, various "cooling" strategies, aka. simulated annealing method, could be implemented and optimized in the BM process for a better and faster convergence 24 .

Comments 9:
(6) The authors described the BM operation as a function of run or iteration number. How long the one run or iteration is taken? Because of the slow switching process of the fabricated device, the BM system can be further slow. The author should address this comment.

Response:
We thank the reviewer for this comment. As has been addressed in our response to Comments 5, the switching speed of the BM is mainly limited by the SnO x /MoS 2 device. In this work, each iteration of the BM operation takes around 3 seconds as the switching process of the fabricated device costs most of the running time in each iteration. In the MAX-SAT application we demonstrated in the manuscript, six such fabricated devices are required to form the DUTs, and they could be run in parallel simultaneously. In this MAX-SAT problem, we include six different Boolean variables and compose five clauses in total. As shown in Figure 3 (d-e), this optimization converges after ten iterations. This means the total runtime for solving such a representative MAX-SAT problem is around 30 seconds, and we also confirmed this in experimental tests.
As discussed in or response to Comments 5, by optimizing the material properties and device dimension, we can further improve the speed of the BM system.

Comments 10:
(7) The baseline obtained from the theoretical calculation or simulation need to be provided in the BM result to compare with the experimental results.

Response:
We thank the reviewer for this very helpful comment. The simulation of the Boltzmann machine operation was conducted, and the simulated results are shown in Figure R9. Due to the stochastic nature of BM, we ran the simulation of BM for 50 times and plotted the averaged energy evolution under four "temperature cooling" strategies. The plot indicates the effect of temperature strategy on the optimization process of the BM, which matches well with the experimental results in Figure 4d. Figure R9 in Supplementary S10 with relevant discussion.

Changes to supplementary information: We have added
Comments 11: (8) The authors insisted that an appropriate cooling strategy can lead to more efficient performance in the BM. Can the author suggest any guideline for the appropriate cooling strategy?

Response:
We thank the reviewer for this comment. As discussed in the manuscript, implementing simulated annealing with an appropriate cooling strategy could lead to a better and faster convergence and prevent premature convergence in the BM.
A general guideline for the appropriate cooling strategy is to apply high effective "temperature" at the beginning of the optimization process to prevent premature convergence and applying lower effective "temperature" at the later stages of the optimization process to secure the convergence to global (or best local) optimum. The best detailed temperature cooling profile in an optimization process may vary based on the property of the optimization problem, such as the complexity of the problem, requirement for feasible convergence etc 18,19 . Despite it is more experiment-driven, a cooling strategy with linearly decreased (used in this paper) or a step-decreased temperature strategy (commonly used in annealing process) can be used as appropriate cooling strategy for initial try. Figure R9 The simulated result of energy evolution in BM under 4 different strategies for comparison to experimental result in Figure 4d.
It seems some of the parts in the manuscript is appropriately revised and improved according to the reviewer's comments in the first round of revision. However, the additional experiments and discussion for the switching mechanism are still unclear. The references [1][2][3][4][5][6] cited in the rebuttal letter seem to be a rather improper because the used metal-oxide, observed switching behaviors, and device structure were not the same; other metal-oxide materials (ZnO in Ref [1], NiO in Ref [2], NiO in Ref [3]), bipolar switching behaviors of SnOx-based memristors (Ref. [4,5]), and other switching characteristics (Vset < Vreset) in SnOx/SnSe/SnOx junction (Ref [6]).
In addition, the additional experiments the author performed need further discussion and explanation for supporting the author's argument. For this reason, I think that this manuscript needs to be more revised with some supporting data and additional discussion.
My concerns are provided as below: 1) The authors added several published references to support their claim, where the device structure was fundamentally different. As the authors already know, the different material combinations and device structure can strongly affect the switching principle and characteristics of memristors. despite the high current level, which is different from the switching characteristics in Fig. 1e (Vreset = -3 V in negative scan).
3) The device schematic of the device needs to be revised. From the optical image in Fig. R4, it is better to consider the MoS2 lateral dimension for clarification. Also, the ground symbol needs to be revised. The three dashes should be orientated horizontally, with the shortest one placed at the bottom rather than vertically. Various experiments and discussions should be required to study the origin of gate-tunable characteristics clearly. The suggested band diagram needs to be provided with some evidence and references, such as bandgap.
5) I wonder how the forming process of the device used in the manuscript occurs. Moreover, I don't agree that the authors did properly address the high gate voltage. The MOS capacitor in the gate region needs the power to charge up, and this power is proportional to ½C×V^2×f. In addition, the amplitude of the used VG in Fig. R7 should be presented, and the current level of ID in Fig. R7 is much lower up to ~2-3 order, compared with I-V curves in Fig. 1e, 1f,  6, eaba9901 (2020)).
In the AFM experiment, the authors performed the SET process by directly probing the AFM tip of 20-nm diameter (what kinds of metal tip was used?). This means, from the reviewer's perspective, that the tip acts as the 20-nm diameter top electrode practically. For the more correct c-AFM measurement, I think that it is necessary to perform the c-AFM measurements after etching the top electrode of the set-and resetcompleted device. Hence, the device may not have a 20-nm conductive filament. Additionally, the current map in Fig. R2 needs to add the colorbar for current-level, and the current of green region is too low ~105 pA. Furthermore, the right I-V curve in Fig. R3 does not exhibit the reset in the reverse sweep (-8 V → 0 V) despite the high current level, which is different from the switching characteristics in Fig. 1e (Vreset = -3 V in negative scan).

Response:
We appreciate the reviewer for this comment. We used a standard AFM tip with 10 nm Cr fullycoated on the front side, which has the same metal configuration as the device in the manuscript.
First, we would like to mention that it is impossible to accurately etch the top-electrode of the set-and resetcompleted device without impacting the formed filament inside. As the reviewer has mentioned in his other comments, the metal contacts on both the top and bottom electrodes assist in the formation and stabilization of the filaments. During the etching process, the etchant would be highly reactive with the oxygen vacancies on the surface of SnOx once the metal is totally move. Moreover, even if the filament could survive the etching process, the nature of the device heterostructure in which the filament is stabilized would have become very different without the top contact.
In comparison, our current setup performs the measurement in-situ with the same device configuration as in other parts of our manuscript and is probably the best feasible configuration to be tested with our cAFM system. Moreover, this method has been extensively to support the switching mechanism of RRAM in other Regarding the second part of the reviewer's question about the current level of the cAFM current mapping, we in fact intentionally kept the current level very low (voltage bias at -1 V for the original test device) during the c-AFM test to ensure the probing current would not disturb the existing filaments. Here, following the reviewer's request, we included a new measurement from another test device (as shown in Figure RR1) performed at a bias voltage of -2 V in this new device, which shows higher filament current level above 10 nA. A color bar is also added for the current mapping image.
Regarding the reset mentioned by the reviewer, the current compliance used here (and also in the previous Fig. R3) in the cAFM measurements (~30 A) is not high enough to observe the reset in the reverse sweep. As shown in Figure 1e of the manuscript, it usually needs larger current (~250 A) to observe such reset in the device.
Changes to supplementary information: We deleted the original Figure S4 and S5 in Supplementary and updated with a new Figure S4 with relevant discussion in Supplementary S3.

Comments 3:
The device schematic of the device needs to be revised. From the optical image in Fig. R4, it is better to consider the MoS2 lateral dimension for clarification. Also, the ground symbol needs to be revised. The three dashes should be orientated horizontally, with the shortest one placed at the bottom rather than vertically.

Response:
We thank the reviewer for pointing this out. The device schematic shown in manuscript Fig. 1a (same as the previous Fig. R4) has been updated as shown below.
Changes to Manuscript: We have updated Fig. 1a in the manuscript. Figure RR1, a, First set curve (represents forming process) and subsequent set curve. b, Current maps read by -2V after device is set in the c-AFM characterization. The green (and bright) area shows the position of higher current density.