Implementation of multilayer perceptron network with highly uniform passive memristive crossbar circuits

The progress in the field of neural computation hinges on the use of hardware more efficient than the conventional microprocessors. Recent works have shown that mixed-signal integrated memristive circuits, especially their passive (0T1R) variety, may increase the neuromorphic network performance dramatically, leaving far behind their digital counterparts. The major obstacle, however, is immature memristor technology so that only limited functionality has been reported. Here we demonstrate operation of one-hidden layer perceptron classifier entirely in the mixed-signal integrated hardware, comprised of two passive 20 × 20 metal-oxide memristive crossbar arrays, board-integrated with discrete conventional components. The demonstrated network, whose hardware complexity is almost 10× higher as compared to previously reported functional classifier circuits based on passive memristive crossbars, achieves classification fidelity within 3% of that obtained in simulations, when using ex-situ training. The successful demonstration was facilitated by improvements in fabrication technology of memristors, specifically by lowering variations in their I–V characteristics.

Started more than half a century ago, the field of neural computation has known its ups and downs, but since 2012, it exhibits an unprecedented boom triggered by the dramatic breakthrough in the development of deep convolutional neuromorphic networks 1,2 .The breakthrough 3 was enabled not by any significant algorithm advance, but rather by the use of high performance graphics processors 4 , and the further progress is being fueled now by the development of even more powerful graphics processors and custom integrated circuits [5][6][7] .Nevertheless, the energy efficiency of these implementations of convolutional networks (and other neuromorphic systems 8- 11 ) remains well below that of their biological prototypes 12,13 , even when the most advanced CMOS technology is used.The main reason for this efficiency gap is that the use of digital operations for mimicking biological neural networks, with their high redundancy and intrinsic noise, is inherently unnatural.On the other hand, recent works have shown [11][12][13][14][15][16] that analog and mixed-signal integrated circuits, especially using nanoscale devices, may increase the neuromorphic network performance dramatically, leaving far behind both their digital counterparts and biological prototypes and approaching the energy efficiency of the brain.The background for these advantages is that in such circuits the key operation performed by any neuromorphic network, the vector-by-matrix multiplication, is implemented on the physical level by utilization of the fundamental Ohm and Kirchhoff laws.The key component of this circuit is a nanodevice with adjustable conductance G -essentially an analog nonvolatile memory cell -used at each crosspoint of a crossbar array, and mimicking the biological synapse.
Though potential advantages of specialized hardware for neuromorphic computing had been recognized several decades ago 17,18 , up until recently, adjustable conductance devices were mostly implemented using the standard CMOS technology 13 .This approach was used to implement several sophisticated, efficient systemssee, e.g.Refs.[14, 15].However, these devices have relatively large areas leading to higher interconnect capacitance and hence larger time delays.Fortunately, in the last decade, another revolution has taken place in the field of nanoelectronic memory devices.Various types of emerging nonvolatile memories are now being actively investigated for their use in fast and energy-efficient neuromorphic networks  . Of paticular importance, is the development of the technology for programmable, nonvolatile twoterminal devices called ReRAM or "memristors" 42,43 .The low-voltage conductance G of these devices may be continuously adjusted by the application of short voltage pulses of higher amplitude (>1 V) 27,42 .These devices were used to demonstrate first simple neuromorphic network providing pattern classification 21,26,27,28,30,32,40 .The memristors can have a very low chip footprint, which is determined only by the overlap area of the metallic electrodes, and may be scaled down below 10 nm without sacrificing their endurance, retention, and tuning accuracy, with some of the properties (such as the ON/OFF conductance ratio) being actually improved 44 .
The main result of this paper is the experimental demonstration of a fully functional, boardintegrated, mixed-signal neuromorphic network based on integrated metal-oxide memristive devices.The demonstrated network is comprised of almost an order of magnitude higher number of devices as compared to the previously reported neuromorphic classifiers based on passive crossbar circuits. 26,27.Our particular focus is on passive ("0T1R") memristive crossbar circuits, which are naturally suitable for three-dimensional integration [45][46][47] .Due to their extremely high effective integration density, such circuits may be instrumental for keeping all the synaptic weights of a large-scale artificial neural networks locally, thus cutting dramatically the energy and latency overheads of the off-chip communications.

Integrated Memristors
The passive 20×20 crossbar arrays with Pt/Al2O3/TiO2-x/Ti/Pt memristor at each crosspoint were fabricated using a technique similar to that reported in Refs.26, 27 (Fig. 1).Specifically, the bilayer binary oxide stack was deposited using low temperature reactive sputtering method.The crossbar electrodes were evaporated using oblique angle physical vapor deposition (PVD) and patterned by lift-off technique using lithographical masks with 200-nm lines separated by 400-nm gaps.Each crossbar electrode is contacted to a thicker (Ni/Cr/Au 400 nm) metal line / bonding pad, which are formed at the last step of the fabrication process.As evident from Figure 1a, b, due to the utilized undercut in the photoresist layer and tilted PVD sputtering in the lift-off process, the metal electrodes have roughly triangular shape with ~ 250 nm width.Such shape of the bottom electrodes ensured better step coverage for the following processing layers and, in particular, helped to reduce the top electrode resistance.The externally measured (pad-to-pad) crossbar line resistance for the bonded chip is around 800 Ω.It is similar to that of smaller crossbar circuit reported in Refs.26, 27 due to the dominant contribution of the contact between crossbar electrode and thicker bonding lines.
Majority of the devices required an electroforming step which consisted of one-time application of a high current ramp bias.The devices were formed one at a time, and to speed up the whole process, an automated setup has been developed (Fig. S1a).Such setup was used for early screening of defective samples, and has allowed a successful forming and testing of numerous crossbar arrays (Fig. 2).
Memristor I-V characteristics are nonlinear (Fig. 1c) due to the alumina barrier between the bottom electrode and the switching layer.I-V's nonlinearity provides selector functionality, which is essential for the conductance tuning in the crossbar circuit.In particular, it limits leakage currents and reduces disturbance of half-selected devicessee Sect. 3 of the Supplementary Information for more discussion of this point.
Most importantly, memristive devices in the fabricated 20×20 crossbar circuits have uniform characteristics with gradual "analog" switching.Specifically, the distributions of the effective set and reset voltages are sufficiently narrow (Fig. 2) to allow precise tuning of devices' conductances to the desired values in the whole array (Fig. 3).To the best of our knowledge, this is the first report of such a precise adjustment on this integration scale.For example, an analog tuning was essential for other demonstrations based on passive memristive circuits, though was performed with much cruder precision 19,39 .A comparable tuning accuracy was demonstrated in Ref. 40, though for less dense but much more robust to variations "1T1R" structures, in which each memory cell is coupled with a dedicated transistor.

Multilayer Perceptron Implementation
Two 20×20 crossbar circuits were packaged and integrated with discrete CMOS components on two printed circuit boards (Fig. S2b) to implement the multilayer perceptron (MLP) (Fig. 4).The MLP network features 16 inputs, 10 hidden-layer neurons, and 4-outputs, which is sufficient to perform classification of 4×4-pixel black-and-white patterns (Fig. 4d) into 4 classes.With account of bias inputs, the implemented neural network has 170 and 44 synaptic weights in the first and second layers, respectively.
The integrated memristors implement synaptic weights, while discrete CMOS circuitry implements switching matrix and neurons.Each synaptic weight is implemented with a pair of memristors, so that 17×20 and 11×8 contiguous subarrays were involved in the experiment (Fig. 4a), i.e. almost all of the available memristors in the first crossbar and about a quarter of the devices in the second one.The switching matrix was implemented with analog discrete component multiplexers and designed to operate in two different modes.The first one is utilized for on-board forming of memristors as well as their conductance tuning during weight import.In this operation mode, the switching matrix allows the access to any selected row and column and, simultaneously, the application of a common voltage to all remaining (half-selected) crossbar lines, including an option of floating them.The voltages are generated by an external parameter analyzer.In the second, inference mode the switching matrix connects the crossbar circuits to the neurons as shown in Fig. 4a and enables the application of ±0.2 V inputs, corresponding to white and black pixels of the input patterns.Concurrently, the measurement of output voltages of the perceptron network is carried out.The whole setup is controlled by a general-purpose computer (Fig. S2c).
The neuron circuitry is comprised of three distinct stages (Fig. S2a).The first stage consists of inverting operational amplifier, which maintains a virtual ground on the crossbar row electrodes.
Its voltage output is a weighted sum between the input voltages, applied to crossbar columns (Fig. 4a), and the conductances of the corresponding crosspoint devices.The second stage op-amp computes the difference between two weighted sums calculated for the adjacent rows of the crossbar.The operational amplifier's output in this stage is allowed to saturate for large input currents, thus effectively implementing tanh-like activation function.In the third and final stage of the neuron circuit, the output voltage is scaled down to be within -0.2 V to +0.2 V range before applying it to the next layer.The voltage scaling is only implemented for the hidden layer neurons to ensure negligible disturbance of the state of memristors in the second crossbar array.
With such implementation, perceptron operation for the first and second layers is described by the following equations: Here V in , V H , V out are, respectively, perceptron input, hidden layer output, and perceptron output voltages.G (1)± and G (2)± are the device conductances in the first and second crossbar circuits, with +/-superscripts denoting a specific device of a differential pair, while I ± are the currents flowing into the corresponding neurons. 1 ≤ j ≤ 10 and 1 ≤ k ≤ 4 are hidden and output neuron indexes, while 1 ≤ i ≤ 16 is the pixel index of an input pattern.The additional bias inputs are

Pattern Classification
The multilayer perceptron was trained ex-situ by first finding the synaptic weights in the software-implemented network, and then importing the weights into the hardware.(Some of the preliminary results for in-situ training are discussed in Sect. 2 of the Supplementary Information.) In particular, the software-based perceptron was trained with conventional backpropagation algorithm, using four sets of patterns representing four classes of letters (Fig. 4d).In the software network, the neuron activation function was approximated with tangent hyperbolic with a slope specific to the hardware implementation.We assumed a linear I-V characteristics for the memristors, which is a good approximation for the considered range of voltages used for inference operation (Fig. 1c).During the training the weights were clipped within [10 μS, 100 μS] conductance range, which is an optimal range for the considered memristors.
In addition, two different approaches for modeling weights were considered in the software network.In the simplest "hardware-oblivious" approach, all memristors were assumed to be perfectly functional, while in a more advanced "hardware-aware" approach, the software model utilized additional information about the defective memristors.These were the devices whose conductances were experimentally found to be stuck at some values, and hence could not be changed during tuning.
The calculated synaptic weights were imported into the hardware by tuning memristors' conductances to the desired values using an automated write-and-verify algorithm. 48The stuck devices were excluded from tuning for the hardware-aware training approach.To speed up weight import, the maximum tuning error was set to 30% of the target conductance (Fig. 5a, b), which is adequate import precision for the considered benchmark according to the simulation results (Fig. S5).After weight import had been completed, the inference was performed by applying ±0.2V inputs specific to the pattern pixels and measuring four analog voltage outputs.Fig. 5c shows typical transient response.Though the developed system was not optimized for speed, the experimentally measured classification rate was quite highabout 300,000 patterns per second and was mainly limited by the chip-to-chip propagation delay of analog signals on the printed circuit board.for the hardware-oblivious approach were respectively 95% and 79.06% (Fig. 5d), as compared to 100% and 82.34% achieved in the software (Fig. S5).As expected, the experimental results were much better for hardware-aware approach, i.e. 100% for the training patterns and 81.4% for the test ones (Fig. 5e).
It should be noted that the achieved classification fidelity on test patterns is far from ideal 100% value due to rather challenging benchmark.In our demonstration, the input images are small and addition of noise, by flipping one pixel, resulted in many test patterns being very similar to each other.In fact, many of them are very difficult to classify even for a human, especially distinguishing between test patterns "V" and "X".

Discussion and Summary
We believe that the presented work is an important milestone towards implementation of extremely energy efficient and fast mixed-signal neuromorphic hardware.Though demonstrated network has rather low complexity to be useful for practical applications, it has all major features of more practical large-scale deep learning hardwarea nonlinear neuromorphic circuit based on metal-oxide memristive synapses integrated with silicon neurons.The successful board-level demonstration was mainly possible due to the advances in memristive circuit fabrication technology, in particular much improved uniformity and reliability of memristors.
Perhaps the only practically useful way to scale up the neuromorphic network complexity further is via monolithical integration of memristors with CMOS circuits.Such work has already been started by several groups 19,30 , including ours 47 .We envision that the most promising implementations will be based on passive memristor technology, i.e. similar to the one demonstrated in this paper, because it is suitable for monolithical back-end-of-line integration of multiple crossbar layers 46 .The three dimensional nature of such circuits 49 will enable neuromorphic networks with extremely high synaptic density, e.g.potentially reaching 10 13 synapses in one square centimeter for 100-layer 10-nm memristive crossbar circuits, which is only hundred times less compared to the total number of synapses in a human brain.(Reaching such extremely high integration density of synapses would also require substantially increasing crossbar dimensions -see discussion of this point in Section 3 of Supplementary Information.)Storing all network weights locally would eliminate overhead of the off-chip communication and lead to unprecedented system-level energy efficiency and speed for large-scale networks.For example, the crude estimates showed that energy-delay product for the inference operation of a large-scale deep learning neural networks implemented with mixed-signal circuits based on the 200-nm memristor technology similar to the one discussed in this paper could be six orders of magnitude smaller as compared to that of the advanced digital circuits, while more than eight orders of magnitude smaller when utilizing three-dimensional 10-nm memristor circuits 50      (%) (%) "A" "T" "V" "X" "A" "T" "V" "X" "A" "T" "V" "X" "A" "T" "V" "X" turned on during forming of other devices.Finally, if the device could not be formed within allowed number of attempts for the second time, it is recorded as defective.About 1% to 2.5% of the devices in the crossbar array (i.e. 10 or less out of 400 total) could not be formed with the forming algorithm parameters that we used.It might have been possible to form even these devices by applying larger stress but we have not tried it in this experiment to avoid permanently damaging the crossbar circuit.Typically, the failed devices were stuck at some conductance state, comparable to the range of conductances utilized in the experiment, so failed device had negligible effect on the tuning accuracy.
Figure S2 shows additional details of the MLP implementation and the measurement setup.
We have used AD8034 discrete operational amplifiers for the CMOS-based neurons and ADG1438 discrete analog multiplexers to implement on-board switch matrix.
Figure S3 shows absolute values of resistances and absolute error for the data presented in Figure 3.

Additional Details and Results for Pattern Classification Experiment
Because of limited size of the classifier, we have used custom 4-class benchmark, which is comprised of a total of 40 training (Fig. 4d) and 640 test (Fig. S4) 4×4-pixel black and white patterns representing stylized letters "A", "T", "V", and "X".As Figure S5     In addition to ex-situ method (see Figure 5 of the main text and its discussion), we have also trained the network "in-situ", i.e. directly in a hardware [1, 2].(Similar to our previous work [1,   2], only inference stage was performed in a hardware during such in-situ training, while other operations, such as computing and storing the necessary weight updates, were assisted by an external computer.)Unfortunately, because of limitations of our current experimental setup, we could only implement in-situ training using fixed-amplitude training pulses (which is similar to Manhattan rule).The classification performance for this method was always worse as compared to that of both hardware-aware and hardware-oblivious ex-situ approaches.For example, the experimentally measured fidelity for 3-pattern classification task was 70% (Fig. S6), as compared to 100% classification performance achieved on training set using both ex-situ approaches.This is expected because in ex-situ training the feedback from read measurements of the tuning algorithm allows to effectively cope with switching threshold variations by uniquely adjusting write pulse amplitude for each memristor, which is not the case for the fixed-amplitude weight update (Fig. S6).We expect that fidelity of in-situ trained network can be further improved using either variable-amplitude implementation [3].

Crossbar Circuit Scaling
An important future work, in addition to the monolithic integration with CMOS subsystem discussed in the main text, is increasing the dimensions of the crossbar circuits which would allow higher connectivity among neurons and improve integration density (i.e. by lowering relative peripheral overhead).Here let us first stress again that in our implementation, crossbar lines are never floated so that sneak path currents do not affect directly the measured currents at the outputs.
Scaling up crossbar dimensions, however, increases currents flowing in the crossbar lines.Because of the potential voltage drops across the crossbar lines the voltages applied to the crosspoint memristors could be different from the ones applied at the periphery.
For example, Figure S7 shows the dependence of the worse-case voltage drop as a function of the length of the finite resistor ladder, which is useful for analyzing crossbar circuit operation.In this figure, one set of lines shows the voltage drop assuming electrode resistance per wire segment (Rw) comparable to the one in our experiment, while the other one is for more aggressive (though quite realistic) parameters which are representative of high-aspect ratio copper wires.For simplicity, the memristor conductances G(V) can be estimated using the corresponding average value measured at bias V, specific to the type of considered operation.It should be noted that in a properly trained network, the weights are typically normally distributed so that the representative average value is rather close to the minimum of the used conductance range.
Let us now consider in detail three operations which might be impacted by voltage drop on the crossbar lines, namely classifier inference, and read and write phases of the tuning algorithm:

 Write operation
Naturally, the voltage drops are the most significant for write operation because of the larger voltages applied and higher currents passed.For the conductance tuning, however, we do not rely on precise conductance update with write pulses but rather adjust applied write voltages gradually based on precise read measurements.Therefore, any potential voltage drop will be compensated dynamically during tuning by applying larger voltage pulses, with the largest applied voltage (and hence crossbar dimensions) limited by the condition of not disturbing half-selected devices.
Specifically, let us assume the V/3 biasing scheme, i.e. with ±VW/2 applied to the selected lines and ±VW/6 to the remaining lines.From Fig. 1c and 2, up to (VTH SET )max ≈ +1.3 V set and (VTH RESET )max ≈ -1.9 V reset voltages must be applied to switch the devices with the largest switching thresholds.(Here, we neglect the tails of the distributions on Fig. 2, which are typically contributed by the devices at the edges of the array.This is similar to the dummy line technique commonly used in conventional memories.)The corresponding average memristor conductances at one third of such biases can be roughly estimated to be <G((VTH SET )max/3)> ≈ 30 µS for set and <G((VTH SET )max/3)> ≈ 50 µS for reset transitions.On the other hand, the largest voltages, which can be safely applied to the half-selected devices without disturbing memristors with the smallest switching thresholds are (VTH SET )min ≈ +0.7 V for set and (VTH RESET )min ≈ -1 V for reset transitions.

 Read operation
Let us assume that during read operation, one of the selected lines is biased at +VR, while the other selected line and all of the remaining ones are grounded.(This is exactly the scheme that we used for conductance tuning in this work.)In this case, the current running via grounded selected crossbar line is small (only contributed by one selected memristor) and does not dependent on the crossbar dimensions.Therefore, the substantial voltage drops may occur only on the biased selected crossbar line.Such voltage drop would be naturally much less than that of the write operation and, moreover, it can be easily taken into account when reading the state of the devices.For example, it is straightforward to compute the actual applied voltage across the specific memristor knowing the conductive states of all other half-selected devices of the biased selected crossbar electrode.

 Inference operation
As discussed in main text, during inference, one set of lines (vertical in Figure 3a) receive voltages V ≤ VR, while all orthogonal lines are virtually grounded.Because of the smaller applied voltages, the crossbar line currents, and hence the corresponding voltage drops, are the smallest for inference operations.However, the inference operation (just like read) is more sensitive as compared to write operation to the voltage variations and even small voltage drops may lead to the lower effective precision of the vector-by-matrix computation.For example, assuming representative 10 µS average device conductance, and 70×70 and 400×400 crossbar arrays discussed in write operation above, the worst-case voltage drop on one line is around 7% (Fig. S7b).
Using our examples, inference operations would likely be a limiting factor for scaling though are several reserves for improvements.For example, the conductances of each memristor can be uniquely increased to compensate for the potential voltage drops during inference.(Unlike read operation, such adjustment cannot be exact because of the input-dependent voltage drop on the virtually-grounded lines.)The loss of precision for the worst case largest currents might be also acceptable, e.g. if it leads to the saturation of the neuron.It is also important to note that precision loss at inference due to voltage drops is common problem for the devices with or without selectors.
If fact, the problem is likely more severe for 1T1R structures, because of their larger device area and potentially larger Rw.
The crude estimate above show that the developed device technology, with some further optimization of the electrodes, should be suitable for implementing much larger, up to 400×400 crossbar circuit.The discussed analysis is also applicable to 10 nm memristors, if we assume that both the resistance of the crossbar line segment and memristor operating (average) currents would scale down at the same rate.(For that memristor currents should decrease at slightly faster rate than its linear device dimensions to compensate for the additional increase in metal resistivity due to scattering effects.)That is certainly plausible scenario for smaller currents at voltages below VR (e.g., relevant to the inference operation and read phase of the tuning algorithm) considering that the off-state conductance is typically limited by the device leakages which are proportional to the device electrode area.Ensuring the same scaling in the context of the write phase of the tuning algorithm would require enhancing I-V nonlinearity and/or decreasing write currents, which we believe is also plausible given the observed write current dependence on the electrode area in our devices and further optimization of the tunneling barrier layer.

Temperature Sensitivity
Practical neuromorphic hardware should be able to operate correctly under wide temperature ranges.Even though we have not measured experimentally the sensitivity of the functional performance to the ambient temperature variations let us note that for the proposed circuits, the change in memristor conductance with ambient temperature (Fig. S8a, b) is already partially compensated because of differential synapses implementation, with each weight W ≡ G + -G -

Figure 5d ,
Figure 5d,e shows classification results for the considered benchmark using the two different approaches.(In both software simulations and hardware experiments, the winning class was determined by the neuron with maximum output voltage.)The generalization functionality was tested on a 640 noisy test patterns (Fig. S4), obtained by flipping one of the pixels in the training images (Fig. 4d).The experimentally measured fidelity on a training and test set patterns

Figure 2 .
Figure 2.Set and reset threshold statistics for seven 20×20-device crossbar arrays at memristor switching with current and voltage pulses.The set / reset thresholds are defined as the smallest voltages at which the device resistance is increased / decreased by more than 5% at the application of a voltage or current pulse of the corresponding polarity.The legends show the corresponding averages and standard deviations for the switching threshold distributions.Note that the variations are naturally better when only considering devices within a single crossbar circuit, and in addition, excluding memristors at the edges of the circuit, which typically contribute to the long tails of the histograms.For example, µ = 1.0 V and σ = 0.13 V for voltage controlled set, while it is µ = -1.2V and σ = 0.15 V for reset for one of the crossbars used in the experiment.

Figure 3 .Figure 4 .
Figure 3. High precision tuning in 20×20 memristive crossbar: (a) the desired "smiley face" pattern, quantized to 256 gray levels.(b) The actual resistance values measured after tuning alldevices with the nominal 5% accuracy, using the automated tuning algorithm48 , and (c) the corresponding statistics of the tuning errors, which is defined as normalized absolute difference between the target and actual conductance values.On panel (a), the white / black pixels correspond to 84 KΩ / 7 KΩ, measured at 0.2 V bias.The tuning was performed with 500-µs-long voltage pulses with amplitudes in a [0.8 V, 1.5 V] / [-1.8 V, -0.8 V] range to increase / decrease device conductance.

a)Figure 5 .
Figure 5. Experimental results: (a, b) The normalized difference between the target and the actual conductances after tuning in (a) the first and (b) the second layer of the network for the hardwareoblivious training approach; (c) Time response of the trained network for 6 different input patterns, in particular showing less than 5 μs propagation delay.Perceptron output voltage for (d) hardwareoblivious and (e) hardware-aware ex-situ training approaches, with top / bottom panels showing measured results for training / test patterns.

Figure S1 .
Figure S1.(a) Flow diagram of the automatic current-controlled memristor forming procedure.The the adjustment of Istop' value was so far performed manually after the failure to form a device automatically (in ~10% of all cases).(b) All forming I-V curves for one of the crossbars used in the experimental demonstration (with Istart = 180 µA, Istop = 540 µA, Istep = 20 µA, Vreset = -1.3V, Rmin = 5).

Figure S2 .
Figure S2.Experimental setup and board details: (a) Circuit diagram of the implemented neurons.Note that the output scaling stage is not implemented in the output neurons; (b) Photos of the two printed circuit boards with one hosting wire-bonded memristive crossbar chips and the switching matrix and the other one implementing discrete CMOS neurons; (c) Block diagram of the experimental setup controlled by a personal computer.

Figure S3 :
Figure S3: Absolute device resistances (top) and absolute tuning error (bottom) for the smiley face experiment.

Figure S4 .
Figure S4.A complete set of 640 test patterns.

42E+04 4 .PatternFigure S5 .
Figure S5.Simulated classification performance for several networks as function of weight import precisions: (a) Comparison of the best fidelity obtained for single layer perceptron and MLPs with different number of hidden layer neurons (shown in parenthesis in the legend).(b, c) The results for 10-hidden layer perceptron, similar to the one used in the experiment for classification of (b) training and (c) test patterns.The weight error was modeled by adding, to its optimized value, a normally distributed noise with the shown standard deviation.The red, blue (rectangles), and black (segment) markers denote, respectively, the median, the 25%-75% percentile, and the minimum and maximum values for 100 simulation runs.

Figure S6 .
Figure S6.In-situ training for 3-pattern classification ("A", "V", and "T"): (a) Experimentally measured and simulated error decay dynamics for the training set patterns.In experiment, conductances of all memristors were updated, one row of the crossbar at a time, at the end of each epoch.The weight update in each row was done in parallel in two steps by applying 500-µs fixed amplitude (± 1.3 V) voltage pulses using V/2 biasing technique.(b) Example of devices' switching kinetics and it's variations obtained using simple device model from Ref. [1].Such model was used for the in-situ training simulations shown in panel asee supplementary matlab code for more details.

Figure S7 .
Figure S7.Voltage drop in resistor ladder: (a) The considered circuit and (b) the relative worst-case voltage drop for several representative parameters specific to the implemented crossbar circuits.AR stands for the electrode height-towidth aspect ratio.
The maximum crossbar dimensions, specific to the wire resistance, memristor I-V and its variations (i.e.parameters Rw,