Introduction

Big data has emerged as a critical driving force to the data-driven industrial revolution, which underpins decision-making processes in modern society, enhances efficiency, and promotes innovation1,2. The ability to collect, analyze, and interpret big data supports companies and societies involved in artificial intelligence in maintaining a competing edge. Among the various types of data, time-series data, which are sequential observations collected over specific time frames, have become increasingly important as dynamic information in applications such as biometric analysis3, weather forecasting4, stock chart estimation5, and water inflow prediction6. Their importance arises from the inherently ordered nature, which allows predicting future behaviors in dynamic information using estimated models that describe correlated features.

In artificial intelligence, time-series data have been extensively explored through approaches including network, statistical, and transformer models7,8. A promising approach is the implementation of reservoir computing (RC) based on nonlinear dynamic systems such as physical reservoirs (PRs) to replace computationally intensive software-based methods9,10. Remarkably, memristive systems feature inherent nonlinearity and short-term memory capability, which enable a single memristor node to replace a complex nonlinearly coupled loop formed by multiple nodes in a software reservoir11,12,13,14,15,16,17,18. Since the first demonstration of memristive physical RC11, several approaches based on memristors, such as the virtual node concept14, self-organized nanowire network17, input masking16, and a fully analog framework18, have been suggested to date. Although these proposed physical RC systems are noteworthy, most of them are fabricated on single-stacked and two-dimensional (2D) memristive device frameworks and inevitably restricted to a single reservoir architecture. The 2D-based approaches eventually encounter physical limitations in addressing multi-variant time information. In addition, several multilayered physical RC systems have been recently proposed19,20. These systems demonstrated the promising aspects of the advanced reservoir architecture, including improved reservoir capacity and optimization. However, these approaches have a 2D lateral architecture or rely on the input design rule, which requires additional multilayered PR architecture capable of parallel processing, simplification, and versatility to efficiently address multi-variable temporal inputs (detailed in Supplementary Note 1). Hence, it is desirable to realize a wide physical RC with multiple-stacked PRs based on three-dimensional (3D) integrated memristive architecture. This 3D approach is anticipated to enable the extraction of richer features and multiple local traits from temporal inputs simultaneously, performing a more accurate, reliable, and rapid RC than the 2D-based single physical RC. In addition, the multiple-stacked PRs can provide signal mapping capabilities with a smaller footprint than a single PR. Consequently, 3D-integrated memristive PRs architecture is a significant step in the development of a next-generation physical RC. However, the achievement of the wide RC based on multiple-stacked PRs remains elusive within existing memristive frameworks as well as other device platforms due to the complex fabrication steps for 3D stacked structures and the stringent requirements on the PR properties.

For instance, many memristors show unreliable switching characteristics such as variation, instability, and low yield21,22 owing to the intrinsic nature of switching filaments, which makes it extremely challenging to meet the high reproducibility requirement of PR states from cycle to cycle and from device to device. Additionally, the matrix array structure demands additional components, such as transistors or selectors, to ensure reliable programming and reading by preventing crosstalk23,24. This leads to complex device structure and fabrication process (i.e., incorporating 1 T) as well as increased power consumption and active node size. Moreover, memristors at matrix nodes must operate in a low programming voltage range because time-series data processing requires a considerable number of input pulses with varying intervals (Δt) over a specific time frame for low-power operation14,15. Furthermore, addressing and predicting correlations between various dynamic information sources require the ability to handle multiple time series concurrently in each time frame without mutual interference25,26,27. In other words, many independent reservoir states controlled by sequential input pulses with varying Δt are required for multiple dynamic tasks to be performed simultaneously without interference. To effectively tackle these challenges all at once, we propose and demonstrate the wide RC based on a multilayered 3D-integrated memristive array structure with several independent reservoir states at each matrix layer, crosstalk-free and robust switching characteristics, high device yield, and low programming voltages. In other words, the primary objective of our research is to design, fabricate, and implement a wide physical RC platform, enabling its capacity to be widely applied in sophisticated reservoir configurations and providing functional benefits that go beyond simply increasing the density of devices.

Our approach can efficiently capture and process 3D local dynamic features of multiple time series by implementing a 3D stacked 3 × 10 × 10 tungsten oxide (WOx) array architecture. Each memristor cell is designed based on asymmetric interfacial Schottky barrier engineering to achieve crosstalk-free of signals throughout the array. The 3D stacked array exhibits highly reliable and robust operation with characteristics such as a high yield (>98%), exceptional stability (>4000 consecutive sweeps and >105 cycles), switching uniformity (~0.24%), and low operating voltage (~0.7 V). We demonstrate reliable reservoir operations under various time-dependent electrical inputs without overlapping reservoir states. As a proof of concept, we perform spatiotemporal pattern classification of biological cell positions and behavior prediction of a time-dependent chaotic Lorenz attractor. Our proposal achieves a higher and faster (~4.5 times) classification accuracy for 3D positions of a biological cell using 60% fewer memristor cells, and a tenfold reduction in prediction error for the chaotic Lorenz attractor compared with the use of a single reservoir. The proposed 3D physical RC architecture supported by reliable memristors opens an avenue for efficient and compact physical structure for reservoir computing capable of time-series processing in artificial intelligence systems.

Results

Three-layer stacked physical reservoirs

Figure 1a illustrates a wide RC system consisting of three reservoir layers (yellow, green, and purple) and an output layer. These reservoirs can perform a nonlinear projection of temporal inputs (un(t)) onto reservoir states \(({x}_{n}^{i}(t))\) per feature space via the different nodes of the reservoirs (blue circles), where i and n are the order of the reservoir layer (i = 1, 2, 3) and number of nodes (n = 1, 2,…, n), respectively9,10. As the collective reservoir states are delivered from each reservoir layer to the output layer, the weights (\({W}_{n}^{i}\)) between them can be iteratively updated for the desired outputs (yn(t)) according to an algorithm (e.g., backpropagation, linear regression). Such learning process through the output layer can be readily executed based on a linear combination of the weighted reservoir states, thereby reducing the learning costs when compared with other time-series processing algorithms. Moreover, it has been demonstrated through an algorithmic and theoretical approach that the utilization of a wide RC system is more effective in handling multiple dynamic time information simultaneously by extracting rich local features through multiple independent reservoir layers, in comparison to a single RC system (Supplementary Fig. 1 and Supplementary Note 2)25,26,27. As shown in Figs. 1a, b, the wide RC system can be physically mapped onto a 3D stacked three-layer memristive crossbar array, where each layer acts as a reservoir (three reservoirs in total). Figure 1b shows a top-view optical microscopy image of the fabricated 3D stacked three-layer 3 × 10 × 10 memristive crossbar array, where each layer consists of Pt/WOx/W memristor cells at each crosspoint (300 cells). In Fig. 1b, the first, second, and third WOx layers are indicated in orange, green, and purple, respectively, confirming the formation of a 3D stacked array. Figures 1c, d show the 3D stacked 3 × 2 × 3 subarray (Fig. 1c) with a magnified view of the vertically integrated cells at a crosspoint (Fig. 1d), revealing a cell line width of approximately 100 μm. To facilitate the understanding of the 3D stacked architecture, the corresponding schematics are shown in Figs. 1e, f. Array fabrication with a cell line width of 20 μm is also demonstrated using conventional photolithography (Supplementary Fig. 2). Moreover, similar switching feature at nanoscale is demonstrated, as detailed below. Figures 1g–k show cross-sectional high-resolution transmission electron microscopy (HR-TEM) images and the corresponding energy dispersive spectroscopy (EDS) results of the 3D stacked array at a crosspoint, verifying the well-defined vertical integration of the three Pt/WOx/W junction structures. The detailed fabrication processes are provided in the Supplementary Information (Supplementary Fig. 3) and Methods section. Note that the benefits of our approach are further discussed in Supplementary Figs. 4, 5, Supplementary Table 1, and Supplementary Note 3, 4 of Supplementary Information.

Fig. 1: 3D stacked WOx physical reservoir (PR) arrays.
figure 1

a Schematic of a wide reservoir computing (RC) system with multiple reservoirs (see Supplementary Note 2). b Top-view optical image of fabricated 3D stacked three-layer 3 × 10 × 10 crossbar array with vertical integration of three Pt/WOx/W memristors at each crosspoint. The first (1st), second (2nd), and third (3rd) WOx layers are shown in orange, green, and purple, respectively. The fabrication of each layer is described in the Supplementary Information (Supplementary Fig. 3). The 3D stacked array can naturally map a wide RC system onto hardware. c, d Top-view optical image of 3 × 2 × 3 sub-array (c) and magnified view at crosspoint (d) with line width of 100 μm. The scale bar for the top and bottom image is 200 μm and 100 μm, respectively. e Schematic of fabricated 3D stacked three-layer memristive crossbar array. Each memristor cell located at the first, second, and third layers is shown in orange, green, and purple, respectively. f Magnified schematic of vertical integration of three Pt/WOx/W memristors at a certain crosspoint (3 × 1 × 1) in panel (e) (red dotted line). g–k Cross-sectional high-resolution transmission electron microscopy (HR-TEM) image of three stacked memristors at crosspoint (g) and its corresponding energy dispersive spectroscopy (EDS) results for all (h) and each (i–k) of its elements. The scale bar for all (g–k) is 100 nm.

Full size image

Electrical characteristics of the 3D physical reservoirs

Figure 2a shows a representative self-rectifying current–voltage (I–V) switching curve of a memristor in the 3D stacked WOx PRs. The ION/IOFF and rectification (ION/ISneak) ratios are approximately 4.1 × 102 and 4.8 × 102, respectively, where ION and IOFF are the ON and OFF currents at reading voltage VREAD = 0.5 V, respectively, and ISneak is the sneak current at −VREAD. The programming voltage ranges in ±1.8 V. An initial electroforming step is not required for the fabricated array memristors. This switching phenomenon is attributed to the robust Schottky barrier formed at the Pt/WOx interface and dynamic modulation of the Schottky barrier at the WOx/W interface, which is controlled by the electric-field-driven migration of oxygen vacancies28,29. It can be also observed even in the absence of an applied compliance current (Fig. 2a). This operational advantage can reduce the burden on peripheral circuitry for strict current control, hence, it enables our 3D stacking array to be denser and more scalable. Note that the shift of voltage position for minimum currents in Fig. 2a might be associated with the transient formation of an internal electric field30,31,32 (Supplementary Note 5). Figure 2b shows the statistical histograms of ION, IOFF, and ISneak for all the memristors from the first to third layers of the 3D stacked WOx PRs (300 memristors). The I‒V switching curves of all the reservoir layers are provided in the Supplementary Information (Supplementary Figs. 6–8). Similar I‒V switching curves are observed regardless of the location of the reservoir layers. The average values for ION, IOFF, and ISneak are (1.40 ± 0.10) × 10−6 A, (4.05 ± 0.38) × 10−9 A, and (2.23 ± 0.16) × 10−9 A, respectively. All the distributions of ION, IOFF, and ISneak are well-fitted by the lognormal distribution curve (Supplementary Fig. 9). Hence, well-defined switching parameters with acceptably low variations are observed in the 3D stacked WOx PRs. The device yield of the layers is 98.3% (97, 100, and 98/100 for the first, second, and third layers, respectively), as illustrated in the schematic of Fig. 2c (see Supplementary Figs. 6–8). These results support the reproducibility and expandability of the 3D stacked WOx PRs for large-scale physical RC systems.

Fig. 2: Electrical characteristics of 3D Pt/WOx/W memristor array.
figure 2

a Representative I−V curve of Pt/WOx/W memristor without initial forming and exhibiting self-rectifying bipolar switching. The top Pt electrode and bottom W electrode were biased and grounded, respectively. The inset shows the same I−V curve on a linear scale. b Statistical histograms of switching parameters ION, IOFF, and ISneak for all the memristors from the first to third layers (300 memristors). c Schematic of 3D stacked WOx array illustrating switching success and failure (i.e., short circuit) as colored and red dotted boxes, respectively. d Operation under consecutive I−V sweeps over 4000 times for WOx memristor in 3D stacked array. e Endurance cycling test of WOx memristor over 105 cycles by programming at 1.8 V and −1.8 V for W = 500 μs. ION and IOFF were read at VREAD = 0.5 V. f Operating voltage range |V| for WOx memristor (red box) and other self-rectifying memristors. g Dynamic I behavior over time t for WOx memristor after applying voltage pulses of 1.8 V for W = 1 ms, 100 μs, and 50 μs, demonstrating a self-decaying nature (i.e., short-term memory) with different to (inset).

Full size image

Figure 2d, e show the consecutive switching curves for more than 4000 times and over 105 cycles with an ION/IOFF ratio of approximately 5 × 102, respectively, demonstrating an excellent operating stability with average cycle variation of 0.24% (Supplementary Fig. 10). In addition, the multiple 3D stacked WOx PRs exhibit the lowest operating voltage range (at least |0.7 | V, Fig. 2f and Supplementary Fig. 11) with robust self-rectification (Supplementary Fig. 12), indicating the possibility of lower power consumption than existing self-rectifying memristors31,33,34,35,36,37,38,39,40,41,42,43,44,45. Figure 2g shows the current responses of the memristor cell over time after applying single voltage pulses of different widths (W ). The current abruptly increases close to ION soon after applying a single voltage pulse and then decays spontaneously and exponentially over time (t) under VREAD (i.e., I ∝ exp(−t/to), where to is the characteristic decay time), eventually returning to IOFF. This behavior was still observed to persist despite the application of strong programming voltage scheme in both air and vacuum environments (Supplementary Fig. 13). This spontaneous decay in the switching current cannot be attributed to the protonation-assisted switching mechanism, as the metallic HxWO3 phase is absent and the switching occurs over a short period of time46. This self-decaying nature (i.e., short-term memory) can describe time-dependent internal states that enable temporal processing. Because width W can control to, a 3D stacked WOx memristor cell can process time-series data over varying time frames. Note that to was observed to decrease as the W decreased based on a power function (Supplementary Fig. 14). Consequently, as compared with other reported memristor devices47,48,49,50, our proposed WOx memristor can exhibit highly reliable selector-less short-term switching performances, operational advantages, and scalability for showcasing the 3D-integrated multilayered PR arrays.

Switching mechanism of the 3D physical reservoirs

The interfacial barrier heights were investigated according to the ION and IOFF states to understand the self-rectifying switching behavior of the 3D stacked WOx memristor cell. Initially, the Pt/WOx/W memristor was designed based on the asymmetric distribution of oxygen vacancy (VO) in WOx between Pt and W electrodes. Figure 3a shows the ex-situ X-ray photoelectron spectroscopy (XPS) results with timed Ar+ bombardment for the W ion peaks at the top and bottom WOx interfaces. Compared with the top WOx interface, there are two peaks corresponding to W5+ and W0 at the bottom WOx interface (top and bottom of Fig. 3a)51,52. This indicates the enrichment of VO in the direction of the bottom WOx interface, which increases the concentration of VO (\({C}_{{V}_{{{{{{\rm{O}}}}}}}}\)) at the WOx/W interface. Because the Fermi level of WOx increases (decreases) with high (low) \({C}_{{V}_{O}}\)53, asymmetric barrier heights \({\varPhi }_{B}^{1}\) and \({\varPhi }_{B}^{2}\) at the WOx/W and Pt/WOx interfaces can be formed. A detailed discussion and additional XPS results are presented in Supplementary Note 5 and Supplementary Figs. 15, 16. The combined effects of the sputtering parameters and electrodes were also investigated (see the Methods section and Supplementary Figs. 17–19). The structural analysis results are consistent with the visual inspection of WOx (Supplementary Fig. 20).

Fig. 3: Switching mechanism of 3D Pt/WOx/W memristor.
figure 3

a W 4 f XPS results taken from top and bottom interfaces of WOx layer, exhibiting asymmetric profile in concentration of oxygen vacancies (\({C}_{{V}_{O}}\)) depending on the position of the WOx layer. b Arrhenius plot of ln(I/T2) according to 1/kBT for ION and IOFF states of Pt/WOx/W memristor. Variables \({\varPhi }_{B}^{1}\) (blue) and \({\varPhi }_{B}^{2}\) (red) indicate each Schottky barrier height at the W/WOx (bottom) and Pt/WOx (top) interfaces, as shown in the inset. The changes in \({\varPhi }_{B}^{1}\) and \({\varPhi }_{B}^{2}\) were estimated from the slopes of ln(I/T2) according to 1/kBT for the ION and IOFF states of the Pt/WOx/W memristor, respectively. c Changes in estimated \({\varPhi }_{B}^{1}\) and \({\varPhi }_{B}^{2}\) for ION and IOFF states. The inset shows the different band diagrams for the ION and IOFF states and state-dependent \({C}_{{V}_{O}}\). d, e Conductive atomic force microscopy (c-AFM) images of current distributions in 1 × 1 μm2 WOx/W for tip bias (Vbias) = 3 V (d) and Vbias = −3 V (e). Note that spatial current distribution (\(\bar{I}\)) was found to be 2.63 nA (d) and 8.45 pA (e) on the average. The scale bar for both (d, e) is 100 nm. f Statistical ION and IOFF values for 15 WOx memristors according to memristor area.

Full size image

Figure 3b shows the temperature (T)-dependent ION and IOFF states of the Pt/WOx/W memristor and their interfacial barrier heights (\({\varPhi }_{B}^{1}\) at WOx/W and \({\varPhi }_{B}^{2}\) at Pt/WOx) through the Arrhenius plot. Based on the fitting results shown in Fig. 3b (solid lines; see the Methods section), \({\varPhi }_{B}^{1}\) and \({\varPhi }_{B}^{2}\) are shown according to the ION and IOFF states in Fig. 3c. The asymmetry between \({\varPhi }_{B}^{1}\) and \({\varPhi }_{B}^{2}\) contributes to the self-rectifying feature shown in Fig. 2a. A larger change in \({\varPhi }_{B}^{1}\) than in \({\varPhi }_{B}^{2}\) is observed during the switching transition, implying that charge transfer through \({\varPhi }_{B}^{1}\) is dominant in determining the switching states. When a positive voltage is applied to the Pt electrode, VO migrates downward, and \({C}_{{V}_{O}}\) increases at the bottom WOx interface28,29. In that case, \({\varPhi }_{B}^{1}\) is largely decreased at VREAD = 0.5 V; then, the conductance state is switched to ION. When a negative voltage is applied to the Pt electrode, the behaviors occur inversely, and the conductance state switches to IOFF. Additional experimental results for different thicknesses of WOx under ambient or vacuum environments and the structural analysis of WOx are described in the Supplementary Information (Supplementary Figs. 17, 21–23, Supplementary Note 5). Note that there is a possibility that electron trapping/detrapping process may partially contribute to the switching mechanism, but it is likely to have a minor effect on the switching compared to the electric field-driven VO migration (Supplementary Fig. 24 and Supplementary Note 5).

Figures 3d and 3e show maps of the spatial current distribution (\(\bar{I}\)) on the Pt-tip/WOx/W junction structure according to the tip bias (Vbias = ±3 V) using conductive atomic force microscopy (c-AFM). The estimated contact radius of the Pt tip is approximately 2.1 nm (Supplementary Fig. 25). Details of the c-AFM measurements are shown in Supplementary Fig. 25 and Methods section. The average values of \(\bar{I}\) are 2.63 nA and 8.45 pA at Vbias of 3 V and −3 V, respectively, demonstrating a uniform change at the whole scanning area without evident current hotspots (i.e., conductive filaments)54. The I–V switching feature at the nanoscale using c-AFM is consistent with the switching behaviors observed in Fig. 2 (Supplementary Fig. 25). Figure 3f shows statistical ION and IOFF according to the memristor size (with a constant ION-IOFF ratio, Supplementary Fig. 26), demonstrating the homogeneous enrichment and depletion of VO near the WOx interfaces55. Note that dependency of the ION/IOFF on the size is almost negligible (Supplementary Fig. 26).

Physical reservoir computing for multiple time-series data

Temporal processing of multiple dynamic inputs was quantitatively evaluated considering the nonlinear dynamics and short-term memory of 3D stacked WOx PRs (Fig. 4 and Supplementary Figs. 27–30). Figure 4a shows the change in PR states (i.e., current responses) of the memristor cell in response at each timestep (t1–t4) over a voltage pulse sequence. Pulse sequence [1111] indicates that a voltage pulse (VINPUT = 1.8 V for 200 ns) is sequentially applied at each timestep (t1–t4), whereas sequence [0000] indicates that no voltage pulses are applied. Hence, 16 reservoir states are successfully programmed at t4 without notable cycle variations (10 cycles per sequence, Supplementary Fig. 27). These distinct reservoir states may be attributed to the self-rectifying properties and the consistent cycling switching. The trajectories of change in the reservoir states by the voltage pulse sequences are similarly observed on the memristors from the first to third layers of 3D stacked WOx PRs (Supplementary Fig. 28).

Fig. 4: Physical reservoir computing for cell position classification.
figure 4

a Change in physical reservoir (PR) states of memristor cell at read voltage (VREAD) = 0.5 V per timestep (t1–t4) and 16 voltage pulse sequences ([1111], [0111], …, [0000]). A voltage pulse (VINPUT) is 1.8 V for 200 ns with interval (P) = 7 μs. Each reservoir state at t4 is clearly separated regardless of the cycle variation. b, c Illustrations of biological cell showing nucleus, cytoplasm, and background (b), and its corresponding 5 × 5-pixel image (c). d Schematics of five cell positions: top left (TL), top right (TR), center (C), bottom left (BL), and bottom right (BR). e, f Schematics of three 5 × 5-pixel images of nucleus, cytoplasm, and background for case C (e), and their corresponding time-series sequences applied to three PR layers (nucleus for third layer, cytoplasm for second layer, and background for first layer) (f). The time-series inputs were voltage pulses converted from each local feature and introduced into different locations of the memristors in each layer (5 memristors per layer for the spatial domain) at different times (t1–t5 in the temporal domain). g Final reservoir states at t5 obtained from first to third reservoir layers (5 memristors per layer). h Schematic of reservoir computing (RC) configuration consisting of input layer for time-series data, three PRs, and output layer. Weights (\({W}_{n}^{i}\)) between PRs and output layer was trained to reduce the error (i.e., loss function) by comparing outputs (yn(t)) and the desired value. i Loss functions for single (navy) and multiple (yellow) reservoirs according to training epoch. The inset shows the classification accuracies at 100 epochs and numbers of used memristors for the single and multiple reservoirs. j, k Confusion matrices for single (j) and multiple (k) reservoirs.

Full size image

To demonstrate the capability of 3D stacked WOx PRs to process multiple time series without mutual interference, a detection task of moving biological cells was conducted based on their layer separation in nucleus, cytoplasm, and background, as shown in Figs. 4b–k56. Figs. 4b, c show a centered cell image (Fig. 4b) and its simplified 5 × 5-pixel image (Fig. 4c), where the blue, yellow, and black pixels represent the nucleus, cytoplasm, and background, respectively. As shown in Fig. 4d, the moving cell can be localized in one of five possible cell positions with respect to its nucleus: top left (TL), top right (TR), center (C), bottom left (BL), or bottom right (BR). In three layers of 5 × 5-pixel images corresponding to the nucleus, cytoplasm, and background images, each layer is encoded according to the five position cells using different voltage sequences in the spatial and time domains. For the centered cell (C) in Figs. 4e, f, various voltage sequences for the five WOx memristors in each layer are encoded. The multiple 3D stacked WOx PRs can individually capture three local features of the biological cell from the first to third layers. Figure 4g shows the final reservoir states at the center position in terms of these three features. The features are spatiotemporally distinguishable, demonstrating the success of feature extraction using dynamic cell positioning. The results for the other cell positions (TL, TR, BL, and BR) are shown in Supplementary Fig. 29.

When each spatiotemporal input for a cell position enters each reservoir layer, the reservoir state, \({x}_{n}^{i}\), in each layer (i = 1, 2, 3) is generated. It is then connected to a digital output layer (y1(t), y2(t), y3(t), y4(t), and y5(t)) to perform learning based on the simple single-layer gradient descent (Fig. 4h). Details about learning and prediction are provided in the Methods section. Figure 4i shows the loss function over epoch for a single reservoir and multiple reservoirs. As shown in Fig. 4i, the loss function for multiple reservoirs rapidly decreases with the number of epochs (~4.5 times faster), leading to a higher classification accuracy. Note that the further discussion on the convergence speed is described in Supplementary Note 6. In this detection task, the accuracy of the multiple 3D stacked WOx PRs is 100% at 100 epochs, being 7% higher than that of a single reservoir. Thus, for multiple PRs, only the diagonal boxes are fully saturated with yellow in the confusion matrices for the classification results between the actual and predicted positions (Fig. 4k). However, for a single reservoir, the blue saturation of several off-diagonal boxes is observed, indicating incorrect predictions (Fig. 4j). In addition, a single reservoir requires a larger number of memristors (25) than multiple reservoirs (15) for this detection task, and it cannot distinguish between the nucleus and cytoplasm at the overlapping position (Supplementary Fig. 30). Therefore, multiple PRs can extract rich features from time-series data because each reservoir learns distinct local features, even when using 60% fewer memristors than a single reservoir, thus surpassing the performance of a simple increase in packing density. Additional discussions on the wide RC are described in Supplementary Note 2, 6.

As another example of physical RC, we considered a time-dependent Lorenz attractor, which is a nonlinear and deterministic 3D chaotic dynamic system for representing convection in the atmosphere that is difficult to predict57. We attempted the Lorenz attractor prediction and simulation using multiple 3D stacked WOx PRs as a proof of concept. Figure 5a shows the 3D trajectory obtained from the Lorenz attractor equations with specific parameters (see the Methods section). As shown in Figs. 5b, c, by deconvoluting the 3D Lorenz equation into three time-dependent one-dimensional equations (i.e., x-, y-, and z-axis components over a timestep) (Fig. 5b), the 3D stacked WOx PRs could directly handle their corresponding chaotic inputs to generate separable reservoir states, which were then transferred to the output layer (Fig. 5c).

Fig. 5: Physical RC for prediction of time-dependent Lorenz attractor.
figure 5

a 3D trajectory of Lorenz attractor in the x, y, and z components. b, c One-dimensional time-dependent Lorenz attractors over time deconvoluted from 3D Lorenz attractor (b), and schematic of three physical reservoirs (PRs) for prediction of the 3D Lorenz attractor (c). d Actual and predicted behaviors of 3D Lorenz attractor after 1400 timesteps. e Individual actual behaviors of x, y, and z components over time (colored solid lines), and their corresponding prediction results (red dotted lines) using multiple 3D stacked 3 ×10 × 10 WOx PRs after 1400 timesteps (blue dotted line). f Normalized mean squared error (NMSE) between actual and predicted behaviors at timestep 2788 for single (navy) and multiple (yellow) reservoirs in the x, y, and z components.

Full size image

Based on the linear regression algorithm, the learning process was performed at the output layers considering 100 nodes per reservoir (300 nodes in total). As shown in Fig. 5d, after initialization and learning over 1400 timesteps, the predicted behavior closely matches the actual behavior with a negligible deviation. Figure 5e shows the amplitudes of the x, y, and z components over time for the actual and predicted behaviors (2788 timesteps). Hence, we demonstrate the successful prediction of a time-dependent 3D Lorenz attractor using multiple 3D stacked 3 × 10 × 10 WOx PRs. The average normalized mean squared error (NMSE) between the predicted and actual behaviors is approximately 2.62 × 10−4, being >10 times better than that when using a single reservoir with 300 nodes (NMSE of 1.35 × 10−3) (Fig. 5f and Supplementary Fig. 31). If the reservoir capacity is insufficient, even when considering multiple reservoirs, the predicted behavior substantially deviates from the actual behavior (Supplementary Fig. 32). Details about prediction are provided in the Methods section. The results indicate that the optimized \({W}_{n}^{i}\) in the output layer obtained from learning allows to suitably predict the future behavior in response to multiple reservoir states (three layers) at the current timestep. As a result, 3D stacked WOx memristor arrays promote the prediction accuracy and efficiency of physical RC for multiple time-series processing. Note that, in Supplementary Note 6 and Supplementary Figs. 33, 34, we show relevant discussions of physical RC based on multiple 3D stacked WOx PRs as compared with traditional approaches and suggest how they could be used at the circuit level.

Discussion

We design and fabricate a 3D stacked three-layer 3 × 10 × 10 WOx memristive crossbar array to implement a wide physical reservoir system for processing multiple time-series information. This design exhibits a forming-free feature, high switching uniformity and device yield (>98%) for a three-layer stacked array, robust endurance (>105 cycles), and low programming voltage (~0.7 V) for self-rectifying switching. Moreover, it demonstrates essential reservoir properties, including separation and memory fading. The 3D stacked WOx memristive crossbar array for wide physical RC allows each array layer to individually portray and process distinct local features of spatiotemporal inputs such that the reservoir states can be readily analyzed at the output layer for classification and prediction tasks. By leveraging this strategy, for dimensional classification of biological cell positions, the multiple PRs achieved better classification accuracy (~100%) and ~4.5 times faster convergence using 60% fewer memristors (15) compared with a single reservoir (93% accuracy with 25 memristors). In addition, for the prediction of the chaotic Lorenz attractor, they achieved a tenfold lower prediction NMSE (~2.62 × 10−4) than a single reservoir (NMSE of 1.35 × 10−3). Although the array size is still small as a proof-of-principle demo, both the principles and device structure are applicable to large scales. The proposed 3D stacked WOx PR array represents a leap forward toward an efficient wide RC platform, promising for time-series processing in edge computing.

Methods

Fabrication of 3D stacked three-layer memristive crossbar array

The 3D stacked three-layer 3 × 10 × 10 WOx memristive crossbar array was fabricated on either a glass substrate or 285 nm-thick SiO2/p++-Si substrate (1.5 cm × 1.5 cm). First, the substrates were prepared using a conventional cleaning process with acetone, isopropyl alcohol, and deionized water via ultra-sonication for 3 min each. To form 1st bottom electrode lines (< ~10 nm thick) on the substrate, W was deposited by sputtering system with 5 min pre-sputtering step, gas flow of Ar 10 sccm, working pressure of 10 mTorr, and gun power of 100 W with the patterned shadow mask (1.5 cm × 1.5 cm) with line width of 100 μm. Note that all the following fabrication processes were performed through sputtering and shadow masks to circumvent the sidewall issue that is critical for the 3D stacked structure. The same-sized shadow masks with the substrate were used to facilitate the patterning alignments. Subsequently, the WOx (~180 nm thick) was deposited on the center of the 1st bottom W electrodes except for the contact pads. The patterning alignment was carefully carried out with an optical microscopy. During WOx deposition, various combinational sputtering parameters were used, such as 20 min pre-sputtering step, gas flow of Ar 60 sccm and O2 1 sccm, working pressure of 20 mTorr, gun power of 150 W, and substrate temperature of 40 °C, to optimize a nonlinear dynamic switching with self-rectification (see Supplementary Fig. 17). Perpendicularly to the 1st bottom W electrode lines, the 1st top Pt electrode lines (< ~10 nm) were formed on the 1st WOx layer by using a shadow mask and sputter. The Pt was deposited with 5 min pre-sputtering step, gas flow of Ar 10 sccm, working pressure of 10 mTorr, and gun power of 100 W. After the Pt deposition, W was directly sputtered on the 1st top Pt electrodes to form 2nd bottom W electrode lines. The 2nd WOx layer was then formed on the 2nd bottom W electrode/1st top Pt electrode lines, except for the all electrode pads. The same sputtering parameters used during the formation of 1st WOx were employed. Similar fabrication steps were repeated until completing the 3D stacked of the three-layer memristive crossbar array. The top-view optical images of the fabrication processes and corresponding schematics are shown in Supplementary Fig. 3.

Structural analysis

To identify the formation of 3D stacked junction structure at a certain crosspoint, the sample for TEM and EDS analysis was prepared via focused ion beam technique (FIB, Helios G4 HX) and then cross-sectional TEM image and EDS results of the 3D stacked memristors were obtained by using a FEI Double Cs and monochromated TEM instrument. The depth-profiling XPS analysis (PHI 5000 VersaProbe) was performed by using 100 μm-diameter sized single Pt/WOx/W memristor (80-nm thick WOx).

Electrical characterization

A semiconductor parameter analyzer (4155 C, Agilent), pulse generator (81104 A, Keysight), and low-leakage switch mainframe (E5250A, Keysight) were used to electrically investigate and characterize the I–V switching curves, consecutive sweeps, endurance cycles, voltage ranges, temperature-dependent Schottky barrier height, and size dependency of the 3D stacked three-layer 3 × 10 × 10 WOx memristive crossbar array. Memristors in the 3D stacked arrays were randomly accessible when contacting their corresponding electrode pads. For example, to select a certain memristor of the 1st layer, the 1st bottom W electrode pad was grounded, whereas the 1st top Pt & 2nd bottom W electrode pad was biased. Considering that the 1st top Pt electrode is underneath the 2nd bottom W electrode, the Pt/WOx/W memristor of the 1st layer can be characterized. This measurement can be considered a reasonable method because our WOx-based memristors operate based on interfacial switching. The time-decay features and current responses (i.e., reservoir states) as functions of various time-dependent inputs were measured by using a Keithley 4200 semiconductor characterization system that can program arbitrary voltage schemes into memristors. To investigate self-decaying nature, a VREAD of 0.5 V for 5 μs was first applied into the selected memristor in the 3D stacked array to confirm IOFF state. Then, a programming pulse with amplitude of 1.8 V for pulse widths (W) = 1 ms, 100 μs or 50 μs, which can switch the device into ION state, was introduced into the memristor. The rising and falling edges of the pulses were set to be 500 ns. After the programming, a VREAD of 0.5 V was used to investigate the current decaying behaviors over time. To generate reservoir states, time-varying voltage pulse schemes consisting of four timesteps were used (Fig. 4a). As shown in the upper panel of Fig. 4a, 16 binary encoding sequences (1 or 0) consist of 4 timesteps (t1–t4) at P = 7 μs, and each timestep contains either a VINPUT of 1.8 V for W = 200 ns when its corresponding encoding is 1 or no pulse when its corresponding encoding is 0. The rising and falling edges of the programming pulse were set to be 50 ns. After each timestep, VREAD of 0.5 V at the time interval of 5 ns was used to record reservoir states (i.e., current behaviors). Note that for the random access, the same method used in the electrical characterization was employed.

Conductive atomic force microscopy

To identify the filament-free and homogeneous switching phenomena of the WOx memristor, conductive atomic force microscopy (c-AFM) measurement was performed. In order to map the spatial current distribution (\(\bar{I}\)), a Pt/Ir-coated c-AFM tip was used to directly scan the top surface of the WOx/W sample and configure the Pt/WOx/W junction structure (Supplementary Fig. 25). The Pt/Ir-coated tip was set to be grounded, and the bottom W electrode was set to be biased; hence, tip bias (Vbias) was represented as the opposite bias polarity. The tip (~24 nm) and estimated contact radii (~2.1 nm) are discussed in the Supplementary Information (Supplementary Fig. 25). The electrical characteristics were measured in scanning mode with a scan area of 1 × 1 μm2 by using a DLPCA-2000 built-in current amplifier (Electro-Optical Components) under a humidity below 15%. After the Pt/Ir tip contacted the WOx/W samples, −3 V and 3 V were applied to the bottom W electrode during scanning, respectively. Then, the spatial current distribution (\(\bar{I}\)) at Vbias = 3 V and −3 V were mapped on the 1 × 1 μm2 scan area and it was verified that there is no occurrence of a conductive filament.

Estimation of Schottky barrier height

To determine the Schottky barrier heights of the Pt/WOx/W memristor, temperature-dependent I–V curves were investigated and plotted according to the following thermionic emission equations:

$$I=A{A}^{*}{T}^{2}\exp \left(\frac{-q{\varPhi }_{B}}{{K}_{B}T}\right),$$
(1)
$${ln}\frac{I}{{T}^{2}}={{{{{\rm{ln}}}}}}(A{A}^{*})-\frac{q{\varPhi }_{B}}{{K}_{B}T},$$
(2)

where A, A*, T, q, ΦB, and KB are the effective area, Richardson constant, temperature, electronic charge, Schottky barrier height, and Boltzmann constant, respectively. Hence, \({\varPhi }_{B}^{1}\) and \({\varPhi }_{B}^{2}\) for the ON and OFF states at VREAD = ± 0.5 V are quantitatively estimated from the slope of the straight fitting lines shown in Fig. 3b.

Dimensional classification of biological cell positions

The dimensional classification of five biological cell positions was simulated based on the individual PR states encoded from each layer of the 3D stacked WOx array. Three layers of 5 × 5-pixel images corresponding to the nucleus, cytoplasm, and background images are individually encoded by each layer of the 3D stacked WOx array. A Keithley 4200 system, capable of generating arbitrary voltage pulse patterns, was employed to measure the reservoir states (i.e., current responses) at each timestep (from t1 to t5) by applying the spatiotemporal inputs shown in Figs. 4e–g. Five memristors per layer were selected in each 10 × 10 crossbar array for spatial information processing of the biological cell, and all reservoir states were read at VREAD past 5 μs after the application of each programming pulse (1.8 V for 200 ns). The programming and reading disturbances in the 3D stacked 3 × 10 × 10 WOx array can be minimized owing to the self-rectifying function. The reservoir states of the different memristors in each layer were individually recorded by applying time-dependent voltage pulses to the top Pt electrode line of the selected memristor with a grounded bottom W electrode line.

Subsequently, a single-layer neural network with a supervised learning algorithm was employed as the output layer. By considering the PR states at the final time frame (t5) as input x, output y was generated through the softmax activation function:

$${y}_{j}=\frac{{e}^{\, {z}_{j}}}{\mathop{\sum }\nolimits_{k=1}^{j}{e}^{{z}_{k}}},{z}_{j}={W}_{i,j}\cdot {x}_{i}+b,$$
(3)

where Wi,j is the synaptic weight between a reservoir and output layer, b is the bias, and i and j are integers identifying the reservoir nodes and output neurons, respectively. The number of output neurons was five, corresponding to the considered cell positions (top left (TL), top right (TR), center (C), bottom left (BL), or bottom right (BR)). During learning, the cross-entropy loss function (ε) was decreased using delta rule of gradient descent as follows:

$$\varepsilon=-\frac{1}{n}\mathop{\sum }\limits_{m=1}^{n}\mathop{\sum }\limits_{k=1}^{j}{\hat{y}}_{k}^{(m)}\log (\, {y}_{k}^{(m)}),$$
(4)
$${W}_{i,j}\equiv {W}_{i,j}-\eta \frac{\partial \varepsilon }{\partial W},$$
(5)

where n is the number of input data points, \({\hat{y}}_{j}\) is the one-hot encoding of yj, and η is the learning rate. After learning, additional images were generated per cell position and used during prediction (Supplementary Fig. 35), and also their corresponding PR states were produced based on the fitting results (Supplementary Fig. 36).

Prediction of time-dependent Lorenz attractor

The 3D Lorenz attractor is a set of chaotic solutions of the Lorenz system described by three ordinary differential equations according to the axis:

$$\frac{{dx}}{{dt}}=\alpha ( \, y-x),$$
(6)
$$\frac{{dy}}{{dt}}=x\left( \, \beta -z\right)-y,$$
(7)
$$\frac{{dz}}{{dt}}={xy}-\gamma z.$$
(8)

These equations can describe various chaotic behaviors and physical phenomena over time depending on the values of parameters α, β, and γ. By setting α = 10, β = 28, and γ = 8/3 for the chaotic behaviors, the Lorenz attractor was normalized and deconvoluted into time-varying amplitudes for the x, y, and z components. These values were applied into each PR after normalizing the inputs in the operating voltage range from 0 V to 1.8 V, as shown in Figs. 5b, c. At each reservoir, 100 memristors were utilized and simulated to enrich the reservoir states because some device variations can produce qualitatively similar but quantitatively different current responses (Supplementary Fig. 28 and Supplementary Note 2, 6). The reservoir states obtained from the three PRs were fed into the corresponding output layers, and the weights were optimized using the supervised learning algorithm of linear regression with the pseudo inverse:

$$W={\left({X}^{{{{{{\rm{T}}}}}}}X\right)}^{-1}\,{X}^{{{{{{\rm{T}}}}}}}Y$$
(9)

X, W, and Y are a set of reservoir states, weights, and actual behaviors, respectively. After initialization during the initial 100 timesteps, the weights were updated until 1400 timesteps. Initialization was required to prepare and assist the prediction processing of the RC system without learning because the initial conditions of the Lorenz attractor could strongly affect its future behavior. When learning was complete, the actual behaviors at the current timestep (t) were used to predict the behaviors at the next timestep (t + 1).

Normalized mean squared error

The normalized mean squared error (NMSE) used for performance evaluation was calculated as

$${{{{{\rm{NMSE}}}}}}=\frac{{\sum }_{t}{\sum }_{j}{({y}_{j}\left(t\right)-{t}_{j}\left(t\right))}^{2}}{{\sum }_{t}{\sum }_{j}{t}_{j}^{2}(t)},$$
(10)

where tj(t) and yj(t) are the actual and predicted behaviors, respectively.