Introduction

Chaos has been widely studied for decades in physics, mathematics, and engineering1,2,3,4. Since chaos is generally defined as aperiodic dynamical behavior of deterministic systems that exhibit a high sensitivity to initial conditions1, it could be considered predictable in a mathematical sense, assuming all relevant information about the system is known. In practical terms, however, it is extremely difficult to accomplish this task hindered by unknown factors such as noise and interactions with the surrounding environment. Therefore, it remains a formidable challenge to predict chaotic behavior in practice.

In recent years, data-driven approaches have been employed to analyze physical systems, such as finding the parameters/functions of governing equations5,6,7,8. Recurrent neural networks (RNNs) constitute a powerful machine learning approach for processing and predicting time-series data9,10,11 (see Supplementary Fig. 1a for schematic illustration of a standard RNN). Due to such capabilities, RNNs or their variations have been applied for dynamics problems12,13. In particular, these neural networks with recurrent connections have shown successful attempts to predict the dynamics of chaotic systems14,15,16,17,18,19. However, RNNs are usually considered to be a “black box” for learning and predicting time series. Thus, interpretation of the neurons’ processing a time series, especially for chaotic data, has remained elusive. This is partly because the activation for each neuron in an RNN for the current time step depends on the activation of every other neuron at the previous time step (denoted by a red circle in Supplementary Fig. 1b). This deep level of coupling interaction makes it challenging to extract meaningful information about the effects of individual neurons.

More recently, quasi-recurrent neural networks (QRNNs) have been developed, particularly for natural language analysis (see Supplementary Note 1 and Supplementary Fig. 1c)20. They exhibit faster processing of time-series data and competitive performance compared with other RNNs. Most notably, the hidden states of a QRNN can be readily visualized and interpreted without additional processing (e.g., introducing self-attention to visualize how the input data are processed21). The QRNN is composed of convolutional layers which process time-series data in parallel across each time step, and pooling layers in which recurrent relations can be implemented (Supplementary Fig. 1c). Due to the element-wise calculation in the pooling function, activation of each neuron does not depend on the past outputs of other neurons in the same pooling layer.

Here, we demonstrate not only prediction of both periodic and chaotic data, but also analysis of hidden units’ distinctive responses to such dynamic conditions by using the QRNN. To examine the effectiveness of our approach, especially in the experimental context, we need to produce unique sets of dynamic data containing periodic, subharmonic, and chaotic trajectories in a controllable manner. One of the examples showing such behavior is a structure with intrinsic bistability22,23,24,25. In this study, we design and fabricate a versatile bistable mechanical system based on origami unit cells, specifically triangulated cylindrical origami (TCO)26. These TCO cells, in serial connection, can provide highly tunable properties27,28,29 as well as multi-degree-of-freedom nature, thus providing an ideal playground to examine the efficacy of our data-driven prediction. Based on the experimentally measured time-series data, we explore the feasibility of extracting meaningful system information, such as whether the dynamic response of our origami system is chaotic or not, from our data-driven approach. Since no mathematical model of the system nor the knowledge of its dynamical nature (e.g., definition of chaos) is required, our data-driven approach is model-free and can be used to analyze complex dynamics in absence of prior knowledge of the underlying physics of a system.

Results

Experimental demonstration of chaotic behavior of the triangulated cylindrical origami

Origami has been extensively studied recently due to its tailorable static responses. We show that a unique TCO-based platform can produce rich data sets from its complex dynamics, especially chaos, thus enabling to examine the effectiveness of our data-driven approach. One of the interesting features of the TCO is that its axial and rotational motions are coupled with each other (see Fig. 1a where the folding sequence of the TCO unit cell is depicted, and also Supplementary Movie 1 for folding animation). Figure 1b shows the flat sheet with crease patterns of the TCO and its folded shape. To describe the initial shape of the TCO unit cell, we define the initial height (h0), initial rotational angle (θ0), and radius of the cross-section (R) as shown in Fig. 1c, d.

Fig. 1: Folding behavior of the triangulated cylindrical origami (TCO) unit cells.
figure 1

a Folding sequence of the TCO. bd The flat sheet with crease patterns of the TCO (b) is folded into a 3D cylindrical shape (c). The red lines (e.g., \(\overline{{A}_{a}B}\)) and blue lines (e.g., \(\overline{{A}_{b}B}\)) indicate mountain and valley crease lines, respectively. The parameters to define the initial shape are the initial height (h0), initial rotational angle (θ0), and radius of the cross-section (R) as shown in the top-down view (d). Actual prototype of the TCO unit cells with (h0θ0R) = (50, 70°, 36) in its initial (e) and second (f) stable configuration. g Energy landscape as a function of compressive displacement (δ), which is calculated from static experiments on the paper prototypes, shows the bistable behavior, i.e., there exists two energy minima denoted by the triangle markers. The distance between these two energy minima is denoted by Lb. Here, the energy (U) is normalized by the height (h0) and stiffness (K) at the initial unstretched state (i.e., δ = 0).

To analyze the potential energy, we fabricate prototypes by using construction paper sheets cut by a laser cutting machine (see “Methods”; Supplementary Movie 2). Figure 1e, f shows our fabricated prototype of the TCO unit cell with (h0θ0R) = (50 mm, 70°, 36 mm) in the first and second stable states, respectively. We first conduct quasi-static cyclic loading tests on this paper prototype (see Supplementary Fig. 2) to examine and enhance the repeatability of folding/unfolding behavior. After this cyclic loading, we extract the force–displacement relationship from compression tests and obtain the energy curve as shown in Fig. 1g. Here, the energy is normalized by the initial height (h0) and stiffness (K) at the initial unstretched state, i.e., no axial displacement δ = 0 (see the inset illustration in Fig. 1g). We observe the bistable behavior, such that the TCO unit cell possesses two local minima in its energy landscape where the normalized distance between these two stable states (Lb) is 0.35 (see the x-axis in Fig. 1g). This characteristic distance will be used to aid in analyzing the dynamic behavior of the TCO.

Forced dynamic tests of a system of two connected TCO unit cells are used to create the chaotic response data sets. The unit cells have properties (h0θ0R) = (50 mm,  ±70°, 36 mm), with the left-most cross-section attached to a shaker that generates harmonic excitation (see Fig. 2a; “Methods”). The experiment is conducted for different excitation frequency ranging from 5 to 25 Hz. The folding motion of these two TCO unit cells is measured by two action cameras together with a customized digital image correlation program. In the experiments, we measure displacement (ui) and rotational angle (φi) of each cross-section (i = 0, 1, 2, see Fig. 2b for the notation). We use these measured data as well as the velocities, \({\dot{u}}_{i}\) and \({\dot{\varphi }}_{i}\), numerically calculated from ui(t) and φi(t). These measured data are separated into two data sets, the first of which is used for training, and the second for testing (Fig. 2b).

Fig. 2: Dynamic testing on the two-triangulated cylindrical origami (TCO) structure.
figure 2

a Actual testing set up for the vibration test. The structure consists of two TCO unit cells with the design parameters (h0θ0R) = (50 mm,  ±70°, 36 mm) where h0, θ0, and R are the initial height, initial rotational angle, and radius of the cross-section, respectively. The input excitation is applied by a shaker, and the folding motion of these two TCO unit cells is captured by two action cameras, together with a customized digital image correlation program. b Conceptual illustration of operation on the measurement data for training and testing the quasi-recurrent neural network (QRNN). From the digital image correlation, we obtain displacement (ui where i = 0, 1, 2) and rotational angle (φi where i = 0, 1, 2) from the equilibrium position in the first stable state. The measured data are separated into two data sets: training data and testing data. The experiment was conducted for different excitation frequency (fex) ranging from 5 to 25 Hz. Each data set contains n = 12 components (\({u}_{i},{\dot{u}}_{i},{\varphi }_{i},{\dot{\varphi }}_{i}\) for three different cross-sections). c Classification of different folding motions (chaotic, interwell, and intrawell periodic) of the bistable TCO unit cell through the normalized potential energy \(U/K{h}_{0}^{2}\) (green line), where U and K are the potential and stiffness, respectively. The three different gray colored curves with the arrow heads represent oscillatory motions in potential wells. d, e Measurement results for the excitation frequency of 12 Hz. The displacement of the left-most section (u0), which is attached to the shaker, shows sinusoidal waves, while u1 indicates chaotic motion (d). Let δi = ui−1 − ui, we plot a phase portrait as a function of δ1 and its derivative \(\dot{{\delta }_{1}}\), in which blue dots are all measurement data, while red dots represent Poincare map (e).

A schematic diagram of the dynamic folding behavior is shown in Fig. 2c overlaid on the underlying double-well potential energy landscape. We can define three different regimes24: intrawell, interwell (periodic), and interwell (chaotic) vibrations as shown in Fig. 2c. The intrawell oscillation means that the system exhibits small oscillations about one of the two local potential minima. If the system overcomes the energy barrier and goes to the other stable state, we observe the interwell vibrations, which is typically either periodic or chaotic. Quasi-periodic responses may also occur in nonlinear systems, though that was not observed in the origami experiments. Figure 2d shows the measurement results for an excitation frequency (fex) of 12 Hz. The displacement of the left-most section (u0), which is attached to the shaker, shows input sinusoidal waves. However, u1 indicates chaotic motion. Here we define δi = ui−1 − ui and plot a phase plane for δ1 in which blue dots are all measurement data and red dots represent Poincaré map30 as shown in Fig. 2e. Note that in the Poincaré map, periodic oscillations will show a single fixed point, or N-points for a N-periodic response, whereas chaotic behavior will lead to a large collection of points. The horizontal axis is normalized by Lb. Therefore δi/Lb = 1 indicates that the TCO unit cell transits to the second stable regime. It should be noted, however, that it is possible for a forced dynamic response to transit over the unstable potential hilltop, and reverse course without ever reaching the stable potential minimum. Thus, this normalization provides only a nominal indicator of a completed transit between the two stable equilibria.

The experiment result for fex = 12 Hz clearly shows that the vibrations take place not only around the first stable state, but also around the other energy local minimum state aperiodically, which corresponds to the chaotic interwell vibration. This manifests the capability of our TCO system to form chaotic dynamics (further analysis on this chaotic behavior, specifically Lyapunov exponent calculation, to be explained later). Although demonstrations of chaotic behaviors of origami structures with single degree of freedom have been reported25,31, our system is notable in that we experimentally create the chaotic structural vibrations by using multi-DOF origami structures (see Supplementary Movie 3 for experimental measurements for different excitation frequencies).

Prediction based on quasi-recurrent neural networks

Based on the experimentally measured data, we study a data-driven approach to predict multi-DOF folding motion of our origami structure by employing the QRNN technique20 (see Supplementary Note 1 for the detailed information about the QRNN). This prediction relies solely on the data obtained from the experiment, and therefore, prior knowledge about a mathematical model of the system is not required. To predict the chaotic/periodic folding motion of the TCO structure, we use the QRNN consisting of three hidden layers. Each layer is composed of 352 units. The input data X contains n = 12 components (\({u}_{i},{\dot{u}}_{i},{\varphi }_{i},{\dot{\varphi }}_{i}\) for three different cross-sections), and each component has T = 128 data points (i.e., time steps) which correspond to data length of 0.53 s given the action camera’s sampling frequency of 240 fps.

Based on the input data of \({\bf{X}}\in {{\mathbb{R}}}^{128\times 12}\) from t1 to t128, the QRNN predicts all 12 variables for next 32 time steps. Then, by using the predicted 32 time steps and the last 96 time steps from the previous input data of 128 time steps, we predict another 32 time steps. We repeat this process to predict the dynamic behavior of our origami structure. The total duration of the measured data contains 8000 time steps (33.3 s), and we use first 5600 time steps (23.3 s) for training and the other 2400 time steps (10.0 s) for evaluating the prediction. We obtain 21 sets of such time-series data for 21 frequency steps (i.e., fex = 5, 6, 7, …, 25 Hz, see Fig. 2b for the schematic illustration of our data sets composed of training and testing data in various frequencies). We run the training for 100 epochs (see “Methods” for the parameters used for the QRNN training; also see Supplementary Fig. 3 for the training results). Please note that we train one QRNN system by using all frequency cases, which enables to predict both periodic and chaotic cases, instead of training neural networks for a specific frequency and predict the dynamics at the corresponding frequency case.

  Figure 3a–d shows the predictions made by the QRNN (denoted by red color in the figure) compared with the actual data from the measurements (denoted by gray color) for four different excitation frequencies: fex = 7 Hz (periodic), 12 Hz (chaotic), 16 Hz (periodic), and 17 Hz (chaotic) (see the Supplementary Movie 4 for the entire folding motions of the TCO unit cells reconstructed from both experiments and predictions). In Fig. 3a–d, the “displacements” insets show the displacement-time history of u1 and u2, where time is normalized by excitation period (Tex = 1/fex). The prediction of the QRNN shows excellent agreement with the experimental data for the periodic cases (fex = 7 and 16 Hz). For the chaotic cases (fex = 12 and 17 Hz), the QRNN exhibits quantitatively accurate prediction through ~30 excitation cycles compared with the later part of the prediction, especially for 17 Hz case, and then the deviation begins growing.

Fig. 3: Quasi-recurrent neural network (QRNN) predictions for the folding motion of the triangulated cylindrical origami (TCO) system.
figure 3

Predictions from the QRNN (denoted by red color) are compared with the testing data from the experiments (denoted by gray color) for different excitation frequencies; fex = a 7 Hz, b 12 Hz, c 16 Hz, and d 17 Hz. In the “displacement” plots, the displacement of each section (u1 and u2, respectively) is normalized by the initial height (h0) and time is normalized by excitation period (Tex = 1/fex). Black solid lines are the initial input data for the QRNN, which are obtained from the testing data. In the “FFT” plots, the fast Fourier transform (FFT) is applied to u1 and u2, where the frequency spectrum (F(Ω)) is normalized by its maximum peak value (F(Ω)max). Blue dashed line indicates the excitation frequency (fex). “Phase plane” plots show different folding behaviors of the TCO unit cell, intrawell, interwell, and chaotic motions, as a function of strain δi and its derivative \(\dot{{\delta }_{i}}\). Note that the distance between the two energy minima Lb is used to normalize the data. Spectrum analysis of the FFT normalized by the maximum peak value of the spectra for different excitation frequencies from 5 to 25 Hz is applied to the displacement of the first section (u1) and second section (u2). Areas bounded by gray dashed vertical lines indicate chaotic regime, and there is a good agreement between e experiments and f predictions.

It is worth noting that, even with a well-characterized chaotic system (i.e., having access to the governing equations), the sensitivity to initial conditions (and numerical rounding errors) means that quantitative deviation is always expected. Hence, qualitative matching is a more realistic goal. This can be seen by the fact that the frequency spectrum obtained from time series of u1 and u2 shows qualitatively similar trend as plotted in the “FFT” insets in Fig. 3a–d. Specifically, the increase of lower frequency components for chaotic responses is successfully captured. In addition, even though the prediction from the QRNN deviated from the experimental results as the number of excitation cycle increases, the QRNN outputs surprisingly reasonable peak deformations of both TCO unit cells. For example, the given TCO configuration with θ0 = +70° follows the counter-clockwise rotation (φ > 0) under compression and vice versa (i.e., clockwise rotation under tension), which is also predicted by the QRNN (see Supplementary Fig. 4 for the configuration space of the first and second TCO units as a function of u and φ).

The phase portraits, plotted as a function of displacement and velocity, are shown in the “phase plane” insets in Fig. 3a–d. These show the capability of the QRNN to produce complicated folding behaviors of the TCO structures. For instance, lower frequency excitation creates intrawell oscillation for both TCO unit cells, whereas fex = 16 Hz triggers intermittent intrawell and interwell oscillations in the same structure. This unique behavior is accurately captured in the prediction. In addition to these periodic responses, fex = 17 Hz case exhibits interwell chaotic motion of the first unit cell, while the second unit cell shows interwell periodic oscillations. This is also well expressed by the QRNN (see Supplementary Movie 5 for measured and predicted phase portraits for all excitation frequency cases).

To analyze qualitative behaviors for all excitation frequency cases, we perform spectral analysis on different excitation frequencies from 5 to 25 Hz and construct surface plots as shown in Fig. 3e, f. In this figure, areas bounded by gray dashed vertical lines indicate chaotic response, and show good agreement between experiments and predictions (compare surface maps in panels e and f). Given the presence of unknown factors in the experiment, the QRNN shows remarkably accurate prediction capability based purely on experimentally measured data with noise.

Visualization of the response of the hidden layers to periodic/chaotic data

This section focuses on the analysis of the hidden state QRNN computing. Figure 4a shows the schematic illustration of the QRNN structure composed of three hidden layers. Note that the element-wise multiplication in QRNN fo-pooling enables the analysis on individual hidden units, because different hidden units do not interact directly in a single pooling layer. This allows independent calculation of each hidden unit until the next QRNN hidden layer. For example, the responses of each hidden state to the initial input data at fex = 7 Hz are visualized as a function of time and hidden unit index in Fig. 4b. The hidden state is composed of Ct vectors with 352 hidden units. Dark red/blue colors indicate strong neuron activation, whereas the white color implies reduced activation. These visualizations show several patterns in the hidden states. Extracting the activation history of specific hidden units from the final third hidden layer also shows how a specific neuron processes input data. Figure 4c, d shows the initial input data (u1 and u2) and the neuron activation of 71st and 313th hidden units, respectively. Interestingly, these hidden units show different neuron activation behaviors between periodic (fex = 7 Hz in Fig. 4c, d) and chaotic (fex = 12 Hz in Fig. 4c, d) cases. In addition, in the case of fex = 20 Hz (chaotic), we observe notably different behavior between the 71st and 313th hidden units in response to the same input data.

Fig. 4: Visualization of the hidden states of the quasi-recurrent neural network (QRNN).
figure 4

a Schematic illustration of our QRNN configuration composed of three hidden layers is shown. Here, the input data of X are composed of the vectors \({{\bf{x}}}_{t}\in {{\mathbb{R}}}^{n}\) from the initial time step (t = 0) to the maximum number of time steps (t = T) for one data set. We calculate a sequence \({\bf{Z}}\in {{\mathbb{R}}}^{T\times m}\) composed of candidate vectors (z(ti)), and F and O composed of f(ti) and o(ti), respectively, for the gates required for the pooling components (see Supplementary Note 1 for the details). Also, m is a number of hidden units in a layer, and the filter width (k) is used for each convolution layer (see the dark colored regions for each layer). b The hidden states of each layer for fex = 7 Hz (periodic) case are visualized as surface plots where color denotes activation of each hidden unit. To show how the individual hidden units respond to the input data, c the displacement-time plots of the initial input data for the first and second sections, u1 (blue) and u2 (gray) normalized by the initial height of each unit cell (h0), and d the neuron activation of the 71st (orange) and 313th (green) hidden units are shown. We show the three different excitation frequency cases: fex = 7, 12, and 20 Hz. To compare all the frequency cases, we perform e the fast Fourier transform (FFT) analysis on u1(t) and f the Lyapunov exponent calculation, which are compared with the hidden state responses for the third layer as shown in (g) whose color bar is shown in (b). To quantify neuron activation for each excitation frequency, we extract and calculate the average neuron activation value Ct for h 71st and i 313th neurons.

To analyze the neuron activation for different periodic/chaotic regimes, we compare FFT analysis on the experimentally measured data u1(t) (Fig. 4e) with the Lyapunov exponent calculation based on the Rosenstein’s method32 (Fig. 4f, see also Supplementary Note 2 and Supplementary Fig. 6 for details of the Lyapunov exponent calculation). From the frequency spectrum and the Lyapunov exponent analysis, we identify the three different chaotic regimes: 10–13, 17, and 19–22 Hz (bounded by the gray dashed lines in Fig. 4e–i). The response of the third hidden layer is also visualized in Fig. 4g. Based on our observation of the 71st and 313th hidden units above, we extract the neuron activation of those two units and calculate the average activation value over the time length of the input data (i.e., 0.53 s). Figure 4h, i shows the average activation of 71st and 313th hidden units, respectively. The 71st hidden unit shows drastically different behaviors depending on the regimes, i.e., positive (negative) values for periodic (chaotic) cases. In addition, it is interesting that 313th hidden unit exhibits weaker activation only for the first two chaotic regimes. Note that although there are variations between different frequencies in the same periodic/chaotic regimes (especially fex = 14 Hz), we confirm this unique response of hidden units from ten different training runs, and these multiple trained systems could enhance the capability of distinguishing between periodic and chaotic cases (see Supplementary Fig. 5 for different patterns of hidden unit responses, and also “Methods” for how we find such a hidden unit with unique response).

We observe that the QRNN allows not only the prediction of origami folding behavior, but also the classification of chaotic/periodic input data of the TCO systems by simply monitoring the neuron activation of the hidden units in the final hidden layer. We show that this neuron activation pattern can successfully mimic the Lyapunov exponent, which is a conventional method to characterize chaotic behavior. Therefore, the hidden layer analysis can be useful to understand how the QRNN responses to the input data, and this simple approach has potential to characterize and determine chaotic vibrations, providing a reason behind the decision. Note that the concept of chaos is not implemented specifically during training process. Thus, the QRNN itself acquires prediction capability in the process of the training based on the hidden units’ distinctive responses to input data between periodic and chaos cases.

Discussion

In this study, we have demonstrated a data-driven approach to predict and analyze chaotic/periodic behaviors by using QRNN. Given challenges of chaos prediction, we have built a mechanical platform composed of origami unit cells and have generated different types of time-series data based on intrawell, interwell periodic, and interwell chaotic vibrations. By utilizing experimentally measured data, we have trained the QRNN composed of three hidden layers and demonstrated the effectiveness of predicting chaotic/periodic time series. One of the unique features of the QRNN-based approach is that it allows hidden layer analysis readily without adding extra functions to neural networks. By leveraging this feature, we have calculated average neuron activation of the hidden units in the final layer to examine their response to the input data. This simple approach has revealed the different responses of the QRNN’s hidden units to the system’s dynamic condition, depending on whether it is chaotic or not. Also, this approach has potential to provide the reason why the QRNN produces chaotic data, and specifically what parts of the input data lead to different responses of the QRNN by monitoring the hidden units’ activation. Given the general nature of the origami system and the QRNN method, this approach can be applied to predict complex dynamics of various engineered and biological systems (e.g., aircraft wing vibration, chaotic snap-through of buckled plates33, flight trajectory prediction and safety assessment34, and tissue organization35) by allowing to access evolutionary steps and by providing more parameters to assess their dynamical response. Based on these unique features of the QRNN, our approach can contribute to better understanding of both chaos and machine learning techniques for complex dynamic systems.

Methods

Prototype fabrication and compression test

We used construction paper sheets (Strathmore 500 Series 3-PLY BRISTOL; 0.5 mm paper thickness) for the main origami body and extruded acrylic plate (United States Plastic; 1.6 mm thick) for the interfacial polygon (hexagon in current study), which are both cut by a laser cutting machine (VLS 4.6, Universal Laser Systems). Here, folding lines or the crease lines are based on the compliant mechanisms for accurate and consistent folding behavior. These are then assembled into TCO unit cells with adhesive sheets (Archival Double Tack Mounting Film, Grafix). See Supplementary Movie 2 for fabrication process. Since TCO unit cells are assembled by hand, each unit cell exhibits different force–displacement behavior. This uncertainty in the quality of each prototype significantly influences the repeatability and consistency of the folding/unfolding motion. To avoid this, each unit cell underwent 200 cycles with a controlled displacement from −3 mm (tension) to 15 mm (compression) at 6 mm/s as a preconditioning process (detailed can be found in ref. 29).

Dynamic test

We conducted the dynamic test on a chain composed of two TCO unit cells with (h0θ0R) = (50 mm,  ±70°, 36 mm). The left end of the two-TCO-unit system was connected to the shaker (LDS V406 M4-CE, Brüel & Kjær). The shaker was excited to apply single-frequency harmonic excitation to the system. To track the folding/unfolding motion of the TCO unit cells during the dynamic testing, we used two action cameras (GoPro Hero4) and customized Python codes for noncontact digital image correlation. We captured images from these two cameras at 240 fps, and identified the coordinates of the spherical markers attached to the corner of the interfacial polygons (see Fig. 2a) based on the triangulation method29.

Prediction based on quasi-recurrent neural networks

The QRNN used for the prediction consists of three hidden layers, and the filter width (k) of 6 is used for each convolution layer (see Fig. 4a). Training was carried out on minibatches of 50 data sets by using the Adam optimization algorithm36 with learning rate of 0.001 and decay rate of 0.8. To create each data set, we randomly chose excitation frequency out of 21 different frequencies (fex = 5–25 Hz), and from the selected frequency data with 5600 time steps, we randomly selected input data with 128 time steps and next 32 times step data to calculate error of prediction. We implemented this QRNN in custom codes written in Python by using the machine learning library, TensorFlow, and we performed the training/prediction.

Analysis on the hidden units

To compare the activation of the hidden units for different frequencies, we calculated the average value over the duration of the input data (0.53 s). Let cave,n(fex) be the average activation of nth hidden unit in the final hidden layer for the excitation frequency of fex, we constructed a vector composed of Cave,n(fex) for different frequencies as \({{\bf{C}}}_{{\rm{ave}},n}=[{c}_{{\rm{ave}},n}(5\,{\rm{Hz}}),{c}_{{\rm{ave}},n}(6\,{\rm{Hz}}),\cdots \ ,{c}_{{\rm{ave}},n}(25\,{\rm{Hz}})]\), which is visualized as a bar graph as shown in Fig. 4h. To find unique response of a hidden unit, we examined every hidden units by calculating the difference between Cave,n and a desired response pattern. For example, if we look for a hidden unit showing negative activation for chaotic data, we set a vector \({{\bf{C}}}_{{\rm{target}}}=\left[{c}_{5},{c}_{6},\cdots \ ,{c}_{i},\cdots \ ,{c}_{25}\right]\) where:

$${c}_{i}=\left\{\begin{array}{ll}0.5&({\rm{periodic}})\\ -0.5&({\rm{chaotic}})\end{array}\right.\quad (i=5,6,\cdots \ ,25).$$
(1)

Then, we selected the hidden unit which has the minimum value of the norm Cave,n − Ctarget. By changing Ctarget, we can explore the different types of hidden unit activation patterns (see Supplementary Fig. 5).