Abstract
The dynamical relationship between magnetic storms and magnetospheric substorms is one of the most controversial issues of contemporary space research. Here, we address this issue through a causal inference approach to two corresponding indices in conjunction with several relevant solar wind variables. We find that the vertical component of the interplanetary magnetic field is the strongest and common driver of both storms and substorms. Further, our results suggest, at least based on the analyzed indices, that there is no statistical evidence for a direct or indirect dependency between substorms and storms and their statistical association can be explained by the common solar drivers. Given the powerful statistical tests we performed (by simultaneously taking into account time series of indices and solar wind variables), a physical mechanism through which substorms directly or indirectly drive storms or vice versa is, therefore, unlikely.
Introduction
The identification of spurious associations and potentially causal relationships is key to an improved processbased understanding of various geoscientific processes. Specifically in magnetospheric physics, the understanding of the relationship between magnetic storms and magnetospheric substorms as a part of the solar wind–magnetosphere system is of paramount importance for the development of numerical simulation models of the magnetosphere^{1}. In particular, the existence and directionality of the storm – substorm interaction is one of the most controversial aspects of magnetospheric dynamics^{2}. The original concept of storms being the cumulative result of successive substorms put forward by Akasofu in 1961^{3} has been disputed in subsequent analyses^{2,4,5}. While several modelbased studies have shown a distinct impact of substorm injections on the stormtime ring current enhancement^{6,7,8}, other studies have suggested that the ring current buildup could in principle be directly driven by the solar wind electric field^{9,10}. In this case, magnetospheric substorms do not drive magnetic storms and the two phenomena are independent and share a common cause–the southward interplanetary magnetic field (IMF) driver^{11}.
Recent work (see for instance the review by Balasis et al.^{12}) points to a considerable importance of entropybased measures for identifying and quantifying linear and nonlinear interdependencies between different geophysical variables, variability at different scales, and other characteristics. Time series analyses based on informationtheoretic measures have been used to shed light on the stormsubstorm interaction^{13} and the solar wind drivers of the outer radiation belt^{14} through the general perspective of quantifying information transfer, including linear and nonlinear mechanisms. In particular, DeMichelis et al.^{13} applied a bivariate transfer entropy^{15} (bivTE) analysis to the geomagnetic activity indices AL and SYMH. SYMH is the highresolution (1min) version of the hourly Disturbance stormtime (Dst) index, which is used as a proxy of magnetospheric ring current strength and, thus, as a measure of magnetic storm intensity. AL belongs to the set of the 1min Auroral Electrojet indices (AE, AL, AU and AO) and is used to determine the onset of the substorm growth phase^{16}. DeMichelis et al. suggested that information flow from AL to SYMH dominates in the case of small geomagnetic disturbances, while the reverse situation is observed in presence of strong geomagnetic disturbances.
However, bivariate measures such as mutual information (MI) or bivTE do not allow to exclude the very frequent influence of other variables as common drivers, rendering MI and bivTE associations spurious. Multivariate extensions of TE, on the other hand, are severely limited because their estimators don’t work well in high dimensions^{17}. In the present study, we contrast bivariate measures with a directional, multivariate informationtheoretic causality measure based on lowdimensionally estimated graphical models^{17,18,19}. This multivariate measure for the influence of a subprocess X of a system on another subprocess Y is called information transfer to Y (ITY) and allows for more powerful tests on the absence or potential presence of a causal relationship, which is crucial for developing a better “mechanistic” understanding of the governing processes.
Here, we investigate time series of various solar wind parameters including the IMF’s magnitude B and vertical component B_{Z}, its velocity V_{SW} and dynamic pressure P_{dyn}, as well as the AL and SYMH indices. We focus on data from 2001, near a solar activity maximum. Our goal is to clarify whether substorm activity could causally drive storm dynamics or – on the contrary – whether solar wind variables can explain the statistical associations between storm and substorm activity.
Results
Data
The present study is focused on the year 2001, a year with very strong solar activity, and data on solar wind parameters as well as geomagnetic activity indices. As possible solar driving factors we include only those quantities with at least a measurable statistical dependency (mutual information) with either AL or SYMH. These are B, B_{Z}, V_{SW}, and P_{dyn}. The original oneminutely data were aggregated to a 20minute time resolution by averaging over nonoverlapping 20minute blocks. This resolution was selected based on iterative tests to obtain a compromise between resolving time lags and still keeping the computational load and multiple testing problems low. Additionally, DeMichelis et al.^{13} found, on average, a net information flow from AL to SYMH attaining its maximum at a typical time delay of about 1 h which is well resolved with our chosen time resolution.
As solar wind time series inevitably contain missing values due to satellite failures, in the aggregation we masked samples for 20 min periods with more than 50% missing values. We also accounted for masked samples in the lagged analyses (up to τ_{max} = 6 × 20 min) to avoid a selection bias. This leads to 18,384 nonmasked 20min samples instead of about 26,000 samples for the whole year. No further preprocessing was applied. Figure 1 shows the corresponding time series.
Mutual information and bivariate transfer entropy analysis
This study aims to shed light on the possible existence of a driverresponse relationship between storms and substorms, which is a reflection of the dynamic processes within the coupled solar windmagnetosphereionosphere system. Because there has been accumulating evidence that the involved interrelations are of a nonlinear nature^{20,21,22,23,24,25,26} and very long data series are available, we employ a nonparametric (modelfree) approach here. Information theory provides a genuine framework for the modelfree study of couplings among time series. Here we invoke three informationtheoretic measures with increasing power to detect spurious dependencies due to autocorrelation, common drivers or indirect relationships.
The first and simplest association measure applying information theory to time series is the lagged (cross)mutual information^{27} given by
using Shannon entropies \(H(X)=\,\int \,p(x)\,\mathrm{ln}\,p(x)dx\) (correspondingly for conditional entropies) in units of nats with the natural logarithm as a base. For τ > 0, MI measures the information in the past of X that is contained in the present of Y. The weaknesses of MI as a measure of information transfer have been discussed early on, most notably by Schreiber^{15}. A first step to arrive at a directional notion of information transfer is to exclude information from the past of Y. Implementing this idea, Schreiber introduced the transfer entropy (TE)^{15} between two variables, which is the informationtheoretic analogue of Granger causality and can be defined in a lagspecific variant as
based on the conditional mutual information. To estimate all CMIs in this study, we use an advanced nearestneighbor estimator^{28,29} that is most suitable for variables with a continuous range of values (details in Methods).
However, bivTE can yield spurious results if more than two processes are interacting: For the interaction example in Fig. 2(b) both the MI I(X_{t−1};Y_{t}) and the TE I(X_{t−1};Y_{t}Y_{t−1}) are larger than zero due to the common driver Z, even though no direct or indirect physical mechanism exists by which X drives Y or vice versa. The detailed timeresolved graph in Fig. 2(a) shows that, X_{t−1} and Y_{t} are not independent given only the past of Y or only the common driver Z_{t−2} as a condition. Rather, in order to unveil the spurious dependency, the CMI must be conditioned on both Y_{t−1} and Z_{t−2} to exclude all causal paths connecting X_{t−1} and Y_{t} (see ref.^{30} for a definition of causal paths).
In Fig. 3 we investigate bivariate MI and bivTE lag functions of all considered solar variables with AL and SYMH, including the interaction between these two. We restrict the maximum time delay to τ_{max} = 6 × 20 min. For example, the panel B_{Z} → AL shows the lag function I(B_{Z,t−τ}; AL_{t}) of MI [Eq. (1), gray] and I(B_{Z,t−τ}; AL_{t}AL_{t−1}) of bivTE excluding the past lag of AL [Eq. (2), black]. The multivariate ITY [Eq. (3), blue] is discussed in the next section. The solid line marks the significance threshold. All (C)MI values have been rescaled to the (partial) correlation scale via \(I\to \sqrt{1{e}^{2I}}\in [0,1]\)^{27} and rescaled values above 0.4 can, thus, be considered as moderate to strong. In the tables (Table 1 in main article and Supplementary Tables S2–S4), on the other hand, the CMI values are given in nats.
In Fig. 3, MI lag functions (grey) show large values for all possible driver variables. Here the peak of the MI lag function is often shifted compared to bivTE (black). Such an effect can arise from strong autocorrelations as studied in ref.^{31}. Overall, the bivariate TE has sharper peaks than MI. B_{Z} clearly is the strongest driver of both AL and SYMH, and all other drivers are comparably weak (except for the autodependencies in panels AL → AL and SYMH → SYMH). The reason for this behavior is that some MI values are ‘inflated’, again, due to strong autocorrelations^{18}, especially V_{sw} is strongly autodependent. This makes MI values and the peak of MI lag functions hard to interpret.
The interactions AL → SYMH and SYMH → AL have been studied in ref.^{13} where a relationship from substorms towards storms was found with a binning estimator of bivTE. Our results reproduce this finding with a nearestneighbor estimator^{28,29}. The other direction, from storms to substorms, is not very significant here. Note that values at lag τ = 0 min cannot be interpreted in a directional sense in our analysis.
Multivariate informationtheoretic causality analysis
Mutual information and bivariate informationtheoretic measures, such as MI and bivTE, cannot account for common drivers and indirect transitive relationships. As illustrated in Fig. 2(b), a common driver (Z) can lead to a spurious association, either linear or nonlinear, between X and Y. The complex multivariate causal interaction structure can be captured with the concept of a time series graph^{32,33} as shown in Fig. 2(a), originating from the theory of graphical models. As further defined in ref.^{30}, each node in a time series graph represents a subprocess at a certain time. Past nodes at t′ < t have a link towards a subprocess at time t if and only if they are not independent conditionally on the past of the whole process. In this graph the parents \({{\mathscr{P}}}_{\cdot }\) of a variable are given by all nodes with an arrow towards it (blue boxes in Fig. 2(a)).
While these parents could be estimated by testing the CMI between each X_{t−τ} and Y_{t} conditional on all other lagged variables, this approach, similar to multivariate or conditional TE, does not work well due to its high dimensionality^{17} leading to weak statistical power and many false positives. In ref.^{19} an efficient algorithm for the estimation of the parents of a variable Y (further details in Methods) is detailed. In a second stage we use the estimated set of parents to measure the information transfer to Y (ITY)^{18} for all lagged variables X_{t−τ} (including the parents)
which will be zero if and only if X_{t−τ} and Y_{t} are independent conditionally on \({{\mathscr{P}}}_{{Y}_{t}}\). Unfortunately, no analytical results exist on the finitesample distribution of the nearestneighbor estimator under the null hypothesis of conditional independence. For significance testing, we use a blockshuffle surrogate test here following refs.^{34} and^{35} as described in Methods. The algorithm was run with maximum lag τ_{max} = 6 × 20 min as before. We assess significance at the 95% level.
Table 1 shows iteration steps with the selected conditions and the conditional mutual information (CMI) values and significance of the AL → SYMH and SYMH → AL links in each step. The AL → SYMH link becomes nonsignificant using the condition set (SYMH(t − 1), B_{Z}(t − 2), P_{dyn}(t − 1), V_{sw}(t − 2)). This implies that these solar drivers can explain the spurious link AL → SYMH at a lag of 20 min. Note that this set is only a sufficient explanatory set and other drivers might also induce this spurious association. Also the much weaker link SYMH → AL becomes nonsignificant after including few solar drivers (B_{Z}, V_{sw}, B, at different lags).
The ITY estimates with these parents are shown in Fig. 3 (blue markers). ITY now accounts for autocorrelation in the driven variable (like bivTE), but additionally for the influence of the other parents as common drivers or indirect mediators^{36}. Now the ITY lag functions are peaked and significant (markers above solid line) only at a few selected lags.
Figure 4 visualizes the significant drivers of AL and SYMH in a process graph as in Fig. 2(b). Edges correspond to directional lagged links, and the labels indicate their lags. If more than one lag is significant, they are listed in the order of their strength. Both, the edge color and width, indicate the value at the lag with the largest ITY. The node color depicts the strength of the lag1 autodependency for AL and SYMH. Note that the weak ITY value in B_{Z} → SYMH is due to B_{Z} occurring with two neighboring lags in the parents of SYMH, which reduces the information transfer of either of them.
In conjunction with some further robustness studies for another substorm index and other method parameters (Supplementary Figs. S1 and S2), our major results can be summarized as follows: The main drivers of substorms as measured by AL are B_{Z} and V_{SW}. These also drive storms as measured by SYMH. P_{dyn} and especially B are less robustly related to both storms and substorms. Regarding time lags, the AL index responds to B_{Z} at a lag ≈ 20–40 min, while the lags with the weaker other drivers are less robust (see Supplementary Figs. S1 and S2). The SYMH index responds to B_{Z} at a lag ≈ 40 min, to V_{SW} at 40 min, to P_{dyn} at 20 min, and rather weakly with nonrobust lags to B.
Regarding the previously found link AL → SYMH^{13}, we find that mainly B_{Z} and to a lesser degree V_{SW} and P_{dyn} are sufficient to explain this statistical association. These results are also verified by applying the same tools to an AE  SYMH analysis and for other estimation parameters (see Supplementary Figs. S1 and S2 and Tables S2–S4). Thus, we find that there is no direct or indirect transfer of information AL → SYMH or SYMH → AL.
Discussion
DeMichelis et al.^{13} investigated the transfer of information between substorms and storms by means of a bivariate transfer entropy analysis of AL and SYMH time series from 1981 (near solar maximum). They found a significant information flow from substorms to storms attaining its maximum with a typical time delay of about 1 h and suggested that the direction of information flow between substorms and storms depends on the global magnetospheric activity level. Our analysis goes beyond the study of ref.^{13} by utilizing a directional, multivariate informationtheoretic causality measure that simultaneously takes into account solar wind variables and geomagnetic indices data, allowing for more powerful statistical tests on the absence or potential presence of a causal relationship between substorms and storms.
Our secondary finding that the main drivers of substorms (as measured by AL) and storms (as measured by SYMH) are B_{Z} and V_{SW} is consistent with the fact that the energy transfer from the solar wind to the magnetosphere is proportional to B_{Z} and V_{SW}. P_{dyn} and especially B are less relevant for both storms and substorms. We conclude that these directed information transfers constitute robust interrelationships between solar wind parameters and dynamic processes in the magnetosphere. These findings confirm earlier studies on solar wind drivers and their storm and substorm manifestations, including variations of the indices SYMH and AL (refs.^{37,38} and refs. therein).
Johnson et al.^{39} recently showed that the transfer of information from V_{SW} or VB_{south} (where VB_{south} is V_{SW} × southward IMF B_{Z}) to Dst (similar to SYMH but with a lower resolution) lasts more than 100 hours, which may correspond to the long time scale of the ring current decay (e.g., ref.^{40}). We have also considered similar time scales when having analyzed VB_{south} and Dst for another study focusing solely on storms^{41}. Here, we analyze 1 minuteresolution indices and solar wind data in order to look at shorter time scales that cover lags up to 2 hours, focusing both on storms and substorms. The finding of IMF B_{Z} and V_{SW} as the drivers of storms are consistent with Johnson et al.^{39}, which also used information theoretic tools (transfer entropy and cumulant based analysis) in their analysis.
Our most important finding is that our iterative causal discovery algorithm analysis suggests that mainly B_{Z}, and to a lesser degree V_{SW} and P_{dyn} are sufficient to explain the previously found link AL → SYMH^{13}. Thus, we find no statistical evidence for a link AL → SYMH. We also find no link SYMH → AL and these results are robust also for another substorm index (AE) and for other estimation parameters. The results by Iyemori and Rao^{42} supported the idea that the geomagnetic storms and substorms are independent processes; that is, the ringcurrent development is not the result of the frequent occurrence of substorms, but that of enhanced convection caused by the large southward IMF. Although some later studies^{43,44}, based on insitu observations, have shown that the contribution of ion injections to the ring current energy gain is substantial, our results do not favor the role of substorms in the enhancement of the stormtime ring current through accumulative ion injections during consecutive substorms, in agreement with the earlier studies by Iyemori and Rao^{42}. A possible reason for the absence of information transfer from AL → SYMH might be that not all ion injections to the stormtime ring current are reflected in the AL variations. A recent study^{45} showed that smallscale injections are not captured by AL. Another study^{46} showed that low and highenergy protons vary in quite different ways on stormtime timescales and accordingly suggested that the relation between ion injections and ring current growth may be more complicated than previously perceived. In summary, it is possible that substorms are required for the particle injection to the ring current [e.g. seen in Energetic Neutral Atom imaging] but not sufficient (since nonstorm substorms appear to lead to no intensification of the ring current) and strong convection is also required. This remains a debate.
Before concluding, let us discuss the methodological limitations pertaining to such a statistical causality analysis. The presence of significant links in our analysis can be called causal only with respect to the included set of variables. Nonobserved variables can still be the cause of a link here and the obtained links should, therefore, serve more as an hypothesis for further studies that include more possible explanatory variables. From a theoretical standpoint, a more robust finding is that of the absence of a link: if there exists no statistical evidence for a dependency between two variables, a physical mechanism between the two is unlikely. Hence, the nonsignificance of direct or indirect dependencies between the commonly used AL and SYMH indices leads us to the conclusion that there exists no physical mechanism by which perturbations in substorms are transported to storms or vice versa. However, from a practical standpoint in the general context of the limitations associated with every statistical information quantity, below we summarize possible deficiencies and/or weaknesses that may accompany the application of ITY, even though we consider them rather insignificant or of low probability to occur for the present study.
Firstly, the information measure might not capture the dependency. We should note that our informationtheoretic approach allows to take into account almost any type of nonlinear relationship, both in excluding it as a common driver, and also in detecting it. This is in contrast to linear correlation or linear Granger causality studies. The price for this “generality” is lower statistical power: For a particular dependency, the more general CMI will have less power compared to a measure that is optimized for this type of dependency, for example correlation for linear dependencies. Weaker power means that weaker dependencies might not be detected for small sample sizes, especially for highdimensional conditions^{30}. Our method is designed to avoid highdimensionality by an iterative approach (especially compared to multivariate TE) and has demonstrated high power in numerical experiments^{30}. Additionally, here we have a very large sample size, leading us to the conclusion that if there is a dependency, it must be very weak. Also, our major finding is robust when using other estimation parameters.
Secondly, we analyzed the whole year 2001 to obtain a sufficiently large sample size. Possibly, a causal relationship is present only during shorter periods and absent in other periods, which would be difficult to assess given too short sample sizes and the length of characteristic time scales of the processes.
Thirdly, the physical mechanism might be present mostly during the missing values excluded in the analysis. If satellite failures are indeed strongly related with the hypothesized mechanism, this would imply a nonavoidable selection bias in our analysis.
Lastly, the indices have serious limitations as to their ability of monitoring a particular current system: (1) They are scalars and may be insufficient to deduce a 2D current system; (2) The indices are derived from a very limited number of stations and as such are subject to a number of artifacts and limitations; (3) The ground perturbations are due to all currents – near and far. In summary, AL is a limited measure of the 2D westward electrojet distribution (intensity, structure and dynamics) and SYMH is a limited measure of the ring current (intensity, structure and dynamics)^{47}. Here we tested two kinds of indices for substorms (AL and AE) and got robust results.
These limitations (generalitypower tradeoff, missing values, proxy data quality) apply to any statistical coupling analysis. The main shortcoming of previous approaches based on bivariate measures is that these did not take into account possible common drivers, hence weakening a possible causal interpretation. Multivariate approaches have a stronger causal interpretation at the cost of weaker detection power due to higher dimensionality, which our method alleviates as much as possible. In light of these qualifications, we conclude that a direct or indirect physical mechanism by which substorms drive storms or vice versa is unlikely.
Conclusions
There has been only one study so far that utilized information theory tools to study the stormsubstorm relation. De Michelis et al.^{13} used the bivariate measures of delayed mutual information and transfer entropy to analyze SYMH and AL indices from a year near solar maximum (1981). Their findings suggested that information flow from AL to SYMH dominates in the case of weak geomagnetic disturbances, while the reverse situation is observed in the presence of strong geomagnetic disturbances. The present study goes beyond the analysis performed by De Michelis et al.^{13} by contrasting bivariate measures with a directional, multivariate informationtheoretic causality measure based on lowdimensionally estimated graphical models. This multivariate measure is called information transfer to Y and allows for more powerful tests on the absence or potential presence of a causal relationship, which is crucial for developing a better “mechanistic” understanding of the governing processes. Thus, we are able to simultaneously handle SYMH and AL indices along with the magnetospheric activity solar wind variables including the IMF’s magnitude B and vertical component B_{Z}, its velocity V_{SW} and dynamic pressure P_{dyn} using an information measure technique. This is the first time, to our knowledge, that the variations of the various parameters describing the input and output of the solar wind – magnetosphere system are treated all together by a causality measure that is capable to identify information flow between all these parameters. We conclude on nonsignificant direct or indirect dependencies between AL and SYMH indices, and therefore, between substorms and storms, which means that the previously applied bivariate measures were not able to accurately depict or resolve the interdependencies between the system’s parameters.
Additionally, a secondary conclusion of our study is that we are able to confirm earlier results about the solar wind drivers of storms and substorms (i.e., B_{Z}) from a different point of view, utilizing the modern and versatile toolbox of information theory. We have achieved this, for the first time, by considering the solar windmagnetosphere system as a whole and applying a multivariate informationtheoretic approach able to simultaneously handle the system’s input (solar wind drivers) and output (magnetospheric activity indicators) in contrast to several previous important but distinct studies, where the bulk of information on the solar wind driver of the magnetosphere has been accumulated [e.g. refs.^{37,38}]. This demonstrates the great potential that the application of information theory may have to treat space physics problems, where vast amounts of related datasets are continuously accumulated either from spaceborne or groundbased measurements. Moreover, in the light of our findings, the application of the multivariate causality measure of ITY was able to explain the spurious link AL → SYMH found previously by bivariate causality measures^{13} simply by the variations of the solar wind drivers.
The results of this study contribute to the ongoing debate of the stormsubstorm relationship and to the debate of plasma injection to the inner magnetosphere. For example, Angelopoulos et al.^{48} concluded that burstybulk flows (BBFs) are sufficient to account for all the energy deposition in the ionosphere and inner magnetosphere. However, Ohtani et al.^{49} reported that fast plasma sheet flows do not reach the geosynchronous orbit or lead to dipolarization. The results of the present study offer an interesting possibility that substorm led injections or BBFs, in general, may not travel all the way to the inner magnetosphere. However, substorm BBFs accompanied by strong convection (VB_{south}) may penetrate the inner magnetosphere and contribute to the ring current.
Our analysis demonstrates the great potential of combining a causal discovery algorithm with a multivariate and lagspecific extension of transfer entropy for tackling contemporary research questions in magnetospheric physics, such as the stormsubstorm relationship, which is one of the most controversial topics of magnetospheric dynamics and solarterrestrial coupling. Further analyses using a causal pathwayanalysis^{36,50} can shed light on the interaction mechanism among the solar drivers and the magnetosphere. The obtained causal drivers, on the other hand, can also be relevant for optimal prediction schemes^{51}. We expect that our results will contribute to a better understanding of the dynamic processes related to the coupled solar wind  magnetosphere  ionosphere system by fostering a paradigm shift in our perception of the stormsubstorm relationship. They may also have a direct impact on magnetosphere modeling and, consequently, space weather forecasting efforts.
Methods
The algorithm in ref.^{19} for the estimation of the parents of a variable Y uses the idea to successively test for conditional independence between Y_{t} and each possible past driver (including the past of Y) conditioned on iteratively more conditions. Thereby, the condition dimension stays as low as possible in every iteration step which helps to alleviate high dimensionality in estimating CMIs. Here we test only the most relevant set of conditions with the highest CMIs in the previous step. The algorithm then is as follows: We first initialize the preliminary parents \({\mathscr{P}}({Y}_{t})=({{\bf{X}}}_{t1},{{\bf{X}}}_{t2},\ldots ,{{\bf{X}}}_{t{\tau }_{{\rm{\max }}}})\) containing the past of all variables (including Y). Starting with p = 0, we iteratively increase p → p + 1 in an outer loop and, in an inner loop, test for all variables \({X}_{t\tau }^{i}\) from \({\mathscr{P}}({Y}_{t})\) whether
where \({{\mathscr{P}}}^{(p)}({Y}_{t})\) are the p strongest parents among \({\mathscr{P}}({Y}_{t})\backslash \{{X}_{t\tau }^{i}\}\) according to their CMI. If the CMI is zero at some significance level α using the test described below, we remove a link from \({\mathscr{P}}({Y}_{t})\) at the end of each piteration. The algorithm converges if no larger conditioning sets can be tested. We sort \({\mathscr{P}}({Y}_{t})\) after every iteration according to the CMI values.
We use an advanced nearestneighbor estimator^{28,29} of CMI that is most suitable for variables with a continuous range of values. This estimator has as a parameter the number of nearestneighbors k which determines the size of hypercubes around each (highdimensional) sample point and, therefore, can be viewed as a density smoothing parameter (although it is dataadaptive unlike fixedbandwidth estimators). For large k, the underlying dependencies are strongly smoothed and may not resolve nonlinearities. We tested different values of k to verify the robustness of our results. Larger k have larger bias and are more computationally expensive, but have smaller variance. Note that the estimated CMI values can be slightly negative while CMI is a nonnegative quantity. In Figs. 3 and 4 and Supplementary Figs. S1 and S2 the (C)MI values have been rescaled to the (partial) correlation scale via \(I\to \sqrt{1{e}^{2I}}\in [0,1]\)^{27}. In the tables, on the other hand, the CMI values are given in nats.
For significance testing, either a fixed threshold or shuffle surrogates are the only choice here. Surrogate tests are especially helpful for proper significance tests because they adapt to the bias for higherdimensional CMIs. In ref.^{17} a shuffle test has been used, but for strongly autocorrelated time series, as in the present case, this test is too weak. Therefore, we use a blockshuffle surrogate test here following refs.^{34} and^{35}. An ensemble of M values of \(I({X}_{t\tau }^{\ast };{Y}_{t}Z)\) is generated where \({X}_{t\tau }^{\ast }\) is a blockshuffled sample of X_{t−τ}, i.e., with blocks of the original time series permuted. As an optimal blocklength we use the approach described in refs.^{34} and^{35} for nonoverlapping blocks. The optimal blocklength (Eq. (6) in ref.^{35}) involves the decay rate of the envelope of the autocorrelation function γ(τ). The latter is estimated up to a maximum delay of 5% of the (nonmasked) samples and the envelope was estimated using the Hilbert transform. Then a function Cϕ^{τ} is fitted to the envelope with constant C to obtain the decay rate ϕ. Finally, the CMI values are sorted and a pvalue is obtained as the fraction of surrogates with CMI greater or equal than the estimated CMI value. We use an ensemble of 200 surrogates. Confidence intervals (errorbars in figures) were estimated using bootstrap resampling involving only estimated nearestneighbor statistics with 200 samples. The blockshuffle approach is only an approximation to obtain the true null distribution.
Software Availability
Software is available online under https://github.com/jakobrunge/tigramite.
References
 1.
Pulkkinen, T. I. et al. Solar wind  magnetosphere coupling: A review of recent results. J. Atmos. and Solar Terr. Phys. 69, 256–264 (2007).
 2.
Sharma, A. S. et al. The StormSubstorm Relationship: Current Understanding and Outlook, in Disturbances in Geospace: The StormSubstorm Relationship (American Geophysical Union, Washington DC, 2003).
 3.
Akasofu, S., Cain, J. C. & Chapman, S. The Magnetic Field of a Model Radiation Belt, Numerically Computed. Journal of Geophysical Research 66, 4013–4026 (1961).
 4.
Gonzalez, W. D., Joselyn, J. A. & Kamide, Y. What is a geomagnetic storm? Journal of Geophysical Research: Space Physics 99, 5771–5792 (1994).
 5.
Kamide, Y. et al. Current understanding of magnetic storms: Stormsubstorm relationships. Journal of Geophysical Research: Space Physics 103, 17705–17728 (1998).
 6.
Fok, M.C., Moore, T. E. & Delcourt, D. C. Modeling of inner plasma sheet and ring current during substorms. Journal of Geophysical Research 104, 14557–14570 (1999).
 7.
Daglis, I. A. & Kamide, Y. The role of substorms in stormtime particle acceleration. In Kamide, Y. & Gurbax, S. (eds) Disturbances in Geospace: The StormSubstorm Relationship (American Geophysical Union, Washington, DC, 2004).
 8.
Ganushkina, N. Y., Pulkkinen, T. I. & Fritz, T. Role of substormassociated impulsive electric fields in the ring current development during storms. Annales Geophysicae 23, 579–591 (2005).
 9.
Kamide, Y. Is substorm occurrence a necessary condition for a magnetic storm? Journal of Geomagnetism and Geoelectricity 44, 109–117 (1992).
 10.
McPherron, R. L., Baker, D. N., Bargatze, L. F., Clauer, C. R. & Holzer, R. E. IMF control of geomagnetic activity. Advances in Space Research 8, 71–86 (1988).
 11.
Daglis, I. A. et al. Intense space storms: Critical issues and open disputes. Journal of Geophysical Research: Space Physics 108 (2003).
 12.
Balasis, G. et al. Statistical Mechanics and InformationTheoretic Perspectives on Complexity in the Earth System. Entropy 15, 4844–4888 (2013).
 13.
De Michelis, P., Consolini, G., Materassi, M. & Tozzi, R. An information theory approach to the stormsubstorm relationship. Journal of Geophysical Research: Space Physics 116 (2011).
 14.
Wing, S., Johnson, J. R., Camporeale, E. & Reeves, G. D. Information theoretical approach to discovering solar wind drivers of the outer radiation belt. Journal of Geophysical Research: Space Physics 121 (2016).
 15.
Schreiber, T. Measuring information transfer. Physical Review Letters 85, 461–464 (2000).
 16.
Gjerloev, J. W., Hoffman, R. A., Friel, M. M., Frank, L. A. & Sigwarth, J. B. Substorm behavior of the auroral electrojet indices. Annales Geophysicae 22, 2135–2149 (2004).
 17.
Runge, J., Heitzig, J., Petoukhov, V. & Kurths, J. Escaping the Curse of Dimensionality in Estimating Multivariate Transfer Entropy. Physical Review Letters 108, 258701 (2012).
 18.
Runge, J., Heitzig, J., Marwan, N. & Kurths, J. Quantifying causal coupling strength: A lagspecific measure for multivariate time series related to transfer entropy. Physical Review E 86, 061121 (2012).
 19.
Runge, J., Nowack, P., Kretschmer, M., Flaxman, S. & Sejdinovic, D. Detecting causal associations in large nonlinear time series datasets. arXiv:1702.07007v2 1–46 (2018).
 20.
Bargatze, L. F., Baker, D. N., Hones, E. W. & Mcpherron, R. L. Magnetospheric impulse response for many levels of geomagnetic activity. Journal of 90, 6387–6394 (1985).
 21.
Tsurutani, B. T., Goldstein, B. E., Sugiura, M., Iyemori, T. & Gonzalez, W. D. The nonlinear response of AE to the IMF Bs driver  A spectral break at 5 hours. Geophysical Research Letters 17, 279 (1990).
 22.
Consolini, G., Marcucci, M. & Candidi, M. Multifractal Structure of Auroral Electrojet Index Data. Physical Review Letters 76, 4082–4085 (1996).
 23.
Klimas, A. J., Vassiliadis, D., Baker, D. N. & Roberts, D. A. The organized nonlinear dynamics of the magnetosphere. Journal of Geophysical Research: Space Physics 101, 13089 (1996).
 24.
Vassiliadis, D., Klimas, A., Baker, D. N. & Roberts, D. The nonlinearity of models of the vBsouthAL coupling NN, Xn + l F (Xn, Un). Journal of Geophysical Research: Space Physics 101, 779–787 (1996).
 25.
Hnat, B., Chapman, S. & Rowlands, G. Scaling of solar wind and the AU, AL and AE indices as seen by WIND. Geophysical Research Letters 29, 35–1 (2002).
 26.
Balasis, G. et al. Investigating dynamical complexity in the magnetosphere using various entropy measures. Journal of Geophysical Research: Space Physics 114 (2009).
 27.
Cover, T. M. & Thomas, J. A. Elements of Information Theory (John Wiley & Sons, Hoboken, 2006).
 28.
Kraskov, A., Stögbauer, H. & Grassberger, P. Estimating mutual information. Physical Review E 69, 066138 (2004).
 29.
Frenzel, S. & Pompe, B. Partial Mutual Information for Coupling Analysis of Multivariate Time Series. Physical Review Letters 99, 204101 (2007).
 30.
Runge, J. Causal network reconstruction from time series: From theoretical assumptions to practical estimation. Chaos An Interdiscip. J. Nonlinear Sci. 28, 075310 (2018).
 31.
Runge, J., Petoukhov, V. & Kurths, J. Quantifying the Strength and Delay of Climatic Interactions: The Ambiguities of Cross Correlation and a Novel Measure Based on Graphical Models. Journal of Climate 27, 720–739 (2014).
 32.
Dahlhaus, R. Graphical interaction models for multivariate time series. Metrika 51, 157–172 (2000).
 33.
Eichler, M. Graphical modelling of multivariate time series. Probability Theory and Related Fields 153, 233–268 (2012).
 34.
Peifer, M. et al. On Studentising and Blocklength Selection for the Bootstrap on Time Series. Biometrical Journal 47, 346–357 (2005).
 35.
Mader, M., Mader, W., Sommerlade, L., Timmer, J. & Schelter, B. Blockbootstrapping for noisy data. Journal of Neuroscience Methods 219, 285–91 (2013).
 36.
Runge, J. Quantifying information transfer and mediation along causal pathways in complex systems. Phys. Rev. E 92, 062829 (2015).
 37.
Vassiliadis, D. Systems theory for geospace plasma dynamics. Reviews of Geophysics 44, RG2002 (2006).
 38.
Tsyganenko, N. A. Databased modelling of the Earth’s dynamic magnetosphere: A review. Annales Geophysicae 31, 1745–1772 (2013).
 39.
Johnson, J. R., Wing, S. & Camporeale, E. Transfer entropy and cumulantbased cost as measures of nonlinear causal relationships in space plasmas: applications to dst. Annales Geophysicae 36, 945–952 (2018).
 40.
Kozyra, J. et al. Multistep dst development and ring current composition changes during the 4–6 june 1991 magnetic storm. Journal of Geophysical Research: Space Physics 107, SMP–33 (2002).
 41.
Balasis, G. et al. From prestorm activity to magnetic storms: a transition described in terms of fractal dynamics. Annales Geophysicae 24, 3557–3567 (2006).
 42.
Iyemori, T. & Rao, D. R. Decay of the Dst field of geomagnetic disturbance after substorm onset and its implication to stormsubstorm relation. Annales Geophysicae 14, 608–618 (1996).
 43.
Daglis, I. A. et al. “Fine structure” of the stormsubstorm relationship: Ion injections during Dst decrease. Advances in Space Research 25, 2369–2372 (2000).
 44.
Gkioulidou, M. et al. The role of smallscale ion injections in the buildup of Earth’s ring current pressure: Van Allen Probes observations of the 17 March 2013 storm. Journal of Geophysical Research: Space Physics 119, 7327–7342 (2014).
 45.
Gkioulidou, M. et al. Spatial structure and temporal evolution of energetic particle injections in the inner magnetosphere during the 14 July 2013 substorm event. Journal of Geophysical Research: Space Physics 120, 1924–1938 (2015).
 46.
Gkioulidou, M., Ukhorskiy, A., Mitchell, D. G. & Lanzerotti, L. J. Stormtime dynamics of ring current protons: Implications for the longterm energy budget in the inner magnetosphere. Geophysical Research Letters 43, 4736–4744 (2016).
 47.
Newell, P. & Gjerloev, J. Supermagbased partial ring current indices. Journal of Geophysical Research: Space Physics 117 (2012).
 48.
Angelopoulos, V. et al. Multipoint analysis of a bursty bulk flow event on april 11, 1985. Journal of Geophysical Research: Space Physics 101, 4967–4989 (1996).
 49.
Ohtani, S., Singer, H. & Mukai, T. Effects of the fast plasma sheet flow on the geosynchronous magnetic configuration: Geotail and goes coordinated study. Journal of Geophysical Research: Space Physics 111 (2006).
 50.
Runge, J. et al. Identifying causal gateways and mediators in complex spatiotemporal systems. Nature Communications 6, 8502 (2015).
 51.
Runge, J., Donner, R. V. & Kurths, J. Optimal modelfree prediction from multivariate time series. Phys. Rev. E 91, 052909 (2015).
Acknowledgements
We acknowledge the following sources of data: the Kyoto World Data Center (WDC) for Geomagnetism and the observatories that produce and make AE and SYMH indices available at http://wdc.kugi.kyotou.ac.jp/ as well as NASA’s Space Physics Data Facility (SPDF) that hosts the Coordinated Data Analysis Web (CDAWeb) services along with the science teams that provided interplanetary data through http://cdaweb.gsfc.nasa.gov/. J.R. received funding by the James S. McDonnell Foundation. R.V.D. acknowledges financial support via the BMBFfunded Young Investigators Group “Complex Systems Approaches to Understanding Causes and Consequences of Past, Present and Future Climate Change” (CoSyCC^{2}, grant no. 01LN1306A). This work has been financially supported by the joint Greek–German IKYDA 2013 project “Transdisciplinary assessment of dynamical complexity in magnetosphere and climate: A unified description of the nonlinear dynamics across extreme events” funded by IKY and DAAD. The authors thank Ciaron Linstead for help with highperformance computing and gratefully acknowledge the European Regional Development Fund (ERDF), the German Federal Ministry of Education and Research (BMBF) and the Land Brandenburg for supporting this project by providing resources on the high performance computer system at the Potsdam Institute for Climate Impact Research.
Author information
Affiliations
Contributions
G.B., J.R., R.V.D. and I.A.D. designed the study, C.P. prepared the data, J.R. analyzed the data. All authors discussed the results and contributed to editing the manuscript.
Corresponding author
Ethics declarations
Competing Interests
The authors declare no competing interests.
Additional information
Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Electronic supplementary material
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Runge, J., Balasis, G., Daglis, I.A. et al. Common solar wind drivers behind magnetic storm–magnetospheric substorm dependency. Sci Rep 8, 16987 (2018). https://doi.org/10.1038/s41598018352505
Received:
Accepted:
Published:
Keywords
 Solar Wind Driver
 Common Driver
 Solar Windmagnetosphere System
 Transfer Entropy (TE)
 Stormtime Ring Current
Further reading

Dynamical Complexity of the 2015 St. Patrick’s Day Magnetic Storm at Swarm Altitudes Using Entropy Measures
Entropy (2020)

Measuring Information Coupling between the Solar Wind and the Magnetosphere–Ionosphere System
Entropy (2020)

Applications of Information Theory in Solar and Space Physics
Entropy (2019)

Recurrence‐Based Quantification of Dynamical Complexity in the Earth's Magnetosphere at Geospace Storm Timescales
Journal of Geophysical Research: Space Physics (2019)

Detecting and quantifying causal associations in large nonlinear time series datasets
Science Advances (2019)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.