A distributional multivariate approach for assessing performance of climate-hydrology models

One of the ultimate goals of climate studies is to provide projections of future scenarios: for this purpose, sophisticated models are conceived, involving lots of parameters calibrated via observed data. The outputs of such models are used to investigate the impacts on related phenomena such as floods, droughts, etc. To evaluate the performance of such models, statistics like moments/quantiles are used, and comparisons with historical data are carried out. However, this may not be enough: correct estimates of some moments/quantiles do not imply that the probability distributions of observed and simulated data match. In this work, a distributional multivariate approach is outlined, also accounting for the fact that climate variables are often dependent. Suitable statistical tests are described, providing a non-parametric assessment exploiting the Copula Theory. These procedures allow to understand (i) whether the models are able to reproduce the distributional features of the observations, and (ii) how the models perform (e.g., in terms of future climate projections and changes). The proposed methodological approach is appropriate also in contexts different from climate studies, to evaluate the performance of any model of interest: methods to check a model per se are sketched out, investigating whether its outcomes are (statistically) consistent.


Climate-hydrology analysis (bivariate): variables (P, T, Q)
Kendall τ and Spearman ρ In this Section, a bivariate statistical analysis of the variables (P, T, Q) will be presented, for all the five available river sections (viz., Boretto, Borgoforte, Cremona, Piacenza, and Pontelagoscuro): details can be found in the paper. For each station, the following plots are presented.
Top panel. Plotted are the estimates of the Kendall τ (white bars) and the Spearman ρ (grey bars), as well as the corresponding p-Values (star markers), for the pair (P, T ), corresponding to the following data sets: Observations (Obs), Control (Ctrl), and RCP4.5 (RCP45). The dashed horizontal line corresponds to the 5% reference level.
Middle panel. Same as top panel, for the pair (P, Q).
Bottom panel. Same as top panel, for the pair (T, Q).

Obs Ctrl RCP45
Boretto: (P,T In this Section, a multivariate statistical analysis of the variables (P, T, Q) will be presented, for all the five available river sections (viz., Boretto, Borgoforte, Cremona, Piacenza, and Pontelagoscuro): details can be found in the paper. For each station, the following plots are presented, as indicated in the corresponding sub-section title.
Data, Pseudo-observations and Empirical Copulas. The left column concerns the Observations data, the middle column concerns the Control data, and the right column concerns the RCP4.5 data. , the joint distribution of (P, T ) (F:(P,T)), the copula of (P, T ) (C:(P,T)), the joint distribution of (P, Q) (F:(P,Q)), the copula of (P, Q) (C:(P,Q)), the joint distribution of (T, Q) (F:(T,Q)), the copula of (T, Q) (C:(T,Q)), the joint distribution of (P, T, Q) (F:(P,T,Q)), and the copula of (P, T, Q) (C:(P,T,Q)). The dashed horizontal line corresponds to the 5% reference level.

Kolmogorov-Smirnov (KS) and Anderson-Darling (AD) homogeneity tests
In this Section, a univariate statistical analysis of the drought occurrences will be presented. In particular, all the five available river sections (viz., Boretto, Borgoforte, Cremona, Piacenza, and Pontelagoscuro) will be considered: details can be found in the paper.
Each sub-section contains the following plots related to a single station, as indicated in the corresponding sub-section title. Here, the variables I and D are investigated, considering different pairs of data sets. The non-parametric Kolmogorov-Smirnov (KS) and Anderson-Darling (AD) homogeneity tests are used to check whether the Null hypothesis "H 0 : the (univariate) samples come from the same distribution" should be rejected. Note that the KS test is more powerful concerning the body of the (unknown) distribution, while the AD one is more specific for the tails. Shown are the p-Values of the tests: since Ties are present for D, the corresponding box-plots are over all the N R randomizations. The dashed horizontal line corresponds to the 5% reference level.
Plotted are the p-Values of the KS (white) and AD (grey) tests, for the variables I (top panel) and D (bottom panel), and the threshold Q 300 , corresponding to the following pairs of data sets: the Observations-Control (Obs,Ctrl), the Observations-RCP4.5 (Obs,RCP45), and the Control-RCP4.5 (Ctrl,RCP45).

Drought analysis (bivariate): variables (I, D)
Kendall τ and Spearman ρ In this Section, a bivariate statistical analysis of the drought occurrences will be presented. In particular, all the five available river sections (viz., Boretto, Borgoforte, Cremona, Piacenza, and Pontelagoscuro) will be considered: details can be found in the paper. Each sub-section contains the following plots, related to a single station, as indicated in the corresponding sub-section title.
Plotted are the estimates of the Kendall τ (white bars) and the Spearman ρ (grey bars), as well as the corresponding p-Values (star markers), for the pair (I, D) and the threshold Q 300 , corresponding to the following data sets: Observations (Obs), Control (Ctrl), and RCP4.5 (RCP45). The dashed horizontal line corresponds to the 5% reference level.

Obs
Ctrl RCP45 In this Section, a bivariate statistical analysis of the drought occurrences will be presented. In particular, all the five available river sections (viz., Boretto, Borgoforte, Cremona, Piacenza, and Pontelagoscuro) will be considered: details can be found in the paper. Each sub-section contains the following plots, related to a single station, as indicated in the corresponding sub-section title.