Introduction

Surface incident solar radiation (Rs) is the basic energy of biological, physical and chemical processes, and the essential input parameters of biological physics models and hydrological simulation mathematical models1,2. Ground-based stations provide the best estimate of Rs, but it is still insufficient for estimation in remote areas, especially in high latitudes, and in plateaus or mountainous areas, due to the sparsity and heterogeneity of stations3,4,5,6. Currently, there exists a range of gridded global Rs products with higher spatial resolution exist from remote sensing7,8 and reanalysis9,10. Satellite remote sensing is one of the most practical ways to derive Rs with relatively higher accuracy, but temporal coverage is limited by transit time of satellite11,12.

In addition to satellite-based products, scientists have been developing reanalysis Rs products to restore long-term historical climate records for numerical weather forecasting using data assimilation techniques since the late 1980s13. Reanalysis methods provide dynamically consistent global analysis of the global atmospheric characteristics by combining the geophysical fluid-dynamic model of the atmosphere and measurements. The model contains important physical processes, such as radiative transfer and convection. Observations are used to constrain the dynamic model to optimize the properties of complete coverage and accuracy14. Reanalysis data provides global and effective Rs of long time series, which alleviate the deficiency in radiation data and greatly promote the development of modern atmospheric science. As reanalysis data are obtained by numerical simulation, they cannot completely replace the observed data for describing the real three-dimensional state of the atmosphere15. Due to the heterogeneity of various data sources and difference of data assimilation schemes, there exist errors in radiation reanalysis products16. Thus, understanding the uncertainty and deviation of reanalysis data is a prerequisite for the rational use of reanalysis data17.

Since the mid-1990s, the United States, European Union, and Japan have organized and implemented a series of global reanalysis projects on atmospheric data to restore and reconstruct historical records of climate change. There are six representative reanalysis products: ERA-Interim of the European Centre for Medium-Range Weather Forecasts (ECMWF); MERRA-2 of the National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC)’s Global Modeling and Assimilation Office (GMAO); NCEP–NCAR reanalysis, NCEP-DOE reanalysis, Climate Forecast System Reanalysis (CFSR) from the National Centers for Environmental Prediction (NCEP); JRA-55 from the Japan Meteorological Agency (JMA). These reanalysis data are widely used in the field of atmospheric sciences, for example, surface temperature changes, regional precipitation distribution, and surface solar radiation distribution18,19,20. Among them, reanalysis Rs products are widely used in ecological conservation, agricultural production, and civil engineering and various solar energy applications21,22,23.

In recent decades, multi-source Rs data such as ground measurements and satellite inversed products, have been used to perform multi-scale verification of reanalysis Rs data at stations, regions, or on a global scale. It shows that the Rs of reanalysis data is usually larger than ground measurements in most cases24,25,26,27. Wang and Zeng24 found that both the MERRA and ERA-Interim slightly overestimate Rs by 1.56–5.00 W/m2 on a daily scale based on nine stations from the Coordinated Enhanced Observing Period (CEOP)-Asia-Australia Monsoon Project (CAMP/Tibet). Yue28 found that the annual average of Rs from MERRA is 3–4 W/m2 higher than the observed value in the Yangtze River Delta region, but their inter-annual changes are basically the same. Yue28 further analyzed the influence of aerosol content on the Rs of MERRA. Fu et al.29 assessed the applicability of four reanalysis data (NCEP-1, NCEP-2, ERA-Interim and JCDAS) in the southeast polar region by using the radiative observation data of the Panda-1 station from February 2011 to January 2012. The results showed that the ERA-Interim is state-of-the-art, and its average deviation (bias) is −22.70 W/m2. The deviation is small in summer (4.82 W/m2) and large in winter (24.70 W/m2). The applicability of reanalysis data is distinctly different in different stations, different regions or different periods. However, above studies are limited by the number of stations. Thus, more observation data are needed to confirm the applicability of reanalysis Rs. Zhang et al.17 used 674 ground-based observation stations to conduct research on the seasonal changes and spatial distribution at the global scale taking advantage of six kinds of reanalysis Rs products (NCEP–NCAR, NCEP-DOE, CFSR, ERA-Interim, MERRA, and JRA-55). The study showed that all products overestimate Rs in China, and the monthly difference between reanalysis Rs and measurements ranges from 23.15 to 71.95 W/m2. ERA-Interim and MERRA have relatively better quality in China compared with other products, and the data quality shows obvious seasonal differences. Zhang et al.17 preliminarily discussed that the underestimation of cloud coverage on the global scale may lead to the overestimation of reanalysis Rs. Boilley and Wald30 compared the meteorological reanalysis from ERA-Interim and MERRA and measurements of daily solar irradiation on surface in Europe, Africa and Atlantic Ocean, which proved that the reanalysis often mistakes cloudy conditions as clear skies. Penna et al.31 demonstrated that AOD could cause a difference of Rs up to 30 W/m2 over the Amazon region in MERRA-2, which was associated with the absorption of aerosols. In fact, solar radiation varies on spatial-temporal scales and influenced by many factors, such as cloud coverage, AOD, water vapor content, ozone concentration, surface albedo and other factors. Previous studies have concentrated on the evaluation of reanalysis Rs but ignored the causes of the errors or only consider a single factor in a large region. It is vital to have a good knowledge of the main impact factors in the different seasons and regions for the purpose of proper application and accurate data correction.

In this paper, we analyze the spatial-temporal errors of the MERRA-2 and ERA-Interim in China and identify the causes of the errors. The observation station data from the China Meteorological Administration (CMA) and Multi-layer Simulation and Data Assimilation Center of the Tibetan Plateau (DAM) are used to assess Rs of MERRA-2 and ERA-Interim. Meanwhile, the MERRA-2 and ERA-Interim products are compared to determine the difference in their spatial distributions and seasonal variations. The Clouds and Earth’s Radiant Energy Systems (CERES) Energy Balanced and Filled (EBAF) Rs dataset8 is used to identify regions with large radiation deviations in different seasons as hotspots through comparison to reanalysis radiation data. Considering the influence of atmospheric factors on the reanalysis Rs products and the spatial heterogeneity of the distribution of atmospheric factors, we introduce the Geodetector32 to quantitatively analyze the causes of the spatial-temporal errors of Rs in the hotspots and utilize CERES-EBAF atmospheric products to verify the results of the dominant atmospheric influence factors. The results are useful for the proper application and accurate data correction about the two representative global reanalysis data.

This paper is organized as follows. The reanalysis products (ERA-Interim and MERRA-2) and station measurements (CMA and DAM stations) used for this study region, the description of the quality control of the stations and the rationale of using the Geodetector are detailed in Section 2. The errors of reanalysis are validated, and the domain impact factor based on the Geodetector is analyzed in Section 3. A short summary and conclusions are presented in Section 4.

Experiments

Data

Five Rs data sources are used in this study: two reanalysis products, one satellite remote sensing based product, one ground measurement and one model simulation dataset.

Reanalysis products

Two reanalysis products, ERA-Interim and MERRA-2, are used in this study. These products are different in many aspects, such as physical parameterizations of numerical models, numerical schemes, observational data used for assimilation, and the assimilation schemes17,24. The monthly mean Rs data of these two reanalysis datasets from 2000 to 2009 are evaluated in this research. The details of these datasets are described in the following paragraphs.

The ERA-Interim9 is provided by the European Centre for Medium Range Weather Forecasts (ECMWF), which is one of the most important reanalysis centers in the world. The spatial resolution of the ERA-Interim is 0.75° (approximately 80 km) in the horizontal direction, which is divided into 60 levels from the ground surface to the 0.1hPa altitude in the vertical direction. The temporal resolution of the ERA-Interim is 3 h. An improved 3DVar assimilation technology is used as the assimilation scheme. The data sources used in the assimilation scheme include ground measurements and satellite remote sensing data. The radiation scheme is based on the Rapid Radiation Transfer Model (RRTM)33. The prognostic cloud variables (cloud cover, cloud condensed water) and water vapor from the meteorological model and climatologic values for aerosols, carbon dioxide, trace gases and ozone are used in the radiative transfer model30,34.

The MERRA-210 is provided by the GSFC’s Global Modeling and Assimilation Office (GMAO). It was introduced to replace the original MERRA dataset because of the advances made in the assimilation system that enable assimilation of modern hyperspectral radiance and microwave observations, along with GPS-Radio Occultation datasets. It also uses NASA’s ozone profile observations that began in late 2004. Additional advances in both the GEOS (Goddard Earth Observing System Data Assimilation System) model and the GSI (Gridpoint Statistical Interpolation) assimilation system are included in MERRA-2. Spatial resolution remains about 50 km in the latitudinal direction. Along with the enhancements in the meteorological assimilation, MERRA-2 takes some significant steps towards GMAO’s target of an Earth System reanalysis. The radiation scheme is based on the method proposed by Chou and Suarez35. In the MERRA-2 reanalysis, the meteorological and aerosol observations are simultaneously assimilated within the GEOS-5. The MODIS Neural Net Retrieval and AVHRR Neural Net Retrieval at 550 nm are used to assimilate the AOD observations30.

CMA ground measurements

The CMA provides ground radiation measurements of 122 stations in China (except Taiwan, Hong Kong, and Macao). The CMA surface observations include three basic physical variables: the total surface solar radiation Rs, direct solar radiation Rdir, and diffuse radiation Rdif from 1957 to the present. The instruments of stations used are: DFY-4 and TBQ-2 total radiation meter; DFY-3 and TBS-2 direct radiation meter (both with solar tracking frame); DFP-1 shading ring; and RYJ-4 automatic radiation recorder. The above-mentioned radiation meters are all electrothermal type, which consists of two parts: the induction surface and the thermopile36,37. The radiation meter and measuring instrument (voltmeter, ammeter) constitute a set of radiation instruments. Detail information of instruments for CMA stations are available on the website: http://data.cma.cn/site/index.html. The world radiation reference (WRR) is used as the standard to carry out the value transmission in these stations, ensuring the comparability of solar radiation measurements worldwide. The distribution of these radiation stations is shown in Fig. 1 (red triangles).

Figure 1
figure 1

Geographical distribution of the observation stations (812 stations in total) used in this study from CMA (96 stations in red triangles), and DAM (716 stations in black dots). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image

Daily surface solar radiation dataset of the DAM

Tang et al.38,39,40 developed hybrid model and daily meteorological data (including air temperature, atmospheric pressure, relative humidity, sunshine duration hours, and precipitation) to estimate the daily Rs of the 716 CMA meteorological stations based on published data from the 716 regular meteorological stations of the Weather Information Centre of the CMA. Neural network models are built on each CMA 96 radiation station, and then the trained models are used to estimate the Rs of the 716 CMA stations. Finally, the estimation results of the artificial neural network model are used to correct those of the hybrid model for the purpose of obtaining 716 daily Rs data in China from 1961 to 2010 (http://www.tpedatabase.cn/portal/MetaDataInfo.jsp?MetaDataId=249399). The sparse and uneven spatial distribution of CMA stations is compensated in this way. The distribution of the radiation stations is also shown in Fig. 1 (black circles).

CERES-EBAF product

The CERES-EBAF product is derived from the clouds, AOD, and earth radiant energy systems detected by the TRMM, Terra and Aqua satellites. It provides a reasonable inversion of atmospheric parameters such as clouds, water vapor and AOD and Rs estimation. Atmospheric parameters are derived from A-train Constellation (the Cloud Aerosol Lidar with Orthogonal Polarization (CALIOP) on the Cloud Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite, the CloudSat Cloud Profiling Radar (CPR), and the Aqua Moderate Resolution Imaging Spectrometer (MODIS)). Rs are derived from a radiative transfer model of CERES with k-distribution and correlated-k for radiation (FLCKKR) with a two stream approximation, which is consistent with the radiative flux from the surface to the top of the atmosphere41. The CERES-EBAF product contains 1° regional, zonal and global monthly means of Top-of-Atmosphere (TOA) and surface (SFC) longwave (LW), shortwave (SW), and net (NET) fluxes under clear and all-sky conditions. The CERES-EBAF has higher accuracy than the other grid Rs products like CMIP5, NCEP-NCAR, NCEP-DOE, CFSR, ERA-Interim, MERRA and JRA-55 et al.17,41,42. Zhang et al.17 found that the CERES-EBAF Rs data are more consistent with ground measurements than the reanalysis data, and provide more accurate atmospheric products such as clouds and aerosols. Li et al.43 proved that AODs used by CERES have mean biases less than 0.1 and small Root Mean Square Error (RMSE) over most sites in China compared with China Remote Sensing Network (CARSNET). Yan et al.44 found that the time series of CERES cloud properties, for the most part, closely track the variations of the surface retrievals and their correlation coefficient reaches up to 0.83. Liu et al.45 evaluated the water vapor product of MODIS at the 83 stations in China. The results show that the water vapor has high correlation coefficient (more than 0.91) with station measurements, yielding a mean bias and RMSE less than 2.58 mm and 6.02 mm respectively. The EBAF Rs, AOD, Cloud Coverage and Water Vapor Content of CERES are used in this study. Due to the lack of higher spatiotemporal resolution ground measurements, in this paper, the CERES data were averaged in a long time series for subsequent analysis and comparisons, which greatly reduced the uncertainty of the data and enhanced the credibility of the experiment.

Methodology

Station quality control

There are some quality problems in the CMA ground measurements, resulting from the accuracy and calibration of radiation instruments, the human factors in the operation of instruments, and the positional changes in stations46. When using CMA radiation data as validation data, a quality inspection should be carried out first. The original station data have been checked by a simple physical quality control of the daily Rs data; that is, the Rs is equal to the sum of diffuse radiation and the direct radiation (Original threshold control: \({R}_{s}={R}_{dif}+{R}_{dir}\)). Furthermore, an upper limit is set for further validation. Referring to Shi’s research37, the upper limit is the solar radiation G0 received by the earth’s top atmosphere per day:

$${G}_{0}=\frac{24\times 3600}{\pi }{I}_{0}k({\rm{c}}{\rm{o}}{\rm{s}}\varphi {\rm{c}}{\rm{o}}{\rm{s}}\delta {\rm{s}}{\rm{i}}{\rm{n}}{w}_{s}+\frac{\pi {w}_{s}}{180}\,\sin \,\varphi \,\sin \,\delta )$$
(A1)

where I0 is the solar constant (approximately 1367 W/m2), δ is the solar declination angle, φ is the station latitude, ws is the sunset angle, and k is the earth orbit eccentricity correction factor. The daily earth orbit eccentricity correction factor k is calculated using the following formula47:

$$k=1+0.033{\rm{c}}{\rm{o}}{\rm{s}}(\frac{360{d}_{n}}{365})$$
(A2)

where k is the ordinal day of the year and dn represents the day of the year. The daily solar declination angle δ and sunset angle ws are calculated using the following formulas48,49:

$$\delta =23{\rm{.45sin}}(360\frac{284+{d}_{n}}{365})$$
(A3)
$${w}_{{\rm{s}}}={\rm{c}}{\rm{o}}{{\rm{s}}}^{-1}(-\,\tan \,\phi \,\tan \,\delta )$$
(A4)

From a physical point of view, it is difficult to set a strict lower limit for the Rs. Due to the existence of scattered radiation; the Rs must be greater than zero. Geiger et al.50 based on the statistical characteristics of the radiation data, the lower limit is set to 3% of the daily solar radiation received by the top of the earth’s atmosphere every day (see Eq. (A5)). Such simple quality control is proved to be effective can eliminate most of the error51,52,53,54. Moreover, CMA data is checked for errors at daily scale. The final used data is the monthly average data calculated from daily Rs. Table 1 summarizes the results of the physical threshold test. From the table, we can see that the incidence of errors is lower in summer. The average error rate of the daily Rs observations from 2000 to 2009 is 1.98%, and the winter error rate is higher than that of the summer. In summary, only ground measurements that pass the following physical criterion are used for validation.

$$0.03\ast {G}_{0} < {R}_{s} < {G}_{0}$$
(A5)
Table 1 Monthly distribution of the number of days for daily Rs data of CMA failing the physical threshold test from 2000 to 2009.
Full size table

Considering that the CMA stations are sparse and unevenly distributed, the 716 Rs data of the DAM is also used in this study. The daily surface solar radiation dataset was produced by merging two data sets. One is the hybrid model estimate at 716 CMA stations and the other is the ANN-based model estimate at 96 radiation stations. The latter, which has higher accuracy, was used to correct the hybrid model estimate dynamically at a monthly scale. Tang et al.39,40 verified the accuracy of the assimilated radiation data set And showed that the bias and RMSE of the hybrid model at the 96 CMA radiation stations are 0.7 and 2 MJ/m2 respectively, While that of the corrected hybrid model are −0.1 and 1.8 MJ/m2 respectively. The accuracy of the corrected radiation dataset is significantly higher than that of the traditional locally calibrated model. Therefore, the 716 Rs data can be used as valid data for evaluating the reanalysis data. In this study, both CMA and DAM are used to evaluate the reanalysis products for a better comparison. But only DAM is used to analyze the error causes of reanalysis because it is highly consistent with CMA and has more data than CMA.

Time series estimation

In this study, the Rs data from 2000 to 2009 by ERA-Interim and MERRA-2 are compared with the CMA and DAM measurements, and the data error of reanalysis at different time scales is analyzed. The daily Rs estimates of ERA-Interim and MERRA-2 are horizontally interpolated using a bilinear interpolation technique with inverse distance weights for four most closest surrounding grid cells30.

To investigate the error distribution of the reanalysis data at the different stations, the seasonal distribution of the average relative bias (RB) for the reanalysis and measured Rs from 2000 to 2009 is calculated. The expression of the RB is as follows:

$$RB=\frac{{R}_{s\_Reanalysis}-{R}_{s\_sites}}{{R}_{s\_sites}}\ast 100$$
(A6)

where \({R}_{s\_Reanalysis}\) and \({R}_{s\_sites}\) represent the Rs of reanalysis products and stations, respectively. Where RB indicates relative bias, and positive values, indicate that the reanalysis data is higher than the ground observation data, while negative values indicate that the reanalysis data is lower than the ground observation data.

The expression of the monthly anomalies is as follows:

$$D{A}_{i}=SS{R}_{i}-\overline{SSR}$$
(A7)

where SSRi and \(\overline{SSR}\) are the average monthly Rs and the average monthly Rs for all years of that month, respectively.

The expression of the RMSE is as follows:

$$RMSE=\sqrt{\frac{1}{m}{\mathop{\sum }\limits_{i=1}^{m}({R}_{s\_\mathrm{Re}analysis}-{R}_{s\_sites})}^{2}}$$
(A8)

where m is the numbers of month.

Geodetector

The Geodetector is a method used to explore the influence mechanism of geographical spatial zoning factors on disease risk in infancy55. It can effectively identify the relationship between multiple factors and geographical phenomena, so it has been gradually applied to the study of geography and humanities56,57. The factor detector of the Geodetector can verify the spatial heterogeneity of a single variable. Atmospheric factors such as AOD, cloud coverage and water vapor content are typical category variables and have an important influence on the reanalysis of Rs. Therefore, it is suitable to use the Geodetector method to reveal the influence of the regional atmospheric factors on the reanalysis of the surface radiative error. The Geodetector software is divided into four parts: the risk detector (superposition of the related data, then comparison of whether the difference is significant, and the major role of the significant factor in the risk is determined), the factor detector (using the q value55 to test the association between two variables Y and X, according to the coupling between their spatial distributions, without assumption of linearity), the ecological detector (using the variance to compare), and the interaction detectors (including synergy, antagonism, double synergy, single antagonism and mutual independence). The factor detector section is used in this study.

To analyze the significance of the atmospheric influence factors on the Rs errors in the different regions of China, we actually analyze 10 factors at first, including the reanalysis ozone concentration, surface albedo, content of ice and water clouds, cloud coverage, AOD and water vapor content, etc. It is found that the influence of AOD, cloud coverage and water vapor content on the Rs is obvious, while other factors are not significant. This is basically consistent with the results of sensitivity analysis of radiation transmission models47. Therefore, this paper focuses on the quantitative influence level of AOD, cloud coverage and water vapor content on the Rs error in the different regions. Taking the RB of Rs at each regional station as the spatial stratification variable Y, the influence of atmospheric factors on the RB of reanalysis Rs is measured based on the power of determinant (PD) value of the Geodetector model. The region used to analyze the cause of error must satisfy the following conditions: the station distribution in the region is even and as many stations as possible, and error analysis is conducted on the region where the reanalysis Rs deviation is large. These regions are called hotspots in this study. Studies3,8,17,42 have shown that CERES-EBAF has a higher accuracy than the other grid Rs products; thus, the CERES-EBAF Rs data are used as verification data. Zhang et al.17 proved that almost all the reanalysis Rs products showed better accuracy in summer (June, July, and August) than in winter (December, January, and February), and the difference between summer and winter is more prominent than other seasons. Section 3.1 of this study further confirmed that both ERA-Interim and MERRA-2 have lager frequency distribution with smaller RB in summer than winter (Fig. 2). So, the reanalysis monthly mean Rs products were divided into summer and winter seasons to assess the seasonal error causes (Fig. 3). To perform the comparison, a bilinear interpolation of a weighted average of pixel in the nearest 2-by-2 neighborhood was used to unify the resolution of the reanalysis products to CERES-EBAF. According to the map, this study selects the large deviation regions of the reanalysis Rs in summer and winter as a hotspot area. Two hotspots in each season are selected, and a total of 8 hotspots are numbered 1–8. For the ERA-Interim, the southwestern region (hotspot 1) and the eastern region (hotspot 2) are selected as the hotspots during the summer, while the southern coastal region (hotspot 3) and the northeastern region (hotspot 4) are selected in winter. For the MERRA-2, the southern coastal region (hotspot 5) and the Sichuan Basin region (hotspot 6) are selected in summer, and the southern coastal region (hotspot 7) and the north-eastern region (hotspot 8) are selected in winter. The same region but different hotspot numbers was defined when the seasons or reanalysis product are different for better comparison, such as the hotspots 3, 5 and 7, and the hotspots 4 and 8.

Figure 2
figure 2

RB frequency distribution of reanalysis radiation products (ERA-Interim and MERRA-2) and ground measurements (CMA and DAM) from 2000 to 2009, (a,b) represent the RB frequency distribution of ERA-Interim and CMA or DAM in different months, respectively. (c,d) represent the RB frequency distribution of MERRA-2 and CMA or DAM in different months, respectively. The vertical axis of frequency distribution (month) represents the numbers of month which RB is in the classification range of the horizontal axis.

Full size image
Figure 3
figure 3

The average Rs hotspot regions’ selection of ERA-Interim and MERRA-2 from 2000 to 2009 ((a,b) represent ERA-Interim’s hotspot regions in summer and winter, respectively. (c,d) represent MERRA-2’s hotspot regions in summer and winter, respectively.). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image

Based on the reanalysis seasonal differences, we use Geodetector to analyze the causes of the errors in the hotspots. As shown in Fig. 4, K-means clustering analysis is first used to divide the hotspot into k sub-regions with atmospheric factor X. The number of Rs stations and the variance of the RB of the Rs of the stations in the sub-regions are recorded as \({N}_{d1},\,{N}_{d2},\ldots ,{N}_{dk}\) and \({\sigma }_{d1}^{2},\,{\sigma }_{d2}^{2},\ldots ,{\sigma }_{di}^{2}\) respectively.

$$PD=1-\frac{{\sum }_{i=1}^{{\rm{K}}}({N}_{di}\times {\sigma }_{di}^{2})}{{N}_{D}\times {\sigma }_{D}^{2}}$$
(A9)

where i=1, 2, 3, …, k, represents the number of sub-regions; Ndi is the number of stations in the sub-region of the i; ND is the number of stations in the entire hotspot, \({N}_{D}={\sum }_{i=1}^{{\rm{K}}}({N}_{di})\); and \({\sigma }_{D}^{2}\) represents the discrete variance of the RB of the Rs over the whole hotspot’s station. The study area is divided into k sub-regions based on atmospheric factors. When the atmospheric factor X has a decisive force on the RB of the station, the gap between the atmospheric factors in each sub-region is small while the gap between the sub-regions is large. The discrete variance \({\sigma }_{di}^{2}\,\) of the station is small in each sub-region, while the variance between the sub-regions is very large. When \({\sigma }_{di}^{2}\) is close to 0, the PD value tends to be 1, which is the ideal state indicating that the RB changes are completely determined by atmospheric factor X. If the RB of the Rs is irrelevant to the atmospheric factor, then the weighted sum of the discrete variance \({\sigma }_{di}^{2}\) and the number of stations in each sub-region Ndi is closer to VarD \((VarD={N}_{D}\times {\sigma }_{D}^{2})\), thus PD = 0. Therefore, the range of the PD value is [0, 1]. A larger PD value indicates a greater correlation between the reanalysis atmospheric factors and RB of the reanalysis Rs.

Figure 4
figure 4

The principle of the geographical detector.

Full size image

Results and Discussion

Correlation and trend analysis

Figure 5 shows the scatter plots of monthly average surface radiation for the reanalysis and stations (CMA and DAM) from 2000 to 2009. The correlation coefficient (R) between ERA-Interim and station measurements is 0.91 and 0.89 respectively; the RMSE values are 32.36 and 28.43 W/m2 respectively; and the bias is 18.28 and 15.79 W/m2, respectively. The R between the MERRA-2 and stations is 0.93 and 0.98, respectively; the RMSE is 45.76 and 44.96 W/m2, respectively; and bias is 43.84 and 35.68 W/m2, respectively. The difference between the two reanalysis products and CMA or DAM is very small; the difference of RMSE is 3.93 and 0.80 W/m2, respectively; and the difference of the bias is 2.49 and 8.16 W/m2, respectively, which further illustrates that the DAM datasets have high consistency with CMA and can be used as validation data. In addition, the experiment shows that the reanalysis of the Rs data by ERA-Interim and MERRA-2 is higher than the ground observation station data, which is consistent with the previous conclusions17,58,59. The biases of the ERA-Interim and MERRA-2 are 15.79–18.28 W/m2 and 35.68–43.84 W/m2, respectively, and the RMSEs are 28.43–32.36 W/m2 and 44.96–45.76 W/m2, respectively. In general, the Rs in china is overestimated by both reanalysis products, but the overestimation by MERRA-2 is more obvious.

Figure 5
figure 5

Average monthly Rs scatter plots between reanalysis products (ERA-Interim and MERRA-2) and surface measurements (CMA and DAM) from 2000 to 2009. (The color code represents the dot density, and the dot density in the red area is higher than that in the blue area. Deseasonalized correlations in the lower right corner.).

Full size image

Reanalysis shows seasonal differences (Table 2), and Figs. 6 to 7 show the seasonal scatter plots of the reanalysis with the stations from 2000 to 2009. The ERA-Interim shows obvious seasonal differences under the verification of CMA and DAM. The ranges of bias and RMSE in summer and autumn are 3.04–12.08 W/m2 and 18.51–27.05 W/m2, respectively, but are 23.02–38.21 W/m2 and 21.51 W/m2–38.21 in spring and winter, respectively, which is higher than that in summer and autumn season. The seasonal characteristics of the MERRA-2 are not obvious, and the range of bias in summer and autumn is 29.55–45.62 W/m2, which close to that in spring and winter. However, the range of the RMSE in spring and winter is 35.59–53.09 W/m2, which is slightly higher than that in summer and autumn.

Table 2 Evaluation of the seasonal Rs from the ERA-Interim and MERRA-2 products using surface measurements collected from the CMA and DAM stations from 2000 to 2009.
Full size table
Figure 6
figure 6

Seasonal average monthly Rs scatter plots between ERA-Interim and surface measurements (CMA and DAM) from 2000 to 2009. ((A1) – (A4) represent the scatter plots of the average monthly Rs of ERA-Interim and CMA in different seasons. (B1) – (B4) represent the scatter plots of average monthly Rs of ERA-Interim and DAM in different seasons. The color code represents the dot density, and the dot density in the red area is higher than that in the blue area. Deseasonalized correlations in the lower right corner.).

Full size image
Figure 7
figure 7

Seasonal average monthly Rs scatter plots between MERRA-2 and surface measurements (CMA and DAM) from 2000 to 2009. ((C1) – (C4) represent the scatter plots of average monthly Rs of MERRA-2 and CMA in different seasons. (D1) – (D4) represent the scatter plots of average monthly Rs of MERRA-2 and DAM in different seasons. The color code represents the dot density, and the dot density in the red area is higher than that in the blue area. Deseasonalized correlations in the lower right corner.).

Full size image

As shown in Figs. 2 and 8, the monthly mean Rs of the four datasets peaks around July, reaching a valley value around January every year. The monthly mean Rs of the datasets in annual cycle is shown in Fig. 8(a). The CMA and DAM datasets are basically fitted with a range from 80 to 230 W/m2. The Rs ranges of the ERA-Interim and MERRA-2 are 97–249 W/m2 and 105–280 W/m2, respectively. The MERRA-2 is higher than the ERA-Interim and surface measurements. The monthly mean anomalies Rs of the four datasets in annual cycle is shown in Fig. 8(b). The CMA and DAM datasets are basically fitted with a range from −11 to 12 W/m2. The Rs anomaly ranges of the ERA-Interim and MERRA-2 are −16–13 W/m2 and −13–11 W/m2, respectively. It can be seen that larger values of the Rs relative anomalies are more frequent and of larger magnitude in the MERRA-2/CMA-DAM than the ECMWF/CMA-DAM as shown in Fig. 2(c), implying that Rs of MERRA-2 is more changeable comparing to the ECMWF. The frequency distribution diagrams between the reanalysis and surface measurements are shown in Fig. 2(a–d). From June to October, the distribution frequency of ERA-Interim is higher with a smaller RB (−5 to 10%), while from December to April, the distribution frequency is higher with a larger RB (15 to 35%). Although the seasonal difference is not obvious in the MERRA-2, it can also be seen that the RB is smaller (10 to 25%) and the frequency is higher in June to October, but larger (25 to 35%) from December to April. This results are consistent with studies of Zhang et al.17 and Jia et al.58. Both ERA-Interim and MERRA-2 have lager frequency distribution with smaller RB in summer than winter and the difference between summer and winter is more prominent than other seasons (Fig. 2). The seasonal differences in the reanalysis products may be due to the seasonal differences in atmospheric factors mentioned in Introduction Section, which will be further analyzed in section 3.3 and 3.4.

Figure 8
figure 8

Time series of reanalysis radiation products (ERA-Interim and MERRA-2) and ground measurements from 2000 to 2009 (CMA and DAM), (a) monthly mean, (b) anomalies, (c) relative anomalies.

Full size image

Bias and error assessments

The reanalysis monthly mean products are divided into the summer and winter seasons to assess the seasonal dependency of their accuracy. Figure 9 shows the average Rs distribution in the different seasons of ERA-Interim and MERRA-2 from 2000 to 2009. The ranges of the radiation in summer are 100.72–309.06 W/m2 and 186.10–359.12 W/m2 for ERA-Interim and MERRA-2, respectively. The ranges in winter are 78.45–233.97 W/m2 and 56.52–222.53 W/m2, respectively. The largest Rs in summer is located in Qinghai, Tibet and Xinjiang. The smallest Rs in winter is located in Sichuan and Guizhou. The Rs of the MERRA-2 in summer is higher than that of the ERA-Interim in summer, while it is lower than that of the ERA-Interim in the north and Sichuan Basin in winter. The ERA-Interim and MERRA-2 basically show the same distribution characteristics of Rs in China, but there are obvious differences in the Rs values.

Figure 9
figure 9

Average Rs distribution in the different seasons of ERA-Interim and MERRA-2 from 2000 to 2009. ((a,b) represent the radiation of ERA-Interim in summer and winter, respectively. (c,d) represent the radiation of MERRA-2 in summer and winter, respectively). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image

To investigate the error distribution of the reanalysis data at the different stations, the seasonal distribution of average RB between ERA-Interim and surface measurements from 2000 to 2009 is shown in Figs. 10 and 11, respectively. For the ERA-Interim, in summer, the stations with positive RB values are mainly distributed in southeast China, central east China and the Sichuan Basin. The maximum RB of the CMA stations reaches 38%, and the maximum RB of the DAM stations reaches 35%. The stations with negative RB values are mainly distributed in plateaus and in southwest and northeast China. The smallest RB of the CMA stations is −32%, and the smallest RB of the DAM stations is −48%. In contrast, the RB is basically positive in winter, indicating that the ERA-Interim Rs data show overestimation. Specifically, the stations with large positive RB values are mainly distributed in the southeast China, central east China and the Sichuan Basin, which is the same as that in summer, and the maximum RB of the CMA stations is larger than 50%, and the maximum RB of the DAM station reaches 45%. In addition, the ERA-Interim shows seasonal differences; for example, the Rs in the northeast and southwest regions is underestimated during the summer and overestimated during the winter. The Rs in the southeast region, the central east regions and the Sichuan Basin are overestimated, but the overestimation is more pronounced in winter.

Figure 10
figure 10

Seasonal distribution of the average Rs RB between ERA-Interim and surface measurements (CMA and DAM) from 2000 to 2009. ((a,b) represent the average radiation RB of ERA-Interim and CMA in summer and winter, respectively. (c,d) represent the average radiation RB of ERA-Interim and DAM in summer and winter, respectively.). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image
Figure 11
figure 11

Seasonal distribution of average Rs RB between MERRA-2 and surface measurements (CMA and DAM) from 2000 to 2009. ((a,b) represent the average radiation RB of MERRA-2 and CMA in summer and winter, respectively. (c,d) represent the average radiation RB of MERRA-2 and DAM in summer and winter, respectively.). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image

For the MERRA-2, in summer, the stations with a positive RB are mainly distributed in southwest China, central east China and the plateaus. The maximum RB of the CMA stations reaches 61%, and the maximum RB of DAM stations reaches 81%. The stations with a negative RB are mainly distributed in southeast China, especially in the coastal areas. The smallest RB of the CMA stations is −12%, and the smallest RB of the DAM stations is −17%. The errors may be affected by cloud coverage and water vapor content in the coastal areas in summer, which will be further analyzed in section 3.3 and 3.4. In winter, the RB is basically positive and more pronounced than in summer. The Rs data show overestimation. The stations with large positive RBs are mainly distributed in southeast and central east China. The maximum RB of the CMA and DAM can exceed 90%.

The data quality of reanalysis Rs showed regional and seasonal characteristics. The underestimation of ERA-Interim mainly occurred in the southwestern and northeastern regions. The underestimation of MERRA-2 in summer is mainly in the southern coastal region and in the northeastern region in winter. In general, both reanalysis data overestimate the Rs in most regions of China, and the overestimation by MERRA-2 is more obvious. The MERRA-2 and ERA-Interim both have large errors in the Sichuan Basin, possibly because of the reanalysis of atmospheric parameters, especially the influence of water vapor and cloud coverage. The further analysis will be shown in Section 3.3.

Analysis of influence factors

The ECMWF only has MACC (Monitoring Atmospheric Composition and Climate) AOD reanalysis product, which is different from AOD used in ERA-Interim. The AOD used in the ERA-Interim radiation transfer model is Climatology AOD data, which don’t have long-term variation. So, there is no AOD product provided in the ERA-Interim, we don’t consider the impact of AOD to the Rs of ERA-Interim. Table 3 shows the PD values for each environmental factor in hotspot areas. The table header gives the names of the environmental factors: AOD, cloud coverage and water vapor content. For the ERA-Interim, in summer, the Rs error of the reanalysis product is mainly influenced by cloud coverage in hotspot 1 with the PD values of 0.4241. Although the influence of the water vapor content is small, the impact on the Rs could not be ignored since the PD value is also greater than 0.1000. Hotspot 2 is mainly affected by cloud coverage, and the PD value is as high as 0.2410, and the influence of water vapor content can be ignored as PD value (0.0912) is less than 0.1000. In winter, the dominant factors in hotspots 3 and 4 are cloud coverage, and the PD value of these factors is 0.3057, 0.2125, respectively. The water vapor content in hotspots 3 and 4 also passed the significance test, but the PD value is small. For the MERRA-2, in summer, the influence factors of hotspot 5 are ordered as follows: AOD > cloud coverage > water vapor content, and the PD value is 0.2452, 0.1342 and 0.1244 respectively. The influence factors of hotspot 6 are ordered as follows: AOD > cloud coverage > water vapor content, and the PD value is 0.2124, 0.1623 and 0.1398, respectively. In winter, Hotspots 7 and 8 are affected by AOD, cloud coverage and water vapor content, but they are mainly influenced by AOD in hotspot 7 and cloud coverage in hotspot 8. In summary, cloud coverage and AOD are the main influencing factors for the two reanalysis Rs products, and with the increase in cloud coverage, the influence of the water vapor content cannot be ignored.

Table 3 Seasonal PD values of the hotspot regions in China (the calculation of the influence factors is based on the DAM datasets, the numbers 1–8 represent different hotspots).
Full size table

Atmospheric parameter verification

Section 3.3 explained the PD value of the atmospheric influence factors in each hotspot. The section determines whether the influence factors are representative and accurate. As shown in Figs. 12–15, the average cloud coverage, AOD and water vapor content of CERES-EBAF from 2000 to 2009 are compared with the ERA-Interim and MERRA-2 atmospheric products. The PD value (Table 3) and the deviation of the radiation (Fig. 3) in the hotspots are combined to verify whether the change in the atmospheric parameters of the reanalysis is consistent with the dominant atmospheric factors in the hotspots.

Figure 12
figure 12

Comparison of the ERA-Interim and CERES average atmosphere product in summer on a regional scale from 2000 to 2009. ((a,b) correspond to ERA-Interim, CERES and ERA-Interim minus CERES, respectively.). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image
Figure 13
figure 13

Comparison of the ERA-Interim and CERES average atmosphere product in winter on a regional scale from 2000 to 2009. ((a,b) correspond to ERA-Interim, CERES and ERA-Interim minus CERES, respectively.). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image
Figure 14
figure 14

Comparison of the MERRA-2 and CERES average atmosphere product in summer on a regional scale from 2000 to 2009. ((a–c) correspond to MERRA-2, CERES and MERRA-2 minus CERES, respectively.). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image
Figure 15
figure 15

Comparison of the MERRA-2 and CERES average atmosphere product in winter on a regional scale from 2000 to 2009. ((a–c) Correspond to MERRA-2, CERES and MERRA-2 minus CERES, respectively.). (Generated by Arcgis 10.7 software, https://www.esri.com/en-us/home).

Full size image

For the ERA-Interim, the Rs is underestimated in the hotspot 1 (Fig. 3a), and the order of influence power of atmospheric factors is cloud coverage > water vapor (Table 3). The underestimation of the water vapor only can compensate part of the underestimation of the Rs due to overestimation of the cloud coverage. Because cloud coverage is the dominant factor of this region (Fig. 12a), whose power is much larger than that of water vapor (Table 3). The dominant factor in hotspot 2 is the cloud coverage (Table 3), and the underestimation of cloud coverage (Fig. 12a) leads to the overestimation of the Rs in the hotspot 2 (Fig. 3a). It is difficult to obtain an accurate Rs in the southeast coastal areas (hotspot 3), because of the rapid dynamic change of the cloud cover. The results show that the dominant factors in hotspot 3 is cloud coverage (Table 3), which is underestimated (Fig. 13a), resulting in an overestimation of the ERA-Interim Rs in this region (Fig. 3b). In hotspot 4, the influence of cloud coverage is larger than that of other factors (Table 3). The cloud coverage is overestimated (Fig. 13a), which led to the underestimation of the Rs (Fig. 3b). Boilley and Wald30 also proved that ERA-Interim often mistakes cloudy conditions as clear skies. The opposite is also true though less pronounced: actual clear sky conditions are predicted as cloudy.

For the MERRA-2, the Rs is slightly underestimated in the southeast coastal areas (hotspot 5), and the order of influence power of atmospheric factors is AOD > cloud coverage > water vapor (Table 3). The underestimation of the AOD cannot compensate the underestimation of Rs due to the overestimation of cloud coverage and water vapor content (Fig. 14a–c), because the sum of power of cloud coverage and water vapor (PD = 0.2586) is greater than AOD (PD = 0.2452). The Rs in the Sichuan Basin (hotspot 6) is affected by the topography of the basin, and the estimation of the Rs is complicated. The influence power of AOD and cloud coverage is greater than water vapor content (Table 3). Therefore, the overestimation of water vapor content cannot compensate the overestimation of Rs due to the underestimation of AOD and cloud coverage (Fig. 14a,c). Therefore, the Rs in this region is overestimated (Fig. 3c). The dominant factors’ order in hotspot 7 is as follows: AOD > cloud coverage > water vapor content (Fig. 15a–c), and all the influence factors are obviously underestimated in this area, which led to the overestimation of the Rs in hotspot 7, as shown in Fig. 3d. The cloud coverage and AOD in hotspot 8 are overestimated (Fig. 15a,b), resulting in an underestimation of the Rs in this area. The results consistent with Feng and Wang’s60 study that MERRA-2 have a high mean bias over China due to their incorrect estimation of cloud fraction, which is greater in southern China. The bias in trend tend to reduce due to its underestimation of aerosol assimilation. However, MERRA-2 show a positive bias in trend of Rs likely due to aerosol-cloud interaction.

The study shows that the Rs error of the MERRA-2 and ERA-Interim in the southeast coastal areas are mainly influenced by the cloud coverage, and when the PD value of cloud coverage is great, the influence of water vapor content also becomes greater. The error causes in the Sichuan Basin are mainly affected by AOD and water vapor content, and those in the northeast and middle east of China is mainly affected by cloud coverage and AOD.

Conclusions

This study presents the validation and inter-comparison of the reanalysis Rs estimation from 2000–2009 provided by the ERA-Interim and MERRA-2 using quality-controlled surface measurements at 96 stations from CMA and 716 Rs data from the DAM. The reanalysis products are also compared with the satellite retrievals of CERES-EBAF from different perspectives, including accuracy, spatial distribution, periodic variation, seasonal variation, inter-annual variation and regional variation. In addition, considering the influence of atmospheric factors on the earth’s Rs and the spatially stratified heterogeneity of the atmospheric distribution, the geographical detector is used to find out the errors causes in different hotspots in China.

Overall, the bias between the ERA-Interim products and surface measurements at all stations range from 15.79 to 18.28 W/m2, and the RMSE ranges from 28.43 to 32.36 W/m2. The bias between the MERRA-2 products and surface measurements at all stations range from 35.68 to 43.84 W/m2, and the RMSE ranges from 44.96 to 45.76 W/m2. Both the ERA-Interim and MERRA-2 reanalysis radiation data overestimate Rs in China, and the overestimation of MERRA-2 is more pronounced. The ERA-Interim has strong seasonal differences, and data in summer and autumn are better than those in spring and winter. In contrast, MERRA-2’s seasonal differences are not obvious. The study shows that Rs error of MERRA-2 and ERA-Interim in the southeast coastal areas are mainly influenced by cloud coverage. When the PD value of cloud coverage is large, the influence of water vapor content also becomes greater. The error in the Sichuan Basin is mainly affected by AOD and water vapor content, and the error in the northeast and middle east of China is mainly affected by cloud coverage and AOD.

Generally, further improvements and efforts are required for higher accuracy by using more surface measurements, and improvement of the accuracy in the atmospheric parameters would make the verification results more convincing. The results are useful for the proper application and accurate data correction about the two representative global reanalysis data. Further study will focus on establishing the empirical relationship between the factors’ PD values and the radiative errors of reanalysis, as well as correct the radiative errors.