Introduction

Increasing global temperature driven by growing greenhouse gas (GHG) emissions is a major global concern1. GHGs mainly result from the energy-related combustion of fossil fuels2. The question emerges, which factors drive energy demand and emissions, and to what extent they are malleable. This question is relevant at many levels, from products to individuals, households, and cities up to continental and even global totals. We here focus on countries, because many decision-making processes occur at the national level. A widespread approach for cross-country analyses classifies the factors influencing resource use and emissions into population, affluence, and technology3,4,5. These factors are captured in the STIRPAT (Stochastic Impacts by Regression on Population, Affluence and Technology) framework3, derived from the classical IPAT (impact = population × affluence × technology) approach5,6.

Economic activity (called affluence in IPAT-style analyses, usually measured as Gross Domestic Product or GDP) is acknowledged as a major determinant of energy use and greenhouse gas (GHG) emissions7,8,9. The discussion mainly focuses on whether GDP can be decoupled from emissions, i.e., whether energy use and emissions can be reduced while GDP is growing. This may be possible e.g., through more efficient technologies, but the debate is so far inconclusive10,11. While recent studies revealed examples of growing national economies where policies implemented in the last decade achieved reductions in energy demand and CO2 emissions12,13,14, neither those studies nor a meta-analysis15 yielded evidence for reductions in energy use and GHG emissions consistent with ambitious climate targets.

Other potential determinants of energy demand and emissions receive less attention than GDP, even though additional entry points to accelerate decarbonization are urgently required. Population density has been studied with varied outcomes. An econometric study of OECD countries 1980–2011 suggested an inverse relation between population density and CO2 emissions16; a regression analysis of materials used in >100 countries in 2000 found a similar effect17. A global regression analysis18 found no effect of population density on energy demand, whereas a panel analysis of 11 Asian countries from 1960–200419 to conditions that influence energy demand for heating or cooling of buildings20, fuel prices affecting demand for transport energy21, and the urbanization rate (urban population as percent of the total population). A suitable proxy for the climate dependency of heating energy demand are heating-degree days (abbreviated as HDD)20. The price of gasoline (abbreviated as PGAS) is an energy price indicator that is strongly related to settlement patterns21. Some studies used urban population as a percent of the total population (abbreviated as UPOP) as a development indicator22.

At the urban scale, the influence of population density and the spatial layout of urban areas on cities’ resource demand has been widely studied20,23,24,25,26,27,28. There are several reasons why the extent and spatial layout (density and form) of built structures—henceforth denoted as material stock patterns29, i.e., the spatial patterns of societies’ material stocks in infrastructures and buildings—could affect energy demand and CO2 emissions. The accumulation of material stocks requires massive amounts of resources such as steel-reinforced concrete, mortar, bricks, timber, plastics, glass, gravel, or sand30,31,32,33, which are associated with high GHG emissions34. Heating, cooling, and lighting of buildings and production processes in industrial plants require much energy20,23,26,35, as does the mobility of goods and people on roads and railways33,36.

Despite these insights from urban studies, material stock patterns are seldom considered in debates on national-level analyses of factors determining levels of energy demand and emissions, as well as their possible decoupling from GDP11,15. A systematic review of the empirical literature on these questions15,37 revealed only one study38 considering material stock patterns in analyzing transport-related emissions. Hence little is known about the effects of material stock patterns on energy demand and CO2 emissions beyond the city level. This results from a scale mismatch: maps of material stock patterns provide fine-grained spatial detail30, often focused on specific regions, that cannot be included in national-level analyses of factors driving energy demand and emissions. For cross-country analyses, consistent indicators need to be developed from spatial data and then aggregated at the national level in a manner which preserves key information on patterns and supports comparative analyses across countries and world regions.

In this work, we develop national-level indicators of characteristics of built structures that, based on urban studies, can be potential determinants of resource use and emissions. We quantify them for 113 countries comprising 91.2% of the world population and 97.3% of global GDP. We analyze two independent variables: (1) yearly total per-capita final energy consumption (abbreviated as TFC) and (2) yearly per-capita CO2 emissions (abbreviated as CO2). We test material stock pattern indicators against other variables that have been widely used in national-level analyses of determinants of energy use and emissions, here denoted as conventional factors. As conventional factors, we use GDP/cap/yr (abbreviated as GDP), population density (DENS), UPOP as a development indicator, HDD as a proxy of climate dependency of energy demand, and PGAS as an energy price indicator. Extensive variables are expressed as per-capita values to facilitate country comparisons and remove countries’ population numbers from the analysis.

Results

National-level indicators of material stock patterns

We test three hypotheses based on aggregated indicators of material stock patterns (Fig. 1). Material stock patterns are represented by three types of indicators: (1) The area of built-up land is represented by two indicators, one as a fraction of a nation’s inhabited land, the other per inhabitant. Other indicators describe patterns of built-up areas, including their spatial clustering, form, and distribution, which reflect geomorphological factors as well as historical contingencies. (2) Road indicators that describe the density (length per unit area) of roads in urban and rural regions and the relations between urban and rural road lengths and densities. (3) Railway indicators are defined in the same manner as those for roads (Table 1).

Fig. 1: Workflow of this study.
figure 1

National-level indicators of the extent and spatial patterns of settlements and infrastructures (material stock patterns) were derived from global maps, here illustrated using Paraguay, the UK, Kenia, and Bangladesh as examples. Results were statistically analyzed along with the conventional factors assumed to co-determine energy use and CO2 emissions. The main aim was to test the hypotheses formulated in the lower-right box. Copyright for administrative boundaries: © Eurogeographics.

Full size image
Table 1 Indicators of the extent and patterns of built-up land and transport infrastructures
Full size table

Bivariate and semi-partial correlations

In terms of their Pearson coefficients in bivariate correlations, several indicators of material stock patterns are as strongly correlated with TFC and CO2 as the conventional factors (Fig. 2a). GDP is positively correlated with both TFC and CO2. HDD and the fraction of the urban population also show the expected pattern, while PGAS and DENS are largely uncorrelated. Almost all material stock pattern indicators are correlated with both TFC and CO2. The extent of built-up land (BLcap and BLfract) is positively correlated with both TFC and CO2, as are total and rural road density and the dispersion of built-up land and most railway-related indicators. The correlation coefficients of BLcap with TFC and CO2 are both ~0.7; BLcap is the second-best predictor of both TFC and CO2 after GDP. As expected from the urban literature, urban population density (UPdens) is inversely correlated with CO2 and energy, whereas the share of urban population (UPOP) is strongly positively correlated with CO2 and TFC. Inverse relations prevail for BLmono, BLcomp, and the urban-to-rural relations of infrastructure density.

Fig. 2: Correlation analyses of total final energy demand per capita (TFC) and per-capita CO2 emissions (CO2) with conventional factors and material stock pattern indicators.
figure 2

a Pearson’s zero correlation coefficients of correlations between TFC (left) and CO2 (right) and material stock pattern indicators as well as conventional factors. Natural logarithms of the variables were analyzed. Squaring the correlation coefficients gives the percentage of the cross-country variation of CO2 or TFC explained by the respective indicator alone. Correlations were not significant for variables marked with an asterisk (p < 0.1). b Semi-partial correlations between material stock pattern indicators and TFC (left) and CO2 (right) controlling for GDP and DENS. Distance from the vertical axis indicates the correlation coefficient of the semi-partial correlation, and distance from the horizontal axis is the correlation coefficient of the bivariate (uncontrolled) correlation. Red contours indicate insignificant results (significance level p < 0.1).

Full size image

Semi-partial correlations of the material stock pattern indicators controlled for GDP and population density (DENS) are shown in Fig. 2b. The part of each material stock pattern indicator correlated with GDP and DENS is removed, revealing the strength of the linear correlation between TFC or CO2 and the remaining part of the respective variable. The distance from the vertical axis either to the right (positive correlation) or to the left (inverse correlation) depicts the additional explanatory power of the respective indicator over a model considering only GDP and DENS. Several indicators provide additional explanations over GDP and DENS alone. The area of built-up land (both BLcap and BLfract) is positively correlated with TFC and CO2, as are the density of rail and road infrastructures, especially in rural areas. UPOP is inversely correlated with TFC and CO2, as is urban road density (not significant) and BLmono. PGAS, which had not been significant in the bivariate correlations, emerges as an important factor, which is also observed for RDurban and other indicators. UPdens lose importance, most likely due to its high correlation with GDP.

Multivariate lasso analysis

The capability of the material stock pattern indicators to add insights beyond conventional factors is further analyzed in Table 2. We use the least absolute shrinkage and selection operator (lasso) approach to select variables for multivariate statistical models capable of predicting cross-country patterns of TFC and CO2. Lasso is a widely-used procedure for automatically performing variable selection in linear regression models39,40. It overcomes the drawbacks of overfitting and multicollinearity associated with ordinary least square (OLS) methods41. The lasso method penalizes complexity to derive the best parsimonious model for any predefined value of λ (the factor determining how strongly model complexity is penalized; for detail, see Methods section), thereby allowing to identify the factors that most strongly affect TFC and CO2. The standard procedure to select λ is cross-validation, where randomly chosen samples of countries are used to develop a model used for the prediction of patterns in out-of-sample countries. The procedure gradually reduces λ, which generally results in more indicators being selected41. Alternative Lasso schemes and other variable selection techniques, such as stepwise regression, yield very similar results (SI).

Table 2 Multivariate analysis using cross-validation lasso
Full size table

The leftmost three columns of Table 2 show the knots in the lasso paths, i.e., the λ values at which indicators are added (or removed due to collinearity). Knots are arranged in decreasing order of λ, with indicators being ranked in order of selection. In the first column, the λ value marked with an asterisk (*) indicates the optimal model identified by the cross-validation. GDP is always selected as the first indicator, but various material stock pattern indicators appear very early on (i.e., at high λ values), and remain active in the optimal model. BLcap is the second-chosen indicator for both CO2 and TFC, and several other material stock pattern indicators are selected much earlier than widely-used conventional factors such as population density (DENS). Heating-degree days (HDD) and the price of gas (PGAS) are important for both CO2 and TFC, whereas UPOP is important for predicting CO2 but not selected by lasso for TFC.

The rightmost three columns of Table 2 report the estimated coefficients of models selected by cross-validation as well as measures of in-sample and out-of-sample fit (r2). Models A comprises only variables selected by lasso (path shown in the left part of the table) among all indicators. For the benchmark Models B, only conventional factors are selected. Including material stock pattern indicators in Model A yields better prediction than Models B for both CO2 and TFC in terms of in-sample and out-of-sample goodness of fit, and improves the Bayesian Information Criterion (BIC) of model selection. Other criteria to develop optimal models, and alternative variable selection procedures yielded similar results (section 5 in the SI). Even models that include only material stock pattern indicators and exclude all conventional factors (including GDP) achieve good predictions (out-of-sample r2 of 0.65 for TFC and 0.62 for CO2; see results for Model C shown in section 5.2 of the SI).

Discussion

The analysis shows that extent and spatial patterns of built structures, here denoted as material stock patterns, play an important role in co-determining and predicting the level of resource use, here TFC and CO2, in a cross-country analysis. This implies that insights from urban studies23,24,25,26,42 generally hold at the national scale. Despite the unavoidable loss of information resulting from the aggregation of maps to the national scale, the indicators in Table 1 maintain key information representing important characteristics of material stock patterns that strongly affect cross-country patterns of TFC and CO2. The indicators have substantial additional explanatory and predictive power over conventional factors. They can help develop much stronger models of national-level TFC and CO2 than usual IPAT-type approaches, and will enable researchers to broaden their analysis and scenario modeling capabilities by including material stock patterns as crucial factors for analyzing the possible decoupling of energy use and emissions from GDP.

Population density plays a smaller role than widely assumed, while many material stock patterns strongly influence the cross-country differences in energy demand and CO2 emissions. The material stock pattern indicator with the most consistent predictive power across all analyses is the area of built-up land per capita (BLcap), which emerges as the second-most important variable (after GDP) in most of our statistical analyses, even in analyses considering the GDP effect. This is plausible because infrastructures and buildings require energy for being built and used, which results in CO2 emissions in current fossil fuel-dominated energy systems34,43. Higher BLcap also means larger floor size and longer distances between destinations, which all raises energy demand in buildings and transport. These findings corroborate and expand on previous analyses (that used entirely different models, did not capture spatial patterns, and mostly referred to temporal trends), suggesting that challenges for climate-change mitigation strongly depend on the past and future accumulation of material stocks in buildings and infrastructures43,44,45,46. This is worrying because material stocks are growing globally largely in unison with GDP32,46.

Our indicators and results create options for analyzing which of the characteristics of material stock patterns are most important in determining and predicting energy demand and emissions in cross-country analyses. Analyses presented in Fig. 2, Table 2 as well as the SI clearly show that many specific aspects of material stock patterns play a role in determining cross-country differences in energy demand and emissions. The multivariate analysis also shows that these patterns interact in many ways that are difficult to disentangle due to the collinearities of the material stock pattern indicators. Future research could employ refined study designs addressing how changes over time and space affect these relationships and elucidate the different causal pathways involved.

Our results have implications for countries pursuing ambitious climate targets2,43. They demonstrate that insights from urban-level studies translate to the national scale, which could so far not be investigated at that level due to lacking data. This suggests that the indicators presented above offer opportunities to investigate the importance of built structures also for other aspects of national economies’ resource use than those analyzed here. Empirical urban studies have consistently shown that urban form and infrastructure affect travel demand and, therefore, GHG emissions47. The relationship between built structures and GHG emissions has also been predicted by theoretical analyses in urban economics21,48. This suggests that our national-level observations—e.g., that large, sprawling material stocks in built structures represent an important determinant of a nation’s per-capita level of energy demand and emissions—are very likely underpinned by similar causal relationships. Of course, more research, in particular on how these relationships change over time, would be desirable. Because high-resolution maps of decadal trends in built structures are gradually becoming available49,50, such analyses may soon be feasible. We conclude that the area and patterns of built-up land emerge as an important entry point for efforts at reducing energy demand and CO2 emissions at national levels51, suggesting that limiting built-up area per capita could be a worthwhile policy goal, not only from a land-use perspective, but also to limit future GHG emissions.

Methods

The material stock pattern indicators were derived from crowd-sourced data on infrastructures (roads and railways)52 as well as high-resolution land-cover data53. The indicators refer to the years 2015 (built-up land) and 2020 (roads and railroads). To reduce random fluctuations for the energy and emission data as well as the conventional factors, we calculated averages for as many years in the period 2015-2020 as were available in the statistical sources. Extensive variables were represented by per-capita values to facilitate comparisons between different-sized countries, following the convention to regard population as a scaling factor with an elasticity of one in STIRPAT analyses5.

Data used to derive spatial indicators

The spatial indicators presented in this study rely on three spatially explicit datasets; (1) built-up features and urban agglomerations, (2) main infrastructure features (road and railway), and (3) an inhabited land area layer used for reference. All variables that express quantities as fractions of a country’s area, relate to inhabited land. We used the LC100 grid of the Copernicus Global Land Cover Service53 to derive relevant land cover information. The choice of input data and data quality checks are discussed in the SI; see also Supplementary Fig. 1 (SI). In contrast to other built-up land datasets54,55,56, the LC100 not only provides globally consistent information on built-up areas, it also includes all other complementary land cover types at a global scale. For data preparation, we vectorized the LC100 grid to allow further spatial intersection procedures (e.g., to clip to national borders). The NOAA-DEM grid was used as a global digital elevation model. We utilized the Geofabrik-download hub57 to obtain the entire global OSM data. The GISCO archive from EUROSTAT58 provides the applied country borders. Details on the data sources are given in Supplementary Table 1 (SI); datasets59 and software code60 are available online.

Sources of energy and emission data as well as conventional factors

Gross domestic product (GDP) in constant 2015 US$ was sourced from UN Statistics Division National Accounts61. Population data used to calculate population density (DENS) and urban population rates (UPOP) were taken from official census data 2017 of the World Bank62,63. The World Bank database was also used to source data on pump prices for gasoline (PGAS)64. Heating-degree days (HDD) were calculated as a population-weighted average of °C days above 18 °C and sourced from the International Energy Agency (IEA) Weather for Energy Tracker65. Territorial CO2 emissions from fossil fuel combustion and cement production (CO2) were sourced from the Global Carbon Project database66,67 and total final energy consumption (TFC) from the IEA energy balances, which could only be accessed for the latest year 201768. The record date for the download of these indicators was 25 March 2021. We always used the latest available year for our cross-sectional analysis, but also conducted robustness checks using arithmetic averages over the latest five years for all indicators to exclude potential bias from outstanding annual values. Details on the data are available in Supplementary Table 2 in the SI.

Deriving the built-up land layer

The built-up land vector (BL) for every country is one primary data product derived by the vectorized LC100 grid. This standard BL layer comprised all national built-up features and was used to derive the BL-related indicators shown in part (1) of Table 1. To map each country’s urban agglomerations, and thereby distinguish urban from rural land respectively infrastructures (see Supplementary Figs. 2–4, SI), we clustered features of the BL vector layer using an empirical growing-neighborhood approach: We started with the country’s largest BL feature and created convex hulls, which were buffered with the fifth of the area-equal radius. We then identified intersecting BL features within this buffered hull and created a new hull and buffer. These two steps were repeated as long as the BL area of all intersecting features reached 33.3% of the area of the buffered hull (used to check the intersections), and the BL area increased at least 0.5%, compared to the BL area of the previous iteration. If these criteria were not given, the growing procedure terminated and the collected BL features dissolved to one BL agglomeration. The algorithm subsequently went on with the next-biggest BL feature, which had not been assigned to an already-created agglomeration, again starting the growing-neighborhood procedure. The whole process terminated when the next BL feature (to restart the growing-neighborhood procedure again) represented less than 0.1% of the total BL area of the respective country. The maps in Supplementary Fig. 5 (SI) show the built-up (BL) land indicators. Fractions of land areas refer to inhabited land.

The infrastructure layers

The preprocessed OSM database comprises globally consistent road (R) and railway vector data (RW), but the availability of OSM data varies strongly between countries. While the OSM data in countries of the global North (industrial or even postindustrial countries) also include minor road and train track categories (e.g., cycleways, steps, or private gauges), OSM data in countries of the global South only comprise the main road and railway network. To reduce this inconsistency, we excluded minor OSM classes in order to derive a more comparable global database (Supplementary Table 3). The maps showing the road network indicators are in Supplementary Fig. 6 (SI). Fractions of the area refer to an inhabited land. The railway indicators were derived from OSM in the same manner as those for roads and are shown in Supplementary Fig. 7 (SI). The railway types considered are defined in Supplementary Table 4. Fractions refer to an inhabited land. The planar extent of road and railway networks was calculated using width data reported in Supplementary Table 5. The distinction between urban and rural infrastructures was based on a spatial intersection of OSM road and railway data with the BL features, which resulted in layers of urban and rural networks (example shown in Supplementary Fig. 4, SI).

Reference layer for inhabited land

The definition of some material stock pattern indicators requires a reference area (AREF). The area of the total national territory (ANT) may not be suitable, given that in some countries (almost), the entire area is inhabitable, whereas other countries contain large tracts of land unsuitable for human habitation and hence largely uninhabited. We, therefore, developed the proxy layer inhabited land (IH) as a more suitable area reference (AIH). In contrast to existing similar datasets69, this IH mainly uses land cover information of the LC100 grid. This guarantees thematic consistency in spatial intersections with BL data that were derived using the same dataset. The IH includes not only area that is covered with settlements or infrastructures, but also cropland areas and areas with ambiguous land cover that fall within a zone of influence around existing built-up areas, which is approximated by a buffer that depends on the area of a built-up land feature (Eq. 1). The IH is based on the current settlement and cropland extent, as well as the spatial distribution, density, and elevation occurrence of built-up areas according to the LC100 grid.

To calculate IH, we first cut out high-altitude regions from the country’s total territory. We calculated the elevational distribution of BL features and excluded all areas above the area-weighted 99th percentile of BL-elevations. In a second step, we excluded the LC100 land cover types bare/sparse vegetation (deserts and rocks), moss & lichen, snow & ice, and permanent water bodies. Thirdly, we spatially intersected the map resulting from the previous two steps with a synthetic layer that represents the gapless inhabited land area. To derive this layer, we applied an area-dependent buffer for all BL features. See Eq. (1) for the dynamic BL buffer width (wBL); ABL feature denotes the area of an exemplary built-up land feature.

$${{w}_{{BL}}={A}_{{BL}\;{feature}}}^{\frac{2}{3}}$$
(1)
$$\max \{{w}_{{BL}}:100{km}\}$$

Finally, we added cropland areas (from LC100) and the original BL areas to re-include those BL areas excluded by the elevation threshold. Please note that for spatial indicators that depend on \({A}_{{REF}}\), the specific spatial pattern (shape) of IH is not relevant: we just use the national total area of IH instead of the area of the national territory as reference value. The potential usefulness of this IH layer for other research questions, in particular where its spatial accuracy is of high importance, needs to be tested and is not in focus in this study.

Definitions of spatial pattern indicators

The spatial pattern indicators were derived from the built-up, road, and railway layers. Details on the definitions of the indicators (Table 1) are given in the SI in section 3, Supplementary Tables 6–8.

Pearson correlations

The Pearson correlation is a measure of linear association between two variables. The coefficient can be obtained from bivariate data \(\left\{\left({X}_{1},{Y}_{1}\right), . . . ,\left({X}_{n},{Y}_{n}\right)\right\}\) as \({r}_{{XY}}={S}_{{XY}}/{S}_{X}{S}_{Y}\), where \({S}_{{XY}}\) and \({S}_{i}\) denote the sample covariance and standard deviation. The correlation coefficient is between −1 and 1. Correlations equal to 1 (or −1) indicate a perfect linear association, with data points lying exactly on a positive (negative) line. A value equal to zero indicates the absence of any linear association. The squared correlation coefficient \({r}_{{xy}}^{2}\) is the coefficient of determination (R-squared) of the linear regression of variable x on variable y; it measures the fraction of the variance explained by the regression line.

Semi-partial correlations

Suppose that \(Y\) is determined by \({{{\bf{X}}}}=\left\{{X}_{1},....,{X}_{k}\right\}\). Then, the semi-partial correlation between \(Y\) and \({X}_{i}\), controlled for the other predictors \({{{{\bf{X}}}}}_{-i}\), attempts to measure the correlation between \(Y\) and \({X}_{i}\) that would be observed if the effect of \({{{{\bf{X}}}}}_{-i}\) would be removed from \({X}_{i}\) but not from\(Y\). This means that the semi-partial regression measures the correlation between \(Y\) and the part of \({X}_{i}\) that is orthogonal to the other variables \({{{{\bf{X}}}}}_{-i}\). It is calculated by constructing a new variable \({X}_{i}^{{\prime} }\) that is orthogonal (i.e., entirely uncorrelated) to all previously considered variables (i.e., those controlled for).

The squared semi-partial correlation coefficient measures the fraction of the variance of the dependent variable \(Y\) that is uniquely explained by \({X}_{i}\). Thus, it can also be interpreted as the increase (decrease) in the model R-squared value that results from including (removing) \({X}_{i}\) from the set of predictors \({{{\bf{X}}}}\).

The semi-partial correlation is calculated by first fitting a linear regression of \(Y\) on \({{{\bf{X}}}}\) and computing the coefficient as:

$${r}_{Y\left({X}_{i} | {{{{\bf{X}}}}}_{-i}\right)}={{{\rm{sign}}}}\left(t\right)\sqrt{\frac{{t}^{2}\left(1-{R}^{2}\right)}{n-k}}$$
(2)

In (2) \(t\) is the t-statistic of variable \({X}_{i}\) in the previous regression, \({R}^{2}\) is the R-squared, \(k\) is the number of independent variables plus the constant, and \(n\) is the sample size. Finally, the significance level is given by \(2/{{{\rm{Pr }}}}\left({t}_{n-k} \, > \, \left|t\right|\right)\), where \({{{\rm{Pr }}}}\) is the probability, \(t\) is as described above and \({t}_{n-k}\) follows a Student’s \(t\) distribution with \(n-k\) degrees of freedom. Further details on the correlation techniques used are available e.g. here70.

Lasso analysis

The least absolute shrinkage and selection operator (lasso) approach is used to select variables for multivariate statistical models for predicting cross-country patterns of TFC and CO241,71. Lasso is a standard regularization technique for model selection and prediction that overcomes the disadvantages of other regularization techniques such as Ridge regression72. It can select a parsimonious set of variables from many potential covariates, even if covariates are collinear. Lasso minimizes the sum of squared residuals, but unlike standard least square fit approaches, it penalizes complexity in the objective function to derive the best parsimonious model for any predefined value of \(\lambda \), i.e., the factor penalizing model complexity. If \(\lambda \) is set to zero, lasso delivers standard least squares estimates, which corresponds to a model with the maximum complexity. In general, the larger the \(\lambda \), the smaller the number of non-zero coefficients.

Consider a linear specification \(Y={B}_{0}+{B}_{1}{X}_{1}+\ldots+{B}_{P}{X}_{P}+\epsilon \), where variables have been previously standardized to account for differences in scales. Lasso finds estimates for model coefficient B keeping the model sparse by minimizing the following term:

$$\frac{1}{2N}{\left(Y-{{{\bf{XB}}}}\right)}^{{\prime} }\left(Y-{{{\bf{XB}}}}\right)+\lambda {\sum }_{j=1}^{P}\left|{B}_{j}\right|$$
(3)

The first part of the term (3) is the in-sample squared error minimized by a classical least-squares approach. Lasso also includes the absolute sum of coefficients in the objective function, which penalizes complexity driving some of the estimated coefficients to zero.

The value of λ is typically chosen so that the estimated model satisfies a predetermined condition. Several criteria can be employed, the most common of which is cross-validation. Cross-validation selects the \(\lambda \) to minimize the out-of-sample prediction error. First, sample observations are split into K random folds (validation sets). For each validation set, the model is fitted using data from the other folds, and the out-of-sample deviance for the observations in the validation set is computed (i.e., using data not employed for estimation). Finally, the overall out-of-sample performance of the model in all the validation sets is assessed by the mean-square prediction error (MSPE), a statistical parameter in squared (log) units required for model selection. Cross-validation selects the \(\lambda \) over a grid of possible values such that the corresponding model has the minimum MSPE71. In Table 2, MSPE is transformed into r2, i.e., the goodness of fit within the sample of countries, and oSr2, i.e., the cross-validated out-of-sample goodness of fit for the optimal models. If no variable is added or removed at λ*, this is reported in the left columns of Table 2 as Unchanged. Note that beyond \({\lambda }^{*}\), more variables could be added but would not further improve the out-of-sample prediction.