Main

For several decades, US housing investment policies, such as community development block grants, have distributed more than US$5.8B per year (all $ in US$ henceforth) in public assistance to distressed communities1. Block grants offer flexible mechanisms that preserve local control over and prioritization of administered public funds. However, evidence that these policies effectively serve low- and moderate-income (LMI) households has been unclear. A fundamental challenge in quantifying these programme’s benefits is that the civic data needed for impact evaluation are often siloed across city information systems. Block grants administered by the US Department of Housing and Urban Development (HUD) address the human dimensions of increasing urban density and land use. More generally, these housing policies can be important mediating strategies in scenarios of human affluence and environmental impact2, including estimates of building energy use or resource consumption at various geographic scales3,4. This notion of housing as a driver of resource consumption is important as renewed federalism debates over public funds for housing assistance also affect modelling assumptions about sustainable urban growth, social equity, and climate resilience. Yet, despite over three decades of programmatic evidence and evaluation, the energy and sustainability outcomes of HUD-funded programmes have been largely missing from public decision-making (Supplementary Note 1). Consequently, the analysis of sustainability trade-offs or sustainability co-benefits from housing investment has been invisible to the policy process.

A hurdle for policymakers is that the community benefits of these programmes are often hard to measure. Scholars have suggested dedicated funds be set aside to develop more sophisticated, holistic approaches for impact evaluation5. A principal limitation for evaluators is usually structural. For example, the disaggregated data required to rigorously evaluate the benefits of these programmes, such as energy, water or resource use, are often inaccessible across information systems or city bureaucracies. Other performance data reported to HUD are commonly in the form of community surveys or self-reported information from programme participants. However, due to the limited availability of contemporaneous data from non-participants, which are necessary to construct credible reference groups for evaluation, obtaining consistent, reliable estimates of programme impacts for block grants is rare.

In this Article, we describe a multiyear effort on the use of civic data analytics in the public sector. We linked siloed data on energy consumption and housing programme participation. Using an open data hub, we created an automated housing registry to process large datasets from over a dozen independently administered databases and multiple city departments (Supplementary Note 2). This process of combining and standardizing records from relational databases is referred to in the data science community as data fusion6. The housing registry’s capabilities include access to open data with geographic information systems (GIS) mapping and a community engagement analytics platform. Importantly, the registry links housing and utility consumption records at the property-address level. These records are more granular than common evaluation studies at the parcel or county scale, which typically do not permit analyses of individual household behaviour.

We investigated long-term sustainability outcomes for two of the largest HUD-administered block grants, the Community Development Block Grant (CDBG) entitlement programme and the HOME Investment Partnerships (HOME) programme (Supplementary Note 3). We analysed 16 years of evidence (2004–2019) from the City of Albany, Georgia—a typical, small-to-mid-sized entitlement community in the US Southeast. The data include 5.9 million monthly observations of participating and non-participating households. We asked whether HUD-administered block grants, which fund housing capital improvements, could generate hidden spillover benefits to private citizens through energy savings. In quantifying possible spillover benefits of housing assistance, we investigated potential policy innovation to use housing programme targeting as an entry strategy to include LMI communities often left out of energy conservation upgrades. Although the connection between block grant programmes and energy efficiency might not be immediately obvious, we found that home upgrades and rehabilitation greatly affect household resource consumption. We document that housing programmes can accelerate energy efficiency in LMI communities, including households with a lower awareness of or interest in energy efficiency.

Block grants for LMI households

Programme ‘targeting’ is a central tenet of US federal housing policies but creates fundamental challenges for research and evaluation. HUD-administrated block grants are mean-tested policies that distribute targeted federal resources to state and local officials to build more resilient communities1. A key feature of the block grant funding mechanism is that cities have decentralized authority over these funds.

Advocates for block grants praise the programme’s flexibility. They suggest that local public administrators will seek the most efficient and cost-effective means to deliver programme services as those officials have better information about community needs. Local administrators are also presumed to be more ‘visible’ and thus can be held more accountable by citizens versus federal administrators1,7. However, critics argue that block grants have too much flexibility. For example, grantees can redirect programme targeting away from individuals with the greatest need or shift programme services with long-term pay-offs in favour of short-term initiatives with less impact.

When local governments receive federal grants, it often stimulates higher levels of spending than theory would predict from local revenues, a phenomenon known as the ‘flypaper effect’8,9. Nevertheless, debates persist about whether increased expenditures actually lead to higher levels of public service provision, especially within LMI communities10,11. In the context of block grants and housing, scholars have argued that the actual value of block grant funding tends to diminish over time12. As a result, block grant programmes have been criticized for gradually decreasing services to the neediest or most vulnerable populations5,13,14. CDBG and HOME programmes reduce capital and information barriers for entitlement communities to be able to access and receive governmental assistance. However, it remains an open question whether income-qualified households meaningfully participate in and benefit from these federal funds. We therefore investigated whether a broader range of co-benefits from housing assistance, such as energy savings and other unmeasured sustainability benefits, might be generated by housing block grants.

The energy-relevant programme activities under CDBG include energy efficiency projects, rental rehabilitation and emergency repairs, that is, roof replacements, heating, ventilation and air conditioning (HVAC), electrical, plumbing and other repairs that bring structures up to current building codes. Under HOME grants, the relevant programme activities include the rehabilitation of owner-occupied housing units, acquisition, rehabilitation or construction of rental units as affordable housing for low-income individuals, and tenant-based rental assistance. For a more detailed list of CDBG and HOME activities and programme rules, see refs. 1,15.

Our field site, Albany, Georgia, is 74% Black or African American, and its poverty rate is 30.8%—nearly triple the US national average (Supplementary Discussion). Albany is ideally suited for formula targeting under block grant eligibility rules due to its population size, ageing housing stock and high poverty rate. On the basis of the city’s data, we calculated the average household energy consumption in Albany as 13,255 kilowatt-hours (kWh) per year, making it an important population of interest. This is nearly 25% higher than the US national average (10,791 kWh)16. For a review of causes and correlates of household energy burden in low-income communities, see ref. 17. Given the high energy and cost burdens, we could expect to observe substantial improvements from policy intervention.

Results and Discussion

Programme targeting

We evaluated the household characteristics of programme participants in Albany to assess evidence of programme targeting. The most common housing problems include cost burdening, crowding and substandard housing (for example, lacking complete plumbing, kitchen facilities or weatherization)18. The city has 7,985 LMI renter households and 2,232 LMI owner households that spend more than 30% of their monthly income on housing costs, including utilities (Supplementary Note 4). Given this high share of cost-burdened households, it is understandable that the demand for housing assistance far exceeds the available funds, with long waiting lists of income-qualified applicants. In Fig. 1, we show, by census tract, the spatial distribution of 549 HUD-funded housing projects participating in either CDBG or HOME among 16,680 properties in the city. While there are funded activities across all six city wards, a high proportion of participating properties are also located within Federal Opportunity Zones (Fig. 1).

Fig. 1: Housing policies target households with higher energy burdens.
figure 1

The locations of 549 CDBG and HOME participating households in US Census tracts within the City of Albany, Georgia. The households receiving federal assistance are generally concentrated in areas with relatively higher electricity consumption per square foot and/or higher poverty rate, including many in Albany’s federally designated Opportunity Zones. Over 90% of participating households are at or below 80% of the area median income, which provides evidence of the effective programme targeting for energy efficiency.

Source data

Full size image

Although energy intensity is not a consideration in targeting or eligibility criteria, we found that participating households that received HOME or CDBG funds are heavily concentrated in areas with high poverty rates and where energy burdens are prevalent (Fig. 1). Out of 10,127 households at 80% of the area median income or below, 5,714 have a severe cost burden (4,440 renters and 1,274 owners); these households comprise a substantial share, 38% of LMI households18. In these tracts, the ratio of median electricity bill charges to median income is 8–12%—higher than the national threshold (Fig. 1). This type of evidence has previously been hard to discover given the persistent data silos and lack of researcher access to integrated utility data for evaluation. Next, we compared programme targeting under federal housing rules to more conventional policies targeting energy efficiency.

Previous work has shown that self-selection into energy efficiency programmes generally has low take-up rates among LMI households, even when energy efficiency services are subsidized or free19,20. Given the upfront investment, administrative or coordination costs necessary to achieve large-scale savings, dedicated energy efficiency programmes can actually have negative rates of return20. Further, when standalone energy efficiency policies are not means tested, they also tend to attract participation in higher-income areas21. This situation raises questions about inframarginal participation—whether participants would have invested in energy efficiency without public subsidies or benefits22. Yet, although not all activities covered under HOME and CDBG may be relevant for energy conservation, we argue that programme targeting under housing block grant rules could be a favourable alternative to standalone energy efficiency policies that are not necessarily means tested or have a low service take-up. This is because the housing programme selection process simultaneously attracts the most energy-intensive and energy-burdened households in situations where the demand for services is also strong.

Surprisingly, we found that housing policies can accelerate participation in energy efficiency among capital-constrained homeowners or renters, even in cases where participants were not initially motivated by energy conservation measures. For example, one resident said, ‘When they put the roof on it was like night and day. I could feel the warmth of the house’. In the next section, we quantify the realized energy savings within targeted LMI communities.

Energy savings from housing programmes

We estimated the long-run energy savings in kWh per square foot for participating properties in CDBG and HOME programmes. These spillover energy savings can be conceptualized as a bonus in programme performance beyond core housing programme objectives. To calculate energy savings, we implemented several matching models with regression adjustment to construct suitable statistical reference groups pre- and post-programme participation. To mitigate observational bias, we used algorithmic matching procedures with a genetic search algorithm23 to achieve covariate balance between treated and counterfactual observations. We also implemented staggered difference-in-differences (DiD) estimators that mitigate potential biases of two-way fixed effects with heterogeneous effects (Methods). For transparency in protocols, we report the bias reduction in Fig. 2 and note that in staggered DiD models without matching, the energy savings can be understated (Supplementary Table 1). We report the most conservative estimates, robust to various matching procedures and estimators (for more details, see Methods).

Fig. 2: Matching algorithms reduce observational bias.
figure 2

The relative performance of genetic matching and PSM in standardized percent bias. The key conditioning and testing variables shown include property, demographic and neighbourhood characteristics. The conditioning variables are identified with asterisks and include observable property characteristics (average baseline consumption, property size and age, number of beds and baths). To mitigate the effect of possible unobservables on energy use, the market value of the property was added to the set of matching variables as a proxy for unobserved quality attributes. Genetic matching achieved 90.6% bias reduction, while PSM achieved 77.8% bias reduction; therefore, the remaining bias in standardized percent bias is −9.6% and −22.2%, respectively. Although both methods substantially reduce median bias and offer a high degree of covariate balance, the genetic matching algorithm is preferred over PSM.

Source data

Full size image

HUD-funded housing projects in Albany generated statistically significant monthly average energy savings of 5–11% for participating households as compared with multivariate matched properties with similar characteristics (Table 1). For the subset of energy-relevant projects estimated by staggered DiD estimators (that is, Energy Efficiency, Emergency Repairs and Homeowner Rehabilitation), we report energy savings of 11–14% after correcting for potential estimation biases due to treatment effect heterogeneity under staggered participation (Table 1 and Supplementary Table 2). We note that point estimates can be higher when considering staggered designs. While there is year-to-year variability in performance depending on the mix of implemented projects, the energy savings for housing participants are relatively stable across years, with increasing performance in the last 2 years of the study period (Supplementary Fig. 1).

Table 1 Long-run energy savings from housing programmes 2004–2019
Full size table

Overall, HUD-funded block grants in Albany reduced electricity use by 4.72 million kWh over the study period. The reduction in non-baseload emissions is equivalent to 3.70 million pounds of coal not being burned or the carbon sequestered by 3,695 acres of forest (Supplementary Note 5). These long-term savings are remarkable, given that energy efficiency is not an explicit criterion for these policies.

Participating properties in the CDBG programme achieved monthly savings of 6–14% (Supplementary Table 1, 95% confidence interval (CI)). Emergency repairs, where households could elect for one critical repair (for example, HVAC), comprising 248 projects, generated 6% energy savings. Albany’s CDBG-funded Energy Efficiency programme, offering new insulation and windows, comprising 62 projects, generated ~13% energy savings. The largest savings came from the CDBG-funded Rental Rehabilitation programme, which focuses on structural upgrades (for example, roof) to city-owned rental properties, comprising 22 projects, generating 32% energy savings. This performance is consistent with the high savings associated with major building upgrades reported in voluntary and information-based programmes21,24,25.

The HOME portfolio had more mixed results. On the one hand, Homeowner Rehabilitation, which provided households with a full range of repairs, comprising 29 projects, generated ~11% energy savings. HOME had a larger share of projects not relevant to energy savings (for example, Tenant Based Rental Assistance). Unsurprisingly, these 160 unrelated HOME projects were associated with a 15% increase in energy consumption. Therefore, we found evidence of energy savings across a broad portfolio of CDBG projects and, to a more limited extent, HOME projects.

To further contextualize savings from the HUD-funded CDBG programmes, we translated the lower and upper range of estimated energy savings (for example, 6% for Emergency Repairs and 32% for Rental Rehabilitation) to dollar amounts using an average monthly electricity bill in Albany ($125). When annualized, housing participants saved anywhere from $75 to $482 in direct kWh charges. According to the Bureau of Labor Statistics consumer price index26, these savings are equivalent to nearly two months of groceries for households in the region (Supplementary Table 3).

Housing spillovers versus energy conservation programmes

We evaluated how meaningful these savings are in comparison with dedicated energy efficiency programmes reported in the literature. First, we compared the magnitude of energy savings for both non-LMI- and LMI-targeted programmes and found that housing spillovers meet or exceed the reported energy savings from standalone programmes. For example, savings ranging from 0% to 25% were reported for a broad range of interventions involving capital upgrades24. Savings from behavioural and information-based interventions also ranged from 0% to 20%, depending on the intervention type and methodology27,28,29,30.

We found that energy savings from housing programme spillovers (which range from 6% to 32%; Supplementary Table 1) are generally consistent with and sometimes exceed previous reports for non-LMI-targeted interventions. In another review, energy savings of 0.9% to 8.2% were reported for non-LMI-targeted informational nudges for energy conservation31. Although energy savings from capital improvements often generate substantially larger savings, we acknowledge that information and behavioural nudges can also offer other benefits. For example, treatment effects from information-based interventions can persist for years after the treatments are discontinued32,33; or they can generate conservation spillovers from one form of resource consumption to another. Reported cases include water to energy savings34; waste sorting to waste reduction35; or hot water savings to space heating conservation36. We acknowledge that energy savings may not be the only important outcome measure for programme evaluation.

In addition, we benchmarked the energy savings from housing spillovers to standalone energy efficiency programmes where LMI households were the principal recipients of the energy savings. Studies of standalone energy efficiency programmes geared toward LMI households, such as the Weatherization Assistance Program (WAP), Low-Energy Efficiency Plus (LEEP-Plus) and Energy Savings Assistance Program (ESAP), have reported energy savings in the range of 2% to 7%, albeit with challenges in programme uptake20,37,38. Therefore, given the range of treatment effects in this study, we found that housing spillovers are competitive with and occasionally exceed the energy savings from standalone energy efficiency programmes targeting LMI communities.

Comparatively, housing spillovers are also meaningful in effectively reaching a broader range of LMI households versus standalone programmes. This is because LMI households in need of home repairs are generally a larger subset of the population than those actively seeking specialized energy efficiency support. Notably, the majority of grantees are simultaneously concentrated in areas with high poverty rates and, surprisingly, high energy consumption, which has been previously unknown (Fig. 1). We believe this profile is notable as it differs from descriptions of low LMI participation in dedicated energy efficiency programmes20,37.

Cost-effective comparisons

Although energy savings is not the intended aim of CDBG and HOME block grants, we calculated cost-effectiveness ratios in kWh saved per dollar spent for four energy-relevant housing programmes: Emergency Repairs, Energy Efficiency, and Rental Rehabilitation (under CDBG) and Homeowner Rehabilitation (under HOME). Because of our unique partnership with City of Albany public administrators, we were able to access programme and administrative costs at the project level. The fiscal period for which we had access to the costs is October 2007 to May 2018, spanning 11 years. We noted that such long-term evaluations of block grant outcomes have been uncommon5. For details on cost-effectiveness calculations, see Methods. Within CDBG, we report cost-effectiveness ratios of 83.5 kWh $−1 for Rental Rehabilitation, 10.8 kWh $−1 for Energy Efficiency and 3.7 kWh $−1 for Emergency Repairs. Within the HOME programme, we report the cost-effectiveness ratio of 0.8 kWh $−1 for Homeowner Rehabilitation.

Further, we benchmarked the cost-effectiveness ratios (in 2021 $) of housing spillovers against reported estimates from dedicated energy efficiency programmes. We considered recent meta-reviews24,31 and other highly cited studies published in the past 20 years. In Fig. 3, we provide a comparison, beginning with standalone Capital Upgrades programmes, which include both LMI- and non-LMI-targeted programmes. We also compared housing spillovers to non-LMI-targeted programmes including Information & Behavioural Programmes and Rebates & Financial Incentives. We found that housing spillovers from Rental Rehabilitation in the CDBG programme are nearly 2.9 times more cost effective than common Capital Upgrades programmes, such as utility-based retrofitting (that is, 29.0 kWh $−1)17. As the Rental Rehabilitation funds upgrades in city-owned properties, we learned that Rental Rehabilitation is revenue generating (unlike non-city-owned properties in homeowner rehabilitation). Therefore, the cost-effectiveness ratio is substantially higher because administrators can also leverage programme income to re-invest in additional upgrades. We consider split incentives issues within rental rehabilitation in Supplementary Discussion. Spillovers from Emergency Repairs and other block grant programmes are also within the reported cost-effectiveness ratios from dedicated programmes that target LMI communities, including WAP, LEEP-Plus and ESAP17,20,38. Similarly, we find that cost-effectiveness ratios from housing spillovers are also competitive with non-LMI-targeted Capital Upgrades programmes, such as building labels and building codes (that is, ranging from 21.3 to 4.7 kWh $−1)21,39,40 (Fig. 3).

Fig. 3: Cost effectiveness of housing spillovers versus standalone energy efficiency programmes.
figure 3

A comparison of cost-effectiveness ratios (in kWh saved per dollar) for housing spillovers in this study with other dedicated energy efficiency programmes. This includes peer-reviewed point estimates for the most common interventions including Capital Upgrades, Information & Behavioural Nudges, and Rebates & Financial Incentives. Values for refs. 28,32,44,47 were derived from ref. 31. Values for refs. 39,41,42,45,46 were derived from ref. 24. Values for refs. 17,20,27,37,38,48 were derived from information reported in those studies. Values are exact and have been scaled to 2021 $.

Source data

Full size image

As expected, the cost-effectiveness ratios of housing spillovers are less favourable than those estimated for Information & Behavioural programmes27,28,32,33,41,42 (that is, ranging from 64.3 to 0.1 kWh $−1) (Fig. 3), which do not typically involve capital upgrades. We also compared cost-effectiveness ratios in this study to Rebates & Financial Incentives, such as appliance replacement (refrigerator, heat pump), electricity bill credits and other rebates (that is, ranging from 29.3 to 0.4 kWh $−1)43,44,45,46,47,48. In contrast to nudge interventions, we find that the cost-effectiveness ratios in this study are generally competitive with Rebates & Financial Incentives (Fig. 3). This is intriguing since direct monetary incentives for energy efficiency, unless restricted by programme rules, do not generally target LMI communities. Although outside the scope of this paper, we did back-of-the-envelope calculations of the implied internal rates of return for housing programme spillovers for interested readers (Supplementary Note 8)49. Over the study period, the implied internal rates of return are ~40% and higher. Many dedicated energy efficiency programmes, such as weatherization, have reported variable rates of return as low as 3% to over 100% (refs. 20,30). For further discussion of rates of return in energy efficiency programme evaluation, see refs. 50,51,52,53,54,55,56,57,58.

In summary, whether a comparable programme is LMI-targeted or not, we found that the cost-effectiveness ratios from housing spillovers are generally competitive with dedicated energy efficiency programmes across a broad range of intervention types.

Evidence of programme uptake

To further understand the drivers of performance in CDBG and HOME programme administration, we conducted semi-structured interviews with public administrators and residents (see Supplementary Discussion). Engaging with public administrators and residents allowed us to compare programme uptake for dedicated energy efficiency programmes with the uptake for housing programmes. This is important because programme uptake has been a critical barrier to accelerating energy efficiency participation in LMI communities. Through our interviews, we found evidence of persistent barriers contributing to low programme uptake in dedicated energy efficiency programmes and strong drivers of programme uptake within housing.

We know from the literature that barriers to uptake of residential energy efficiency programmes can typically include: (1) capital, resource and liquidity constraints; (2) information barriers and behavioural or cognitive biases; and (3) transaction and process costs24. We found evidence for many of these same barriers in Albany, including a less-documented barrier: (4) local mistrust of government. First, evidence of low uptake of energy efficiency programmes is commonly due to a lack of capital and other resources. According to the housing programme director, most applicants in Albany are ‘elderly and on fixed incomes’. A resident shared, “The [financial] barrier is having those resources to conserve”. Another resident stated, “… a lot of my fellow homeowners cannot afford homeowners insurance, without which you cannot get weatherization and stuff”. Second, when asked why more residents were not participating in the programmes, a resident proffered that they “don’t understand and don’t get the information right”. Another said, “I don’t know what type of appliance would be available to say, this will help you decrease your electricity”. Third, evidence of process and transaction costs came up in several interviews. For programme participation to occur, public administrators for the City of Albany must ‘see a lot of customers’; work ‘24/7’; always be ‘on call’; and put in ‘110 or more percent’. One resident shared, “[The administrators] have funds available for Energy Assistance, but they take you through so much to get whatever they’re going to give you. If they’re going to help you, you’ll be so burned out because it takes so much”. While we confirmed that high-involvement processes might be necessary at the local level, additional transaction costs could limit the scalability and so decrease the uptake of dedicated energy efficiency programmes.

A fourth barrier, local mistrust of government, has been discussed in the public management literature for a broad range of services, but less so for energy efficiency59,60,61. Public administrators in Albany are aware of this issue. For example, one official shared, “It’s hard to convince people to do energy efficiency and let folks into their homes”. According to some public administrators, certain residents have “… perceptions that [the city government is] going to put a lien on [their property]”. They say, “The mistrust is enormous” and that residents “don’t believe [city administrators are] doing what it is they say they’re doing”. Evidence from our interviews demonstrates that mistrust of local government service delivery, in addition to capital constraints, cognitive biases and transaction costs (among others), may also limit energy efficiency programme uptake in LMI communities.

In contrast, housing programmes have high demand and participation. These programmes attract a broad range of eligible participants from LMI households. According to the City of Albany’s 5-Year Consolidated Plan, “Over 2,000 families are on waiting lists for a total of just 1,117 public housing units, and the occupancy rate for existing units is virtually 100%”18. This evidence of high take-up of public housing assistance—nearly twice the availability—reveals the broad reach of the city’s housing programmes’ HUD block grants in our study. Stakeholder meetings conducted by the city revealed that ‘high utility costs may be a common issue for low-income, disabled, senior and minority households living in older and less energy efficient homes’. These households comprise the vast majority of entitlement grantees in Albany. Other stakeholders testified that ‘while households may be able to afford their homes, units may lack appliances or are in need of major repairs’18. Reports of high utility costs and the need for housing repairs confirm the high complementarity between energy efficiency and housing programme uptake. Residents’ interviews further illustrate the potential impacts. “You’re talking about [sic] putting… money toward buying food and groceries versus paying utility bills; so [the housing policies] can have a big impact”, said one resident. Another stated, “I only get $1,200 a month, and my utilities is $4 almost $5 [hundred], and my mortgage is $765”. Such resident feedback confirms that the policies can have an impact in financially struggling households regardless of awareness of or interest in energy efficiency measures. Considering that housing policies have strong demand, we conclude that expansions in housing programme participation can lead to strong energy and sustainability co-benefits for a broader range of LMI households.

Dedicated energy efficiency policies tend to attract affluent and informed households but suffer from low participation rates among LMI households37. We found substantial energy savings from housing programme spillovers in situations where demand for services is also strong. These sustainability co-benefits have remained largely hidden from programme evaluation and policy decision-making due to widespread data silos at the city scale. Through data innovation in record-linkage procedures, we have been able to uncover previously unmeasured energy savings impacting low- and moderate-income communities. For a family facing trade-offs between essential household needs, the quantified energy savings can make a dramatic difference: nearly two months of groceries. For the community writ large, the energy co-benefits accelerate long-term participation from households facing structural and persistent barriers to energy efficiency. We argue that energy and sustainability-oriented outcomes should be further integrated into federal housing programme evaluation criteria, and we expect that doing so will uncover a multitude of other hard-to-measure social benefits.

Methods

Ethics statement

Human participant protocols were conducted under Georgia Tech Institutional Review Board (IRB) protocol number H20089.

Data and programme details

Administrators in the City of Albany have used open data tools to respond to local demands for greater transparency and accountability in the delivery of public services. These open data initiatives are becoming increasingly common among similar-sized cities across the United States. For the current study, the city provided data access to 5.9 M housing-related open data records from more than 12 city departments. The dataset included monthly electricity consumption for all residential properties in Albany from 2004 to 2019. After we linked housing and energy consumption data by property identifiers, we obtained a proper subset of 2,931,406 panel observations covering 16,680 residential properties.

Out of nearly 20 programmes funded under HOME and CDBG, we focused the analysis on programmes directly related to household energy use. These are Energy Efficiency, Emergency Repairs, Homeowner Rehabilitation and Rental Rehabilitation. These energy-relevant projects comprise 65% of the whole project portfolio during the analysis period from 2004 to 2019. Emergency Repairs, for example, constituted an important share of the total housing portfolio, and it represented more than 30% of all treated properties in our analysis. Programmes unrelated to energy use, such as Tenant Based Rental Assistance or New Construction, in which rental support can travel with the individual and not necessarily the housing unit, were used for falsification (placebo) testing.

The unit of analysis is the property address (we used property address and household interchangeably). The dependent variable used for analysis is the monthly electricity consumption in kWh per square foot. We log transformed the dependent variable and multiplied this by 100 for ease of interpreting the estimated coefficients directly as a percentage change. The policy indicator variable was coded as 1 for months in which CDBG or HOME projects started and continued to be active and 0 otherwise before a project’s implementation. The policy indicator variable for properties that never received treatment and were thus available for counterfactual analysis was coded as 0 for all the periods. Given the large dataset of counterfactual, non-treated observations, we mitigated selection bias by matching households on the basis of similar baseline electricity usage and household characteristics within the same city21,42. We combined matching models for bias reduction and covariate balance with staggered DiD or two-way fixed effects (TWFE) estimators for estimation efficiency. For more details, see Supplementary Note 10.

To evaluate the characteristics of treated and control units, we compiled data from the 2019 5-year American Community Survey62 and the Dougherty County Tax Assessor’s database of property records. This dataset included important property, demographic and neighbourhood characteristics known to affect household energy consumption. The most important pre-treatment property-level characteristics included the average monthly baseline energy consumption (in kWh per month per household), property size (in square feet), property age (in years), number of bedrooms and number of bathrooms. Demographic and neighbourhood characteristics included the household median income (in dollars), the share of female head of household (in percentage), the share of Black or African American population (in percentage) and alternative economic measures at the tract level (such as the share of the population below poverty level (in percentage), the share of the households with gross rent more than 35% of household income (in percentage) and the population on the Supplemental Nutrition Assistance Programme (SNAP (in percentage)). These physical and demographic characteristics are widely used in the building energy efficiency literature as matching or conditioning variables to reduce imbalance between treated and control properties21,63.

To mitigate the effect of possible unobservables on energy use, we included the fair market property value as a proxy for other potentially unobserved quality attributes64. Because property values could be influenced by housing programme criteria with the explanatory variable, we conducted additional analyses to show the main results with and without the property value as a conditioning variable to check for any potential biasing effect. Excluding property value in the conditioning variables generated somewhat higher treatment effects of 10%–19% (Supplementary Table 4). However, given possible unobserved factors related to housing stock quality, we included the property value in our models and reported the more conservative estimates. We also conducted additional robustness checks with an expanded set of testing variables related to age, homeownership and disability status to confirm bias reduction across further occupant characteristics. To mitigate other time-varying factors related to outdoor ambient temperature effects on energy demand, we also included archival weather station data from the National Oceanic and Atmospheric Administration (NOAA) to adjust for seasonal heating and cooling degree-days65. We used data for the nearest weather station in Albany, which is located 4 miles from downtown Albany at the Southwest Georgia regional airport.

Selection bias and protocols for bias reduction

As expected in impact evaluation studies with voluntary programmes, we found evidence of strong self-selection bias. Before implementing the matching models, the treated and non-treated properties had large differences in observable property characteristics. Descriptive statistics revealed statistically significant differences across key testing variables (Supplementary Table 5). For example, participating properties receiving HUD funding are ~30% smaller in square footage and have almost two times lower property values (Supplementary Table 5), which characterizes the profile of units that typically receive federal housing assistance. For further pre-treatment comparisons across other conditioning variables, including demographic and neighbourhood features, see Supplementary Table 5. Figure 2 shows a summary of the pre- and post-matching differences and covariate balance between treated and non-treated properties expressed as standardized percent bias (Supplementary Note 9).

Matching algorithms

Before analysis by difference-in-differences, we implemented multivariate matching procedures as a pre-processing step to construct statistical reference groups for analysis and to mitigate observational bias. Previous research in building energy efficiency has demonstrated notable performance gains in large datasets, particularly with the availability of high-performance computing resources21. We implemented algorithmic matching procedures with genetic matching, which automatically finds the optimal solution and fitness parameters that achieve maximum covariate balance23,66. Genetic matching automates the process of covariate balancing under various objective functions such as maximizing P values or minimizing standardized mean differences in empirical quantile–quantile distances across all matching variables.

We used matching protocols ‘with replacement’ that allowed us to preserve a larger sample size while not exceeding the ratio of controls over treated units that degrade performance. We ran the Genmatch script with all possible ratios of treated to control observations in the range from 1 to 100. This grid search resulted in a local optimum at a ratio of 19:1, meaning that up to 19 untreated properties weighted on their characteristics were available to each of the treated units for comparison. To fine-tune the ratio parameter, we implemented a rule-based optimization procedure that (1) maximized the average reduction in standardized mean differences and (2) minimized the number of pruned observations in the counterfactual66. Supplementary Fig. 2 shows the sensitivity of the standardized mean differences to changes in the ratio parameter for genetic matching, while Supplementary Fig. 3 shows the sensitivity of standardized mean differences to changes in observations pruned for the same values of the ratio parameter. Given the extended run times for genetic matching, we used multiple cores on a high-performance computing cluster to reduce computation time.

To benchmark our matching results, we conducted propensity score matching (PSM). We found a local optimum for bias reduction at a ratio of 21:1 of non-treated to treated units. In Supplementary Fig. 4, we show the sensitivity of standardized mean differences to changes in the ratio parameter, while Supplementary Fig. 5 shows the sensitivity of standardized mean differences to changes in observations pruned for the same values of the treated to untreated ratio.

Our best-performing model was genetic matching, which achieved an average and median bias reduction of 91% and 93%, respectively. This is better than the 78% average and 84% median bias reduction achieved with PSM across our conditioning and testing variables in Fig. 2. One limitation of propensity score models is that they might require a researcher’s discretion in the selection of parameters of interest67. For this reason, we favoured use of the automated methods with genetic matching, which also achieves better bias reduction in this application.

Balance–size matching frontier

To provide additional evidence on the comparative performance of the matching models, we implemented the Matching Frontier technique68, which allows us to estimate the theoretical limit to jointly maximize covariate balance and sample size. We used a specialized R package that allows for synchronous optimization of covariate balance and sample size (for details, see the Code Availability Statement). These results are presented in Supplementary Fig. 6. Genetic matching achieves a larger bias reduction, but it also produces a lower absolute loss imbalance (L1) compared with the PSM approach. These findings confirm that genetic matching is more efficient and gets closer to the balance–sample size frontier. The genetic matching procedures weakly dominate PSM matching across the key conditioning and testing variables. Therefore, given the richness of the current dataset, we were able to confirm that genetic matching is the preferred matching algorithm for this domain of building energy efficiency, as introduced in ref. 21.

Sensitivity of matching procedures to unobservables

We conducted Rosenbaum’s sensitivity analysis using protocols described in refs. 69,70. We calculated the critical value of the sensitivity parameter Γ, which captures the level of influence an unobserved confounder should need to affect the monthly kWh ft−2 outcome to change our inference. We estimated the changes in P values or significance levels on the basis of different values of Γ from 1 to 3 with a step size of 0.05. The critical gamma value is 1.45, where the confidence interval includes zero (Supplementary Table 6). This means that an unobserved covariate would have to change the energy intensity (in kWh ft−2) of participating households by ~45% before changing our inference at the 90% confidence level.

Although there could be other selection processes or time-varying unobservables not captured in our conditioning and testing variables, we believe it is unlikely because an unobserved confounder would have to exceed our threshold of 45% of the impact on the outcome variable in kWh ft−2.

Estimating treatment effects

To estimate causal programme impacts, we analysed the panel data using 16 years of monthly energy consumption records (in kWh ft−2) with and without matching. We used a TWFE with standard errors clustered at the property-address level, as reported in Table 1, as well as staggered DiD estimators. We provide additional details on the policy indicator in Supplementary Note 10. The reported treatment effects are robust to various levels of one-way and two-way clustering options (Supplementary Table 7).

To address potential estimation biases due to treatment effect heterogeneity in the presence of staggered programme adoption71, we implemented staggered DiD estimators72,73. We implemented two alternative protocols. The first approach72 uses not-yet-treated observations in a given period as counterfactual, while the second approach73 calculates the average treatment effect among switchers. We note that not every HUD-funded project in our study is subject to staggered adoption, which means that concerns about potential estimation biases with fixed effects estimators apply only to a subset of the studied projects. In Table 1, we report the results for three out of four energy-related projects that had staggered participation based on the project start date (for example, Energy Efficiency, Emergency Repairs and Homeowner Rehabilitation).

Supplementary Fig. 1 compares the dynamic DiD treatment effects with TWFE estimators after matching. Although there is some divergence in the dynamic treatment effect estimates in the later periods after more than 10 years or 40 quarters of performance data, we found that the staggered DiD treatment effect estimates were broadly consistent and within the 95% confidence intervals of each other for nearly all years in the study period (Supplementary Table 2). We also provide evidence of parallel trends for years before the start of housing projects and programme data collection in Supplementary Fig. 7. Importantly, given the quality of the data, we note that we did not rely on cross-sectional results for statistical significance, and we were able to measure year-to-year impacts using multiple approaches with matching before estimation of the event study (Supplementary Fig. 1). Due to covariate imbalances, the coupling of matching with DiD estimators was preferred such that covariates of never-treated units matched treated units. Recent econometric literature also points to the merits of matching before DiD analysis74,75. For a more general discussion of design issues to staggered DiD approaches, see refs. 71,76,77,78.

Placebo tests and other robustness checks

We implemented placebo tests in multiple ways to confirm the validity of our technical approach. First, we implemented a placebo test by analysing treated properties before any HUD investment from 2004 to 2007, where no effects are logically possible. We found treatment effects not statistically different from zero with two-way fixed effects and in models with and without matching, as shown in our main results in Table 1. As an additional falsification test, we considered funded CDBG and HOME projects not directly related to energy consumption, such as Tenant Based Rental Assistance or New Construction, to test for the direction of treatment effects. As shown in Supplementary Table 1, we found positive treatment effects up to 15% for non-energy projects with and without matching, as expected.

Another potential concern in treatment effect estimation is the uncertainty of the exact date ranges of project completion. This could introduce a source of measurement error, even as the benefits of capital improvements (HVAC unit, window sealing, roof repairs and so on) persist. Following ref. 30, we tested additional specifications by dropping observations where the treatment status is uncertain. Of 549 treated projects, we excluded 43 projects tagged as ‘incomplete’ (7.8% of treated projects). We confirmed that results with and without incomplete projects are all within the reported 95% confidence intervals under our three main specifications (Supplementary Table 8). This is expected given that the share of ‘incomplete projects’ in the sample is relatively small compared with the overall number of studied projects. Access to project status, tracked by the programme administrators and subsequently shared with the researchers, indicates minimal uncertainty in the date range as a possible source of evaluation error.

Cost effectiveness

To calculate the cost-effectiveness ratios, we considered the total kWh saved across all programme years divided by the total cost, which includes programme plus administrative costs. We used the most conservative treatment effect estimates (that is, genetic matching with two-way fixed effects), which provide a lower bound on the cost-effectiveness ratios. The programme costs are the direct entitlement funds, and administrative costs are the share of indirect costs as reported to HUD, excluding programme income (Supplementary Note 7). For this analysis, we did not consider other indirect costs, such as the social cost of carbon.

Administrator interviews and community engagement

To understand the localized administrative drivers of the CDBG and HOME programmes, we conducted 10 semi-structured interviews with public administrators, including the City of Albany’s Manager’s Office, the Department of Community and Economic Development (DCED), which manages the HUD projects and funding, Technology and Communications, and Utility Operations departments. We also conducted 40 semi-structured interviews with Albany residents to assess the programme effectiveness in the field. Of the 40 interviewees, 24 received a CDBG or HOME treatment at some point during the project period, and 16 did not receive the treatment. Participants in the Emergency Repairs programme made up 55% of all interviewees and 92% of all treated households. All interviews were conducted via phone from May 2020 to August 2020. We recruited resident interviewees in several ways: cold called past participants from DCED lists; mailed 927 postcards to past participants, which included contact information and a link to an online form to sign up for the interviews; circulated a press release and social media posts via the city’s communications office (from which we received two press articles); and sent personalized hand-addressed letters to 15 past HOME participants. All interviewees gave their informed consent for research purposes; personal data were anonymized and saved separately from interview recordings and transcripts.

Reporting summary

Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.