Public trust demands greater openness from those whose research is used to set policy, argues Stefan Pfenninger.
The global transition towards a clean and sustainable energy future is well under way. New figures from Europe this month show that the continent is on track to reach its goal of a 20% renewable-energy share by 2020, and renewable capacity in China and the United States is also rising. But many technical, political and economic uncertainties remain, not least in the data and models used to underpin such policies. These uncertainties need open discussion, and yet energy strategies all over the world are based on research not open to scrutiny.
Researchers who seek, for example, to study the economic and energy model used by the US government (called NEMS) are met with a forbidding warning. On its website, the Energy Information Administration, which is developing the model, pronounces: “Most people who have requested NEMS in the past have found out that it was too difficult or rigid to use.”
At least NEMS (National Energy Modelling System) is publicly available. Most assumptions, systems, models and data used to set energy policy are not. These black-box simulations cannot be verified, discussed or challenged. This is bad for science, bad for the public and spreads distrust. Energy research needs to catch up with the open-software and open-data movements. We energy researchers should make our computer programs and data freely accessible, and academic publishing should shun us until we do.
Our community’s models are relevant to policy because they explore alternative scenarios or seek to understand the technical constraints on deploying new energy technologies. It is modelling for insight (by an academic exploring a range of qualitatively different scenarios for a clean energy supply, say) and for numbers (as in a government agency deciding on the remuneration level of a technology-support scheme).
Trust in this research matters because it contributes to policies on energy — and, by extension, on climate mitigation — that produce winners and losers throughout the global economy, and so can be hotly contested. Such policies are among the crucial driving forces that led to the current surge in the development of wind and solar power.
The list of reasons why energy models and data are not openly available is long: business confidentiality; concerns over the security of critical infrastructure; a desire to avoid exposure and scrutiny; worries about data being misrepresented or taken out of context; and a lack of time and resources.
Black-box simulations cannot be verified, discussed or challenged.
This secrecy is problematic, because it is well known that closed systems hide and perpetuate mistakes. A classic example is the spreadsheet error discovered in the influential Reinhart–Rogoff paper used to support economic policies of national austerity. The European Commission’s Energy Roadmap 2050 was based on a model that could not be viewed by outsiders, leaving it open to criticism. Assumptions that remain hidden, like the costs of technologies, can largely determine what comes out of such models. In the United Kingdom, opaque and overly optimistic cost assumptions for onshore wind went into models used for policymaking, and that may well have delayed the country’s decarbonization.
This closed culture is alien to younger researchers, who grew up with collaborative online tools and share code and data on platforms such as GitHub. Yet academia’s love affair with metrics and the pressure to publish set the wrong incentives: every hour spent on cleaning up a data set for public release or writing open-source code is time not spent working on a peer-reviewed paper.
Nevertheless, some academic-led projects are pushing towards more openness. The Enipedia project is building a worldwide open database on power plants, with data such as their locations and emissions. The Open Power System Data project gathers data such as electricity consumption from government agencies and transmission-network operators, and pushes for clarity on the licensing under which these data are made available. The Open Energy Modelling Initiative is emerging as a platform for coordinating and strengthening such efforts.
Regulation can also help. The European Union has mandated open access to electricity-market data, resulting in the creation of the ENTSO-E Transparency Platform to hold it, and there are good arguments for the creation of national energy-data agencies to coordinate the collection and archiving of a range of important data.
The vast majority of published research is still untouched by these fledgling initiatives. Only one energy journal — Energy Economics — currently requires data and models alongside submissions. Other journals should follow suit.
The open sharing of code and data is also important because it permits more meaningful collaboration between academics. Sharing a DNA sequence in an established format is, of course, easier than sharing the unstructured assumptions behind a techno-economic scenario study, for which no standard format exists yet. So the energy community must decide on standards for sharing code, data and assumptions.
A change in journal policies would help to kick-start these discussions. In policy-focused research, where one ‘truth’ does not exist, one cannot assess whether a modelled scenario is ‘correct’, so the important yardstick is not truth, but trust. The arrival of the post-truth world shows that trust in experts is lower than ever — and surely this is partly the experts’ fault.
Related links in Nature Research
Related external links
About this article
Cite this article
Pfenninger, S. Energy scientists must show their workings. Nature 542, 393 (2017). https://doi.org/10.1038/542393a
Scientific Reports (2021)
Implications to the electricity system of Paraguay of different demand scenarios and export prices to Brazil
Energy Systems (2021)
BMC Energy (2020)
Nature Communications (2020)
Back to the basic: toward improvement of technoeconomic representation in integrated assessment models
Climatic Change (2020)