Abstract
Public energy research and development (R&D) is recognized as a key policy tool for transforming the world’s energy system in a cost-effective way. However, managing the uncertainty surrounding technological change is a critical challenge for designing robust and cost-effective energy policies. The design of such policies is particularly important if countries are going to both meet the ambitious greenhouse-gas emissions reductions goals set by the Paris Agreement and achieve the required harmonization with the broader set of objectives dictated by the Sustainable Development Goals. The complexity of informing energy technology policy requires, and is producing, a growing collaboration between different academic disciplines and practitioners. Three analytical components have emerged to support the integration of technological uncertainty into energy policy: expert elicitations, integrated assessment models, and decision frameworks. Here we review efforts to incorporate all three approaches to facilitate public energy R&D decision-making under uncertainty. We highlight emerging insights that are robust across elicitations, models, and frameworks, relating to the allocation of public R&D investments, and identify gaps and challenges that remain.
This is a preview of subscription content, access via your institution
Relevant articles
Open Access articles citing this article.
-
Assessing the energy transition in China towards carbon neutrality with a probabilistic framework
Nature Communications Open Access 10 January 2022
Access options
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Rent or buy this article
Get just this article for as long as you need it
$39.95
Prices may be subject to local taxes which are calculated during checkout




References
Vaclav, S. Energy At a Crossroads: Global Perspectives and Uncertainties (MIT Press, 2005).
Sutherland, W. J. & Burgman, M. Policy advice: Use experts wisely. Nature 526, 317–318 (2015).
Anadon, L. D., Baker, E., Bosetti, V. & Reis, L. A. Expert views — and disagreements — about the potential of energy technology R&D. Climatic Change 136, 677–691 (2016). This paper brings together three sets of expert elicitations covering five important energy technologies and presents insights that include aggregated and disaggregated results and the impact of R&D on expected costs.
Baker, E., Bosetti, V. & Anadon, L. D. Special issue on defining robust energy R&D portfolios. Energy Policy 80, 215–218 (2015).
Nemet, G. F., Anadon, L. D. & Verdolini, E. Quantifying the effects of expert selection and elicitation design on experts’ confidence in their judgments about future energy technologies. Risk Anal. 37, 315–330 (2016).
Weyant, J. P., Knopf, B., De Cian, E. D., Keppo, I. & Van Vuuren, D. P. Introduction to the EMF28 study on scenarios for transforming the european energy system. Clim. Change Econ. 4, 1302001 (2013).
Morgan, M. G. Use (and abuse) of expert elicitation in support of decision making for public policy. Proc. Natl Acad. Sci. USA 111, 7176–7184 (2014). This paper provides an overview of the key practices and issues in the use of expert elicitations to inform policy, with examples that include environment and energy applications.
Baker, E., Olaleye, O. & Aleluia Reis, L. Decision frameworks and the investment in R&D. Energy Policy 80, 275–285 (2015). This paper isolates the impact of decision frameworks on R&D decisions using distributions over future technology costs from a combined set of expert elicitations.
Manne, A. S. Hedging Strategies for Global Carbon Dioxide Abatement: A Summary of Poll Results — EMF 14 Subgroup Analysis for Decisions under Uncertainty (Energy Modelling Forum, 1996).
Manne, A. S. & Richels, R. G. Buying Greenhouse Insurance: the Economic Costs of Carbon Dioxide Emission Limits (MIT press, 1992).
Varian, H. Micoreconomic Analsysis (Norton, 1992).
Dixit, A. K. & Pindyck, R. S. R. Investment Under Uncertainty (Princeton Univ. Press, 1994).
Pizer, W. A. et al. Optimal Choice of Policy Instrument and Stringency Under Uncertainty: the Case of Climate Change (Citeseer, 1997).
Lorenz, A., Schmidt, M. G. W., Kriegler, E. & Held, H. Anticipating climate threshold damages. Environ. Model. Assess. 17, 163–175 (2012).
Heal, G. & Kriström, B. Uncertainty and climate change. Environ. Resour. Econ. 22, 3–39 (2002).
Usher, W. & Strachan, N. An expert elicitation of climate, energy and economic uncertainties. Energy Policy 61, 811–821 (2013).
Santen, N. R. & Anadon, L. D. Balancing solar PV deployment and RD&D: a comprehensive framework for managing innovation uncertainty in electricity technology investment planning. Renew. Sustain. Energy Rev. 60, 560–569 (2016). This paper explores the role of the uncertainty framework (including approximate dynamic programming) on the balance between investments in solar R&D versus solar deployment given uncertainty around the future costs of solar PV.
Labriet, M., Kanudia, A. & Loulou, R. Climate mitigation under an uncertain technology future: a TIAM-WORLD analysis. Energy Econ. 34, S366–S377 (2012).
Verdolini, E., Bosetti, V., Anadón, L. D., Baker, E. & Aleluia Reis, L. The Future Prospects of Energy Technologies: Insights from Expert Elicitations Working Paper 47 2016 (FEEM, 2016).
Nemet, G. F. Beyond the learning curve: factors influencing cost reductions in photovoltaics. Energy Policy 34, 3218–3232 (2006).
Wirth, H. Recent Facts about Photovoltaics in Germany (Fraunhofer ISE, 2016).
Anadon, L. D., Chan, G. & Lee, A. in Transforming US Energy Innovation (eds Anadon, L. D., Bunn, M. & Narayanamurti, V. ) Ch. 2, 36–75 (Cambridge Univ. Press, 2014).
Verdolini, E., Anadon, L. D., Lu, J. & Nemet, G. F. The effects of expert selection, elicitation design, and R&D assumptions on experts’ estimates of the future costs of photovoltaics. Energy Policy 80, 233–243 (2015).
Gritsevskyi, A. & Nakićenovi, N. Modeling uncertainty of induced technological change. Energy Policy 28, 907–921 (2000).
Webster, M., Santen, N. & Parpas, P. An approximate dynamic programming framework for modeling global climate policy under decision-dependent uncertainty. Comput. Manag. Sci. 9, 339–362 (2012).
Clarke, L. et al. in Climate Change 2014: Mitigation of Climate Change (eds Edenhofer, O. et al.) Ch. 6, 413–510 (Cambridge Univ. Press, 2014). This section of the IPCC report covers the wide range of literature considering the impact of technology costs on mitigation without focusing on the role of public energy R&D.
Bellman, R. Dynamic programming and Lagrange multipliers. Proc. Natl Acad. Sci. USA 42, 767–769 (1956).
Powell, W. B. Approximate Dynamic Programming: Solving the Curses of Dimensionality (John Wiley & Sons, 2007).
Baker, E. Increasing risk and increasing informativeness: equivalence theorems. Operations Res. 54, 26–36 (2006).
Pugh, G. et al. Energy R&D portfolio analysis based on climate change mitigation. Energy Econ. 33, 634–643 (2011).
Ricci, E. C., Bosetti, V., Baker, E. & Jenni, K. E. From Expert Elicitations to Integrated Assessment: Future Prospects of Carbon Capture Technologies (FEEM, 2014).
Iyer, G., Hultman, N., Fetter, S. & Kim, S. H. Implications of small modular reactors for climate change mitigation. Energy Econ. 45, 144–154 (2014).
Barron, R. & McJeon, H. The differential impact of low-carbon technologies on climate change mitigation cost under a range of socioeconomic and climate policy scenarios. Energy Policy 80, 264–274 (2015).
Olaleye, O. & Baker, E. Large scale scenario analysis of future low carbon energy options. Energy Economics 49, 203–216 (2015).
Anderson, B., Borgonovo, E., Galeotti, M. & Roson, R. Uncertainty in climate change modeling: can global sensitivity analysis be of help? Risk Anal. 34, 271–293 (2014).
Bosetti, V. et al. Sensitivity to energy technology costs: a multi-model comparison analysis. Energy Policy 80, 244–263 (2015). This paper conducts a structured sensitivity analysis informed by expert elicitations over energy technology costs across three important energy–economic models, shedding light on the impact of model structure on mitigation costs.
Lehtveer, M. & Hedenus, F. How much can nuclear power reduce climate mitigation cost? — Critical parameters and sensitivity. Energy Strategy Rev. 6, 12–19 (2015).
Morgan, M. G. Our knowledge of the world is often not simple: policymakers should not duck that fact, but should deal with it. Risk Anal. 35, 19–20 (2015).
Gillenwater, M. Probabilistic decision model of wind power investment and influence of green power market. Energy Policy 63, 1111–1125 (2013).
Iman, R. L. & Conover, W. J. A distribution-free approach to inducing rank correlation among input variables. Commun. Stat. Simulation Comput. 11, 311–334 (1982).
Chan, G. & Anadon, L. D. Improving decision making for public R&D investment in energy: utilizing expert elicitation in parametric models. Preprint at https://doi.org/10.17863/CAM.7842 (2016). This paper combines expert elicitations and LHS to optimize public R&D investments in six energy technology areas affecting costs in a total of 25 technologies, identifying areas with the largest returns.
Olaleye, O. P. Role of Low Carbon Energy Technologies in Near Term Energy Policy. PhD thesis, Univ. Mass. Amherst (2016).
Lemoine, D. & McJeon, H. C. Trapped between two tails: trading off scientific uncertainties via climate targets. Environ. Res. Lett. 8, 34019 (2013).
Baker, E., Chon, H. & Keisler, J. Carbon capture and storage: combining economic analysis with expert elicitations to inform climate policy. Climatic Change 96, 379–408 (2009).
Baker, E., Chon, H. & Keisler, J. Advanced solar R&D: combining economic analysis with expert elicitations to inform climate policy. Energy Econ. 31, S37–S49 (2009).
Baker, E., Chon, H. & Keisler, J. Battery technology for electric and hybrid vehicles: expert views about prospects for advancement. Technol. Forecast. Soc. Change 77, 1139–1146 (2010).
Nemet, G. F. & Baker, E. Demand subsidies versus R&D: comparing the uncertain impacts of policy on a pre-commercial low-carbon energy technology. Energy J. 30, 49–80 (2009).
Jenni, K. E., Baker, E. D. & Nemet, G. F. Expert elicitations of energy penalties for carbon capture technologies. Int. J. Greenhouse Gas Control 12, 136–145 (2013).
Nemet, G. F., Baker, E. & Jenni, K. E. Modeling the future costs of carbon capture using experts’ elicited probabilities under policy scenarios. Energy 56, 218–228 (2013).
Nemet, G. F., Baker, E., Barron, B. & Harms, S. Characterizing the effects of policy instruments on the future costs of carbon capture for coal power plants. Climatic Change 133, 155–168 (2015).
Kahneman, D. Thinking, Fast and Slow (Macmillan, 2011).
Baker, E. & Solak, S. Management of energy technology for sustainability: how to fund energy technology research and development. Products Operat. Manag. 23, 348–365 (2014).
McJeon, H. Energy Technology Development And Climate Change Mitigation. PhD Thesis, Univ. Maryland (2012).
Barron, R., Djimadoumbaye, N. & Baker, E. How grid integration costs impact the optimal R&D portfolio into electricity supply technologies in the face of climate change. Sustain. Energy Technol. Assess. 7, 22–29 (2014).
Baker, E. & Solak, S. Climate change and optimal energy technology R&D policy. Eur. J. Oper. Res. 213, 442–454 (2011).
Marangoni, G., de Maere d’Aertrycke, G. & Bosetti, V. Optimal Clean Energy R&D Investments Under Uncertainty. FEEM Working Paper no. 2017.016 (2017).
Barron, R. Analysis of the Impact of Technological Change on the Cost of Achieving Climate Change Mitigation Targets. PhD Thesis, Univ. Mass. Amherst (2015).
Bistline, J. E. Energy technology R&D portfolio management: modeling uncertain returns and market diffusion. Appl. Energy 183, 1181–1196 (2016).
Baker, E., Bosetti, V., Anadon, L. D., Henrion, M. & Aleluia Reis, L. Future costs of key low-carbon energy technologies: harmonization and aggregation of energy technology expert elicitation data. Energy Policy 80, 219–232 (2015).
Baker, E., Clarke, L. & Shittu, E. Technical change and the marginal cost of abatement. Energy Econ. 30, 2799–2816 (2008).
Bistline, J. E. Electric sector capacity planning under uncertainty: climate policy and natural gas in the US. Energy Econ. 51, 236–251 (2015).
The Power of Change: Innovation for Development and Deployment of Increasingly Clean Electric Power Technologies (US National Academies Press, 2016).
Mowery, D. & Rosenberg, N. The influence of market demand upon innovation: a critical review of some recent empirical studies. Res. Policy 8, 102–153 (1979).
Jaffe, A. B., Newell, R. G. & Stavins, R. N. A tale of two market failures: technology and environmental policy. Ecol. Econ. 54, 164–174 (2005). This paper discusses some of the key reasons why public investment in energy R&D is very important.
Gallagher, K. S. & Anadon, L. D. DOE Budget Authority for Energy Research, Development & Demonstration Database (Belfer Center for Science and International Affairs, 2016); https://www.belfercenter.org/publication/doe-budget-authority-energy-research-development-demonstration-database-2
Munsell, M. US solar market sets new record, installing 7.3 GW of solar PV in 2015. Solar Energy Industries Association (22 February 2016).
Weiner, J. Median installed price of solar in the United States fell by 5–12% in 2015. Berkeley Lab News Center (24 August 2016).
Ziolkowska, J. R. Evaluating sustainability of biofuels feedstocks: a multi-objective framework for supporting decision making. Biomass Bioenergy 59, 425–440 (2013).
Pietzcker, R. C. et al. System integration of wind and solar power in Integrated Assessment Models: A cross-model evaluation of new approaches. Energy Econ. http://doi.org/10.1016/j.eneco.2016.11.018(2016).
Ackoff, R. L. Some unsolved problems in problem solving. Operat. Res. Q. 13, 1–11 (1962).
Anadón, L. D. Missions-oriented RD&D institutions in energy between 2000 and 2010: a comparative analysis of China, the United Kingdom, and the United States. Res. Policy 41, 1742–1756 (2012).
Rittel, H. W. & Webber, M. M. Dilemmas in a general theory of planning. Policy Sci. 4, 155–169 (1973).
Kriegler, E. et al. The role of technology for achieving climate policy objectives: overview of the EMF 27 study on global technology and climate policy strategies. Climatic Change 123, 353–367 (2014). This paper presents a large effort to estimate the value of investing in R&D to improve energy technologies without a direct focus on uncertainty or on optimizing R&D investments across a range of areas.
Weyant, J. P. Accelerating the development and diffusion of new energy technologies: beyond the ‘valley of death’. Energy Econ. 33, 674–682 (2011).
Nemet, G. Inter-technology knowledge spillovers for energy technologies. Energy Econ. 34, 1259–1270 (2012).
Wiesenthal, T., Leduc, G., Haegeman, K. & Schwarz, H.-G. Bottom-up estimation of industrial and public R&D investment by technology in support of policy-making: the case of selected low-carbon energy technologies. Res. Policy 41, 116–131 (2012).
Mission (Department of Energy, 2015); http://energy.gov/technologytransitions/mission
Office of Technology Transitions — Overview (US Department of Energy, accessed 10 April 2017); https://energy.gov/technologytransitions/downloads/office-technology-transitions-overview
Carbon Valuation in UK Policy Appraisal: a Revised Approach (Department of Energy and Climate Change, 2009).
Quality Assurance Tools and Guidance in BEIS (Department for Business, Energy and Industrial Strategy, 2016).
Baker, E., Bosetti, V. & Anadon, L. D. Special issue on defining robust energy R&D portfolios. Energy Policy 80, 215–218 (2015).
Fiorese, G., Catenacci, M., Bosetti, V. & Verdolini, E. The power of biomass: experts disclose the potential for success of bioenergy technologies. Energy Policy 65, 94–114 (2014).
Fiorese, G., Catenacci, M., Verdolini, E. & Bosetti, V. Advanced biofuels: future perspectives from an expert elicitation survey. Energy Policy 56, 293–311 (2013).
Baker, E. & Keisler, J. M. Cellulosic biofuels: expert views on prospects for advancement. Energy 36, 595–605 (2011).
Rao, A. B., Rubin, E. S., Keith, D. W. & Granger Morgan, M. Evaluation of potential cost reductions from improved amine-based CO2 capture systems. Energy Policy 34, 3765–3772 (2006).
Chung, T. S., Patiño-Echeverri, D. & Johnson, T. L. Expert assessments of retrofitting coal-fired power plants with carbon dioxide capture technologies. Energy Policy 39, 5609–5620 (2011).
Chan, G., Anadon, L. D., Chan, M. & Lee, A. Expert elicitation of cost, performance, and RD&D budgets for coal power with CCS. Energy Procedia 4, 2685–2692 (2011).
Prospective Evaluation of Applied Energy Research and Development at DOE (Phase Two) (NRC, 2007). This report by the US National Research Council makes the case that public energy R&D decisions need to be informed by methods that account for the inherent uncertainty in the results of such investments.
Abdulla, A., Azevedo, I. L. & Morgan, M. G. Expert assessments of the cost of light water small modular reactors. Proc. Natl Acad. Sci. USA 110, 9686–9691 (2013).
Anadón, L. D., Bosetti, V., Bunn, M., Catenacci, M. & Lee, A. Expert judgments about RD&D and the future of nuclear energy. Environ. Sci. Technol. 46, 11497–11504 (2012).
Baker, E., Chon, H. & Keisler, J. M. Advanced nuclear power: combining economic analysis with expert elicitations to inform climate policy. Preprint at http://dx.doi.org/10.2139/ssrn.1407048 (2008).
Curtright, A. E., Morgan, M. G. & Keith, D. W. Expert assessments of future photovoltaic technologies. Environ. Sci. Technol. 42, 9031–9038 (2008).
Bosetti, V., Catenacci, M., Fiorese, G. & Verdolini, E. The future prospect of PV and CSP solar technologies: an expert elicitation survey. Energy Policy 49, 308–317 (2012).
Mason, I. How Low Will Photovoltaic Prices Go? An Expert Discussion (Near Zero, 2012).
Catenacci, M., Verdolini, E., Bosetti, V. & Fiorese, G. Going electric: expert survey on the future of battery technologies for electric vehicles. Energy Policy 61, 403–413 (2013).
Bistline, J. E. Energy technology expert elicitations: an application to natural gas turbine efficiencies. Technol. Forecast. Soc. Change 86, 177–187 (2014).
Gillenwater, M. Probabilistic decision model of wind power investment and influence of green power market. Energy Policy 63, 1111–1125 (2013).
Wiser, R. et al. Expert elicitation survey on future wind energy costs. Nat. Energy 1, 16135 (2016).
Nemet, G. F., Baker, E., Barron, B. & Harms, S. Characterizing the effects of policy instruments on the future costs of carbon capture for coal power plants. Clim. Change 133, 155–168 (2015).
Nemet, G. F. & Baker, E. Demand subsidies versus R&D: comparing the uncertain impacts of policy on a pre-commercial low-carbon energy technology. Energy J. 30, 49–80 (2009).
Bistline, J. E. & Weyant, J. P. Electric sector investments under technological and policy-related uncertainties: a stochastic programming approach. Clim. Change 121, 143–160 (2013). This work uses a stochastic dynamic programming approach without relying on expert elicitations, and thus gives a sense of the literature that does not include such elicitations for parameterizing uncertainty.
McJeon, H. C. et al. Technology interactions among low-carbon energy technologies: what can we learn from a large number of scenarios? Energy Econ. 33, 619–631 (2011).
Baker, E. D., Bosetti, V. & Salo, A. Finding common ground when experts disagree: belief dominance over portfolios of alternatives Working Paper 46.2016 (FEEM, 2016).
Webster, M., Donohoo, P. & Palmintier, B. Water–CO2 trade-offs in electricity generation planning. Nat. Clim. Change 3, 1029–1032 (2013).
Anadon, L. D., Bunn, M. & Narayanamurti, V. (eds) Transforming US Energy Innovation (Cambridge Univ. Press, 2014).
Anadon, L., Bosetti, V., Bunn, M., Catenacci, M. & Lee, A. Expert judgments about RD&D and the future of nuclear energy. Environ. Sci. Technol. 46, 11497–11504 (2012).
Fishbone, L. G. & Abilock, H. Markal, a linear-programming model for energy systems analysis: technical description of the bnl version. Int. J. Energy Res. 5, 353–375 (1981).
Weyant, J. et al. in Climate Change 1995: Economic and Social Dimensions of Climate Change (eds Bruce, J. P., Lee, H. & Haites, E. F. ) Ch. 10, 367–396 (IPCC, Cambridge Univ. Press, 1996).
Pindyck, R. S. Climate change policy: what do the models tell us? J. Econ. Lit. 51, 860–872 (2013).
Nordhaus, W. D. A Question of Balance: Weighing the Options on Global Warming Policies. (Yale Univ. Press, 2014).
Edmonds, J. & Reilly, J. Global energy and CO2 to the year 2050. Energy J. 4, 21–47 (1983).
Bosetti, V., Carraro, C., Galeotti, M., Massetti, E. & Tavoni, M. WITCH A world induced technical change hybrid model. Energy J. 27, 13–37 (2006).
Tversky, A. & Kahneman, D. Judgment under uncertainty: heuristics and biases. Science 185, 1124–1131 (1974).
Barbose, G. L. et al. Tracking the Sun VIII: the Installed Price of Residential and Non-Residential Photovoltaic Systems in the United States (Lawrence Berkeley National Laboratory, 2015).
Cooke, R. M. The aggregation of expert judgment: do good things come to those who weight? Risk Anal. 35, 12–15 (2015).
Acknowledgements
We are grateful to Nidhi Santen for useful insights and feedback on the description on ADP, to Franklyn Kanyako for assistance with the figures, and to Patricia McLaughlin for her work proofreading it. L.D.A. is grateful to the Climate Change Initiative of the Doris Duke Charitable Foundation and the Science, Technology and Public Policy programme at the Harvard Kennedy School for supporting early work on this paper and to the EU H2020 research and innovation programme under the Grant Agreement No 730403 (INNOPATHS). V.B. is grateful to funding from the European Research Council under the European Community’s Programme ‘Ideas’ — Call identifier: ERC-2013-StG / ERC grant agreement number 336703 — project RISICO ‘RISk and uncertainty in developing and Implementing Climate change pOlicies’. We also acknowledge financial support from the United Nations Environment Program through the Green Growth Knowledge Platform (GGKP) Research Committee on Technology and Innovation for the preparation of GGKP Working paper 01-2016, a report that helped us identify the need for this Review. E.B. acknowledges the NSF-sponsored IGERT: Offshore Wind Energy Engineering, Environmental Science, and Policy (NSF DGE 1068864).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Rights and permissions
About this article
Cite this article
Anadón, L., Baker, E. & Bosetti, V. Integrating uncertainty into public energy research and development decisions. Nat Energy 2, 17071 (2017). https://doi.org/10.1038/nenergy.2017.71
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/nenergy.2017.71
This article is cited by
-
Assessing the energy transition in China towards carbon neutrality with a probabilistic framework
Nature Communications (2022)
-
Probabilistic feasibility space of scaling up green hydrogen supply
Nature Energy (2022)
-
Energy innovation funding and institutions in major economies
Nature Energy (2022)
-
Looking forward and back
Nature Energy (2021)
-
Rescue US energy innovation
Nature Energy (2017)