The risk elicitation puzzle

Published online:


Evidence shows that people’s preference for risk changes considerably when measured using different methods, which led us to question whether the common practice of using a single behavioural elicitation method (EM) reflects a valid measure. The present study addresses this question by examining the across-methods consistency of observed risk preferences in 1,507 healthy participants using six EMs. Our analyses show that risk preferences are not consistent across methods when operationalized on an absolute scale, a rank scale or the level of model parameters of cumulative prospect theory. This is at least partly explained by the finding that participants do not consistently follow the same decision strategy across EMs. After controlling for methodological and human factors that may impede consistency, our results challenge the view that different EMs manage to stably capture risk preference. Instead, we interpret the results as suggesting that risk preferences may be constructed when they are elicited, and different cognitive processes can lead to varying preferences.

  • Subscribe to Nature Human Behaviour for full access:



Additional access options:

Already a subscriber?  Log in  now or  Register  for online access.


  1. 1.

    Arrow, K. J. Aspects of the Theory of Risk-Bearing. (Yrjö Jahnssonin Foundation, Helsinki, 1965).

  2. 2.

    Booske, B. C., Sainfort, F. & Hundt, A. S. Eliciting consumer preferences for health plans. Health Serv. Res. 34, 839–854 (1999).

  3. 3.

    Saigal, S., Stoskopf, B. L., Burrows, E., Streiner, D. L. & Rosenbaum, P. L. Stability of maternal preferences for pediatric health states in the perinatal period and 1 year later. Arch. Pediatr. Adolesc. Med. 157, 261–269 (2003).

  4. 4.

    Samuelson, P. A. Lifetime portfolio selection by dynamic stochastic programming. Rev. Econ. Stat. 51, 239–246 (1969).

  5. 5.

    Hertwig, R. & Erev, I. The description–experience gap in risky choice. Trends Cogn. Sci. 13, 517–523 (2009).

  6. 6.

    Charness, G., Gneezy, U. & Halladay, B. Experimental methods: pay one or pay all. J. Econ. Behav. Organ. 131, 141–150 (2016).

  7. 7.

    Tversky, A., Sattath, S. & Slovic, P. Contingent weighting in judgment and choice. Psychol. Rev. 95, 371–384 (1988).

  8. 8.

    Anderson, L. R. & Mellor, J. M. Are risk preferences stable? Comparing an experimental measure with a validated survey-based measure. J. Risk Uncertain. 39, 137–160 (2009).

  9. 9.

    Dave, C., Eckel, C. C., Johnson, C. A. & Rojas, C. Eliciting risk preferences: when is simple better? J. Risk Uncertain. 41, 219–243 (2010).

  10. 10.

    Reynaud, A. & Couture, S. Stability of risk preference measures: results from a field experiment on French farmers. Theory Decis. 73, 203–221 (2012).

  11. 11.

    Dulleck, U., Fooken, J. & Fell, J. Within-subject intra- and inter-method consistency of two experimental risk attitude elicitation methods. German Econ. Rev. 16, 104–121 (2013).

  12. 12.

    Deck, C., Lee, J., Reyes, J. A. & Rosen, C. C. A failed attempt to explain within subject variation in risk taking behavior using domain specific risk attitudes. J. Econ. Behav. Organ. 87, 1–24 (2013).

  13. 13.

    Szrek, H., Chao, L. W., Ramlagan, S. & Peltzer, K. Predicting (un)healthy behavior: a comparison of risk-taking propensity measures. Judgm. Decis. Mak. 7, 716–727 (2012).

  14. 14.

    Bruner, D. M. Changing the probability versus changing the reward. Exp. Econ. 12, 367–385 (2009).

  15. 15.

    Ihli, H. J., Chiputwa, B. & Musshoff, O. Do changing probabilities or payoffs in lottery-choice experiments affect risk preference outcomes? Evidence from rural Uganda. J. Agr. Resour. Econ. 41, 324–345 (2016).

  16. 16.

    Deck, C., Lee, J. & Reyes, J. Investing versus gambling: experimental evidence of multi-domain risk attitudes. Appl. Econ. Let. 21, 19–23 (2013).

  17. 17.

    Isaac, R. M. & James, D. Just who are you calling risk averse? J. Risk Uncertain. 20, 177–187 (2000).

  18. 18.

    Berg, J., Dickhaut, J. & McCabe, K. Risk preference instability across institutions: a dilemma. Proc. Natl Acad. Sci. USA 102, 4209–4214 (2005).

  19. 19.

    Harbaugh, W. T., Krause, K. & Vesterlund, L. The fourfold pattern of risk attitudes in choice and pricing tasks. Econ. J. 120, 595–611 (2010).

  20. 20.

    Menkhoff, L. & Sakha, S. Estimating risky behavior with multiple-item risk measures. J. Econ. Psychol. 59, 59–86 (2017).

  21. 21.

    Crosetto, P. & Filippin, A. A theoretical and experimental appraisal of four risk elicitation methods. Exp. Econ. 19, 613–641 (2016).

  22. 22.

    He, P., Veronesi, M. & Engel, S. Consistency of Risk Preference Measures and the Role of Ambiguity: An Artefactual Field Experiment from China (WP 3, Working Paper Series, Department of Economics, Univ. Verona, 2016).

  23. 23.

    Loomes, G. & Pogrebna, G. Measuring individual risk attitudes when preferences are imprecise. Econ. J. 124, 569–593 (2014).

  24. 24.

    Nielsen, T., Keil, A. & Zeller, M. Assessing farmers’ risk preferences and their determinants in a marginal upland area of Vietnam: a comparison of multiple elicitation techniques. Agric. Econ. 44, 255–273 (2013).

  25. 25.

    Drichoutis, A. & Lusk, J. What can multiple price lists really tell us about risk preferences? J. Risk Uncertain. 53, 89–106 (2016).

  26. 26.

    Fausti, S. W. & Gillespie, J. M. A comparative analysis of risk preference elicitation procedures using mail survey results In Annual Meetings of the Western Agricultural Economics Association, Vancouver, Canada (Western Agricultural Economics Association, 2000).

  27. 27.

    Von Neumann, J. & Morgenstern, O. Theory of Games and Economic Behavior (Princeton Univ. Press, Princeton, NJ, 1947).

  28. 28.

    Holt, C. A. & Laury, S. K. Risk aversion and incentive effects. Am. Econ. Rev. 92, 1644–1655 (2002).

  29. 29.

    Lejuez, C. et al. Evaluation of a behavioral measure of risk taking: the Balloon Analogue Risk Task (BART). J. Exp. Psychol. Appl. 8, 75 (2002).

  30. 30.

    Hey, J. D. & Orme, C. Investigating generalizations of expected utility theory using experimental data. Econometrica 62, 1291–1326 (1994).

  31. 31.

    Figner, B., Mackinlay, R. J., Wilkening, F. & Weber, E. U. Affective and deliberative processes in risky choice: age differences in risk taking in the Columbia Card Task. J. Exp. Psychol. Learn. Mem. Cogn. 35, 709–730 (2009).

  32. 32.

    Dutilh, G. & Rieskamp, J. Comparing perceptual and preferential decision making. Psychon. Bull. Rev. 23, 723–737 (2015).

  33. 33.

    Markowitz, H. Portfolio selection. J. Finance 7, 77–91 (1952).

  34. 34.

    Starmer, C. Developments in non-expected utility theory: the hunt for a descriptive theory of choice under risk. J. Econ. Lit. 38, 332–382 (2000).

  35. 35.

    Lejarraga, T., Pachur, T., Frey, R. & Hertwig, R. Decisions from experience: from monetary to medical gambles. J. Behav. Decis. Mak. 29, 67–77 (2016).

  36. 36.

    Pachur, T., Mata, R. & Hertwig, R. Who dares, who errs? Disentangling cognitive and motivational roots of age differences in decisions under risk. Psychol. Sci. 28, 504–518 (2017).

  37. 37.

    Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S. & Garcia-Retamero, R. Measuring risk literacy: the Berlin numeracy test. J. Behav. Decis. Mak. 7, 25–47 (2012).

  38. 38.

    Wallsten, T. S., Pleskac, T. J. & Lejuez, C. Modeling behavior in a clinically diagnostic sequential risk-taking task. Psychol. Rev. 112, 862–880 (2005).

  39. 39.

    Lichtenstein, S. & Slovic, P. Reversals of preference between bids and choices in gambling decisions. J. Exp. Psychol. 89, 46–55 (1971).

  40. 40.

    Payne, J. W. Contingent decision behavior. Psychol. Bull. 92, 382–402 (1982).

  41. 41.

    Slovic, P. & Lichtenstein, S. Preference reversals: a broader perspective. Am. Econ. Rev. 73, 596–605 (1983).

  42. 42.

    Tversky, A. & Kahneman, D. in Environmental Impact Assessment, Technology Assessment, and Risk Analysis (eds Covello, V. T., Mumpower, J. L., Stallen, P. J. M. & Uppuluri, V. R. R.) 107–129 (Springer-Verlag, Berlin, Heidelberg, 1985).

  43. 43.

    Hey, J. D. & Orme, C. Investigating generalizations of expected utility theory using experimental data. Econometrica 62, 1291–1326 (1994).

  44. 44.

    Harless, D. W. & Camerer, C. F. The predictive utility of generalized expected utility theories. Econometrica 62, 1251–1289 (1994).

  45. 45.

    Bruhin, A., Fehr-Duda, H. & Epper, T. Risk and rationality: uncovering heterogeneity in probability distortion. Econometrica 78, 1375–1412 (2010).

  46. 46.

    Conte, A., Hey, J. D. & Moffatt, P. G. Mixture models of choice under risk. J. Econ. 162, 79–88 (2011).

  47. 47.

    Slovic, P. The construction of preference. Am. Psychol. 50, 364–371 (1995).

  48. 48.

    Ferguson, C. J. A meta-analysis of normal and disordered personality across the life span. J. Pers. Soc. Psychol. 98, 659–667 (2010).

  49. 49.

    Vieider, F. M. et al. Common components of risk and uncertainty attitudes across contexts and domains: evidence from 30 countries. J. Eur. Econ. Assoc. 13, 421–452 (2015).

  50. 50.

    Mesquita, B., Barrett, L. F. & Smith, E. R. The Mind in Context (Guilford Press, New York, 2010).

  51. 51.

    Josef, A. K. et al. Stability and change in risk-taking propensity across the adult life span. J. Pers. Soc. Psychol. 111, 430–450 (2016).

  52. 52.

    Frey, R., Pedroni, A., Mata, R., Rieskamp, J. & Hertwig, R. Risk preference shares the psychometric structure of major psychological traits. Sci. Adv. (in the press).

  53. 53.

    Hertwig, R., Barron, G., Weber, E. U. & Erev, I. Decisions from experience and the effect of rare events in risky choice. Psychol. Sci. 15, 534–539 (2004).

  54. 54.

    Lejuez, C. et al. Evaluation of a behavioral measure of risk taking: the balloon analogue risk task (BART). J. Exp. Psychol. Appl. 8, 75–84 (2002).

  55. 55.

    Figner, B., Mackinlay, R. J., Wilkening, F. & Weber, E. U. Affective and deliberative processes in risky choice: age differences in risk taking in the Columbia Card Task. J. Exp. Psychol. Learn. Mem. Cogn. 35, 709–730 (2009).

  56. 56.

    Dutilh, G. & Rieskamp, J. Comparing perceptual and preferential decision making. Psychon. Bull. Rev. 23, 723–737 (2015).

  57. 57.

    Von Helversen, B. & Rieskamp, J. Does the influence of stress on financial risk taking depend on the riskiness of the decision? In 35th Annual Conference of the Cognitive Science Soc. 1546–1551 (2013).

  58. 58.

    Rieskamp, J. The probabilistic nature of preferential choice. J. Exp. Psychol. Learn. Mem. Cogn. 34, 1446–1465 (2008).

  59. 59.

    Glöckner, A. & Pachur, T. Cognitive models of risky choice: parameter stability and predictive accuracy of prospect theory. Cognition 123, 21–32 (2012).

  60. 60.

    Prelec, D. The probability weighting function. Econometrica 66, 497–527 (1998).

Download references


This paper benefited from many helpful comments from the members of the Center for Economic Psychology at the University of Basel. We thank S. Goss and L. Wiles for editing this manuscript. This work was supported by the Swiss National Science Foundation with a grant to J.R. and R.H. (CRSII1_136227). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information


  1. Economic Psychology, Department of Psychology, University of Basel, 4055, Basel, Switzerland

    • Andreas Pedroni
    • , Gilles Dutilh
    •  & Jörg Rieskamp
  2. Methods of Plasticity Research, Department of Psychology, University of Zurich, 8050, Zurich, Switzerland

    • Andreas Pedroni
  3. Center for Cognitive and Decision Sciences, Department of Psychology, University of Basel, 4055, Basel, Switzerland

    • Renato Frey
  4. Center for Adaptive Rationality, Max Planck Institute for Human Development, 14195, Berlin, Germany

    • Renato Frey
    •  & Ralph Hertwig
  5. Faculty of Business and Economics (HEC Lausanne), University of Lausanne, 1015, Lausanne, Switzerland

    • Adrian Bruhin
    •  & Ralph Hertwig


  1. Search for Andreas Pedroni in:

  2. Search for Renato Frey in:

  3. Search for Adrian Bruhin in:

  4. Search for Gilles Dutilh in:

  5. Search for Ralph Hertwig in:

  6. Search for Jörg Rieskamp in:


A.P., R.F., A.B., G.D., R.H. and J.R. designed the research and wrote the paper. A.P. and R.F. performed the experimental studies. A.P., R.F. and A.B. analysed the data.

Competing interests

The authors declare no competing interests.

Corresponding author

Correspondence to Andreas Pedroni.

Electronic supplementary material

  1. Supplementary Information

    Supplementary Figures 1–6, Supplementary Tables 1–7, Supplementary Discussion.

  2. Life Science Reporting Summary