Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

The risk elicitation puzzle

Nature Human Behaviour volume 1, pages 803–809 (2017)Cite this article

Subjects

Abstract

Evidence shows that people’s preference for risk changes considerably when measured using different methods, which led us to question whether the common practice of using a single behavioural elicitation method (EM) reflects a valid measure. The present study addresses this question by examining the across-methods consistency of observed risk preferences in 1,507 healthy participants using six EMs. Our analyses show that risk preferences are not consistent across methods when operationalized on an absolute scale, a rank scale or the level of model parameters of cumulative prospect theory. This is at least partly explained by the finding that participants do not consistently follow the same decision strategy across EMs. After controlling for methodological and human factors that may impede consistency, our results challenge the view that different EMs manage to stably capture risk preference. Instead, we interpret the results as suggesting that risk preferences may be constructed when they are elicited, and different cognitive processes can lead to varying preferences.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Degree of consistency in risk preferences across EMs.
Fig. 2: Types of decision-makers across EMs.

Similar content being viewed by others

References

  1. Arrow, K. J. Aspects of the Theory of Risk-Bearing. (Yrjö Jahnssonin Foundation, Helsinki, 1965).

    Google Scholar 

  2. Booske, B. C., Sainfort, F. & Hundt, A. S. Eliciting consumer preferences for health plans. Health Serv. Res. 34, 839–854 (1999).

    CAS  PubMed  PubMed Central  Google Scholar 

  3. Saigal, S., Stoskopf, B. L., Burrows, E., Streiner, D. L. & Rosenbaum, P. L. Stability of maternal preferences for pediatric health states in the perinatal period and 1 year later. Arch. Pediatr. Adolesc. Med. 157, 261–269 (2003).

    Article  PubMed  Google Scholar 

  4. Samuelson, P. A. Lifetime portfolio selection by dynamic stochastic programming. Rev. Econ. Stat. 51, 239–246 (1969).

    Article  Google Scholar 

  5. Hertwig, R. & Erev, I. The description–experience gap in risky choice. Trends Cogn. Sci. 13, 517–523 (2009).

    Article  PubMed  Google Scholar 

  6. Charness, G., Gneezy, U. & Halladay, B. Experimental methods: pay one or pay all. J. Econ. Behav. Organ. 131, 141–150 (2016).

    Article  Google Scholar 

  7. Tversky, A., Sattath, S. & Slovic, P. Contingent weighting in judgment and choice. Psychol. Rev. 95, 371–384 (1988).

    Article  Google Scholar 

  8. Anderson, L. R. & Mellor, J. M. Are risk preferences stable? Comparing an experimental measure with a validated survey-based measure. J. Risk Uncertain. 39, 137–160 (2009).

    Article  Google Scholar 

  9. Dave, C., Eckel, C. C., Johnson, C. A. & Rojas, C. Eliciting risk preferences: when is simple better? J. Risk Uncertain. 41, 219–243 (2010).

    Article  Google Scholar 

  10. Reynaud, A. & Couture, S. Stability of risk preference measures: results from a field experiment on French farmers. Theory Decis. 73, 203–221 (2012).

    Article  Google Scholar 

  11. Dulleck, U., Fooken, J. & Fell, J. Within-subject intra- and inter-method consistency of two experimental risk attitude elicitation methods. German Econ. Rev. 16, 104–121 (2013).

    Article  Google Scholar 

  12. Deck, C., Lee, J., Reyes, J. A. & Rosen, C. C. A failed attempt to explain within subject variation in risk taking behavior using domain specific risk attitudes. J. Econ. Behav. Organ. 87, 1–24 (2013).

    Article  Google Scholar 

  13. Szrek, H., Chao, L. W., Ramlagan, S. & Peltzer, K. Predicting (un)healthy behavior: a comparison of risk-taking propensity measures. Judgm. Decis. Mak. 7, 716–727 (2012).

    PubMed  PubMed Central  Google Scholar 

  14. Bruner, D. M. Changing the probability versus changing the reward. Exp. Econ. 12, 367–385 (2009).

    Article  Google Scholar 

  15. Ihli, H. J., Chiputwa, B. & Musshoff, O. Do changing probabilities or payoffs in lottery-choice experiments affect risk preference outcomes? Evidence from rural Uganda. J. Agr. Resour. Econ. 41, 324–345 (2016).

    Google Scholar 

  16. Deck, C., Lee, J. & Reyes, J. Investing versus gambling: experimental evidence of multi-domain risk attitudes. Appl. Econ. Let. 21, 19–23 (2013).

    Article  Google Scholar 

  17. Isaac, R. M. & James, D. Just who are you calling risk averse? J. Risk Uncertain. 20, 177–187 (2000).

    Article  Google Scholar 

  18. Berg, J., Dickhaut, J. & McCabe, K. Risk preference instability across institutions: a dilemma. Proc. Natl Acad. Sci. USA 102, 4209–4214 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Harbaugh, W. T., Krause, K. & Vesterlund, L. The fourfold pattern of risk attitudes in choice and pricing tasks. Econ. J. 120, 595–611 (2010).

    Article  Google Scholar 

  20. Menkhoff, L. & Sakha, S. Estimating risky behavior with multiple-item risk measures. J. Econ. Psychol. 59, 59–86 (2017).

    Article  Google Scholar 

  21. Crosetto, P. & Filippin, A. A theoretical and experimental appraisal of four risk elicitation methods. Exp. Econ. 19, 613–641 (2016).

    Article  Google Scholar 

  22. He, P., Veronesi, M. & Engel, S. Consistency of Risk Preference Measures and the Role of Ambiguity: An Artefactual Field Experiment from China (WP 3, Working Paper Series, Department of Economics, Univ. Verona, 2016).

  23. Loomes, G. & Pogrebna, G. Measuring individual risk attitudes when preferences are imprecise. Econ. J. 124, 569–593 (2014).

    Article  Google Scholar 

  24. Nielsen, T., Keil, A. & Zeller, M. Assessing farmers’ risk preferences and their determinants in a marginal upland area of Vietnam: a comparison of multiple elicitation techniques. Agric. Econ. 44, 255–273 (2013).

    Article  Google Scholar 

  25. Drichoutis, A. & Lusk, J. What can multiple price lists really tell us about risk preferences? J. Risk Uncertain. 53, 89–106 (2016).

    Article  Google Scholar 

  26. Fausti, S. W. & Gillespie, J. M. A comparative analysis of risk preference elicitation procedures using mail survey results In Annual Meetings of the Western Agricultural Economics Association, Vancouver, Canada (Western Agricultural Economics Association, 2000).

  27. Von Neumann, J. & Morgenstern, O. Theory of Games and Economic Behavior (Princeton Univ. Press, Princeton, NJ, 1947).

    Google Scholar 

  28. Holt, C. A. & Laury, S. K. Risk aversion and incentive effects. Am. Econ. Rev. 92, 1644–1655 (2002).

    Article  Google Scholar 

  29. Lejuez, C. et al. Evaluation of a behavioral measure of risk taking: the Balloon Analogue Risk Task (BART). J. Exp. Psychol. Appl. 8, 75 (2002).

    Article  CAS  PubMed  Google Scholar 

  30. Hey, J. D. & Orme, C. Investigating generalizations of expected utility theory using experimental data. Econometrica 62, 1291–1326 (1994).

    Article  Google Scholar 

  31. Figner, B., Mackinlay, R. J., Wilkening, F. & Weber, E. U. Affective and deliberative processes in risky choice: age differences in risk taking in the Columbia Card Task. J. Exp. Psychol. Learn. Mem. Cogn. 35, 709–730 (2009).

    Article  PubMed  Google Scholar 

  32. Dutilh, G. & Rieskamp, J. Comparing perceptual and preferential decision making. Psychon. Bull. Rev. 23, 723–737 (2015).

    Article  Google Scholar 

  33. Markowitz, H. Portfolio selection. J. Finance 7, 77–91 (1952).

    Google Scholar 

  34. Starmer, C. Developments in non-expected utility theory: the hunt for a descriptive theory of choice under risk. J. Econ. Lit. 38, 332–382 (2000).

    Article  Google Scholar 

  35. Lejarraga, T., Pachur, T., Frey, R. & Hertwig, R. Decisions from experience: from monetary to medical gambles. J. Behav. Decis. Mak. 29, 67–77 (2016).

    Article  Google Scholar 

  36. Pachur, T., Mata, R. & Hertwig, R. Who dares, who errs? Disentangling cognitive and motivational roots of age differences in decisions under risk. Psychol. Sci. 28, 504–518 (2017).

    Article  PubMed  Google Scholar 

  37. Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S. & Garcia-Retamero, R. Measuring risk literacy: the Berlin numeracy test. J. Behav. Decis. Mak. 7, 25–47 (2012).

    Google Scholar 

  38. Wallsten, T. S., Pleskac, T. J. & Lejuez, C. Modeling behavior in a clinically diagnostic sequential risk-taking task. Psychol. Rev. 112, 862–880 (2005).

    Article  PubMed  Google Scholar 

  39. Lichtenstein, S. & Slovic, P. Reversals of preference between bids and choices in gambling decisions. J. Exp. Psychol. 89, 46–55 (1971).

    Article  Google Scholar 

  40. Payne, J. W. Contingent decision behavior. Psychol. Bull. 92, 382–402 (1982).

    Article  Google Scholar 

  41. Slovic, P. & Lichtenstein, S. Preference reversals: a broader perspective. Am. Econ. Rev. 73, 596–605 (1983).

    Google Scholar 

  42. Tversky, A. & Kahneman, D. in Environmental Impact Assessment, Technology Assessment, and Risk Analysis (eds Covello, V. T., Mumpower, J. L., Stallen, P. J. M. & Uppuluri, V. R. R.) 107–129 (Springer-Verlag, Berlin, Heidelberg, 1985).

  43. Hey, J. D. & Orme, C. Investigating generalizations of expected utility theory using experimental data. Econometrica 62, 1291–1326 (1994).

    Article  Google Scholar 

  44. Harless, D. W. & Camerer, C. F. The predictive utility of generalized expected utility theories. Econometrica 62, 1251–1289 (1994).

    Article  Google Scholar 

  45. Bruhin, A., Fehr-Duda, H. & Epper, T. Risk and rationality: uncovering heterogeneity in probability distortion. Econometrica 78, 1375–1412 (2010).

    Article  Google Scholar 

  46. Conte, A., Hey, J. D. & Moffatt, P. G. Mixture models of choice under risk. J. Econ. 162, 79–88 (2011).

    Article  Google Scholar 

  47. Slovic, P. The construction of preference. Am. Psychol. 50, 364–371 (1995).

    Article  Google Scholar 

  48. Ferguson, C. J. A meta-analysis of normal and disordered personality across the life span. J. Pers. Soc. Psychol. 98, 659–667 (2010).

    Article  PubMed  Google Scholar 

  49. Vieider, F. M. et al. Common components of risk and uncertainty attitudes across contexts and domains: evidence from 30 countries. J. Eur. Econ. Assoc. 13, 421–452 (2015).

    Article  Google Scholar 

  50. Mesquita, B., Barrett, L. F. & Smith, E. R. The Mind in Context (Guilford Press, New York, 2010).

    Google Scholar 

  51. Josef, A. K. et al. Stability and change in risk-taking propensity across the adult life span. J. Pers. Soc. Psychol. 111, 430–450 (2016).

    Article  PubMed  PubMed Central  Google Scholar 

  52. Frey, R., Pedroni, A., Mata, R., Rieskamp, J. & Hertwig, R. Risk preference shares the psychometric structure of major psychological traits. Sci. Adv. (in the press).

  53. Hertwig, R., Barron, G., Weber, E. U. & Erev, I. Decisions from experience and the effect of rare events in risky choice. Psychol. Sci. 15, 534–539 (2004).

    Article  PubMed  Google Scholar 

  54. Lejuez, C. et al. Evaluation of a behavioral measure of risk taking: the balloon analogue risk task (BART). J. Exp. Psychol. Appl. 8, 75–84 (2002).

    Article  CAS  PubMed  Google Scholar 

  55. Figner, B., Mackinlay, R. J., Wilkening, F. & Weber, E. U. Affective and deliberative processes in risky choice: age differences in risk taking in the Columbia Card Task. J. Exp. Psychol. Learn. Mem. Cogn. 35, 709–730 (2009).

    Article  PubMed  Google Scholar 

  56. Dutilh, G. & Rieskamp, J. Comparing perceptual and preferential decision making. Psychon. Bull. Rev. 23, 723–737 (2015).

    Article  Google Scholar 

  57. Von Helversen, B. & Rieskamp, J. Does the influence of stress on financial risk taking depend on the riskiness of the decision? In 35th Annual Conference of the Cognitive Science Soc. 1546–1551 (2013).

  58. Rieskamp, J. The probabilistic nature of preferential choice. J. Exp. Psychol. Learn. Mem. Cogn. 34, 1446–1465 (2008).

    Article  PubMed  Google Scholar 

  59. Glöckner, A. & Pachur, T. Cognitive models of risky choice: parameter stability and predictive accuracy of prospect theory. Cognition 123, 21–32 (2012).

    Article  PubMed  Google Scholar 

  60. Prelec, D. The probability weighting function. Econometrica 66, 497–527 (1998).

    Article  Google Scholar 

Download references

Acknowledgements

This paper benefited from many helpful comments from the members of the Center for Economic Psychology at the University of Basel. We thank S. Goss and L. Wiles for editing this manuscript. This work was supported by the Swiss National Science Foundation with a grant to J.R. and R.H. (CRSII1_136227). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Economic Psychology, Department of Psychology, University of Basel, 4055, Basel, Switzerland

    Andreas Pedroni, Gilles Dutilh & Jörg Rieskamp

  2. Methods of Plasticity Research, Department of Psychology, University of Zurich, 8050, Zurich, Switzerland

    Andreas Pedroni

  3. Center for Cognitive and Decision Sciences, Department of Psychology, University of Basel, 4055, Basel, Switzerland

    Renato Frey

  4. Center for Adaptive Rationality, Max Planck Institute for Human Development, 14195, Berlin, Germany

    Renato Frey & Ralph Hertwig

  5. Faculty of Business and Economics (HEC Lausanne), University of Lausanne, 1015, Lausanne, Switzerland

    Adrian Bruhin & Ralph Hertwig

Authors
  1. Andreas Pedroni
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Renato Frey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adrian Bruhin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gilles Dutilh
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ralph Hertwig
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jörg Rieskamp
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

A.P., R.F., A.B., G.D., R.H. and J.R. designed the research and wrote the paper. A.P. and R.F. performed the experimental studies. A.P., R.F. and A.B. analysed the data.

Corresponding author

Correspondence to Andreas Pedroni.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Supplementary Information

Supplementary Figures 1–6, Supplementary Tables 1–7, Supplementary Discussion.

Life Science Reporting Summary

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pedroni, A., Frey, R., Bruhin, A. et al. The risk elicitation puzzle. Nat Hum Behav 1, 803–809 (2017). https://doi.org/10.1038/s41562-017-0219-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-017-0219-x

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing