Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Coherency-maximizing exploration in the supermarket

Abstract

In uncertain environments, effective decision makers balance exploiting options that are currently preferred against exploring alternative options that may prove superior1,2. For example, a honeybee foraging for nectar must decide whether to continue exploiting the current patch or move to a new location36. When the relative reward of options changes over time, humans explore in a normatively correct fashion, exploring more often when they are uncertain about the relative value of competing options711. However, rewards in these laboratory studies were objective (for example, monetary payoff), whereas many real-world decision environments involve subjective evaluations of reward (for example, satisfaction with food choice). In such cases, rather than choices following preferences, preferences may follow choices with subjective reward (that is, value) to maximize coherency between preferences and behaviour12,13. If so, increasing coherency would lessen the tendency to explore while uncertainty increases, contrary to previous findings. To evaluate this possibility, we examined the exploratory choices of more than 280,000 anonymized individuals in supermarkets over several years. Consumers’ patterns of exploratory choice ran counter to normative models for objective rewards79,14—the longer the exploitation streak for a product, the less likely people were to explore an alternative. Furthermore, customers preferred coupons to explore alternative products when they had recently started an exploitation streak. These findings suggest interventions to promote healthy lifestyle choices.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: Predicted exploration patterns for uncertainty minimization and coherency maximization.
Figure 2: Exploration changes locally but not globally.
Figure 3: Predicting exploration from exploitation streak lengths.
Figure 4: Coupon redemption depending on current exploitation streak length.

References

  1. 1

    Cohen, J. D., McClure, S. M. & Yu, A. J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Phil. Trans. R. Soc. B 362, 933–942 (2007).

    Article  Google Scholar 

  2. 2

    Hills, T. T. et al. Exploration versus exploitation in space, mind, and society. Trends Cogn. Sci. 19, 46–54 (2015).

    Article  Google Scholar 

  3. 3

    Hutchinson, J. M., Wilke, A. & Todd, P. M. Patch leaving in humans: can a generalist adapt its rules to dispersal of items across patches? Anim. Behav. 75, 1331–1349 (2008).

    Article  Google Scholar 

  4. 4

    Kramer, D. L. & Weary, D. M. Exploration versus exploitation: a field study of time allocation to environmental tracking by foraging chipmunks. Anim. Behav. 41, 443–449 (1991).

    Article  Google Scholar 

  5. 5

    Krebs, J. R., Kacelnik, A. & Taylor, P. Test of optimal sampling by foraging great tits. Nature 275, 27–31 (1978).

    Article  Google Scholar 

  6. 6

    Mobbs, D. et al. Foraging under competition: the neural basis of input-matching in humans. J. Neurosci. 33, 9866–9872 (2013).

    CAS  Article  Google Scholar 

  7. 7

    Blanco, N. J., Otto, A. R., Maddox, W. T., Beevers, C. G. & Love, B. C. The influence of depression symptoms on exploratory decision-making. Cognition 129, 563–568 (2013).

    Article  Google Scholar 

  8. 8

    Knox, W. B., Otto, A. R., Stone, P. & Love, B. The nature of belief-directed exploratory choice in human decision-making. Front. Psychol. 2, 398 (2012).

    Article  Google Scholar 

  9. 9

    Otto, A. R., Knox, W. B., Markman, A. B. & Love, B. C. Physiological and behavioral signatures of reflective exploratory choice. Cogn. Affect. Behav. Neurosci. 14, 1167–1183 (2014).

    Article  Google Scholar 

  10. 10

    Riefer, P. S. & Love, B. C. Unfazed by both the bull and bear: strategic exploration in dynamic environments. Games 6, 251–261 (2015).

    Article  Google Scholar 

  11. 11

    Speekenbrink, M. & Konstantinidis, E. Uncertainty and exploration in a restless bandit problem. Top. Cogn. Sci. 7, 351–367 (2015).

    Article  Google Scholar 

  12. 12

    Hall, L., Johansson, P., Tärning, B., Sikström, S. & Deutgen, T. Magic at the marketplace: choice blindness for the taste of jam and the smell of tea. Cognition 117, 54–61 (2010).

    Article  Google Scholar 

  13. 13

    Johansson, P., Hall, L. & Chater, N. in Neuroscience of Preference and Choice: Cognitive and Neural Mechanisms (eds Dolan, R. J. & Sharot, T. ) 121–138 (Academic Press, 2012).

    Book  Google Scholar 

  14. 14

    Blanco, N. J. et al. A frontal dopamine system for reflective exploratory behavior. Neurobiol. Learn. Mem. 123, 84–91 (2015).

    CAS  Article  Google Scholar 

  15. 15

    Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).

    CAS  Article  Google Scholar 

  16. 16

    Egan, L. C., Bloom, P. & Santos, L. R. Choice-induced preferences in the absence of choice: evidence from a blind two choice paradigm with young children and capuchin monkeys. J. Exp. Soc. Psychol. 46, 204–207 (2010).

    Article  Google Scholar 

  17. 17

    Sharot, T., Fleming, S. M., Yu, X., Koster, R. & Dolan, R. J. Is choice-induced preference change long lasting? Psychol. Sci. 23, 1123–1129 (2012).

    Article  Google Scholar 

  18. 18

    Sharot, T., Velasquez, C. M. & Dolan, R. J. Do decisions shape preference? Evidence from blind choice. Psychol. Sci. 21, 1231–1235 (2010).

    Article  Google Scholar 

  19. 19

    Hall, L., Johansson, P. & Strandberg, T. Lifting the veil of morality: choice blindness and attitude reversals on a self-transforming survey. PloS ONE 7, e45457 (2012).

    CAS  Article  Google Scholar 

  20. 20

    Johansson, P., Hall, L., Sikström, S. & Olsson, A. Failure to detect mismatches between intention and outcome in a simple decision task. Science 310, 116–119 (2005).

    CAS  Article  Google Scholar 

  21. 21

    Festinger, L. A Theory of Cognitive Dissonance (Stanford Univ. Press, 1957).

    Google Scholar 

  22. 22

    Frey, D. Recent research on selective exposure to information. Adv. Exp. Soc. Psychol. 19, 41–80 (1986).

    Google Scholar 

  23. 23

    Jonas, E., Schulz-Hardt, S., Frey, D. & Thelen, N. Confirmation bias in sequential information search after preliminary decisions: an expansion of dissonance theoretical research on selective exposure to information. J. Pers. Soc. Psychol. 80, 557–571 (2001).

    CAS  Article  Google Scholar 

  24. 24

    Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998).

    Google Scholar 

  25. 25

    Hills, T. T., Todd, P. M. & Goldstone, R. L. Search in external and internal spaces evidence for generalized cognitive search processes. Psychol. Sci. 19, 802–808 (2008).

    Article  Google Scholar 

  26. 26

    Frank, M. J., Doll, B. B., Oas-Terpstra, J. & Moreno, F. Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neurosci. 12, 1062–1068 (2009).

    CAS  Article  Google Scholar 

  27. 27

    Johansson, P., Hall, L., Tärning, B., Sikström, S. & Chater, N. Choice blindness and preference change: you will like this paper better if you (believe you) chose to read it! J. Behav. Decis. Making 27, 281–289 (2014).

    Article  Google Scholar 

  28. 28

    Ariely, D. & Norton, M. I. How actions create—not just reveal—preferences. Trends Cogn. Sci. 12, 13–16 (2008).

    Article  Google Scholar 

Download references

Acknowledgements

We thank P. Todd for comments. This work was supported by the Leverhulme Trust grant RPG-2014-075, National Institutes of Health (grant 1P01HD080679) and Wellcome Trust Senior Investigator Award WT106931MA to B.C.L. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. At the time of submission, R.P., N.B. and G.P. were employed by dunnhumby Ltd. This work was carried out as part of P.S.R.’s PhD thesis, which was co-sponsored by dunnhumby Ltd. and UCL. dunnhumby Ltd. did not place any restrictions on the design, data collection and analysis, decision to publish or preparation of the manuscript, beyond the requirement that this work was to be done in compliance with its data policy.

Author information

Affiliations

Authors

Contributions

P.S.R. was involved in all parts of this project, supported by R.P., N.B. and G.P. regarding the data analysis and with input from B.C.L. for the design, analysis and write-up of the manuscript.

Corresponding author

Correspondence to Peter S. Riefer.

Ethics declarations

Competing interests

The authors declare no competing interests.

Supplementary information

Supplementary Information

Supplementary Data and Analyses, Supplementary Tables 1–4, Supplementary References (PDF 272 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Riefer, P., Prior, R., Blair, N. et al. Coherency-maximizing exploration in the supermarket. Nat Hum Behav 1, 0017 (2017). https://doi.org/10.1038/s41562-016-0017

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing