In uncertain environments, effective decision makers balance exploiting options that are currently preferred against exploring alternative options that may prove superior1,2. For example, a honeybee foraging for nectar must decide whether to continue exploiting the current patch or move to a new location3–6. When the relative reward of options changes over time, humans explore in a normatively correct fashion, exploring more often when they are uncertain about the relative value of competing options7–11. However, rewards in these laboratory studies were objective (for example, monetary payoff), whereas many real-world decision environments involve subjective evaluations of reward (for example, satisfaction with food choice). In such cases, rather than choices following preferences, preferences may follow choices with subjective reward (that is, value) to maximize coherency between preferences and behaviour12,13. If so, increasing coherency would lessen the tendency to explore while uncertainty increases, contrary to previous findings. To evaluate this possibility, we examined the exploratory choices of more than 280,000 anonymized individuals in supermarkets over several years. Consumers’ patterns of exploratory choice ran counter to normative models for objective rewards7–9,14—the longer the exploitation streak for a product, the less likely people were to explore an alternative. Furthermore, customers preferred coupons to explore alternative products when they had recently started an exploitation streak. These findings suggest interventions to promote healthy lifestyle choices.
Subscribe to Journal
Get full journal access for 1 year
only $8.67 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Cohen, J. D., McClure, S. M. & Yu, A. J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Phil. Trans. R. Soc. B 362, 933–942 (2007).
Hills, T. T. et al. Exploration versus exploitation in space, mind, and society. Trends Cogn. Sci. 19, 46–54 (2015).
Hutchinson, J. M., Wilke, A. & Todd, P. M. Patch leaving in humans: can a generalist adapt its rules to dispersal of items across patches? Anim. Behav. 75, 1331–1349 (2008).
Kramer, D. L. & Weary, D. M. Exploration versus exploitation: a field study of time allocation to environmental tracking by foraging chipmunks. Anim. Behav. 41, 443–449 (1991).
Krebs, J. R., Kacelnik, A. & Taylor, P. Test of optimal sampling by foraging great tits. Nature 275, 27–31 (1978).
Mobbs, D. et al. Foraging under competition: the neural basis of input-matching in humans. J. Neurosci. 33, 9866–9872 (2013).
Blanco, N. J., Otto, A. R., Maddox, W. T., Beevers, C. G. & Love, B. C. The influence of depression symptoms on exploratory decision-making. Cognition 129, 563–568 (2013).
Knox, W. B., Otto, A. R., Stone, P. & Love, B. The nature of belief-directed exploratory choice in human decision-making. Front. Psychol. 2, 398 (2012).
Otto, A. R., Knox, W. B., Markman, A. B. & Love, B. C. Physiological and behavioral signatures of reflective exploratory choice. Cogn. Affect. Behav. Neurosci. 14, 1167–1183 (2014).
Riefer, P. S. & Love, B. C. Unfazed by both the bull and bear: strategic exploration in dynamic environments. Games 6, 251–261 (2015).
Speekenbrink, M. & Konstantinidis, E. Uncertainty and exploration in a restless bandit problem. Top. Cogn. Sci. 7, 351–367 (2015).
Hall, L., Johansson, P., Tärning, B., Sikström, S. & Deutgen, T. Magic at the marketplace: choice blindness for the taste of jam and the smell of tea. Cognition 117, 54–61 (2010).
Johansson, P., Hall, L. & Chater, N. in Neuroscience of Preference and Choice: Cognitive and Neural Mechanisms (eds Dolan, R. J. & Sharot, T. ) 121–138 (Academic Press, 2012).
Blanco, N. J. et al. A frontal dopamine system for reflective exploratory behavior. Neurobiol. Learn. Mem. 123, 84–91 (2015).
Daw, N. D., O’Doherty, J. P., Dayan, P., Seymour, B. & Dolan, R. J. Cortical substrates for exploratory decisions in humans. Nature 441, 876–879 (2006).
Egan, L. C., Bloom, P. & Santos, L. R. Choice-induced preferences in the absence of choice: evidence from a blind two choice paradigm with young children and capuchin monkeys. J. Exp. Soc. Psychol. 46, 204–207 (2010).
Sharot, T., Fleming, S. M., Yu, X., Koster, R. & Dolan, R. J. Is choice-induced preference change long lasting? Psychol. Sci. 23, 1123–1129 (2012).
Sharot, T., Velasquez, C. M. & Dolan, R. J. Do decisions shape preference? Evidence from blind choice. Psychol. Sci. 21, 1231–1235 (2010).
Hall, L., Johansson, P. & Strandberg, T. Lifting the veil of morality: choice blindness and attitude reversals on a self-transforming survey. PloS ONE 7, e45457 (2012).
Johansson, P., Hall, L., Sikström, S. & Olsson, A. Failure to detect mismatches between intention and outcome in a simple decision task. Science 310, 116–119 (2005).
Festinger, L. A Theory of Cognitive Dissonance (Stanford Univ. Press, 1957).
Frey, D. Recent research on selective exposure to information. Adv. Exp. Soc. Psychol. 19, 41–80 (1986).
Jonas, E., Schulz-Hardt, S., Frey, D. & Thelen, N. Confirmation bias in sequential information search after preliminary decisions: an expansion of dissonance theoretical research on selective exposure to information. J. Pers. Soc. Psychol. 80, 557–571 (2001).
Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 1998).
Hills, T. T., Todd, P. M. & Goldstone, R. L. Search in external and internal spaces evidence for generalized cognitive search processes. Psychol. Sci. 19, 802–808 (2008).
Frank, M. J., Doll, B. B., Oas-Terpstra, J. & Moreno, F. Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. Nat. Neurosci. 12, 1062–1068 (2009).
Johansson, P., Hall, L., Tärning, B., Sikström, S. & Chater, N. Choice blindness and preference change: you will like this paper better if you (believe you) chose to read it! J. Behav. Decis. Making 27, 281–289 (2014).
Ariely, D. & Norton, M. I. How actions create—not just reveal—preferences. Trends Cogn. Sci. 12, 13–16 (2008).
We thank P. Todd for comments. This work was supported by the Leverhulme Trust grant RPG-2014-075, National Institutes of Health (grant 1P01HD080679) and Wellcome Trust Senior Investigator Award WT106931MA to B.C.L. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. At the time of submission, R.P., N.B. and G.P. were employed by dunnhumby Ltd. This work was carried out as part of P.S.R.’s PhD thesis, which was co-sponsored by dunnhumby Ltd. and UCL. dunnhumby Ltd. did not place any restrictions on the design, data collection and analysis, decision to publish or preparation of the manuscript, beyond the requirement that this work was to be done in compliance with its data policy.
The authors declare no competing interests.
About this article
Cite this article
Riefer, P., Prior, R., Blair, N. et al. Coherency-maximizing exploration in the supermarket. Nat Hum Behav 1, 0017 (2017). https://doi.org/10.1038/s41562-016-0017
Behavior Research Methods (2019)
PLOS Computational Biology (2019)
Proceedings of the National Academy of Sciences (2019)
Scientific Reports (2019)
Computational Brain & Behavior (2019)