Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Dissociable mechanisms govern when and how strongly reward attributes affect decisions

Abstract

Theories and computational models of decision-making usually focus on how strongly different attributes are weighted in choice, for example, as a function of their importance or salience to the decision-maker. However, when different attributes affect the decision process is a question that has received far less attention. Here, we investigated whether the timing of attribute consideration has a unique influence on decision-making by using a time-varying drift diffusion model and data from four separate experiments. Experimental manipulations of attention and neural activity demonstrated that we can dissociate the processes that determine the relative weighting strength and timing of attribute consideration. Thus, the processes determining either the weighting strengths or the timing of attributes in decision-making can independently adapt to changes in the environment or goals. Quantifying these separate influences of timing and weighting on choice improves our understanding and predictions of individual differences in decision behaviour.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Fig. 1: Details of the different food choice tasks used in each study.
Fig. 2: Patterns of behaviour predicted by this tDDM.
Fig. 3: Influence of taste and healthiness attributes on choice outcomes and RTs.
Fig. 4: Choice patterns and separate attribute consideration onset tDDM parameter estimates for the IAC study by condition.
Fig. 5: Changes in health challenge success and tDDM parameter estimates following tDCS over the left dlPFC.

Data availability

The data analysed in this paper are openly available on the Open Science Framework at https://osf.io/g76fn/. Additional data for the MRT experiments from Sullivan et al.16 are available at https://osf.io/jmiwn/.

Code availability

The code for fitting the diffusion models and running the other analyses is openly available on the Open Science Framework at https://osf.io/g76fn/.

References

  1. 1.

    Mazur, J. E. in Quantitative Analyses of Behavior: The Effect of Delay and of Intervening Events on Reinforcement Value Vol. 5 (eds Commons, M. L. et al.) 55–73 (Erlbaum, 1987).

  2. 2.

    Fehr, E. & Schmidt, K. M. A theory of fairness, competition, and cooperation. Q. J. Econ. 114, 817–868 (1999).

    Google Scholar 

  3. 3.

    Laibson, D. Golden eggs and hyperbolic discounting. Q. J. Econ. 112, 443–477 (1997).

    Google Scholar 

  4. 4.

    Kahneman, D. & Tversky, A. Prospect theory: an analysis of decisions under risk. Econometrica 47, 263–292 (1979).

    Google Scholar 

  5. 5.

    Palminteri, S., Wyart, V. & Koechlin, E. The importance of falsification in computational cognitive modeling. Trends Cogn. Sci. 21, 425–433 (2017).

    PubMed  Google Scholar 

  6. 6.

    Carland, M. A., Marcos, E., Thura, D. & Cisek, P. Evidence against perfect integration of sensory information during perceptual decision making. J. Neurophysiol. 115, 915–930 (2016).

    PubMed  Google Scholar 

  7. 7.

    Summerfield, C. & Tsetsos, K. Building bridges between perceptual and economic decision-making: neural and computational mechanisms. Front. Neurosci. 6, 70 (2012).

    PubMed Central  PubMed  Google Scholar 

  8. 8.

    Gluth, S., Hotaling, J. M. & Rieskamp, J. The attraction effect modulates reward prediction errors and intertemporal choices. J. Neurosci. 37, 371–382 (2017).

    CAS  PubMed Central  PubMed  Google Scholar 

  9. 9.

    Krajbich, I., Armel, C. & Rangel, A. Visual fixations and the computation and comparison of value in simple choice. Nat. Neurosci. 13, 1292–1298 (2010).

    CAS  PubMed  Google Scholar 

  10. 10.

    Reeck, C., Wall, D. & Johnson, E. J. Search predicts and changes patience in intertemporal choice. Proc. Natl Acad. Sci. USA 114, 11890–11895 (2017).

    CAS  PubMed  Google Scholar 

  11. 11.

    Hunt, L. T., Dolan, R. J. & Behrens, T. E. Hierarchical competitions subserving multi-attribute choice. Nat. Neurosci. 17, 1613–1622 (2014).

    CAS  PubMed Central  PubMed  Google Scholar 

  12. 12.

    Kunar, M. A., Watson, D. G., Tsetsos, K. & Chater, N. The influence of attention on value integration. Atten Percept. Psychophys. 79, 1615–1627 (2017).

    PubMed Central  PubMed  Google Scholar 

  13. 13.

    Forstmann, B. U., Ratcliff, R. & Wagenmakers, E. J. Sequential sampling models in cognitive neuroscience: advantages, applications, and extensions. Annu. Rev. Psychol. 67, 641–666 (2016).

    CAS  PubMed  Google Scholar 

  14. 14.

    Clithero, J. A. Improving out-of-sample predictions using response times and a model of the decision process. J. Econ. Behav. Organ 148, 344–375 (2018).

    Google Scholar 

  15. 15.

    Webb, R. The (neural) dynamics of stochastic choice. Manage. Sci. https://doi.org/10.1287/mnsc.2017.2931 (2018).

  16. 16.

    Sullivan, N., Hutcherson, C., Harris, A. & Rangel, A. Dietary self-control is related to the speed with which attributes of healthfulness and tastiness are processed. Psychol. Sci. 26, 122–134 (2015).

    PubMed  Google Scholar 

  17. 17.

    Lim, S. L., Penrod, M. T., Ha, O. R., Bruce, J. M. & Bruce, A. S. Calorie labeling promotes dietary self-control by shifting the temporal dynamics of health- and taste-attribute integration in overweight individuals. Psychol. Sci. 29, 447–462 (2018).

    PubMed  Google Scholar 

  18. 18.

    Hare, T. A., Malmaud, J. & Rangel, A. Focusing attention on the health aspects of foods changes value signals in vmPFC and improves dietary choice. J. Neurosci. 31, 11077–11087 (2011).

    CAS  PubMed Central  PubMed  Google Scholar 

  19. 19.

    Ratcliff, R. A note on modeling accumulation of information when the rate of accumulation changes over time. J. Math. Psychol. 21, 178–184 (1980).

    Google Scholar 

  20. 20.

    Ratcliff, R. A theory of memory retrieval. Psychol. Rev. 85, 59–108 (1978).

    Google Scholar 

  21. 21.

    Ratcliff, R., Smith, P. L., Brown, S. D. & McKoon, G. Diffusion decision model: current issues and history. Trends Cogn. Sci. 20, 260–281 (2016).

    PubMed Central  PubMed  Google Scholar 

  22. 22.

    Holmes, W. R., Trueblood, J. S. & Heathcote, A. A new framework for modeling decisions about changing information: the piecewise linear ballistic accumulator model. Cogn. Psychol. 85, 1–29 (2016).

    PubMed Central  PubMed  Google Scholar 

  23. 23.

    Cisek, P., Puskas, G. A. & El-Murr, S. Decisions in changing conditions: the urgency-gating model. J. Neurosci. 29, 11560–11571 (2009).

    CAS  PubMed Central  PubMed  Google Scholar 

  24. 24.

    Usher, M. & McClelland, J. L. The time course of perceptual choice: the leaky, competing accumulator model. Psychol. Rev. 108, 550–592 (2001).

    CAS  Google Scholar 

  25. 25.

    Brown, S. D. & Heathcote, A. The simplest complete model of choice response time: linear ballistic accumulation. Cogn. Psychol. 57, 153–178 (2008).

    Google Scholar 

  26. 26.

    Busemeyer, J. R. & Townsend, J. T. Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. Psychol. Rev. 100, 432–459 (1993).

    CAS  Google Scholar 

  27. 27.

    Roe, R. M., Busemeyer, J. R. & Townsend, J. T. Multialternative decision field theory: a dynamic connectionist model of decision making. Psychol. Rev. 108, 370–392 (2001).

    CAS  PubMed  Google Scholar 

  28. 28.

    Stewart, N., Chater, N. & Brown, G. D. Decision by sampling. Cogn. Psychol. 53, 1–26 (2006).

    PubMed  Google Scholar 

  29. 29.

    Ulrich, R., Schröter, H., Leuthold, H. & Birngruber, T. Automatic and controlled stimulus processing in conflict tasks: superimposed diffusion processes and delta functions. Cogn. Psychol. 78, 148–174 (2015).

    PubMed  Google Scholar 

  30. 30.

    Ratcliff, R. & Rouder, J. N. Modeling response times for two-choice decisions. Psychol. Sci. 9, 347–356 (1998).

    Google Scholar 

  31. 31.

    Alós‐Ferrer, C. A dual-process diffusion model. J. Behav. Decis. Mak. 31, 203–218 (2018).

    Google Scholar 

  32. 32.

    Caplin, A. & Martin, D. The dual-process drift diffusion model: evidence from response times. Econ. Inq. 54, 1274–1282 (2016).

    Google Scholar 

  33. 33.

    Foerde, K. et al. Assessment of test–retest reliability of a food choice task among healthy individuals. Appetite 123, 352–356 (2018).

    PubMed Central  PubMed  Google Scholar 

  34. 34.

    Krawczyk, D. C. Contributions of the prefrontal cortex to the neural basis of human decision making. Neurosci. Biobehav. Rev. 26, 631–664 (2002).

    PubMed  Google Scholar 

  35. 35.

    Han, J. E., Boachie, N., Garcia-Garcia, I., Michaud, A. & Dagher, A. Neural correlates of dietary self-control in healthy adults: a meta-analysis of functional brain imaging studies. Physiol. Behav. 192, 98–108 (2018).

    CAS  PubMed  Google Scholar 

  36. 36.

    Kable, J. W. & Glimcher, P. W. The neural correlates of subjective value during intertemporal choice. Nat. Neurosci. 10, 1625–1633 (2007).

    CAS  PubMed Central  PubMed  Google Scholar 

  37. 37.

    Rushworth, M. F. & Behrens, T. E. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat. Neurosci. 11, 389–397 (2008).

    CAS  PubMed  Google Scholar 

  38. 38.

    Ruff, C. C., Ugazio, G. & Fehr, E. Changing social norm compliance with noninvasive brain stimulation. Science 342, 482–484 (2013).

    CAS  PubMed  Google Scholar 

  39. 39.

    Shen, B. et al. High-definition tDCS alters impulsivity in a baseline-dependent manner. NeuroImage 143, 343–352 (2016).

    PubMed  Google Scholar 

  40. 40.

    Essex, B. G., Clinton, S. A., Wonderley, L. R. & Zald, D. H. The impact of the posterior parietal and dorsolateral prefrontal cortices on the optimization of long-term versus immediate value. J. Neurosci. 32, 15403–15413 (2012).

    CAS  PubMed Central  PubMed  Google Scholar 

  41. 41.

    Cho, S. S. et al. Effect of continuous theta burst stimulation of the right dorsolateral prefrontal cortex on cerebral blood flow changes during decision making. Brain Stimul. 5, 116–123 (2012).

    PubMed Central  PubMed  Google Scholar 

  42. 42.

    Camus, M. et al. Repetitive transcranial magnetic stimulation over the right dorsolateral prefrontal cortex decreases valuations during food choices. Eur. J. Neurosci. 30, 1980–1988 (2009).

    PubMed  Google Scholar 

  43. 43.

    Figner, B. et al. Lateral prefrontal cortex and self-control in intertemporal choice. Nat. Neurosci. 13, 538–539 (2010).

    CAS  PubMed  Google Scholar 

  44. 44.

    Knoch, D. et al. Studying the neurobiology of social interaction with transcranial direct current stimulation-the example of punishing unfairness. Cereb. Cortex 18, 1987–1990 (2008).

    PubMed  Google Scholar 

  45. 45.

    Kool, W., McGuire, J. T., Wang, G. J. & Botvinick, M. M. Neural and behavioral evidence for an intrinsic cost of self-control. PLoS ONE 8, e72626 (2013).

    CAS  PubMed Central  PubMed  Google Scholar 

  46. 46.

    Rudorf, S. & Hare, T. A. Interactions between dorsolateral and ventromedial prefrontal cortex underlie context-dependent stimulus valuation in goal-directed choice. J. Neurosci. 34, 15988–15996 (2014).

    CAS  PubMed Central  PubMed  Google Scholar 

  47. 47.

    Zysset, S. et al. The neural implementation of multi-attribute decision making: a parametric fMRI study with human subjects. NeuroImage 31, 1380–1388 (2006).

    PubMed  Google Scholar 

  48. 48.

    Kahnt, T., Heinzle, J., Park, S. Q. & Haynes, J. D. Decoding different roles for vmPFC and dlPFC in multi-attribute decision making. NeuroImage 56, 709–715 (2011).

    PubMed  Google Scholar 

  49. 49.

    Enax, L., Hu, Y., Trautner, P. & Weber, B. Nutrition labels influence value computation of food products in the ventromedial prefrontal cortex. Obesity 23, 786–792 (2015).

    PubMed  Google Scholar 

  50. 50.

    Luo, S., Ainslie, G., Pollini, D., Giragosian, L. & Monterosso, J. R. Moderators of the association between brain activation and farsighted choice. NeuroImage 59, 1469–1477 (2012).

    PubMed  Google Scholar 

  51. 51.

    Hare, T. A., Camerer, C. F. & Rangel, A. Self-control in decision-making involves modulation of the vmPFC valuation system. Science 324, 646–648 (2009).

    CAS  PubMed  Google Scholar 

  52. 52.

    Hare, T. A., Hakimi, S. & Rangel, A. Activity in dlPFC and its effective connectivity to vmPFC are associated with temporal discounting. Front. Neurosci. 8, 50 (2014).

    PubMed Central  PubMed  Google Scholar 

  53. 53.

    Maier, S. U., Makwana, A. B. & Hare, T. A. Acute stress impairs self-control in goal-directed choice by altering multiple functional connections within the brain’s decision circuits. Neuron 87, 621–631 (2015).

    CAS  PubMed  Google Scholar 

  54. 54.

    Marechal, M. A., Cohn, A., Ugazio, G. & Ruff, C. C. Increasing honesty in humans with noninvasive brain stimulation. Proc. Natl Acad. Sci. USA 114, 4360–4364 (2017).

    CAS  PubMed  Google Scholar 

  55. 55.

    Brevet-Aeby, C., Brunelin, J., Iceta, S., Padovan, C. & Poulet, E. Prefrontal cortex and impulsivity: interest of noninvasive brain stimulation. Neurosci. Biobehav. Rev. 71, 112–134 (2016).

    PubMed  Google Scholar 

  56. 56.

    Caplin, A. Measuring and modeling attention. Annu. Rev. Econ. 8, 379–403 (2016).

    Google Scholar 

  57. 57.

    Bhatia, S. Associations and the accumulation of preference. Psychol. Rev. 120, 522–543 (2013).

    PubMed  Google Scholar 

  58. 58.

    Johnson, E. J., Haubl, G. & Keinan, A. Aspects of endowment: a query theory of value construction. J. Exp. Psychol. Learn. Mem. Cogn. 33, 461–474 (2007).

    PubMed  Google Scholar 

  59. 59.

    Weber, E. U. & Johnson, E. J. Mindful judgment and decision making. Annu. Rev. Psychol. 60, 53–85 (2009).

    PubMed  Google Scholar 

  60. 60.

    Payne, J. W., Bettman, J. R. & Johnson, E. J. The Adaptive Decision Maker (Cambridge Univ. Press, 1993).

  61. 61.

    Aron, A. R. From reactive to proactive and selective control: developing a richer model for stopping inappropriate responses. Biol. Psychiatry 69, e55–e68 (2011).

    Google Scholar 

  62. 62.

    Castiglione, A., Wagner, J., Anderson, M. & Aron, A. R. Preventing a thought from coming to mind elicits increased right frontal beta just as stopping action does. Cereb. Cortex 29, 2160–2172 (2019).

    PubMed Central  PubMed  Google Scholar 

  63. 63.

    Sims, C. A. Implications of rational inattention. J. Monetary Econ. 50, 665–690 (2003).

    Google Scholar 

  64. 64.

    Caplin, A. & Dean, M. Revealed preference, rational inattention, and costly information acquisition. Am. Econ. Rev. 105, 2183–2203 (2015).

    Google Scholar 

  65. 65.

    Krajbich, I. & Dean, M. How can neuroscience inform economics? Curr. Opin. Behav. Sci. 5, 51–57 (2015).

    Google Scholar 

  66. 66.

    Giuliani, N. R. & Berkman, E. T. Craving is an affective state and its regulation can be understood in terms of the extended process model of emotion regulation. Psychol. Inq. 26, 48–53 (2015).

    PubMed Central  PubMed  Google Scholar 

  67. 67.

    Gross, J. J. Emotion regulation: current status and future prospects. Psychol. Inq. 26, 1–26 (2015).

    Google Scholar 

  68. 68.

    Eldar, E., Bae, G. J., Kurth-Nelson, Z., Dayan, P. & Dolan, R. J. Magnetoencephalography decoding reveals structural differences within integrative decision processes. Nat. Hum. Behav. 2, 670–681 (2018).

    Google Scholar 

  69. 69.

    Maier, S. U. Mastering Temptation: Behavioral, Neural and Physiological Investigations of Self-Control in Goal-Directed Choice. PhD thesis, Univ. of Zurich (2016).

  70. 70.

    Nitsche, M. A. & Paulus, W. Sustained excitability elevations induced by transcranial DC motor cortex stimulation in humans. Neurology 57, 1899–1901 (2001).

    CAS  PubMed  Google Scholar 

  71. 71.

    Fritsch, B. et al. Direct current stimulation promotes BDNF-dependent synaptic plasticity: potential implications for motor learning. Neuron 66, 198–204 (2010).

    CAS  PubMed Central  PubMed  Google Scholar 

  72. 72.

    Mullen, K., Ardia, D., Gil, D. L., Windover, D. & Cline, J. DEoptim: an R package for global optimization by differential evolution. J. Stat. Soft. 40, 1–26 (2011).

    Google Scholar 

  73. 73.

    Kruschke, J. K. Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan. 2nd edn (Academic Press, Elsevier, 2015).

  74. 74.

    Gneiting, T. & Raftery, A. E. Strictly proper scoring rules, prediction, and estimation. J. Am. Stat. Assoc. 102, 359–378 (2007).

    CAS  Google Scholar 

  75. 75.

    Kruschke, J. K. Bayesian estimation supersedes the t test. J. Exp. Psychol. Gen. 142, 573–603 (2013).

    PubMed  Google Scholar 

Download references

Acknowledgements

This study has been supported by Swiss National Science Foundation (SNSF) grants 320030_143443 and 32003B_166566 (to C.C.R. and T.A.H.), a Marie Curie International Incoming Fellowship PIIF-GA-2012-327196 (to A.R.B.) and EU FP7 grant 607310 (to T.A.H.). C.C.R. received support from the SNSF (grant no. 100019L_173248) and from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 725355, BRAINCODES). The funders had no role in the conceptualization, design, data collection, analysis, decision to publish or preparation of the manuscript. The authors thank A. Makwana and A. Cubillo for contributing code for the food choice and control tasks, J. Price for help with data collection and documentation, and G. Lombardi for help with data collection and documentation as well as useful discussions about analysis strategies and implementation.

Author information

Affiliations

Authors

Contributions

S.U.M., A.R.B., R.P., C.C.R. and T.A.H. designed one or more aspects of the research. S.U.M. and A.R.B. collected the novel data for the GFC and tDCS studies. R.P. and T.A.H. designed the tDDM with separate attribute consideration onset times. S.U.M., A.R.B., R.P. and T.A.H. analysed the data. S.U.M., A.R.B., R.P., C.C.R. and T.A.H. wrote the paper.

Corresponding authors

Correspondence to Silvia U. Maier or Anjali Raja Beharelle or Todd A. Hare.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Primary Handling Editor: Marike Schiffer

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Parameter recovery for the time-varying DDM.

Parameter recovery for the time-varying DDM with separate consideration onset times for tastiness and healthiness attributes. The plots in the first column show the distributions of all 272 generating and recovered relative weighting (a) and timing parameters (b). There was no significant difference between generating and recovered relative weighting (mean difference = 0.01, 95% HDI = [−0.36, 0.54], posterior probability of a difference > 0 = 0.662, Bayes factor = 0.140) or relative timing parameters (mean difference = −0.01, 95% HDI = [−0.03, 0.01], posterior probability of a difference > 0 = 0.105, Bayes factor = 0.024). The panels in the second column show the correlations between the generating and recovered relative weighting (c) and timing parameters (d). The red dotted line indicates the x = y identity line. Panel e) plots the error in relative weight recovery against the error in relative timing recovery. This plot shows that there is no significant correlation between the two types of error when fitting the model (r = − 0.1, 95% HDI = [−0.215; 0.018], posterior probability of observing a negative correlation = 0.95). The grey shaded area (panels c-e) signifies the 95% confidence interval.

Extended Data Fig. 2 Cumulative response time distributions for sDDM, tDDM and bDDM.

Cumulative distributions for response times by participant type, choice outcome and data source. Participants estimated to consider taste or health first are plotted in the top and bottom rows, respectively. Response times for choices in favour of (91) less healthy but more tasty (LH_MT), (2) more healthy but less tasty (MH_LT), or (3) both more healthy and more tasty (MH_MT) outcomes are shown in columns 1-3, respectively. Choices in favour of the option rated as less healthy and less tasty were rarely made (less than 5% of trials) and are omitted for clarity. Responses generated by human participants are shown in green lines. Responses generated by simulated agents using the best-fitting sDDM, tDDM, and bDDM parameters are shown in orange, purple, and magenta lines respectively. All three models can recreate the RT patterns in the empirical data equally well when choice outcomes align with the attribute participants consider first. However, the sDDM and bDDM both generate response times that are too fast relative to the empirical data when participants that consider taste first ultimately choose in favour of a more healthy, but less tasty option (row 1, column 2) or if participants that consider health first ultimately choose in favour of a less healthy, but more tasty option (row 2, column 1). In contrast, the tDDM is able to reproduce the observed response time distributions in these cases well.

Extended Data Fig. 3 Relative start time for all participants in each dataset.

Relative start times in seconds for healthiness compared to tastiness for all participants in each study. Positive values indicate that tastiness is considered before healthiness and negative values that healthiness is considered before tastiness. In each column every dot is a separate participant. The thick black horizontal bars represent within-study means and the rectangular bands indicate the 95% highest density intervals (HDIs). Dataset abbreviations: MRT = data from the computer-mouse response trials in Sullivan et al 2015; IAC = data from the natural choice condition in Hare et al 2011; GFC = newly collected data from the first session/day of an experiment combining gambles and food choices; TDCS = newly collected data from the pre-stimulation baseline choices in our tDCS experiment.

Supplementary information

Supplementary Information

Supplementary Results, Supplementary Discussion, Supplementary Methods, Supplementary Figs. 1 and 2, Supplementary Tables 1–28 and Supplementary References.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Maier, S.U., Raja Beharelle, A., Polanía, R. et al. Dissociable mechanisms govern when and how strongly reward attributes affect decisions. Nat Hum Behav 4, 949–963 (2020). https://doi.org/10.1038/s41562-020-0893-y

Download citation

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing