The nature and neural implementation of emotions is the subject of vigorous debate. Here, we use Bayesian decision theory to address key complexities in this field and conceptualize emotions in terms of their relationship to survival-relevant behavioural choices. Decision theory indicates which behaviours are optimal in a given situation; however, the calculations required are radically intractable. We therefore conjecture that the brain uses a range of pre-programmed algorithms that provide approximate solutions. These solutions seem to produce specific behavioural manifestations of emotions and can also be associated with core affective dimensions. We identify principles according to which these algorithms are implemented in the brain and illustrate our approach by considering decision making in the face of proximal threat.
Your institute does not have access to this article
Open Access articles citing this article.
Molecular Psychiatry Open Access 24 September 2021
Scientific Reports Open Access 25 September 2020
Scientific Reports Open Access 16 July 2020
Subscribe to Nature+
Get immediate online access to the entire Nature family of 50+ journals
Subscribe to Journal
Get full journal access for 1 year
only $4.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Darwin, C. The Expression of the Emotions in Man and Animals Vol. 1965 (Univ. of Chicago Press, 1872).
Russell, J. A. A circumplex model of affect. J. Pers Soc. Psychol. 39, 1161–1178 (1980).
Kuppens, P., Tuerlinckx, F., Russell, J. A. & Barrett, L. F. The relation between valence and arousal in subjective experience. Psychol. Bull. 139, 917–940 (2013).
Scherer, K. R., Schorr, A. & Johnstone, T. Appraisal Processes in Emotion: Theory, Methods, Research (Oxford Univ. Press, 2001).
Oatley, K. & Johnson-Laird, P. N. Cognitive approaches to emotions. Trends Cogn. Sci. 18, 134–140 (2014).
Ekman, P. & Oster, H. Facial expressions of emotion. Annu. Rev. Psychol. 30, 527–554 (1979).
Izard, C. E. Basic emotions, relations among emotions, and emotion–cognition relations. Psychol. Rev. 99, 561–565 (1992).
Calhoon, G. G. & Tye, K. M. Resolving the neural circuits of anxiety. Nat. Neurosci. 18, 1394–1404 (2015).
Burgdorf, J. & Panksepp, J. The neurobiology of positive emotions. Neurosci. Biobehav. Rev. 30, 173–187 (2006).
Herry, C. & Johansen, J. P. Encoding of fear learning and memory in distributed neuronal circuits. Nat. Neurosci. 17, 1644–1654 (2014).
Stephan, K. E. et al. Charting the landscape of priority problems in psychiatry, part 1: classification and diagnosis. Lancet Psychiatry 3, 77–83 (2016).
LeDoux, J. E. Coming to terms with fear. Proc. Natl Acad. Sci. USA 111, 2871–2878 (2014).
Blanchard, D. C., Griebel, G., Pobbe, R. & Blanchard, R. J. Risk assessment as an evolved threat detection and analysis process. Neurosci. Biobehav. Rev. 35, 991–998 (2011).
Scherer, K. R. in Appraisal Processes in Emotion (eds Scherer, K. R., Schorr, A. & Johnstone, T.) 92–120 (Oxford Univ. Press, 2001).
Wilensky, A. E., Schafe, G. E., Kristensen, M. P. & LeDoux, J. E. Rethinking the fear circuit: the central nucleus of the amygdala is required for the acquisition, consolidation, and expression of Pavlovian fear conditioning. J. Neurosci. 26, 12387–12396 (2006).
Dayan, P., Niv, Y., Seymour, B. & Daw, N. D. The misbehavior of value and the discipline of the will. Neural Netw. 19, 1153–1160 (2006).
Daw, N. D., Niv, Y. & Dayan, P. Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8, 1704–1711 (2005).
Lindquist, K. A. & Barrett, L. F. A functional architecture of the human brain: emerging insights from the science of emotion. Trends Cogn. Sci. 16, 533–540 (2012).
Jack, R. E., Garrod, O. G. & Schyns, P. G. Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr. Biol. 24, 187–192 (2014).
Nowlis, H. H. & Nowlis, V. The description and analysis of mood. Ann. NY Acad. Sci. 65, 345–355 (1956).
Marr, D. C. & Poggio, T. From understanding computation to understanding neural circuitry. Neurosci. Res. Program Bull. 15, 470–491 (1977).
Kording, K. P. & Wolpert, D. M. Bayesian decision theory in sensorimotor control. Trends Cogn. Sci. 10, 319–326 (2006).
Bach, D. R. & Dolan, R. J. Knowing how much you don't know: a neural organization of uncertainty estimates. Nat. Rev. Neurosci. 13, 572–586 (2012).
Pouget, A., Beck, J. M., Ma, W. J. & Latham, P. E. Probabilistic brains: knowns and unknowns. Nat. Neurosci. 16, 1170–1178 (2013).
Graebner, A. K., Iyer, M. & Carter, M. E. Understanding how discrete populations of hypothalamic neurons orchestrate complicated behavioral states. Front. Syst. Neurosci. 9, 111 (2015).
Berridge, K. C. & Kringelbach, M. L. Pleasure systems in the brain. Neuron 86, 646–664 (2015).
Samuelson, P. A. Note on the pure theory of consumers' behaviour. Economica 5, 61–71 (1938).
Blondel, V. D. & Tsitsiklis, J. N. A survey of computational complexity results in systems and control. Automatica 36, 1249–1274 (2000).
Hinton, G. E. & Nowlan, S. J. How learning can guide evolution. Complex Syst. 1, 495–502 (1987).
Niv, Y., Daw, N. D., Joel, D. & Dayan, P. Tonic dopamine: opportunity costs and the control of response vigor. Psychopharmacology 191, 507–520 (2007).
Choi, J. E., Vaswani, P. A. & Shadmehr, R. Vigor of movements and the cost of time in decision making. J. Neurosci. 34, 1212–1223 (2014).
Dayan, P. Instrumental vigour in punishment and reward. Eur. J. Neurosci. 35, 1152–1168 (2012).
Mowrer, O. H. 2-Factor learning theory — summary and comment. Psychol. Rev. 58, 350–354 (1951).
Maia, T. V. Two-factor theory, the actor–critic model, and conditioned avoidance. Learn. Behav. 38, 50–67 (2010).
Lloyd, K. & Dayan, P. Safety out of control: dopamine and defence. Behav. Brain Funct. 12, 15 (2016).
Trimmer, P. C., Paul, E. S., Mendl, M. T., McNamara, J. M. & Houston, A. I. On the evolution and optimality of mood States. Behav. Sci. (Basel) 3, 501–521 (2013).
Bach, D. R. Anxiety-like behavioural inhibition is normative under environmental threat-reward correlations. PLoS Comput. Biol. 11, e1004646 (2015).
Mackintosh, N. J. Conditioning and Associative Learning (Oxford Univ. Press, 1983).
Dickinson, A. Contemporary Animal Learning Theory Vol. 1 (CUP Archive, 1980).
Dickinson, A. & Balleine, B. in Steven's Handbook of Experimental Psychology: Learning, Motivation and Emotion Vol. 3 (eds Pashler, H. & Gallistel, R.) 497–534 (Wiley, 2002).
Doya, K. What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? Neural Netw. 12, 961–974 (1999).
Adams, C. D. & Dickinson, A. in Information Processing in Animals: Memory Mechanisms (eds Spear, N. E. & Miller, R. E.) 143–165 (Psychology Press,1981).
Yeomans, J. S., Li, L., Scott, B. W. & Frankland, P. W. Tactile, acoustic and vestibular systems sum to elicit the startle reflex. Neurosci. Biobehav. Rev. 26, 1–11 (2002).
Wilson, R. C. & Niv, Y. Inferring relevance in a changing world. Front. Hum. Neurosci. 5, 189 (2011).
Garcia, J., McGowan, B. K., Ervin, F. R. & Koelling, R. A. Cues: their relative effectiveness as a function of the reinforcer. Science 160, 794–795 (1968).
Rozin, P., Gruss, L. & Berk, G. Reversal of innate aversions: attempts to induce a preference for chili peppers in rats. J. Comp. Physiol. Psychol. 93, 1001–1014 (1979).
Cain, C. K. & LeDoux, J. E. Escape from fear: a detailed behavioral analysis of two atypical responses reinforced by CS termination. J. Exp. Psychol. Anim. Behav. Process 33, 451–463 (2007).
Bach, D. R. A cost minimisation and Bayesian inference model predicts startle reflex modulation across species. J. Theor. Biol. 370, 53–60 (2015).
Kehoe, E. J. & Macrae, M. in A Neuroscientist's Guide to Classical Conditioning 171–231 (Springer, 2002).
Blanchard, D. C. & Blanchard, R. J. Ethoexperimental approaches to the biology of emotion. Annu. Rev. Psychol. 39, 43–68 (1988).
McNaughton, N. & Corr, P. J. A two-dimensional neuropsychology of defense: fear/anxiety and defensive distance. Neurosci. Biobehav. Rev. 28, 285–305 (2004).
Gray, J. A. & McNaughton, N. The Neuropsychology of Anxiety: an Enquiry into the Functions of the Septohippocampal System Vol. 2 (Oxford Univ. Press, 2000).
Fanselow, M. S. & Lester, L. S. in Evolution and Learning (eds Bolles, R. C. & Beecher, M. D) 185–212 (Psychology Press, 1988).
Siddle, D. A. Orienting, habituation, and resource allocation: an associative analysis. Psychophysiology 28, 245–259 (1991).
Schutzwohl, A. Surprise and schema strength. J. Exp. Psychol. Learn. Mem. Cogn. 24, 1182–1199 (1998).
Hailman, J. P. The Ontogeny of an Instinct: the Pecking Response in Chicks of the Laughing Gull (Larus atricilla L.) and Related Species (Brill, 1967).
Tinbergen, N. The Study of Instinct (Oxford Univ. Press, 1951).
Byrne, R. W. & Byrne, J. M. E. Complex leaf-gathering skills of mountain gorillas (Gorilla g. beringei): variability and standardization. Am. J. Primatol. 31, 241–261 (1993).
Zhang, S., Mano, H., Ganesh, G., Robbins, T. & Seymour, B. Dissociable learning processes underlie human pain conditioning. Curr. Biol. 26, 52–58 (2016).
Gross, C. T. & Canteras, N. S. The many paths to fear. Nat. Rev. Neurosci. 13, 651–658 (2012).
LeDoux, J. E., Sakaguchi, A., Iwata, J. & Reis, D. J. Auditory emotional memories: establishment by projections from the medial geniculate nucleus to the posterior neostriatum and/or dorsal amygdala. Ann. NY Acad. Sci. 444, 463–464 (1985).
Kim, J. J. & Fanselow, M. S. Modality-specific retrograde amnesia of fear. Science 256, 675–677 (1992).
Ohman, A. & Mineka, S. Fears, phobias, and preparedness: toward an evolved module of fear and fear learning. Psychol. Rev. 108, 483–522 (2001).
Dolan, R. J. & Dayan, P. Goals and habits in the brain. Neuron 80, 312–325 (2013).
Korn, C. W. & Bach, D. R. Maintaining homeostasis by decision-making. PLoS Comput. Biol. 11, e1004301 (2015).
Rangel, A. Regulation of dietary choice by the decision-making circuitry. Nat. Neurosci. 16, 1717–1724 (2013).
Fonio, E., Benjamini, Y. & Golani, I. Freedom of movement and the stability of its unfolding in free exploration of mice. Proc. Natl Acad. Sci. USA 106, 21335–21340 (2009).
Bach, D. R. The cognitive architecture of anxiety-like behavioral inhibition. J. Exp. Psychol. Hum. Percept. Perform. 43, 18–29 (2017).
Alonso, R., Brocas, I. & Carrillo, J. D. Resource allocation in the brain. Rev. Econom. Studies 81, 501–534 (2014).
Dayan, P. & Berridge, K. C. Model-based and model-free Pavlovian reward learning: revaluation, revision, and revelation. Cogn. Affect Behav. Neurosci. 14, 473–492 (2014).
Tomkins, S. S. & McCarter, R. What and where are the primary affects? Some evidence for a theory. Percept. Mot. Skills 18, 119–158 (1964).
Ekman, P., Friesen, W. V. & Ellsworth, P. Emotion in the Human Face: Guide-Lines for Research and an Integration of Findings (Pergamon Press, 1972).
Ekman, P. Are there basic emotions? Psychol. Rev. 99, 550–553 (1992).
Brandao, M. L., Zanoveli, J. M., Ruiz-Martinez, R. C., Oliveira, L. C. & Landeira-Fernandez, J. Different patterns of freezing behavior organized in the periaqueductal gray of rats: association with different types of anxiety. Behav. Brain Res. 188, 1–13 (2008).
Keay, K. A., Clement, C. I., Owler, B., Depaulis, A. & Bandler, R. Convergence of deep somatic and visceral nociceptive information onto a discrete ventrolateral midbrain peraqueductal gray region. Neuroscience 61, 727–732 (1994).
Phillips, R. G. & LeDoux, J. E. Differential contribution of amygdala and hippocampus to cued and contextual fear conditioning. Behav. Neurosci. 106, 274–285 (1992).
McHugh, S. B., Deacon, R. M., Rawlins, J. N. & Bannerman, D. M. Amygdala and ventral hippocampus contribute differentially to mechanisms of fear and anxiety. Behav. Neurosci. 118, 63–78 (2004).
Bach, D. R. et al. Human hippocampus arbitrates approach–avoidance conflict. Curr. Biol. 24, 541–547 (2014).
Reynolds, S. M. & Berridge, K. C. Fear and feeding in the nucleus accumbens shell: rostrocaudal segregation of GABA-elicited defensive behavior versus eating behavior. J. Neurosci. 21, 3261–3270 (2001).
Reynolds, S. M. & Berridge, K. C. Positive and negative motivation in nucleus accumbens shell: bivalent rostrocaudal gradients for GABA-elicited eating, taste “liking”/“disliking” reactions, place preference/avoidance, and fear. J. Neurosci. 22, 7308–7320 (2002).
Reynolds, S. M. & Berridge, K. C. Emotional environments retune the valence of appetitive versus fearful functions in nucleus accumbens. Nat. Neurosci. 11, 423–425 (2008).
Sharpe, M. J. & Killcross, S. The prelimbic cortex uses higher-order cues to modulate both the acquisition and expression of conditioned fear. Front. Syst. Neurosci. 8, 235 (2014).
Ohl, F. W., Wetzel, W., Wagner, T., Rech, A. & Scheich, H. Bilateral ablation of auditory cortex in Mongolian gerbil affects discrimination of frequency modulated tones but not of pure tones. Learn. Mem. 6, 347–362 (1999).
Letzkus, J. J. et al. A disinhibitory microcircuit for associative fear learning in the auditory cortex. Nature 480, 331–335 (2011).
Winston, J. S., Gottfried, J. A., Kilner, J. M. & Dolan, R. J. Integrated neural representations of odor intensity and affective valence in human amygdala. J. Neurosci. 25, 8903–8907 (2005).
Lewis, P., Critchley, H., Rotshtein, P. & Dolan, R. Neural correlates of processing valence and arousal in affective words. Cereb. Cortex 17, 742–748 (2007).
Chikazoe, J., Lee, D. H., Kriegeskorte, N. & Anderson, A. K. Population coding of affect across stimuli, modalities and individuals. Nat. Neurosci. 17, 1114–1122 (2014).
Lindquist, K. A., Satpute, A. B., Wager, T. D., Weber, J. & Barrett, L. F. The brain basis of positive and negative affect: evidence from a meta-analysis of the human neuroimaging literature. Cereb. Cortex 26, 1910–1922 (2016).
Critchley, H. D. & Rolls, E. T. Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. J. Neurophysiol. 75, 1673–1686 (1996).
Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Killcross, S. & Coutureau, E. Coordination of actions and habits in the medial prefrontal cortex of rats. Cereb. Cortex 13, 400–408 (2003).
Balleine, B. W. Neural bases of food-seeking: affect, arousal and reward in corticostriatolimbic circuits. Physiol. Behav. 86, 717–730 (2005).
Bassareo, V. & Di Chiara, G. Differential responsiveness of dopamine transmission to food-stimuli in nucleus accumbens shell/core compartments. Neuroscience 89, 637–641 (1999).
Corbit, L. H. & Balleine, B. W. The general and outcome-specific forms of Pavlovian-instrumental transfer are differentially mediated by the nucleus accumbens core and shell. J. Neurosci. 31, 11786–11794 (2011).
Liljeholm, M. & O'Doherty, J. P. Contributions of the striatum to learning, motivation, and performance: an associative account. Trends Cogn. Sci. 16, 467–475 (2012).
Amat, J., Paul, E., Zarza, C., Watkins, L. R. & Maier, S. F. Previous experience with behavioral control over stress blocks the behavioral and dorsal raphe nucleus activating effects of later uncontrollable stress: role of the ventral medial prefrontal cortex. J. Neurosci. 26, 13264–13272 (2006).
Dayan, P. Twenty-five lessons from computational neuromodulation. Neuron 76, 240–256 (2012).
Fallon, J. H. Topographic organization of ascending dopaminergic projections. Ann. NY Acad. Sci. 537, 1–9 (1988).
Hale, M. W. & Lowry, C. A. Functional topography of midbrain and pontine serotonergic systems: implications for synaptic regulation of serotonergic circuits. Psychopharmacology 213, 243–264 (2011).
Berridge, C. W. & Waterhouse, B. D. The locus coeruleus-noradrenergic system: modulation of behavioral state and state-dependent cognitive processes. Brain Res. Brain Res. Rev. 42, 33–84 (2003).
Liang, K. C. et al. Corticotropin-releasing factor: long-lasting facilitation of the acoustic startle reflex. J. Neurosci. 12, 2303–2312 (1992).
Cavanagh, J. F., Eisenberg, I., Guitart-Masip, M., Huys, Q. & Frank, M. J. Frontal theta overrides pavlovian learning biases. J. Neurosci. 33, 8541–8548 (2013).
Tovote, P., Fadok, J. P. & Luthi, A. Neuronal circuits for fear and anxiety. Nat. Rev. Neurosci. 16, 317–331 (2015).
Likhtik, E. et al. Prefrontal entrainment of amygdala activity signals safety in learned fear and innate anxiety. Nat. Neurosci. 17, 106–113 (2014).
Adhikari, A., Topiwala, M. A. & Gordon, J. A. Synchronized activity between the ventral hippocampus and the medial prefrontal cortex during anxiety. Neuron 65, 257–269 (2010).
Adhikari, A., Topiwala, M. A. & Gordon, J. A. Single units in the medial prefrontal cortex with anxiety-related firing patterns are preferentially influenced by ventral hippocampal activity. Neuron 71, 898–910 (2011).
Roseman, I. J. & Smith, C. A. in Appraisal Processes in Emotion Ch. 1 (eds Scherer, K. R., Schorr, A. & Johnstone, T.) 3–19 (Oxford Univ. Press, 2001).
Frijda, N. H. The Laws of Emotion (Lawrence Erlbaum Associates, 2007).
Dolan, R. J. Emotion, cognition, and behavior. Science 298, 1191–1194 (2002).
Mendl, M., Burman, O. H. & Paul, E. S. An integrative and functional framework for the study of animal emotion and mood. Proc. Biol. Sci. 277, 2895–2904 (2010).
Eldar, E., Rutledge, R. B., Dolan, R. J. & Niv, Y. Mood as representation of momentum. Trends Cogn. Sci. 20, 15–24 (2016).
DeWall, C. N., Baumeister, R. F., Chester, D. S. & Bushman, B. J. How often does currently felt emotion predict social behavior and judgment? A meta-analytic test of two theories. Emot. Rev. 8, 136–143 (2016).
Huys, Q. J. et al. Bonsai trees in your head: how the pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol. 8, e1002410 (2012).
Redish, A. D. Vicarious trial and error. Nat. Rev. Neurosci. 17, 147–159 (2016).
Loewenstein, G. & Lerner, J. S. in Handbook of Affective Sciences (eds Davidson, R. J., Scherer, K. R. & Goldsmith, H. H.) (Oxford University Press, 2003).
Pessoa, L. & Adolphs, R. Emotion processing and the amygdala: from a 'low road' to 'many roads' of evaluating biological significance. Nat. Rev. Neurosci. 11, 773–783 (2010).
Pessoa, L. On the relationship between emotion and cognition. Nat. Rev. Neurosci. 9, 148–158 (2008).
Barrett, L. F. The theory of constructed emotion: an active inference account of interoception and categorization. Soc. Cogn. Affect Neurosci. 12, 1–23 (2016).
Berger, J. O. Statistical Decision Theory and Bayesian Analysis (Springer Science & Business Media, 2013).
Sutton, R. S. & Barto, A. G. Reinforcement Learning: an Introduction (MIT Press, 1998).
Bertsekas, D. P. Dynamic Programming and Optimal Control Vol. 1 (Athena Scientific Belmont, 2005).
Schwartz, B. & Williams, D. R. The role of the response-reinforcer contingency in negative automaintenance. J. Exp. Anal. Behav. 17, 351–357 (1972).
Breland, K. & Breland, M. The misbehavior of organisms. Am. Psychol. 16, 681–684 (1961).
Dickinson, A. & Balleine, B. Motivational control of goal-directed action. Anim. Learn. Behav. 22, 1–18 (1994).
Boureau, Y. L., Sokol-Hessner, P. & Daw, N. D. Deciding how to decide: self-control and meta-decision making. Trends Cogn. Sci. 19, 700–710 (2015).
Dayan, P. How to set the switches on this thing. Curr. Opin. Neurobiol. 22, 1068–1074 (2012).
Alexander, W. H. & Brown, J. W. Computational models of performance monitoring and cognitive control. Top. Cogn. Sci. 2, 658–677 (2010).
Botvinick, M. & Weinstein, A. Model-based hierarchical reinforcement learning and human action control. Phil. Trans. R. Soc. B Biol. Sci. 369, 20130480 (2014).
Keramati, M., Dezfouli, A. & Piray, P. Speed/accuracy trade-off between the habitual and the goal-directed processes. PLoS Comput. Biol. 7, e1002055 (2011).
Pezzulo, G., Rigoli, F. & Chersi, F. The mixed instrumental controller: using value of information to combine habitual choice and mental simulation. Front. Psychol. 4, 92 (2013).
Kahneman, D. Thinking, Fast and Slow (Farrar, 2011).
Camerer, C. Behavioral Game Theory: Experiments in Strategic Interaction (Princeton Univ. Press, 2003).
Devaine, M., Hollard, G. & Daunizeau, J. Theory of mind: did evolution fool us? PLoS ONE 9, e87619 (2014).
Johnson, D. D. & Fowler, J. H. The evolution of overconfidence. Nature 477, 317–320 (2011).
Giske, J. et al. Effects of the emotion system on adaptive behavior. Am. Nat. 182, 689–703 (2013).
Seth, A. K. Interoceptive inference, emotion, and the embodied self. Trends Cogn. Sci. 17, 565–573 (2013).
The authors thank G. Castegnetti, L. Harris, C. Korn, M. Mendl, H. Nakahara, L. Paul, the reviewers and many others for inspiring discussions during the writing of this article. This work was supported by the University of Zurich (to D.R.B.), the Gatsby Charitable Foundation (to P.D.) and a grant from the UK National Centre for the Replacement Refinement and Reduction of Animals in Research (K/00008X/1; to M. Mendl, E. Paul and P.D.). The Wellcome Trust Centre for Neuroimaging is supported by core funding from the Wellcome Trust (091593/Z/10/Z).
The authors declare no competing financial interests.
- Decision theory
A computational-level theory for making choices given information about states and resulting utilities. Bayesian decision theory is a formally optimal (normative) decision theory.
In this article, an algorithm denotes an abstract, self-contained set of operations or effective procedures that maps sensory input and internal state to external and internal actions.
In this article, a controller corresponds to a realized neural circuit that is capable of implementing one or several algorithms for choosing or emitting actions.
- Constructionist approaches
A family of theoretical approaches that view subjectively experienced mental categories (such as feelings) as constructed representations of more-basic psychological operations, which are not consciously accessible.
- Utility functions
A real utility function quantifies how useful or dangerous certain outcomes are to an agent, in a given situation, and is realized in the output of actual neural circuits. A virtual utility function is an as-if construct that provides quantifications that are consistent with behavioural choices, but without necessarily underlying those choices.
Choice consistency, or independence, denotes that if A is preferred over B, then A + C is preferred over B + C, irrespective of what C is. This is a fundamental component of expected utility theory and of revealed choice theory.
Assuming that A is preferred over B and that B is preferred over C, these preferences are said to be transitive if A is also preferred over C. This is a fundamental component of expected utility theory and of revealed choice theory.
In this article, pre-programming refers to any restriction on the workings of a controller that can be cast in Bayesian decision theory terms as an immutable, prior mapping of state or prediction to action, or utility function.
- Action contingency
The causal relationship between the execution of actions and the outcomes that result.
In this article, the term Pavlovian is used to denote an algorithm or a controller making a choice of actions that is insensitive to the actual consequences of those actions. Here, the term is not used to denote design characteristics of experiments (as is sometimes the case).
In this article, the term instrumental refers to an algorithm or a controller making choices that are contingent on their past or predicted future consequences. Here, the term does not refer to design characteristics of experiments.
In this article, the term model-based is used to characterize algorithms that exploit a model of the structure of the environment and the outcomes that it affords to make long-run predictions about the future. Predictions need not be action contingent and thus can support either Pavlovian or instrumental controllers.
In this article, the term model-free is used to describe algorithms that learn to make long-run predictions by caching or saving experiences from the past, generally by enforcing self-consistency in successive outputs. Predictions are typically scalar, for instance, of summed future value and consequently do not encode the specific outcomes underpinning those values. Model-free predictions need not be action contingent and thus can support either Pavlovian or instrumental controllers.
- Appraisal theory
A family of emotion theories, all of which posit that manifestations of emotions (feelings, motivational processes, bodily reactions, and so on) are the output of a set of cognitive appraisals or encompass such appraisals. Theories differ widely according to the appraisals that they consider part of the set.
About this article
Cite this article
Bach, D., Dayan, P. Algorithms for survival: a comparative perspective on emotions. Nat Rev Neurosci 18, 311–319 (2017). https://doi.org/10.1038/nrn.2017.35
Nature Reviews Neuroscience (2022)
Molecular Psychiatry (2022)
Nature Human Behaviour (2020)
Scientific Reports (2020)