Skip to main content

Thank you for visiting You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Rational quantitative attribution of beliefs, desires and percepts in human mentalizing


Social cognition depends on our capacity for ‘mentalizing’, or explaining an agent’s behaviour in terms of their mental states. The development and neural substrates of mentalizing are well-studied, but its computational basis is only beginning to be probed. Here we present a model of core mentalizing computations: inferring jointly an actor’s beliefs, desires and percepts from how they move in the local spatial environment. Our Bayesian theory of mind (BToM) model is based on probabilistically inverting artificial-intelligence approaches to rational planning and state estimation, which extend classical expected-utility agent models to sequential actions in complex, partially observable domains. The model accurately captures the quantitative mental-state judgements of human participants in two experiments, each varying multiple stimulus dimensions across a large number of stimuli. Comparative model fits with both simpler ‘lesioned’ BToM models and a family of simpler non-mentalistic motion features reveal the value contributed by each component of our model.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.


All prices are NET prices.

Figure 1: Experimental scenario and model schema.
Figure 2: Example experimental stimuli.
Figure 3: The four factors varied in the factorial design of Experiment 1.
Figure 4: Experiment 1 results.
Figure 5: Comparing BToM and mean human (n=16) desire and belief inferences across all individual scenarios.
Figure 6: Experiment 2 results.
Figure 7: Experiment 2 results, comparing models and mean human (n=176) percept inferences across all individual scenarios.


  1. 1

    Knill, D. & Richards, W. Perception as Bayesian Inference (Cambridge Univ. Press, 1996).

    Book  Google Scholar 

  2. 2

    Marr, D . Vision (Freeman, 1982).

    Google Scholar 

  3. 3

    Weiss, Y., Simoncelli, E. P. & Adelson, E. H . Motion illusions as optimal percepts. Nat. Neurosci. 5, 598–604 (2002).

    CAS  Article  Google Scholar 

  4. 4

    Carey, S . The Origin of Concepts (Oxford Univ. Press, 2009).

    Book  Google Scholar 

  5. 5

    Csibra, G., Biró, S., Koós, O. & Gergely, G . One-year-old infants use teleological representations of actions productively. Cogn. Sci. 27, 111–133 (2003).

    Article  Google Scholar 

  6. 6

    Hamlin, J. K., Wynn, K. & Bloom, P. Social evaluation by preverbal infants. Nature 450, 557–560 (2007).

    CAS  Article  Google Scholar 

  7. 7

    Leslie, A. M., Friedman, O. & German, T. P . Core mechanisms in ‘theory of mind’. Trends Cogn. Sci. 8, 528–533 (2005).

    Article  Google Scholar 

  8. 8

    Onishi, K. H. & Baillargeon, R. Do 15-month-old infants understand false beliefs? Science 308, 255–258 (2005).

    CAS  Article  Google Scholar 

  9. 9

    Woodward, A. L. Infants selectively encode the goal object of an actor’s reach. Cognition 69, 1–34 (1998).

    CAS  Article  Google Scholar 

  10. 10

    Wellman, H. M. Making Minds: How Theory of Mind Develops (Oxford Univ. Press, 2014).

    Book  Google Scholar 

  11. 11

    Wimmer, H. & Perner, J . Beliefs about beliefs: representation and constraining function of wrong beliefs in young children’s understanding of deception. Cognition 13, 103–128 (1983).

    CAS  Article  Google Scholar 

  12. 12

    Battaglia, P. W., Hamrick, J. B. & Tenenbaum, J. B . Simulation as an engine of physical scene understanding. Proc. Natl Acad. Sci. USA 110, 18327–18332 (2013).

    CAS  Article  Google Scholar 

  13. 13

    Körding, K. P. & Wolpert, D. M. Bayesian integration in sensorimotor learning. Nature 427, 244–247 (2004).

    Article  Google Scholar 

  14. 14

    Baker, C. L., Saxe, R. & Tenenbaum, J. B. Action understanding as inverse planning. Cognition 113, 329–349 (2009).

    Article  Google Scholar 

  15. 15

    Jern, A. & Kemp, C. A decision network account of reasoning about other people’s choices. Cognition 142, 12–38 (2015).

    Article  Google Scholar 

  16. 16

    Jara-Ettinger, J., Gweon, H., Tenenbaum, J. B. & Schulz, L. E. Children’s understanding of the costs and rewards underlying rational action. Cognition 140, 14–23 (2015).

    Article  Google Scholar 

  17. 17

    Lucas, C. G. et al. The child as econometrician: a rational model of preference understanding in children. PLoS ONE 9, e92160 (2014).

    Article  Google Scholar 

  18. 18

    E. Oztop, D. Wolpert & M. Kawato . Mental state inference using visual control parameters. Cogn. Brain Res. 22, 129–151 (2005).

    Article  Google Scholar 

  19. 19

    Pantelis, P. C. et al. Inferring the intentional states of autonomous virtual agents. Cognition 130, 360–379 (2014).

    Article  Google Scholar 

  20. 20

    Blythe, P. W., Todd, P. M. & Miller, G. F. in Simple Heuristics that Make Us Smart (eds Gigerenzer G., Todd, P. M. & the ABC Research Group) 257–286 (Oxford Univ. Press, 1999).

    Google Scholar 

  21. 21

    Zacks, J. M. Using movement and intentions to understand simple events. Cogn. Sci. 28, 979–1008 (2004).

    Article  Google Scholar 

  22. 22

    Goodman, N. D., Baker, C. L. & Tenenbaum, J. B . Cause and intent: social reasoning in causal learning. In Proc. 31st Annu. Conf. Cognitive Science Society (eds Taatgen, N. & van Rijn, H.) 2759–2764 (Cognitive Science Society, 2009).

  23. 23

    Shafto, P., Goodman, N. D. & Frank, M. C . Learning from others: the consequences of psychological reasoning for human learning. Persp. Psychol. Sci. 7, 341–351 (2012).

    Article  Google Scholar 

  24. 24

    Rafferty A. N., LaMar M. M. & Griffiths, T. L. Inferring learners’ knowledge from their actions. Cogn. Sci. 39, 584–618 (2015).

    Article  Google Scholar 

  25. 25

    Hawthorne-Madell, D. & Goodman, N. D. So good it has to be true: wishful thinking in theory of mind. In Proc. 37th Annu. Conf. Cognitive Science Society (eds Noelle, D. C. et al.) 884–889 (Cognitive Science Society, 2015).

  26. 26

    Butterfield, J., Jenkins, O. C., Sobel, D. M. & Schwertfeger, J. Modeling aspects of theory of mind with Markov random fields. Int. J. Soc. Robot. 1, 41–51 (2009).

    Article  Google Scholar 

  27. 27

    Shafto, P., Eaves, B., Navarro, D. J. & Perfors, A . Epistemic trust: modeling children’s reasoning about others’ knowledge and intent. Dev. Sci. 15, 436–447 (2012).

    Article  Google Scholar 

  28. 28

    Gergely, G. & Csibra, G . Teleological reasoning in infancy: the naïve theory of rational action. Trends Cogn. Sci. 7, 287–292 (2003).

    Article  Google Scholar 

  29. 29

    Perner, J . Understanding the Representational Mind (MIT Press, 1991).

    Google Scholar 

  30. 30

    Kaelbling, L. P., Littman, M. L. & Cassandra, A. R . Planning and acting in partially observable stochastic domains. Artificial Intelligence 101, 99–134 (1998).

    Article  Google Scholar 

  31. 31

    von Neumann, J. & Morgenstern, O . Theory of Games and Economic Behavior (Princeton Univ. Press, 1953).

    Google Scholar 

  32. 32

    Gmytrasiewicz, P. J. & Doshi, P. A framework for sequential planning in multi-agent settings. J. Artif. Intell. Res. 24, 49–79 (2005).

    Article  Google Scholar 

  33. 33

    Davis, L. S. & Benedikt, M. L . Computational models of space: isovists and isovist fields. Comput. Graph. Image Process. 11, 49–72 (1979).

    Article  Google Scholar 

  34. 34

    Morariu, V. I., Prasad, V. S. N. & Davis L. S . Human activity understanding using visibility context. In IEEE/RSJ IROS Workshop: From Sensors to Human Spatial Concepts (FS2HSC) (2007).

  35. 35

    Tremoulet, P. D. & Feldman, J . The influence of spatial context and the role of intentionality in the interpretation of animacy from motion. Percept. Psychophys. 29, 943–951 (2006).

    Article  Google Scholar 

  36. 36

    Perner, J. & Ruffman, T. Infants’ insight into the mind: how deep? Science 308, 214–216 (2005).

    CAS  Article  Google Scholar 

  37. 37

    Csibra, G. & Volein, Á . Infants can infer the presence of hidden objects from referential gaze information. Br. J. Dev. Psychol. 26, 1–11 (2008).

    Article  Google Scholar 

  38. 38

    Moll, H. & Tomasello, M . 12- and 18-month-old infants follow gaze to spaces behind barriers. Dev. Sci. 7, F1–F9 (2004).

    Article  Google Scholar 

  39. 39

    Hsu, D., Lee, W. S. & Rong, N . What makes some POMDP problems easy to approximate? In Advances in Neural Information Processing Systems (NIPS 2007) Vol. 20 (eds Platt, J. C. ) (NIPS Foundation, 2007).

    Google Scholar 

  40. 40

    Silver, D. & Veness, J . Monte-Carlo planning in large POMDPs. In Advances in Neural Information Processing Systems (NIPS 2010) Vol. 23 (eds Lafferty, J. D. et al.) (NIPS Foundation, 2010).

    Google Scholar 

  41. 41

    Somani, A., Ye, N., Hsu, D. & Lee, W. S. Despot: online POMDP planning with regularization. In Advances in Neural Information Processing Systems (NIPS 2013) Vol. 26 (eds Burges, C. J. C. et al.) (NIPS Foundation, 2013).

    Google Scholar 

  42. 42

    Lovejoy, W. S . Computationally feasible bounds for partially observed Markov decision processes. Oper. Res. 39, 162–175 (1991).

    Article  Google Scholar 

  43. 43

    Kurniawati, H., Hsu, D. & Lee, W. S. SARSOP: efficient point-based POMDP planning by approximating optimally reachable belief spaces. In Proc. Robotics: Science and Systems Vol. 4 (eds Brock, O., Trinkle, J. & Ramos, F.) 65–72 (MIT, 2009).

    Google Scholar 

  44. 44

    Kulkarni, T. D., Kohli, P., Tenenbaum, J. B. & Mansinghka, V . Picture: an imperative probabilistic programming language for scene perception. In IEEE Conf. Computer Vision and Pattern Recognition 4390–4399 (Computer Vision Foundation, 2015).

  45. 45

    de Villiers, J. G. & de Villiers, P. A. in Language Acquisition (ed. Foster-Cohen, S. ) Ch. 7, 169–195 (Palgrave Macmillan, 2009).

    Book  Google Scholar 

  46. 46

    Baker, C. L., Goodman, N. D. & Tenenbaum, J. B . Theory-based social goal inference. In Proc. 30th Annu. Conf. Cognitive Science Society 1447–1455 (Cognitive Science Society, 2008).

  47. 47

    Hamlin, J. K., Ullman, T. D., Tenenbaum, J. B., Goodman, N. D. & Baker, C. L. The mentalistic basis of core social cognition: experiments in preverbal infants and a computational model. Dev. Sci. 16, 209–226 (2013).

    Article  Google Scholar 

  48. 48

    Littman, M. L . Markov games as a framework for multi-agent reinforcement learning. In Proc. 11th Int. Conf. Machine Learning Vol. 9166 (ed. Perner, P.) 157–163 (Springer, 1994).

  49. 49

    Yoshida, W., Dolan, R. J. & Friston, K. J. Game theory of mind. PLoS Comput. Biol. 4, 1–14 (2008).

    Article  Google Scholar 

  50. 50

    Doshi, P., Qu, X., Goodie, A. & Young, D. Modeling recursive reasoning by humans using empirically informed interactive POMDPs. In Proc. 9th Int. Conf. Autonomous Agents and Multiagent Systems (AAMAS) (International Foundation for Autonomous Agents and Multiagent Systems, 2010).

  51. 51

    Stuhlmüller, A. & Goodman, N. D . Reasoning about reasoning by nested conditioning: modeling theory of mind with probabilistic programs. J. Cogn. Syst. Res. 28, 80–99 (2013).

    Article  Google Scholar 

  52. 52

    Spelke, E. S. & Kinzler, K. D. Core knowledge. Dev. Sci. 10, 89–96 (2007).

    Article  Google Scholar 

  53. 53

    Kahneman, D . Thinking, Fast and Slow (Farrar, Straus and Giroux, 2011).

    Google Scholar 

  54. 54

    Ong, S. C. W., Png, S. W., Hsu, D. & Lee, W. S. POMDPs for robotic tasks with mixed observability. In Robotics: Science and Systems Vol. 5 (MIT Press, 2009).

    Google Scholar 

  55. 55

    Cohen, P. R. Empirical Methods in Artificial Intelligence (MIT Press, 1995).

    Google Scholar 

Download references


This work was supported by the Center for Brains, Minds & Machines (CBMM), under NSF STC award CCF-1231216; by NSF grant IIS-1227495 and by DARPA grant IIS-1227504. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information




C.L.B., R.S. and J.B.T. designed Experiment 1. C.L.B. ran Experiment 1, implemented the models and performed the analyses of Experiment 1. J.J.E., C.L.B. and J.B.T. designed Experiment 2. J.J.-E. and C.L.B. ran Experiment 2, implemented the models and performed the analyses of Experiment 2. C.L.B. and J.B.T. wrote the manuscript.

Corresponding author

Correspondence to Joshua B. Tenenbaum.

Ethics declarations

Competing interests

The authors declare no competing interests.

Supplementary information

Supplementary Information

Supplementary Methods, Supplementary Figures, Supplementary References. (PDF 731 kb)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Baker, C., Jara-Ettinger, J., Saxe, R. et al. Rational quantitative attribution of beliefs, desires and percepts in human mentalizing. Nat Hum Behav 1, 0064 (2017).

Download citation

Further reading


Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing