Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

Probabilistic brains: knowns and unknowns

Subjects

Abstract

There is strong behavioral and physiological evidence that the brain both represents probability distributions and performs probabilistic inference. Computational neuroscientists have started to shed light on how these probabilistic representations and computations might be implemented in neural circuits. One particularly appealing aspect of these theories is their generality: they can be used to model a wide range of tasks, from sensory processing to high-level cognition. To date, however, these theories have only been applied to very simple tasks. Here we discuss the challenges that will emerge as researchers start focusing their efforts on real-life computations, with a focus on probabilistic learning, structural learning and approximate inference.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: The visuo-haptic multisensory experiment of Ernst and Banks11.
Figure 2: Probabilistic population code using a basis function decomposition of the log probability.
Figure 3: Taking a product of likelihood functions with probabilistic population codes.
Figure 4: Neural network for Chinese character identification.
Figure 5: Incremental structural learning.

Similar content being viewed by others

References

  1. Van Horn, K.S. Constructing a logic of plausible inference: a guide to Cox's theorem. Int. J. Approx. Reason. 34, 3–24 (2003).

    Article  Google Scholar 

  2. De Finetti, B., Machi, A. & Smith, A. Theory of Probability: a Critical Introductory Treatment (Wiley, New York, 1993).

  3. Bayes, T. An essay towards solving a problem in the doctrine of chances. Philos. Trans. R. Soc. Lond. 53, 370–418 (1763).

    Article  Google Scholar 

  4. Laplace, P.S. Theorie Analytique des Probabilites (Ve Courcier, Paris, 1812).

  5. Stigler, S.M. Stigler's law of eponymy. Trans. N. Y. Acad. Sci. 39, 147–158 (1980).

    Article  Google Scholar 

  6. Mach, E. Contributions to the Analysis of the Sensations (Open Court Pub., 1897).

  7. Helmholtz, H.v. Versuch einer erweiterten Anwendung des Fechnerschen Gesetzes im Farbensystem. Z. Psychol. Physiol. Sinnesorgane 2, 1–30 (1891).

    Google Scholar 

  8. Knill, D.C. & Richards, W. Perception as Bayesian Inference (Cambridge University Press, New York, 1996).

  9. van Beers, R.J., Sittig, A.C. & Gon, J.J. Integration of proprioceptive and visual position-information: an experimentally supported model. J. Neurophysiol. 81, 1355–1364 (1999).

    Article  CAS  PubMed  Google Scholar 

  10. Knill, D.C. Surface orientation from texture: ideal observers, generic observers and the information content of texture cues. Vision Res. 38, 1655–1682 (1998).

    Article  CAS  PubMed  Google Scholar 

  11. Ernst, M.O. & Banks, M.S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433 (2002).

    Article  CAS  PubMed  Google Scholar 

  12. Jacobs, R.A. Optimal integration of texture and motion cues to depth. Vision Res. 117, 3621–3629 (1999).

    Article  Google Scholar 

  13. Wolpert, D.M., Ghahramani, Z. & Jordan, M. An internal model for sensorimotor integration. Science 269, 1880–1882 (1995).

    Article  CAS  PubMed  Google Scholar 

  14. Todorov, E. Optimality principles in sensorimotor control. Nat. Neurosci. 7, 907–915 (2004).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Körding, K.P. & Wolpert, D.M. Bayesian integration in sensorimotor learning. Nature 427, 244–247 (2004).

    Article  PubMed  CAS  Google Scholar 

  16. Chater, N., Tenenbaum, J.B. & Yuille, A. Probabilistic models of cognition: conceptual foundations. Trends Cogn. Sci. 10, 287–291 (2006).

    Article  PubMed  Google Scholar 

  17. Gopnik, A. et al. A theory of causal learning in children: causal maps and Bayes nets. Psychol. Rev. 111, 3–32 (2004).

    Article  PubMed  Google Scholar 

  18. Tenenbaum, J.B., Griffiths, T.L. & Kemp, C. Theory-based Bayesian models of inductive learning and reasoning. Trends Cogn. Sci. 10, 309–318 (2006).

    Article  PubMed  Google Scholar 

  19. Tenenbaum, J.B. & Griffiths, T.L. Theory-based causal inference. in Advances in Neural Information Processing Systems (eds. Becker, S., Thrun, S. & Obermayer, K.) 35–42 (MIT Press, 2003).

  20. Steyvers, M., Griffiths, T.L. & Dennis, S. Probabilistic inference in human semantic memory. Trends Cogn. Sci. 10, 327–334 (2006).

    Article  PubMed  Google Scholar 

  21. Jurafsky, D. A probabilistic model of lexical and syntactic access and disambiguation. Cogn. Sci. 20, 137–194 (1996).

    Article  Google Scholar 

  22. Levy, R. & Jaeger, T.F. Speakers optimize information density through syntactic reduction. in Advances in Neural Information Processing Systems (eds. Schlökopf, B., Platt, J.C. & Hofmann, T.) 849–856 (MIT Press, 2007).

  23. Tenenbaum, J.B., Kemp, C., Griffiths, T.L. & Goodman, N.D. How to grow a mind: statistics, structure and abstraction. Science 331, 1279–1285 (2011).

    Article  CAS  PubMed  Google Scholar 

  24. van Beers, R.J., Sittig, A.C. & Denier van der Gon, J.J. How humans combine simultaneous proprioceptive and visual position information. Exp. Brain Res. 111, 253–261 (1996).

    Article  CAS  PubMed  Google Scholar 

  25. Alais, D. & Burr, D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257–262 (2004).

    Article  CAS  PubMed  Google Scholar 

  26. Ratcliff, R. & Rouder, J.N. Modeling response times for two-choice decisions. Psychol. Sci. 9, 347–356 (1998).

    Article  Google Scholar 

  27. Mazurek, M.E., Roitman, J.D., Ditterich, J. & Shadlen, M.N. A role for neural integrators in perceptual decision making. Cereb. Cortex 13, 1257–1269 (2003).

    Article  PubMed  Google Scholar 

  28. Krajbich, I., Armel, C. & Rangel, A. Visual fixations and the computation and comparison of value in simple choice. Nat. Neurosci. 13, 1292–1298 (2010).

    Article  CAS  PubMed  Google Scholar 

  29. Kappen, H.J., Gómez, V. & Opper, M. Optimal control as a graphical model inference problem. Mach. Learn. 87, 159–182 (2012).

    Article  Google Scholar 

  30. Todorov, E. General duality between optimal control and estimation. in 47th IEEE Conference on Decision and Control 4286–4292 (2008).

  31. Barlow, H.B. Pattern recognition and the responses of sensory neurons. Ann. NY Acad. Sci. 156, 872–881 (1969).

    Article  CAS  PubMed  Google Scholar 

  32. Koechlin, E., Anton, J.L. & Burnod, Y. Bayesian inference in populations of cortical neurons: a model of motion integration and segmentation in area MT. Biol. Cybern. 80, 25–44 (1999).

    Article  CAS  PubMed  Google Scholar 

  33. Anastasio, T.J., Patton, P.E. & Belkacem-Boussaid, K. Using Bayes' rule to model multisensory enhancement in the superior colliculus. Neural Comput. 12, 1165–1187 (2000).

    Article  CAS  PubMed  Google Scholar 

  34. Hoyer, P.O. & Hyvarinen, A. Interpreting neural response variability as Monte Carlo sampling of the posterior. in Neural Informatoin Processing Systems (eds. Becker, S., Thrun, S. & Obermayer, K.) 293–300 (MIT Press, 2003).

  35. Paulin, M.G. Evolution of the cerebellum as a neuronal machine for Bayesian state estimation. J. Neural Eng. 2, S219–S234 (2005).

    Article  CAS  PubMed  Google Scholar 

  36. Lee, T.S. & Mumford, D. Hierarchical Bayesian inference in the visual cortex. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 1434–1448 (2003).

    Article  PubMed  Google Scholar 

  37. Achler, T. & Amir, E. Input feedback networks: classification and inference based on network structure. Proc. Artificial General Intelligence 1, 15–26 (2008).

    Google Scholar 

  38. Rao, R.P. Bayesian computation in recurrent neural circuits. Neural Comput. 16, 1–38 (2004).

    Article  PubMed  Google Scholar 

  39. Jazayeri, M. & Movshon, J.A. Optimal representation of sensory information by neural populations. Nat. Neurosci. 9, 690–696 (2006).

    Article  CAS  PubMed  Google Scholar 

  40. Denève, S., Duhamel, J.R. & Pouget, A. Optimal sensorimotor integration in recurrent cortical networks: a neural implementation of Kalman filters. J. Neurosci. 27, 5744–5756 (2007).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  41. Beck, J.M. & Pouget, A. Exact inferences in a neural implementation of a hidden Markov model. Neural Comput. 19, 1344–1361 (2007).

    Article  PubMed  Google Scholar 

  42. Bogacz, R. & Gurney, K. The basal ganglia and cortex implement optimal decision making between alternative actions. Neural Comput. 19, 442–477 (2007).

    Article  PubMed  Google Scholar 

  43. Gold, J.I. & Shadlen, M.N. Neural computations that underlie decisions about sensory stimuli. Trends Cogn. Sci. 5, 10–16 (2001).

    Article  PubMed  Google Scholar 

  44. Anderson, C. Neurobiological computational systems. in Computational Intelligence: Imitating Life (eds. Marks, R.J., Zurada, J.M. & Robinson, C.J.) 213–222 (IEEE Press, New York, 1994).

  45. Zemel, R.S., Dayan, P. & Pouget, A. Probabilistic interpretation of population code. Neural Comput. 10, 403–430 (1998).

    Article  CAS  PubMed  Google Scholar 

  46. Poggio, T. A theory of how the brain might work. Cold Spring Harb. Symp. Quant. Biol. 55, 899–910 (1990).

    Article  CAS  PubMed  Google Scholar 

  47. Ma, W.J., Beck, J.M., Latham, P.E. & Pouget, A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432–1438 (2006).

    Article  CAS  PubMed  Google Scholar 

  48. Huys, Q.J., Zemel, R.S., Natarajan, R. & Dayan, P. Fast population coding. Neural Comput. 19, 404–441 (2007).

    Article  PubMed  Google Scholar 

  49. Sanger, T.D. Probability density estimation for the interpretation of neural population codes. J. Neurophysiol. 76, 2790–2793 (1996).

    Article  CAS  PubMed  Google Scholar 

  50. Foldiak, P. The 'ideal homunculus': statistical inference from neural population responses. in Computation and Neural Systems (eds. Eeckman, F. & Bower, J.) 55–60 (Kluwer Academic Publishers, Norwell, Massachusetts, USA, 1993).

  51. Graf, A.B., Kohn, A., Jazayeri, M. & Movshon, J.A. Decoding the activity of neuronal populations in macaque primary visual cortex. Nat. Neurosci. 14, 239–245 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Berens, P. et al. A fast and simple population code for orientation in primate V1. J. Neurosci. 32, 10618–10626 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Fiser, J., Berkes, P., Orban, G. & Lengyel, M. Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn. Sci. 14, 119–130 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  54. Moreno-Bote, R., Knill, D.C. & Pouget, A. Bayesian sampling in visual perception. Proc. Natl. Acad. Sci. USA 108, 12491–12496 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  55. Fetsch, C.R., Pouget, A., Deangelis, G.C. & Angelaki, D.E. Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 15, 146–154 (2012).

    Article  CAS  Google Scholar 

  56. Beck, J.M. et al. Bayesian decision making with probabilistic population codes. Neuron 60, 1142–1152 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Churchland, A.K. et al. Variance as a signature of neural computations during decision making. Neuron 69, 818–831 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  58. Beck, J.M., Latham, P.E. & Pouget, A. Marginalization in neural circuits with divisive normalization. J. Neurosci. 31, 15310–15319 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  59. Ma, W.J., Navalpakkam, V., Beck, J.M., Berg, R. & Pouget, A. Behavior and neural basis of near-optimal visual search. Nat. Neurosci. 14, 783–790 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  60. Beck, J., Heller, K. & Pouget, A. Complex inference in neural circuits with probabilistic population codes and topic models. in Advances in Neural Information Processing Systems (ed. Bartlett, P.) 3068–3076 (MIT Press, 2012).

  61. Deneve, S., Latham, P.E. & Pouget, A. Reading population codes: a neural implementation of ideal observers. Nat. Neurosci. 2, 740–745 (1999).

    Article  CAS  PubMed  Google Scholar 

  62. Deneve, S., Latham, P.E. & Pouget, A. Efficient computation and cue integration with noisy population codes. Nat. Neurosci. 4, 826–831 (2001).

    Article  CAS  PubMed  Google Scholar 

  63. Eliasmith, C. & Anderson, C.H. Neural Engineering: Computation, Representation and Dynamics in Neurobiological Systems (MIT Press, 2003).

  64. Barber, M.J., Clark, J.W. & Anderson, C.H. Neural representation of probabilistic information. Neural Comput. 15, 1843–1864 (2003).

    Article  CAS  PubMed  Google Scholar 

  65. Anderson, J.S., Lampl, I., Gillespie, D.C. & Ferster, D. The contribution of noise to contrast invariance of orientation tuning in cat visual cortex. Science 290, 1968–1972 (2000).

    Article  CAS  PubMed  Google Scholar 

  66. MacKay, D.J.C. Bayesian Interpolation. Neural Comput. 4, 415–447 (1992).

    Article  Google Scholar 

  67. Toyoizumi, T., Pfister, J.P., Aihara, K. & Gerstner, W. Generalized Bienenstock-Cooper-Munro rule for spiking neurons that maximizes information transmission. Proc. Natl. Acad. Sci. USA 102, 5239–5244 (2005).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Bohte, S.M. & Mozer, M.C. Reducing the variability of neural responses: a computational theory of spike timing–dependent plasticity. Neural Comput. 19, 371–403 (2007).

    Article  PubMed  Google Scholar 

  69. Parra, L.C., Beck, J.M. & Bell, A.J. On the maximization of information flow between spiking neurons. Neural Comput. 21, 2991–3009 (2009).

    Article  PubMed  Google Scholar 

  70. Bishop, C.M. Pattern Recognition and Machine Learning (Springer, 2006).

  71. MacKay, D.J.C. A practical Bayesian framework for backpropagation networks. Neural Comput. 4, 448–472 (1992).

    Article  Google Scholar 

  72. Collins, A. & Koechlin, E. Reasoning, learning and creativity: frontal lobe function and human decision-making. PLoS Biol. 10, e1001293 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  73. Braun, D.A., Mehring, C. & Wolpert, D.M. Structure learning in action. Behav. Brain Res. 206, 157–165 (2010).

    Article  PubMed  PubMed Central  Google Scholar 

  74. Kemp, C. & Tenenbaum, J.B. The discovery of structural form. Proc. Natl. Acad. Sci. USA 105, 10687–10692 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  75. Quartz, S.R. & Sejnowski, T.J. The neural basis of cognitive development: a constructivist manifesto. Behav. Brain Sci. 20, 537–556, discussion 556–596 (1997).

    Article  CAS  PubMed  Google Scholar 

  76. Holtmaat, A., Wilbrecht, L., Knott, G.W., Welker, E. & Svoboda, K. Experience-dependent and cell type–specific spine growth in the neocortex. Nature 441, 979–983 (2006).

    Article  CAS  PubMed  Google Scholar 

  77. Isope, P. & Barbour, B. Properties of unitary granule cell→Purkinje cell synapses in adult rat cerebellar slices. J. Neurosci. 22, 9668–9678 (2002).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  78. Ballard, D.H., Hayhoe, M.M., Pook, P.K. & Rao, R.P. Deictic codes for the embodiment of cognition. Behav. Brain Sci. 20, 723–742, discussion 743–767 (1997).

    Article  CAS  PubMed  Google Scholar 

  79. Gallistel, C.R. & King, A.P. Memory and the Computational Brain: Why Cognitive Science Will Transform Neuroscience (Wiley/Blackwell, New York, 2009).

  80. Smolensky, P. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artif. Intell. 46, 159–217 (1990).

    Article  Google Scholar 

  81. Plate, T. Holographic Reduced Representations (CSLI Publication, Stanford, California, 2003).

  82. Stewart, T. & Eliasmith, C. Compositionality and biologically plausible models. in Oxford Handbook of Compositionality (eds. Hinzen, W., Machery, E. & Werning, M.) (2011).

  83. Gigerenzer, G.T. & Todd, P.M. Simple Heuristics that Make Us Smart (Oxford University Press, New York, 1999).

  84. Fajen, B.R. & Warren, W.H. Behavioral dynamics of intercepting a moving target. Exp. Brain Res. 180, 303–319 (2007).

    Article  PubMed  Google Scholar 

  85. Bowers, J.S. & Davis, C.J. Bayesian just-so stories in psychology and neuroscience. Psychol. Bull. 138, 389–414 (2012).

    Article  PubMed  Google Scholar 

  86. Griffiths, T.L., Chater, N., Norris, D. & Pouget, A. How the Bayesians got their beliefs (and what those beliefs actually are): comment on Bowers and Davis (2012). Psychol. Bull. 138, 415–422 (2012).

    Article  PubMed  Google Scholar 

  87. Knill, D.C. & Pouget, A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712–719 (2004).

    Article  CAS  PubMed  Google Scholar 

  88. Chomsky, N. Aspects of the Theory of Syntax (MIT Press, 1965).

  89. Hsu, A.S., Chater, N. & Vitanyi, P.M. The probabilistic analysis of language acquisition: theoretical, computational and experimental analysis. Cognition 120, 380–390 (2011).

    Article  PubMed  Google Scholar 

  90. Simard, P.Y., LeCun, Y., Denke, J.S. & Victorri, B. Transformation invariance in pattern recognition–tangent distance and tangent propagation. in Neural Networks: Tricks of the Trade (eds. Montavon, G., Orr, G.B. & Müller, K.-R.) 235–269 (2012).

  91. Poggio, T. & Edelman, S. A network that learns to recognize three-dimensional objects. Nature 343, 263–266 (1990).

    Article  CAS  PubMed  Google Scholar 

  92. Beck, J.M., Ma, W.J., Pitkow, X., Latham, P.E. & Pouget, A. Not noisy, just wrong: the role of suboptimal inference in behavioral variability. Neuron 74, 30–39 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  93. MacKay, D. Information Theory, Inference and Learning Algorithms (Cambridge University Press, 2003).

Download references

Acknowledgements

P.E.L. is supported by the Gatsby Charitable Foundation, W.J.M. by National Eye Institute grant R01EY020958-01, National Science Foundation grant IIS-1132009 (Collaborative Research in Computational Neuroscience), and Army Research Office grant W911NF-12-1-0262, and A.P. by National Science Foundation grant #BCS0446730, Multi-University Research Initiative grant #N00014-07-1-0937, National Institute on Drug Abuse grants #BCS0346785, the Swiss National Fund (31003A 143707) and a research grant from the James S. McDonnell Foundation.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alexandre Pouget.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pouget, A., Beck, J., Ma, W. et al. Probabilistic brains: knowns and unknowns. Nat Neurosci 16, 1170–1178 (2013). https://doi.org/10.1038/nn.3495

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3495

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing