Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Perspective
  • Published:

Bayesian encoding and decoding as distinct perspectives on neural coding

Nature Neuroscience volume 26, pages 2063–2072 (2023)Cite this article

Subjects

Abstract

The Bayesian brain hypothesis is one of the most influential ideas in neuroscience. However, unstated differences in how Bayesian ideas are operationalized make it difficult to draw general conclusions about how Bayesian computations map onto neural circuits. Here, we identify one such unstated difference: some theories ask how neural circuits could recover information about the world from sensory neural activity (Bayesian decoding), whereas others ask how neural circuits could implement inference in an internal model (Bayesian encoding). These two approaches require profoundly different assumptions and lead to different interpretations of empirical data. We contrast them in terms of motivations, empirical support and relationship to neural data. We also use a simple model to argue that encoding and decoding models are complementary rather than competing. Appreciating the distinction between Bayesian encoding and Bayesian decoding will help to organize future work and enable stronger empirical tests about the nature of inference in the brain.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Visualization of Bayesian encoding.
Fig. 2: Visualization of Bayesian decoding.
Fig. 3: Side-by-side comparison of Bayesian encoding and Bayesian decoding.
Fig. 4: Encoding by sampling followed by decoding orientation from the samples.

Similar content being viewed by others

Code availability

Two panels in Fig. 4 were generated by simulation. The code is available at https://github.com/haefnerlab/bayesian-encoding-decoding/.

References

  1. von Helmholtz, H. Treatise on Physiological Optics (The Optical Society of America, 1925).

  2. Knill, D. C. & Richards, W. Perception as Bayesian Inference (Cambridge Univ. Press, 1996).

  3. Kersten, D., Mamassian, P. & Yuille, A. Object perception as Bayesian inference. Annu. Rev. Psychol. 55, 271–304 (2004).

  4. Fiser, J., Berkes, P., Orbán, G. & Lengyel, M. Statistically optimal perception and learning: from behavior to neural representations. Trends Cogn. Sci. 14, 119–130 (2010).

    PubMed  PubMed Central  Google Scholar 

  5. Pouget, A., Beck, J. M., Ma, W. J. & Latham, P. E. Probabilistic brains: knowns and unknowns. Nat. Neurosci. 16, 1170–1178 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  6. Shivkumar, S., Lange, R. D., Chattoraj, A. & Haefner, R. M. A probabilistic population code based on neural samples. Adv. Neural Inf. Process. Syst. 31, 7070–7079 (2018).

    Google Scholar 

  7. Olshausen, B. A. & Field, D. J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996).

    CAS  PubMed  Google Scholar 

  8. Olshausen, B. A. & Field, D. J. Sparse coding with an incomplete basis set: a strategy employed by V1? Vis. Res. 37, 3311–3325 (1997).

    CAS  PubMed  Google Scholar 

  9. Zemel, R. S., Dayan, P. & Pouget, A. Probabilistic interpretation of population codes. Neural Comput. 10, 403–430 (1998).

    CAS  PubMed  Google Scholar 

  10. Berkes, P., Orbán, G., Lengyel, M. & Fiser, J. Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment. Science 331, 83–87 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Dayan, P., Hinton, G. E., Neal, R. & Zemel, R. The Helmholtz machine. Neural Comput. 7, 889–904 (1995).

    CAS  PubMed  Google Scholar 

  12. Dayan, P. & Abbott, L. F. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, 2001).

  13. Bell, A. J. & Sejnowski, T. J. The ‘independent components’ of scenes are edge filters. Vis. Res. 37, 3327–3338 (1997).

    CAS  PubMed  Google Scholar 

  14. Murphy, K. P. Machine Learning: a Probabilistic Perspective (MIT Press, 2012).

  15. Wainwright, M. J. & Jordan, M. I. Graphical models, exponential families, and variational inference. Found. Trends Mach. Learn. 1, 1–305 (2008).

    Google Scholar 

  16. Bishop, C. M. Pattern Recognition and Machine Learning (Springer, 2006).

  17. Sanborn, A. N. Types of approximation for probabilistic cognition: sampling and variational. Brain Cogn. 112, 98–101 (2015).

    PubMed  Google Scholar 

  18. Gershman, S. J. & Beck, J. M. in Computational models of brain and behavior (ed. A. Moustafa), 453–466 (Wiley-Blackwell, 2016).

  19. Ma, W. J., Beck, J. M., Latham, P. E. & Pouget, A. Bayesian inference with probabilistic population codes. Nat. Neurosci. 9, 1432–1438 (2006).

    CAS  PubMed  Google Scholar 

  20. Beck, J. M. et al. Probabilistic population codes for Bayesian decision-making. Neuron 36, 1142–1152 (2008).

    Google Scholar 

  21. Fetsch, C. R., Pouget, A., DeAngelis, G. C. & Angelaki, D. E. Neural correlates of reliability-based cue weighting during multisensory integration. Nat. Neurosci. 15, 146–154 (2011).

    PubMed  PubMed Central  Google Scholar 

  22. Fetsch, C. R., DeAngelis, G. C. & Angelaki, D. E. Bridging the gap between theories of sensory cue integration and the physiology of multisensory neurons. Nat. Rev. Neurosci. 14, 429–442 (2013).

    CAS  PubMed  Google Scholar 

  23. Hou, H., Zheng, Q., Zhao, Y., Pouget, A. & Gu, Y. Neural correlates of optimal multisensory decision making under time-varying reliabilities with an invariant linear probabilistic population code. Neuron 104, 1010–1021 (2019).

    CAS  PubMed  Google Scholar 

  24. Orbán, G., Berkes, P., Fiser, J. & Lengyel, M. Neural variability and sampling-based probabilistic representations in the visual cortex. Neuron 92, 530–543 (2016).

    PubMed  PubMed Central  Google Scholar 

  25. Haefner, R. M., Berkes, P. & Fiser, J. Perceptual decision-making as probabilistic inference by neural sampling. Neuron 90, 649–660 (2016).

    CAS  PubMed  Google Scholar 

  26. Bányai, M. et al. Stimulus complexity shapes response correlations in primary visual cortex. Proc. Natl Acad. Sci. USA 116, 2723–2732 (2019).

    PubMed  PubMed Central  Google Scholar 

  27. Lange, R. D. & Haefner, R. M. Task-induced neural covariability as a signature of approximate Bayesian learning and inference. PLoS Comput. Biol. 18, e1009557 (2022).

  28. Campbell, D. T. in Studies in the Philosophy of Biology (ed. F. J. Ayala) 179–186 (Macmillan, 1974).

  29. Yablo, S. Mental causation. Philos. Rev. 101, 245–280 (1992).

    Google Scholar 

  30. Morris, R. Developments of a water-maze procedure for studying spatial learning in the rat. J. Neurosci. Methods 11, 47–60 (1984).

    CAS  PubMed  Google Scholar 

  31. Beck, J. M., Heller, K. & Pouget, A. Complex inference in neural circuits with probabilistic population codes and topic models. Adv. Neural Inf. Process. Syst. 25, 3068–3076 (2012).

    Google Scholar 

  32. Ernst, M. O. & Banks, M. S. Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415, 429–433 (2002).

    CAS  PubMed  Google Scholar 

  33. Knill, D. C. & Pouget, A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27, 712–719 (2004).

    CAS  PubMed  Google Scholar 

  34. Alais, D. & Burr, D. The ventriloquist effect results from near-optimal bimodal integration. Curr. Biol. 14, 257–262 (2004).

    CAS  PubMed  Google Scholar 

  35. Körding, K. P. Decision theory: what ‘should’ the nervous system do? Science 318, 606–610 (2007).

  36. Angelaki, D. E., Gu, Y. & DeAngelis, G. C. Multisensory integration: psychophysics, neurophysiology, and computation. Curr. Opin. Neurobiol. 19, 452–458 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  37. Walker, E. Y., Cotton, R. J., Ma, W. J. & Tolias, A. S. A neural basis of probabilistic computation in visual cortex. Nat. Neurosci. 23, 122–129 (2019).

    PubMed  Google Scholar 

  38. Rahnev, D. The Bayesian brain: what is it and do humans have it? Behav. Brain Sci. 42, e238 (2019).

    PubMed  PubMed Central  Google Scholar 

  39. Koblinger, Á., Fiser, J. & Lengyel, M. Representations of uncertainty: where art thou? Curr. Opin. Behav. Sci. 38, 150–162 (2021).

    PubMed  PubMed Central  Google Scholar 

  40. Jaynes, E. T. Probability Theory: the Logic of Science (Cambridge Univ. Press, 2003).

  41. Aitchison, L. & Lengyel, M. The hamiltonian brain: efficient probabilistic inference with excitatory–inhibitory neural circuit dynamics. PLoS Comput. Biol. 12, e1005186 (2016).

    PubMed  PubMed Central  Google Scholar 

  42. Echeveste, R., Aitchison, L., Hennequin, G. & Lengyel, M. Cortical-like dynamics in recurrent circuits optimized for sampling-based probabilistic inference. Nat. Neurosci. 23, 1138–1149 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  43. George, D. & Hawkins, J. Towards a mathematical theory of cortical micro-circuits. PLoS Comput. Biol. 5, e1000532 (2009).

    PubMed  PubMed Central  Google Scholar 

  44. Raju, R. V. & Pitkow, X. in Advances in Neural Information Processing Systems vol. 30 (eds. Lee, D. D. et al.) 2029–2037 (Curran Associates, 2016).

  45. Grabska-Barwinska, A., Beck, J. M., Pouget, A. & Latham, P. E. in Advances in Neural Information Processing Systems vol. 26 (eds. Burges, C. J. C. et al.) 1968–1976 (Curran Associates, 2013).

  46. George, D. et al. Cortical microcircuits from a generative vision model. Preprint at https://arxiv.org/abs/1808.01058 (2018).

  47. Friston, K. J. A theory of cortical responses. Philos. Trans. R. Soc. Lond. B Biol. Sci. 360, 815–836 (2005).

    PubMed  PubMed Central  Google Scholar 

  48. Lavin, A., Guntupalli, J. S., Lázaro-gredilla, M., Lehrach, W. & George, D. Explaining visual cortex phenomena using recursive cortical network. Preprint at bioRxiv https://doi.org/10.1101/380048 (2018).

  49. Vertes, E. & Sahani, M. in Advances in Neural Information Processing Systems vol. 31 (eds Bengio, S. et al.) (Curran Associates, 2018).

  50. Moreno-Bote, R., Knill, D. C. & Pouget, A. Bayesian sampling in visual perception. Proc. Natl Acad. Sci. USA 108, 12491–12496 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  51. Gershman, S. J., Vul, E. & Tenenbaum, J. B. Multistability and perceptual inference. Neural Comput. 24, 1–24 (2012).

    PubMed  Google Scholar 

  52. Hohwy, J., Roepstorff, A. & Friston, K. J. Predictive coding explains binocular rivalry: an epistemological review. Cognition 108, 687–701 (2008).

    PubMed  Google Scholar 

  53. Hoyer, P. O. & Hyvärinen, A. Interpreting neural response variability as monte carlo sampling of the posterior. Adv. Neural Inf. Process. Syst. 17, 293–300 (2003).

    Google Scholar 

  54. Schwartz, O. & Simoncelli, E. P. Natural signal statistics and sensory gain control. Nat. Neurosci. 4, 819–825 (2001).

    CAS  PubMed  Google Scholar 

  55. Bornschein, J., Henniges, M. & Lücke, J. Are V1 simple cells optimized for visual occlusions? a comparative study. PLoS Comput. Biol. 9, e1003062 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  56. Coen-Cagli, R., Kohn, A. & Schwartz, O. Flexible gating of contextual influences in natural vision. Nat. Neurosci. 18, 1648–1655 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  57. Graf, A. B. A., Kohn, A., Jazayeri, M. & Movshon, J. A. Decoding the activity of neuronal populations in macaque primary visual cortex. Nat. Neurosci. 14, 239–247 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  58. Orhan, A. E. & Ma, W. J. Efficient probabilistic inference in generic neural networks trained with non-probabilistic feedback. Nat. Commun. 8, 138 (2017).

    PubMed  PubMed Central  Google Scholar 

  59. Block, N. If perception is probabilistic, why does it not seem probabilistic? Philos. Trans. R. Soc. Lond. B Biol. Sci. 373, 20170341 (2018).

    PubMed  PubMed Central  Google Scholar 

  60. Rahnev, D., Block, N., Jehee, J. & Denison, R. Is perception probabilistic? Clarifying the definitions. Preprint at https://psyarxiv.com/f8v5r/ (2020).

  61. Sahani, M. & Dayan, P. Doubly distributional population codes: simultaneous representation of uncertainty and multiplicity. Neural Comput. 15, 2255–2279 (2003).

    PubMed  Google Scholar 

  62. Vul, E. & Rich, A. N. Independent sampling of features enables conscious perception of bound objects. Psychol. Sci. 21, 1168–1175 (2010).

    PubMed  Google Scholar 

  63. Sanborn, A. N., Griffiths, T. L. & Navarro, D. J. Rational approximations to rational models: alternative algorithms for category learning. Psychol. Rev. 117, 1144–1167 (2010).

    PubMed  Google Scholar 

  64. Lieder, F., Hsu, M. & Griffiths, T. L. The high availability of extreme events serves resource-rational decision-making. In Proc. 36th Annual Conference of the Cognitive Science Society (eds. Bello, P. et al.) 2567–2572 (Cognitive Science Soc., 2014).

  65. Vul, E., Goodman, N. D., Griffiths, T. L. & Tenenbaum, J. B. One and done? Optimal decisions from very few samples. Cogn. Sci. 38, 599–637 (2014).

    PubMed  Google Scholar 

  66. Sanborn, A. N. & Chater, N. Bayesian brains without probabilities. Trends Cogn. Sci. 20, 883–893 (2016).

    PubMed  Google Scholar 

  67. Lieder, F., Griffiths, T. L., Huys, Q. J. M. & Goodman, N. D. The anchoring bias reflects rational use of cognitive resources. Psychon. Bull. Rev. 25, 322–334 (2017).

    Google Scholar 

  68. Zhu, J.-Q., Sanborn, A. N. & Chater, N. The Bayesian sampler: generic Bayesian inference causes incoherence in human probability judgments. Psychol. Rev. 127, 719–748 (2020).

    PubMed  PubMed Central  Google Scholar 

  69. Daw, N. D., Courville, A. C. & Dayan, P. in the Probabilistic Mind: Prospects for Bayesian Cognitive Science (eds. Chater, N. & Oaksford, M.) https://doi.org/10.1093/acprof:oso/9780199216093.003.0019 (Oxford Scholarship Online, 2008).

  70. Sanborn, A. N. & Silva, R. Constraining bridges between levels of analysis: a computational justification for locally Bayesian learning. J. Math. Psychol. 57, 94–106 (2013).

    Google Scholar 

  71. Lange, R. D., Chattoraj, A., Beck, J. M., Yates, J. L. & Haefner, R. M. A confirmation bias in perceptual decision-making due to hierarchical approximate inference. PLoS Comput. Biol. 17, e1009517 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  72. Griffiths, T. L., Vul, E. & Sanborn, A. N. Bridging levels of analysis for probabilistic models of cognition. Curr. Dir. Psychol. Sci. 21, 263–268 (2012).

    Google Scholar 

  73. Jaakkola, T. S. & Jordan, M. I. in Learning in Graphical Models (ed. Jordan, M. I.) 163–174 (Kluwer, 1998).

  74. Lange, R. D., Benjamin, A. S., Haefner, R. M. & Pitkow, X. Interpolating between sampling and variational inference with infinite stochastic mixtures. Proc. Machine Learn. Res. 180, 1063–1073 (2022).

    Google Scholar 

  75. Walker, E. Y. et al. Studying the neural representations of uncertainty. Nat. Neurosci. 26, 1857–1867 (2023).

    CAS  PubMed  Google Scholar 

  76. Pecevski, D., Buesing, L. & Maass, W. Probabilistic inferences general graphical models through sampling in stochastic networks of spiking neurons. PLoS Comput. Biol. 7, e1002294 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  77. Buesing, L., Bill, J., Nessler, B. & Maass, W. Neural dynamics as sampling: a model for stochastic computation in recurrent networks of spiking neurons. PLoS Comput. Biol. 7, e1002211 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  78. Savin, C. & Denève, S. Spatio-temporal representations of uncertainty in spiking neural networks. In Advances in Neural Information Processing Systems vol. 27 2024–2032 (NIPS, 2014).

  79. Probst, D. et al. Probabilistic inference in discrete spaces can be implemented into networks of LIF neurons. Front. Comput. Neurosci. 9, 13 (2015).

    PubMed  PubMed Central  Google Scholar 

  80. Festa, D., Aschner, A., Davila, A., Kohn, A. & Coen-Cagli, R. Neuronal variability reflects probabilistic inference tuned to natural image statistics. Nat. Commun. 12, 3635 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  81. Tajima, C. I. et al. Population code dynamics in categorical perception. Sci. Rep. 6, 22536 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  82. Beck, J. M., Latham, P. E. & Pouget, A. Marginalization in neural circuits with divisive normalization. J. Neurosci. 31, 15310–15319 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank the many colleagues with whom we have discussed the ideas in this paper, especially M. Lengyel and J. Drugowitsch for their detailed comments on an earlier version of this manuscript. This work was supported by the National Institutes of Health (NIH) R01 grant EY028811, NIH U19 grant 1U19NS118246-01 and a National Science Foundation CAREER grant IIS-2143440 to R.M.H.

Author information

Author notes

  1. Richard D. Lange

    Present address: Department of Computer Science, Rochester Institute of Technology, Rochester, NY, USA

  2. Sabyasachi Shivkumar

    Present address: Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA

  3. These authors contributed equally: Sabyasachi Shivkumar, Ankani Chattoraj.

Authors and Affiliations

  1. Department of Neurobiology, University of Pennsylvania, Philadelphia, PA, USA

    Richard D. Lange

  2. Department of Brain and Cognitive Sciences, University of Rochester, Rochester, NY, USA

    Sabyasachi Shivkumar, Ankani Chattoraj & Ralf M. Haefner

Authors
  1. Richard D. Lange
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sabyasachi Shivkumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ankani Chattoraj
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ralf M. Haefner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Richard D. Lange.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Neuroscience thanks Máté Lengyel, Cristina Savin, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Note containing derivation of equation (5).

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lange, R.D., Shivkumar, S., Chattoraj, A. et al. Bayesian encoding and decoding as distinct perspectives on neural coding. Nat Neurosci 26, 2063–2072 (2023). https://doi.org/10.1038/s41593-023-01458-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41593-023-01458-6

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing