Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Information theory and neural coding

Nature Neuroscience volume 2, pages 947–957 (1999)Cite this article

Abstract

Information theory quantifies how much information a neural response carries about the stimulus. This can be compared to the information transferred in particular models of the stimulus–response function and to maximum possible information transfer. Such comparisons are crucial because they validate assumptions present in any neurophysiological analysis. Here we review information-theory basics before demonstrating its use in neural coding. We show how to use information theory to validate simple stimulus–response models of neural coding of dynamic stimuli. Because these models require specification of spike timing precision, they can reveal which time scales contain information in neural coding. This approach shows that dynamic stimuli can be encoded efficiently by single neurons and that each spike contributes to information transmission. We argue, however, that the data obtained so far do not suggest a temporal code, in which the placement of spikes relative to each other yields additional information.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: A mock neuron is tested with different stimulus intensities (from 0 to 10).
Figure 2: Flow chart of how to measure the channel capacity of a neuron.
Figure 3: Summary diagram for calculation of upper and lower bounds on information transfer.
Figure 4: An example of reverse reconstruction.
Figure 5: Example of upper and lower bound of information as calculated from the spike train of a fly motion-sensitive nerve cell (H1 cell).

References

  1. Britten, K. H., Shadlen, M. N., Newsome, W. T. & Movshon, J. A. The analysis of visual motion: A comparison of neuronal and psychophysical performance. J. Neurosci. 12, 4745– 4746 (1992).

    Article  CAS  Google Scholar 

  2. Gabbiani, F., Metzner, W., Wessel, R. & Koch, C. From stimulus encoding to feature extraction in weakly electric fish. Nature 384, 564–567 (1996).

    Article  CAS  Google Scholar 

  3. Parker, A. J. & Newsome, W. T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998).

    Article  CAS  Google Scholar 

  4. Shannon, C. E. The mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948).

    Article  Google Scholar 

  5. MacKay, D. N. & McCulloch, W. S. The limiting information capacity of a neuronal link. Bull. Math. Biophys. 14, 127–135 (1952).

    Article  Google Scholar 

  6. Stein, R. B., French, A. S. & Holden, A. V. The frequency response, coherence, and information capacity of two neuronal models. Biophys. J. 12, 295–322 (1972).

    Article  CAS  Google Scholar 

  7. Eckhorn, R. & Popel, B. Rigorous and extended application of information theory to the afferent visual system of the cat. I. Basic concepts. Biol. Cybern. 16, 191– 200 (1974).

    CAS  Google Scholar 

  8. Eckhorn, R. & Popel, B. Rigorous and extended application of information theory to the afferent visual system of the cat. II. Experimental results. Biol. Cybern. 17, 7– 17 (1975).

    Article  Google Scholar 

  9. de Ruyter van Steveninck, R. & Bialek, W. Real-time performance of movement-sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences. Proc. R. Soc. Lond. B Biol. Sci. 234, 379–414 ( 1988).

    Article  Google Scholar 

  10. Bialek, W., Rieke, F., de Ruyter van Steveninck, R. R. & Warland, D. Reading a neural code. Science 252, 1854 –1857 (1991).

    Article  CAS  Google Scholar 

  11. Theunissen, F., Roddey, J. C., Stufflebeam, S., Clague, H. & Miller, J. P. Information theoretic analysis of dynamical encoding by four identified interneurons in the cricket cercal system. J. Neurophysiol. 75, 1345– 1364 (1996).

    Article  CAS  Google Scholar 

  12. Wessel, R., Koch, C. & Gabbiani, F. Coding of time-varying electric field amplitude modulation in a wave-type electric fish. J. Neurophysiol. 75, 2280–2293 (1996).

    Article  CAS  Google Scholar 

  13. Haag, J. & Borst, A. Encoding of visual motion information and reliability in spiking and graded potential neurons. J. Neurosci. 17, 4809–4819 ( 1997).

    Article  CAS  Google Scholar 

  14. Rieke, F., Warland, D., de Ruyter van Steveninck, R. & Bialek, W. Spikes: Exploring the Neural Code (MIT Press, Cambridge, Massachusetts, 1997).

    Google Scholar 

  15. de Ruyter van Steveninck, R. R., Lewen, G. D., Strong, S. P., Koberle, R. & Bialek, W. Reproducibility and variability in neural spike trains. Science 275, 1805–1808 ( 1997).

    Article  CAS  Google Scholar 

  16. Buracas, G. T., Zador, A. M., DeWeese, M. R. & Albright, T. D. Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron 20, 959 –969 (1998).

    Article  CAS  Google Scholar 

  17. Knudsen, E. I. & Konishi, M. Mechanisms of sound localization in the barn owl (Tyto alba). J. Comp. Physiol. 133, 13–21 ( 1979).

    Article  Google Scholar 

  18. Simmons, J. A. Perception of echo phase information in bat sonar. Science 204, 1336–1338 (1979).

    Article  CAS  Google Scholar 

  19. Mainen, Z. F. & Sejnowski, T. J. Reliability of spike timing in neocortical neurons. Science 268, 1503 –1506 (1995).

    Article  CAS  Google Scholar 

  20. Theunissen, F. E. & Miller, J. P. Temporal encoding in nervous systems: a rigorous definition. J. Comput. Neurosci. 2, 149–162 ( 1995).

    Article  CAS  Google Scholar 

  21. McClurkin, J. W., Optican, L. M., Richmond, B. J. & Gawne, T. J. Concurrent processing and complexity of temporally encoded neuronal messages in visual perception. Science 253, 675– 677 (1991).

    Article  CAS  Google Scholar 

  22. Vaadia, E. I. et al. Dynamics of neuronal interactions in monkey cortex in relation to behavioral events. Nature 373, 515– 518 (1995).

    Article  CAS  Google Scholar 

  23. Laurent, G. & Davidowitz, W. Encoding of olfactory information with oscillating neural assemblies. Science 265, 1872–1875 (1994).

    Article  CAS  Google Scholar 

  24. Laurent, G. Odor images and tunes. Neuron 16, 473– 476 (1996).

    Article  CAS  Google Scholar 

  25. Theunissen, F. E. & Doupe, A. J. Temporal and spectral sensitivity of complex auditory neurons in the nucleus HVc of male zebra finches. J. Neurosci. 18, 3786– 3802 (1998).

    Article  CAS  Google Scholar 

  26. Theunissen, F. E. & Miller, J. P. Representation of sensory information in the cricket cercal sensory system. II Information theoretic calculation of system accuracy and optimal tuning curve width of four primary interneurons. J. Neurophysiol. 66, 1690–1703 (1991).

    Article  CAS  Google Scholar 

  27. Kjaer, T. W., Hertz, J. A. & Richmond, B. J. Decoding cortical neuronal signals: network models, information estimation and spatial tuning. J. Comput. Neurosci. 1, 109–139 ( 1994).

    Article  CAS  Google Scholar 

  28. Strong, S. P., Koberle, R., de Ruyter van Steveninck, R. R. & Bialek, W. Entropy and information in neural spike trains. Phys. Rev. Lett. 80, 197–200 (1998).

    Article  CAS  Google Scholar 

  29. Thomson, D. J. & Chave, A. D. in Advances in Spectrum Analysis and Array Processing Vol 1. (ed. Haykin, S.) 58–113 (Prentice Hall, Englewood Cliffs, New Jersey, 1991).

    Google Scholar 

  30. Theunissen, F. E. An Investigation of Sensory Coding Principles Using Advanced Statistical Techniques. Thesis, Univ. California, Berkeley (1993).

    Google Scholar 

  31. Marmarelis, P. & Marmarelis, V. The White Noise Approach (Plenum, New York, 1978).

    Google Scholar 

  32. Boer, E. & Kuyper, P. Triggered correlation. IEEE Trans. Biomed. Eng. 15, 169–179 (1968).

    Article  Google Scholar 

  33. DeAngelis, G. C., Ohzawa, I. & Freeman, R. D. Receptive-field dynamics in the central visual pathways. Trends Neurosci. 18, 451– 458 (1995).

    Article  CAS  Google Scholar 

  34. Eggermont, J. J., Johannesma, P. I. M. & Aertsen, A. M. H. J. Reverse correlation methods in auditory research. Q. Rev. Biophys. 16, 341– 414 (1983).

    Article  CAS  Google Scholar 

  35. Gabbiani, F. Coding of time-varying signals in spike trains of linear and half-wave rectifying neurons. Network Comput. Neural Sys. 7, 61–65 (1996).

    Google Scholar 

  36. Roddey, J. C., Girish, B. & Miller, J. P. Assessing the performance of neural encoding models in the presence of noise. J. Comput. Neurosci. (in press).

  37. Haag, J. & Borst, A. Active membrane properties and signal encoding in graded potential neurons. J. Neurosci. 18, 7972–7986 (1998).

    Article  CAS  Google Scholar 

  38. Clague, H., Theunissen, F. & Miller, J. P. Effects of adaptation on neural coding by primary sensory interneurons in the cricket cercal system. J. Neurophysiol. 77, 207–220 ( 1997).

    Article  CAS  Google Scholar 

  39. Rieke, F., Bodnar, D. & Bialek, W. Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory neurons. Proc. R. Soc. Lond. B Biol. Sci. 262, 259– 265 (1995).

    Article  CAS  Google Scholar 

  40. Warland, D., Landolfa, M. A., Miller, J. P. & Bialek, W. in Analysis and Modeling of Neural Systems (ed. Eeckman, F.) 327–333 (Kluwer, Norwell, Massachusetts, 1991).

    Google Scholar 

  41. Perkel, D. H. & Bullock, T. H. Neural coding. Neurosci. Res. Prog. Bull. 6, 223–344 (1967).

    Google Scholar 

  42. Suga, N. Cortical computational maps for auditory imaging. Neural Net. 3, 3–21 (1990).

    Article  Google Scholar 

  43. Margoliash, D. & Fortune, E. S. Temporal and harmonic combination-sensitive neurons in the zebra finch's HVc. J. Neurosci. 12, 4309–4326 (1992).

    Article  CAS  Google Scholar 

  44. O'Scalaidhe, S. P., Wilson, F. A. & Goldman-Rakic, P. S. Areal segregation of face-processing neurons in prefrontal cortex. Science 278, 1135– 1138 (1997).

    Article  CAS  Google Scholar 

  45. Ferster, D. & Spruston, N. Cracking the neuronal code. Science 270, 756–757 ( 1995).

    Article  CAS  Google Scholar 

  46. Abbott, L. F. Decoding neuronal firing and modeling neural networks. Q. Rev. Biophys. 27, 191–331 ( 1994).

    Article  Google Scholar 

  47. Miller, G. A. in Information theory in Psychology: Problems and Methods 95– 100 (Free Press, Glencoe, Illinois, 1955).

    Google Scholar 

  48. Golomb, D., Hertz, J., Panzeri, S., Richmond, B. & Treves, A. How well can we estimate the information carried in neuronal responses from limited samples? Neural Comput. 9, 649–665 (1997).

    Article  CAS  Google Scholar 

  49. Koppl, C. Phase locking to high frequencies in the auditory nerve and cochlear nucleus magnocellularis of the barn owl, Tyto alba. J. Neurosci. 17, 3312–3321 ( 1997).

    Article  CAS  Google Scholar 

  50. Warland, D. K., Reinagel, P. & Meister, M. Decoding visual information from a population of retinal ganglion cells. J. Neurophysiol. 78, 2336 –2350 (1997).

    Article  CAS  Google Scholar 

  51. Roddey, J. C. & Jacobs, G. A. Information theoretic analysis of dynamical encoding by filiform mechanoreceptors in the cricket cercal system. J. Neurophysiol. 75, 1365– 1376 (1996).

    Article  CAS  Google Scholar 

  52. Gray, C. M., Konig, P., Engel, A. K. & Singer, W. Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338, 334–337 (1989).

    Article  CAS  Google Scholar 

  53. Abeles, M., Bergman, H., Margalit, E. & Vaadia, E. Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. J. Neurophysiol. 70, 1629– 1638 (1993).

    Article  CAS  Google Scholar 

  54. DeCharms, R. C. & Merzenich, M. M. Primary cortical representation of sounds by the coordination of action-potential timing. Nature 381, 610–613 ( 1996).

    Article  CAS  Google Scholar 

Download references

Acknowledgements

We are grateful to B. Bialek, A. Doupe, F. Gabbiani and J. Haag for comments on the manuscript.

Author information

Authors and Affiliations

  1. ESPM-Division of Insect Biology, University of California, Berkeley, 94720, California, USA

    Alexander Borst

  2. Dept. of Psychology, University of California, Berkeley, 94720, California, USA

    Frédéric E. Theunissen

Authors
  1. Alexander Borst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Frédéric E. Theunissen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexander Borst.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Borst, A., Theunissen, F. Information theory and neural coding. Nat Neurosci 2, 947–957 (1999). https://doi.org/10.1038/14731

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1038/14731

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing