Information theory quantifies how much information a neural response carries about the stimulus. This can be compared to the information transferred in particular models of the stimulus–response function and to maximum possible information transfer. Such comparisons are crucial because they validate assumptions present in any neurophysiological analysis. Here we review information-theory basics before demonstrating its use in neural coding. We show how to use information theory to validate simple stimulus–response models of neural coding of dynamic stimuli. Because these models require specification of spike timing precision, they can reveal which time scales contain information in neural coding. This approach shows that dynamic stimuli can be encoded efficiently by single neurons and that each spike contributes to information transmission. We argue, however, that the data obtained so far do not suggest a temporal code, in which the placement of spikes relative to each other yields additional information.
This is a preview of subscription content, access via your institution
Open Access articles citing this article.
Machine Intelligence Research Open Access 14 July 2022
Nature Communications Open Access 12 May 2022
npj Systems Biology and Applications Open Access 27 January 2022
Subscribe to Journal
Get full journal access for 1 year
only $6.58 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Britten, K. H., Shadlen, M. N., Newsome, W. T. & Movshon, J. A. The analysis of visual motion: A comparison of neuronal and psychophysical performance. J. Neurosci. 12, 4745– 4746 (1992).
Gabbiani, F., Metzner, W., Wessel, R. & Koch, C. From stimulus encoding to feature extraction in weakly electric fish. Nature 384, 564–567 (1996).
Parker, A. J. & Newsome, W. T. Sense and the single neuron: probing the physiology of perception. Annu. Rev. Neurosci. 21, 227–277 (1998).
Shannon, C. E. The mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948).
MacKay, D. N. & McCulloch, W. S. The limiting information capacity of a neuronal link. Bull. Math. Biophys. 14, 127–135 (1952).
Stein, R. B., French, A. S. & Holden, A. V. The frequency response, coherence, and information capacity of two neuronal models. Biophys. J. 12, 295–322 (1972).
Eckhorn, R. & Popel, B. Rigorous and extended application of information theory to the afferent visual system of the cat. I. Basic concepts. Biol. Cybern. 16, 191– 200 (1974).
Eckhorn, R. & Popel, B. Rigorous and extended application of information theory to the afferent visual system of the cat. II. Experimental results. Biol. Cybern. 17, 7– 17 (1975).
de Ruyter van Steveninck, R. & Bialek, W. Real-time performance of movement-sensitive neuron in the blowfly visual system: coding and information transfer in short spike sequences. Proc. R. Soc. Lond. B Biol. Sci. 234, 379–414 ( 1988).
Bialek, W., Rieke, F., de Ruyter van Steveninck, R. R. & Warland, D. Reading a neural code. Science 252, 1854 –1857 (1991).
Theunissen, F., Roddey, J. C., Stufflebeam, S., Clague, H. & Miller, J. P. Information theoretic analysis of dynamical encoding by four identified interneurons in the cricket cercal system. J. Neurophysiol. 75, 1345– 1364 (1996).
Wessel, R., Koch, C. & Gabbiani, F. Coding of time-varying electric field amplitude modulation in a wave-type electric fish. J. Neurophysiol. 75, 2280–2293 (1996).
Haag, J. & Borst, A. Encoding of visual motion information and reliability in spiking and graded potential neurons. J. Neurosci. 17, 4809–4819 ( 1997).
Rieke, F., Warland, D., de Ruyter van Steveninck, R. & Bialek, W. Spikes: Exploring the Neural Code (MIT Press, Cambridge, Massachusetts, 1997).
de Ruyter van Steveninck, R. R., Lewen, G. D., Strong, S. P., Koberle, R. & Bialek, W. Reproducibility and variability in neural spike trains. Science 275, 1805–1808 ( 1997).
Buracas, G. T., Zador, A. M., DeWeese, M. R. & Albright, T. D. Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex. Neuron 20, 959 –969 (1998).
Knudsen, E. I. & Konishi, M. Mechanisms of sound localization in the barn owl (Tyto alba). J. Comp. Physiol. 133, 13–21 ( 1979).
Simmons, J. A. Perception of echo phase information in bat sonar. Science 204, 1336–1338 (1979).
Mainen, Z. F. & Sejnowski, T. J. Reliability of spike timing in neocortical neurons. Science 268, 1503 –1506 (1995).
Theunissen, F. E. & Miller, J. P. Temporal encoding in nervous systems: a rigorous definition. J. Comput. Neurosci. 2, 149–162 ( 1995).
McClurkin, J. W., Optican, L. M., Richmond, B. J. & Gawne, T. J. Concurrent processing and complexity of temporally encoded neuronal messages in visual perception. Science 253, 675– 677 (1991).
Vaadia, E. I. et al. Dynamics of neuronal interactions in monkey cortex in relation to behavioral events. Nature 373, 515– 518 (1995).
Laurent, G. & Davidowitz, W. Encoding of olfactory information with oscillating neural assemblies. Science 265, 1872–1875 (1994).
Laurent, G. Odor images and tunes. Neuron 16, 473– 476 (1996).
Theunissen, F. E. & Doupe, A. J. Temporal and spectral sensitivity of complex auditory neurons in the nucleus HVc of male zebra finches. J. Neurosci. 18, 3786– 3802 (1998).
Theunissen, F. E. & Miller, J. P. Representation of sensory information in the cricket cercal sensory system. II Information theoretic calculation of system accuracy and optimal tuning curve width of four primary interneurons. J. Neurophysiol. 66, 1690–1703 (1991).
Kjaer, T. W., Hertz, J. A. & Richmond, B. J. Decoding cortical neuronal signals: network models, information estimation and spatial tuning. J. Comput. Neurosci. 1, 109–139 ( 1994).
Strong, S. P., Koberle, R., de Ruyter van Steveninck, R. R. & Bialek, W. Entropy and information in neural spike trains. Phys. Rev. Lett. 80, 197–200 (1998).
Thomson, D. J. & Chave, A. D. in Advances in Spectrum Analysis and Array Processing Vol 1. (ed. Haykin, S.) 58–113 (Prentice Hall, Englewood Cliffs, New Jersey, 1991).
Theunissen, F. E. An Investigation of Sensory Coding Principles Using Advanced Statistical Techniques. Thesis, Univ. California, Berkeley (1993).
Marmarelis, P. & Marmarelis, V. The White Noise Approach (Plenum, New York, 1978).
Boer, E. & Kuyper, P. Triggered correlation. IEEE Trans. Biomed. Eng. 15, 169–179 (1968).
DeAngelis, G. C., Ohzawa, I. & Freeman, R. D. Receptive-field dynamics in the central visual pathways. Trends Neurosci. 18, 451– 458 (1995).
Eggermont, J. J., Johannesma, P. I. M. & Aertsen, A. M. H. J. Reverse correlation methods in auditory research. Q. Rev. Biophys. 16, 341– 414 (1983).
Gabbiani, F. Coding of time-varying signals in spike trains of linear and half-wave rectifying neurons. Network Comput. Neural Sys. 7, 61–65 (1996).
Roddey, J. C., Girish, B. & Miller, J. P. Assessing the performance of neural encoding models in the presence of noise. J. Comput. Neurosci. (in press).
Haag, J. & Borst, A. Active membrane properties and signal encoding in graded potential neurons. J. Neurosci. 18, 7972–7986 (1998).
Clague, H., Theunissen, F. & Miller, J. P. Effects of adaptation on neural coding by primary sensory interneurons in the cricket cercal system. J. Neurophysiol. 77, 207–220 ( 1997).
Rieke, F., Bodnar, D. & Bialek, W. Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory neurons. Proc. R. Soc. Lond. B Biol. Sci. 262, 259– 265 (1995).
Warland, D., Landolfa, M. A., Miller, J. P. & Bialek, W. in Analysis and Modeling of Neural Systems (ed. Eeckman, F.) 327–333 (Kluwer, Norwell, Massachusetts, 1991).
Perkel, D. H. & Bullock, T. H. Neural coding. Neurosci. Res. Prog. Bull. 6, 223–344 (1967).
Suga, N. Cortical computational maps for auditory imaging. Neural Net. 3, 3–21 (1990).
Margoliash, D. & Fortune, E. S. Temporal and harmonic combination-sensitive neurons in the zebra finch's HVc. J. Neurosci. 12, 4309–4326 (1992).
O'Scalaidhe, S. P., Wilson, F. A. & Goldman-Rakic, P. S. Areal segregation of face-processing neurons in prefrontal cortex. Science 278, 1135– 1138 (1997).
Ferster, D. & Spruston, N. Cracking the neuronal code. Science 270, 756–757 ( 1995).
Abbott, L. F. Decoding neuronal firing and modeling neural networks. Q. Rev. Biophys. 27, 191–331 ( 1994).
Miller, G. A. in Information theory in Psychology: Problems and Methods 95– 100 (Free Press, Glencoe, Illinois, 1955).
Golomb, D., Hertz, J., Panzeri, S., Richmond, B. & Treves, A. How well can we estimate the information carried in neuronal responses from limited samples? Neural Comput. 9, 649–665 (1997).
Koppl, C. Phase locking to high frequencies in the auditory nerve and cochlear nucleus magnocellularis of the barn owl, Tyto alba. J. Neurosci. 17, 3312–3321 ( 1997).
Warland, D. K., Reinagel, P. & Meister, M. Decoding visual information from a population of retinal ganglion cells. J. Neurophysiol. 78, 2336 –2350 (1997).
Roddey, J. C. & Jacobs, G. A. Information theoretic analysis of dynamical encoding by filiform mechanoreceptors in the cricket cercal system. J. Neurophysiol. 75, 1365– 1376 (1996).
Gray, C. M., Konig, P., Engel, A. K. & Singer, W. Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338, 334–337 (1989).
Abeles, M., Bergman, H., Margalit, E. & Vaadia, E. Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. J. Neurophysiol. 70, 1629– 1638 (1993).
DeCharms, R. C. & Merzenich, M. M. Primary cortical representation of sounds by the coordination of action-potential timing. Nature 381, 610–613 ( 1996).
We are grateful to B. Bialek, A. Doupe, F. Gabbiani and J. Haag for comments on the manuscript.
About this article
Cite this article
Borst, A., Theunissen, F. Information theory and neural coding. Nat Neurosci 2, 947–957 (1999). https://doi.org/10.1038/14731
This article is cited by
npj Systems Biology and Applications (2022)
Nature Communications (2022)
Machine Intelligence Research (2022)
Nonlinear Dynamics (2022)
Decoding neurobiological spike trains using recurrent neural networks: a case study with electrophysiological auditory cortex recordings
Neural Computing and Applications (2022)