Decorrelation and efficient coding by retinal ganglion cells

Journal name:
Nature Neuroscience
Volume:
15,
Pages:
628–635
Year published:
DOI:
doi:10.1038/nn.3064
Received
Accepted
Published online

Abstract

An influential theory of visual processing asserts that retinal center-surround receptive fields remove spatial correlations in the visual world, producing ganglion cell spike trains that are less redundant than the corresponding image pixels. For bright, high-contrast images, this decorrelation would enhance coding efficiency in optic nerve fibers of limited capacity. We tested the central prediction of the theory and found that the spike trains of retinal ganglion cells were indeed decorrelated compared with the visual input. However, most of the decorrelation was accomplished not by the receptive fields, but by nonlinear processing in the retina. We found that a steep response threshold enhanced efficient coding by noisy spike trains and that the effect of this nonlinearity was near optimal in both salamander and macaque retina. These results offer an explanation for the sparseness of retinal spike trains and highlight the importance of treating the full nonlinear character of neural codes.

At a glance

Figures

  1. Decorrelation of naturalistic stimuli.
    Figure 1: Decorrelation of naturalistic stimuli.

    (a,b) Sample frames of naturalistic and white noise stimuli, projected onto a 3.4-mm square on the retina. (c) Responses of two RGCs to a short segment of the naturalistic stimulus, displayed as rasters of spikes on 250 identical repeats. (d) A sample spatio-temporal receptive field for an OFF ganglion cell, measured as the spike-triggered average stimulus and integrated over one spatial dimension for ease of display. Note the spatial center-surround antagonism (red regions above and below blue) and the biphasic time course (red region left of blue). (e) Spatial receptive fields of two OFF cells, including 1-s.d. outlines of the receptive field centers (solid) and surrounds (dotted). (f) Cross-correlation function between two ganglion cell spike trains, indicating the frequency of spike pairs as a function of their delay. The shaded area encompasses most of the central peak and indicates the range of delays used to compute the quoted correlation coefficients. (g,h) Correlation coefficient between the responses of two ganglion cells as a function of their distance under a white noise (g) or naturalistic (h) stimulus. Each pair of cells contributes a point; lines represent median correlation for pairs at similar distance. Comparisons are restricted within a cell type (solid lines) or across cell types (dashed lines). For reference, the correlation between stimulus pixels is shown (thin lines).

  2. Nonlinearity accounts for much of decorrelation.
    Figure 2: Nonlinearity accounts for much of decorrelation.

    (a,b) Spatial correlation functions for neurons and models under naturalistic stimulation. Cells with the same polarity preference (OFF-OFF or ON-ON pairs) have positive correlations (a) and those with opposite polarity preferences (OFF-ON pairs) have negative correlations (b). Curves are presented as in Figure 1h for the stimulus, trial-averaged firing rates, spike trains and linear models. The stimulus correlations are shown with opposite sign for ease of comparison in b. Results from many cell pairs are summarized by the median correlation for pairs at similar retinal distance; error bars indicate the central quartiles. L: center and L: center-surround designate linear models using receptive fields including the center component only or both center and surround. (c) The origins of decorrelation in different response components. The full circle represents the median correlation present in the stimulus after filtering by the receptive field centers at a retinal distance of 300 μm (arrowheads in a). The empty wedge (C) is the much smaller remaining correlation between the ganglion cell spike trains. The red wedge represents the decorrelation caused by lateral inhibition from receptive field surrounds. The difference between the linear response and the observed firing rate is a result of nonlinear processing and is responsible for over half the decorrelation implemented by the retina (green wedge). The trial-to-trial variation contributes an additional small amount of decorrelation (blue wedge). (d) Decorrelation in the time domain. Autocorrelation functions of salamander ganglion cell responses and linear models are plotted as a function of delay during naturalistic stimulation. The linear filter's first lobe, ~100 ms wide (inset, black), introduced excess correlation beyond that in the stimulus. The antagonistic second lobe (inset, red) counteracted those, but overcompensated, introducing anticorrelations at long delays. The observed correlations in the firing rate were much smaller still. (e,f) Spatial (e) and temporal (f) correlations in macaque RGCs, displayed as in a, b and d. Macaque RGC responses were approximated by an LN model13, 23, using published spatio-temporal receptive field parameters36 (equations (4–6)) and sigmoidal nonlinearities23 (equation (10)). The output noise was modeled as sub-Poisson variation (equation (11)) with parameters derived from published spike trains21 (see Online Methods). The stimulus was scaled in space and time to compensate for the different scales of primate and salamander receptive fields. L, receptive field filter only; LN, including the nonlinearity; LN + noise, including the noise.

  3. Sparseness in retinal responses.
    Figure 3: Sparseness in retinal responses.

    (a) Spike rasters for two salamander ganglion cells over ten repetitions of a naturalistic stimulus. Firing events are brief, separated by long silences, and have some trial-to-trial variability. (b) Mean firing rates for the same neurons, with shading that indicates the s.d. about the mean in time bins of 50 ms. (c) The linear response generated from convolving the stimulus with the spatiotemporal receptive fields of those two cells. This linear model generally captures the times of firing events, but differs markedly in sparseness. (d) Depiction of three factors contributing to decorrelation between two caricatured neural responses: event timing, sparseness and noise.

  4. Decorrelation and efficient coding in the LNP model.
    Figure 4: Decorrelation and efficient coding in the LNP model.

    (a) Schematic of two visual neurons that each respond according to the LNP model. For each cell (top and bottom), the stimulus is processed by a linear filter that includes lateral inhibition in space. This signal is passed through a sigmoid nonlinearity and the result modulates the rate of a Poisson process that generates spikes; the spike counts in discrete time windows are the response variable. (b) Four sample nonlinearities with sigmoid shape and high or low gain (solid or dashed lines), high or low threshold (thick or thin lines), and various peak rates. The shaded curve indicates the probability distribution of the filtered stimulus signal at the input to the nonlinearity. (c) The effects of such a nonlinear transform on the correlations between two jointly Gaussian variables. Note that the output correlation is always less than that of the input. A low threshold (thin lines) affects the correlation only weakly, but at high threshold (thick lines) the output correlation is greatly reduced, especially for negative values. The precise shape of the nonlinearity (dashed versus solid) is less important and the peak rate has no effect. (d) The ratio of output correlation to input correlation decreases with increasing threshold, shown here for the sigmoid nonlinearity applied to two variables with input correlation C = ±0.6. (e) When the two outputs are affected by independent additive noise, this reduces the output correlation by a factor determined by the signal-to-noise ratio (equation (14)). (f) Influence of the nonlinearity on information transmission. In the framework of the LNP model, the threshold and gain of the sigmoid nonlinearity determine how much information about the stimulus is transmitted by the spikes (grayscale and contour lines). The average firing rate was fixed at 1.1 Hz (the median over the salamander ganglion cells). Threshold and 1/gain are measured in s.d. of the input signal distribution. Insets illustrate nonlinearities (solid lines) at different thresholds and gains relative to the input distribution (shaded area). (g,h) When multiple neurons receive correlated inputs, raising the threshold makes their outputs more redundant (g) even as the total information increases (h) and correlation decreases (d). All neurons had pairwise correlation coefficients of 0.9, equal thresholds, optimal (infinite) gain and a fixed mean firing rate of 1.1 Hz. The optimal threshold varies only weakly with population size (N = 1,...,8).

  5. Efficiency of stimulus coding by RGCs.
    Figure 5: Efficiency of stimulus coding by RGCs.

    (a) Cumulative distribution of the spike count in 50-ms time bins, averaged over multiple repeats of the stimulus. Data (thin lines) for three sample ganglion cells and their fit with a model (thick lines) parametrized by θ, g and K (equation (20)). (b) The information transmitted by model firing rate distributions with a fixed mean firing rate of 1.1 Hz, whose shape is parametrized by θ and g. Noise was assumed to be sub-Poisson as observed empirically (equation (11), Supplementary Fig. 3). The blue dot indicates the globally maximal rate of information transmission at this mean rate. Red dots indicate the parameters of the rate distribution measured from salamander ganglion cells. These cells have widely varying mean firing rates. The contour plot of information transmission varies slightly with mean rate, but is shown here for illustration purposes only at one typical mean rate. (c) Histogram of information efficiencies over the population of salamander RGCs. For each cell, the information rate is calculated directly from the empirical spike counts. To calculate efficiency, we compared this information rate to the maximal information rate possible for the measured mean firing rate (Online Methods). (d) Information transmission estimated for macaque RGCs, displayed as in b. Red dots are parameters describing the firing rate distribution obtained from published spike rasters in response to white noise stimulation21. The contour plot shows the information transmission for different firing rate distributions while fixing the mean rate and time window to typical values, namely 30 Hz and 10 ms, respectively.

References

  1. Attneave, F. Some informational aspects of visual perception. Psychol. Rev. 61, 183193 (1954).
  2. Barlow, H.B. Possible principles underlying the transformation of sensory messages. in Sensory Communication (ed. Rosenblith, W.A.) 217234 (MIT Press, Cambridge, MA, 1961).
  3. Srinivasan, M.V., Laughlin, S.B. & Dubs, A. Predictive coding: a fresh view of inhibition in the retina. Proc. R. Soc. Lond. B Biol. Sci. 216, 427459 (1982).
  4. Atick, J.J. & Redlich, A.N. What does the retina know about natural scenes? Neural Comput. 4, 196210 (1992).
  5. Atick, J.J. & Redlich, A.N. Convergent algorithm for sensory receptive field development. Neural Comput. 5, 4560 (1993).
  6. Atick, J.J. & Redlich, A.N. Could information theory provide an ecological theory of sensory processing? Network 3, 213251 (1992).
  7. van Hateren, J.H. Real and optimal neural images in early vision. Nature 360, 6870 (1992).
  8. van Hateren, J.H. Spatiotemporal contrast sensitivity of early vision. Vision Res. 33, 257267 (1993).
  9. Field, D.J. Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, 23792394 (1987).
  10. Atick, J.J. & Redlich, A.N. Toward a theory of early visual processing. Neural Comput. 2, 308320 (1990).
  11. Dan, Y., Atick, J.J. & Reid, R.C. Efficient coding of natural scenes in the lateral geniculate nucleus: experimental test of a computational theory. J. Neurosci. 16, 33513362 (1996).
  12. Puchalla, J.L., Schneidman, E., Harris, R.A. & Berry, M.J. Redundancy in the population code of the retina. Neuron 46, 493504 (2005).
  13. Chichilnisky, E.J. A simple white noise analysis of neuronal light responses. Network 12, 199213 (2001).
  14. Warland, D.K., Reinagel, P. & Meister, M. Decoding visual information from a population of retinal ganglion cells. J. Neurophysiol. 78, 23362350 (1997).
  15. Segev, R., Puchalla, J. & Berry, M.J. Functional organization of ganglion cells in the salamander retina. J. Neurophysiol. 95, 22772292 (2006).
  16. Enroth–Cugell, C. & Robson, J.G. Functional characteristics and diversity of cat retinal ganglion cells. Basic characteristics and quantitative description. Invest. Ophthalmol. Vis. Sci. 25, 250267 (1984).
  17. Berry, M.J. & Meister, M. Refractoriness and neural precision. J. Neurosci. 18, 22002211 (1998).
  18. Burrone, J. & Lagnado, L. Synaptic depression and the kinetics of exocytosis in retinal bipolar cells. J. Neurosci. 20, 568578 (2000).
  19. Demb, J.B., Zaghloul, K., Haarsma, L. & Sterling, P. Bipolar cells contribute to nonlinear spatial summation in the brisk-transient (Y) ganglion cell in mammalian retina. J. Neurosci. 21, 74477454 (2001).
  20. Field, G.D. & Rieke, F. Nonlinear signal transfer from mouse rods to bipolar cells and implications for visual sensitivity. Neuron 34, 773785 (2002).
  21. Uzzell, V.J. & Chichilnisky, E.J. Precision of spike trains in primate retinal ganglion cells. J. Neurophysiol. 92, 780789 (2004).
  22. Pillow, J.W. et al. Spatio-temporal correlations and visual signaling in a complete neuronal population. Nature 454, 995999 (2008).
  23. Chichilnisky, E.J. & Kalmar, R.S. Functional asymmetries in ON and OFF ganglion cells of primate retina. J. Neurosci. 22, 27372747 (2002).
  24. Croner, L.J., Purpura, K. & Kaplan, E. Response variability in retinal ganglion cells of primates. Proc. Natl. Acad. Sci. USA 90, 81288130 (1993).
  25. Schwartz, O., Pillow, J.W., Rust, N.C. & Simoncelli, E.P. Spike-triggered neural characterization. J. Vis. 6, 484507 (2006).
  26. Lancaster, H.O. Some properties of the bivariate normal distribution considered in the form of a contingency table. Biometrika 44, 289292 (1957).
  27. de la Rocha, J., Doiron, B., Shea–Brown, E., Josic, K. & Reyes, A. Correlation between neural spike trains increases with firing rate. Nature 448, 802806 (2007).
  28. Berry, M.J., Warland, D.K. & Meister, M. The structure and precision of retinal spike trains. Proc. Natl. Acad. Sci. USA 94, 54115416 (1997).
  29. Reinagel, P. How do visual neurons respond in the real world? Curr. Opin. Neurobiol. 11, 437442 (2001).
  30. Baccus, S.A. & Meister, M. Fast and slow contrast adaptation in retinal circuitry. Neuron 36, 909919 (2002).
  31. Stein, R.B. The information capacity of nerve cells using a frequency code. Biophys. J. 7, 797826 (1967).
  32. Shamai, S. Capacity of a pulse amplitude modulated direct detection photon channel. IEE Proc. Commun. Speech Vis. 137, 424430 (1990).
  33. Keat, J., Reinagel, P., Reid, R.C. & Meister, M. Predicting every spike: a model for the responses of visual neurons. Neuron 30, 803817 (2001).
  34. Balasubramanian, V. & Berry, M.J. A test of metabolically efficient coding in the retina. Network 13, 531552 (2002).
  35. Hosoya, T., Baccus, S.A. & Meister, M. Dynamic predictive coding by the retina. Nature 436, 7177 (2005).
  36. Croner, L.J. & Kaplan, E. Receptive fields of P and M ganglion cells across the primate retina. Vision Res. 35, 724 (1995).
  37. Barlow, H.B. & Levick, W.R. The mechanism of directionally selective units in rabbit′s retina. J. Physiol. (Lond.) 178, 477504 (1965).
  38. Ölveczky, B.P., Baccus, S.A. & Meister, M. Segregation of object and background motion in the retina. Nature 423, 401408 (2003).
  39. Levick, W.R. Receptive fields and trigger features of ganglion cells in the visual streak of the rabbits retina. J. Physiol. (Lond.) 188, 285307 (1967).
  40. Gollisch, T. & Meister, M. Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron 65, 150164 (2010).
  41. Dacey, D.M. Origins of perception: retinal ganglion cell diversity and the creation of parallel visual pathways. in The Cognitive Neurosciences (ed. Gazzaniga, M.S.) 281301 (MIT Press, Cambridge, Massachusetts, 2004).
  42. Laughlin, S.B. A simple coding procedure enhances a neuron′s information capacity. Z. Naturforsch. C 36c, 910912 (1981).
  43. Olshausen, B.A. & Field, D.J. Sparse coding of sensory inputs. Curr. Opin. Neurobiol. 14, 481487 (2004).
  44. Ringach, D.L. & Malone, B.J. The operating point of the cortex: neurons as large deviation detectors. J. Neurosci. 27, 76737683 (2007).
  45. van Vreeswijk, C.A. Whence sparseness? Adv. Neural Inf. Process. Syst. 13, 189195 (2001).
  46. Vinje, W.E. & Gallant, J.L. Sparse coding and decorrelation in primary visual cortex during natural vision. Science 287, 12731276 (2000).
  47. Wang, X.J., Liu, Y., Sanchez–Vives, M.V. & McCormick, D.A. Adaptation and temporal decorrelation by single neurons in the primary visual cortex. J. Neurophysiol. 89, 32793293 (2003).
  48. Rucci, M. & Casile, A. Fixational instability and natural image statistics: implications for early visual representations. Network 16, 121138 (2005).
  49. Cleland, T.A. Early transformations in odor representation. Trends Neurosci. 33, 130139 (2010).
  50. Wiechert, M.T., Judkewitz, B., Riecke, H. & Friedrich, R.W. Mechanisms of pattern decorrelation by recurrent neuronal circuits. Nat. Neurosci. 13, 10031010 (2010).
  51. Meister, M., Pine, J. & Baylor, D.A. Multi–neuronal signals from the retina: acquisition and analysis. J. Neurosci. Methods 51, 95106 (1994).
  52. Segev, R., Puchalla, J. & Berry, M.J. Functional organization of ganglion cells in the salamander retina. J. Neurophysiol. 95, 22772292 (2006).
  53. Himstedt, W. Prey selection in salamanders. in Analysis of Visual Behavior (eds. Ingale, D.J., Goodale, M.A. & Mansfield, R.J.W.) 4766 (MIT Press, Cambridge, Massachusetts, 1982).
  54. Dong, D.W. & Atick, J.J. Statistics of natural time-varying images. Network 6, 345358 (1995).
  55. Croner, L.J. & Kaplan, E. Receptive fields of P and M ganglion cells across the primate retina. Vision Res. 35, 724 (1995).
  56. Chichilnisky, E.J. & Kalmar, R.S. Functional asymmetries in ON and OFF ganglion cells of primate retina. J. Neurosci. 22, 27372747 (2002).
  57. Berry, M.J. & Meister, M. Refractoriness and neural precision. J. Neurosci. 18, 22002211 (1998).
  58. Uzzell, V.J. & Chichilnisky, E.J. Precision of spike trains in primate retinal ganglion cells. J. Neurophysiol. 92, 780789 (2004).
  59. Schneidman, E., Bialek, W. & Berry, M.J. Synergy, redundancy and independence in population codes. J. Neurosci. 23, 1153911553 (2003).
  60. Pillow, J.W. et al. Spatio–temporal correlations and visual signaling in a complete neuronal population. Nature 454, 995999 (2008).

Download references

Author information

Affiliations

  1. Department of Brain and Cognitive Sciences, University of Rochester, Rochester, New York, USA.

    • Xaq Pitkow
  2. Molecular and Cellular Biology, Center for Brain Science, Harvard University, Cambridge, Massachusetts, USA.

    • Markus Meister

Contributions

X.P. and M.M. designed the study. X.P. performed all of the experiments, analysis and modeling. X.P. and M.M. wrote the article.

Competing financial interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to:

Author details

Supplementary information

PDF files

  1. Supplementary Text and Figures (516K)

    Supplementary Figures 1–3

Additional data