Sparse coding and high-order correlations in fine-scale cortical networks

Journal name:
Date published:
Published online

Connectivity in the cortex is organized at multiple scales1, 2, 3, 4, 5, suggesting that scale-dependent correlated activity is particularly important for understanding the behaviour of sensory cortices and their function in stimulus encoding. We analysed the scale-dependent structure of cortical interactions by using maximum entropy models6, 7, 8, 9 to characterize multiple-tetrode recordings from primary visual cortex of anaesthetized macaque monkeys (Macaca mulatta). We compared the properties of firing patterns among local clusters of neurons (<300μm apart) with those of neurons separated by larger distances (600–2,500μm). Here we report that local firing patterns are distinctive: whereas multi-neuronal firing patterns at larger distances can be predicted by pairwise interactions, patterns within local clusters often show evidence of high-order correlations. Surprisingly, these local correlations are flexible and rapidly reorganized by visual input. Although they modestly reduce the amount of information that a cluster conveys, they also modify the format of this information, creating sparser codes by increasing the periods of total quiescence, and concentrating information into briefer periods of common activity. These results imply a hierarchical organization of neuronal correlations: simple pairwise correlations link neurons over scales of tens to hundreds of minicolumns, but on the scale of a few minicolumns, ensembles of neurons form complex subnetworks whose moment-to-moment effective connectivity is dynamically reorganized by the stimulus.

At a glance


  1. Analysis of multi-tetrode recordings.
    Figure 1: Analysis of multi-tetrode recordings.

    a, Multi-tetrode recording geometry: red dots indicate tetrode centres; grey circles indicate approximate recording areas of each tetrode. b, Visual stimuli consisted of pseudorandom chequerboards. c, Extracellularly recorded spikes are sorted and binned (10- or 15-ms bin width). Multi-neuron firing patterns are identified and tallied to form a firing pattern distribution. Maximum entropy models are fitted to this distribution.

  2. A pairwise maximum entropy model frequently fails for local networks of neurons.
    Figure 2: A pairwise maximum entropy model frequently fails for local networks of neurons.

    a, The independent model (Mind, ordinate) fails to predict the observed (Mobs, abscissa) frequency of firing patterns over a range of spatial scales. b, Box plots (inter-quartile range) of log2 likelihood ratios (LLRs) of the independent model (Mind), referenced to a perfect model (Mobs) for 60s of data. Notches indicate median values; outliers plotted as plus signs. Across spatial scales, Mind is a poor fit. c, The pairwise model (Mpair, ordinate) fails to predict the observed (Mobs, abscissa) frequency of firing patterns for ensembles of neurons separated by <300μm (red dots), but not at larger separations. d, Box plots of LLRs of the pairwise model (Mpair), referenced to a perfect model (Mobs). At distances <300μm (red), Mpair frequently fails to account for the observed distribution of firing patterns; at larger separations, Mpair is nearly perfect.

  3. High-order stimulus dependent correlations in multi-neuron firing patterns.
    Figure 3: High-order stimulus dependent correlations in multi-neuron firing patterns.

    a, The Jensen-Shannon divergence between firing pattern distributions conditional on each pixel (insets on right) yields a spatiotemporal map of mutual information between the stimulus and the population response. For a maximally informative pixel, Mpair is fitted to these conditional distributions. b, Conditioning on maximally informative pixels (blue dots) has a large effect on the goodness of fit of Mpair; conditioning on random pixels (red dots) has a small effect. For maximally-informative pixels, confidence intervals (95%, see Methods) are smaller than the plotted symbols. For random pixels, the error bars show 95% confidence intervals for the mean. c, Expanded view of b, near the origin. d, Histogram of the change in average interaction strength (Jij) following conditioning. Conditioning on informative pixels (blue bars), but not random pixels (red bars) lead to significantly different interaction strengths.

  4. Effect of correlations on stimulus encoding.
    Figure 4: Effect of correlations on stimulus encoding.

    a, Mutual information between the stimulus (maximally informative pixel) and neural response generated by Mobs (Iobs; ordinate) and Mpair (Ipair; abscissa) for neurons recorded at <300μm. Higher-than-second order correlations make no contribution to the mutual information transmitted. b, Pairwise correlations slightly reduce the information encoded by local ensembles; for most sites, Ipair is slightly smaller than Iind. c, At distances <300μm (red), neuronal activity (Mobs) has a higher frequency of the all-silent firing pattern than predicted from independent firing (Mind), indicating that correlations in neuronal firing increase the sparseness of local networks. No difference was seen at larger distances (600μm (blue) or >1,000μm (grey), confluent on the diagonal). d, The effect of correlations on the information encoded when the network is active. On average, correlations increase the information transmitted when the network is active.


  1. Yoshimura, Y. & Callaway, E. M. Fine-scale specificity of cortical networks depends on inhibitory cell type and connectivity. Nature Neurosci. 8, 15521559 (2005)
  2. Yoshimura, Y., Dantzker, J. L. & Callaway, E. M. Excitatory cortical neurons form fine-scale functional networks. Nature 433, 868873 (2005)
  3. Song, S., Sjostrom, P. J., Reigl, M., Nelson, S. & Chklovskii, D. B. Highly nonrandom features of synaptic connectivity in local cortical circuits. PLoS Biol. 3, e68 (2005)
  4. Hubel, D. H. & Wiesel, T. N. Sequence regularity and geometry of orientation columns in the monkey striate cortex. J. Comp. Neurol. 158, 267293 (1974)
  5. Das, A. & Gilbert, C. D. Long-range horizontal connections and their role in cortical reorganization revealed by optical recording of cat primary visual cortex. Nature 375, 780784 (1995)
  6. Shlens, J. et al. The structure of multi-neuron firing patterns in primate retina. J. Neurosci. 26, 82548266 (2006)
  7. Tang, A. et al. A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro . J. Neurosci. 28, 505518 (2008)
  8. Schneidman, E., Berry, M. J. II, Segev, R. & Bialek, W. Weak pairwise correlations imply strongly correlated network states in a neural population. Nature 440, 10071012 (2006)
  9. Yu, S., Huang, D., Singer, W. & Nikolic, D. A small world of neuronal synchrony. Cereb. Cortex 18, 28912901 (2008)
  10. Aertsen, A. M., Gerstein, G. L., Habib, M. K. & Palm, G. Dynamics of neuronal firing correlation: modulation of “effective connectivity”. J. Neurophysiol. 61, 900917 (1989)
  11. Smith, M. A. & Kohn, A. Spatial and temporal scales of neuronal correlation in primary visual cortex. J. Neurosci. 28, 1259112603 (2008)
  12. Buzsaki, G. Large-scale recording of neuronal ensembles. Nature Neurosci. 7, 446451 (2004)
  13. Thomson, M. G. Beats, kurtosis and visual coding. Network 12, 271287 (2001)
  14. Samonds, J. M., Zhou, Z., Bernard, M. R. & Bonds, A. B. Synchronous activity in cat visual cortex encodes collinear and cocircular contours. J. Neurophysiol. 95, 26022616 (2006)
  15. Puchalla, J. L., Schneidman, E., Harris, R. A. & Berry, M. J. Redundancy in the population code of the retina. Neuron 46, 493504 (2005)
  16. Reich, D. S., Mechler, F. & Victor, J. D. Independent and redundant information in nearby cortical neurons. Science 294, 25662568 (2001)
  17. Gawne, T. J., Kjaer, T. W., Hertz, J. A. & Richmond, B. J. Adjacent visual cortical complex cells share about 20% of their stimulus-related information. Cereb. Cortex 6, 482489 (1996)
  18. Gawne, T. J. & Richmond, B. J. How independent are the messages carried by adjacent inferior temporal cortical neurons? J. Neurosci. 13, 27582771 (1993)
  19. Gray, C. M., Konig, P., Engel, A. K. & Singer, W. Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature 338, 334337 (1989)
  20. Meister, M. Multineuronal codes in retinal signaling. Proc. Natl Acad. Sci. USA 93, 609614 (1996)
  21. Cover, T. M. & Thomas, J. A. Elements of Information Theory (Wiley, 1991)
  22. Nirenberg, S., Carcieri, S. M., Jacobs, A. L. & Latham, P. E. Retinal ganglion cells act largely as independent encoders. Nature 411, 698701 (2001)
  23. Mountcastle, V. B. Modality and topographic properties of single neurons of cat's somatic sensory cortex. J. Neurophysiol. 20, 408434 (1957)
  24. Mountcastle, V. B. The columnar organization of the neocortex. Brain 120, 701722 (1997)
  25. Henze, D. A. et al. Intracellular features predicted by extracellular recordings in the hippocampus in vivo . J. Neurophysiol. 84, 390400 (2000)
  26. Stevens, C. F. & Wang, Y. Facilitation and depression at single central synapses. Neuron 14, 795802 (1995)
  27. Victor, J. D., Mechler, F., Repucci, M. A., Purpura, K. P. & Sharpee, T. Responses of V1 neurons to two-dimensional hermite functions. J. Neurophysiol. 95, 379400 (2006)
  28. Reid, R. C., Victor, J. D. & Shapley, R. M. The use of m-sequences in the analysis of visual neurons: linear receptive field properties. Vis. Neurosci. 14, 10151027 (1997)
  29. Fee, M. S., Mitra, P. P. & Kleinfeld, D. Automatic sorting of multiple unit neuronal signals in the presence of anisotropic and non-Gaussian variability. J. Neurosci. Methods 69, 175188 (1996)
  30. Kennel, M. B., Shlens, J., Abarbanel, H. D. & Chichilnisky, E. J. Estimating entropy rates with Bayesian confidence intervals. Neural Comput. 17, 15311576 (2005)

Download references

Author information


  1. Department of Neurology and Neuroscience, Weill Cornell Medical College, New York, New York 10065, USA

    • Ifije E. Ohiorhenuan,
    • Ferenc Mechler,
    • Keith P. Purpura,
    • Anita M. Schmid,
    • Qin Hu &
    • Jonathan D. Victor


I.E.O. conceived the project and carried out the data analysis. I.E.O. and J.D.V. wrote the manuscript. J.D.V. supervised the project. I.E.O., F.M., K.P.P., A.M.S., Q.H. and J.D.V. collected experimental data. F.M., K.P.P. and A.M.S. provided feedback on the manuscript.

Competing financial interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to:

Author details

Supplementary information

PDF files

  1. Supplementary Information (753K)

    This file contains Supplementary Methods and Supplementary Figures 1-7 with legends.

Additional data