Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Inferring learning rules from distributions of firing rates in cortical neurons

Nature Neuroscience volume 18, pages 1804–1810 (2015)Cite this article

Subjects

Abstract

Information about external stimuli is thought to be stored in cortical circuits through experience-dependent modifications of synaptic connectivity. These modifications of network connectivity should lead to changes in neuronal activity as a particular stimulus is repeatedly encountered. Here we ask what plasticity rules are consistent with the differences in the statistics of the visual response to novel and familiar stimuli in inferior temporal cortex, an area underlying visual object recognition. We introduce a method that allows one to infer the dependence of the presumptive learning rule on postsynaptic firing rate, and we show that the inferred learning rule exhibits depression for low postsynaptic rates and potentiation for high rates. The threshold separating depression from potentiation is strongly correlated with both mean and s.d. of the firing rate distribution. Finally, we show that network models implementing a rule extracted from data show stable learning dynamics and lead to sparser representations of stimuli.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Visual response of inferior temporal cortical (ITC) neurons to novel and familiar stimuli.
Figure 2: Inferring learning rules from distributions of firing rates.
Figure 3: Effect of visual experience in ITC neurons and their dependence on different cell types.
Figure 4: Effect of visual experience in individual ITC neurons and regulation of learning rules.
Figure 5: Comparison between simulated and experimental data.

Similar content being viewed by others

References

  1. Hebb, D.O. The Organization of Behavior (Wiley, 1949).

  2. Bi, G.Q. & Poo, M.M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464–10472 (1998).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Bliss, T.V. & Lomo, T. Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. J. Physiol. (Lond.) 232, 331–356 (1973).

    Article  CAS  Google Scholar 

  4. Dudek, S.M. & Bear, M.F. Homosynaptic long-term depression in area CA1 of hippocampus and effects of N-methyl-D-aspartate receptor blockade. Proc. Natl. Acad. Sci. USA 89, 4363–4367 (1992).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  5. Markram, H., Lubke, J., Frotscher, M. & Sakmann, B. Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs. Science 275, 213–215 (1997).

    Article  CAS  PubMed  Google Scholar 

  6. Sjöström, P.J., Turrigiano, G.G. & Nelson, S.B. Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron 32, 1149–1164 (2001).

    Article  PubMed  Google Scholar 

  7. Dan, Y. & Poo, M.M. Spike timing-dependent plasticity: from synapse to perception. Physiol. Rev. 86, 1033–1048 (2006).

    Article  PubMed  Google Scholar 

  8. Feldman, D.E. Synaptic mechanisms for plasticity in neocortex. Annu. Rev. Neurosci. 32, 33–55 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Fox, K. & Wong, R.O. A comparison of experience-dependent plasticity in the visual and somatosensory systems. Neuron 48, 465–477 (2005).

    Article  CAS  PubMed  Google Scholar 

  10. Dayan, P. & Abbott,, L.F. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems (MIT Press, 2005).

  11. Gerstner, W. & Kistler, W.M. Spiking Neuron Models: Single Neurons, Populations, Plasticity (Cambridge Univ. Press, 2002).

  12. Bienenstock, E.L., Cooper, L.N. & Munro, P.W. Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2, 32–48 (1982).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Song, S., Miller, K.D. & Abbott, L.F. Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nat. Neurosci. 3, 919–926 (2000).

    Article  CAS  PubMed  Google Scholar 

  14. Amit, D.J. & Brunel, N. Model of global spontaneous activity and local structured activity during delay periods in the cerebral cortex. Cereb. Cortex 7, 237–252 (1997).

    Article  CAS  PubMed  Google Scholar 

  15. Sohal, V.S. & Hasselmo, M.E. A model for experience-dependent changes in the responses of inferotemporal neurons. Network 11, 169–190 (2000).

    Article  CAS  PubMed  Google Scholar 

  16. Miyashita, Y. Inferior temporal cortex: where visual perception meets memory. Annu. Rev. Neurosci. 16, 245–263 (1993).

    Article  CAS  PubMed  Google Scholar 

  17. Tanaka, K. Inferotemporal cortex and object vision. Annu. Rev. Neurosci. 19, 109–139 (1996).

    Article  CAS  PubMed  Google Scholar 

  18. Kobatake, E., Wang, G. & Tanaka, K. Effects of shape-discrimination training on the selectivity of inferotemporal cells in adult monkeys. J. Neurophysiol. 80, 324–330 (1998).

    Article  CAS  PubMed  Google Scholar 

  19. Li, L., Miller, E.K. & Desimone, R. The representation of stimulus familiarity in anterior inferior temporal cortex. J. Neurophysiol. 69, 1918–1929 (1993).

    Article  CAS  PubMed  Google Scholar 

  20. Logothetis, N.K., Pauls, J. & Poggio, T. Shape representation in the inferior temporal cortex of monkeys. Curr. Biol. 5, 552–563 (1995).

    Article  CAS  PubMed  Google Scholar 

  21. Freedman, D.J., Riesenhuber, M., Poggio, T. & Miller, E.K. Experience-dependent sharpening of visual shape selectivity in inferior temporal cortex. Cereb. Cortex 16, 1631–1644 (2006).

    Article  PubMed  Google Scholar 

  22. Woloszyn, L. & Sheinberg, D.L. Effects of long-term visual experience on responses of distinct classes of single units in inferior temporal cortex. Neuron 74, 193–205 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Meyer, T., Walker, C., Cho, R.Y. & Olson, C.R. Image familiarization sharpens response dynamics of neurons in inferotemporal cortex. Nat. Neurosci. 17, 1388–1394 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  24. Op de Beeck, H.P., Wagemans, J. & Vogels, R. Effects of perceptual learning in visual backward masking on the responses of macaque inferior temporal neurons. Neuroscience 145, 775–789 (2007).

    Article  CAS  PubMed  Google Scholar 

  25. McKee, J.L., Thomas, S.L. & Freedman, D.J. Soc. Neurosci. Abstr. 65.16/HH18 (2013).

  26. Buzsáki, G. & Mizuseki, K. The log-dynamic brain: how skewed distributions affect network operations. Nat. Rev. Neurosci. 15, 264–278 (2014).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  27. Hromádka, T., Deweese, M.R. & Zador, A.M. Sparse representation of sounds in the unanesthetized auditory cortex. PLoS Biol. 6, e16 (2008).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  28. Roxin, A., Brunel, N., Hansel, D., Mongillo, G. & van Vreeswijk, C. On the distribution of firing rates in networks of cortical neurons. J. Neurosci. 31, 16217–16226 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Anderson, J.S., Lampl, I., Gillespie, D.C. & Ferster, D. The contribution of noise to contrast invariance of orientation tuning in cat visual cortex. Science 290, 1968–1972 (2000).

    Article  CAS  PubMed  Google Scholar 

  30. Rauch, A., La Camera, G., Luscher, H.R., Senn, W. & Fusi, S. Neocortical pyramidal cells respond as integrate-and-fire neurons to in vivo-like input currents. J. Neurophysiol. 90, 1598–1612 (2003).

    Article  PubMed  Google Scholar 

  31. Kirkwood, A., Rioult, M.C. & Bear, M.F. Experience-dependent modification of synaptic plasticity in visual cortex. Nature 381, 526–528 (1996).

    Article  CAS  PubMed  Google Scholar 

  32. Toyoizumi, T., Kaneko, M., Stryker, M.P. & Miller, K.D. Modeling the dynamic interaction of Hebbian and homeostatic plasticity. Neuron 84, 497–510 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Zenke, F., Hennequin, G. & Gerstner, W. Synaptic plasticity in neural networks needs homeostasis with a fast rate detector. PLoS Comput. Biol. 9, e1003330 (2013).

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  34. Miller, K.D. & MacKay, D.J.C. The role of constraints in Hebbian learning. Neural Comput. 6, 100–126 (1994).

    Article  Google Scholar 

  35. Bourne, J.N. & Harris, K.M. Coordination of size and number of excitatory and inhibitory synapses results in a balanced structural plasticity along mature hippocampal CA1 dendrites during LTP. Hippocampus 21, 354–373 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Amit, D.J. & Fusi, S. Learning in neural networks with material synapses. Neural Comput. 6, 957–982 (1994).

    Article  Google Scholar 

  37. Sejnowski, T.J. Storing covariance with nonlinearly interacting neurons. J. Math. Biol. 4, 303–321 (1977).

    Article  CAS  PubMed  Google Scholar 

  38. Cooper, L.N., Intrator, N., Blais, B.S. & Shouval, H.Z. Theory of Cortical Plasticity (World Scientific, 2004).

  39. Brunel, N. Hebbian learning of context in recurrent neural networks. Neural Comput. 8, 1677–1710 (1996).

    Article  CAS  PubMed  Google Scholar 

  40. Miyashita, Y. Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature 335, 817–820 (1988).

    Article  CAS  PubMed  Google Scholar 

  41. Bogacz, R. & Brown, M.W. Comparison of computational models of familiarity discrimination in the perirhinal cortex. Hippocampus 13, 494–524 (2003).

    Article  PubMed  Google Scholar 

  42. Norman, K.A. & O'Reilly, R.C. Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. Psychol. Rev. 110, 611–646 (2003).

    Article  PubMed  Google Scholar 

  43. Riesenhuber, M. & Poggio, T. Hierarchical models of object recognition in cortex. Nat. Neurosci. 2, 1019–1025 (1999).

    Article  CAS  PubMed  Google Scholar 

  44. Yamins, D.L. et al. Performance-optimized hierarchical models predict neural responses in higher visual cortex. Proc. Natl. Acad. Sci. USA 111, 8619–8624 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. DiCarlo, J.J., Zoccolan, D. & Rust, N.C. How does the brain solve visual object recognition? Neuron 73, 415–434 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  46. Tomita, H., Ohbayashi, M., Nakahara, K., Hasegawa, I. & Miyashita, Y. Top-down signal from prefrontal cortex in executive control of memory retrieval. Nature 401, 699–703 (1999).

    Article  CAS  PubMed  Google Scholar 

  47. Clopath, C., Busing, L., Vasilaki, E. & Gerstner, W. Connectivity reflects coding: a model of voltage-based STDP with homeostasis. Nat. Neurosci. 13, 344–352 (2010).

    Article  CAS  PubMed  Google Scholar 

  48. Graupner, M. & Brunel, N. Calcium-based plasticity model explains sensitivity of synaptic changes to spike pattern, rate, and dendritic location. Proc. Natl. Acad. Sci. USA 109, 3991–3996 (2012).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  49. Pfister, J.P. & Gerstner, W. Triplets of spikes in a model of spike timing-dependent plasticity. J. Neurosci. 26, 9673–9682 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Shouval, H.Z., Bear, M.F. & Cooper, L.N. A unified model of NMDA receptor-dependent bidirectional synaptic plasticity. Proc. Natl. Acad. Sci. USA 99, 10831–10836 (2002).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  51. McCormick, D.A., Connors, B.W., Lighthall, J.W. & Prince, D.A. Comparative electrophysiology of pyramidal and sparsely spiny stellate neurons of the neocortex. J. Neurophysiol. 54, 782–806 (1985).

    Article  CAS  PubMed  Google Scholar 

  52. Ibos, G. & Freedman, D.J. Dynamic integration of task-relevant visual features in posterior parietal cortex. Neuron 83, 1468–1480 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  53. Paxinos, G., Huang, X.F. & Toga, A.W. The Rhesus Monkey Brain in Stereotaxic Coordinates (Academic, 2000).

Download references

Acknowledgements

We thank S. Dieudonné and D. Higgins for discussions and Y. Aljadeff, K. Burbank, and M. de Pittà for feedback on the manuscript. D.L.S. has been supported by grants from the National Science Foundation (SBE-0542013) and the US National Institutes of Health (R01EY14681). D.J.F. has been supported by a NSF CAREER award, a McKnight Scholar award and the Alfred P. Sloan Foundation. J.L.M. is a recipient of a Natural Sciences and Engineering Research Council of Canada (NSERC) fellowship.

Author information

Authors and Affiliations

  1. Department of Neurobiology, University of Chicago, Chicago, Illinois, USA

    Sukbin Lim, Jillian L McKee, David J Freedman & Nicolas Brunel

  2. Department of Neuroscience, Columbia University, New York, New York, USA

    Luke Woloszyn

  3. Department of Statistics, University of Chicago, Chicago, Illinois, USA

    Yali Amit & Nicolas Brunel

  4. Department of Computer Science, University of Chicago, Chicago, Illinois, USA

    Yali Amit

  5. Department of Neuroscience, Brown University, Providence, Rhode Island, USA

    David L Sheinberg

Authors
  1. Sukbin Lim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jillian L McKee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luke Woloszyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yali Amit
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. David J Freedman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. David L Sheinberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nicolas Brunel
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

S.L., Y.A. and N.B. designed the research. S.L. analyzed the data, performed network simulations and prepared the figures. J.L.M., L.W., D.J.F. and D.L.S. contributed the electrophysiological data. S.L. and N.B. wrote the manuscript, with contributions from all authors.

Corresponding author

Correspondence to Nicolas Brunel.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Schematics of the method to infer learning rules.

From the distributions of firing rates (top), dependence of learning rules on post-synaptic rates (bottom) is obtained in 4 steps, shown in yellow boxes. Assumptions made at each step are shown in purple boxes.

Supplementary Figure 2 Transfer function and input changes with non-Gaussian distribution of input currents for novel stimuli.

a,c,e, Normalized Student’s t-distributions of input currents with different degrees of freedom ν (a), inferred transfer function (c), and input changes as a function of rates (e). b,d,f, Normalized Gamma distributions of input currents with different shape parameters k (b), transfer function (d), and input changes (f). Although the shape of transfer functions and input changes varies for the different input distributions, the derived input change is qualitatively similar, showing depression for low rates and potentiation for high rates without changing a threshold separating potentation from depression (e,f).

Supplementary Figure 3 Transfer functions and input changes derived under the assumption of a Gaussian distribution of input currents for novel stimuli and for familiar stimuli.

Transfer functions (a,c) and input changes (b,d) derived under the assumption of a Gaussian distribution of input currents for novel stimuli (a,b) and for familiar stimuli (c,d).

Supplementary Figure 4 Input changes computed under a relaxed rank-preservation assumption.

Black dots are the same as one in Fig. 2g, which was computed under the assumption that the rank of a stimuli among all stimuli is preserved with learning. We perturbed the rank of stimuli – we added noise to input currents to familiar stimuli and for each realization of noise, we recalculated the rank of the familiar stimuli. With the perturbed rank of the familiar stimuli (but without noise in input currents to familiar stimuli), we obtained difference of the input currents as with the unperturbed rank. Solid curves and shaded areas are mean and the standard deviation of input changes computed with 10,000 realization of noise (a–c). Standard deviation of noise added to input currents varied from 0.1 to 0.5 (0.1 in a, 0.2 in b and 0.5 in c) in the unit of the standard deviation of input currents for different stimuli.

Supplementary Figure 5 Transfer functions of individual excitatory and inhibitory neurons and their average with normalized firing rates.

Transfer functions of individual excitatory (a,c) and inhibitory (b,d) neurons and their average (e,f) with normalized firing rates. Dark-colored curves in a-d are example transfer functions, and the colored area in e and f represents the variability of the transfer functions over different neurons. Both in the excitatory and inhibitory neurons in c-f, the transfer functions with normalized firing rates are very similar to each other within each class.

Supplementary Figure 6 Simulation without a constraint on the sum of synaptic weights.

a,b, Distributions of normalized firing rates of excitatory (a) and inhibitory (b) neurons for novel (grey) and familiar (black) stimuli, obtained from the experiment (same as in Fig. 5a,b). c,d, Distributions of normalized firing rates for novel (red) and familiar (blue) stimuli, obtained from the simulation. The simulation implemented the same learning rule as in Fig. 5, but without the constraint on the total sum of synaptic strengths onto post-synaptic neurons. Due to too much depression in the synaptic strengths, the effect of learning one particular stimulus on the activity pattern becomes very small. e,f, Changes of 25th, 50th, 75th and 95th percentiles of normalized firing rates in the data (black) and in the simulation (red cross). Unlike the data showing decrease in activity at most percentiles, the simulation shows slight increase both in excitatory (e) and inhibitory (f) neurons.

Supplementary Figure 7 Time course of mean and maximal visual responses of ITC neurons obtained in a passive viewing task and simulation.

Time course of mean and maximal visual responses of ITC neurons obtained in a passive viewing task (a–d) and simulation (e-h). In the data, solid curves are activities averaged over all neurons and all novel (red) or familiar (blue) stimuli and error bars represent mean ± SEM of activities averaged over individual neurons (a-d). In the simulation, solid curves and error bars represent mean and mean ± SEM of network activities, respectively (e-h). The equations for the dynamics and parameters are the same as in Figure 5, except that to mimic the temporal profile of visual responses to novel stimuli, the external input is multiplied by a factor -3exp(-t/20ms)+2.5exp(-t/50ms)+0.5, where t is time after activity onset with a delay of 100ms in activity onset after the stimulus onset at time 0. The input-output transfer functions and the rate-dependent learning rule in the E-to-E connections are the same as in Figure 5, which were derived from firing rates averaged in the time window between 75 ms and 200 ms after stimulus onset (see Online Methods).

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–7 and Supplementary Note 1 (PDF 1080 kb)

Supplementary Methods Checklist (PDF 116 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lim, S., McKee, J., Woloszyn, L. et al. Inferring learning rules from distributions of firing rates in cortical neurons. Nat Neurosci 18, 1804–1810 (2015). https://doi.org/10.1038/nn.4158

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.4158

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing