Abstract
Population coding is widely regarded as an important mechanism for achieving reliable behavioral responses despite neuronal variability. However, standard reinforcement learning slows down with increasing population size, as the global reward signal becomes less and less related to the performance of any single neuron. We found that learning speeds up with increasing population size if, in addition to global reward, feedback about the population response modulates synaptic plasticity.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
214,86 € per year
only 17,91 € per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Pouget, A., Dayan, P. & Zemel, R. Nat. Rev. Neurosci. 1, 125–132 (2000).
Averbeck, B.B., Latham, P.E. & Pouget, A. Nat. Rev. Neurosci. 7, 358–366 (2006).
Seung, H.S. Neuron 40, 1063–1073 (2003).
Fiete, I.R. & Seung, H.S. Phys. Rev. Lett. 97, 048104 (2006).
Pfister, J.P., Toyoizumi, T., Barber, D. & Gerstner, W. Neural Comput. 18, 1318–1348 (2006).
Florian, R.V. Neural Comput. 19, 1468–1502 (2007).
Izhikevich, E.M. Cereb. Cortex 17, 2443–2452 (2007).
Wirth, S. et al. Science 300, 1578–1581 (2003).
Rumelhart, D.E., Hinton, G.E. & Williams, R.J.D. Parallel Distributed Processing, Vol. 1 (eds. Rumelhart, E. & McClelland, J.L.) (MIT Press, Cambridge, Massachusetts, 1986).
Foehring, R.C. & Lorenzon, N.M. Can. J. Exp. Psychol. 53, 45–61 (1999).
Centonze, D., Gubellini, P., Pisani, A., Bernardi, G. & Calabresi, P. Rev. Neurosci. 14, 207–216 (2003).
Gütig, R. & Sompolinsky, H. Nat. Neurosci. 9, 420–428 (2006).
Acknowledgements
We thank W. Gerstner for helpful comments on the manuscript.
Author information
Authors and Affiliations
Contributions
R.U. designed and executed the research in close interaction with W.S. The manuscript was written jointly.
Corresponding author
Supplementary information
Supplementary Text and Figures
Supplementary Figure 1 and Supplementary Methods (PDF 553 kb)
Rights and permissions
About this article
Cite this article
Urbanczik, R., Senn, W. Reinforcement learning in populations of spiking neurons. Nat Neurosci 12, 250–252 (2009). https://doi.org/10.1038/nn.2264
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.2264
