Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks

Nature Neuroscience volume 24, pages 129–139 (2021)Cite this article

Subjects

Abstract

Cortical neurons process information on multiple timescales, and areas important for working memory (WM) contain neurons capable of integrating information over a long timescale. However, the underlying mechanisms for the emergence of neuronal timescales stable enough to support WM are unclear. By analyzing a spiking recurrent neural network model trained on a WM task and activity of single neurons in the primate prefrontal cortex, we show that the temporal properties of our model and the neural data are remarkably similar. Dissecting our recurrent neural network model revealed strong inhibitory-to-inhibitory connections underlying a disinhibitory microcircuit as a critical component for long neuronal timescales and WM maintenance. We also found that enhancing inhibitory-to-inhibitory connections led to more stable temporal dynamics and improved task performance. Finally, we show that a network with such microcircuitry can perform other tasks without disrupting its pre-existing timescale architecture, suggesting that strong inhibitory signaling underlies a flexible WM network.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: RNN model and experimental data.
Fig. 2: RNN model trained on the DMS task and the dlPFC data contain units with long timescales.
Fig. 3: Long τ units maintain cue stimulus information during the delay period robustly.
Fig. 4: Inhibitory synaptic weights lead to task-specific timescales.
Fig. 5: I → I connectivity strength strongly mediates both neuronal timescales and task performance.
Fig. 6: Two oppositely tuned inhibitory subgroups mutually inhibit each other for WM maintenance.
Fig. 7: High trial-to-trial spike-count variability during fixation corresponds to long neuronal timescale.
Fig. 8: Strong I → I connections might be intrinsic to prefrontal cortex.

Similar content being viewed by others

Data availability

The trained RNN models used in the present study are deposited as MATLAB-formatted data in Open Science Framework, https://osf.io/md4wg. The experimental data used in the study can be obtained from Constantinidis et al.21.

Code availability

The code for the analyses performed in this work is available at https://github.com/rkim35/wmRNN.

References

  1. Murray, J. D. et al. A hierarchy of intrinsic timescales across primate cortex. Nat. Neurosci. 17, 1661–1663 (2014).

    Article  CAS  Google Scholar 

  2. Chaudhuri, R., Knoblauch, K., Gariel, M.-A., Kennedy, H. & Wang, X.-J. A large-scale circuit mechanism for hierarchical dynamical processing in the primate cortex. Neuron 88, 419–431 (2015).

    Article  CAS  Google Scholar 

  3. Cavanagh, S. E., Wallis, J. D., Kennerley, S. W. & Hunt, L. T. Autocorrelation structure at rest predicts value correlates of single neurons during reward-guided choice. eLife 5, e18937 (2016).

    Article  Google Scholar 

  4. Miller, E. K., Erickson, C. A. & Desimone, R. Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167 (1996).

    Article  CAS  Google Scholar 

  5. Fuster, J. M. & Alexander, G. E. Neuron activity related to short-term memory. Science 173, 652–654 (1971).

    Article  CAS  Google Scholar 

  6. Fascianelli, V., Tsujimoto, S., Marcos, E. & Genovesio, A. Autocorrelation structure in the macaque dorsolateral, but not orbital or polar, prefrontal cortex predicts response-coding strength in a visually cued strategy task. Cereb. Cortex 29, 230–241 (2017).

    Article  Google Scholar 

  7. Cavanagh, S. E., Towers, J. P., Wallis, J. D., Hunt, L. T. & Kennerley, S. W. Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex. Nat. Commun. 9, 3498 (2018).

  8. Wasmuht, D. F., Spaak, E., Buschman, T. J., Miller, E. K. & Stokes, M. G. Intrinsic neuronal dynamics predict distinct functional roles during working memory. Nat. Commun. 9, 3499 (2018).

    Article  CAS  Google Scholar 

  9. Kim, D. et al. Distinct roles of parvalbumin- and somatostatin-expressing interneurons in working memory. Neuron 92, 902–915 (2016).

    Article  CAS  Google Scholar 

  10. Kamigaki, T. & Dan, Y. Delay activity of specific prefrontal interneuron subtypes modulates memory-guided behavior. Nat. Neurosci. 20, 854–863 (2017).

    Article  CAS  Google Scholar 

  11. Xu, H. et al. A disinhibitory microcircuit mediates conditioned social fear in the prefrontal cortex. Neuron 102, 668–682 (2019).

    Article  CAS  Google Scholar 

  12. Cummings, K. A. & Clem, R. L. Prefrontal somatostatin interneurons encode fear memory. Nat. Neurosci. 23, 61–74 (2019).

    Article  Google Scholar 

  13. Krabbe, S. et al. Adaptive disinhibitory gating by VIP interneurons permits associative learning. Nat. Neurosci. 22, 1834–1843 (2019).

    Article  CAS  Google Scholar 

  14. Pfeffer, C. K., Xue, M., He, M., Huang, Z. J. & Scanziani, M. Inhibition of inhibition in visual cortex: the logic of connections between molecularly distinct interneurons. Nat. Neurosci. 16, 1068–1076 (2013).

    Article  CAS  Google Scholar 

  15. Tremblay, R., Lee, S. & Rudy, B. GABAergic interneurons in the neocortex: from cellular properties to circuits. Neuron 91, 260–292 (2016).

    Article  CAS  Google Scholar 

  16. Medalla, M., Gilman, J. P., Wang, J.-Y. & Luebke, J. I. Strength and diversity of inhibitory signaling differentiates primate anterior cingulate from lateral prefrontal cortex. J. Neurosci. 37, 4717–4734 (2017).

    Article  CAS  Google Scholar 

  17. Mongillo, G., Rumpel, S. & Loewenstein, Y. Inhibitory connectivity defines the realm of excitatory plasticity. Nat. Neurosci. 21, 1463–1470 (2018).

    Article  CAS  Google Scholar 

  18. Kim, R., Li, Y. & Sejnowski, T. J. Simple framework for constructing functional spiking recurrent neural networks. Proc. Natl Acad. Sci. USA 116, 22811–22820 (2019).

    Article  CAS  Google Scholar 

  19. Qi, X.-L., Meyer, T., Stanford, T. R. & Constantinidis, C. Changes in prefrontal neuronal activity after learning to perform a spatial working memory task. Cereb. Cortex 21, 2722–2732 (2011).

    Article  Google Scholar 

  20. Meyer, T., Qi, X.-L., Stanford, T. R. & Constantinidis, C. Stimulus selectivity in dorsal and ventral prefrontal cortex after training in working memory tasks. J. Neurosci. 31, 6266–6276 (2011).

    Article  CAS  Google Scholar 

  21. Constantinidis, C., Qi, X.-L. & Meyer, T. Single-neuron spike train recordings from macaque prefrontal cortex during a visual working memory task before and after training. CRCNS https://doi.org/10.6080/K0ZW1HVD (2016).

  22. Stokes, M. G. et al. Dynamic coding for cognitive control in prefrontal cortex. Neuron 78, 364–375 (2013).

    Article  CAS  Google Scholar 

  23. Spaak, E., Watanabe, K., Funahashi, S. & Stokes, M. G. Stable and dynamic coding for working memory in primate prefrontal cortex. J. Neurosci. 37, 6503–6516 (2017).

    Article  CAS  Google Scholar 

  24. Miller, E. K. & Cohen, J. D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).

    Article  CAS  Google Scholar 

  25. Goldman-Rakic, P. S. in Comprehensive Physiology (ed. Terjung, R. L.) 373–417 (American Cancer Society, 2011).

  26. Meyer, T., Qi, X.-L. & Constantinidis, C. Persistent discharges in the prefrontal cortex of monkeys naïve to working memory tasks. Cereb. Cortex 17, i70–i76 (2007).

    Article  Google Scholar 

  27. Yang, G. R., Murray, J. D. & Wang, X.-J. A dendritic disinhibitory circuit mechanism for pathway-specific gating. Nat. Commun. 7, 12815 (2016).

  28. Wang, X.-J. & Yang, G. R. A disinhibitory circuit motif and flexible information routing in the brain. Curr. Opin. Neurobiol. 49, 75–83 (2018).

    Article  CAS  Google Scholar 

  29. Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013).

    Article  CAS  Google Scholar 

  30. Bouchacourt, F. & Buschman, T. J. A flexible model of working memory. Neuron 103, 147–160 (2019).

    Article  CAS  Google Scholar 

  31. Song, H. F., Yang, G. R. & Wang, X.-J. Training excitatory-inhibitory recurrent neural networks for cognitive tasks: a simple and flexible framework. PLoS Comput. Biol. 12, e1004792 (2016).

    Article  Google Scholar 

  32. Miconi, T. Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. eLife 6, e20899 (2017).

    Article  Google Scholar 

  33. Orhan, A. E. & Ma, W. J. A diverse range of factors affect the nature of neural representations underlying short-term memory. Nat. Neurosci. 22, 275–283 (2019).

    Article  CAS  Google Scholar 

  34. Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X.-J. Task representations in neural networks trained to perform many cognitive tasks. Nat. Neurosci. 22, 297–306 (2019).

    Article  CAS  Google Scholar 

  35. Kepecs, A. & Fishell, G. Interneuron cell types are fit to function. Nature 505, 318–326 (2014).

    Article  CAS  Google Scholar 

  36. Batista-Brito, R. et al. Developmental dysfunction of VIP interneurons impairs cortical circuits. Neuron 95, 884–895.e9 (2017).

    Article  CAS  Google Scholar 

  37. Pi, H.-J. et al. Cortical interneurons that specialize in disinhibitory control. Nature 503, 521–524 (2013).

    Article  CAS  Google Scholar 

  38. Karnani, M. M. et al. Opening holes in the blanket of inhibition: localized lateral disinhibition by VIP interneurons. J. Neurosci. 36, 3471–3480 (2016).

    Article  CAS  Google Scholar 

  39. Wang, X.-J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).

    Article  CAS  Google Scholar 

  40. Kim, Y. et al. Brain-wide maps reveal stereotyped cell-type-based cortical architecture and subcortical sexual dimorphism. Cell 171, 456–469.e22 (2017).

    Article  CAS  Google Scholar 

  41. Medalla, M. & Barbas, H. Synapses with inhibitory neurons differentiate anterior cingulate from dorsolateral prefrontal pathways associated with cognitive control. Neuron 61, 609–620 (2009).

    Article  CAS  Google Scholar 

  42. Wimmer, K., Nykamp, D. Q., Constantinidis, C. & Compte, A. Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nat. Neurosci. 17, 431–439 (2014).

    Article  CAS  Google Scholar 

  43. Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG and MEG data. J. Neurosci. Methods 164, 177–190 (2007).

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to B. Tsuda, Y. Chen and J. Fleischer for helpful discussions and feedback on the manuscript. We also thank J. Aldana for assistance with computing resources. This work was funded by the National Institute of Mental Health (grant no. F30MH115605-01A1 to R.K.). We also gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 graphics processing unit used for this research. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Computational Neurobiology Laboratory, Salk Institute for Biological Studies, La Jolla, CA, USA

    Robert Kim & Terrence J. Sejnowski

  2. Neurosciences Graduate Program, University of California San Diego, La Jolla, CA, USA

    Robert Kim

  3. Medical Scientist Training Program, University of California San Diego, La Jolla, CA, USA

    Robert Kim

  4. Institute for Neural Computation, University of California San Diego, La Jolla, CA, USA

    Terrence J. Sejnowski

  5. Division of Biological Sciences, University of California San Diego, La Jolla, CA, USA

    Terrence J. Sejnowski

Authors
  1. Robert Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Terrence J. Sejnowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

R.K. and T.J.S. designed the study and wrote the manuscript. R.K. performed the analyses and simulations.

Corresponding authors

Correspondence to Robert Kim or Terrence J. Sejnowski.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Neuroscience thanks Dean Buonomano, Timothy Buschman, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kim, R., Sejnowski, T.J. Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks. Nat Neurosci 24, 129–139 (2021). https://doi.org/10.1038/s41593-020-00753-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41593-020-00753-w

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing