Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks

Abstract

Cortical neurons process information on multiple timescales, and areas important for working memory (WM) contain neurons capable of integrating information over a long timescale. However, the underlying mechanisms for the emergence of neuronal timescales stable enough to support WM are unclear. By analyzing a spiking recurrent neural network model trained on a WM task and activity of single neurons in the primate prefrontal cortex, we show that the temporal properties of our model and the neural data are remarkably similar. Dissecting our recurrent neural network model revealed strong inhibitory-to-inhibitory connections underlying a disinhibitory microcircuit as a critical component for long neuronal timescales and WM maintenance. We also found that enhancing inhibitory-to-inhibitory connections led to more stable temporal dynamics and improved task performance. Finally, we show that a network with such microcircuitry can perform other tasks without disrupting its pre-existing timescale architecture, suggesting that strong inhibitory signaling underlies a flexible WM network.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Fig. 1: RNN model and experimental data.
Fig. 2: RNN model trained on the DMS task and the dlPFC data contain units with long timescales.
Fig. 3: Long τ units maintain cue stimulus information during the delay period robustly.
Fig. 4: Inhibitory synaptic weights lead to task-specific timescales.
Fig. 5: I → I connectivity strength strongly mediates both neuronal timescales and task performance.
Fig. 6: Two oppositely tuned inhibitory subgroups mutually inhibit each other for WM maintenance.
Fig. 7: High trial-to-trial spike-count variability during fixation corresponds to long neuronal timescale.
Fig. 8: Strong I → I connections might be intrinsic to prefrontal cortex.

Data availability

The trained RNN models used in the present study are deposited as MATLAB-formatted data in Open Science Framework, https://osf.io/md4wg. The experimental data used in the study can be obtained from Constantinidis et al.21.

Code availability

The code for the analyses performed in this work is available at https://github.com/rkim35/wmRNN.

References

  1. Murray, J. D. et al. A hierarchy of intrinsic timescales across primate cortex. Nat. Neurosci. 17, 1661–1663 (2014).

    CAS  Article  Google Scholar 

  2. Chaudhuri, R., Knoblauch, K., Gariel, M.-A., Kennedy, H. & Wang, X.-J. A large-scale circuit mechanism for hierarchical dynamical processing in the primate cortex. Neuron 88, 419–431 (2015).

    CAS  Article  Google Scholar 

  3. Cavanagh, S. E., Wallis, J. D., Kennerley, S. W. & Hunt, L. T. Autocorrelation structure at rest predicts value correlates of single neurons during reward-guided choice. eLife 5, e18937 (2016).

    Article  Google Scholar 

  4. Miller, E. K., Erickson, C. A. & Desimone, R. Neural mechanisms of visual working memory in prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167 (1996).

    CAS  Article  Google Scholar 

  5. Fuster, J. M. & Alexander, G. E. Neuron activity related to short-term memory. Science 173, 652–654 (1971).

    CAS  Article  Google Scholar 

  6. Fascianelli, V., Tsujimoto, S., Marcos, E. & Genovesio, A. Autocorrelation structure in the macaque dorsolateral, but not orbital or polar, prefrontal cortex predicts response-coding strength in a visually cued strategy task. Cereb. Cortex 29, 230–241 (2017).

    Article  Google Scholar 

  7. Cavanagh, S. E., Towers, J. P., Wallis, J. D., Hunt, L. T. & Kennerley, S. W. Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex. Nat. Commun. 9, 3498 (2018).

  8. Wasmuht, D. F., Spaak, E., Buschman, T. J., Miller, E. K. & Stokes, M. G. Intrinsic neuronal dynamics predict distinct functional roles during working memory. Nat. Commun. 9, 3499 (2018).

    CAS  Article  Google Scholar 

  9. Kim, D. et al. Distinct roles of parvalbumin- and somatostatin-expressing interneurons in working memory. Neuron 92, 902–915 (2016).

    CAS  Article  Google Scholar 

  10. Kamigaki, T. & Dan, Y. Delay activity of specific prefrontal interneuron subtypes modulates memory-guided behavior. Nat. Neurosci. 20, 854–863 (2017).

    CAS  Article  Google Scholar 

  11. Xu, H. et al. A disinhibitory microcircuit mediates conditioned social fear in the prefrontal cortex. Neuron 102, 668–682 (2019).

    CAS  Article  Google Scholar 

  12. Cummings, K. A. & Clem, R. L. Prefrontal somatostatin interneurons encode fear memory. Nat. Neurosci. 23, 61–74 (2019).

    Article  Google Scholar 

  13. Krabbe, S. et al. Adaptive disinhibitory gating by VIP interneurons permits associative learning. Nat. Neurosci. 22, 1834–1843 (2019).

    CAS  Article  Google Scholar 

  14. Pfeffer, C. K., Xue, M., He, M., Huang, Z. J. & Scanziani, M. Inhibition of inhibition in visual cortex: the logic of connections between molecularly distinct interneurons. Nat. Neurosci. 16, 1068–1076 (2013).

    CAS  Article  Google Scholar 

  15. Tremblay, R., Lee, S. & Rudy, B. GABAergic interneurons in the neocortex: from cellular properties to circuits. Neuron 91, 260–292 (2016).

    CAS  Article  Google Scholar 

  16. Medalla, M., Gilman, J. P., Wang, J.-Y. & Luebke, J. I. Strength and diversity of inhibitory signaling differentiates primate anterior cingulate from lateral prefrontal cortex. J. Neurosci. 37, 4717–4734 (2017).

    CAS  Article  Google Scholar 

  17. Mongillo, G., Rumpel, S. & Loewenstein, Y. Inhibitory connectivity defines the realm of excitatory plasticity. Nat. Neurosci. 21, 1463–1470 (2018).

    CAS  Article  Google Scholar 

  18. Kim, R., Li, Y. & Sejnowski, T. J. Simple framework for constructing functional spiking recurrent neural networks. Proc. Natl Acad. Sci. USA 116, 22811–22820 (2019).

    CAS  Article  Google Scholar 

  19. Qi, X.-L., Meyer, T., Stanford, T. R. & Constantinidis, C. Changes in prefrontal neuronal activity after learning to perform a spatial working memory task. Cereb. Cortex 21, 2722–2732 (2011).

    Article  Google Scholar 

  20. Meyer, T., Qi, X.-L., Stanford, T. R. & Constantinidis, C. Stimulus selectivity in dorsal and ventral prefrontal cortex after training in working memory tasks. J. Neurosci. 31, 6266–6276 (2011).

    CAS  Article  Google Scholar 

  21. Constantinidis, C., Qi, X.-L. & Meyer, T. Single-neuron spike train recordings from macaque prefrontal cortex during a visual working memory task before and after training. CRCNS https://doi.org/10.6080/K0ZW1HVD (2016).

  22. Stokes, M. G. et al. Dynamic coding for cognitive control in prefrontal cortex. Neuron 78, 364–375 (2013).

    CAS  Article  Google Scholar 

  23. Spaak, E., Watanabe, K., Funahashi, S. & Stokes, M. G. Stable and dynamic coding for working memory in primate prefrontal cortex. J. Neurosci. 37, 6503–6516 (2017).

    CAS  Article  Google Scholar 

  24. Miller, E. K. & Cohen, J. D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).

    CAS  Article  Google Scholar 

  25. Goldman-Rakic, P. S. in Comprehensive Physiology (ed. Terjung, R. L.) 373–417 (American Cancer Society, 2011).

  26. Meyer, T., Qi, X.-L. & Constantinidis, C. Persistent discharges in the prefrontal cortex of monkeys naïve to working memory tasks. Cereb. Cortex 17, i70–i76 (2007).

    Article  Google Scholar 

  27. Yang, G. R., Murray, J. D. & Wang, X.-J. A dendritic disinhibitory circuit mechanism for pathway-specific gating. Nat. Commun. 7, 12815 (2016).

  28. Wang, X.-J. & Yang, G. R. A disinhibitory circuit motif and flexible information routing in the brain. Curr. Opin. Neurobiol. 49, 75–83 (2018).

    CAS  Article  Google Scholar 

  29. Mante, V., Sussillo, D., Shenoy, K. V. & Newsome, W. T. Context-dependent computation by recurrent dynamics in prefrontal cortex. Nature 503, 78–84 (2013).

    CAS  Article  Google Scholar 

  30. Bouchacourt, F. & Buschman, T. J. A flexible model of working memory. Neuron 103, 147–160 (2019).

    CAS  Article  Google Scholar 

  31. Song, H. F., Yang, G. R. & Wang, X.-J. Training excitatory-inhibitory recurrent neural networks for cognitive tasks: a simple and flexible framework. PLoS Comput. Biol. 12, e1004792 (2016).

    Article  Google Scholar 

  32. Miconi, T. Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks. eLife 6, e20899 (2017).

    Article  Google Scholar 

  33. Orhan, A. E. & Ma, W. J. A diverse range of factors affect the nature of neural representations underlying short-term memory. Nat. Neurosci. 22, 275–283 (2019).

    CAS  Article  Google Scholar 

  34. Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X.-J. Task representations in neural networks trained to perform many cognitive tasks. Nat. Neurosci. 22, 297–306 (2019).

    CAS  Article  Google Scholar 

  35. Kepecs, A. & Fishell, G. Interneuron cell types are fit to function. Nature 505, 318–326 (2014).

    CAS  Article  Google Scholar 

  36. Batista-Brito, R. et al. Developmental dysfunction of VIP interneurons impairs cortical circuits. Neuron 95, 884–895.e9 (2017).

    CAS  Article  Google Scholar 

  37. Pi, H.-J. et al. Cortical interneurons that specialize in disinhibitory control. Nature 503, 521–524 (2013).

    CAS  Article  Google Scholar 

  38. Karnani, M. M. et al. Opening holes in the blanket of inhibition: localized lateral disinhibition by VIP interneurons. J. Neurosci. 36, 3471–3480 (2016).

    CAS  Article  Google Scholar 

  39. Wang, X.-J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).

    CAS  Article  Google Scholar 

  40. Kim, Y. et al. Brain-wide maps reveal stereotyped cell-type-based cortical architecture and subcortical sexual dimorphism. Cell 171, 456–469.e22 (2017).

    CAS  Article  Google Scholar 

  41. Medalla, M. & Barbas, H. Synapses with inhibitory neurons differentiate anterior cingulate from dorsolateral prefrontal pathways associated with cognitive control. Neuron 61, 609–620 (2009).

    CAS  Article  Google Scholar 

  42. Wimmer, K., Nykamp, D. Q., Constantinidis, C. & Compte, A. Bump attractor dynamics in prefrontal cortex explains behavioral precision in spatial working memory. Nat. Neurosci. 17, 431–439 (2014).

    CAS  Article  Google Scholar 

  43. Maris, E. & Oostenveld, R. Nonparametric statistical testing of EEG and MEG data. J. Neurosci. Methods 164, 177–190 (2007).

    Article  Google Scholar 

Download references

Acknowledgements

We are grateful to B. Tsuda, Y. Chen and J. Fleischer for helpful discussions and feedback on the manuscript. We also thank J. Aldana for assistance with computing resources. This work was funded by the National Institute of Mental Health (grant no. F30MH115605-01A1 to R.K.). We also gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 graphics processing unit used for this research. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

Authors

Contributions

R.K. and T.J.S. designed the study and wrote the manuscript. R.K. performed the analyses and simulations.

Corresponding authors

Correspondence to Robert Kim or Terrence J. Sejnowski.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Neuroscience thanks Dean Buonomano, Timothy Buschman, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary Information

Supplementary Figs. 1–14.

Reporting Summary

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kim, R., Sejnowski, T.J. Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks. Nat Neurosci 24, 129–139 (2021). https://doi.org/10.1038/s41593-020-00753-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41593-020-00753-w

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing