Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Multiplexing working memory and time in the trajectories of neural networks

Nature Human Behaviour volume 7, pages 1170–1184 (2023)Cite this article

Subjects

Abstract

Working memory (WM) and timing are generally considered distinct cognitive functions, but similar neural signatures have been implicated in both. To explore the hypothesis that WM and timing may rely on shared neural mechanisms, we used psychophysical tasks that contained either task-irrelevant timing or WM components. In both cases, the task-irrelevant component influenced performance. We then developed recurrent neural network (RNN) simulations that revealed that cue-specific neural sequences, which multiplexed WM and time, emerged as the dominant regime that captured the behavioural findings. During training, RNN dynamics transitioned from low-dimensional ramps to high-dimensional neural sequences, and depending on task requirements, steady-state or ramping activity was also observed. Analysis of RNN structure revealed that neural sequences relied primarily on inhibitory connections, and could survive the deletion of all excitatory-to-excitatory connections. Our results indicate that in some instances WM is encoded in time-varying neural activity because of the importance of predicting when WM will be used.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Humans implicitly learn the timing component of a WM task and the WM component of a timing task.
Fig. 2: Differential dynamics for the encoding of WM and time across in RNNs trained on three tasks.
Fig. 3: Transition from low-dimensional ramping to high-dimensional neural sequences over the course of training in the T + WM task.
Fig. 4: Multiplexing of time and WM.
Fig. 5: Circuit motifs underlying the generation of multiple neural sequences.
Fig. 6: In the presence of high noise ablation of all Ex → Ex connections has little effect on performance in the T + WM task.
Fig. 7: Neural sequences instantiate dynamic attractors.

Similar content being viewed by others

Data availability

Human data and code for analysis are available at https://osf.io/HXSUG/.

Code availability

Code for the RNN simulations and analysis is available at https://github.com/BuonoLab/Timing-WM_RNN_2022.

References

  1. D’Esposito, M. & Postle, B. R. The cognitive neuroscience of working memory. Annu. Rev. Psychol. 66, 115–142 (2015).

    Article  PubMed  Google Scholar 

  2. Baddeley, A. D. & Hitch, G. J. in Psychology of Learning and Motivation (ed. Bower G. H.) 47–89 (Academic Press, 1974).

  3. Paton, J. J. & Buonomano, D. V. The neural basis of timing: distributed mechanisms for diverse functions. Neuron 98, 687–705 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Merchant, H., Harrington, D. L. & Meck, W. H. Neural basis of the perception and estimation of time. Annu. Rev. Neurosci. 36, 313–336 (2013).

    Article  CAS  PubMed  Google Scholar 

  5. Issa, J. B., Tocker, G., Hasselmo, M. E., Heys, J. G. & Dombeck, D. A. Navigating through time: a spatial navigation perspective on how the brain may encode time. Annu. Rev. Neurosci. 43, 73–93 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  6. Goldman-Rakic, P. S. Cellular basis of working memory. Neuron 14, 477–485 (1995).

    Article  CAS  PubMed  Google Scholar 

  7. Nobre, A. C. & van Ede, F. Anticipated moments: temporal structure in attention. Nat. Rev. Neurosci. 19, 34–48 (2018).

    Article  CAS  PubMed  Google Scholar 

  8. Coull, J. T., Cheng, R.-K. & Meck, W. H. Neuroanatomical and neurochemical substrates of timing. Neuropsychopharmacology 36, 3–25 (2011).

    Article  PubMed  Google Scholar 

  9. Stokes, M. G. ‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework. Trends Cogn. Sci. 19, 394–405 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  10. Lundqvist, M., Herman, P. & Miller, E. K. Working memory: delay activity, yes! Persistent activity? Maybe not. J. Neurosci. 38, 7013–7019 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Cueva, C. J. et al. Low-dimensional dynamics for working memory and time encoding. Proc. Natl Acad. Sci. USA 117, 23021–23032 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Pastalkova, E., Itskov, V., Amarasingham, A. & Buzsaki, G. Internally generated cell assembly sequences in the rat hippocampus. Science 321, 1322–1327 (2008).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Goldman, M. S. Memory without feedback in a neural network. Neuron 61, 621–634 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Rajan, K., Harvey, C. D. & Tank D. W. Recurrent network models of sequence generation and memory. Neuron 90, 128–142 (2016).

  15. Funahashi, S., Bruce, C. J. & Goldman-Rakic, P. S. Mnemonic coding of visual space in the monkey’s dorsolateral prefrontal cortex. J. Neurophysiol. 61, 331–349 (1989).

    Article  CAS  PubMed  Google Scholar 

  16. Fuster, J. M. & Alexander, G. E. Neuron activity related to short-term memory. Science 173, 652–654 (1971).

    Article  CAS  PubMed  Google Scholar 

  17. Fuster, J. M., Bodner, M. & Kroger, J. K. Cross-modal and cross-temporal association in neurons of frontal cortex. Nature 405, 347–351 (2000).

    Article  CAS  PubMed  Google Scholar 

  18. Constantinidis, C. et al. Persistent spiking activity underlies working memory. J. Neurosci. 38, 7020–7028 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Mongillo, G., Barak, O. & Tsodyks, M. Synaptic theory of working memory. Science 319, 1543–1546 (2008).

    Article  CAS  PubMed  Google Scholar 

  20. Buonomano, D. V. & Maass, W. State-dependent computations: spatiotemporal processing in cortical networks. Nat. Rev. Neurosci. 10, 113–125 (2009).

    Article  CAS  PubMed  Google Scholar 

  21. Coull, J. T. & Nobre, A. C. Where and when to pay attention: the neural systems for directing attention to spatial locations and to time intervals as revealed by both PET and fMRI. J. Neurosci. 18, 7426–7435 (1998).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Rainer, G., Rao, S. C. & Miller, E. K. Prospective coding for objects in primate prefrontal cortex. J. Neurosci. 19, 5493–5505 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Matell, M. S., Meck, W. H. & Lustig, C. Not ‘just’ a coincidence: frontal‐striatal interactions in working memory and interval timing. Memory 13, 441–448 (2005).

    Article  PubMed  Google Scholar 

  24. van Ede, F., Niklaus, M. & Nobre, A. C. Temporal expectations guide dynamic prioritization in visual working memory through attenuated α oscillations. J. Neurosci. 37, 437–445 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  25. Liu, J. K. & Buonomano, D. V. Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J. Neurosci. 29, 13172–13181 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Murray, J. M. & Escola, G. S. Learning multiple variable-speed sequences in striatum via cortical tutoring. eLife 6, e26084 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  27. Yang, G. R. & Wang, X.-J. Artificial neural networks for neuroscientists: a primer. Neuron 107, 1048–1070 (2020).

    Article  CAS  PubMed  Google Scholar 

  28. Townsend, J. & Ashby, F. in Cognitive Theory Vol. 3 (eds Castellan, J. & Restle, F.) 200–239 (Erlbaum, 1978).

  29. Nobre, A. C., Correa, A. & Coull, J. T. The hazards of time. Curr. Opin. Neurobiol. 17, 465–470 (2007).

    Article  CAS  PubMed  Google Scholar 

  30. Taxidis, J. et al. Differential emergence and stability of sensory and temporal representations in context-specific hippocampal sequences. Neuron 108, 984–998 (2020).

  31. MacDonald, C. J. et al. Hippocampal ‘time cells’ bridge the gap in memory for discontiguous events. Neuron 71, 737–749 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Gouvea, T. S. et al. Striatal dynamics explain duration judgments. eLife 4, e11386 (2015).

    Article  PubMed  PubMed Central  Google Scholar 

  33. Zhou, S., Masmanidis, S. C. & Buonomano, D. V. Neural sequences as an optimal dynamical regime for the readout of time. Neuron 108, 651–658.e655 (2020).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  34. Heys, J. G. & Dombeck, D. A. Evidence for a subcircuit in medial entorhinal cortex representing elapsed time during immobility. Nat. Neurosci. 21, 1574–1582 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Shimbo, A., Izawa, E.-I. & Fujisawa, S. Scalable representation of time in the hippocampus. Sci. Adv. 7, eabd7013 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  36. Brody, C. D., Hernandez, A., Zainos, A. & Romo, R. Timing and neural encoding of somatosensory parametric working memory in macaque prefrontal cortex. Cereb. Cortex 13, 1196–1207 (2003).

    Article  PubMed  Google Scholar 

  37. Bae, J. W. et al. Parallel processing of working memory and temporal information by distinct types of cortical projection neurons. Nat. Commun. 12, 4352 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Emmons, E. B. et al. Rodent medial frontal control of temporal processing in the dorsomedial striatum. J. Neurosci. 37, 8718–8733 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Jazayeri, M., Shadlen, & Michael, N. A neural mechanism for sensing and reproducing a time interval. Curr. Biol. 25, 2599–2609 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X.-J. Task representations in neural networks trained to perform many cognitive tasks. Nat. Neurosci. 22, 297–306 (2019).

  41. Zhou, S., Masmanidis, S. C. & Buonomano, D. V. Encoding time in neural dynamic regimes with distinct computational tradeoffs. PLoS Comput. Biol. 18, e1009271 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Murray, J. D. et al. Stable population coding for working memory coexists with heterogeneous neural dynamics in prefrontal cortex. Proc. Natl Acad. Sci. USA 114, 394–399 (2017).

    Article  CAS  PubMed  Google Scholar 

  43. Hahnloser, R. H. R., Kozhevnikov, A. A. & Fee, M. S. An ultra-sparse code underlies the generation of neural sequence in a songbird. Nature 419, 65–70 (2002).

    Article  CAS  PubMed  Google Scholar 

  44. Hardy, N. F. & Buonomano, D. V. Encoding time in feedforward trajectories of a recurrent neural network model. Neural Comput. 30, 378–396 (2018).

    Article  CAS  PubMed  Google Scholar 

  45. Fiete, I. R., Senn, W., Wang, C. Z. H. & Hahnloser, R. H. R. Spike-time-dependent plasticity and heterosynaptic competition organize networks to produce long scale-free sequences of neural activity. Neuron 65, 563–576 (2010).

    Article  CAS  PubMed  Google Scholar 

  46. Medina, J. F., Garcia, K. S., Nores, W. L., Taylor, N. M. & Mauk, M. D. Timing mechanisms in the cerebellum: testing predictions of a large-scale computer simulation. J. Neurosci. 20, 5516–5525 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Laje, R. & Buonomano, D. V. Robust timing and motor patterns by taming chaos in recurrent neural networks. Nat. Neurosci. 16, 925–933 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Monteforte, M. & Wolf, F. Dynamic flux tubes form reservoirs of stability in neuronal circuits. Phys. Rev. X 2, 041007 (2012).

    CAS  Google Scholar 

  49. Stokes, M. G. et al. Dynamic coding for cognitive control in prefrontal cortex. Neuron 78, 364–375 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Kim, R. & Sejnowski, T. J. Strong inhibitory signaling underlies stable temporal dynamics and working memory in spiking neural networks. Nat. Neurosci. 24, 129–139 (2021).

    Article  CAS  PubMed  Google Scholar 

  51. Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl Acad. Sci. USA 79, 2554–2558 (1982).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  52. Wang, X.-J. Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci. 24, 455–463 (2001).

    Article  CAS  PubMed  Google Scholar 

  53. Inagaki, H. K., Fontolan, L., Romani, S. & Svoboda, K. Discrete attractor dynamics underlies persistent activity in the frontal cortex. Nature 566, 212–217 (2019).

    Article  CAS  PubMed  Google Scholar 

  54. Park, J. C., Bae, J. W., Kim, J. & Jung, M. W. Dynamically changing neuronal activity supporting working memory for predictable and unpredictable durations. Sci. Rep. 9, 15512 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  55. Masse, N. Y., Yang, G. R., Song, H. F., Wang, X.-J. & Freedman, D. J. Circuit mechanisms for the maintenance and manipulation of information in working memory. Nat. Neurosci. 22, 1159–1167 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Motanis, H., Seay, M. J. & Buonomano, D. V. Short-term synaptic plasticity as a mechanism for sensory timing. Trends Neurosci. 41, 701–711 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  57. Bakhurin, K. I. et al. Differential encoding of time by prefrontal and striatal network dynamics. J. Neurosci. 37, 854–870 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  58. Tiganj, Z., Jung, M. W., Kim, J. & Howard, M. W. Sequential firing codes for time in rodent medial prefrontal cortex. Cereb. Cortex 27, 5663–5671 (2017).

    Article  PubMed  Google Scholar 

  59. Tsao, A. et al. Integrating time from experience in the lateral entorhinal cortex. Nature 561, 57–62 (2018).

  60. Kim, J., Ghim, J.-W., Lee, J. H. & Jung, M. W. Neural correlates of interval timing in rodent prefrontal cortex. J. Neurosci. 33, 13834–13847 (2013).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Saez, A., Rigotti, M., Ostojic, S., Fusi, S. & Salzman, C. D. Abstract context representations in primate amygdala and prefrontal cortex. Neuron 87, 869–881 (2015).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Genovesio, A., Tsujimoto, S. & Wise, S. P. Feature- and order-based timing representations in the frontal cortex. Neuron 63, 254–266 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  63. Goodfellow, I., Bengio, Y. & Courville, A. Deep Learning (MIT Press, 2016).

  64. Miller, K. D. & Troyer, T. W. Neural noise can explain expansive, power-law nonlinearities in neural response functions. J. Neurophysiol. 87, 653–659 (2002).

    Article  PubMed  Google Scholar 

  65. Rauch, A., Camera, G. L., Lüscher, H.-R., Senn, W. & Fusi, S. Neocortical pyramidal cells respond as integrate-and-fire neurons to in vivo-like input currents. J. Neurophysiol. 90, 1598–1612 (2003).

    Article  PubMed  Google Scholar 

  66. Orhan, A. E. & Ma, W. J. A diverse range of factors affect the nature of neural representations underlying short-term memory. Nat. Neurosci. 22, 275–283 (2019).

    Article  CAS  PubMed  Google Scholar 

  67. Ghazizadeh, E. & Ching, S. Slow manifolds within network dynamics encode working memory efficiently and robustly. PLoS Comput. Biol. 17, e1009366 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  68. Bernacchia, A., Fiser, J., Hennequin, G. & Lengyel, M. Adaptive erasure of spurious sequences in sensory cortical circuits. Neuron 110, 1857–1868 (2022).

  69. Tupikov, Y. & Jin, D. Z. Addition of new neurons and the emergence of a local neural circuit for precise timing. PLoS Comput. Biol. 17, e1008824 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  70. Schacter, D. L., Addis, D. R. & Buckner, R. L. Remembering the past to imagine the future: the prospective brain. Nat. Rev. Neurosci. 8, 657–661 (2007).

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

We thank A. Tanwar for assistance with the psychophysics experiments, J. Rissman for helpful advice and J. Fuster for his comments on an earlier version of this manuscript. This research was supported by National Institutes of Health grant no. NS116589 awarded to D.V.B. and P.G. The funders had no role in study design, data collection and analysis, the decision to publish or the preparation of the paper.

Author information

Author notes

  1. These authors contributed equally: Shanglin Zhou, Michael Seay.

Authors and Affiliations

  1. Department of Neurobiology, David Geffen School of Medicine, University of California, Los Angeles, CA, USA

    Shanglin Zhou & Dean V. Buonomano

  2. Department of Psychology, University of California, Los Angeles, CA, USA

    Michael Seay & Dean V. Buonomano

  3. Program in Neurosciences and Mental Health, Hospital for Sick Children, Toronto, Ontario, Canada

    Jiannis Taxidis

  4. Department of Physiology, University of Toronto, Toronto, Ontario, Canada

    Jiannis Taxidis

  5. Department of Neurology, David Geffen School of Medicine, University of California, Los Angeles, Los Angeles, CA, USA

    Peyman Golshani & Dean V. Buonomano

  6. Integrative Center for Learning and Memory, Brain Research Institute, University of California, Los Angeles, Los Angeles, CA, USA

    Peyman Golshani

  7. UCLA Semel Institute for Neuroscience and Behavioral Sciences, University of California, Los Angeles, Los Angeles, CA, USA

    Peyman Golshani

  8. West Los Angeles VA Medical Center, Los Angeles, CA, USA

    Peyman Golshani

Authors
  1. Shanglin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael Seay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiannis Taxidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peyman Golshani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dean V. Buonomano
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors participated in the conceptualizing of the study. M.S. collected the human psychophysics data. M.S. and D.V.B. analysed the human experimental data. S.Z. and D.V.B. performed the model simulations and analysed the modelling data. All authors wrote and revised the paper.

Corresponding author

Correspondence to Dean V. Buonomano.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Human Behaviour thanks Nicholas Myers, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, S., Seay, M., Taxidis, J. et al. Multiplexing working memory and time in the trajectories of neural networks. Nat Hum Behav 7, 1170–1184 (2023). https://doi.org/10.1038/s41562-023-01592-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-023-01592-y

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing