Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Decoding cognition from spontaneous neural activity

Abstract

In human neuroscience, studies of cognition are rarely grounded in non-task-evoked, ‘spontaneous’ neural activity. Indeed, studies of spontaneous activity tend to focus predominantly on intrinsic neural patterns (for example, resting-state networks). Taking a ‘representation-rich’ approach bridges the gap between cognition and resting-state communities: this approach relies on decoding task-related representations from spontaneous neural activity, allowing quantification of the representational content and rich dynamics of such activity. For example, if we know the neural representation of an episodic memory, we can decode its subsequent replay during rest. We argue that such an approach advances cognitive research beyond a focus on immediate task demand and provides insight into the functional relevance of the intrinsic neural pattern (for example, the default mode network). This in turn enables a greater integration between human and animal neuroscience, facilitating experimental testing of theoretical accounts of intrinsic activity, and opening new avenues of research in psychiatry.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Fig. 1: A (relative) dichotomy in human neuroscience.
Fig. 2: ‘Representation-rich’ paradigm of spontaneous neural activity.
Fig. 3: Coordination between hippocampal SWRs and large-scale cortical activity across species.

References

  1. Uddin, L. Q. Bring the noise: reconceptualizing spontaneous neural activity. Trends Cogn. Sci. 24, 734–746 (2020).

    PubMed  PubMed Central  Google Scholar 

  2. Zhang, D. & Raichle, M. E. Disease and the brain’s dark energy. Nat. Rev. Neurol. 6, 15–28 (2010).

    PubMed  Google Scholar 

  3. Becker, R., Van De Ville, D. & Kleinschmidt, A. Alpha oscillations reduce temporal long-range dependence in spontaneous human brain activity. J. Neurosci. 38, 755–764 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  4. Allaman, L., Mottaz, A., Kleinschmidt, A. & Guggisberg, A. G. Spontaneous network coupling enables efficient task performance without local task-induced activations. J. Neurosci. 40, 9663–9675 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  5. Smith, S. M. et al. Functional connectomics from resting-state fMRI. Trends Cogn. Sci. 17, 666–682 (2013).

    PubMed  PubMed Central  Google Scholar 

  6. Smith, S. M. et al. Correspondence of the brain’s functional architecture during activation and rest. Proc. Natl Acad. Sci. USA 106, 13040–13045 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  7. Tavor, I. et al. Task-free MRI predicts individual differences in brain activity during task performance. Science 352, 216–220 (2016). This study shows that individual differences in brain activity during task performance can be predicted on the basis of its neural profile off-task (that is, during rest).

    CAS  PubMed  PubMed Central  Google Scholar 

  8. Rudoy, J. D., Voss, J. L., Westerberg, C. E. & Paller, K. A. Strengthening individual memories by reactivating them during sleep. Science 326, 1079–1079 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  9. Rasch, B., Büchel, C., Gais, S. & Born, J. Odor cues during slow-wave sleep prompt declarative memory consolidation. Science 315, 1426–1429 (2007).

    CAS  PubMed  Google Scholar 

  10. Wang, B. et al. Targeted memory reactivation during sleep elicits neural signals related to learning content. J. Neurosci. 39, 6728–6736 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  11. Cairney, S. A., El Marj, N. & Staresina, B. P. Memory consolidation is linked to spindle-mediated information processing during sleep. Curr. Biol. 28, 948–954.e4 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  12. Pajani, A., Kok, P., Kouider, S. & de Lange, F. P. Spontaneous activity patterns in primary visual cortex predispose to visual hallucinations. J. Neurosci. 35, 12947–12953 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  13. Chew, B. et al. Endogenous fluctuations in the dopaminergic midbrain drive behavioral choice variability. Proc. Natl Acad. Sci. USA 116, 18732–18737 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  14. Tambini, A. & Davachi, L. Awake reactivation of prior experiences consolidates memories and biases cognition. Trends Cogn. Sci. 23, 876–890 (2019).

    PubMed  PubMed Central  Google Scholar 

  15. Higgins, C. et al. Replay bursts in humans coincide with activation of the default mode and parietal alpha networks. Neuron 109, 882–893.e7 (2021). This study shows that human replays happen in bursts and are coupled with activation of the DMN, as well as high-frequency power increase in the temporal lobe.

    CAS  PubMed  PubMed Central  Google Scholar 

  16. Yeshurun, Y., Nguyen, M. & Hasson, U. The default mode network: where the idiosyncratic self meets the shared social world. Nat. Rev. Neurosci. 22, 181–192 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  17. Sutherland, G. R. & McNaughton, B. Memory trace reactivation in hippocampal and neocortical neuronal ensembles. Curr. Opin. Neurobiol. 10, 180–186 (2000).

    CAS  PubMed  Google Scholar 

  18. Tambini, A. & Davachi, L. Persistence of hippocampal multivoxel patterns into postencoding rest is related to memory. Proc. Natl Acad. Sci. USA 110, 19591–19596 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  19. Kriegeskorte, N., Mur, M. & Bandettini, P. A. Representational similarity analysis-connecting the branches of systems neuroscience. Front. Syst. Neurosci. 2, 4 (2008).

    PubMed  PubMed Central  Google Scholar 

  20. Diedrichsen, J. & Kriegeskorte, N. Representational models: a common framework for understanding encoding, pattern-component, and representational-similarity analysis. PLoS Comput. Biol. 13, e1005508 (2017).

    PubMed  PubMed Central  Google Scholar 

  21. Eldar, E., Lièvre, G., Dayan, P. & Dolan, R. J. The roles of online and offline replay in planning. eLife 9, e56911 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  22. Kurth-Nelson, Z., Economides, M., Dolan, R. J. & Dayan, P. Fast sequences of non-spatial state representations in humans. Neuron 91, 194–204 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  23. Liu, Y., Dolan, R. J., Kurth-Nelson, Z. & Behrens, T. E. J. Human replay spontaneously reorganizes experience. Cell 178, 640–652 (2019). This study shows that human replay (measured with MEG) exhibits similarities to rodent replay, while representing sensory and structural information independently, facilitating generalization in a novel context.

    CAS  PubMed  PubMed Central  Google Scholar 

  24. Liu, Y., Mattar, M. G., Behrens, T. E., Daw, N. D. & Dolan, R. J. Experience replay is associated with efficient nonlocal learning. Science 372, eabf1357 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  25. Schuck, N. W. & Niv, Y. Sequential replay of nonspatial task states in the human hippocampus. Science 364, eaaw5181 (2019). This study shows that human replay (measured with fMRI) can be captured in the hippocampus and is related to maintaining a neural representation of task space in the orbitofrontal cortex.

    CAS  PubMed  PubMed Central  Google Scholar 

  26. Burgess, N. & O’Keefe, J. Neuronal computations underlying the firing of place cells and their role in navigation. Hippocampus 6, 749–762 (1996).

    CAS  PubMed  Google Scholar 

  27. Skaggs, W. E. & McNaughton, B. L. Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience. Science 271, 1870–1873 (1996).

    CAS  PubMed  Google Scholar 

  28. Nádasdy, Z., Hirase, H., Czurkó, A., Csicsvari, J. & Buzsáki, G. Replay and time compression of recurring spike sequences in the hippocampus. J. Neurosci. 19, 9497–9507 (1999).

    PubMed  PubMed Central  Google Scholar 

  29. Louie, K. & Wilson, M. A. Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron 29, 145–156 (2001).

    CAS  PubMed  Google Scholar 

  30. Foster, D. J. & Wilson, M. A. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680 (2006).

    CAS  PubMed  Google Scholar 

  31. Diba, K. & Buzsáki, G. Forward and reverse hippocampal place-cell sequences during ripples. Nat. Neurosci. 10, 1241 (2007).

    CAS  PubMed  PubMed Central  Google Scholar 

  32. Ji, D. & Wilson, M. A. Coordinated memory replay in the visual cortex and hippocampus during sleep. Nat. Neurosci. 10, 100 (2007).

    CAS  PubMed  Google Scholar 

  33. Davidson, T. J., Kloosterman, F. & Wilson, M. A. Hippocampal replay of extended experience. Neuron 63, 497–507 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  34. Karlsson, M. P. & Frank, L. M. Awake replay of remote experiences in the hippocampus. Nat. Neurosci. 12, 913 (2009).

    CAS  PubMed  PubMed Central  Google Scholar 

  35. Gupta, A. S., van der Meer, M. A., Touretzky, D. S. & Redish, A. D. Hippocampal replay is not a simple function of experience. Neuron 65, 695–705 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  36. Carr, M. F., Jadhav, S. P. & Frank, L. M. Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval. Nat. Neurosci. 14, 147 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  37. Ambrose, R. E., Pfeiffer, B. E. & Foster, D. J. Reverse replay of hippocampal place cells is uniquely modulated by changing reward. Neuron 91, 1124–1136 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  38. Ólafsdóttir, H. F., Carpenter, F. & Barry, C. Coordinated grid and place cell replay during rest. Nat. Neurosci. 19, 792 (2016).

    PubMed  Google Scholar 

  39. O’Neill, J., Boccara, C. N., Stella, F., Schoenenberger, P. & Csicsvari, J. Superficial layers of the medial entorhinal cortex replay independently of the hippocampus. Science 355, 184–188 (2017).

    PubMed  Google Scholar 

  40. Wu, C.-T., Haggerty, D., Kemere, C. & Ji, D. Hippocampal awake replay in fear memory retrieval. Nat. Neurosci. 20, 571–580 (2017).

    CAS  PubMed  PubMed Central  Google Scholar 

  41. Buzsáki, G. Hippocampal sharp wave-ripple: a cognitive biomarker for episodic memory and planning. Hippocampus 25, 1073–1188 (2015). This study shows that hippocampal SWRs are implicated in many cognitive processes, the deficit of which occurs in certain psychiatric conditions, such as schizophrenia.

    PubMed  PubMed Central  Google Scholar 

  42. Jadhav, S. P., Kemere, C., German, P. W. & Frank, L. M. Awake hippocampal sharp-wave ripples support spatial memory. Science 336, 1454–1458 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  43. Foster, D. J. Replay comes of age. Annu. Rev. Neurosci. 40, 581–602 (2017).

    CAS  PubMed  Google Scholar 

  44. Ólafsdóttir, H. F., Bush, D. & Barry, C. The role of hippocampal replay in memory and planning. Curr. Biol. 28, R37–R50 (2018).

    PubMed  PubMed Central  Google Scholar 

  45. Wittkuhn, L. & Schuck, N. W. Dynamics of fMRI patterns reflect sub-second activation sequences and reveal replay in human visual cortex. Nat. Commun. 12, 1795 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  46. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018).

  47. Sutton, R. S. Dyna, an integrated architecture for learning, planning, and reacting. ACM Sigart Bull. 2, 160–163 (1991).

    Google Scholar 

  48. Mattar, M. G. & Daw, N. D. Prioritized memory access explains planning and hippocampal replay. Nat. Neurosci. 21, 1609 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  49. Guggenmos, M., Sterzer, P. & Cichy, R. M. Multivariate pattern analysis for MEG: a comparison of dissimilarity measures. NeuroImage 173, 434–447 (2018).

    PubMed  Google Scholar 

  50. Norman, K. A., Polyn, S. M., Detre, G. J. & Haxby, J. V. Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 10, 424–430 (2006).

    PubMed  Google Scholar 

  51. Haxby, J. V. et al. Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science 293, 2425–2430 (2001).

    CAS  PubMed  Google Scholar 

  52. Peelen, M. V. & Downing, P. E. Using multi-voxel pattern analysis of fMRI data to interpret overlapping functional activations. Trends Cogn. Sci. 11, 4–5 (2007).

    PubMed  Google Scholar 

  53. Liu, Y. et al. Temporally delayed linear modelling (TDLM) measures replay in both animals and humans. eLife 10, e66917 (2021). This study shows that fast neural sequences of spontaneous reactivations can be captured non-invasively in the human brain, and the same method can be used to quantify rodent replay in electrophysiology recordings.

    CAS  PubMed  PubMed Central  Google Scholar 

  54. Harris, K. D. Nonsense correlations in neuroscience. Preprint at bioRxiv https://doi.org/10.1101/2020.11.29.402719 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  55. Doll, B. B., Duncan, K. D., Simon, D. A., Shohamy, D. & Daw, N. D. Model-based choices involve prospective neural activity. Nat. Neurosci. 18, 767 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  56. Eldar, E., Bae, G. J., Kurth-Nelson, Z., Dayan, P. & Dolan, R. J. Magnetoencephalography decoding reveals structural differences within integrative decision processes. Nat. Hum. Behav. 2, 670–681 (2018).

    PubMed  Google Scholar 

  57. Kurth-Nelson, Z., Barnes, G., Sejdinovic, D., Dolan, R. & Dayan, P. Temporal structure in associative retrieval. eLife 4, e04919 (2015).

    PubMed Central  Google Scholar 

  58. Momennejad, I., Otto, A. R., Daw, N. D. & Norman, K. A. Offline replay supports planning in human reinforcement learning. eLife 7, e32548 (2018).

    PubMed  PubMed Central  Google Scholar 

  59. Wimmer, G. E., Liu, Y., Vehar, N., Behrens, T. E. J. & Dolan, R. J. Episodic memory retrieval success is associated with rapid replay of episode content. Nat. Neurosci. 23, 1025–1033 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  60. Wimmer, G. E. & Shohamy, D. Preference by association: how memory mechanisms in the hippocampus bias decisions. Science 338, 270–273 (2012).

    CAS  PubMed  Google Scholar 

  61. Wise, T., Liu, Y., Chowdhury, F. & Dolan, R. J. Model-based aversive learning in humans is supported by preferential task state reactivation. Sci. Adv. 7, eabf9616 (2021).

    PubMed  PubMed Central  Google Scholar 

  62. Wittkuhn, L., Chien, S., Hall-McMaster, S. & Schuck, N. W. Replay in minds and machines. Neurosci. Biobehav. Rev. 129, 367–388 (2021).

    PubMed  Google Scholar 

  63. Schapiro, A. C., McDevitt, E. A., Rogers, T. T., Mednick, S. C. & Norman, K. A. Human hippocampal replay during rest prioritizes weakly learned information and predicts memory performance. Nat. Commun. 9, 3920 (2018).

    PubMed  PubMed Central  Google Scholar 

  64. Genzel, L. et al. A consensus statement: defining terms for reactivation analysis. Phil. Trans. R. Soc. B 375, 20200001 (2020).

    PubMed  PubMed Central  Google Scholar 

  65. Tambini, A., Ketz, N. & Davachi, L. Enhanced brain correlations during rest are related to memory for recent experiences. Neuron 65, 280–290 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  66. Schönauer, M. et al. Decoding material-specific memory reprocessing during sleep in humans. Nat. Commun. 8, 15404 (2017).

    PubMed  PubMed Central  Google Scholar 

  67. Shanahan, L. K., Gjorgieva, E., Paller, K. A., Kahnt, T. & Gottfried, J. A. Odor-evoked category reactivation in human ventromedial prefrontal cortex during sleep promotes memory consolidation. eLife 7, e39681 (2018).

    PubMed  PubMed Central  Google Scholar 

  68. Deuker, L. et al. Memory consolidation by replay of stimulus-specific neural activity. J. Neurosci. 33, 19373–19383 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  69. Antony, J. W. & Schapiro, A. C. Active and effective replay: systems consolidation reconsidered again. Nat. Rev. Neurosci. 20, 506–507 (2019).

    CAS  PubMed  Google Scholar 

  70. Schuck, N. W. et al. Human orbitofrontal cortex represents a cognitive map of state space. Neuron 91, 1402–1412 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  71. Kornysheva, K. et al. Neural competitive queuing of ordinal structure underlies skilled sequential action. Neuron 101, 1166–1180.e3 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  72. Hahamy, A., Wilf, M., Rosin, B., Behrmann, M. & Malach, R. How do the blind ‘see’? The role of spontaneous brain activity in self-generated perception. Brain 144, 340–353 (2021).

    PubMed  Google Scholar 

  73. Shohamy, D. & Daw, N. D. Integrating memories to guide decisions. Curr. Opin. Behav. Sci. 5, 85–90 (2015).

    Google Scholar 

  74. Dolan, R. J. & Dayan, P. Goals and habits in the brain. Neuron 80, 312–325 (2013). This article provides a road map for the neuroscience research of decision-making, with implications for the ‘representation-rich’ paradigm.

    CAS  PubMed  PubMed Central  Google Scholar 

  75. Behrens, T. E. J. et al. What is a cognitive map? organizing knowledge for flexible behavior. Neuron 100, 490–509 (2018). This article discusses the functional relevance of a cognitive map, especially in terms of its computational role in human cognition as well as artificial intelligence.

    CAS  PubMed  Google Scholar 

  76. Vaz, A. P., Wittig, J. H., Inati, S. K. & Zaghloul, K. A. Replay of cortical spiking sequences during human memory retrieval. Science 367, 1131–1134 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  77. Norman, Y. et al. Hippocampal sharp-wave ripples linked to visual episodic recollection in humans. Science 365, eaax1030 (2019).

    CAS  PubMed  Google Scholar 

  78. Ólafsdóttir, H. F., Barry, C., Saleem, A. B., Hassabis, D. & Spiers, H. J. Hippocampal place cells construct reward related sequences through unexplored space. eLife 4, e06063 (2015).

    PubMed  PubMed Central  Google Scholar 

  79. Moneta, N., Garvert, M. M., Heekeren, H. R. & Schuck, N. W. Parallel representation of context and multiple context-dependent values in ventro-medial prefrontal cortex. Preprint at bioRxiv https://doi.org/10.1101/2021.03.17.435844 (2021).

    Article  Google Scholar 

  80. Wang, F., Schoenbaum, G. & Kahnt, T. Interactions between human orbitofrontal cortex and hippocampus support model-based inference. PLoS Biol. 18, e3000578 (2020).

    PubMed  PubMed Central  Google Scholar 

  81. Schuck, N. W. et al. Medial prefrontal cortex predicts internally driven strategy shifts. Neuron 86, 331–340 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  82. Woodward, N. D. & Cascio, C. J. Resting-state functional connectivity in psychiatric disorders. JAMA Psychiatry 72, 743–744 (2015).

    PubMed  PubMed Central  Google Scholar 

  83. Snyder, A. Z. & Raichle, M. E. A brief history of the resting state: the Washington University perspective. Neuroimage 62, 902–910 (2012).

    PubMed  Google Scholar 

  84. Smith, S. M. et al. Resting-state fMRI in the human connectome project. Neuroimage 80, 144–168 (2013).

    PubMed  Google Scholar 

  85. Raichle, M. E. & Snyder, A. Z. A default mode of brain function: a brief history of an evolving idea. Neuroimage 37, 1083–1090 (2007).

    PubMed  Google Scholar 

  86. Gusnard, D. A. & Raichle, M. E. Searching for a baseline: functional imaging and the resting human brain. Nat. Rev. Neurosci. 2, 685–694 (2001).

    CAS  PubMed  Google Scholar 

  87. Raichle, M. E. The brain’s default mode network. Annu. Rev. Neurosci. 38, 433–447 (2015).

    CAS  PubMed  Google Scholar 

  88. Buckner, R. L. & DiNicola, L. M. The brain’s default network: updated anatomy, physiology and evolving insights. Nat. Rev. Neurosci. 20, 593–608 (2019).

    CAS  PubMed  Google Scholar 

  89. Andrews-Hanna, J. R., Reidler, J. S., Sepulcre, J., Poulin, R. & Buckner, R. L. Functional-anatomic fractionation of the brain’s default network. Neuron 65, 550–562 (2010).

    CAS  PubMed  PubMed Central  Google Scholar 

  90. Raichle, M. E. et al. A default mode of brain function. Proc. Natl Acad. Sci. USA 98, 676–682 (2001).

    CAS  PubMed  PubMed Central  Google Scholar 

  91. Buckner, R. L. The serendipitous discovery of the brain’s default network. Neuroimage 62, 1137–1145 (2012).

    PubMed  Google Scholar 

  92. Agnati, L. F., Guidolin, D., Battistin, L., Pagnoni, G. & Fuxe, K. The neurobiology of imagination: possible role of interaction-dominant dynamics and default mode network. Front. Psychol. 4, 296 (2013).

    PubMed  PubMed Central  Google Scholar 

  93. Smallwood, J. & Schooler, J. W. The science of mind wandering: empirically navigating the stream of consciousness. Annu. Rev. Psychol. 66, 487–518 (2015).

    PubMed  Google Scholar 

  94. Schacter, D. L., Addis, D. R. & Buckner, R. L. Remembering the past to imagine the future: the prospective brain. Nat. Rev. Neurosci. 8, 657–661 (2007).

    CAS  PubMed  Google Scholar 

  95. Hassabis, D. & Maguire, E. A. Deconstructing episodic memory with construction. Trends Cogn. Sci. 11, 299–306 (2007).

    PubMed  Google Scholar 

  96. Meyer, M. L., Davachi, L., Ochsner, K. N. & Lieberman, M. D. Evidence that default network connectivity during rest consolidates social information. Cereb. Cortex 29, 1910–1920 (2019).

    PubMed  Google Scholar 

  97. Constantinescu, A. O., O’Reilly, J. X. & Behrens, T. E. J. Organizing conceptual knowledge in humans with a gridlike code. Science 352, 1464 (2016). This study provides evidence for grid-like coding of non-physical space in the brain areas that constitute the DMN.

    CAS  PubMed  PubMed Central  Google Scholar 

  98. Park, S. A., Miller, D. S. & Boorman, E. D. Novel inferences in a multidimensional social network use a grid-like code. Preprint at bioRxiv https://doi.org/10.1101/2020.05.29.124651 (2020).

    Article  PubMed  PubMed Central  Google Scholar 

  99. Baldassano, C., Hasson, U. & Norman, K. A. Representation of real-world event schemas during narrative perception. J. Neurosci. 38, 9689–9699 (2018).

    CAS  PubMed  PubMed Central  Google Scholar 

  100. Vidaurre, D., Myers, N. E., Stokes, M., Nobre, A. C. & Woolrich, M. W. Temporally unconstrained decoding reveals consistent but time-varying stages of stimulus processing. Cereb. Cortex 29, 863–874 (2019).

    PubMed  Google Scholar 

  101. Kaplan, R. et al. Hippocampal sharp-wave ripples influence selective activation of the default mode network. Curr. Biol. 26, 686–691 (2016). This study shows a selective coupling between hippocampal ripples and activation of the DMN in the monkey brain.

    CAS  PubMed  PubMed Central  Google Scholar 

  102. Liu, X. et al. Multimodal neural recordings with Neuro-FITM uncover diverse patterns of cortical–hippocampal interactions. Nat. Neurosci. 24, 886–896 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  103. Wang, M., Foster, D. J. & Pfeiffer, B. E. Alternating sequences of future and past behavior encoded within hippocampal theta oscillations. Science 370, 247 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  104. Buzsáki, G. & Moser, E. I. Memory, navigation and theta rhythm in the hippocampal-entorhinal system. Nat. Neurosci. 16, 130 (2013). This article provides a synthesis of the hippocampal–entorhinal system in supporting both navigation and memory.

    PubMed  PubMed Central  Google Scholar 

  105. Redish, A. D. Vicarious trial and error. Nat. Rev. Neurosci. 17, 147 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  106. Sylvester, C. M. et al. Individual-specific functional connectivity of the amygdala: a substrate for precision psychiatry. Proc. Natl Acad. Sci. USA 117, 3808–3818 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  107. McCutcheon, R. A., Marques, T. R. & Howes, O. D. Schizophrenia — an overview. JAMA Psychiatry 77, 201–210 (2020).

    PubMed  Google Scholar 

  108. Suh, J., Foster, D. J., Davoudi, H., Wilson, M. A. & Tonegawa, S. Impaired hippocampal ripple-associated replay in a mouse model of schizophrenia. Neuron 80, 484–493 (2013).

    CAS  PubMed  Google Scholar 

  109. Altimus, C., Harrold, J., Jaaro-Peled, H., Sawa, A. & Foster, D. J. Disordered ripples are a common feature of genetically distinct mouse models relevant to schizophrenia. Mol. Neuropsychiatry 1, 52–59 (2015).

    PubMed  PubMed Central  Google Scholar 

  110. Zeidman, P. & Maguire, E. A. Anterior hippocampus: the anatomy of perception, imagination and episodic memory. Nat. Rev. Neurosci. 17, 173–182 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  111. Adams, R. A. et al. Impaired theta phase coupling underlies frontotemporal dysconnectivity in schizophrenia. Brain 143, 1261–1277 (2020).

    PubMed  PubMed Central  Google Scholar 

  112. Titone, D., Ditman, T., Holzman, P. S., Eichenbaum, H. & Levy, D. L. Transitive inference in schizophrenia: impairments in relational memory organization. Schizophr. Res. 68, 235–247 (2004).

    PubMed  Google Scholar 

  113. Nour, M. M., Liu, Y., Arumuham, A., Kurth-Nelson, Z. & Dolan, R. Impaired neural replay of inferred relationships in schizophrenia. Cell 184, 4315–4328 (2021). This study is the first demonstration of augmented ripple power, cognitive map deficit and diminished replay in PScz.

    CAS  PubMed  PubMed Central  Google Scholar 

  114. Whitfield-Gabrieli, S. & Ford, J. M. Default mode network activity and connectivity in psychopathology. Annu. Rev. Clin. Psychol. 8, 49–76 (2012).

    PubMed  Google Scholar 

  115. Huys, Q. J. et al. Interplay of approximate planning strategies. Proc. Natl Acad. Sci. USA 112, 3098–3103 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  116. Huys, Q. J. et al. Bonsai trees in your head: how the Pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol. 8, e1002410 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  117. Heller, A. S. & Bagot, R. C. Is hippocampal replay a mechanism for anxiety and depression? JAMA Psychiatry 77, 431–432 (2020).

    PubMed  Google Scholar 

  118. Lewis, P. A., Knoblich, G. & Poe, G. How memory replay in sleep boosts creative problem-solving. Trends Cogn. Sci. 22, 491–503 (2018).

    PubMed  PubMed Central  Google Scholar 

  119. Klinzing, J. G., Niethard, N. & Born, J. Mechanisms of systems memory consolidation during sleep. Nat. Neurosci. 22, 1598–1610 (2019).

    CAS  PubMed  Google Scholar 

  120. Wei, Y., Krishnan, G. P., Marshall, L., Martinetz, T. & Bazhenov, M. Stimulation augments spike sequence replay and memory consolidation during slow-wave sleep. J. Neurosci. 40, 811–824 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  121. Tamminen, J., Lambon Ralph, M. A. & Lewis, P. A. The role of sleep spindles and slow-wave activity in integrating new information in semantic memory. J. Neurosci. 33, 15376–15381 (2013).

    CAS  PubMed  PubMed Central  Google Scholar 

  122. Wilson, M. A. & McNaughton, B. L. Reactivation of hippocampal ensemble memories during sleep. Science 265, 676–679 (1994).

    CAS  PubMed  Google Scholar 

  123. Pfeiffer, B. E. The content of hippocampal “replay”. Hippocampus 30, 6–18 (2020). Human replay (measured with MEG) is found to bear many similarities to rodent replay, and here is shown to represent sensory and structural information independently, thus facilitating generalization in a novel context.

    PubMed  Google Scholar 

  124. Zielinski, M. C., Shin, J. D. & Jadhav, S. P. Hippocampal theta sequences in REM sleep during spatial learning. Preprint at bioRxiv https://doi.org/10.1101/2021.04.15.439854 (2021).

    Article  Google Scholar 

  125. McNamee, D. C., Stachenfeld, K. L., Botvinick, M. M. & Gershman, S. J. Flexible modulation of sequence generation in the entorhinal–hippocampal system. Nat. Neurosci. 24, 851–862 (2021).

    CAS  PubMed  PubMed Central  Google Scholar 

  126. Krause, E. L. & Drugowitsch, J. A large majority of awake hippocampal sharp-wave ripples feature spatial trajectories with momentum. Neuron 110, 722–733.e8 (2021).

    PubMed  Google Scholar 

  127. Stella, F., Baracskay, P., O’Neill, J. & Csicsvari, J. Hippocampal reactivation of random trajectories resembling Brownian diffusion. Neuron 102, 450–461.e7 (2019).

    CAS  PubMed  Google Scholar 

  128. Pereira, S. I. R. & Lewis, P. A. Sleeping through brain excitation and inhibition. Nat. Neurosci. 23, 1037–1039 (2020).

    CAS  PubMed  Google Scholar 

  129. Belal, S. et al. Identification of memory reactivation during sleep by EEG classification. Neuroimage 176, 203–214 (2018).

    PubMed  Google Scholar 

  130. Higgins, C. Uncovering Temporal Structure in Neural Data with Statistical Machine Learning Models (Univ. Oxford, 2019).

  131. Kriegeskorte, N. et al. Matching categorical object representations in inferior temporal cortex of man and monkey. Neuron 60, 1126–1141 (2008).

    CAS  PubMed  PubMed Central  Google Scholar 

  132. Barron, H. C., Mars, R. B., Dupret, D., Lerch, J. P. & Sampaio-Baptista, C. Cross-species neuroscience: closing the explanatory gap. Phil. Trans. R. Soc. B 376, 20190633 (2021).

    PubMed  Google Scholar 

  133. Barron, H. C. et al. Neuronal computation underlying inferential reasoning in humans and mice. Cell 183, 228–243.e21 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  134. Akam, T., Costa, R. & Dayan, P. Simple plans or sophisticated habits? State, transition and learning interactions in the two-step task. PLoS Comput. Biol. 11, e1004648 (2015).

    PubMed  PubMed Central  Google Scholar 

  135. Miranda, B., Malalasekera, W. M. N., Behrens, T. E., Dayan, P. & Kennerley, S. W. Combined model-free and model-sensitive reinforcement learning in non-human primates. PLoS Comput. Biol. 16. e1007944 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  136. Mack, M. L., Preston, A. R. & Love, B. C. Ventromedial prefrontal cortex compression during concept learning. Nat. Commun. 11, 46 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  137. Morton, N. W., Schlichting, M. L. & Preston, A. R. Representations of common event structure in medial temporal lobe and frontoparietal cortex support efficient inference. Proc. Natl Acad. Sci. USA 117, 29338–29345 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  138. Al Roumi, F., Marti, S., Wang, L., Amalric, M. & Dehaene, S. Mental compression of spatial sequences in human working memory using numerical and geometrical primitives. Neuron 109, 2627–2639.e4 (2021).

    CAS  PubMed  Google Scholar 

  139. Bernardi, S. et al. The geometry of abstraction in hippocampus and prefrontal cortex. Cell 183, 954–967.e21 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  140. Higgins, I. et al. Darla: improving zero-shot transfer in reinforcement learning. Proc. Int. Conf. Mach. Learn. 70, 1480–1490 (2017).

    Google Scholar 

  141. Whittington, J. C. et al. The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation. Cell 183, 1249–1263.e23 (2020). This study describes a unifying computational model of the hippocampal–entorhinal system for generalization and inference in an arbitrary relational graph (including physical space and memory).

    CAS  PubMed  PubMed Central  Google Scholar 

  142. Schwartenbeck, P. et al. Generative replay for compositional visual understanding in the prefrontal-hippocampal circuit. Preprint at bioRxiv https://doi.org/10.1101/2021.06.06.447249 (2021).

    Article  Google Scholar 

  143. Kragel, P. A., Knodt, A. R., Hariri, A. R. & LaBar, K. S. Decoding spontaneous emotional states in the human brain. PLoS Biol. 14, e2000106 (2016).

    PubMed  PubMed Central  Google Scholar 

  144. Tusche, A., Smallwood, J., Bernhardt, B. C. & Singer, T. Classifying the wandering mind: revealing the affective content of thoughts during task-free rest periods. Neuroimage 97, 107–116 (2014).

    PubMed  Google Scholar 

  145. Van Calster, L., D’Argembeau, A., Salmon, E., Peters, F. & Majerus, S. Fluctuations of attentional networks and default mode network during the resting state reflect variations in cognitive states: evidence from a novel resting-state experience sampling method. J. Cogn. Neurosci. 29, 95–113 (2017).

    PubMed  Google Scholar 

  146. Smallwood, J. et al. The neural correlates of ongoing conscious thought. iScience 24, 102132 (2021).

    PubMed  PubMed Central  Google Scholar 

  147. Colgin, L. L. Rhythms of the hippocampal network. Nat. Rev. Neurosci. 17, 239 (2016).

    CAS  PubMed  PubMed Central  Google Scholar 

  148. Ólafsdóttir, H. F., Carpenter, F. & Barry, C. Task demands predict a dynamic switch in the content of awake hippocampal replay. Neuron 96, e926 (2017).

    Google Scholar 

  149. Gillespie, A. K. et al. Hippocampal replay reflects specific past experiences rather than a plan for subsequent choice. Neuron 109, 3149–3163.e6 (2021).

    CAS  PubMed  Google Scholar 

  150. Chadwick, A., van Rossum, M. C. & Nolan, M. F. Independent theta phase coding accounts for CA1 population sequences and enables flexible remapping. eLife 4, e03542 (2015).

    PubMed Central  Google Scholar 

  151. Skaggs, W. E., McNaughton, B. L., Wilson, M. A. & Barnes, C. A. Theta phase precession in hippocampal neuronal populations and the compression of temporal sequences. Hippocampus 6, 149–172 (1996).

    CAS  PubMed  Google Scholar 

  152. Wikenheiser, A. M. & Redish, A. D. Hippocampal theta sequences reflect current goals. Nat. Neurosci. 18, 289–294 (2015).

    CAS  PubMed  PubMed Central  Google Scholar 

  153. Johnson, A. & Redish, A. D. Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point. J. Neurosci. 27, 12176–12189 (2007).

    CAS  PubMed  PubMed Central  Google Scholar 

  154. Kay, K. et al. Constant sub-second cycling between representations of possible futures in the hippocampus. Cell 180, 552–567.e25 (2020).

    CAS  PubMed  PubMed Central  Google Scholar 

  155. Schultz, W., Dayan, P. & Montague, P. R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).

    CAS  PubMed  Google Scholar 

  156. Doll, B. B., Simon, D. A. & Daw, N. D. The ubiquity of model-based reinforcement learning. Curr. Opin. Neurobiol. 22, 1075–1081 (2012).

    CAS  PubMed  PubMed Central  Google Scholar 

  157. Daw, N. D. & Dayan, P. The algorithmic anatomy of model-based evaluation. Phil. Trans. R. Soc. B 369, 20130478 (2014).

    PubMed  PubMed Central  Google Scholar 

  158. Siegel, S. & Allan, L. G. The widespread influence of the Rescorla-Wagner model. Psychon. Bull. Rev. 3, 314–321 (1996).

    CAS  PubMed  Google Scholar 

  159. Sutton, R. S. & Barto, A. G. in Proceedings of the Ninth Annual Conference of the Cognitive Science Society (1987).

  160. Gallistel, C. R., LoLordo, V. M., Rozin, P. & Seligman, M. E. P. Robert A. Rescorla (1940–2020). Am. Psychol. 76, 391–392 (2021).

    CAS  PubMed  Google Scholar 

  161. Vikbladh, O. M. et al. Hippocampal contributions to model-based planning and spatial memory. Neuron 102, 683–693.e4 (2019).

    CAS  PubMed  PubMed Central  Google Scholar 

  162. Daw, N. D., Gershman, S. J., Seymour, B., Dayan, P. & Dolan, R. J. Model-based influences on humans’ choices and striatal prediction errors. Neuron 69, 1204–1215 (2011).

    CAS  PubMed  PubMed Central  Google Scholar 

  163. Kool, W., Cushman, F. A. & Gershman, S. J. When does model-based control pay off? PLoS Comput. Biol. 12, e1005090 (2016).

    PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

The authors thank E. Wimmer, C. Higgins and P. Schwartenbeck for helpful discussions and fruitful collaborations regarding the work and ideas presented in this Review. This work was supported by the Fundamental Research Funds for the Central Universities (to Y.L.), a Wellcome Trust Investigator Award (098362/Z/12/Z to R.J.D.), a UCL Welcome PhD Fellowship for Clinicians (102186/B/13/Z to M.M.N.), a Wellcome Trust Senior Research Fellowship (104765/Z/14/Z to T.B.), a Principal Research Fellowship (219525/Z/19/Z to T.B.), a James S. McDonnell Foundation Award (JSMF220020372 to T.B.), an Independent Research Group Grant from the Max Planck Society (M.TN.A.BILD0004 to N.W.S.) and a Starting Grant from the European Union (ERC-2019-StG REPLAY-852669 to N.W.S.). M.M.N. is a predoctoral fellow of the International Max Planck Research School on Computational Methods in Psychiatry and Ageing Research (https://www.mps-ucl-centre.mpg.de/en/comp2psych). The Max Planck UCL Centre for Computational Psychiatry and Ageing Research is supported by UCL and the Max Planck Society. The Wellcome Centre for Human Neuroimaging is supported by core funding from the Wellcome Trust (203147/Z/16/Z). The Wellcome Centre for Integrative Neuroimaging is supported by core funding from the Wellcome Trust (203139/Z/16/Z).

Author information

Authors and Affiliations

Authors

Contributions

Y.L. researched data for article and contributed substantially to discussion of the content, writing, review and editing of the manuscript before submission. T.E.J.B. contributed substantially to discussion of the content of the manuscript and contributed to the writing, review and editing of the manuscript. R.J.D. contributed to the writing, review and editing of the manuscript. M.M.N. and N.W.S. contributed to the writing of the manuscript.

Corresponding author

Correspondence to Yunzhe Liu.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Reviews Neuroscience thanks J. Andrews-Hanna and the other anonymous reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Glossary

Task

An experiment designed to manipulate an assumed cognitive process.

Off-task

Period without explicit task demand (for example, during rest).

Functional connectivity

Temporal dependency of neuronal activation (for example, correlation) between anatomically separated brain regions.

Power

The strength of a signal in a given frequency band.

Decoding

Reading out task-related information from neural activity.

Pairwise multivoxel correlation

Correlation between all pairs of voxels of interest using the entire time course of the functional MRI signal.

Representational similarity analysis

Measure of the similarity of neural activity among different conditions.

Multivoxel patterns

Neural activity profile of multiple voxels in the brain.

Transition matrix

A matrix that stores the probability of transition from state s to state s′.

Regressors

Independent variables in a regression model.

Multivariate decision boundary

A region of a problem space where the output label of a classifier is ambiguous.

Charles Bonnet syndrome

A condition where visual hallucinations occur as a result of vision loss.

Resting states

The states when an explicit task is not being performed.

Brownian diffusive spatial trajectories

Trajectories whose movement is random in space.

Superdiffusive dynamics

Random movement but with sudden jumps.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Liu, Y., Nour, M.M., Schuck, N.W. et al. Decoding cognition from spontaneous neural activity. Nat Rev Neurosci 23, 204–214 (2022). https://doi.org/10.1038/s41583-022-00570-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41583-022-00570-z

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing