The hippocampus has long been known to be involved in spatial navigational learning in rodents1,2, and in memory for events in rodents3,4, primates5 and humans6. A unifying property of both navigation and event memory is a requirement for dealing with temporally sequenced information. Reactivation of temporally sequenced memories for previous behavioural experiences has been reported in sleep in rats7,8. Here we report that sequential replay occurs in the rat hippocampus during awake periods immediately after spatial experience. This replay has a unique form, in which recent episodes of spatial experience are replayed in a temporally reversed order. This replay is suggestive of a role in the evaluation of event sequences in the manner of reinforcement learning models. We propose that such replay might constitute a general mechanism of learning and memory.
Subscribe to Journal
Get full journal access for 1 year
only $3.90 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
O'Keefe, J. & Nadel, L. The Hippocampus as a Cognitive Map (Clarendon, London, 1978)
Morris, R. G. M., Garrud, P., Rawlins, J. N. P. & O'Keefe, J. Place navigation impaired in rats with hippocampal lesions. Nature 297, 681–683 (1982)
Olton, D. S. & Samuelson, R. J. Remembrance of places past: spatial memory in rats. J. Exp. Psychol. Anim. Behav. Process. 2, 97–116 (1976)
Steele, R. J. & Morris, R. G. M. Delay-dependent impairment of a matching-to-place task with chronic and intrahippocampal infusion of the NMDA-antagonist d-AP5. Hippocampus 9, 118–136 (1999)
Gaffan, D. Scene-specific memory for objects: a model of episodic memory impairment in monkeys with fornix transection. J. Cogn. Neurosci. 6, 305–320 (1994)
Scoville, W. B. & Milner, B. Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21 (1957)
Louie, K. & Wilson, M. A. Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron 29, 145–156 (2001)
Lee, A. K. & Wilson, M. A. Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 36, 1183–1194 (2002)
Wilson, M. A. & McNaughton, B. L. Dynamics of the hippocampal ensemble code for space. Science 261, 1055–1058 (1993)
O'Keefe, J. & Dostrovsky, J. The hippocampus as a spatial map: preliminary evidence from unit activity in the freely moving rat. Brain Res. 34, 171–175 (1971)
Press, W. H., Teukolsky, S. A., Vetterling, W. T. & Flannery, B. P. Numerical Recipes in C: The Art of Scientific Computing 2nd edn (Cambridge Univ. Press, Cambridge, 1992)
Ranck, J. B. Jr. Studies on single neurons in dorsal hippocampal formation and in septum in unrestrained rats. I. Behavioral correlates and firing repertoires. Exp. Neurol. 42, 461–531 (1973)
Buzsaki, G. Hippocampal sharp waves: their origin and significance. Brain Res. 398, 242–252 (1986)
Buzsaki, G. A two-stage model of memory trace formation: a role for ‘noisy’ brain states. Neuroscience 31, 551–570 (1989)
Kaelbling, L. P., Littman, M. L. & Moore, A. W. Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)
Sutton, R. S. Learning to predict by the methods of temporal differences. Mach. Learn. 3, 9–44 (1988)
Schultz, W., Dayan, P. & Montague, P. R. A neural subtstrate of prediction and reward. Science 275, 1593–1599 (1997)
Foster, D. J., Morris, R. G. M. & Dayan, P. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10, 1–16 (2000)
Lin, L. J. Self-improving reactive agents based on reinforcement learning, planning and teaching. Mach. Learn. 8, 293–321 (1992)
Sutton, R. S. & Singh, S. P. in Proceedings of the Eighth Yale Workshop on Adaptive and Learning Systems 31–36 (Yale Univ., New Haven, 1994)
Cichosz, P. An analysis of experience replay in temporal difference learning. Cybernet. Syst. 30, 341–363 (1999)
Reynolds, S. Experience Stack Reinforcement Learning for Off-policy Control. Cognitive Science Technical Report CSRP-02–1 (Birmingham Univ., Birmingham, 2002)
Stickgold, R., Whidbee, D., Schirmer, B., Patel, V. & Hobson, J. A. Visual discrimination task improvement: a multi-step process occurring during sleep. J. Cogn. Neurosci. 12, 246–254 (2000)
Siapas, A. G. & Wilson, M. A. Coordinated interactions between hippocampal wripples and cortical spindles during slow-wave sleep. Neuron 21, 1123–1128 (1998)
We thank V. Ego-Stengel, P. Dayan and M. Fee for comments on the manuscript. This work was supported by an NIH grant to M.A.W.
Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.
About this article
Cite this article
Foster, D., Wilson, M. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680–683 (2006). https://doi.org/10.1038/nature04587
Current Opinion in Behavioral Sciences (2021)
Information and communication technologies (ICT)-enabled severe moral communities and how the (Covid19) pandemic might bring new ones
International Journal of Information Management (2021)
Theta Oscillations Coincide with Sustained Hyperpolarization in CA3 Pyramidal Cells, Underlying Decreased Firing
Cell Reports (2020)
Philosophical Transactions of the Royal Society B: Biological Sciences (2020)