Reverse replay of behavioural sequences in hippocampal place cells during the awake state

Abstract

The hippocampus has long been known to be involved in spatial navigational learning in rodents1,2, and in memory for events in rodents3,4, primates5 and humans6. A unifying property of both navigation and event memory is a requirement for dealing with temporally sequenced information. Reactivation of temporally sequenced memories for previous behavioural experiences has been reported in sleep in rats7,8. Here we report that sequential replay occurs in the rat hippocampus during awake periods immediately after spatial experience. This replay has a unique form, in which recent episodes of spatial experience are replayed in a temporally reversed order. This replay is suggestive of a role in the evaluation of event sequences in the manner of reinforcement learning models. We propose that such replay might constitute a general mechanism of learning and memory.

Access options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

Figure 1: A sequence of place fields.
Figure 2: Reverse replay events during a single lap.
Figure 3: Reverse replay events, by lap, for an entire recording session.
Figure 4: Analysis of reverse replay across all recording sessions.

References

  1. 1

    O'Keefe, J. & Nadel, L. The Hippocampus as a Cognitive Map (Clarendon, London, 1978)

    Google Scholar 

  2. 2

    Morris, R. G. M., Garrud, P., Rawlins, J. N. P. & O'Keefe, J. Place navigation impaired in rats with hippocampal lesions. Nature 297, 681–683 (1982)

    ADS  CAS  Article  Google Scholar 

  3. 3

    Olton, D. S. & Samuelson, R. J. Remembrance of places past: spatial memory in rats. J. Exp. Psychol. Anim. Behav. Process. 2, 97–116 (1976)

    Article  Google Scholar 

  4. 4

    Steele, R. J. & Morris, R. G. M. Delay-dependent impairment of a matching-to-place task with chronic and intrahippocampal infusion of the NMDA-antagonist d-AP5. Hippocampus 9, 118–136 (1999)

    CAS  Article  Google Scholar 

  5. 5

    Gaffan, D. Scene-specific memory for objects: a model of episodic memory impairment in monkeys with fornix transection. J. Cogn. Neurosci. 6, 305–320 (1994)

    CAS  Article  Google Scholar 

  6. 6

    Scoville, W. B. & Milner, B. Loss of recent memory after bilateral hippocampal lesions. J. Neurol. Neurosurg. Psychiatry 20, 11–21 (1957)

    CAS  Article  Google Scholar 

  7. 7

    Louie, K. & Wilson, M. A. Temporally structured replay of awake hippocampal ensemble activity during rapid eye movement sleep. Neuron 29, 145–156 (2001)

    CAS  Article  Google Scholar 

  8. 8

    Lee, A. K. & Wilson, M. A. Memory of sequential experience in the hippocampus during slow wave sleep. Neuron 36, 1183–1194 (2002)

    CAS  Article  Google Scholar 

  9. 9

    Wilson, M. A. & McNaughton, B. L. Dynamics of the hippocampal ensemble code for space. Science 261, 1055–1058 (1993)

    ADS  CAS  Article  Google Scholar 

  10. 10

    O'Keefe, J. & Dostrovsky, J. The hippocampus as a spatial map: preliminary evidence from unit activity in the freely moving rat. Brain Res. 34, 171–175 (1971)

    CAS  Article  Google Scholar 

  11. 11

    Press, W. H., Teukolsky, S. A., Vetterling, W. T. & Flannery, B. P. Numerical Recipes in C: The Art of Scientific Computing 2nd edn (Cambridge Univ. Press, Cambridge, 1992)

    Google Scholar 

  12. 12

    Ranck, J. B. Jr. Studies on single neurons in dorsal hippocampal formation and in septum in unrestrained rats. I. Behavioral correlates and firing repertoires. Exp. Neurol. 42, 461–531 (1973)

    Google Scholar 

  13. 13

    Buzsaki, G. Hippocampal sharp waves: their origin and significance. Brain Res. 398, 242–252 (1986)

    CAS  Article  Google Scholar 

  14. 14

    Buzsaki, G. A two-stage model of memory trace formation: a role for ‘noisy’ brain states. Neuroscience 31, 551–570 (1989)

    CAS  Article  Google Scholar 

  15. 15

    Kaelbling, L. P., Littman, M. L. & Moore, A. W. Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237–285 (1996)

    Article  Google Scholar 

  16. 16

    Sutton, R. S. Learning to predict by the methods of temporal differences. Mach. Learn. 3, 9–44 (1988)

    Google Scholar 

  17. 17

    Schultz, W., Dayan, P. & Montague, P. R. A neural subtstrate of prediction and reward. Science 275, 1593–1599 (1997)

    CAS  Article  Google Scholar 

  18. 18

    Foster, D. J., Morris, R. G. M. & Dayan, P. A model of hippocampally dependent navigation, using the temporal difference learning rule. Hippocampus 10, 1–16 (2000)

    CAS  Article  Google Scholar 

  19. 19

    Lin, L. J. Self-improving reactive agents based on reinforcement learning, planning and teaching. Mach. Learn. 8, 293–321 (1992)

    Google Scholar 

  20. 20

    Sutton, R. S. & Singh, S. P. in Proceedings of the Eighth Yale Workshop on Adaptive and Learning Systems 31–36 (Yale Univ., New Haven, 1994)

    Google Scholar 

  21. 21

    Cichosz, P. An analysis of experience replay in temporal difference learning. Cybernet. Syst. 30, 341–363 (1999)

    Article  Google Scholar 

  22. 22

    Reynolds, S. Experience Stack Reinforcement Learning for Off-policy Control. Cognitive Science Technical Report CSRP-02–1 (Birmingham Univ., Birmingham, 2002)

    Google Scholar 

  23. 23

    Stickgold, R., Whidbee, D., Schirmer, B., Patel, V. & Hobson, J. A. Visual discrimination task improvement: a multi-step process occurring during sleep. J. Cogn. Neurosci. 12, 246–254 (2000)

    CAS  Article  Google Scholar 

  24. 24

    Siapas, A. G. & Wilson, M. A. Coordinated interactions between hippocampal wripples and cortical spindles during slow-wave sleep. Neuron 21, 1123–1128 (1998)

    CAS  Article  Google Scholar 

Download references

Acknowledgements

We thank V. Ego-Stengel, P. Dayan and M. Fee for comments on the manuscript. This work was supported by an NIH grant to M.A.W.

Author information

Affiliations

Authors

Corresponding author

Correspondence to David J. Foster.

Ethics declarations

Competing interests

Reprints and permissions information is available at npg.nature.com/reprintsandpermissions. The authors declare no competing financial interests.

Supplementary information

Supplementary Figures

This file contains Supplementary Figures 1–7 (PDF 365 kb)

Supplementary Figure Legends

This file contains text to accompany the above Supplementary Figures. (DOC 23 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Foster, D., Wilson, M. Reverse replay of behavioural sequences in hippocampal place cells during the awake state. Nature 440, 680–683 (2006). https://doi.org/10.1038/nature04587

Download citation

Further reading

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing