Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Learned spatiotemporal sequence recognition and prediction in primary visual cortex

This article has been updated

Abstract

Learning to recognize and predict temporal sequences is fundamental to sensory perception and is impaired in several neuropsychiatric disorders, but little is known about where and how this occurs in the brain. We discovered that repeated presentations of a visual sequence over a course of days resulted in evoked response potentiation in mouse V1 that was highly specific for stimulus order and timing. Notably, after V1 was trained to recognize a sequence, cortical activity regenerated the full sequence even when individual stimulus elements were omitted. Our results advance the understanding of how the brain makes 'intelligent guesses' on the basis of limited information to form visual percepts and suggest that it is possible to study the mechanistic basis of this high-level cognitive ability by studying low-level sensory systems.

This is a preview of subscription content, access via your institution

Access options

Rent or buy this article

Prices vary by article type

from$1.95

to$39.95

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Learned spatiotemporal sequence representations in V1.
Figure 2: Sequence learning is temporally specific.
Figure 3: Learning does not transfer between eyes and requires muscarinic acetylcholine receptors in V1, but not NMDA receptors.
Figure 4: Learned sequence representations are predictive and involve multiple cortical layers.

Similar content being viewed by others

Change history

  • 30 March 2014

    In the version of this article initially published online, the asterisk was defined in the Figure 2 legend as P > 0.05 rather than P < 0.05, and the grant number in the Acknowledgments was given as NIMH:1K99MH099654-01 rather than K99MH099654. The errors have been corrected for the print, PDF and HTML versions of this article.

References

  1. Lashley, K.S. The problem of serial order in behavior. in Cereberal Mechanisms in Behavior (ed. L.A. Jeffress) 112–131 (Wiley, New York, 1951).

  2. Mauk, M.D. & Buonomano, D.V. The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340 (2004).

    Article  CAS  Google Scholar 

  3. Rhodes, B.J., Bullock, D., Verwey, W.B., Averbeck, B.B. & Page, M.P.A. Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives. Hum. Mov. Sci. 23, 699–746 (2004).

    Article  Google Scholar 

  4. Song, S., Howard, J.H. & Howard, D.V. Perceptual sequence learning in a serial reaction time task. Exp. Brain Res. 189, 145–158 (2008).

    Article  Google Scholar 

  5. Gheysen, F., Van Opstal, F., Roggeman, C., Van Waelvelde, H. & Fias, W. The neural basis of implicit perceptual sequence learning. Front. Hum. Neurosci. 5, 137 (2011).

    Article  Google Scholar 

  6. Conway, C.M. & Christiansen, M.H. Sequential learning in non-human primates. Trends Cogn. Sci. 5, 539–546 (2001).

    Article  Google Scholar 

  7. Espinosa, J.S. & Stryker, M.P. Development and plasticity of the primary visual cortex. Neuron 75, 230–249 (2012).

    Article  CAS  Google Scholar 

  8. Frenkel, M.Y. et al. Instructive effect of visual experience in mouse visual cortex. Neuron 51, 339–349 (2006).

    Article  CAS  Google Scholar 

  9. Cooke, S.F. & Bear, M.F. Visual experience induces long-term potentiation in the primary visual cortex. J. Neurosci. 30, 16304–16313 (2010).

    Article  CAS  Google Scholar 

  10. Fahle, M. Perceptual learning: a case for early selection. J. Vis. 4, 879–890 (2004).

    Article  Google Scholar 

  11. Bear, M.F. & Singer, W. Modulation of visual cortical plasticity by acetylcholine and noradrenaline. Nature 320, 172–176 (1986).

    Article  CAS  Google Scholar 

  12. Chubykin, A.A., Roach, E.B., Bear, M.F. & Shuler, M.G. A cholinergic mechanism for reward timing within primary visual cortex. Neuron 77, 723–735 (2013).

    Article  CAS  Google Scholar 

  13. Mitzdorf, U. Current source-density method and application in cat cerebral cortex: investigation of evoked potentials and EEG phenomena. Physiol. Rev. 65, 37–100 (1985).

    Article  CAS  Google Scholar 

  14. Li, N. & DiCarlo, J.J. Unsupervised natural experience rapidly alters invariant object representation in visual cortex. Science 321, 1502–1507 (2008).

    Article  CAS  Google Scholar 

  15. Meyer, T. & Olson, C.R. Statistical learning of visual transitions in monkey inferotemporal cortex. Proc. Natl. Acad. Sci. USA 108, 19401–19406 (2011).

    Article  CAS  Google Scholar 

  16. Eagleman, S.L. & Dragoi, V. Image sequence reactivation in awake V4 networks. Proc. Natl. Acad. Sci. USA 109, 19450–19455 (2012).

    Article  CAS  Google Scholar 

  17. Gollisch, T. & Meister, M. Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron 65, 150–164 (2010).

    Article  CAS  Google Scholar 

  18. McIntyre, C.K., Ragozzino, M.E. & Gold, P.E. Intra-amygdala infusions of scopolamine impair performance on a conditioned place preference task but not a spatial radial maze task. Behav. Brain Res. 95, 219–226 (1998).

    Article  CAS  Google Scholar 

  19. Rao, R.P. & Ballard, D.H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87 (1999).

    Article  CAS  Google Scholar 

  20. Bastos, A.M. et al. Canonical microcircuits for predictive coding. Neuron 76, 695–711 (2012).

    Article  CAS  Google Scholar 

  21. Spratling, M.W. Predictive coding as a model of response properties in cortical area V1. J. Neurosci. 30, 3531–3543 (2010).

    Article  CAS  Google Scholar 

  22. Lee, T.S. & Mumford, D. Hierarchical Bayesian inference in the visual cortex. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 1434–1448 (2003).

    Article  Google Scholar 

  23. Wacongne, C., Changeux, J.P. & Dehaene, S. A neuronal model of predictive coding accounting for the mismatch negativity. J. Neurosci. 32, 3665–3678 (2012).

    Article  CAS  Google Scholar 

  24. Shuler, M.G. & Bear, M.F. Reward timing in the primary visual cortex. Science 311, 1606–1609 (2006).

    Article  CAS  Google Scholar 

  25. Wang, K.H. et al. In vivo two-photon imaging reveals a role of arc in enhancing orientation specificity in visual cortex. Cell 126, 389–402 (2006).

    Article  CAS  Google Scholar 

  26. Xu, S., Jiang, W., Dan, Y. & Poo, M.-m. Activity recall in a visual cortical ensemble. Nat. Neurosci. 15, 449–455 (2012).

    Article  CAS  Google Scholar 

  27. Bao, M., Yang, L., Rios, C., He, B. & Engel, S.A. Perceptual learning increases the strength of the earliest signals in visual cortex. J. Neurosci. 30, 15080–15084 (2010).

    Article  CAS  Google Scholar 

  28. Jehee, J.F., Ling, S., Swisher, J.D., van Bergen, R.S. & Tong, F. Perceptual learning selectively refines orientation representations in early visual cortex. J. Neurosci. 32, 16747–16753 (2012).

    Article  CAS  Google Scholar 

  29. Marr, D. Vision. A Computational Investigation into the Human Representation and Processing of Visual Information (W.H. Freeman and Company, San Francisco, 1982).

  30. Cooke, S.F. & Bear, M.F. How the mechanisms of long-term synaptic potentiation and depression serve experience-dependent plasticity in primary visual cortex. Phil. Trans. R. Soc. Lond. B 369, 20130284 (2014).

    Article  Google Scholar 

  31. Liu, J.K. & Buonomano, D.V. Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J. Neurosci. 29, 13172–13181 (2009).

    Article  CAS  Google Scholar 

  32. Hasselmo, M.E. The role of acetylcholine in learning and memory. Curr. Opin. Neurobiol. 16, 710–715 (2006).

    Article  CAS  Google Scholar 

  33. Minces, V.H., Alexander, A.S., Datlow, M., Alfonso, S.I. & Chiba, A.A. The role of visual cortex acetylcholine in learning to discriminate temporally modulated visual stimuli. Front. Behav. Neurosci. 7, 16 (2013).

    Article  CAS  Google Scholar 

  34. Pinto, L. et al. Fast modulation of visual perception by basal forebrain cholinergic neurons. Nat. Neurosci. 16, 1857–1863 (2013).

    Article  CAS  Google Scholar 

  35. Cajal, S.R. Comparative study of the sensory areas of the human cortex. in Clark University, 1889–1899: Decennial Celebration (ed. W.E. Story) 311–382 (Clark University Press, Worcestor, Massachusetts, 1899).

  36. Douglas, R.J. & Martin, K.A. Mapping the matrix: the ways of neocortex. Neuron 56, 226–238 (2007).

    Article  CAS  Google Scholar 

  37. DiCarlo, J.J., Zoccolan, D. & Rust, N.C. How does the brain solve visual object recognition? Neuron 73, 415–434 (2012).

    Article  CAS  Google Scholar 

  38. Howard, J.H. Jr., Howard, D.V., Japikse, K.C. & Eden, G.F. Dyslexics are impaired on implicit higher-order sequence learning, but not on implicit spatial context learning. Neuropsychologia 44, 1131–1144 (2006).

    Article  Google Scholar 

  39. Carroll, C.A., Boggs, J., O'Donnell, B.F., Shekhar, A. & Hetrick, W.P. Temporal processing dysfunction in schizophrenia. Brain Cogn. 67, 150–161 (2008).

    Article  Google Scholar 

  40. Siegert, R.J., Weatherall, M. & Bell, E.M. Is implicit sequence learning impaired in schizophrenia? A meta-analysis. Brain Cogn. 67, 351–359 (2008).

    Article  Google Scholar 

  41. Smith, J.G. & McDowall, J. The implicit sequence learning deficit in patients with Parkinson’s disease: A matter of impaired sequence integration? Neuropsychologia 44, 275–288 (2006).

    Article  Google Scholar 

Download references

Acknowledgements

We thank A. Heynen and S. Cooke for helpful comments on the manuscript, and S. Meagher for administrative support. This work was supported by the Howard Hughes Medical Institute, the Picower Institute Innovation Fund and a grant from the US National Institute of Mental Health (K99MH099654).

Author information

Authors and Affiliations

Authors

Contributions

J.P.G. and M.F.B contributed to study design and wrote the paper. J.P.G. performed all of the experiments.

Corresponding author

Correspondence to Mark F Bear.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Integrated supplementary information

Supplementary Figure 1 Sequence learning increases response magnitudes, decreases response latencies and modulates spectral power.

(a) Diagram illustrating sequence magnitude quantification. (b) Traces show the average response evoked by the sequence ABCD on day 1 (Baseline, gray) and after training (Day 5, blue) in the experimental group from Fig. 1c. (c) There is a significant effect of training (2–way RM ANOVA, P<0.001) and each sequence element, plotted individually, increases relative to baseline (P<0.001 for A,B and C, P<0.05 for D). (d) Training also has a significant effect on response latency (defined as the interval between stimulus onset and maximal negativity, 2–way RM ANOVA, P<0.001) and there is a decrease in response latency for each element (P<0.001). (e) Plots showing the power spectral density (PSD, estimated using Welch's method, transparent shading shows 95% confidence intervals around means indicated by solid lines) averaged across all animals in each group from Fig. 1c–d (Exp. n=6, Ctrl. n=4) during periods of active stimulus viewing (ABCD, DCBA, and ABCD300) or during inter–stimulus gray screen periods (Gray). (f) To highlight difference between groups, spectral power is plotted as a percent of the average ABCD power at each frequency. In both experimental and scrambled control groups visual stimuli drive higher spectral powers than gray screen at frequencies up to approximately 100 Hz. In the experimental mice, spectral power is on average higher while viewing ABCD than DCBA at low frequencies but this relationship reverses in the high gamma range (note, data was recorded with a 60 Hz notch filter). In control mice the familiar and novel sequence spectrums overlap at both high and low frequencies. In both groups, sequences presented with familiar timing result in more spectral power than a sequence presented with novel timing. Although precise interpretation is difficult, these results are broadly consistent with the idea that familiar and novel sequence either elicit or are modulated by different attentional states.

Supplementary Figure 2 Sequence recall is evident in multi-unit spiking activity.

(a) Each panel shows the multi-unit spike rasters (top) and peri-stimulus time histogram (PSTH, bottom) recorded on a single wire in a multi-channel bundle implanted in a single mouse. Responses are evoked by the sequences ABCD (trained, blue) and DCBA (novel, red). Dashed vertical lines indicate stimulus element onset times. (b) Average PSTH for the 5 channels shown in a. Before averaging, the PSTHs were normalized to the maxium spike rate evoked by either ABCD or DCBA on a per channel basis. Triangles indicate stimulus element onset times. (c) Average normalized PSTH from 6 mice (33 total recording channels).

Supplementary Figure 3 Sequence learning with 200 ms element hold time.

(a) To further verify that sequence learning is not specific to a 150 ms element hold time, a cohort of mice (n=4) was trained with the sequence ABCD using a 200 ms hold time. On the 5th day, the mice were tested on sequences ABCD and DCBA with the trained time and with ABCD using novel hold times (150 and 300 ms). (b) The largest response occurs when the trained sequence is presented with the trained timing. (c) There is a significant effect of sequence (1-way RM ANOVA, P=0.003) and ABCD with the trained timing is significantly larger (P<0.05) than DCBA or ABCD presented with novel timing.

Supplementary Figure 4 CPP blocks SRP but not Sequence Learning.

(a) Mice used in the CPP experiment (see Fig. 3) were reassigned after a 3–day washout period into CPP (n=6) and vehicle control (saline, n=9) groups and exposed to the SRP induction protocol (described previously in detail, see Frenkel et al. 2006). Briefly, on each of four training days the mice were injected with CPP or saline (see methods) and exposed to a sinusoidal grating (0.05 cy/deg, 100% contrast) oriented at θ1 and phase–reversing at 1 Hz. On the 5th day, mice were shown phase–reversing gratings oriented at the familiar and novel orientations (θ1 and θ2). (b) Analysis of phase–reversal locked VEP magnitudes shows a statistically significant interaction between treatment and recording day (days 1 and 5, P<0.001). Post–hoc analysis reveals a significant difference between CPP and control treatments (P<0.05) and a significant VEP increase between days 1 and 5 for θ1 in the control group (P<0.001), but not in the CPP group. There is also a significant interaction between treatment and stimulus orientation on day 5 (P<0.001), a significant difference between the CPP and saline groups (P<0.05), and the response to the familiar orientation is significantly larger than to the novel orientation in the control animals (P<0.001) but not in the CPP animals. These findings are consistent with previous reports that CPP blocks SRP induction and provide a positive control demonstrating the efficacy of the CPP used in the sequence learning experiment. (c) Average voltage traces comparing the trained sequence, ABCD, with a novel sequence, ACBD, and novel timing, ABCD300, on day 5 for the CPP (n=9) and vehicle control (n=6) groups from Figure 4. (d) There is not a significant difference between CPP and control groups on day 5 (2–way RM ANOVA, P=0.752), and both groups show the effect of sequence training (1–way RM ANOVA, P<0.05).

Supplementary Figure 5 Sequence learning requires muscarinic ACh receptors.

(a) Mice were trained on the sequence ABCD 25 minutes after daily injection (i.p.) of either the muscarinic ACh receptor blocker scopolamine (3 mg/kg, n=5) or vehicle (NaCl, n=5). On the fifth day (after drug washout) the mice were tested with ABCD, DCBA and ABCD300. (b) Sequence magnitude quantification shows a significant interaction between treatment and recording day (P<0.001), a significant effect of treatment (P=0.005). Significant within-treatment post-hoc comparisons are indicated by * (P<0.001). Treatment effects on day 1 are not significant (P=0.429). (c) The effect of treatment and sequence on the 5th day of testing are shown. For quantification and statistics, see Fig. 3e. (d) To test the acute effects of scopolamine on evoked potententials, after testing on the 5th day the response to ABCD in mice from the vehicle group (n=4) were recorded immediately before and 25 minutes after scopolamine injection. Evoked responses matched those seen in d, indicating that muscarinic ACh receptor blockade does not affect previously potentiated waveform morphology.

Supplementary Figure 6 Sequence potentiation is prevented by a local blockade of muscarinic ACh receptors in V1.

(a) Schematic illustration of local infusion. (b) 1 mL of scopolamine (0.6 mg/mL) or vehicle (NaCL) was infused locally in V1 via bilateral indwelling cannulae prior to sequence exposure on days 1 and 2. No infusions were performed on days 3 through 5, n=7 mice. (c) While sequence responses in the vehicle treated hemispheres potentiated normally, there was no potentiation in the scopolamine treated hemispheres. (d) After drug washout, responses in the drug treated hemispheres potentiated normally as seen by comparing the waveforms evoked on days 3 through 5 in the scopolamine treated hemisphere with those recorded in the vehicle treated hemisphere on days 1 through 3. (e) There is a significant effect of exposure day on response magnitude (2–Way RM ANOVA, P<0.001) but no significant effect of treatment (P =0.311) or interaction between treatment and recording day (P=0.277) when comparing the first three days of sequence exposure absent scopolamine in the two hemispheres, demonstrating that scopolamine infusion does not damage the cortex. Although V1 is visually response under the acute influence of scopolamine, the day 1 response magnitude of sequence elements B, C, and D is smaller on average than under vehicle in both the systemic and local infusion experiments. This raises the possibility that potentiation is blocked not by antagonism of the ACh receptors but rather because there is not enough evoked cortical activity to drive plasticity. Panels (f–g) show day 2 magnitude driven by elements B–C–D as a function of day 1 magnitude in 12 mice treated with scopolamine (7 local, 5 systemic) and in 35 untreated mice. Untreated mice with small day 1 response (< 125 mV, n=18) show more potentiation than do the scopolamine treated mice (points above the dashed unity line indicate potentiation). There is a significant effect of treatment (2–Way RM ANOVA, P<0.001), interaction between treatment and recording day (P<0.001), and effect of recording day on response magnitude in the untreated mice (P<0.001). This demonstrates that small responses can drive sequence potentiation and suggests that muscarinic ACh receptors are necessary for this to occur.

Supplementary Figure 7 Sequence learning in adult mice.

(a) Most experiments were conducted in adolescent mice, with training commencing around post-natal day 30. To verify that sequence learning occurs in adult mice, a cohort of mice (n=5) were implanted with recording electrodes on day P50 (binocular V1, layer 4). Training with the sequence ABCD (150 ms hold time) commenced on P69, well after the developmental critical period. After viewing the trained sequence on experiment day 3, the mice were also shown the novel sequences DCBA and ABCD300. (b) As in young mice, the average visually evoked response to the sequence ABCD increased across training days and was larger than the response driven by novel sequences. (c) There is a significant effect of training and sequence (one–way RM ANOVA, P<0.001), the response to ABCD on day 3 is significantly larger than on day 1 (Holm– Šidák post hoc test, P<0.001) and is significantly larger than either DCBA (P=0.002) or ABCD300 (P=0.001). All other pairwise post–hoc comparisons are not statistically significant.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–7 (PDF 1526 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gavornik, J., Bear, M. Learned spatiotemporal sequence recognition and prediction in primary visual cortex. Nat Neurosci 17, 732–737 (2014). https://doi.org/10.1038/nn.3683

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3683

This article is cited by

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing