Learning to recognize and predict temporal sequences is fundamental to sensory perception and is impaired in several neuropsychiatric disorders, but little is known about where and how this occurs in the brain. We discovered that repeated presentations of a visual sequence over a course of days resulted in evoked response potentiation in mouse V1 that was highly specific for stimulus order and timing. Notably, after V1 was trained to recognize a sequence, cortical activity regenerated the full sequence even when individual stimulus elements were omitted. Our results advance the understanding of how the brain makes 'intelligent guesses' on the basis of limited information to form visual percepts and suggest that it is possible to study the mechanistic basis of this high-level cognitive ability by studying low-level sensory systems.
Subscribe to Journal
Get full journal access for 1 year
only $17.42 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Rent or Buy article
Get time limited or full article access on ReadCube.
All prices are NET prices.
Lashley, K.S. The problem of serial order in behavior. in Cereberal Mechanisms in Behavior (ed. L.A. Jeffress) 112–131 (Wiley, New York, 1951).
Mauk, M.D. & Buonomano, D.V. The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 307–340 (2004).
Rhodes, B.J., Bullock, D., Verwey, W.B., Averbeck, B.B. & Page, M.P.A. Learning and production of movement sequences: behavioral, neurophysiological, and modeling perspectives. Hum. Mov. Sci. 23, 699–746 (2004).
Song, S., Howard, J.H. & Howard, D.V. Perceptual sequence learning in a serial reaction time task. Exp. Brain Res. 189, 145–158 (2008).
Gheysen, F., Van Opstal, F., Roggeman, C., Van Waelvelde, H. & Fias, W. The neural basis of implicit perceptual sequence learning. Front. Hum. Neurosci. 5, 137 (2011).
Conway, C.M. & Christiansen, M.H. Sequential learning in non-human primates. Trends Cogn. Sci. 5, 539–546 (2001).
Espinosa, J.S. & Stryker, M.P. Development and plasticity of the primary visual cortex. Neuron 75, 230–249 (2012).
Frenkel, M.Y. et al. Instructive effect of visual experience in mouse visual cortex. Neuron 51, 339–349 (2006).
Cooke, S.F. & Bear, M.F. Visual experience induces long-term potentiation in the primary visual cortex. J. Neurosci. 30, 16304–16313 (2010).
Fahle, M. Perceptual learning: a case for early selection. J. Vis. 4, 879–890 (2004).
Bear, M.F. & Singer, W. Modulation of visual cortical plasticity by acetylcholine and noradrenaline. Nature 320, 172–176 (1986).
Chubykin, A.A., Roach, E.B., Bear, M.F. & Shuler, M.G. A cholinergic mechanism for reward timing within primary visual cortex. Neuron 77, 723–735 (2013).
Mitzdorf, U. Current source-density method and application in cat cerebral cortex: investigation of evoked potentials and EEG phenomena. Physiol. Rev. 65, 37–100 (1985).
Li, N. & DiCarlo, J.J. Unsupervised natural experience rapidly alters invariant object representation in visual cortex. Science 321, 1502–1507 (2008).
Meyer, T. & Olson, C.R. Statistical learning of visual transitions in monkey inferotemporal cortex. Proc. Natl. Acad. Sci. USA 108, 19401–19406 (2011).
Eagleman, S.L. & Dragoi, V. Image sequence reactivation in awake V4 networks. Proc. Natl. Acad. Sci. USA 109, 19450–19455 (2012).
Gollisch, T. & Meister, M. Eye smarter than scientists believed: neural computations in circuits of the retina. Neuron 65, 150–164 (2010).
McIntyre, C.K., Ragozzino, M.E. & Gold, P.E. Intra-amygdala infusions of scopolamine impair performance on a conditioned place preference task but not a spatial radial maze task. Behav. Brain Res. 95, 219–226 (1998).
Rao, R.P. & Ballard, D.H. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat. Neurosci. 2, 79–87 (1999).
Bastos, A.M. et al. Canonical microcircuits for predictive coding. Neuron 76, 695–711 (2012).
Spratling, M.W. Predictive coding as a model of response properties in cortical area V1. J. Neurosci. 30, 3531–3543 (2010).
Lee, T.S. & Mumford, D. Hierarchical Bayesian inference in the visual cortex. J. Opt. Soc. Am. A Opt. Image Sci. Vis. 20, 1434–1448 (2003).
Wacongne, C., Changeux, J.P. & Dehaene, S. A neuronal model of predictive coding accounting for the mismatch negativity. J. Neurosci. 32, 3665–3678 (2012).
Shuler, M.G. & Bear, M.F. Reward timing in the primary visual cortex. Science 311, 1606–1609 (2006).
Wang, K.H. et al. In vivo two-photon imaging reveals a role of arc in enhancing orientation specificity in visual cortex. Cell 126, 389–402 (2006).
Xu, S., Jiang, W., Dan, Y. & Poo, M.-m. Activity recall in a visual cortical ensemble. Nat. Neurosci. 15, 449–455 (2012).
Bao, M., Yang, L., Rios, C., He, B. & Engel, S.A. Perceptual learning increases the strength of the earliest signals in visual cortex. J. Neurosci. 30, 15080–15084 (2010).
Jehee, J.F., Ling, S., Swisher, J.D., van Bergen, R.S. & Tong, F. Perceptual learning selectively refines orientation representations in early visual cortex. J. Neurosci. 32, 16747–16753 (2012).
Marr, D. Vision. A Computational Investigation into the Human Representation and Processing of Visual Information (W.H. Freeman and Company, San Francisco, 1982).
Cooke, S.F. & Bear, M.F. How the mechanisms of long-term synaptic potentiation and depression serve experience-dependent plasticity in primary visual cortex. Phil. Trans. R. Soc. Lond. B 369, 20130284 (2014).
Liu, J.K. & Buonomano, D.V. Embedding multiple trajectories in simulated recurrent neural networks in a self-organizing manner. J. Neurosci. 29, 13172–13181 (2009).
Hasselmo, M.E. The role of acetylcholine in learning and memory. Curr. Opin. Neurobiol. 16, 710–715 (2006).
Minces, V.H., Alexander, A.S., Datlow, M., Alfonso, S.I. & Chiba, A.A. The role of visual cortex acetylcholine in learning to discriminate temporally modulated visual stimuli. Front. Behav. Neurosci. 7, 16 (2013).
Pinto, L. et al. Fast modulation of visual perception by basal forebrain cholinergic neurons. Nat. Neurosci. 16, 1857–1863 (2013).
Cajal, S.R. Comparative study of the sensory areas of the human cortex. in Clark University, 1889–1899: Decennial Celebration (ed. W.E. Story) 311–382 (Clark University Press, Worcestor, Massachusetts, 1899).
Douglas, R.J. & Martin, K.A. Mapping the matrix: the ways of neocortex. Neuron 56, 226–238 (2007).
DiCarlo, J.J., Zoccolan, D. & Rust, N.C. How does the brain solve visual object recognition? Neuron 73, 415–434 (2012).
Howard, J.H. Jr., Howard, D.V., Japikse, K.C. & Eden, G.F. Dyslexics are impaired on implicit higher-order sequence learning, but not on implicit spatial context learning. Neuropsychologia 44, 1131–1144 (2006).
Carroll, C.A., Boggs, J., O'Donnell, B.F., Shekhar, A. & Hetrick, W.P. Temporal processing dysfunction in schizophrenia. Brain Cogn. 67, 150–161 (2008).
Siegert, R.J., Weatherall, M. & Bell, E.M. Is implicit sequence learning impaired in schizophrenia? A meta-analysis. Brain Cogn. 67, 351–359 (2008).
Smith, J.G. & McDowall, J. The implicit sequence learning deficit in patients with Parkinson’s disease: A matter of impaired sequence integration? Neuropsychologia 44, 275–288 (2006).
We thank A. Heynen and S. Cooke for helpful comments on the manuscript, and S. Meagher for administrative support. This work was supported by the Howard Hughes Medical Institute, the Picower Institute Innovation Fund and a grant from the US National Institute of Mental Health (K99MH099654).
The authors declare no competing financial interests.
Integrated supplementary information
Supplementary Figure 1 Sequence learning increases response magnitudes, decreases response latencies and modulates spectral power.
(a) Diagram illustrating sequence magnitude quantification. (b) Traces show the average response evoked by the sequence ABCD on day 1 (Baseline, gray) and after training (Day 5, blue) in the experimental group from Fig. 1c. (c) There is a significant effect of training (2–way RM ANOVA, P<0.001) and each sequence element, plotted individually, increases relative to baseline (P<0.001 for A,B and C, P<0.05 for D). (d) Training also has a significant effect on response latency (defined as the interval between stimulus onset and maximal negativity, 2–way RM ANOVA, P<0.001) and there is a decrease in response latency for each element (P<0.001). (e) Plots showing the power spectral density (PSD, estimated using Welch's method, transparent shading shows 95% confidence intervals around means indicated by solid lines) averaged across all animals in each group from Fig. 1c–d (Exp. n=6, Ctrl. n=4) during periods of active stimulus viewing (ABCD, DCBA, and ABCD300) or during inter–stimulus gray screen periods (Gray). (f) To highlight difference between groups, spectral power is plotted as a percent of the average ABCD power at each frequency. In both experimental and scrambled control groups visual stimuli drive higher spectral powers than gray screen at frequencies up to approximately 100 Hz. In the experimental mice, spectral power is on average higher while viewing ABCD than DCBA at low frequencies but this relationship reverses in the high gamma range (note, data was recorded with a 60 Hz notch filter). In control mice the familiar and novel sequence spectrums overlap at both high and low frequencies. In both groups, sequences presented with familiar timing result in more spectral power than a sequence presented with novel timing. Although precise interpretation is difficult, these results are broadly consistent with the idea that familiar and novel sequence either elicit or are modulated by different attentional states.
(a) Each panel shows the multi-unit spike rasters (top) and peri-stimulus time histogram (PSTH, bottom) recorded on a single wire in a multi-channel bundle implanted in a single mouse. Responses are evoked by the sequences ABCD (trained, blue) and DCBA (novel, red). Dashed vertical lines indicate stimulus element onset times. (b) Average PSTH for the 5 channels shown in a. Before averaging, the PSTHs were normalized to the maxium spike rate evoked by either ABCD or DCBA on a per channel basis. Triangles indicate stimulus element onset times. (c) Average normalized PSTH from 6 mice (33 total recording channels).
(a) To further verify that sequence learning is not specific to a 150 ms element hold time, a cohort of mice (n=4) was trained with the sequence ABCD using a 200 ms hold time. On the 5th day, the mice were tested on sequences ABCD and DCBA with the trained time and with ABCD using novel hold times (150 and 300 ms). (b) The largest response occurs when the trained sequence is presented with the trained timing. (c) There is a significant effect of sequence (1-way RM ANOVA, P=0.003) and ABCD with the trained timing is significantly larger (P<0.05) than DCBA or ABCD presented with novel timing.
(a) Mice used in the CPP experiment (see Fig. 3) were reassigned after a 3–day washout period into CPP (n=6) and vehicle control (saline, n=9) groups and exposed to the SRP induction protocol (described previously in detail, see Frenkel et al. 2006). Briefly, on each of four training days the mice were injected with CPP or saline (see methods) and exposed to a sinusoidal grating (0.05 cy/deg, 100% contrast) oriented at θ1 and phase–reversing at 1 Hz. On the 5th day, mice were shown phase–reversing gratings oriented at the familiar and novel orientations (θ1 and θ2). (b) Analysis of phase–reversal locked VEP magnitudes shows a statistically significant interaction between treatment and recording day (days 1 and 5, P<0.001). Post–hoc analysis reveals a significant difference between CPP and control treatments (P<0.05) and a significant VEP increase between days 1 and 5 for θ1 in the control group (P<0.001), but not in the CPP group. There is also a significant interaction between treatment and stimulus orientation on day 5 (P<0.001), a significant difference between the CPP and saline groups (P<0.05), and the response to the familiar orientation is significantly larger than to the novel orientation in the control animals (P<0.001) but not in the CPP animals. These findings are consistent with previous reports that CPP blocks SRP induction and provide a positive control demonstrating the efficacy of the CPP used in the sequence learning experiment. (c) Average voltage traces comparing the trained sequence, ABCD, with a novel sequence, ACBD, and novel timing, ABCD300, on day 5 for the CPP (n=9) and vehicle control (n=6) groups from Figure 4. (d) There is not a significant difference between CPP and control groups on day 5 (2–way RM ANOVA, P=0.752), and both groups show the effect of sequence training (1–way RM ANOVA, P<0.05).
(a) Mice were trained on the sequence ABCD 25 minutes after daily injection (i.p.) of either the muscarinic ACh receptor blocker scopolamine (3 mg/kg, n=5) or vehicle (NaCl, n=5). On the fifth day (after drug washout) the mice were tested with ABCD, DCBA and ABCD300. (b) Sequence magnitude quantification shows a significant interaction between treatment and recording day (P<0.001), a significant effect of treatment (P=0.005). Significant within-treatment post-hoc comparisons are indicated by * (P<0.001). Treatment effects on day 1 are not significant (P=0.429). (c) The effect of treatment and sequence on the 5th day of testing are shown. For quantification and statistics, see Fig. 3e. (d) To test the acute effects of scopolamine on evoked potententials, after testing on the 5th day the response to ABCD in mice from the vehicle group (n=4) were recorded immediately before and 25 minutes after scopolamine injection. Evoked responses matched those seen in d, indicating that muscarinic ACh receptor blockade does not affect previously potentiated waveform morphology.
Supplementary Figure 6 Sequence potentiation is prevented by a local blockade of muscarinic ACh receptors in V1.
(a) Schematic illustration of local infusion. (b) 1 mL of scopolamine (0.6 mg/mL) or vehicle (NaCL) was infused locally in V1 via bilateral indwelling cannulae prior to sequence exposure on days 1 and 2. No infusions were performed on days 3 through 5, n=7 mice. (c) While sequence responses in the vehicle treated hemispheres potentiated normally, there was no potentiation in the scopolamine treated hemispheres. (d) After drug washout, responses in the drug treated hemispheres potentiated normally as seen by comparing the waveforms evoked on days 3 through 5 in the scopolamine treated hemisphere with those recorded in the vehicle treated hemisphere on days 1 through 3. (e) There is a significant effect of exposure day on response magnitude (2–Way RM ANOVA, P<0.001) but no significant effect of treatment (P =0.311) or interaction between treatment and recording day (P=0.277) when comparing the first three days of sequence exposure absent scopolamine in the two hemispheres, demonstrating that scopolamine infusion does not damage the cortex. Although V1 is visually response under the acute influence of scopolamine, the day 1 response magnitude of sequence elements B, C, and D is smaller on average than under vehicle in both the systemic and local infusion experiments. This raises the possibility that potentiation is blocked not by antagonism of the ACh receptors but rather because there is not enough evoked cortical activity to drive plasticity. Panels (f–g) show day 2 magnitude driven by elements B–C–D as a function of day 1 magnitude in 12 mice treated with scopolamine (7 local, 5 systemic) and in 35 untreated mice. Untreated mice with small day 1 response (< 125 mV, n=18) show more potentiation than do the scopolamine treated mice (points above the dashed unity line indicate potentiation). There is a significant effect of treatment (2–Way RM ANOVA, P<0.001), interaction between treatment and recording day (P<0.001), and effect of recording day on response magnitude in the untreated mice (P<0.001). This demonstrates that small responses can drive sequence potentiation and suggests that muscarinic ACh receptors are necessary for this to occur.
(a) Most experiments were conducted in adolescent mice, with training commencing around post-natal day 30. To verify that sequence learning occurs in adult mice, a cohort of mice (n=5) were implanted with recording electrodes on day P50 (binocular V1, layer 4). Training with the sequence ABCD (150 ms hold time) commenced on P69, well after the developmental critical period. After viewing the trained sequence on experiment day 3, the mice were also shown the novel sequences DCBA and ABCD300. (b) As in young mice, the average visually evoked response to the sequence ABCD increased across training days and was larger than the response driven by novel sequences. (c) There is a significant effect of training and sequence (one–way RM ANOVA, P<0.001), the response to ABCD on day 3 is significantly larger than on day 1 (Holm– Šidák post hoc test, P<0.001) and is significantly larger than either DCBA (P=0.002) or ABCD300 (P=0.001). All other pairwise post–hoc comparisons are not statistically significant.
About this article
Cite this article
Gavornik, J., Bear, M. Learned spatiotemporal sequence recognition and prediction in primary visual cortex. Nat Neurosci 17, 732–737 (2014). https://doi.org/10.1038/nn.3683
Science China Life Sciences (2021)
Mesoscopic Mapping of Stimulus-Selective Response Plasticity in the Visual Pathways Modulated by the Cholinergic System
Frontiers in Neural Circuits (2020)
PLOS ONE (2020)
Frontiers in Cellular Neuroscience (2020)
Annals of the New York Academy of Sciences (2020)