Introduction

The outside world provides us only the light, but our visual system is capable of extracting the basic features in low-level areas and understanding them as meaningful concepts in high-level areas. Interactions between these different areas involve both feedforward and feedback processes. However, the underlying mechanisms in these dynamical processes, especially in the temporal domain, are not fully understood.

Predictive coding is a model emphasizing interactions between feedforward and feedback signals. Specifically, it suggests that the brain employs an efficient coding strategy for these interactions by generating predictions in higher-level areas and comparing them with the incoming sensory signals in the lower-level areas1. Previous neuroimaging evidence revealed the existence of such two-way communication2; however, different computational models of predictive coding have suggested very different details of implementation1,3,4,5.

It has been proposed that the feed-forward and feedback in predictive coding could take advantage of oscillations for information processing6,7. On the one hand, recent neurophysiological evidence on laminar-specific oscillations and the functional roles of different layers suggested a faster oscillation for the feed-forward pathway and a slower oscillation for the feedback pathway6,8,9,10,11. On the other hand, recent studies showed a link between behavioral performance and cortical oscillations in perception12,13 and reaction time14,15. Since neural oscillations can reflect the cyclic fluctuations of excitability in a network, investigation of the relationship between trial-to-trial variability and the phase of ongoing oscillations could link specific oscillations to cognitive functions (e.g. attention).

Here, we used this approach to investigate the specific influence of ongoing oscillations on the feed-forward/feedback interactions in predictive coding, by measuring their effect on perception for different pre-stimulus oscillatory phases. A minimal definition of “predictive coding” encompasses any stimulus-driven higher-level activity that modulates, in a feedback manner, the lower-level sensory processing of the same stimulus. In a typical predictive coding experiment, therefore, two conditions must be compared, which differ in the amount or the type of predictive feedback signals. Here, we chose one of the first experimental manipulations used in predictive coding to generate different amounts of predictive feedback16: shape perception.

Specifically, 3D-shape outlines and random-lines versions of the same stimuli, similar to the stimuli used in a previous influential study16, were used in this experiment. It has been shown that 3D-shape outlines can be easily recognized (presumably activating higher-level areas) and thus produce more (or at least, different) predictive feedback than the random-lines versions16. To measure the effect of different amounts of predictive feedback, we asked the subjects to judge the luminance (report the side of the brighter disk) of two gray disks simultaneously displayed on the left and right of fixation on a black background, one containing the 3D-shape outlines and the other containing the random-lines version. Our previous study showed that the 3D-shape (which generates more predictive feedback16) increases the subjective luminance of the disk relative to the random-lines version17. Thus, perceived luminance can be used as a marker for the efficiency of predictive feedback on every trial. Here, the luminance of the disks was adjusted to achieve about 50% choice rate for 3D-shape vs. random-lines disks for all trials (this was achieved by slightly increasing the luminance of the random-lines disk). With our knowledge of the predictive feedback’s effect on luminance judgements17, we can predict that in trials where predictive feedback efficiency is higher than average, subjects would be more likely to report the 3D-shape disk as brighter; and vice versa, when predictive feedback efficiency is lower than average, subjects would more likely report the random-lines disk as brighter. We recorded EEG signals and analyzed the relationship between pre-stimulus oscillation phase and the post-stimulus judgment. If predictive feedback mechanisms during the post-stimulus stage involved one or more periodic processes, then the oscillatory state of the system just before stimulus onset (as reflected in pre-stimulus oscillatory phase) should have an influence on the outcome of predictive feedback, and thus on the perceptual judgment. We found that, independent from the spatial choice (left/right side), the phase of 5 Hz contralateral frontal and 16 Hz contralateral occipital pre-stimulus oscillations modulated the subject’s choice of a brighter 3D-shape disk (more effective predictive feedback) or a brighter random-lines disk (less effective predictive feedback). Since in the predictive coding framework, higher hierarchical level areas are assumed to send predictive feedback and lower hierarchical level areas to send prediction error signals, our results could imply that in the post-stimulus stage, the brain sends predictive feedback periodically at a preferred phase of a theta frequency oscillation in the frontal region, and sends prediction errors periodically at a preferred phase of a beta frequency oscillation in the occipital region.

Results

Human observers judged the luminance of two disks that were presented for 150 ms on the left and right of a central fixation point. The disks contained different versions of the same stimulus, one with a 3D-shape enabling predictive feedback, and the other with a random-lines version of the same shape which impeded predictive feedback (Fig. 1). Before the stimulus onset there was a random period of time (1000–1500 ms) with only the fixation point on the screen. After the stimulus offset a question mark appeared at the center and the subjects were instructed to report the side with the brighter disk. In the main experimental trials, the luminance of the disks was adjusted, such that observers reported the 3D-shape disk as brighter in half of the trials. 15% of trials were catch trials: extreme luminance values were assigned to one disk to monitor the subject’s ability to judge the luminance difference.

Figure 1: Illustration of the experimental paradigm.
figure 1

In each run of trials, a blank screen with only a central fixation point was presented for 1000 to 1500 ms randomly. Then, two circular gray disks, one with a 3D-shape stimulus in the center and the other with random-lines, were presented randomly on either side (left or right) of the fixation point for 150 ms. Subsequently, a question mark appeared in the center of the screen. Subjects were instructed to fixate the fixation point all the time, and report the side of the brighter disk with the corresponding arrow key after the question mark appeared. In the main experimental trials, luminance values of the disks were adjusted to obtain a 50% selection probability of 3D-shape/random-lines disk. In addition, there were 15% catch trials intermixed with the main experimental trials to monitor the subjects’ ability to judge the luminance difference throughout the experiment. In these catch trials, one disk was 20% brighter/darker than in the main experimental trials.

Full size image

Behavioral Results

On average, subjects judged the 3D-shape disk as brighter in half of the trials (49.24% ± 1.57%, mean ± standard error of the mean, SEM) in the main experimental condition, as expected. The luminance judgment correct rate in the catch trials (subjects judged the disk with higher luminance value as brighter or judged the disk with lower luminance value as darker) was high (93.98% ± 1.83%, mean ± SEM), indicating that subjects were adequately engaged in the luminance judgment task.

Electrophysiological Results

We focused on the relationship between the oscillatory phase and the trial-by-trial variations in the efficiency of post-stimulus predictive coding. EEG was recorded during the experiment. Regarding the expected time window of such a relationship, the post-stimulus phase information is driven to a large extent by stimulus-locked activity (e.g. evoked potentials) and is thus further removed from spontaneous activity. On the other hand, since a spontaneous oscillation is a continuous process, pre-stimulus phase information is likely to be related to the oscillatory phase at the (post-stimulus) time point at which predictive coding actually happens. Therefore, the pre-stimulus phase can be a better measurement of the relevant state of a spontaneous oscillation than the post-stimulus phase. Thus, we expected the relation between oscillatory phase and behavior to be most visible in the pre-stimulus time window, where phase information reflects spontaneous fluctuations in neuronal excitability. We used classical stimuli16 for inducing different amounts of predictive feedback on the left and right of the screen: 3D-shape and random-lines versions of the same stimuli (Fig. 1; the 3D-shape version enabling predictive feedback, the random-lines simultaneously impeding it, as described in Murray et al.16). To measure the effective amount of post-stimulus predictive feedback on each trial, we probed the perceived luminance of the disks under the stimuli. We have previously demonstrated that the 3D-shape (producing more predictive feedback, as described by ref. 16) results in a relatively higher perceived luminance than the random-lines17. Here, this net effect was compensated on each trial by slightly lowering the luminance of that disk so that the average likelihood of perceiving either disk brighter was about 50% (see Methods). Therefore, residual fluctuations of luminance perception on every trial can be thought to arise from trial-by-trial fluctuations in the efficiency of post-stimulus predictive coding: the 3D-shape disk may still be perceived brighter on trials where predictive coding was more efficient than average, and darker on trials where it was less efficient than average. Of course, spatial bias and/or trial-by-trial fluctuations in the direction of spatial attention may well also contribute to the choice of which disk appears brighter on a given trial. Thus, for each subject we divided all trials into two datasets based on their spatial choice (left-side choice vs. right-side choice), and we only investigated the relation between pre-stimulus EEG phase and 3D-shape/random-lines choice within each dataset. As the correlates of choosing the prediction-consistent stimulus (3D-shape) were expected to be strongest on electrodes contralateral to that stimulus, which would map onto opposite hemispheres for the two datasets, before plotting any scalp topographies we permuted the electrode positions (symmetrically across the midline axis) of all right-side choice trials. This procedure resulted in a mapping of ipsilateral effects to the spatial choice onto left electrodes, and contralateral effects onto right electrodes.

We estimated the relation between EEG phase and predictive coding via the phase opposition product (POP, see Methods). This measure should be maximal when 3D-shape choice trials (prediction-consistent) and random-lines choice trials (prediction-inconsistent) tend to have opposite phase values. For each subject, dataset, electrode, time point and oscillatory frequency, we obtained surrogate POP values (80,000,000 surrogates) by randomly permuting the trial outcomes, keeping the number of trials constant. Both real and surrogate POP values were averaged across datasets and subjects. The significance was determined as the proportion of surrogate POP values that were more extreme than the observed value. P-values were corrected for multiple comparisons across time points, frequencies and electrodes (100 × 30 × 64) using the FDR method (FDR α = 0.05, corresponding to a P value threshold of 9.53 × 10−6). To show the overall POP in the time-frequency domain, a z-score was computed by comparing the real POP values (combined across all subjects, datasets, and electrodes) to the mean and standard deviation of a null-hypothesis distribution with 10,000 surrogate POP values (generated using the same procedure described before, and also combined across all subjects, datasets, and electrodes). This analysis revealed a significant phase-behavior relation between the post-stimulus 3D-shape/random-lines choice and two pre-stimulus oscillations (Fig. 2A): one theta-frequency oscillation (~3.1 Hz to 7.6 Hz) in the time window from −545 ms to −268 ms, and one beta-frequency oscillation (~13.2 Hz to 25.7 Hz) in the time window from −107 ms to −25 ms. Green outlines mark the significant time-frequency regions (at least one significant electrode) after FDR correction.

Figure 2: Pre-stimulus EEG phase predicts luminance judgment of 3D-shape disk vs. random-lines disk.
figure 2

(A) The relation between pre-stimulus phase and luminance judgment (our measure of predictive coding) is assessed using POP values (phase opposition product; see details in Methods). The time-frequency map is the z-score of observed POP values (combined across all subjects, datasets and electrodes), each value compared with a null-hypothesis distribution of 10,000 surrogate POP values (also combined across all subjects, datasets and electrodes) characterized by its mean and SD. Time 0 indicates stimulus onset. P-values were derived from a comparison of POP values against 80,000,000 surrogates, and corrected for multiple comparisons across all time points, frequencies and electrodes using the FDR method (FDR α = 0.05, corresponding to a P value threshold of 9.53 × 10−6). The green outlines mark the significant time-frequency regions (at least one significant electrode) after FDR correction. A significant relation is apparent between the effect of shape perception on luminance judgments and the EEG phase of ~5 Hz and ~16 Hz pre-stimulus oscillations. (B) Scalp topography of (z-scored) POP values around 16 Hz (frequency range 13.2 to 25.7 Hz; time range −107 to −25 ms). Electrodes highlighted in green have at least one significant time-frequency point (after FDR correction) inside the corresponding red box. Due to our electrode-permutation procedure, in this topography the electrodes ipsilateral to the spatial choice are displayed on the left and those contralateral to the spatial choice are displayed on the right. Thus, the topography shows a contralateral occipital effect for the 16 Hz oscillation. (C) Same as B, but for the ~ 5 Hz oscillations (frequency range 3.1 to 7.6 Hz; time range −545 to −268 ms). The topography shows a contralateral frontal effect for the 5 Hz oscillation.

Full size image

Scalp topographies of the z-score show that the two oscillatory effects involve distinct electrode groups and presumably distinct brain regions (Fig. 2B and C): the theta-frequency effect is maximal over frontal regions and the beta-frequency effect over occipital regions. In both cases, these effects are contralateral to the side that subjects chose as “brighter” (i.e., the right side of the topographies, due to our electrode permutation procedure). Electrodes with at least one significant time-frequency point (after FDR correction) inside the corresponding time-frequency window are highlighted in green.

To quantify the influence of pre-stimulus oscillations on post-stimulus choice, we binned single trials according to the phase at the optimal time-frequency point (for the theta oscillation: −397 ms, 5.4 Hz; for the beta oscillation: −68 ms, 16.5 Hz). Single trials were thus sorted in 13 phase bins based on the average phase of the significant electrodes for each oscillation (four frontal electrodes for the theta oscillation, three occipital electrodes for the beta oscillation). For each phase bin we then computed the post-stimulus choice probability of the 3D-shape disk. These choice probabilities were normalized by dividing them by the overall 3D-shape choice probability across all phase bins. For each experimental dataset (left- vs. right-side choice), phase bins were rotated such that the phase at which 3D-shape disk choice probability was largest was aligned to a phase angle of zero. As a result of this alignment, the 3D-shape choice probability is necessarily maximal at a phase angle of zero; therefore, the zero-phase bin was discarded from further analyses. For both frequencies, the 3D-shape disk choice probability monotonically decreased to a minimum at the opposite phase angle, confirming that pre-stimulus phase affected post-stimulus judgment (Fig. 3). A one-way ANOVA showed that both pre-stimulus theta phase and pre-stimulus beta phase significantly modulated the 3D-shape disk choice probability (for theta oscillation, F(11, 27) = 3.95, p = 2.23 × 10−5; for beta oscillation F(11, 27) = 6.17, p = 3.86 × 10−9). The magnitude of each effect was determined as the difference between the maximum and minimum 3D-shape disk choice probabilities across all phase bins. The frontal theta oscillation accounted for a difference of ~14% of the 3D-shape disk choice probability between phase bins, and the occipital beta oscillation accounted for a difference of ~19%.

Figure 3: Normalized choice probability of 3D-shape disks as a function of pre-stimulus phase.
figure 3

(A) Relationship between frontal pre-stimulus theta phase and choice of 3D-shape disk as the brighter disk. Single trials were binned into 13 bins, centered on the maximal phase bin for each subject (the central value is not displayed because it is not interpretable due to the realignment). The curve indicates that the oscillatory phase of frontal electrodes (4 significant electrodes shown in inset topography) at 5.4 Hz and −397 ms (relative to stimulus onset) modulates the luminance judgment by ~14%. Error bars represent SEM across subjects. (B) Same as A, but for the occipital beta pre-stimulus phase. The oscillatory phase of occipital electrodes (3 significant electrodes shown in inset topography) at 16.5 Hz and −68 ms (relative to stimulus onset) modulates the luminance judgment by ~19%.

Full size image

Because the time-frequency analysis relies on signal convolution with wavelet filters whose duration is non-negligible, one might wonder whether the observed pre-stimulus phase differences could actually be driven by stimulus-evoked activity. For example, at 16 Hz the above time-frequency analysis used a 250 ms time window (4 cycles, 125 ms from the past and 125 ms into the future); thus, significant phase effects observed at −67 ms pre-stimulus may be contaminated by post-stimulus activity. To rule out such contamination, we repeated the POP time-frequency analysis with one-cycle wavelets at all frequencies, and compared the timing of pre-stimulus phase effects with the time-frequency region of possible post-stimulus contamination, determined using the wavelet window length at each frequency (Fig. 4). Both theta- and beta-frequency phase effects were replicated in this analysis, and were found to lie outside of the possible post-stimulus contamination zone.

Figure 4: Significance of POP values in a one-cycle wavelet analysis.
figure 4

(A) P-value map of the POP values combined across previously identified frontal electrodes (green points in inset topography) for the theta-frequency phase effect. The P-values were calculated by comparing the observed POP values with 80,000,000 surrogates. The semi-transparent red area on the time-frequency map indicates the zone of possible contamination by post-stimulus activity (based on the wavelet window length at each frequency, centered on the time of stimulus onset, 0 ms). The previously observed theta-frequency phase effect lies outside of the contamination zone. The P-values here are uncorrected for multiple comparison. (B) Same as A, but for the beta-frequency phase effect. The beta-frequency phase effect also lies outside the contamination zone. Altogether, these findings indicate that pre-stimulus phase differences are not caused by post-stimulus evoked activity.

Full size image

We also ascertained that phase effects were not caused by any eye movement artifacts that may have survived our artifact rejection procedure. For example, the observed pre-stimulus phase differences could be thought to reflect different patterns of eye blink or saccades for different perceptual outcomes. Therefore, we applied our POP time-frequency analysis to the horizontal and vertical EOG (HEOG and VEOG) signals. P-value maps (obtained by comparison of POP values against 80,000,000 surrogates) did not reveal any signs of systematic eye movements in either the theta- or the beta-frequency bands (Fig. 5), ruling out an explanation of our pre-stimulus phase effects in terms of ocular artifacts.

Figure 5: Significance of the POP values of VEOG and HEOG signals.
figure 5

(A) P-value map of the POP values for the VEOG. The P-values were calculated using a similar procedure as in the main analysis: comparing the observed POP values with 80,000,000 surrogate POP values. (B) Same as A, but for the HEOG. There was no significant pre-stimulus time-frequency window with significant POP values in either VEOG or HEOG, indicating that the observed phase effects were not due to ocular artifacts.

Full size image

Discussion

We investigated the temporal dynamics of predictive coding by exploring the relation between pre-stimulus oscillatory phase and the presumed trial-by-trial variations in post-stimulus predictive feedback. We used 3D-shape outlines and random-lines versions of the same stimuli (as in one of the seminal predictive coding studies16) to induce different amounts of predictive feedback (Fig. 1), and measured the corresponding effects on luminance judgment as trial-by-trial markers of the efficiency of predictive coding. (To reiterate our experimental logic: Feedforward processing of the 3D shape likely induced predictive feedback, which increased the perceived brightness of the 3D-shape disk. The more efficient this predictive coding, the larger the effect of predictive feedback, and the brighter the 3D-shape disk should be perceived.). Using a similar analysis method as in well-established studies of the relationship between pre-stimulus phase and post-stimulus behavior18,19, we found that two pre-stimulus ongoing oscillations from different regions and frequencies could strongly influence the post-stimulus luminance judgment, and thus the post-stimulus predictive coding efficiency: a contralateral frontal theta oscillation and a contralateral occipital beta oscillation (Fig. 2). The phase of the theta oscillation before stimulus onset could explain 14% of the luminance judgment difference while the phase of the beta oscillation could explain 19% (Fig. 3). Control analyses ruled out contamination of the phase-behavior relationship by post-stimulus activity (Fig. 4) or ocular artifacts (Fig. 5). These results not only imply that the feedforward/feedback interactions in predictive coding involve periodic processes, but also reveal two periodicities with different sources. Since the occipital and frontal signals likely reflect activity from hierarchically lower and higher areas, respectively, and since predictive coding theory suggests that the brain sends back predictions from higher areas and sends prediction errors from lower areas, our results allow us to speculate on a possible temporal dynamic for predictive coding: predictions sent periodically at a theta frequency, prediction errors sent periodically at a beta frequency. Of course, more direct connectivity studies would be needed in the future to confirm these postulated roles for occipital and frontal signals.

The experimental paradigm used in this study takes advantage of the relationship between shape perception and predictive coding: 3D-shape outlines are assumed to generate more predictive feedback than the random-lines version of the same stimulus. Murray et al.16 used similar stimuli to provide one of the first evidence of predictive coding: compared to the random-lines, 3D-shape outlines increased activity in the lateral occipital complex (LOC), but decreased it in primary visual cortex (V1), suggesting an increase of predictive feedback accompanied by a decrease in prediction errors16,20. Here, we used the same paired stimuli as in the original study, placed them on two gray disks and asked subjects to judge the luminance of the disks. This luminance judgment, associated with perceived contrast, is likely to have a positive and monotonic relationship with neural activity in early visual cortex21,22,23,24. Thus, the variability in luminance judgment associated with the 3D-shape vs. random-lines stimuli could reflect trial-by-trial changes in the effect of predictive feedback on neural activity in early visual cortex.

Previous fMRI studies showed that shape perception could not only reduce16, but also up-regulate neural activity in V125. Our own previous study found that 3D-shape disks were generally perceived brighter than random-lines disks, and that this effect could be attributed to predictive coding rather than attentional biases17. In the present study, we compensated for this net effect by adjusting the disks’ luminance to obtain a 50% selection probability of 3D-shape/random-lines disks. (This operation suggested that predictive feedback has an excitatory effect on lower level activity, which is in contrast to many predictive coding models. This apparent discrepancy was already extensively discussed in our previous paper17. For the present purposes, it is sufficient to note that, if predictive coding is a rhythmic process, both an excitatory and a suppressive main effect of predictive coding would produce the same outcome: a periodic modulation of contrast judgments.) We then focused on the remaining variability in luminance judgement as a trial-by-trial marker of the efficiency of predictive coding. On the other hand, systematic spatial biases (e.g. a general tendency to respond to the left or right stimulus) and/or trial-by-trial fluctuations in the direction of spatial attention can also be expected to affect the luminance judgement26. As a matter of fact, spatial attention itself appears to involve a periodic process12,18,27,28 which could potentially influence the luminance judgement. We carefully avoided these potential confounding factors by dividing the trials into two datasets based on the post-stimulus spatial response (left/right) and performing the analysis within each dataset. If spatial attention biases, for example, were the only cause of the perceived luminance changes, the left-response dataset would pool all trials with a left-side attention bias (and similarly for the right-response dataset), and within each dataset pre-stimulus oscillatory phases would not bear any relation to post-stimulus luminance judgments. Of course, this analysis controls for spatial biases and changes in spatial attention, but cannot rule out object-based attention related to the shape of the object; however, it could be argued that this type of object-based attention is indistinguishable from (or even, identical to) the predictive feedback arising from the predictive coding framework. To sum up, the existence of significant phase-behavior relationships in our analysis can therefore be safely attributed to predictive coding mechanisms, rather than spatial attention or other spatial biases.

Neurophysiological recordings have shown that feedforward and feedback may take advantage of oscillations in different frequencies. Laminar recordings showed that high-frequency oscillations are prominently generated in superficial layers and low-frequency oscillations in deep layers9,10,29. Since superficial and deep layers correspond respectively to the main sources of forward and feedback projections30,31,32, it follows that feedforward communication takes advantage of high-frequency oscillations and feedback takes advantage of low-frequency oscillations. A recent study with simultaneous recordings and micro-stimulation in different layers in V1 and V4 confirmed this notion8. Several authors have independently proposed that low-frequency oscillations send predictions via feedback, while high-frequency oscillations send prediction errors via feedforward6,11,33,34,35,36. Even though the exact frequency bands (theta, beta) are not entirely consistent with those described in the previous studies (e.g. beta, gamma in ref. 11), our results still provide support for distinct functional roles of low and high frequency oscillations in predictive coding at the EEG and behavioral level.

Our results also provide supportive evidence for the hypothesized functions of frontal theta-band and occipital beta-band oscillations. Our conclusions are in line with the notion that 5–10 Hz oscillations could contribute to “top-down” control37,38, which has already been suggested based on attentional phase effects on perception12,18, reaction time14,15,39 and perceptual variability in TMS-induced effects13,40. We found the origin of such theta periodicity in contralateral frontal electrodes, compatible with the involvement of frontal areas in the top-down controlling process2,41,42 and with the involvement of 5–10 Hz oscillations in this region43. On the other hand, local field potential (LFP) recordings showed that, in mammalian visual cortex, beta frequency oscillations are also prominent during the deployment of top-down control44,45,46,47,48. Our findings of beta frequency phase effects on predictive feedback in the occipital area are concordant with such LFP results and suggest a valuable role for the beta frequency oscillations in predictive coding.

In summary, we measured the relation between pre-stimulus oscillations and a predictive feedback-induced effect to investigate the neural oscillations involved in predictive coding. We found that the pre-stimulus phases of frontal theta-frequency oscillations and occipital beta-frequency oscillations jointly determine post-stimulus subjective judgments. These results shed light on the temporal dynamics of predictive coding, and suggest a periodic predictive coding process with faster oscillations in lower areas and slower oscillations in higher areas.

Materials and Methods

Subjects

Fifteen volunteers participated in the experiment. One participant was excluded from the analysis due to the poor behavioral performance in catch trials (<60% trials were correctly reported, with a chance level of 50%, see below). Fourteen participants remained in the sample (8 females, mean age 28.01 ± 4.81 years, four left-handed, four with left eye dominance). All subjects had normal or corrected to normal vision. The study was approved by the local ethics committee “Sud-Ouest et Outre-Mer I” and followed the Code of Ethics of the World Medical Association (Declaration of Helsinki). All subjects provided signed informed consent before starting the experiments.

Apparatus

Stimuli were presented at 57 cm distance using a desktop computer (2.09 GHz Intel processor, Windows XP) with a cathode ray monitor (resolution: 800 × 600 pixels; refresh rate: 140 Hz, Gamma corrected luminance function). Stimuli were designed and presented via the Psychophysics Toolbox (Brainard, 1997) running in MATLAB (MathWorks).

Stimuli and tasks

Stimuli consisted of a central white fixation point (diameter: 0.2 degrees of visual angle) and two circular gray disks (diameter: 4 degrees each) presented randomly to the left and right of fixation (3 degrees eccentricity). One 3D-shape stimulus was in the center of one disk (3D-shape disk) and one random-lines version of the same stimulus in the other (random-lines disk). The 3D-shape and random-lines stimulus pair was randomly chosen from twenty pairs of stimuli generated beforehand using a method similar to Murray et al.16: 3D-shapes were generated by randomly selecting 4–6 vertices, connecting the vertices and adding small extensions to render perceived depth; random-lines stimuli were created by breaking the 3D-shape at its intersections and randomly shifting the lines within the display16. The diameter of both 3D-shape and random-lines stimuli was 3 degrees. The stimulus outlines were black.

Before stimulus onset, there was a blank screen with only the fixation point that lasted from 1000 to 1500 ms (random uniform distribution). Then the two disks and the fixation point appeared for 150 ms. After that, a question mark appeared in the center of the screen. There were two kinds of randomly mixed experimental trials: the main experimental trials and the catch trials. In main experimental trials, the luminance of the disks was adjusted (i.e. the random-line disks were set 1.45% brighter than the 3D-shape disks) based on a previous study17 to obtain an average 50% selection rate of 3D-shape/random-lines disks. In catch trials, one of the disks had its luminance value changed up or down by 20% compared to the luminance used in the main experimental trials, while the other disk kept the same luminance as in the main experimental trials. Subjects were presented with 4 or 8 blocks of 200 trials with 85% main experimental trials and 15% catch trials (the first 6 of the 14 subjects performed only 4 blocks of the present experiment, together with 4 blocks of another experiment that was eventually canceled and whose data were not analyzed). Subjects were instructed to fixate the fixation point all the time, judge the luminance of the disks and respond using the arrow keys (left arrow to indicate that left disk is brighter, right arrow for right disk brighter) on a standard 105 key keyboard when the question mark appeared. There was no feedback after the response.

EEG data acquisition and analysis

EEG was recorded at 1024 Hz using a Biosemi system (64 active electrodes). Horizontal and vertical electro-oculograms (EOG) were recorded by three additional electrodes around the subjects’ eyes. For data pre-processing, the EEG and EOG data were downsampled offline to 256 Hz, re-referenced to average reference and epoched around the stimulus onset in each trial for data analysis via the MATLAB (MathWorks) and EEGLAB toolbox49. Individual electrode data were visually inspected, and channel data containing artifacts were interpolated by the mean of adjacent electrodes (three subjects had one electrode containing artifacts, one subject had two; the positions of the interpolated electrodes were different across subjects).

As the post-stimulus spatial choice was lateralized on each trial to the left or right side, the pre-stimulus oscillatory correlates of the post-stimulus luminance judgment may not only reflect the oscillation’s influence on shape perception and predictive coding, but also its influence on spatial choice (i.e., pre-stimulus oscillations may bias the left/right spatial choice independently of the 3D-shape/random-lines content inside of the disk). To avoid any contribution from the spatial choice, we first divided the trials for each subject into two trial datasets based on the post-stimulus spatial choice, and performed the time-frequency analysis (described below) within each dataset. We reasoned that this analysis would lead to shape perception correlates not on a given fixed set of electrodes, but rather on different electrode groups depending on the side of choice (i.e., electrodes “contralateral” or “ipsilateral” to the spatial choice). Therefore, we arbitrarily chose to permute the electrode locations for the dataset corresponding to a right-side choice: we replaced the left-hemisphere electrodes by the symmetric ones from the right and vice versa (midline electrodes were unaffected). With this new electrode assignment, left-hemisphere electrodes would thus always correspond to those ipsilateral to the spatial choice, and right-hemisphere electrodes to contralateral ones.

For the time-frequency analysis, time-frequency transformations were first generated over all channels using EEGLAB with a function akin to a wavelet transform, starting with 3 cycles at 2 Hz and increasing to 5 cycles at 50 Hz in the multiple-cycle analysis, and with 1 cycle from 2 Hz to 50 Hz in the one-cycle analysis. This yields a complex representation of the amplitude, A, and the phase, φ, for trial j at time t and frequency f:

The phase of this representation can be extracted by normalizing the complex vector to the unit length:

Inter-trial phase coherence (ITPC) measures the phase consistency across trials. We calculated the ITPC using the method described previously50:

where N is the number of trials in one group of trials. Note that the ITPC is a scalar rather than a vector and it is always non-negative.

Here, we wanted to evaluate the relation between the pre-stimulus oscillatory phase and the influence of shape perception on luminance judgment (our measure of the efficiency of predictive coding). Would a particular pre-stimulus phase occur more frequently for trials with post-stimulus 3D-shape disk choice, and the opposite phase for trials with post-stimulus random-lines disk choice ? In the pre-stimulus period, because intertrial intervals are randomized and unpredictable, the phase of the spontaneous EEG signal at a given pre-stimulus time should follow a uniform distribution over all trials. However, if (and only if) there is a systematic relation between EEG phase and behavioral outcome, higher-than-chance inter-trial phase coherence should be observed in each of the trial subgroups. Indeed, if EEG phases of two subgroups corresponding to different behavioral outcomes tend to oppose each other, the phases of one subgroup will concentrate around one phase angle and the phases of the other subgroup will concentrate around the other phase angle; thus, high ITPC values will be observed for both subgroups. Vice versa, if EEG phases of two subgroups do not oppose each other, the phases of the two subgroups will distribute uniformly across different phase angles; thus, low ITPC values will be observed. Therefore, the ITPC values of the two subgroups are reliable indicators of phase opposition. The product or the sum of the two subgroup inter-trial phase coherence values can therefore summarize, in a single variable, the strength of the phase-behavior relation18,19,51. Here we chose to use the product of the inter-trial phase coherence values, a measure we call Phase Opposition Product (POP):

Again, high ITPC values for the two subgroups (suggesting different phase angles for the different trial subgroups) will result in a high POP value; and two low ITPC values (suggesting random/uniformly distributed phases for the different trial subgroups) will result in a low POP value. Thus, the measure of POP will be maximal when two subgroups show strong inter-trial phase coherence, in other words, strong phase opposition between two subgroups.

To accurately assess the significance of the phase-behavior relation without any assumption about the probability distribution of the POP values, we performed a nonparametric permutation test: We first computed the POP values for each point in the time-frequency plane from −650 to 150 ms, from 2 to 50 Hz for each electrode, dataset, and subject and then averaged across all datasets and all subjects. Surrogate POP values were obtained by randomly assigning the trials to one or the other condition for each subject (keeping the number of trials in each condition constant) and recalculating the grand-average POP values. We computed the P value by simply and directly counting the number of surrogate POP values that were more extreme than the observed value. Here, we used 80,000,000 surrogates and thus assigned the P value of 1.25 × 10−8 to the points without any more extreme POP values in the surrogates. The P values were corrected for multiple comparisons over time points, frequencies and electrodes using the FDR method (FDR α = 0.05, corresponding to a P value threshold of 9.53 × 10−6). To show the overall POP in the time-frequency domain, we computed a z-score by combining the observed POPs across all datasets, subjects, and electrodes and comparing the value with the mean and SD of a null-hypothesis distribution with 10,000 surrogate POP values (generated using the procedure described before, and also combined across all electrodes, subjects, and datasets).

Additional Information

How to cite this article: Han, B. and VanRullen, R. The rhythms of predictive coding? Pre-stimulus phase modulates the influence of shape perception on luminance judgments. Sci. Rep. 7, 43573; doi: 10.1038/srep43573 (2017).

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.