Large-scale dynamics of perceptual decision information across human cortex

Perceptual decisions entail the accumulation of sensory evidence for a particular choice towards an action plan. An influential framework holds that sensory cortical areas encode the instantaneous sensory evidence and downstream, action-related regions accumulate this evidence. The large-scale distribution of this computation across the cerebral cortex has remained largely elusive. Here, we develop a regionally-specific magnetoencephalography decoding approach to exhaustively map the dynamics of stimulus- and choice-specific signals across the human cortical surface during a visual decision. Comparison with the evidence accumulation dynamics inferred from behavior disentangles stimulus-dependent and endogenous components of choice-predictive activity across the visual cortical hierarchy. We find such an endogenous component in early visual cortex (including V1), which is expressed in a low (<20 Hz) frequency band and tracks, with delay, the build-up of choice-predictive activity in (pre-) motor regions. Our results are consistent with choice- and frequency-specific cortical feedback signaling during decision formation.


Supplementary Figure 2, Baseline and task-related neural activity in different frequency bands. a.
Pre-stimulus baseline power spectra for all cortical regions shown in main Figure 2. Data are represented as group mean (n=15 subjects, horizontal bars, gray dot), ±SEM (gray error bars). b. Time course of change of band-limited power relative to baseline during the test stimulus presentation, for same cortical regions as in Figure 2. Frequency bands have been defined so as to span the full range of effects evident in our spectral analysis. Left inset (on top of V1), close-up of V1 power responses in the gamma-band and high-frequency (HF) band. Right inset, slope estimates (linear regression across interval 0.5 to 1s) for V1 power responses in different bands. HF, high-frequency band. Asterisks: **, p < 0.01; ***, p < 0,001 (two-sided t-tests). b: t(14)=3.9, p=0.002; g: t(14)=-3.2, p=0.006; HF: t(14)= x-8.6, p=6e -7 ; c. As b, but now for the power response difference between stronger vs. weaker mean test contrast trials. Data in b, c are represented as group mean (n=15 subjects, horizontal bars, gray dot), ±SEM (gray error bars) or individual subjects (dots in inset). Source data are provided as a Source Data file. Figure 3. V1 gamma-band responses as function of contrast (sample) variance. a. Group average time course of change of V1 gamma-band power relative to baseline during the test stimulus presentation, for three different levels of sample contrast variance during the test (equally spaced from minimum to maximum sample variance). Thick straight lines, fits of linear regressions to group average time course for interval 265 to 900 ms from test stimulus onset, capturing the sustained response component exhibiting the linear down-ramp evident in Supplementary Figure 2b. Data are represented as group mean (n=15 subjects, thin lines). b. Group average regression slopes as shown for V1 in a for all visual cortical field maps and as function of sample variance. Data are represented as mean (n=15 subjects, line) and ±SEM (error bars). Asterisks: * = p<0.05, ** = p<0.01 (repeated measures, 1-way ANOVA for factor sample variance). V1: F(2,28)=3.92, p=0.03); V2-V4: F(2,28)=5.3, p=0.01; V3AB: F(2,28)=3.7, p=0.04; IPS0/1: F(2,28)=4.0, p=0.03; LO1/2: F(2,28)=4.9, p=0.01; MT/MST: F(2,28)=6.2, p=0.006; VO1/2: F(2,28)=7.1, p=0.003. Source data are provided as a Source Data file. Figure 4. Correlation between psychophysical kernels and V1 power dynamics. Temporal correlation between the band-power time courses of V1 from Supplementary Figure 3 and the psychophysical kernels from Figure 1b. Each dot is one participant. Data are represented as mean (n=15 subjects, black horizontal lines) and individual participants (colored dots). Asterisks: **, p < 0.01; ***, p < 0.001 (two-sided t-tests). b: t(14)=-3.2, p=6.8e -3 ; g: t(14)=4.5, p=4.7e -4 ; HF: t(14)=4.2, p=9.4e -4 . These correlations were computed excluding the first two samples so as to eliminate the non-specific onset transient response that spanned all frequency bands (see Supplementary Figure 2b). Correlations were qualitatively identical for gamma and high-frequency (HF) bands when including all samples, but then also significantly positive for theta-and alpha-bands, presumably driven by the onset transient followed by power-suppression. Source data are provided as a Source Data file. Figure 5. Correlation between power dynamics contrast decoding in V1. As Supplementary Figure 3, but now correlating the band-power time courses to the V1 contrast decoding profiles from Figure 3B (left, orange line). Data are represented as mean (n=15 subjects, black horizontal lines) and individual participants (colored dots). Asterisks: ***, p < 0.001 (two-sided t-tests). g: t(14)=4.6, p=4.4e -4 ; HF: t(14)=5.1, p=1.7e -4 . The individual temporal profiles of the decoding of momentary contrast samples in V1 ( Figure 3B left, orange line) was strongly correlated with the profiles of the gamma-band and high-frequency responses. Thus, at least part of the decay evident in the contrast decoding profiles may be "inherited" from the decay of the underlying power responses (e.g., due to a decrease in signal-to-noise ratio for decoding). Source data are provided as a Source Data file. Figure 6. Effector choice-related activity lateralization across visuo-motor pathway. a. Time-frequency representations of power lateralization differences "test stronger" vs. "test weaker" choices. We subtracted power values contra-ipsilateral to the hand used for reporting the "test stronger" choice, and then contrasted lateralization values between "test stronger" and "test weaker" choices, separately for the two physical stimulus conditions (see Methods for details). Data are represented as mean (n=15 subjects, color code). Black contour, p<0.05 (cluster-based, two-sided permutation test against 0). Significant choice-related differential activity was evident before the motor response in three frequency bands, and most pronounced in the hand movementselective ROIs M1-hand, IPS/PostCeS, and aIPS: low-frequency (< 8 Hz, contralateral increase), alpha/beta  Hz, contralateral suppression), and high gamma-band (50 -100 Hz, contraleteral increase). Please note that choicepredictive alpha/beta-power lateralization in IPS/PostCeS and M1 became significant about 400 ms after stimulus onset, and then continued to build up throughout decision formation. b. Choice decoding, separately per stimulus category, as in Figure 2C, but now also sorted by the reported confidence level (high vs. low). Data are represented as mean (n=15 subjects, lines) and ±SEM (shaded areas). Dashed horizontal bar: p < 0.05 for the Stimulus: test weaker condition (cluster-corrected two-sided t-test of confidence high AUC < confidence low AUC). Only 13 out the 15 participants could be used for this analysis because data needed to be sorted in a total of eight categories for this analysis and not enough trials were available for at least one of them (specifically, the high-confidence error trials) in the remaining participants. Source data are provided as a Source Data file.  Figure  2C, and decoding AUC-scores were collapsed across categories, after flipping values about AUC-0.5. ROI-colors encode their AUC-values at t=1.1s (black vertical line, see colorbar next to y-axis). Colors are displayed on inflated cortical surfaces in the bottom row. Strongest decoding performance at t=1.1 was found in three ROIs: dorsal premotor cortex (Brodman area 6d, AUC=0.595), M1 (Brodman area 4; AUC=0.59), and the frontal eye field (FEF; AUC FEF=0.59±0.02). Performance in these ROIs was indistinguishable (repeated measures ANOVA, t=1.1 s, F(2,28)=0.42, p=0.66). All other areas showed decoding performance smaller than M1-hand. b. Approach (iii): Time courses (top) and spatial distribution (bottom) of choice decoding accuracy (AUC) from M1-hand (for comparison) and 17 pre-selected ROIs covering dorsolateral prefrontal cortex anterior to premotor cortex. Here, we substantially increased the spatial granularity of the decoding approach by using spectral estimates from each vertex per ROI, and we also included spectral phase (in addition to spectral power) as features for decoding (Methods). Choice-decoding was performed independent of the stimulus category. Again, this showed maximum decoding performance for M1 (AUC=0.63). We found lower decoding performance for all dorsoalateral prefrontal ROIs tested (next best area, Brodman area i6-8 (premotor cortex): AUC=0.59, paired t-test: p=0.002, t=-3.9). Data are represented as mean (n=15 subjects, lines). Source data are provided as a Source Data file.

Signal processing confounds cannot explain the cross-correlation results
The low-frequency kernels were based on spectral estimates for 0-20 Hz activity and the choice decoder time courses were based on mixtures of different frequencies (1-145 Hz), which were partially computed with different time windows (longer for frequencies < 10 Hz, see section Preprocessing, Spectral analysis, and Source Reconstruction of MEG Data). Another concern about the cross-correlation analysis might be that these signal processing differences might induce artefactual time shifts in the cross-correlation functions (e.g. due to power changes in one band being detected earlier than in the other). This concern might render the peak lags observed in Figure 5 difficult to interpret functionally. We ruled out this concern by systematically simulating cross-correlations of pairs of oscillatory signals with various different characteristics (transient vs. sustained vs. ramping power increases, pure vs. noisy oscillations) and then analyzed the signals in the exact same way as our actual MEG data (multi-taper sliding window FFT followed by cross-correlation as described above). We observed much higher positive lags in the data than in any of these simulations (Supplementary Figure 11). This renders it highly unlikely that the peak lags reported in Figure 5 were caused by signal processing artefacts.
Specifically, for each scenario, we simulated (250 simulation runs per scenario) a 10 Hz reference signal and a comparisonsignal of varying frequency (e.g., 50 Hz, Supplementary Figure 11a Figure 11a, c), because the detection of the power increase differed: The power increase in the 10 Hz signal was maximal when the time-window used for power computation was centered on the peak of the transient 10 Hz signal, i.e. after 50 ms (half of a 10 Hz cycle); the power increase of the 50 Hz cycle was maximal after 10 ms (half of a 50 Hz cycle). This yielded 40 ms as the difference between the two half-cycle durations. Other frequency combinations ( ≥ 10 Hz) produced a delay of 0.05 − ! "# s, which was bounded by 50 ms. Even this lag was substantially shorter than the peak lag observed in the data for early visual cortex (areas V1-V4, Figure 5). Further comparison frequencies < 10 Hz used two different time windows for power computation (as we did for the MEG data analysis) yielded negative delays, opposite to what we found in the data ( Figure 5). For the (more realistic) up-ramping signals (linear ramps) we obtained results in between those for sustained and transient signals, with small (<25 ms) absolute values of the peak lags (Supplementary Figure 11d, e).

Comparison of decoding approach with previous approaches for fMRI and MEG data
We here combined atlas-based MEG source reconstruction with a multivariate pattern classification approach that was based on the spectro-spatial patterns of local activity within each region. Current fMRI approaches enable decoding of sensory or cognitive variables from fine-grained multi-voxel patterns in multiple cortical regions 5,6 , but they lack the necessary temporal resolution for tracking the dynamics of decision formation. Conversely, E/MEG decoding studies 7-9 provide the critical temporal resolution, but commonly use the whole sensor array as features for decoding, precluding inferences about the information flow between brain regions. Our current approach is situated between these two lines of previous work and thus provides the opportunity to track large-scale information dynamics across cortical areas.

Neural encoding of decision information in motor vs. non-motor formats
Complementary analyses indicated that choice information in anterior intraparietal (IPS/PostCeS) and (pre-) motor cortical regions was primarily contained in large-scale spatial biases (right vs. left hemisphere) with respect to the action (left vs. right hand button press) prepared to report the choice (Supplementary Figure 6a). Indeed, decoding based on the spectral patterns of lateralization yielded very similar results as in Figure 2c (see source data file). We found no robust choice encoding in regions that did not exhibit such large-scale biases. When using finer-grained spatial patterns of both signal phase and amplitude as features for decoding, we did not find strong choice-predictive activity in more anterior regions of prefrontal cortex, even though this yielded higher choice-prediction values for M1 than the coarser-grained approach (Supplementary Figure 7, compare panels b and a, see Methods for all differences). Previous studies have identified action-independent, choice-predictive signals in human prefrontal cortex during tasks, in which the perceptual choice could not be mapped onto a specific action plan during decision formation 6,10,11 . Our task, however, allowed for such a mapping, mimicking a large body of work in animals [12][13][14][15] . The locus and format (action-dependent vs. -independent) of cortical build-up activity during decisions depends on the task context 16 . In our task, the choice-predictive activity may have been exclusively expressed in the format of motor preparatory activity.