Article | Published:

Persistent neuronal activity in human prefrontal cortex links perception and action

Nature Human Behaviourvolume 2pages8091 (2018) | Download Citation

Abstract

How do humans flexibly respond to changing environmental demands on a subsecond temporal scale? Extensive research has highlighted the key role of the prefrontal cortex in flexible decision-making and adaptive behaviour, yet the core mechanisms that translate sensory information into behaviour remain undefined. Using direct human cortical recordings, we investigated the temporal and spatial evolution of neuronal activity (indexed by the broadband gamma signal) in 16 participants while they performed a broad range of self-paced cognitive tasks. Here we describe a robust domain- and modality-independent pattern of persistent stimulus-to-response neural activation that encodes stimulus features and predicts motor output on a trial-by-trial basis with near-perfect accuracy. Observed across a distributed network of brain areas, this persistent neural activation is centred in the prefrontal cortex and is required for successful response implementation, providing a functional substrate for domain-general transformation of perception into action, critical for flexible behaviour.

Main

The neural mechanisms supporting flexible goal-directed behaviour in humans represent fundamental unresolved questions in neuroscience. Much research has examined how humans and other animals process stimulus information and execute a behavioural response. Yet, how the brain temporally and functionally binds stimulus processing with response execution across modalities and cognitive domains remains poorly understood. Extensive research supports a key role of the prefrontal cortex (PFC) in orchestrating complex behaviours. For example, profound deficits in goal maintenance and decision-making have been reported across species following PFC lesions1,2,3,4. Likewise, numerous neurological, psychiatric and developmental disorders are characterized by PFC dysfunction5,6,7, highlighting the importanc correlation distance matrix e of understanding the role of the PFC in human cognition. Finally, human neuroimaging and animal electrophysiology studies have identified the PFC as the core element in a distributed network of brain regions active during tasks requiring domain-general cognitive processing and response selection11,12,13,14,15,16,11. These findings highlight the importance of the PFC in organizing complex cognitive functions and in translating stimulus properties into task-appropriate behaviours. However, the mechanism by which the PFC bridges stimulus perception with response execution in humans remains undefined.

Single unit and local field potential studies in animals suggest that PFC neurons coordinate activity across functionally linked brain regions through temporally and spatially distributed neuronal computation2. In particular, persistent neuronal activity in the PFC has been observed during decision-making, working memory and response selection tasks featuring pre-defined delay periods11,12,13,14,15,16,17. These studies also reveal that persistent activity within the PFC is functionally heterogeneous, with distinct groups of neurons tracking different task variables, such as task demands, stimulus characteristics and response preference9,15. Temporally, sustained activation has also been reported before response selection in cortical areas outside the PFC, most prominently in sensory regions and the lateral intraparietal area (LIP)18,19,20. Persistent activity in the PFC and parietal regions has been reported in human neuroimaging studies during delayed response tasks (for example, delayed match-to-sample or working memory), where the blood oxygenation level-dependent (BOLD) signal scales with delay duration10,21,22. Evidence of persistent cortical activity during word generation and visual search has also been seen in human intracranial electrophysiology studies, but the behavioural relevance of this activity has not been defined23,24,25. In combination, these findings suggest that persistent activation in the PFC and functionally linked regions may support response selection, decision-making and working memory processes.

However, most studies examining the role of persistent activity in response selection and decision-making rely on tasks with a predefined delay period between stimulus presentation and the required response. Thus, it remains unexplored whether persistent activity, occurring between stimulus presentation and response execution, is specific to working and associative memory processes, as seen in previous work, or whether it reflects a cognitive process generalizable to diverse tasks with self-paced response selection. In addition, key limitations of existing studies, including the coarse temporal resolution of functional magnetic resonance imagine (fMRI) and the restricted number of brain areas and tasks examined in non-human primate studies, leave multiple unresolved questions about the role of persistent activity within the PFC and related brain regions in coordinating behaviour in humans.

Here we hypothesize that persistent neuronal activity indexes domain-general processes linking stimulus and response in humans across tasks varying in difficulty, stimulus properties (visual and auditory) and response modalities. We examine whether persistent neural activity reflects a single cognitive process, as typically reported in working memory studies (for example, engram maintenance and manipulation), or whether it is, instead, common to multiple functionally distinct task-relevant processes co-occurring within the stimulus-to-response window. Finally, we assess the anatomical specificity and cross-regional network interactions of persistent activity across a range of cognitive tasks.

To examine if persistent activity centred in the human PFC and related brain regions functionally bridges stimulus evaluation and response execution across tasks, we analysed electrocorticography (ECoG) data recorded from multiple cortical areas (1344 surface macroelectrodes; Supplemetary Fig. 1a,b and Supplementary Table 1) in 16 participants performing 8 tasks, ranging in cognitive demand, stimulus modality and response execution requirements (Supplementary Fig. 2 and Supplementary Table 2). The examined tasks included face categorization across the gender (male versus female) and emotion (angry versus neutral and happy versus sad) dimensions; visual categorization (cat versus dog, with various degrees of cross-category morphing); auditory categorization (male versus female voice, with various degrees of cross-category morphing); auditory and visual self-referential categorization (yes/no responses to either aurally or visually presented adjectives based on whether they can be used to describe the participant); word repetition (repeating aurally presented words); and antonym generation (saying a word semantically opposite to the aurally presented word). For all tasks, stimulus presentation was temporally discrete and the task performance was self-paced, allowing us to examine brain activity not bound by stimulus presentation or a predefined window for response execution (see Methods).

We first analyse the temporal dynamics underlying processing flow from stimulus perception to response execution across widespread cortical regions. We then provide evidence that persistent neural activity functionally links stimulus perception with response execution across cognitive domains and stimulus modalities.

Results

Temporal dynamics of PFC engagement across tasks

Local field potential power in the broadband high gamma range (HG; 70–150 Hz) was used to index cortical activation (Supplementary Figs. 3 and 4a). Broadband HG is linked to increased local neuronal firing in the underlying cortical tissue and is correlated with the fMRI hemodynamic response26,27,28,29. Thus, it provides an ideal signal to assess the anatomical and functional specificity of persistent activity across tasks. Overall, task-related increases in HG activity above the baseline were observed in 37% of analysed channels (active sites), 31.2% of which were located in the PFC (Supplementary Fig. 1c).

We first examined the temporal propagation of HG activity from stimulus perception to response execution across the cortex by defining three regions of interest (ROIs): Sensory, PFC and Motor (see Methods for ROI definition; Fig. 1a). Temporal differences across ROIs were assessed using the timing of HG onsets (initial neuronal engagement measured as the first significant increase in HG signal relative to the baseline) and HG peak amplitude latency (the point of maximal neuronal firing and synchronization within a given cortical area; Supplementary Fig. 5a). ROI analyses were performed across individual datasets, in that HG values were averaged over active sites within a single participant and task, and across individual trials, where the HG signal was averaged over trials with similar reaction times (RTs).

Fig. 1: Anatomical and functional influences on the chronology of information processing across cortex.
Fig. 1

a, Sites with significant HG activity classified according to anatomical ROIs. b, Latency for HG onsets (green) and peak amplitude (red) across ROIs (N Sensory = 26, N PFC = N Motor = 34). c, Sites with significant HG activity classified according to activity patterns, demonstrating functional heterogeneity within each ROI and anatomical heterogeneity for each HG activity type. d, Latency for HG onsets and peak amplitude calculated only across sites with HG activity pattern predominant in each ROI (Supplementary Table 6; Sensory—early stimulus; PFC—persistent stimulus-to-response; Motor—response; N Sensory = 26, N PFC = N Motor = 34). NS, not significant.

Temporal analyses across datasets

The PFC exhibited later HG onset and peak amplitude latencies across datasets than the Sensory ROI (Fig. 1b, Supplementary Figs. 6a and 7a and Supplementary Table 3; Mann-Whitney test for onset latency: U = 80, P = 6.9 × 10−8; peak latency: U = 61, P = 1.4 × 10−8; N Sensory = 26, N PFC = 34). The temporal delay in HG latencies in the PFC relative to the sensory regions indicates that the PFC is involved in task-related processing subsequent to initial stimulus perception, consistent with previous reports in non-human primates16 as well as in humans24. In contrast, there were no significant differences in HG onset or peak latencies between the PFC and Motor ROIs (Fig. 1b, Supplementary Figs. 6a and 7a and Supplementary Table 3), suggesting that task processing in the PFC temporally overlaps with response preparation and execution. We hypothesized that this temporal overlap between the PFC and Motor ROIs may be due to the functional heterogeneity of anatomically defined brain regions (Fig. 1c,d). This hypothesis is addressed later in the paper.

Temporal features of the HG signal were examined in relation to the response times within each ROI. Across tasks, both HG onset and peak amplitude latencies for the PFC and Motor ROIs were positively correlated with RTs (Fig. 2a and Supplementary Fig. 7b,c; onset latency PFC: r(32) = 0.42, P = 0.01; onset latency Motor: r(32) = 0.35, P = 0.05; peak latency PFC: r(32) = 0.56, P = 5.7 × 10−4; peak latency Motor: r(32) = 0.40, P = 0.02). This relationship was absent for the sensory ROI (onset latency: r(24) = −0.32, P = 0.11; peak latency: r(24) = −0.16, P = 0.44), indicating that the speed of initial sensory processing does not determine the speed of response selection and execution.

Fig. 2: Cortical distribution and temporal dynamics of HG activity across ROIs.
Fig. 2

a, The relationship of HG onset and peak amplitude latencies in the PFC to reaction times (RTs) across datasets (N PFC = 34, sorted by RT; left). Correlations between RTs and HG onset (middle) and HG peak (right) latencies in the PFC. b, Time course of HG activity across ROIs for a representative RT bin. c, The relationship between RTs and the point of divergence between HG signals in the PFC and Motor ROIs across RT bins (N RT bins = 34).

Temporal analyses across trials

We next analysed HG activity across individual trials with matched RTs to examine temporal parameters of processing flow independent of between-subject differences in processing speed and within-task RT variability. Trials for all participants and tasks were consolidated and binned according to their respective RTs, with each bin comprising trials with similar RTs across tasks (within a 50 ms time window; see Fig. 2b for averaged HG signal within a representative RT bin; Supplementary Fig. 8). RT bin analyses revealed similar patterns of HG propagation and RT correlations as seen in the dataset analyses above (see the Supplementary Information for a detailed discussion; Supplementary Figs. 6b and 9 and Supplementary Table 3).

Given the strong relationship between HG latencies and RTs (Supplementary Fig. 9), we conducted a stepwise regression analysis across binned trials to identify which temporal features of the HG signal across ROIs were most predictive of RT (Supplementary Information). In addition to HG onset and peak latencies for each ROI, the regression model also included the time points at which HG activity traces within two temporally adjacent ROIs diverged for the last time before the response. These points of divergence represent a temporal transition in activation as HG activity subsides in one ROI while rising in another (Fig. 2b). HG activity across RT bins peaked first in the Sensory ROI, followed by the PFC, and culminated in the Motor ROI (Supplementary Fig. 6b), resulting in two points of divergence: Sensory-to-PFC (representing the point when HG activity in the Sensory ROI begins to subside while activity in the PFC increases) and PFC-to-Motor (representing the transition of HG activity from the PFC to the Motor ROI). The peak latency for the Motor ROI was excluded from the regression since it was, by definition, linked to response execution. The only significant predictor of RT was the point of divergence between the PFC and Motor ROIs (Fig. 2c; r(32) = 0.93, P < 10−11). These findings indicate that onsets and peaks of HG activity within each ROI are less predictive of RT than the temporal interaction among ROIs reflected in the points of divergence.

The role of persistent activity in the temporal evolution of information processing

Having established that PFC activity temporally links stimulus processing with response execution, we next examined the functional role of persistent activity in stimulus evaluation and response selection. Different temporal patterns of HG activity were observed within each ROI, with some sites exhibiting transient increases time-locked to stimulus presentation or response execution, and other sites exhibiting HG activity temporally sustained from stimulus presentation to response execution (Supplementary Fig. 4a). To objectively identify sites with different temporal patterns of HG activity across tasks, we implemented unsupervised clustering of activity within each dataset (Supplementary Figs. 3 and 4). Each extracted cluster represented a set of sites with similar temporal profiles of HG activity (Supplementary Fig. 4b,c). Four main distinct temporal patterns emerged: early and late stimulus processing, persistent stimulus-to-response, and response, with HG onsets and peak latencies reflecting the time course of processing flow during the task (Fig. 3). Here we focus on the temporal evolution of the persistent stimulus-to-response activation pattern. The remaining HG activity patterns are described in detail in the Supplementary Information.

Fig. 3: Cortical distribution and temporal dynamics of HG activity.
Fig. 3

ad, Four common patterns of HG activity were observed: early stimulus (a), late stimulus (b), persistent stimulus-to-response (c) and response (d). Single trial plots for all participants, tasks and channels sorted by response times (black tick marks) are presented on the left side of each panel. Stimulus offset for each trial is plotted as a grey tick mark. The cortical distribution (left lateral surface) of sites corresponding to each HG pattern is presented on the right side of each panel. The percentage of all active sites exhibiting the corresponding HG pattern is indicated for each activity type.

Persistent stimulus-to-response activity (Fig. 3c), time-locked to stimulus presentation (mean HG onset across datasets = 349.8 ms post stimulus onset, s.e.m. = 39.6 ms, N persist. = 28) and lasting until the response (mean offset = 132.3 ms post response onset, s.e.m. = 25.0 ms), was observed in 32.7% of active sites. The onset and peak latency for the persistent activity occurred later than for late stimulus activity (Supplementary Fig. 6c and Supplementary Tables 3 and 4; onset: U = 107, P = 0.005; peak latency: U = 61, P = 8.1 × 10−5; N late stim. = 15), but preceded response activity (onset: U = 194, P = 0.0004; peak latency: U = 61, P = 1.6 × 10−5; N resp. = 29). This temporal progression was also evident at the single-trial level across RT bins (Supplementary Fig. 6d and Supplementary Table 3) and was present across participants and tasks (Supplementary Tables 4 and 5).

While all other HG patterns featured a single peak with a well-defined rise and fall for each RT bin, persistent stimulus-to-response activity was characterized by a well-defined peak during short RT trials (Supplementary Fig. 10) and an increasingly shallow plateau with activity distributed across the trial window for long RT trials (Supplementary Fig. 5b). Across RT bins, the first plateau peak (the point at which HG activity starts to level off) for persistent activity occurred about 300 ms post HG onset (mean = 268.7 ms, s.e.m. = 23.4 ms, N RT bin = 30) and correlated with RT; r(28) = 0.47, P = 0.01. The offset of the plateau (the last peak before HG activity begins to decrease) was 220.8 ms before the response (s.e.m. = 27.1 ms).

Functional heterogeneity within cortical regions

We observed substantial overlap in HG activity patterns within each ROI. In the PFC, persistent activation sites (64.0% of all active PFC sites) were interleaved with early stimulus (4.9%), late stimulus (13.8%), and response (17.3%) sites. A similar pattern was observed in the Sensory and Motor ROIs, with 63.6% of early stimulus sites located in the Sensory ROI and 61.4% of response sites located in the Motor ROI (Fig. 1c, Supplementary Fig. 1d and Supplementary Table 6). This heterogeneity of HG activity within each ROI affected the ROI-wide estimations of temporal HG propagation described earlier (Fig. 1b). Specifically, while there were no differences between the PFC and Motor ROIs for HG onset and peak latencies calculated across all sites irrespective of HG activity type, a significant difference emerged when we repeated the analysis using only sites with the most prevalent HG activity pattern within each ROI (early stimulus for the Sensory ROI, persistent stimulus-to-response for the PFC, and response for the Motor ROI; Fig. 1d). Both HG onsets and peak latencies in the Sensory ROI preceded those in the PFC (onset: U = 31, P = 5.2 × 10−7; peak latency: U = 39, P = 1.3 × 10−6; N Sensory = 26, N PFC = 34), which, in turn, preceded those in the Motor ROI (U = 209, P = 0.005, peak latency: U = 109, P = 6.2 × 10−6; N PFC = N Motor = 34), revealing a clear chronological propagation of neuronal processing from the sensory regions to the PFC and then from the PFC to the motor regions16,24. This confirms that different types of HG activity are interleaved with the dominant HG activity pattern within each ROI, which influences ROI-wide analyses. For example, early HG onsets observed in the Motor ROI across both datasets and trials (Fig. 1b, Supplementary Fig. 6a,b and Supplementary Table 3) were due to the presence of sites with other activity types (38.6%), including a sizeable number of early stimulus sites (12.9%) located in the perisylvian region of the pre-central and post-central gyri.

Anatomical heterogeneity of persistent activity

Sites with persistent stimulus-to-response activity type were located primarily in the PFC (60.0% of all persistent sites), but were also found across the cortical mantle (Figs. 1c and 3c, Supplementary Figs. 1d and 11a and Supplementary Table 6). Sites with persistent HG activity were present in Sensory (7.1%) and Motor ROIs (17.5%) as well as secondary association cortices (15.4%). The distribution of sites with persistent activity matched previously reported activations in fMRI studies of domain-general cognitive processing (Supplementary Fig. 11b). Thus, although persistent HG activity was centred in the PFC, there was also anatomical heterogeneity across the cortex.

The majority of persistent sites (90.6%) were located in the left hemisphere (Supplementary Fig. 1d; left versus right hemisphere proportion difference: Z = 18.39, P < 10−10; all reported percentages are adjusted for coverage differences between hemispheres). Strong left-hemisphere representation of persistent HG activity was present for both linguistic and nonlinguistic tasks, with greater prevalence for nonlinguistic tasks (9.2% for linguistic tasks, 13.4% for nonlinguistic; proportion difference Z = 2.34, P = 0.003, adjusted to compensate for differences in electrode coverage between linguistic and nonlinguistic tasks). Since responses in all tasks were verbal or were based on semantic categorization of stimuli (regardless of stimulus modality), left hemisphere dominance for persistent HG activity supports modality-independent semantic processing during response selection.

There were also important regional differences in PFC representation of persistent activity. Notably, within the left hemisphere, linguistic tasks primarily recruited persistent sites in the inferior frontal gyrus (IFG) and Broca’s area (28% all recorded sites in the IFG/Broca were classified as persistent relative to 11% in other left PFC regions, Z = 2.64, P = 0.008). Conversely, nonlinguistic tasks engaged other PFC regions (for example, middle frontal (MFG) and cingulate gyrus (ACC) and premotor regions; 27% relative to 14% in IFG/Broca; Z = 4.82, P < 10−6; Supplementary Fig. 12). These results are consistent with the role of the IFG and Broca’s area in language production30,31,32 and suggest that persistent HG activity in these regions reflects verbal response selection. In contrast, persistent activity centred in the MFG, anterior cingulate gyrus and pre-motor regions during nonlinguistic tasks may reflect task set maintenance required for successful response selection9.

Persistent HG activity indexes response selection

As detailed above, left-hemisphere lateralization and PFC distribution of persistent HG activity suggests a potential role in response selection. To test this hypothesis, we examined whether persistent activity is modulated depending on the success or failure of response selection. Relative to successfully completed trials, trials during which participants failed to generate a response in the antonym generation task (the only task that produced enough no-response trials) exhibited delayed onset and diminished HG amplitudes for persistent activity (P < 0.05, FDR-corrected; Fig. 4), providing evidence that sustained activity is essential for successful response selection. Given the temporal progression of HG activity (Fig. 1d, Supplementary Fig. 6c,d and Supplementary Table 3), we suggest that the persistent HG plateau (Supplementary Fig. 5b) reflects the deliberation window for response selection.

Fig. 4: Persistent HG activity is critical for response selection.
Fig. 4

Average traces for the early stimulus (N = 570), persistent stimulus-to-response (N = 1734) and response (N = 596) HG activity patterns for trials with and without a response (matched numbers of trials) in the antonym generation task. Persistent HG activity was delayed and diminished in amplitude, but still present, on trials where no response was generated. Shading on each trace indicates s.e.m. across trials for each time point. Black bold segments along the x axis indicate time points with significant differences between trials with and without responses (P < 0.05, FDR-corrected).

In contrast, response-linked activity was absent on no-response trials, indicating a clear functional dissociation between persistent and response activations. Since persistent activity was present while response activation was absent during these trials, we can conclude that the persistent activity pattern indexes response selection and preparation, but not motor execution (see the Supplementary Information for a detailed discussion of the potential role of response HG activity in response preparation).

Persistent HG activity functionally links stimulus perception and response execution

To be considered a functional link between stimulus and response, persistent activity must be temporally coupled with stimulus processing and must reflect stimulus properties relevant for response selection. Furthermore, persistent activity must also be functionally linked to response execution.

Stimulus processing reflected in persistent HG activity

We first tested whether persistent stimulus-to-response activity is temporally coupled with stimulus processing. We used a stepwise regression across individual trial bins, with the onset of persistent activity as the dependent variable and onset, peak, and offset latencies of early stimulus and late stimulus HG patterns as independent variables. The onset and peak latencies of early stimulus activity and onset and offset latencies of late stimulus activity were significant predictors (in that order) for the onset of persistent stimulus-to-response HG activity (adj. R 2 = 0.61, P = 1.05 × 10−5; N RT bins = 30). Notably, the onset of the persistent activity occurred prior to the offset of both early stimulus and late stimulus activity (Fig. 3; mean difference = 690.4 ms, s.e.m. = 32.4 ms), suggesting that task-relevant persistent processing is triggered well before the completion of initial stimulus processing. These findings indicate that persistent HG activity is temporally linked with early and late stimulus activity, as these three types of neuronal activity reflect different aspects of stimulus processing.

Second, we evaluated whether persistent activity encodes response-relevant stimulus features by examining whether HG amplitude of persistent activity differentiates stimulus categories in the visual and auditory categorization tasks. Six persistent sites (16.6%) exhibited significant HG modulation as a function of stimulus category (dog versus cat or male versus female, mean cluster-wise P = 0.011, s.d. = 0.005 across sites demonstrating the category effect; Fig. 5a,b). However, since there was a direct correspondence between the stimulus and response in these tasks (that is, the image of a dog was always associated with the correct response ‘dog’), it is conceivable that persistent HG activation simply reflects response preparation and not stimulus features. To examine whether persistent activity is sensitive to stimulus properties independent of the response, we examined the antonym generation task, where the response was dissociated from the stimulus category. Significant HG modulation as a function of stimulus category (noun versus adjective) was observed in 10.4% of persistent sites (mean cluster-wise P = 0.013, s.d. = 0.003), highlighting the role of persistent activity in processing stimulus features relevant for response selection. Across these tasks, sites with persistent stimulus-to-response activity sensitive to the stimulus category effect were primarily located on the PFC (the IFG and MFG) as well as the parietal cortex, precentral gyrus and sensory association cortices (Fig. 5a), reinforcing the role of the PFC and related areas in linking stimulus integration with response selection.

Fig. 5: Stimulus features and task demands affect spatial and temporal profiles of persistent stimulus-to-response HG activity.
Fig. 5

a, Persistent stimulus-to-response sites exhibiting a category effect in the antonym generation as well as the visual and auditory categorization tasks (N = 7 participants). b, Representative persistent stimulus-to-response sites showing a category effect in the visual categorization task (dog, N = 91, versus cat, N = 90) and a part of speech effect in the antonym generation task (noun, N = 78, versus adjective, N = 69). Shading on each trace indicates s.e.m. across trials for each time point. Black bold segments along the x axis indicate significant differences between traces (P < 0.05 for at least 100 ms). c, Persistent stimulus-to-response sites active during both the word repetition and antonym generation tasks (‘overlap’, N = 24) or active only during the antonym generation task (‘unique’, N = 24) in three participants (S3, S15, S18; see Supplementary Tables 1 and 2). d, Onsets of persistent stimulus-to-response HG activity averaged across overlap sites in word repetition, overlap sites in antonym generation, and unique sites in antonym generation (S3: N overlap = 11, N unique = 6; S15: N overlap = 6, N unique = 11; S18: N overlap = 10, N unique = 7). Error bars denote s.e.m. across sites. NS, not significant.

Modulation of persistent HG activity as a function of reaction time

We next examined the link between persistent activity and response execution. First, we assessed the relationship between RT and the temporal features of the HG signal (Fig. 6a,b). Across datasets and individual trial bins, significant correlations with RT were observed for both persistent HG onsets (across datasets: r(25)= 0.69, P = 4.4 × 10−5; across trials: r(28) = 0.86, P = 1.6 × 10−9) and HG peak latencies (across datasets: r(25) = 0.79, P = 6.3 × 10−7; across trials: r(28) = 0.51, P = 0.004), with slower RTs associated with later onset and peak of persistent activity. Similar correlations were observed for response activity (Supplementary Fig. 13), which was expected given that the temporal proximity of HG onset and peak latency to response onset was the defining feature for this activation pattern (which was not the case for the persistent HG pattern). In contrast, there was no correlation between RT and HG onset or peak latency for early stimulus and late stimulus sites, indicating that the link between timing of the persistent stimulus-to-response activity and RT cannot be attributed to differences in speed of sensory processing across participants, tasks, stimulus parameters or response modalities. Similar results were obtained when sources of variance were minimized by averaging across participants within each task (Supplementary Fig. 14a) or by examining data from one participant who completed six tasks (Supplementary Fig. 14b).

Fig. 6: Interaction between persistent and response HG activity predicts reaction times.
Fig. 6

a, The relationship of RTs to onset and peak amplitude latencies for persistent stimulus-to-response HG activity across datasets (sorted by RT; N Persistent = 28). b, Chronology of the persistent and response HG activity as a function of RT (shown across select RT bins). The persistent stimulus-to-response plateau (from the first to the last HG peak) is blocked in grey. Shading on each trace indicates s.e.m. across trials for each time point (average number of trials per bin = 439 trials). c, Longer RTs were associated with decreased HG amplitudes of the persistent stimulus-to-response plateau (top) and larger peak amplitudes for response activity (bottom) across RT trial bins (N RT bins = 30).

Since previously explored stimulus properties were reflected in modulation of persistent HG amplitudes across stimulus categories, we next examined whether HG amplitudes also varied as a function of response. The mean magnitude of the persistent HG signal across the plateau time period was inversely correlated with RT (Fig. 6c; r(28) = −0.89, P = 2.6 × 10−11), indicating that persistent activity was diminished in amplitude and more distributed in time for trials on which participants were slower to respond. In contrast, fast reaction times were associated with higher amplitudes and shorter duration of the HG signal (Supplementary Fig. 10). Amplitude of HG activity for early stimulus and late stimulus activity patterns did not correlate with RT, and only peak amplitude of response activity increased as RTs lengthened (r(28) = 0.77, P = 5.4 × 10−7; Fig. 6c), potentially indicating greater response execution effort for trials where a response was not readily available and took longer to generate.

In summary, persistent stimulus-to-response activity predicted behavioural responses and was functionally and temporally linked to stimulus processing, supporting a role in bridging stimulus perception and response execution.

Functional heterogeneity of persistent activation

Using the well-established link between brain anatomy and function, we next tested whether persistent activation encompasses multiple functionally heterogeneous task-specific sub-processes by examining changes in anatomical distribution of persistent HG activity with increased task difficulty and additional cognitive operations. For this analysis, we focused on data from the word repetition and antonym generation tasks, both of which were recorded in three participants (S3, S15 and S18). Both tasks employed identical stimuli, yet required different levels of cognitive processing: antonym generation relied on semantic evaluation and search, absent during word repetition, resulting in longer antonym generation RTs (S3: t(144) = 14.74, P = 1.5 × 10−30; S15: t(259) = 15.08, P = 2.6 × 10−37; S18: t(173) = 8.77, P = 1.6 × 10−15) and greater RT variability (Supplementary Fig. 2 and Supplementary Table 2).

There were more sites with persistent HG activity during antonym generation than word repetition (Fig. 5c). Sites common to both tasks (‘overlap’ sites) were predominantly located on the IFG and pre-central gyrus (Broca’s area), reflecting a common substrate for word production30. Sites ‘unique’ to antonym generation (and not active during word repetition) were also present in the Broca’s area but were more broadly distributed throughout the left lateral PFC, including the middle and superior frontal gyri. Activity at these sites is likely to support semantic search—the dominant cognitive operation unique to antonym generation relative to word repetition. Notably, there were no differences between the two tasks in HG onset for the overlap sites. In contrast, persistent sites unique to antonym generation had later onsets than overlap sites (Fig. 5d; Mann-Whitney test, S3: U = 3.0, P = 1.5 × 10−3; S15: U = 19.0, P = 0.09; S18: U = 15.0, P = 0.03). HG activation for unique antonym generation sites was comparable in amplitude across both short and long RT trials, which excluded the possibility that engagement of unique persistent activation sites was due to task difficulty alone. Instead, these results indicate that task demands and the engagement of additional cognitive operations (such as semantic search in the antonym generation task) alter the spatial topography and elicit functionally and temporally distinct subtypes of temporally sustained activity.

Successful task performance depends on activity across distributed networks

To assess the relationship between persistent HG activity and other activity types on a trial-by-trial basis, we conducted a principal component analysis (PCA) on mean HG amplitudes across trials. The first principal component, accounting for the most variance (24.81% of variance, s.e.m. = 1.9% across tasks), revealed a distributed network of sites with shared trial-by-trial HG modulation. Most cortical sites with persistent activity (57%) were part of this primary functional network, which also included all other types of HG activity patterns (Supplementary Fig. 15). These results are in line with growing literature on the importance of PFC-centred functional networks in spatiotemporal integration of information critical for flexible cognitive processes and successful goal-directed behaviour2,33.

Discussion

In the current study, we examined persistent activity in the PFC and related regions across multiple self-paced tasks ranging in difficulty, required cognitive operations, and behavioural responses. Capitalizing on the superb temporal resolution and spatial specificity of direct cortical recordings, we demonstrate that intrinsically sustained neural activity provides domain-general spatiotemporal integration that links perception and action. Our results suggest that task-relevant cognitive processes in the stimulus-to-response time window are reflected in persistent neural activity in the PFC and functionally linked regions. The observed domain-general persistent activity shares key features with sustained activity previously reported in animals and humans in fixed-delay tasks, such as working memory, in accord with our primary hypothesis of shared morphology and function across a range of behaviours.

Temporally, information integration and decision-making was reflected in persistent stimulus-to-response neuronal activity, triggered during initial stimulus processing and lasting until the response. Stimulus, persistent and response processing occurred in sequential progression with partially overlapping stages. These findings extend previous research on region-specific processing timescales in non-human primates16,34, emphasizing that the chronology of information processing across brain regions is driven by temporal patterns of neuronal activation largely specific to each brain region. Spatially, persistent activation was centred in the lateral PFC, but was also observed in the medial PFC, temporal lobe, anterior cingulate cortex and parietal lobe—areas that feature extensive anatomical connections with the lateral PFC35,36 and that have been implicated in domain general processing10,37,38. This extended cortical network provides an ideal anatomical and physiological substrate for information exchange and integration39. Functionally, persistent stimulus-to-response activity reflected both stimulus processing and response selection. Specifically, amplitude modulation of persistent activity, especially in the PFC, reflected stimulus features and scaled with successful task performance, while also being strongly linked to response speed. In addition, the presence of sites with both stimulus- and response-linked HG activity within the single functional network dominated by sites with persistent HG activity reinforces the role of persistent activity as the link between stimulus processing and response execution.

Critically, we demonstrate that persistent neuronal activity does not represent a single localized process but, instead, subserves a collection of task-specific functions. For example, not all sites with persistent activity performed similar functions, as only a subset was active in different tasks or was sensitive to stimulus properties. Likewise, while persistent activity in the MFG and ACC during non-linguistic tasks supports maintenance and implementation of task-specific instructions on a trial-by-trial basis9, persistent activity in Broca’s area during linguistic tasks supports semantic processing and verbal response selection30. Even within the linguistic tasks, persistent activity demonstrated functional and anatomical dissociation. Specifically, persistent activity common to antonym generation and word repetition was centred in Broca’s area, suggesting a common substrate for word production. In contrast, persistent activity unique to the antonym generation task, which required more effortful semantic search and maintenance of task instructions, was seen in the MFG, ACC and other brain areas. Thus, the high temporal resolution of intracranial EEG allowed us to demonstrate that the increase in brain activation associated with greater cognitive demand is attributable to additional recruitment of local neuronal populations within the PFC, distributed over time, and spatially interleaved with initial task-related activity. These results provide critical evidence that increased brain activation in response to greater processing demands is not simply due to an increase in amplitude of already-present activity, but rather due to a separate and temporally distinct recruitment of additional neuronal resources, even within the same ROI40.

The functional heterogeneity of persistent HG activity is likely to stem from the nature of the HG signal as an index of the cumulative output of local neuronal networks integrated over time and cortical space26,27,28,29,41. Specifically, while some neurons may exhibit persistent firing, other neuronal populations may become active at various stages of the temporal stimulus-to-response window, demonstrating specific tuning to particular stimulus dimensions or response contingencies9,42. Given that HG signal reflects local neuronal firing29,42, our findings indicate that fast behavioural responses rely on increased simultaneous firing across multiple neuronal ensembles, evident from both the temporal convergence and the high amplitudes of stimulus, persistent and response HG activity patterns for short RT trials. In contrast, as the response deliberation window lengthens (due to either response uncertainty or increased neuronal or cognitive signal-to-noise ratio), the amplitude of persistent activity diminishes, indicating decreased, yet temporally distributed, local neuronal firing.

In summary, we observed persistent activity across a diverse range of tasks engaging multiple cognitive processes, and propose that intrinsically persistent neuronal activity in the stimulus-to-response window provides a common functional substrate for information integration and response selection in humans. We also demonstrate that the prefrontal cortex serves as the core element of a distributed cortical network that links stimulus perception with action execution, enabling humans to flexibly respond to ever-changing environmental demands.

Methods

Participants

All procedures were approved by the Institutional Review Boards of University of California (UC) Berkeley, UC San Francisco, Stanford School of Medicine, California Pacific Medical Center (CPMC) and Johns Hopkins University School of Medicine. Eighteen individuals undergoing neurosurgical evaluation for refractory epilepsy were recruited. Patients willing to participate in research signed the informed consent document before testing. Subdural electrode arrays (2.3 mm exposed electrode diameter with 10 mm inter-electrode spacing) were implanted for approximately one week to determine epileptogenic focus (see the Supplementary Information for electrode localization procedures). Electrode number and placement were solely dictated by clinical needs. Electrode coverage for most brain areas was represented in multiple participants (Supplementary Fig. 1c and Supplementary Table 1), which limits the potential influence of any single pathology and enables broad generalization of results. Two participants were excluded: one due to a stroke-related cortical lesion and the other because of missing electrode localization data. The remaining 16 participants (Supplementary Table 1) had normal IQ and were fluent in English, except for one Spanish-speaking participant who completed a Spanish version of the visual categorization task.


Tasks and stimuli

Eight tasks, varying in difficulty and stimulus modality, were used. Task selection for each participant was determined by electrode location, time availability and the participant’s willingness and ability to perform the task. Each participant performed one to six tasks (Supplementary Table 2). Visual stimuli for all tasks were presented using a laptop (15.6″ LCD screen) placed in front of participants at a comfortable distance (0.5–1.0 m). Auditory stimuli (50–60 dB) were presented via two speakers placed on each side of the presentation laptop. Onsets and offsets of stimuli were detected via analogue channels: photodiode for visual stimuli and speaker for auditory. Participants made responses either by pressing the appropriate key on the laptop keyboard or by speaking into a microphone, with verbal response times extracted from an analogue microphone channel.

Visual face categorization tasks

Participants were asked to categorize facial stimuli (NimStim dataset43) either on the dimension of emotion (angry versus neutral or sad versus happy, presented in separate blocks) or gender (female versus male, across the two blocks of emotional faces). Stimulus presentation and response tracking was accomplished with the E-Prime2 software (Psychology Software Tools, Sharpsburg, PA). Stimuli were presented in a randomized order on a white background. Each trial started with a black fixation cross (1,500 ms) followed by a face stimulus (300 ms), which was replaced by a fixation cross, during which participants made the response using a keyboard. The trial was terminated after the response or after 3,000 ms, if no response was detected. Emotion and gender categorization blocks were presented in a counterbalanced order. Stimuli from the two blocks within each categorization condition were combined for analyses.

Auditory categorization task

Participants were asked to categorize gender morphed utterances (the word ‘town’) as being spoken by a female or a male speaker. Stimuli were adapted from the Carnegie Mellon University Arctic Database44. Gender morphs of the category prototypes were constructed in steps of 20, 40, 60 and 80% along the shortest trajectory between formant boundaries45. Participants were presented with a visual cue (blue cross) for 600 ms, followed by a stimulus (524 ms), and were given 1,500 ms to verbally respond, followed by a jittered ~1,000 ms inter-trial interval (ITI). Stimuli were presented in a randomized order using E-Prime2.

Visual categorization task

Participants were asked to categorize morphed visual images as a cat or a dog. Cat and dog prototype and morphed stimuli (20, 40, 60 and 80% morphs) were adapted from previous work46. Participants were presented with a visual cue (blue cross) for 500 ms, followed by a visually presented morphed stimulus (600 ms), and were given 1,500 ms to verbally respond, followed by a jittered ~1,000 ms ITI. Stimuli were presented in a randomized order using E-Prime2.

Auditory word repetition task

Participants were asked to verbally repeat aurally presented words, which were selected from the Affective Norms for English Words47. Stimulus duration range was 295–1,013 ms (mean = 645 ms, s.e.m. = 4 ms). Word stimuli were either nouns or adjectives (equal number of each), and stimuli in the two part-of-speech categories were matched on length, word frequency and emotional content (valence and arousal). Words were presented using MATLAB (The MathWorks, Natick, MA) in a pseudo-random order (no more than two words of the same part of speech presented in a row) with a jittered ~4,000 ms ITI.

Auditory antonym generation task

Participants were asked to verbally generate an antonym to an aurally presented word stimulus. Word stimuli and task structure were identical to those used for the auditory word repetition task, but stimuli were presented in a different pseudo-randomized order. Participants always performed the word repetition task first, with the two tasks never performed back-to-back in any one recording session to avoid habituation effects.

Auditory and visual self-referential tasks

Participants were asked to verbally respond whether each aurally or visually presented word could be used to describe them (‘yes’ or ‘no’ responses). Positive and negative adjectives were selected from the ANEW set and were matched on arousal, valence intensity, word length and word frequency. Audio stimuli (duration range 305–1,024 ms; mean = 690 ms, s.e.m. = 5 ms) were presented using MATLAB. Visual stimuli (400 ms) were presented using E-Prime2. In each task, stimuli were presented with a jittered ~4,000 ms ITI in a pseudo-random order with no more than two stimuli of the same valence presented sequentially. Task order was counterbalanced across participants.


Data acquisition

At UCSF and Stanford, data were acquired using a 128-channel TDT recording system (Tucker-Davis Technologies, Alachua, FL), filtered online at 0.5–300 Hz and sampled at 3,052 Hz (1,526 Hz for one participant). At Johns Hopkins, data were recorded at 1,000 Hz, with a low-pass 300 Hz analogue anti-aliasing filter using a 128-channel Stellate Harmonie system (Stellate Systems, Montreal, Canada). At CMPC, data were recorded at 1,000 Hz using a Nihon-Kohden Neurofax EEG-1200 system (Tokyo, Japan). Analogue channels (microphone, photodiode and speaker output) were recorded synchronously with ECoG signals at 24.4 kHz (UCSF, Stanford) or 1,000 Hz (Johns Hopkins, CPMC). ECoG data were recorded using a subdural electrode as reference (an electrode with minimal or stable signal located away from cortical areas of interest) and a scalp electrode as ground. Sampling rates, online filters and amplification across all recording systems were set to allow comparability across recording sites for the broadband HG (70–150 Hz) signal.


Data preprocessing

Data were recorded from 1,365 ECoG electrodes. Line and equipment noise was removed using an iteratively fit zero-phase Butterworth filter (see Supplementary Methods). Data channels with poor signal quality, epileptiform activity, or those located on subsequently resected tissue were excluded, leaving 1,344 electrodes for further analyses. Since multiple participants completed several tasks (each representing a single dataset), the total number of analysed data channels was 3,051. Data from each channel were down-sampled to 1,000 Hz, whenever needed, and re-referenced to a common average reference within each dataset (channels with poor data quality or periods of epileptiform activity were not included in the common average calculation). Data were visually inspected for periods of transient epileptiform activity or recording artifacts, which were excluded from subsequent analyses.


Spectral decomposition

The analytic amplitude of the broadband HG signal was extracted from the raw ECoG data across the full duration of each recording session. First, a two-way, zero phase-lag, finite impulse response filter (eegfilt.m function, EEGlab toolbox) was applied to extract signal in the 70–150 Hz range. This bandwidth was selected as it excludes any residual line noise and captures most of the broadband HG power48,49,50. Analytic amplitude was calculated by taking the absolute value of the Hilbert transform of the filtered signal, which is comparable to other filtering techniques (for example, wavelets)51. HG signals were next smoothed with a 10 Hz low-pass Butterworth filter. See Supplementary Figs. 3 and 4a for examples of raw data and HG analytic amplitude traces.


Task-active channel selection

To extract stimulus-related activity, data were segmented into trials starting 500 ms before stimulus onset and lasting until 500 ms past the maximum RT for the dataset. To extract response-related activity, trials were segmented from 500 ms before RT to 500 ms post-RT. HG signals were z-score normalized within each trial relative to the 500 ms pre-stimulus baseline (for the visual and auditory categorization tasks, the baseline was taken before the cue). Trials overlapping with artifact epochs or those on which participants did not respond, made an error or responded with hesitation (producing pre-response vocalizations) were excluded. No-response trials in the antonym generation task were analysed separately. Within each dataset, trials with RTs longer or shorter than three standard deviations from the dataset mean were considered to be outliers and were removed from analyses; 19 out of 43 datasets had trials excluded based on the RT criterion, with an average of 1.8 trials (s.e.m. = 0.3) or 1.3% (s.e.m. = 0.1%) excluded in each dataset. All exclusions were solely based on data quality assessment prior to performing any analyses to avoid selection bias. See Supplementary Table 2 for the numbers of trials that were included in final analyses and Supplementary Fig. 4a for examples of HG signal across trials for all analysed channels in one dataset.

To identify task-active channels within each dataset, z-scored HG signals were subjected to one-sample two-tailed t-tests performed across trials for each time point. For all t-test analyses, P values were corrected for multiple comparisons using the false discovery rate correction (FDR; q = 0.05)52. Channels were considered task-active if they contained at least one 100 ms segment of contiguous significance with a mean 10% signal increase from baseline.


Channel clustering based on temporal morphology of the HG signal

To identify channels with common temporal HG patterns (that is, common morphology of the HG signal averaged across trials), we used data-driven classification to avoid subjective biases motivated by a priori hypotheses. Data clustering was performed blind to temporal features of the HG signal. First, we reduced the dimensionality of the clustering space by conducting PCA (using correlation matrix and varimax rotation) on stimulus-locked HG time series (observations) averaged across trials for each active channel (features). Each dataset was analysed separately since the temporal profile of the HG signal depends on RT parameters (mean, range and distribution shape), which are different for each dataset. The number of significant principal components (PCs) was determined using a variant of parallel analysis, whereby comparison data were generated for increasing numbers of components until the observed eigenvalues failed to show significant improvement53 (all PCA analyses were performed using R). Following PCA, channels were clustered based on component weights for significant PCs, which is equivalent to clustering channels based on their location in PC-dimensional space (Supplementary Fig. 3). Clusters were identified using complete-linkage hierarchical clustering of a correlation distance matrix (see Supplementary Methods). Thus, PCA and clustering procedures resulted in a number of clusters (3–5 per dataset) that comprised groups of channels exhibiting similar temporal patterns of HG activity (that is, having similar morphology of the averaged HG signal; Supplementary Fig. 4b,c).


Cluster classification based on temporal patterns of HG activity

Temporal patterns of HG activity typified by each cluster were identified by applying predetermined selection criteria to avoid subjectively biasing the results. First, clusters with low signal-to-noise ratio (<15% post-stimulus HG signal change from baseline for at least 100 ms consecutively) were excluded from further analyses (13% of all identified clusters). Next, for the remaining clusters, onset and offset times of stimulus- and response-locked cluster-wide HG activation (averaged across trials from all channels within the cluster) were determined using one-sample t-tests performed for each time point using the FDR correction for multiple comparisons. To identify clusters with persistent stimulus-to-response HG activation, a binary 0/1 design matrix (trials × time points) was constructed, with ‘ones’ corresponding to each time point starting from the cluster-specific HG onset and lasting to the RT for each trial.

We hypothesized that persistent HG activity may not be homogenous both temporally and functionally. Thus, to account for potential variability in HG offsets relative to the response, a dictionary of design matrices was constructed for each cluster by creating multiple versions of the original design matrix with offsets shifted from 300 ms pre-response to 450 ms post-response in 25 ms steps. Each entry in the design matrix dictionary was correlated (using Pearson’s correlation) with the trial-by-trial HG time series for that cluster (across all trials for each data channel in the cluster).

Given the large number of data points and a potential influence of HG amplitude on r and P values, we implemented a non-parametric significance testing to avoid spurious significant results. The surrogate distribution was built for each design matrix entry by randomly time-shifting HG data series relative to the design matrix data series 10,000 times. On each iteration, a break point was randomly selected in the time series, and the data were circularly shifted around that break point. The circular shift preserves temporal data structure while varying the temporal relationship between the HG signal and the design matrix data series. A significance threshold of P < 0.001 was selected to account for multiple comparisons. Clusters with persistent HG activity were defined as those with a significant above-threshold correlation for any design matrix in the dictionary. To ensure that persistent HG activity was not primarily driven by response execution, the HG peak amplitude had to occur no later than 50 ms pre-response in the response-locked trace. Across all clusters classified as persistent, the average highest correlation across the design matrix entries was r = 0.21 and the average best HG offset (determined by the highest correlation across the design matrix entries) was 106.3 ms post-response (s.d. = 162.9 ms).

Clusters that did not match persistent HG activity criteria were classified as early stimulus, late stimulus or response based on the following set of rules. Clusters with transient HG increases present both post-stimulus and post-response for auditory tasks with verbal responses, or time-locked to stimulus onset and stimulus offset for visual tasks, were classified as early stimulus, representing HG activity sensitive to physical (acoustic or visual) properties of the stimulus (see Supplementary Information for detailed discussion). Clusters exhibiting transient stimulus-locked activity (with HG offset at least 300 ms pre-response) without sensitivity to physical stimulus properties (no HG activity evoked by verbal responses or HG activity sensitive to onset/offset of visual stimuli) were classified as late stimulus (see Supplementary Information for detailed discussion). Finally, clusters exhibiting response-locked activity peaking 50 ms pre-response or later were classified as response.

Five clusters (10 channels) exhibited HG activity sustained throughout the trial irrespective of the RT, probably reflecting tonic task-maintenance activity. Since this type of HG activity was present only in 1% of channels, they were not included in further analyses. In addition, cluster classification was not possible for five datasets due to insufficient RT ranges to discriminate among different types of HG activity (<100 ms minimum-to-maximum spread; see Supplementary Fig. 10 for temporal overlap between activation types at short RTs). Given that these datasets did not provide sufficient separation between stimulus presentation and the response to examine HG activity during the deliberation window, they were excluded from the analyses, leaving 115 clusters and 38 useable datasets (Supplementary Table 2). We took specific precautions to eliminate subjective biases in selection and clustering of HG activity and do not report any results that could be directly attributable to our classification criteria.


Levels of analysis

All analyses were performed across either (1) anatomical regions of interest (ROI) or (2) HG activity patterns. Examined ROIs included Sensory (bilateral superior and middle temporal gyri for auditory stimuli and the occipital cortex for visual stimuli), PFC (bilateral lateral and medial surface of the frontal lobe, excluding the pre-central gyrus and supplementary motor area (SMA)) and Motor (bilateral pre- and post-central gyri and SMA). The relationship between HG parameters and RTs across ROIs or HG activity patterns was analysed across datasets (data from a single participant/task) and within RT-based bins across all trials (irrespective of dataset). For the RT bin analysis, all trials across all datasets within each ROI or HG activation pattern were pooled together and sorted by RTs. Bins were constructed in steps of 50 ms from the minimum RT, and all trials within each bin were averaged together. Bins with low signal-to-noise ratio (<50 trials across all ROIs or HG activation types) were discarded. Thirty RT bins common to all HG activity patterns and 35 RT bins common to all ROIs were included in further analyses.


Determining latency and amplitude parameters of the HG signal

HG onset, HG offset, peak amplitude and peak amplitude latency were identified for each ROI and HG activity type (see Supplementary Fig. 5). In the dataset analysis for all ROIs and HG activity types, temporal and amplitude parameters were calculated for individual channels and then averaged across channels within each dataset. For RT bins, temporal and amplitude parameters were calculated on averaged traces for each bin. To avoid using spurious transient increases in HG activity as markers for HG onset or offset, we instituted a 100 ms threshold for contiguous significance as an indicator of true task-related increases in HG activity. Thus, the HG onset was defined as the first significant time point that was preceded by at least 100 ms where no data point passed the significance threshold (one-sample t-test, P < 0.05 FDR-corrected) and was followed by at least 100 ms where every data point passed the significance threshold. Similarly, the HG offset was defined as the first non-significant data point preceded by at least 100 ms of significant activity and followed by at least 100 ms of non-significant activity.

Peak amplitude and latency were calculated as a maximal HG value in a window from stimulus onset through 500 ms past the longest RT within each dataset or across RT bins. We employed these static windows for all ROIs and HG activity types to avoid potentially biasing influences of RT variability that could be introduced by identifying peak amplitude and latency only in the HG onset-to-offset windows (in which case, the probability of finding a high latency value would be higher for tasks with longer RTs). For 13 early stimulus channels (<1%), HG amplitude was greater following response vocalization than post-stimulus. Since all relevant analyses of early stimulus activity were performed on stimulus-locked HG traces, we re-calculated peak latency values for these channels, restricting the window of analysis from stimulus onset to HG offset.

The HG signal in long RT bins for the PFC ROI and persistent activity type was characterized by a plateau, rather than a distinct peak (Fig. 2b and Supplementary Fig. 5). Thus, additional parameters of the first plateau peak latency (when the rate of change of the positive slope begins to diminish and the activity begins to level off) and last plateau peak latency (the last point before the rate of change of the negative slope begins to increase) were identified. This window was used to calculate mean HG plateau amplitude. Finally, we identified the points of divergence for temporally adjacent ROIs (Sensory–PFC and PFC–Motor) or HG activity types (early-to-late stimulus, late stimulus-persistent, persistent-response). The divergence point is calculated as the last time point at which HG traces for the two ROIs or HG activity types diverge before the RT (that is, the point at which activity at one ROI/HG activity type begins to diminish and activity at the other ROI/HG activity type begins to increase).

The relationship between latency and amplitude parameters of HG activity and RT (across datasets and RT bins for both ROI and HG activity pattern analyses) were examined using Pearson’s correlations. Differences in HG parameters across datasets and RT bins were examined using Mann-Whitney nonparametric tests. FDR corrections for multiple comparisons were applied whenever necessary.


HG activity during trials with no response

HG activity for trials on which participants failed to respond was examined in the antonym generation task. All no-response trials were pooled together and averaged across channels within each HG activation pattern within each participant. For comparison, a matching number of correct trials with longest RTs were selected for each participant to account for the duration of response selection. A two-sample two-tailed t-test with FDR correction was performed on each data point comparing no-response and correct trials for each HG activity pattern. A similar one-sample two-tailed t-test with FDR correction was performed on each data point for response traces to determine whether there were significant HG increases during no-response trials.


Stimulus feature representation indexed by HG activity

HG amplitudes between different stimulus categories were compared for the visual and auditory categorization (100%/80% morphs versus 0%/20% morphs) and antonym generation tasks (nouns versus adjectives). For each channel, HG amplitudes were subjected to a two-sample two-tailed t-test between conditions at every time point across trials. Channels exhibiting significant condition differences (P < 0.05 for at least 100 ms consecutively) were considered sensitive to stimulus features. The false alarm rate for the 100-ms windows with consecutive P < 0.05 significance levels was calculated by examining the distribution of all windows with consecutive P < 0.05 significance levels, regardless of size, across all classified sites for all datasets that were used for this analysis. 96.7% of windows in the distribution were shorter than 100 ms in duration, indicating that the probability of obtaining a 100 ms (or longer) window with consecutive P < 0.05 levels was P = 0.033. Cluster-wise P values were calculated by averaging P values within the identified windows of at least 100 ms in length.


Functional network analysis

To identify sites exhibiting common fluctuations in HG amplitudes on a trial-by-trial basis, PCA (using the correlation matrix) was performed on the mean HG amplitude values within the activation windows (HG onset to HG offset) for each trial (observations) across all channels (features) within each dataset. PCs with eigenvalues above one were retained. To define functional networks based on shared trial-by-trial variability, sites with high weights on each remaining PCs were identified using a threshold calculated as the smallest maximum absolute component weight for a given dataset.


Life Sciences Reporting Summary

Further information on experimental design is available in the Life Sciences Reporting Summary.


Code availability

Analyses reported herein were conducted using custom computer code utilizing tools from MATLAB (including Signal Processing, Cronux, and Statistics toolboxes, with versions updated with new releases), R (version 3.2.2), and Python 2.7 software packages (scikit-learn, numpy, scipy, pandas, and matplotlib). EEGLab 11_0_4_3b was used for raw data visualization and preprocessing. Bioimage Suite 3.01 was used for electrode localization. Custom computer code is available from the corresponding author upon request. Examples of data analysis pipelines can be accessed at GitHub (https://github.com/matarhaller/demoanalysis).


Data availability

The de-identified raw data that support the findings of this study are stored in the Collaborative Research in Computational Neuroscience (CRCNS) database at UC Berkeley (crcns.org). It can be accessed with a free CRCNS account at crcns.org/data-sets/pfc.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

  1. 1.

    Duncan, J., Burgess, P. & Emslie, H. Fluid intelligence after frontal lobe lesions. Neuropsychologia 33, 261–268 (1995).

  2. 2.

    Fuster, J. M., Bodner, M. & Kroger, J. K. Cross-modal and cross-temporal association in neurons of frontal cortex. Nature 405, 347–351 (2000).

  3. 3.

    Stuss, D. T. & Knight, R. T. Principles of Frontal Lobe Function (Oxford Univ. Press, New York, NY, 2012).

  4. 4.

    Szczepanski, S. M. & Knight, R. T. Insights into human behavior from lesions to the prefrontal cortex. Neuron 83, 1002–1018 (2014).

  5. 5.

    Callicott, J. H. et al. Physiological dysfunction of the dorsolateral prefrontal cortex in schizophrenia revisited. Cereb. Cortex 10, 1078–1092 (2000).

  6. 6.

    Just, M. A., Cherkassky, V. L., Keller, T. A. & Minshew, N. J. Cortical activation and synchronization during sentence comprehension in high-functioning autism: evidence of underconnectivity. Brain 127, 1811–1821 (2004).

  7. 7.

    Mayberg, H. in Frontal-Subcortical Circuits in Psychiatric and Neurological Disorders (eds Cummings, J. L. & Lichter, D. G.) 177–206 (Guilford, New York, NY, 2001).

  8. 8.

    Curtis, C. E. & Lee, D. Beyond working memory: the role of persistent activity in decision making. Trends Cogn. Sci. 14, 216–222 (2010).

  9. 9.

    D’Esposito, M. & Postle, B. R. The cognitive neuroscience of working memory. Annu. Rev. Psychol. 66, 115–142 (2015).

  10. 10.

    Fedorenko, E., Duncan, J. & Kanwisher, N. Broad domain generality in focal regions of frontal and parietal cortex. Proc. Natl Acad. Sci. USA 110, 16616–16621 (2013).

  11. 11.

    Goard, M. J., Pho, G. N., Woodson, J. & Sur, M. Distinct roles of visual, parietal, and frontal motor cortices in memory-guided sensorimotor decisions. eLife 5, e13764 (2016).

  12. 12.

    Hernandez, A., Zainos, A. & Romo, R. Temporal evolution of a decision-making process in medial premotor cortex. Neuron 33, 959–972 (2002).

  13. 13.

    Kim, J. N. & Shadlen, M. N. Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat. Neurosci. 2, 176–185 (1999).

  14. 14.

    Rainer, G., Rao, S. C. & Miller, E. K. Prospective coding for objects in primate prefrontal cortex. J. Neurosci. 19, 5493–5505 (1999).

  15. 15.

    Riley, M. R. & Constantinidis, C. Role of prefrontal persistent activity in working memory. Front. Syst. Neurosci. 9, 181 (2016).

  16. 16.

    Siegel, M., Buschman, T. J. & Miller, E. K. Cortical information flow during flexible sensorimotor decisions. Science 348, 1352–1355 (2015).

  17. 17.

    Stokes, M. G. ‘Activity-silent’ working memory in prefrontal cortex: a dynamic coding framework. Trends Cogn. Sci. 19, 394–405 (2015).

  18. 18.

    Chafee, M. V. & Goldman-Rakic, P. S. Matching patterns of activity in primate prefrontal area 8a and parietal area 7ip neurons during a spatial working memory task. J. Neurophysiol. 79, 2919–2940 (1998).

  19. 19.

    Huang, Y., Matysiak, A., Heil, P., Konig, R. & Brosch, M. Persistent neural activity in auditory cortex is related to auditory working memory in humans and nonhuman primates. eLife 5, e15441 (2016).

  20. 20.

    Romo, R. & De Lafuente, V. Conversion of sensory signals into perceptual decisions. Prog. Neurobiol. 103, 41–75 (2013).

  21. 21.

    Curtis, C. E., Rao, V. Y. & D’Esposito, M. Maintenance of spatial and motor codes during oculomotor delayed response tasks. J. Neurosci. 24, 3944–3952 (2004).

  22. 22.

    Curtis, C. E. & Connolly, J. D. Saccade preparation signals in the human frontal and parietal cortices. J. Neurophysiol. 99, 133–145 (2008).

  23. 23.

    Bastin, J. et al. Direct recordings in human cortex reveal the dynamics of gamma-band (50–150 Hz) activity during pursuit eye movement control. Neuroimage 63, 339–347 (2012).

  24. 24.

    Edwards, E. et al. Spatiotemporal imaging of cortical activation during verb generation and picture naming. Neuroimage 50, 291–301 (2010).

  25. 25.

    Ossandon, T. et al. Efficient ‘Pop-Out’ visual search elicits sustained broadband gamma activity in the dorsal attention network. J. Neurosci. 32, 3414–3421 (2012).

  26. 26.

    Manning, J. R., Jacobs, J., Fried, I. & Kahana, M. J. Broadband shifts in local field potential power spectra are correlated with single-neuron spiking in humans. J. Neurosci. 29, 13613–20 (2009).

  27. 27.

    Mukamel, R. et al. Coupling between neuronal firing, field potentials, and fMRI in human auditory cortex. Science 309, 951–954 (2005).

  28. 28.

    Ray, S., Crone, N. E., Niebur, E., Franaszczuk, P. J. & Hsiao, S. S. Neural correlates of high-gamma oscillations (60–200 Hz) in macaque local field potentials and their potential implications in electrocorticography. J. Neurosci. 28, 11526–11536 (2008).

  29. 29.

    Ray, S. & Maunsell, J. H. R. Different origins of gamma rhythm and high-gamma activity in macaque visual cortex. PLoS Biol. 9, e1000610 (2011).

  30. 30.

    Flinker, A. et al. Redefining the role of Broca’s area in speech. Proc. Natl Acad. Sci. USA 112, 2871–2875 (2015).

  31. 31.

    Fedorenko, E., Duncan, J. & Kanwisher, N. Language-selective and domain-general regions lie side by side within Broca’s area. Curr. Biol. 22, 2059–2062 (2012).

  32. 32.

    Sahin, N. T., Pinker, S., Cash, S. S., Schomer, D. & Halgren, E. Sequential processing of lexical, grammatical, and phonological information within Broca’s area. Science 326, 445–449 (2009).

  33. 33.

    Braun, U. et al. Dynamic reconfiguration of frontal brain networks during executive cognition in humans. Proc. Natl Acad. Sci. USA 112, 11678–11683 (2015).

  34. 34.

    Murray, J. D. et al. A hierarchy of intrinsic timescales across primate cortex. Nat. Neurosci. 17, 1661–1663 (2014).

  35. 35.

    Catani, M. et al. Short frontal lobe connections of the human brain. Cortex 48, 273–291 (2012).

  36. 36.

    Sreenivasan, K. K., Curtis, C. E. & D’Esposito, M. Revisiting the role of persistent neural activity during working memory. Trends Cogn. Sci. 18, 82–89 (2014).

  37. 37.

    Duncan, J. & Owen, A. M. Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends Neurosci. 23, 475–483 (2000).

  38. 38.

    Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C. & Wager, T. D. Large-scale automated synthesis of human functional neuroimaging data. Nat. Methods 8, 665–670 (2011).

  39. 39.

    Miller, E. K. & Cohen, J. D. An integrative theory of prefrontal cortex function. Annu. Rev. Neurosci. 24, 167–202 (2001).

  40. 40.

    Cohen, J. D. et al. Temporal dynamics of brain activation during a working memory task. Nature 386, 604–608 (1997).

  41. 41.

    Mukamel, R. & Fried, I. Human intracranial recordings and cognitive neuroscience. Annu. Rev. Psychol. 63, 511–537 (2012).

  42. 42.

    Lundqvist, M. et al. Gamma and beta bursts underlie working memory. Neuron 90, 152–164 (2016).

  43. 43.

    Tottenham, N. et al. The NimStim set of facial expressions: judgments from untrained research participants. Psychiat. Res. 168, 242–249 (2009).

  44. 44.

    Kominek, J. & Black, A. W. The CMU Arctic speech databases. In: Fifth ISCA Speech Synthesis Workshop (eds Black, A. W. & Lenzo, K.) 223–224 (2004).

  45. 45.

    Kawahara, H. & Irino, T. in Speech Separation by Humans and Machines (ed. Divenyi, P.) 167–180 (Springer, Boston, MA, 2005).

  46. 46.

    Freedman, D. J., Riesenhuber, M., Poggio, T. & Miller, E. K. Categorical representation of visual stimuli in the primate prefrontal cortex. Science 291, 312–316 (2001).

  47. 47.

    Bradley, M. M. & Lang, P. J. Affective Norms for English words (ANEW): Instruction Manual and Affective Ratings Technical Report C-1 (Center for Research in Psychophysiology, Univ. Florida, 1999).

  48. 48.

    Crone, N. Functional mapping of human sensorimotor cortex with electrocorticographic spectral analysis. II. Event-related synchronization in the gamma band. Brain 121, 2301–2315 (1998).

  49. 49.

    Lachaux, J.-P., Axmacher, N., Mormann, F., Halgren, E. & Crone, N. E. High-frequency neural activity and human cognition: Past, present and possible future of intracranial EEG research. Prog. Neurobiol. 98, 279–301 (2012).

  50. 50.

    Miller, K. J. et al. Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations. Neuroimage 85, 711–720 (2014).

  51. 51.

    Bruns, A. Fourier-, Hilbert- and wavelet-based signal analysis: Are they really different approaches? J. Neurosci. Methods 137, 321–332 (2004).

  52. 52.

    Benjamini, Y. & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. Series B 57, 298–300 (1995).

  53. 53.

    Ruscio, J. & Roche, B. Determining the number of factors to retain in an exploratory factor analysis using comparison data of known factorial structure. Psychol. Assess. 24, 282–292 (2012).

Download references

Acknowledgements

We thank the patients for their cooperation, patience, and interest—without their help this research would not be possible. We also thank J. N. Hoffman, A. Flinker, R. Ivry, K. Johnson and J. D. Wallis for providing valuable comments and suggestions during manuscript preparation, and K. L. Anderson, M. Cano and V. N. Rangarajan for help in data collection.

This work was supported by the following grants: National Science Foundation (NSF) Graduate Research Fellowship DGE1106400 (M.H.), the National Institute of Mental Health F32MH75317 (A.S.), the National Institute of Neurological Disorders and Stroke (NINDS) R37NS21135 and the Nielsen Corporation (R.T.K.), NINDS R01NS078396 and NSF BCS1358907 (J.P.), NS40596 and NS088606 (N.E.C.), NIH R01DC012379 (E.F.C.). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

The MacBrain Face Stimulus Set was developed by Nim Tottenham (nlt7@columbia.edu) with support from the John D. and Catherine T. MacArthur Foundation Research Network on Early Experience and Brain Development. The dog–cat morph stimuli were provided by E. Miller from the Picower Institute for Learning and Memory and Department of Brain and Cognitive Sciences at the Massachusetts Institute of Technology.

Author information

Affiliations

  1. Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA

    • Matar Haller
    • , Robert T. Knight
    •  & Avgusta Y. Shestyuk
  2. Department of Psychology, University of California, Berkeley, CA, USA

    • John Case
    •  & Robert T. Knight
  3. Department of Neurology, The Johns Hopkins University Medical School, Baltimore, MD, USA

    • Nathan E. Crone
  4. Departments of Neurological Surgery, UCSF Center for Integrative Neuroscience, University of California, San Francisco, CA, USA

    • Edward F. Chang
  5. Department of Neurology and Neurosurgery, California Pacific Medical Center, San Francisco, CA, USA

    • David King-Stephens
    • , Kenneth D. Laxer
    •  & Peter B. Weber
  6. Stanford Human Intracranial Cognitive Electrophysiology Program (SHICEP), Department of Neurology and Neurological Sciences, Stanford University, Stanford, CA, USA

    • Josef Parvizi

Authors

  1. Search for Matar Haller in:

  2. Search for John Case in:

  3. Search for Nathan E. Crone in:

  4. Search for Edward F. Chang in:

  5. Search for David King-Stephens in:

  6. Search for Kenneth D. Laxer in:

  7. Search for Peter B. Weber in:

  8. Search for Josef Parvizi in:

  9. Search for Robert T. Knight in:

  10. Search for Avgusta Y. Shestyuk in:

Contributions

M.H., A.S. and R.T.K. conceived the study. M.H. and A.S. designed the experiments and collected the data. J.P., E.F.C., N.E.C., D.K.S., K.D.L. and P.B.W. recruited and examined the participants and facilitated data recording. M.H., J.C. and A.Y.S. analysed and interpreted the data. M.H., A.Y.S. and R.T.K. wrote the manuscript. A.Y.S. provided direct supervision during study design, data collection, data analysis and interpretation, and manuscript preparation stages.

Competing interests

The authors declare no competing interests.

Corresponding author

Correspondence to Avgusta Y. Shestyuk.

Supplementary information

  1. Supplementary Information

    Supplementary Notes, Supplementary Methods, Supplementary Tables 1–6, Supplementary Figures 1–14.

  2. Life Science Reporting Summary

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/s41562-017-0267-2