Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Distinct cortical systems reinstate the content and context of episodic memories

Abstract

Episodic recall depends upon the reinstatement of cortical activity present during the formation of a memory. Evidence from functional neuroimaging and invasive recordings in humans suggest that reinstatement organizes our memories by time or content, yet the neural systems involved in reinstating these unique types of information remain unclear. Here, combining computational modeling and intracranial recordings from 69 epilepsy patients, we show that two cortical systems uniquely reinstate the semantic content and temporal context of previously studied items during free recall. Examining either the posterior medial or anterior temporal networks, we find that forward encoding models trained on the brain’s response to the temporal and semantic attributes of items can predict the serial position and semantic category of unseen items. During memory recall, these models uniquely link reinstatement of temporal context and semantic content to these posterior and anterior networks, respectively. These findings demonstrate how specialized cortical systems enable the human brain to target specific memories.

Introduction

Episodic recall allows us to remember the past, bringing back memories from a specific place or time. This type of memory retrieval involves the reinstatement of encoding-related neuronal activity that codes for memory attributes1 (e.g., a specific person2 or place3). Neural reinstatement has been proposed as a mechanism for targeting individual memories during memory search4,5, and subsequent studies have demonstrated content2,6,7 and context3,8 reinstatement preceding memory recall. Thus, these two types of reinstatement may serve to target memories that contain certain content or are placed in specific contexts. However, because studies typically focus on either content or context reinstatement in isolation, we know relatively little about how these types of information reinstate across the brain.

Consistent with the longstanding distinction between episodic and semantic memory9 neuroimaging and electrophysiological studies suggest that separable cortical systems support memory for the semantic meaning and the spatiotemporal context of experienced events10,11,12,13,14. Specifically, researchers have identified a posterior medial (PM) network of regions, including parahippocampal, retrosplenial, and posterior parietal cortices, that activate during the processing of contextual information15,16. By contrast, other work has identified an anterior temporal (AT) network of regions that appears critical for semantic and conceptual memory17,18. This network includes the ventral temporal pole and perirhinal cortices19. Because these two networks interact with the hippocampus during memory formation and retrieval14,20, we predicted reinstatement within these two systems would reflect either the content or context of retrieved memories.

Neuroimaging studies that measure population-level neuronal activity have demonstrated reinstatement across cortical systems during memory retrieval. For example, content-related patterns of spectral power from intracranial electroencephalography (iEEG) across prefrontal and temporal cortices reinstate prior to memory recall6,21. Slowly changing patterns of iEEG power in the temporal lobe, consistent with a representation of temporal context5, reinstate during retrieval and account for temporal organization of recall sequences8. Functional MRI studies, which provide greater sampling of cortical systems than invasive recording techniques, have demonstrated contextual reinstatement within the hippocampus22 and functionally connected PM regions23. Multiple regions within the PM network, including the angular gyrus and medial prefrontal cortex, reinstate event-specific patterns of activity24, suggesting this network may be involved in representing the content of memories within a specific episodic context. It is possible that these findings stem from representations of content and context integrated within a single cortical system, coding the full range of attributes in memory. In contrast, the PM and AT networks may independently drive hippocampal-dependent retrieval, serving as distinct cortical pathways to recall.

To examine the contributions of cortico-hippocampal networks to recall behavior, we utilized a computational modeling technique originally developed to predict patterns of brain activity based on the semantic content of individual stimuli25,26,27. This method takes advantage of the sensitivity of neural signals to semantic attributes of presented items. By learning how activity in the brain is shaped by semantic attributes, these models can reliably decode the semantic content of stimuli from an observed pattern of brain activity. We extended this technique to develop context-based models that were trained to predict patterns of brain activity based on the temporal context (i.e., at which point in time in the experiment) in which stimuli were presented. By applying these models to iEEG signals recorded while subjects performed a free-recall task, we tested whether content- and context-based memory representations are reinstated within distinct cortical systems: the AT and PM networks.

Results

We recorded iEEG from neurosurgical patients (n = 69) while they performed a free-recall task with items drawn from 25 distinct categories presented in same-category pairs (Fig. 1a). Patients recalled an average of 31% (0.02 SEM) of list items. Both the serial position and category of items influenced their probability of recall. We observed a primacy effect, as evidenced by a recall advantage for items presented in the first (mean 49%, 0.03 SEM; t68 = 9.2, p < 0.0001) and second (mean 38%, 0.03 SEM; t68 = 4.9, p < 0.0001) serial positions (Fig. 1b). To evaluate category-level differences in recall performance, we compared the proportion of items recalled from each category to the average across all items (Fig. 1c). We found patients had better memory for zoo animals (mean 42%, 0.03 SEM; t68 = 4.1, p = 0.0001) and weather (mean 39%, 0.02 SEM; t68 = 3.3, p = 0.001), and worse memory for electronics (mean 22%, 0.02 SEM; t68 = −4.4, p < 0.0001).

In addition to overall recall performance, the serial position and category of items influenced the order in which items were recalled. Recall clustering (e.g., consecutively recalling same-category items) indicates that the retrieval cue used during memory search targets certain properties of the studied items. Because no retrieval cues are provided to the subject during free recall, the cue must be internally generated by the reinstatement of information present during the study period. To examine category clustering, we used a word embedding model28 to derive a vector representation of each studied item. To illustrate the similarity structure derived from the word2vec model, we projected the 300-dimensional word representations onto a three-dimensional space derived using principal component analysis (Fig. 1a, right). We measured clustering with temporal and categorical factor scores, which indicate whether clustering is above or below chance levels29(see Fig. 1d for example measures). Recall sequences exhibited both categorical (t68 = 30.7, p < 0.0001) and temporal (t68 = 10.8, p < 0.0001) clustering (Fig. 1e). These clustering effects remained after accounting for potential confounds due to the list structure (see Methods). Adjusted measures of categorical (t68 = 25.6, p < 0.0001) and temporal (t68 = 11.4, p < 0.0001) clustering also indicated significant levels of recall organization (Supplementary Fig. 2). Both recall performance and clustering demonstrated reinstatement of both semantic content and temporal context occurred during memory search.

PM and AT networks contain representations of context and content during encoding

To assess representations of temporal context and semantic content within PM and AT networks, we first defined these networks based on an independent resting-state fMRI analysis (Fig. 2a; see Methods for details)30. This analysis follows the standard method of correlating the BOLD time series between a region of interest and other regions distributed throughout the rest of the brain31. Regions with connectivity to temporal polar cortex defined the AT network. These regions included inferior lateral prefrontal cortex, anterior temporal cortex, and inferior angular gyrus (Fig. 2a, green). Regions with connectivity to the posterior angular gyrus defined the PM network (Fig. 2a, purple). Cortical regions within this network included the posterior cingulate cortex, precuneus, parahippocampal cortex, and posterior parietal cortex (see Supplementary Fig. 1 for details regarding electrode coverage).

Synchronous low-frequency activity across the brain signals memory processing32, with theta oscillations responsible for interactions between hippocampus and cortex during memory search33. Prominent theories propose synchronous interactions coordinate network communication34, which would support hippocampal-dependent reinstatement of memory content. Thus, we predicted synchronous activity in both PM and AT networks during the free-recall task. We next tested the hypothesis that PM and AT networks defined by resting-state fMRI would predict correlations in low-frequency power during task performance. Previous work has established correspondence between networks defined by resting-state fMRI and iEEG35,36. However, it is possible that different network structures emerged in iEEG signals during the recall task. For example, these two cortical systems could reconfigure into a single network. We predicted distinct networks to emerge as subjects perform the task, supporting the hypothesis that separable cortical systems are involved in the representation of content and context.

To examine the separability of PM and AT networks, we asked whether theta exhibited greater connectivity within than between the two networks. We measured connectivity by correlating spectral power across trials of the free-recall task. Such trial-by-trial variability in power reflects endogenous fluctuations across neural networks35,37,38. Because these estimates of connectivity can be spuriously elevated due to proximity of recording sites36 (see Supplementary Fig. 3 for evidence of drop off in connectivity with inter-electrode distance), we used a bootstrap matching procedure to control for differences in distance between contacts located within the same network or across both networks (see Methods). Using this approach, low-frequency power was significantly more correlated within the AT network than between the PM and AT network. This effect held at the lower (3 Hz, t64 = 4.97, p < 0.0001), intermediate (5 Hz, t64 = 3.67, p < 0.0001), and upper range of theta (10 Hz, t64 = 3.12, p < 0.0001). Within the PM network, we found evidence for synchronous activity near the upper end of theta (10 Hz, t(64) = 2.13, p = 0.04). We also explored whether network-specific coupling occurred at higher frequencies, spanning beta and gamma ranges (frequencies surviving FDR corrected thresholds are highlighted in Fig. 2a, bottom). We did not identify any higher frequency effects within the AT network. Coupling in the low beta range was enhanced within the PM network (17 Hz, t64 = 2.70, p = 0.009). These results show that changes in spectral power were coherent within each of these two networks, consistent with their identification as dissociable networks from resting-state fMRI.

Having validated the PM and AT networks in our iEEG recordings, we next sought to evaluate their respective roles in the reinstatement of memories’ semantic content and temporal context. To do this, we trained multivariate models to predict brain activity based on the semantic content and temporal context of items presented at encoding. We modeled changes in spectral power from 3 to 180 Hz at each recording site from either the content- or context-based attributes of each item (Fig. 2b). Using these models, we predicted the pattern of spectral power for each serial position or category. We compared these predictions to held-out items in a cross-validation procedure that tested whether it was possible to decode either the serial position or category from brain activity observed within a given network. In addition, applying these models to patterns of spectral power during memory search allowed us to compare reinstatement across the two networks.

We assessed each model’s decoding performance during the encoding phase of the experiment, as subjects studied list items. We used a bootstrap procedure (n = 1000) to estimate decoding performance from five randomly sampled recording sites within each network (this approach allowed us to control for greater electrode coverage in the AT network, see Supplementary Fig. 1b and Methods for further details). Applying this technique, we reliably decoded the category of items in both the PM (median AUC = 0.54, Z = 6.9, p < 0.0001) and AT (median AUC = 0.54, Z = 7.1, p < 0.0001) networks (Fig. 2c), and category decoding did not differ between the two networks (median ΔAUC = 0.002, Z = 0.8, p = 0.45). We also reliably decoded the serial position of items in both the PM (median AUC = 0.71, Z = 7.2, p < 0.0001) and AT (median AUC = 0.70, Z = 7.2, p < 0.0001) networks (Fig. 2d). Here, our serial position model achieved more accurate decoding in the PM than in the AT network (median ΔAUC = 0.002, Z = 2.1, p = 0.04). These bootstrap results provide a lower bound for the true decoding performance for a given network (see Supplementary Fig. 4) and confirm the representation of semantic and temporal information across the PM and AT networks during encoding.

Reinstatement of content and context in PM and AT networks

Having established the ability to decode temporal context (serial position) and semantic content (category) from encoding-period neural activity in the PM and AT networks, we next asked whether these two types of information reinstated during free recall. To measure reinstatement, we tested our category and serial position models on patterns of neural activity in the moments preceding recall (from 900 to 100 msec prior to vocalization, in 20 msec intervals; Fig. 3a). We observed reinstatement of temporal information in the PM network and semantic information in both the PM and AT networks. Temporal context reinstated in the PM network from 600 to 400 msec before overt recall (Fig. 3b, top panel). Content reinstatement was sustained throughout the pre-recall period (Fig. 3b, bottom panel) in both PM and AT networks. Moreover, these two forms of reinstatement differed between the two networks, with greater content reinstatement in the AT network (Fig. 3b; p < 0.05, FWER corrected). These findings demonstrate the specificity of contextual reinstatement within the PM network. In contrast to previous studies that relied upon similarity in neural activity over time to identify contextual information8,22,23,39,40, we demonstrate the ability to recover the serial position of recalled items from reinstatement in the PM network. We also show greater, sustained content reinstatement within the AT network. These unique patterns of reinstatement reveal a dissociation between content and context reinstatement across the PM and AT networks.

Our finding that reinstatement of temporal context is specific to the PM network relies upon chance-level decoding within the AT network. It is possible that by limiting the number of electrodes included in this analysis (which was necessary for comparisons between networks), we impaired our ability to detect reinstatement, particularly if the information was represented in a distributed manner. To evaluate this possibility, we examined decoding performance after incorporating additional electrodes into our analysis (Supplementary Fig. 4). Adding electrodes produced more robust category reinstatement effects in both the PM ($${\chi }_{1}^{2}=5.7$$, p = 0.02) and AT ($${\chi }_{1}^{2}=31.4$$, p < 0.0001) networks. This analysis also revealed stronger context reinstatement in the PM ($${\chi }_{1}^{2}=6.7$$, p = 0.009) but not the AT ($${\chi }_{1}^{2}=0.8$$, p = 0.36) network when incorporating additional features. These findings provide additional support for the specificity of context reinstatement to the PM network and suggest that decoding accuracy was limited by anatomical coverage and the spatial resolution of recordings.

To rule out the possibility that epileptiform activity influenced our ability to detect reinstatement, we examined whether electrodes that were either within the seizure onset zone, exhibited inter-ictal spiking, or located near pathological tissue impacted decoding performance. We repeated our decoding analyses after excluding these electrodes and found a remarkably similar pattern of results (Supplementary Fig. 5). We found no evidence that signals from these electrodes impacted our ability to decode information at encoding (all $${\chi }_{1}^{2}\, < \,0.25$$, $$p^{\prime} s\, > \, 0.61$$) or recall (all $${\chi }_{1}^{2}\, < \,0.43$$, $$p^{\prime} s\, > \,0.53$$). As such, reinstatement reflected changes in spectral content associated with physiologically and cognitively normal processes.

Cortical reinstatement has been shown to predict how memory search unfolds4,21, with cortical representations providing a top-down cue for the memory system to retrieve specific information. As such, it is possible that the PM and AT networks are responsible for targeting stored memories with specific content- or context-based attributes. To test this hypothesis, we examined whether differences in reinstatement across the two networks predicted the tendency of subjects to organize their recall sequences along temporal or categorical dimensions. Consistent with prior work linking cortical reinstatement to memory organization6,8, variability in the organization of a subject’s memory was predicted by cortical reinstatement during memory search (Fig. 3c). Greater reinstatement of context-based representations within the PM network tracked the tendency to consecutively recall items from nearby serial positions (r67 = 0.26, p = 0.028). Increased reinstatement within the AT network was associated with greater organization based on the categorical structure of the list (r67 = −0.35, p = 0.003). Representations encoded in these two cortical systems can differentially guide memory search, biasing how and what we remember.

Discussion

We report evidence that the reinstatement of content- and context-based information occurs within distinct cortical networks. Furthermore, the quality of reinstatement within a given network was predictive of a subject’s tendency to organize their memories along either temporal or categorical dimensions. These results help resolve conflicting viewpoints on how distinct neural representations contribute to memory search. Episodic retrieval occurs when a memory cue converges on the hippocampal formation, prompting associative recall41. Multiple cortical structures have been proposed to represent temporal context, which cues episodic recall42, including lateral prefrontal cortex5 and parahippocampal inputs to the hippocampus43. Despite evidence implicating these structures in representing temporal information8,22,44,45,46,47, it has been unclear whether these brain regions are the primary drivers of episodic recall. Our findings of greater reinstatement of context within the PM network and content within the AT network resolve this ambiguity by characterizing the contributions of these cortico-hippocampal networks to memory search.

Our findings complement previous studies of the neural bases of memory search6,8,21 that have linked neural reinstatement to the organization of human memory. Along with fMRI investigations of temporal coding in the MTL and cortico-hippocampal networks22,23, these previous studies identified candidate regions for the representation of temporal context based on slow changes in the similarity of neural patterns over time. Here, our modeling approach enabled us to recover the temporal position of items within each list, irrespective of the list in which the item was encoded. By doing so, we rule out the possibility that context-like signals emerge from the covert retrieval or maintenance of list items throughout study lists, an inherent limitation of prior work. Given the PM network specifically reinstates contextual codes that predict the temporal organization of recall sequences, neuronal interactions within this network support the ability to target memories from specific episodic contexts.

The observed temporal coding in the PM network parallels the firing of hippocampal neurons in rodents47 and so-called “time cells” in humans48,49. These neurons show selective increases in firing at specific temporal positions, providing a basis for sequence learning and recall. Emergence of this hippocampal code likely depends on two types of time-modulated inputs from the entorhinal cortex50. Neuronal firing within the lateral entorhinal cortex responds to salient stimuli in the environment and decays at rates sufficient to represent ordinal sequences51. The hippocampus also receives inputs from grid cells in the medial entorhinal cortex, which displays coupling to the PM network via parahippocampal cortex52,53. These neurons code for an internal representation of continuous time, as they fire at specific temporal intervals during treadmill running54 and periods of immobility55 when changes in the environment cannot provide temporal information. Although we did not directly relate time cell activity to cortical reinstatement, our findings suggest temporal coding in the medial entorhinal cortex may be crucial to reinstatement in the PM network.

Because the present findings are inherently correlational in nature, future studies could provide causal evidence between network reinstatement and recall organization via noninvasive or direct electrical stimulation. Adaptation of recently developed closed-loop stimulation techniques56,57 may provide a framework to modulate hippocampal activity and bias recall of information from a specific context.

Unlike reinstatement of contextual information that was specific to the PM network, we were able to decode the category of recalled items from patterns of activity in both networks. Converging evidence from fMRI studies suggests that regions within this network58, including the angular gyrus24, represent recollected content in an episode-specific manner. Our findings suggest that temporal coding may be a unique feature of this network, in contrast to representations coded in the AT network. Understanding the nature of these temporal representations, including where they originate and how they are integrated with other forms of episodic information remains to be determined.

Our ability to decode the content and context of items in memory reflects multiple neuronal processes. Specifically, the measures of spectral power we examined reflect a combination of broadband and oscillatory processes, including slower (2–4 Hz) and faster (5–10 Hz) theta rhythms that predominate hippocampal networks59. Broadband shifts in power indicate when memory processing occurs, including concurrent decreases in low frequency and increases in high-frequency content60,61,62. These broadband shifts are associated with changes in neuronal excitability and correlate with firing rates of locally recorded neurons63,64. Thus, increases in neuronal firing within attribute-specific cortical regions would support accurate decoding. At the same time, reduction of oscillations and increased asynchronous activity supports information coding within cortex65,66. Changes in the amplitude of narrowband oscillations would therefore contribute to decoding performance, at timescales constrained by the speed of the oscillation. Understanding the relative contributions of oscillatory and broadband signals to memory reinstatement remains an important question for future research.

Neural models of memory search have suggested that an internal representation of context serves as the primary cue for hippocampal-dependent recall5. Our findings argue for an alternative account regarding the neural basis of memory search. One possibility is that semantic representations within the AT system can cue memories in a context-independent manner, prompting recall of memories with similar semantic attributes. If content-based representations can guide memory search in this fashion, one would expect individuals to organize their memories based on semantic content, rather than the temporal order in which it was learned. Indeed, subjects who showed greater reinstatement in the AT network exhibited greater semantic and less temporal organization. This hypothesis is further supported by evidence that content-based activity in the inferior temporal cortex can be used as a top-down signal to bias retrieval to targeted memories7. Along these lines, the contextual information represented across the PM network focuses retrieval to a specific temporal context. The effects observed here may reflect a general property of memory search, wherein cortical reinstatement serves as a mechanism to target memories based on network-specific representations.

Methods

Participants

Sixty-nine patients (40 male, with an average age of 39 years [SD 12 years]) with medication-resistant epilepsy underwent neurosurgical procedures to implant intracranial electrodes (subdural, depth, or both) to determine epileptogenic regions. On average, the duration of epilepsy at the time of surgery was 19 years (SD 13 years). Seizure onset zones were commonly localized to mesial temporal (n = 25), temporal (n = 22), prefrontal (n = 13), parietal (n = 28), and occipital (n = 7) cortices. An additional two patients had seizure onset zones within the hippocampus. Seizure onset zones were lateralized to the left (n = 31), right (n = 23), or both (n = 11) hemispheres. Data were collected at Dartmouth-Hitchcock Medical Center (Hanover, NH), Emory University Hospital (Atlanta, Georgia), Hospital of the University of Pennsylvania (Philadelphia, PA), Mayo Clinic (Rochester, MN), Thomas Jefferson University Hospital (Philadelphia, PA), Columbia University Medial Center (New York, NY), and University of Texas Southwestern Medical Center (Dallas, TX). Prior to data collection, the research protocol was approved by the Institutional Review Board at each hospital. Informed written consent was obtained from either the participant or their guardians.

Each subject performed a categorized free-recall task in which they studied a list of words with the intention to commit the items to memory. The task was performed at the bedside on a laptop, using PyEPL software67. Analog pulses were sent to available recording channels to enable alignment of experimental events with the recorded iEEG signal. Word presentation lasted for a duration of 1600 ms, followed by a blank inter-stimulus interval (ISI) of 750–1000 ms (see Fig. 1a). Each list contained items from three distinct categories (four items per category), with two same-category items presented consecutively. The total word pool consisted of 25 distinct categories, with individual items selected as prototypical items within each category68. Presentation of word lists was followed by a 20 s post-encoding delay. Subjects performed an arithmetic task during the delay in order to disrupt memory for end-of-list items. Math problems of the form A + B + C = ?? were presented to the participant, with values of A, B, and C set to random single-digit integers. After the delay, a row of asterisks, accompanied by an 800 Hz auditory tone, was presented for a duration of 300 ms to signal the start of the recall period. Subjects were instructed to recall as many words as possible from the most recent list, in any order during the 30 s recall period. Vocal responses were digitally recorded and parsed offline using Penn TotalRecall (http://memory.psych.upenn.edu/TotalRecall). Subjects performed up to 25 lists in a single recall session.

Behavioral analysis

To compute behavioral measures of temporal clustering, we used the temporal factor29 score. Temporal factor measures the percentile rank of the absolute lag between successive recalls from the full distribution of available lags for items that have yet to be recalled. To measure category clustering, we computed a category factor, which assumes that all items within the same category have a distance of zero and items of different categories have a distance of one. These metrics measure the degree to which recall sequences exhibit organization along temporal or categorical dimensions, with random recall sequences falling at the median of the distribution (i.e., 0.5).

We presented same category exemplars in sequential pairs, as in the sequence (A1, A2, B1, B2, C1, C2, B3, B4, …, A4) where Xi is an exemplar of category X. This list structure creates circumstances where random recall sequences produce factor scores that deviate from the expected value of 0.5. Consider the case where a subject recalls four items from a single category in an arbitrary temporal order. After recalling the first item, the subject could either recall the same-category pair (e.g., A2 following A1) or jump to one of the other same-category items (A3 or A4). Because more items are available for recall at long temporal lags, sequences generated with random temporal order produce a lower than expected temporal factor score of 0.45. Measures of category clustering can be similarly biased by the list structure. Consider the case where a subject serially recalls four items starting at an arbitrary position in the list, without regard to semantic information. Because same-category pairs are always recalled consecutively, the expected category factor score for these recall sequences is elevated to 0.64. These examples highlight that factor scores can deviate from 0.5 due to the list structure rather than true recall clustering. To rule out this potential confound, we performed a simulation-based control analysis where we generated null distributions from random recall sequences matched to both the number of items recalled and either the temporal or categorical clustering of the observed sequences. We standardized the observed clustering measures based on these null distributions (n = 1000), which indicate the amount of clustering expected by chance given the list structure. This analysis is sensitive to clustering that is not confounded by the list structure, such as temporal clustering across category boundaries and category clustering across large temporal lags.

Electrophysiological recordings and data processing

iEEG signal was recorded using subdural grids and strips (contacts spaced 10 mm apart) or depth electrodes (contacts spaced 3–10 mm apart) using recording systems at each clinical site. iEEG systems included DeltaMed & XlTek (Natus), Grass Telefactor, and Nihon-Kohden EEG systems. Signals were sampled at 500, 512, 1000, 1024, or 2000 Hz, depending on clinical site. Preprocessing of iEEG signal was performed with custom Python (v3.3) software. Signals recorded at individual contacts were converted to a bipolar montage by computing the difference in signal between adjacent electrode pairs on each strip, grid, and depth electrode. Bipolar signal was notch filtered at 60 Hz with a fourth-order 2 Hz stop-band Butterworth notch filter in order to remove the effects of line noise on the iEEG signal.

Anatomical localization

Anatomical localization of electrode placement was accomplished using independent processing pipelines for depth and surface electrodes. Post-implant CT images were coregistered with presurgical T1 and T2 weighted structural scans using Advanced Normalization Tools69. For patients with MTL depth electrodes, hippocampal subfields and MTL cortices were automatically labeled in a pre-implant, T2-weighted MRI using the automatic segmentation of hippocampal subfields multi-atlas segmentation method70. Subdural electrodes were localized by reconstructing whole-brain cortical surfaces from pre-implant T1-weighted MRIs using Freesurfer71, and snapping electrode centroids to the cortical surface using an energy minimization algorithm72. Reconstructed surfaces were additionally mapped to a population-average surface73 that we used to assign network membership based on resting-state connectivity of cortical regions defined by a multi-modal cortical parcellation30. PySurfer (https://pysurfer.github.io/) was used to display electrode coverage and statistics on the cortical surface.

Network assignment and analysis

We assigned recording sites to networks of interest based on resting-state functional connectivity in an independent set of subjects from the Human Connectome Project30. From the Glasser et al. parcellation, we assigned parcels with high connectivity to the posterior angular gyrus (area PGp) to the PM network and parcels with high connectivity to the temporal pole (area TGd) to the AT network. The assignment was based on partial correlations between each parcel and the seed region, to provide a better estimate of direct network connections. Bipolar pairs were assigned to the nearest network if the bipolar centroid was within 8 mm of a parcel within either network, with the exclusion criteria that they could not be within 4 mm of the other network.

To evaluate the properties of these two cortical networks, we used a bootstrap sampling procedure to randomly sample from bipolar pairs within each network. This approach controls for the effect of distance between recording sites on functional connectivity. As the inter-electrode distance is greater between compared to  within functional networks (leading to a biased estimate of connectivity within each network), we controlled the distances within each distribution of connections (i.e., within AT, within PM, or between networks). For each subject, we randomly sampled connections between the two networks. Connections within a given network were sampled without replacement to match the distribution between network connections by minimizing the differences in connection length. The functional connectivity within and between each network was computed as the Pearson product–moment correlation across encoding events in the experiment. This procedure was repeated 1000 times, and the intrinsic connectivity at a given frequency was estimated from the bootstrap distribution. Some subjects were excluded from analysis (n = 4) in circumstances where electrode coverage prevented well-matched samples across conditions.

A similar bootstrap procedure was used to estimate the decoding accuracy of models trained from each network. Networks with greater electrode coverage are likely to have higher decoding accuracy due to the number of features alone. As a result, we randomly sampled five electrodes from each network prior to estimating the ability to decode content and context information from patterns of brain activity. This sampling procedure was repeated 1000 times per model evaluation, and the average performance across bootstrap distributions was used to indicate model performance for a given subject.

Spectral power

To compute spectral power during word encoding, we applied the Morlet wavelet transform (wave number 5) to all bipolar electrode EEG signals from the onset to the offset of stimulus presentation, across eight logarithmically spaced frequencies from 3 to 180 Hz. Spectral power during recall was estimated from 900 ms to 100 ms preceding the onset of response vocalization for correct recalls. Recall events were required to be free of vocalization onsets in the preceding 1500 ms. Power estimates were log-transformed and down sampled to 50 Hz. To avoid edge artifacts, we included buffers of 1000 ms surrounding events of interest during the computation of spectral power; mirrored buffering was applied to all retrieval-period data. Prior to modeling, all power estimates were standardized using the mean and standard deviation of each session.

Model fitting and testing

We used custom MATLAB code (version R2017b; Natick, MA) to construct two models to predict stimulus-related patterns of neural activity as a function of either (1) the taxonomic category of presented stimuli or (2) the serial position of each presented item within the experiment. For the content model, we modeled the neural response as a function of 300 intermediate category features computed using the word2vec model28 trained from Google News corpora. Category features were computed by averaging semantic representations across all words presented from a given category. This results in the construction of a high-dimensional space that respects the semantic relationships between all of the categories. For the context model, we modeled the neural response to stimuli as a function of serial position within the list, across lists, and across sessions. For subjects in which only a single session was run, the regressor predicting the effect of session on neural activity was excluded.

We fit both the content and context models separately to each neural feature (i.e., spectral power at a given frequency and bipolar pair). Ridge regression was used to identify model parameters that minimized prediction error on the training data, which was randomly assigned using a 5-fold cross-validation procedure, holding out individual words. Within each training fold, we performed an additional 10-fold cross-validation procedure (i.e., nested cross-validation74) to optimize the regularization coefficient used in each fold. Across the ten folds, we selected the regularization coefficient (from 50 potential parameters log-spaced from 10−2 to 1010) that minimized the mean squared error of the model predictions across the training set. The resulting set of model weights across neural features was used to decode either the serial position or category membership for unseen patterns of brain activity. We computed the probability that the observed brain data belonged to each of the 25 categories (or 12 serial positions) by computing the Pearson product–moment correlation between the observed data and the predicted pattern of brain activity for each model. A softmax function was applied to the resultant evidence for each class, and decoding accuracy was measured using area under the receiver operating characteristic curve (AUC) metric across all held-out data.

To quantify reinstatement effects across each network, we applied models trained to predict patterns of brain activity during word encoding to epochs just prior to recall (from 900 ms to 100 ms before vocalization onset). For each sample within this window, we computed the ability of each model to decode either the category or serial position of recalled words in the validation data. The resulting AUC time series were smoothed with a 7 ms FWHM Gaussian kernel prior to statistical analysis for noise reduction.

Statistics and reproducibility

One-tailed tests were used to assess differences versus chance performance in evaluating decoding accuracy and recall organization. Theoretical chance levels (e.g., an AUC of 0.5 one class vs. all others) were further tested by constructing null distributions via permutation of class labels (i.e., category or serial position).

To evaluate the effects of the number of features and potential epileptiform activity on decoding performance, we modeled classifier performance using linear mixed-effects models. Fixed effects included the number of electrodes sampled, and whether the electrode was localized to the seizure onset zone. Intercepts were allowed to vary, treating the subject as a random variable. Inference was performed through model comparison using a likelihood ratio test, dropping the effect of interest. Model fits and normality of residuals were confirmed through visual inspection.

We corrected for multiple comparisons (across time and frequencies) using a nonparametric permutation procedure. In our analysis of intrinsic connectivity, we performed a nonparametric one-sample t test by constructing a null distribution of the maximum t statistic across frequencies, assuming no difference in intrinsic connectivity for connections within and between networks. This assumption was satisfied by random sign flipping of observed values at the subject level. The significance of observed differences in connectivity strength within versus between networks was compared to a distribution constructed from 2000 random permutations, yielding two-tailed significance with PFWE < 0.05.

Multiple comparison correction for network reinstatement followed a similar procedure. We identified significant (PFWE < 0.05) clusters in the moments leading up to recall using threshold-free cluster enhancement (TFCE)75. The TCFE statistic was computed by taking the original test statistics over the pre-recall period and adjusting by weighting by the height (h) and cluster extent (e):

$${\sum }_{k}{h}^{H}e{({h}_{k})}^{E}dh,$$
(1)

where hk is one of k cluster forming thresholds (hk = h0, h + dh, . . . , hmax). Height (H) and extent (E) exponents were set to 2 and 0.5 respectively, with the step size in the cluster threshold (dh) set to 0.01. After computing TCFE statistics, we used nonparametric t tests to identify significant clusters based on null distributions (n = 10000).

Given the limited availability of iEEG data, no statistical methods were used to predetermine sample sizes (number of subjects). The number of trials included in the experiment was determined by practical constraints of patient testing at epilepsy monitoring units. The subjects in this study were a subset of those included in a multi-site collaboration to investigate the modulation of human memory via direct electrical stimulation. Subjects with at least 5 recording sites located within both the PM and AT networks who performed categorized free recall were included in the present study. All experiments and analyses were performed once, without replication in independent cohorts.

Reporting summary

Further information on research design is available in the Nature Research Reporting Summary linked to this article.

Data availability

De-identified data are available at http://memory.psych.upenn.edu/Electrophysiological_DataSource data are provided with this paper.

Code availability

Analysis code for model fitting and evaluation are available at http://memory.psych.upenn.edu/files/pubs/KragEtal21.code.tgz.

References

1. 1.

Danker, J. F. & Anderson, J. R. The ghosts of brain states past: remembering reactivates the brain regions engaged during encoding. Psychol. Bull. 136, 87 (2010).

2. 2.

Gelbard-Sagiv, H., Mukamel, R., Harel, M., Malach, R. & Fried, I. Internally generated reactivation of single neurons in human hippocampus during free recall. Science 322, 96–101 (2008).

3. 3.

Miller, J. F. et al. Neural activity in human hippocampal formation reveals the spatial context of retrieved memories. Science 342, 1111–1114 (2013).

4. 4.

Polyn, S. M., Natu, V. S., Cohen, J. D. & Norman, K. A. Category-specific cortical activity precedes retrieval during memory search. Science 310, 1963–1966 (2005).

5. 5.

Polyn, S. M. & Kahana, M. J. Memory search and the neural representation of context. Trends in Cogn. Sci. 12, 24–30 (2008).

6. 6.

Manning, J. R., Sperling, M. R., Sharan, A., Rosenberg, E. A. & Kahana, M. J. Spontaneously reactivated patterns in frontal and temporal lobe predict semantic clustering during memory search. J. of Neurosci. 32, 8871–8878 (2012).

7. 7.

Norman, Y., Yeagle, E. M., Harel, M., Mehta, A. D. & Malach, R. Neuronal baseline shifts underlying boundary setting during free recall. Nat. Commun. 8, 1301 (2017).

8. 8.

Manning, J. R., Polyn, S. M., Baltuch, G. H., Litt, B. & Kahana, M. J. Oscillatory patterns in temporal lobe reveal context reinstatement during memory search. Proc. of the Natl. Acad. of Sci. 108, 12893–12897 (2011).

9. 9.

E. Tulving. Episodic and semantic memory. In Organization of Memory (eds. Tulving, E. and Donaldson, W.), p. 381–403 (Academic Press, New York, 1972).

10. 10.

Squire, L. R. & Zola-Morgan, S. The medial temporal lobe memory system. Science 253, 1380–1386 (1991).

11. 11.

Eichenbaum, H., Yonelinas, A. P. & Ranganath, C. The medial temporal lobe and recognition memory. Ann. Rev. of Neurosc. 30, 123–152 (2007).

12. 12.

Davachi, L. Item, context and relational episodic encoding in humans. Curr. Opin. in Neurobiol. 16, 693–700 (2006).

13. 13.

Bird, C. M. & Burgess, N. The hippocampus and memory: insights from spatial processing. Nat. Revi. Neurosci. 9, 182 (2008).

14. 14.

Ranganath, C. & Ritchey, M. Two cortical systems for memory-guided behaviour. Nat. Rev. Neurosci. 13, 713 (2012).

15. 15.

Vilberg, K. L. & Rugg, M. D. Memory retrieval and the parietal cortex: a review of evidence from a dual-process perspective. Neuropsychologia 46, 1787–1799 (2008).

16. 16.

Barnett, A. J. et al. Intrinsic connectivity reveals functionally distinct cortico-hippocampal networks in the human brain. PLOS Biology 19, e3001275 (2021).

17. 17.

Binder, J. R., Desai, R. H., Graves, W. W. & Conant, L. L. Where is the semantic system? a critical review and meta-analysis of 120 functional neuroimaging studies. Cereb. Cortex 19, 2767–2796 (2009).

18. 18.

Patterson, K., Nestor, P. J. & Rogers, T. T. Where do you know what you know? the representation of semantic knowledge in the human brain. Nat. Rev. Neurosci. 8, 976 (2007).

19. 19.

Clarke, A. & Tyler, L. K. Object-specific semantic coding in human perirhinal cortex. J. of Neurosci. 34, 4766–4775 (2014).

20. 20.

Ritchey, M., Libby, L. A. & Ranganath, C. Cortico-hippocampal systems involved in memory and cognition: the pmat framework. In Progress in Brain Research, volume 219, pages 45–64. Elsevier, 2015. https://doi.org/10.1016/bs.pbr.2015.04.001.

21. 21.

Morton, N. W. et al. Category-specific neural oscillations predict recall organization during memory search. Cereb. Cortex 23, 2407–2402 (2013).

22. 22.

Hsieh, L.-T., Gruber, M. J., Jenkins, L. J. & Ranganath, C. Hippocampal activity patterns carry information about objects in temporal context. Neuron 81, 1165–1178 (2014).

23. 23.

Hsieh, L.-T. & Ranganath, C. Cortical and subcortical contributions to sequence retrieval: schematic coding of temporal context in the neocortical recollection network. NeuroImage 121, 78–90 (2015).

24. 24.

Kuhl, B. A. & Chun, M. M. Successful remembering elicits event-specific activity patterns in lateral parietal cortex. J. of Neurosci. 34, 8051–8060 (2014).

25. 25.

Mitchell, T. M. et al. Predicting human brain activity associated with the meanings of nouns. Science 320, 1191–1195 (2008).

26. 26.

Huth, A. G., Nishimoto, S., Vu, A. T. & Gallant, J. L. A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76, 1210–1224 (2012).

27. 27.

Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E. & Gallant, J. L. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature 532, 453 (2016).

28. 28.

Mikolov, T., Chen, K., Corrado, G. & Dean, J. Efficient estimation of word representations in vector space. (2013) Preprint at https://arxiv.org/abs/1301.3781v3.

29. 29.

Polyn, S. M., Norman, K. A. & Kahana, M. J. A context maintenance and retrieval model of organizational processes in free recall. Psychol. Rev. 116, 129 (2009).

30. 30.

Glasser, M. F. et al. A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178 (2016).

31. 31.

Deco, G., Jirsa, V. K. & McIntosh, A. R. Emerging concepts for the dynamical organization of resting-state activity in the brain. Nat. Rev. Neurosci. 12, 43–56 (2011).

32. 32.

Solomon, E. A. et al. Widespread theta synchrony and high-frequency desynchronization underlies enhanced cognition. Nat. Commun. 8, 1–14 (2017).

33. 33.

Solomon, E. A., Lega, B. C., Sperling, M. R. & Kahana, M. J. Hippocampal theta codes for distances in semantic and temporal spaces. Proc. of the Natl. Acad. of Sci. 116, 24343–24352 (2019).

34. 34.

Bastos, A. M., Vezoli, J. & Fries, P. Communication through coherence with inter-areal delays. Curr. Opin. in Neurobiol. 31, 173–180 (2015).

35. 35.

Foster, B. L., Rangarajan, V., Shirer, W. R. & Parvizi, J. Intrinsic and task-dependent coupling of neuronal population activity in human parietal cortex. Neuron 86, 578–590 (2015).

36. 36.

Hacker, C. D., Snyder, A. Z., Pahwa, M., Corbetta, M. & Leuthardt, E. C. Frequency-specific electrophysiologic correlates of resting state fmri networks. NeuroImage 149, 446–457 (2017).

37. 37.

Lewis, C. M., Bosman, C. A., Womelsdorf, T. & Fries, P. Stimulus-induced visual cortical networks are recapitulated by spontaneous local and interareal synchronization. Proc. of the Natl. Acad. of Sci. 113, E606–E615 (2016).

38. 38.

Daitch, A. L. et al. Mapping human temporal and parietal neuronal population activity and functional coupling during mathematical cognition. Proc. of the Natl. Acad. of Sci. 113, E7277–E7286 (2016).

39. 39.

Yaffe, R. B. et al. Reinstatement of distributed cortical oscillations occurs with precise spatiotemporal dynamics during successful memory retrieval. Proc. of the Natl. Acad. of Sci. 111, 18727–18732 (2014).

40. 40.

Folkerts, S., Rutishauser, U. & Howard, M. W. Human episodic memory retrieval is accompanied by a neural contiguity effect. J. of Neurosci. 38, 4200–4211 (2018).

41. 41.

Brown, M. W. & Aggleton, J. P. Recognition memory: what are the roles of the perirhinal cortex and hippocampus? Nat. Rev. Neurosci. 2, 51 (2001).

42. 42.

Howard, M. W. & Kahana, M. J. A distributed representation of temporal context. J. of Math. Psychol. 46, 269–299 (2002).

43. 43.

Howard, M. W., Fotedar, M. S., Datey, A. V. & Hasselmo, M. E. The temporal context model in spatial navigation and relational learning: toward a common explanation of medial temporal lobe function across domains. Psychol. Rev. 112, 75 (2005).

44. 44.

Jenkins, L. J. & Ranganath, C. Prefrontal and medial temporal lobe activity at encoding predicts temporal context memory. J. of Neurosci. 30, 15558–15565 (2010).

45. 45.

MacDonald, C. J., Lepage, K. Q., Eden, U. T. & Eichenbaum, H. Hippocampal "time cells” bridge the gap in memory for discontiguous events. Neuron 71, 737–749 (2011).

46. 46.

Eichenbaum, H. Time cells in the hippocampus: a new dimension for mapping memories. Nat. Rev. Neurosci. 15, 732 (2014).

47. 47.

Sun, C., Yang, W., Martin, J. & Tonegawa, S. Hippocampal neurons represent events as transferable units of experience. Nat. Neurosci. 23, 651–663 (2020).

48. 48.

Reddy, L. et al. Human hippocampal neurons track moments in a sequence of events. J. of Neurosci. (2021) https://doi.org/10.1523/JNEUROSCI.3157-20.2021.

49. 49.

Umbach, G. et al. Time cells in the human hippocampus and entorhinal cortex support episodic memory. Proc. of the Natl. Acad. of Sci. 117, 28463–28474 (2020).

50. 50.

Bright, I. M. et al. A temporal record of the past with a spectrum of time constants in the monkey entorhinal cortex. Proc. of the Natl. Acad. of Sci. 117, 20274–20283 (2020).

51. 51.

Tsao, A. et al. Integrating time from experience in the lateral entorhinal cortex. Nature 561, 57 (2018).

52. 52.

Schröder, T. N., Haak, K. V., Zaragoza Jimenez, N. I., Beckmann, C. F. & Doeller, C. F. Functional topography of the human entorhinal cortex. eLife 4, e06738 (2015).

53. 53.

Maass, A., Berron, D., Libby, L. A., Ranganath, C. & Düzel, E. Functional subregions of the human entorhinal cortex. eLife 4, e06426 (2015).

54. 54.

Kraus, B. J. et al. During running in place, grid cells integrate elapsed time and distance run. Neuron 88, 578–589 (2015).

55. 55.

Heys, J. G. & Dombeck, D. A. Evidence for a subcircuit in medial entorhinal cortex representing elapsed time during immobility. Nat. Neurosci. 21, 1574–1582 (2018).

56. 56.

Ezzyat, Y. et al. Direct brain stimulation modulates encoding states and memory performance in humans. Curr. Biol. 27, 1251–1258 (2017).

57. 57.

Ezzyat, Y. et al. Closed-loop stimulation of temporal cortex rescues functional networks and improves memory. Nat. Commun. 9, 365 (2018).

58. 58.

Thakral, P. P., Wang, T. H. & Rugg, M. D. Decoding the content of recollection within the core recollection network and beyond. Cortex 91, 101–113 (2017).

59. 59.

Goyal, A. et al. Functionally distinct high and low theta oscillations in the human hippocampus. Nat. Commun. 11, 1–10 (2020).

60. 60.

Burke, J. F. et al. Human intracranial high-frequency activity maps episodic memory formation in space and time. NeuroImage 85, 834–843 (2014).

61. 61.

Long, N. M., Burke, J. F. & Kahana, M. J. Subsequent memory effect in intracranial and scalp eeg. NeuroImage 84, 488–494 (2014).

62. 62.

Kragel, J. E. et al. Similar patterns of neural activity predict memory function during encoding and retrieval. NeuroImage 155, 60–71 (2017).

63. 63.

Manning, J. R., Jacobs, J., Fried, I. & Kahana, M. J. Broadband shifts in local field potential power spectra are correlated with single-neuron spiking in humans. J. of Neurosci. 29, 13613–13620 (2009).

64. 64.

Miller, K. J., Honey, C. J., Hermes, D., Rao, R. P. N. & Ojemann, J. G. Broadband changes in the cortical surface potential track activation of functionally diverse neuronal populations. NeuroImage 85, 711–720 (2014).

65. 65.

Hanslmayr, S., Staudigl, T. & Fellner, M.-C. Oscillatory power decreases and long-term memory: the information via desynchronization hypothesis. Front. in Human Neurosci. 6, 74 (2012).

66. 66.

Griffiths, B. J. et al. Alpha/beta power decreases track the fidelity of stimulus-specific information. eLife 8, e49562 (2019).

67. 67.

Geller, A. S., Schleifer, I. K., Sederberg, P. B., Jacobs, J. & Kahana, M. J. Pyepl: a cross-platform experiment-programming library. Behav. Res. Methods 39, 950–958 (2007).

68. 68.

Weidemann, C. T. et al. Neural activity reveals interactions between episodic and semantic memory systems during retrieval. Journal of Experimental Psychology: General 148, 1 (2019).

69. 69.

Avants, B. B, Epstein, C. L., Grossman M. & Gee, J. C. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain.  Med. Image Anal. 12, 26–41 (2008).

70. 70.

Yushkevich, P. A. et al. Automated volumetry and regional thickness analysis of hippocampal subfields and medial temporal cortical structures in mild cognitive impairment. Hum. Brain Mapp. 36, 258–287 (2015).

71. 71.

Fischl, B. et al. Automatically parcellating the human cerebral cortex. Cereb. Cortex 14, 11–22 (2004).

72. 72.

Dykstra, A. R. et al. Individualized localization and cortical surface-based registration of intracranial electrodes. NeuroImage 59, 3563–3570 (2012).

73. 73.

Van Essen, D. C., Glasser, M. F., Dierker, D. L., Harwell, J. & Coalson, T. Parcellations and hemispheric asymmetries of human cerebral cortex analyzed on surface-based atlases. Cereb. Cortex 22, 2241–2262 (2012).

74. 74.

Varma, S. & Simon, R. Bias in error estimation when using cross-validation for model selection. BMC Bioinformatics 7, 91 (2006).

75. 75.

Smith, S. M. & Nichols, T. E. Threshold-free cluster enhancement: addressing problems of smoothing, threshold dependence and localisation in cluster inference. NeuroImage 44, 83–98 (2009).

Acknowledgements

We thank Medtronic and Blackrock Microsystems for providing neural recording equipment. This work was supported by the DARPA Restoring Active Memory (RAM) program (Cooperative Agreement N66001-14-2-4032) and NIH grants MH55687 and T32NS047987. We are indebted to all patients who have selflessly volunteered their time to participate in our study. The views, opinions, and/or findings contained in this material are those of the authors and should not be interpreted as representing the official views or policies of the Department of Defense or the U.S. Government. Data were provided in part by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH091657) funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University. We thank Ethan Solomon and Nora Herweg for providing feedback on this work.

Author information

Authors

Contributions

M.J.K. and Y.E. designed the study; J.E.K. analyzed data and drafted the manuscript. J.E.K. and M.J.K. edited the manuscript. J.M.S. performed anatomical localization of depth electrodes. M.R.S., G.W., B.L., R.G., B.C.J., K.A.Z. and S.A.S. recruited subjects and performed clinical duties associated with data collection.

Corresponding authors

Correspondence to James E. Kragel or Michael J. Kahana.

Ethics declarations

Competing interests

M.R.S. is the principal investigator (PI) of research contracts with SK Life Science, Takeda, UCB Pharma, Neurelis, Eisai, Medtronic, Engage, and Cavion. M.R.S. serves as a consultant to Medtronic through Jefferson University which receives all compensation for these services. B.C.J. has research contracts with Neuropace, Medtronic, Sunovion, and Eisai. R.E.G. servers as a consultant to Medtronic, a subcontractor on the RAM project. R.E.G. receives compensation for these services; the terms of this arrangement have been reviewed and approved by Emory University in accordance with its conflict of interest policies. The remaining authors declare no competing interests.

Peer review informationNature Communications thanks the anonymous reviewers for their contribution to the peer review of this work. Peer Review reports are available.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

Kragel, J.E., Ezzyat, Y., Lega, B.C. et al. Distinct cortical systems reinstate the content and context of episodic memories. Nat Commun 12, 4444 (2021). https://doi.org/10.1038/s41467-021-24393-1

• Accepted:

• Published: