Introduction

Ongoing experience is not always focused on the events, or tasks, taking place in the immediate environment. Sometimes we find ourselves absorbed by personally-relevant thoughts that are not directly related to events in the present, an experience that is often studied under the rubric of mind-wandering1,2. Psychological studies have established that there is substantial individual variation in such self-generated experiences. Individuals whose thoughts are frequently directed away from the task in hand, perform poorly on tasks of sustained attention, reading and lecture comprehension, intelligence and working memory3,4,5. However, these individuals can perform better on tasks measuring creativity, problem-solving, or those that are semantic or episodic in nature6,7,8,9,10. The ability to imagine different times and places is often described as mental time travel11 and is assumed to rely on neural mechanisms associated with the memory function12,13. Although we know that different types of ongoing experience make important contributions to daily life, we lack a clear understanding of the mechanisms that influence how and why this common everyday experience varies across people14,15.

Contemporary accounts assume individual differences in thought patterns occur in part because different types of experience depending on specific underlying neurocognitive mechanisms16. Consequently, individual variation in these underlying features will determine the ease with which certain thought patterns are generated or maintained. Consistent with this view, studies highlight attention and control systems as important in the prioritisation and maintenance of patterns of thought in a context-appropriate manner17,18. In contrast, activity patterns within regions allied to the default mode network (DMN), are associated with features of experience that are closely tied to the nature of the representations of the thoughts themselves19,20,21,22,23. In healthy participants, for example, changes in activity within the DMN are linked to patterns of experience with rich subjective details in both episodic memory and working memory tasks contexts. In patients with forms of dementia that target elements of the DMN (including posterior cingulate cortex), patterns of off-task experience are less intense24 and scene construction is impaired, particularly with respect to global features of the experience25. Likewise, patients with semantic dementia, in which atrophy is focussed on the anterior temporal lobes, have deficits in the ability to imagine the future26. Finally, lesions to the hippocampus, a region closely aligned to the DMN, impact on the episodic content that often occupies the absent mind27.

The complex associations between aspects of ongoing experience, measures of cognition and neural function suggests that individual variation in specific patterns of experience may be reflected by composite features, encompassing differences in both cognition and neural function. For example, according to contemporary views of mind-wandering14,16 a chronic tendency towards off-task thoughts may depend on a relative weakness in the ability to perform well on tasks of executive control, relative strengths on the ability to generate cognition based on memory representations16,28, or a combination of both. Moreover, these changes could be mirrored by equally complex neural patterns (e.g. See prior study29). At first glance, this heterogeneity pattern of functional association may seem to undermine attempts to understand the individual differences that underpin different aspects of ongoing thought. However, in our study, we take advantage of this richness by using Sparse Canonical Correlation Analysis (SCCA)30,31, which capitalises on machine learning and multivariate data patterns to infer the underlying dimensional structure that best explains complex data matrices32. Canonical-correlation analysis (CCA) is an advanced multivariate technique that identifies distinct components between two data domains—in our case, brain-region connection-strength values and task performance among several cognitive functions. This modelling approach allows linear combinations of the two variable vectors with correlations among variables to be determined and produces modes in which the functional connectivity data are simultaneously constrained by cognitive measures (and vice versa). To enhance the interpretability of the decomposition solutions, we used a variant of CCA penalized by L1 regularisation, SCCA. This method aims to set a maximum number of brain or behaviour variables to exactly zero to produce more robust decomposition solutions, and which results in a regularized version of the singular value decomposition.

Our prior studies29,33 used SCCA to infer hidden patterns in resting-state data that are linked to individual variation in the patterns of thoughts themselves. In the current study, we focus on whether composite neurocognitive features that describe variation in different types of cognitive process (e.g. executive control, semantic memory) and neural organisation describe meaningful trait variation in patterns of ongoing thoughts. We recorded performance in a large cohort of participants on a battery of tasks, including those more reliant on semantic processing (e.g. Graded naming, semantic association) and those with a higher reliance on executive control (e.g. Digit span and task switching). In the same participants, we acquired measures of both structural and functional brain organisation. In addition, the same participants took part in a multi-day study in which they performed tasks that varied in their working memory demands, to test for context-dependent effects on experience. While participants performed these tasks, we used multi-dimensional experience sampling (MDES), which requires participants to provide self-reported descriptions of their experience. This method asks participants on many occasions to provide reports on multiple questions that describe the content and form of their ongoing thoughts.

In our investigation, we applied multivariate SCCA to the sets of behavioural and brain data to generate a set of modes that reflect joint descriptions of the data from both domains (i.e. Dimensions that reflect commonalities between how a person performed on the tasks and their brain organization at rest). The neurocognitive descriptions revealed by this analysis were used in multivariate multiple analysis to predict patterns of ongoing experience recorded in the laboratory to understand if these neurocognitive modes were related to the experiences that participants reported in the laboratory. We used this analytic approach to evaluate the predictions of the so-called component process account of ongoing thought16. Broadly this account assumes that different components of cognition (for example long term memory, affective processing and executive control) are implicated in different features of ongoing thought. In particular, in this study we focused on the possibility that we use knowledge about the world to generate patterns of ongoing thought that are unrelated to the events in the here and now, while we use executive control to maintain focus on task-relevant information (e.g. McVay and Kane3). In the context of the current study, this would predict that individual variation in the tendency to be absorbed in thoughts related to other times and places, reflects a trade-off between neurocognitive processes that rely on more generative aspects of cognition and those more closely associated with greater executive control16.

Results

Identification of neurocognitive modes of variation

Our current study aims to understand whether composite features that describe an individuals’ brain organisation at rest and their task performance in laboratory conditions, are linked to the individual differences in patterns of ongoing thoughts. We first identified dimensions that describe shared variance in cognitive performance and whole-brain functional connectivity. We applied CCA to the neural and behavioural data and identified two reliable modes that reached statistical significance based on permutation testing (see “Methods”). A meta-analysis of the functional connectivity summary was performed using Neurosynth34 to facilitate our interpretation of the likely functions of these whole-brain patterns (see “Methods”). These cognitive and neural patterns are presented in Fig. 1. The cognitive patterns and the meta-analysis results are both presented as world clouds. The size of the word indicates the strength of the relationship and the colour of the direction of the association.

Figure 1
figure 1

Two reliable neurocognitive modes uncovered through CCA. In both sets of word clouds, the size of the word indicates the strength of the relationship and the colour the direction of the association. The left-hand panel shows the task loadings for each mode in the form of a word cloud (see supplemental material Figure S1 for canonical vector loadings as heat maps). The central panel presents the brain regions that contributed the most to each mode created by calculating the sum of the canonical vectors derived from the functional connectivity. The orange-yellow maps present the overall positively connected nodes and the blue maps show the overall negatively connected modes (see https://github.com/htwangtw/Wang2020_taskSCCA/tree/master/reports/manuscript for raw canonical vector loadings of the functional connectivity data). The right-hand panel shows word clouds that describe the results of a quantitative meta-analysis of this functional data performed using Neurosynth (see “Methods”).

Full size image

Behaviourally, mode 1 (r = 0.57, p = 0.036) is reflected in positive loadings on task switching (both flexibility and inhibition) and picture naming, with relatively weak negative loadings on the specific feature matching and strong semantic association task. These two tasks both required the ability to access semantic knowledge but not flexibility in semantic processing35. In neural terms, functional connectivity was dominated at the positive end of the dimension by lateral visual, the left temporoparietal junction and the left ventral lateral prefrontal cortex. In contrast, the negative end of the functional dimension was dominated by a region of middle cingulate, the anterior motor cortex and lateral occipital cortex. Meta-analysis highlighted positive loadings for terms linked to perceptual processing, including “object recognition”, “motion and visuospatial memory” as well as terms linked to semantic processing (“verbal semantics” and “sentence comprehension”). It is noteworthy that the task loading most heavily on the behavioural features of this dimension involved switching between judgements based on shape, movement and colour. This pattern is broadly reflected in the meta-analysis of the neural data (“motion and visual-spatial processing”) suggesting reasonable convergence between the brain and behavioural descriptions of the mode. Please note that the relative lack of differentiation between tasks makes this mode relatively difficult to interpret in functional terms since it could be related to general properties (such as motivation).

Mode 2 (r = 0.59, p = 0.011) was associated with strong positive loading on tasks that require executive control (“digit span” and “flexibility”) and strong negative loadings on tasks that required self-generation of overt responses based on semantic knowledge (“picture naming” and “category fluency”). The neural patterns emphasised regions of the ventrolateral visual cortex as well as the pre-supplementary area in yellow, while the negative end in blue was dominated by the left angular gyrus, the dorsal precuneus and the dorsomedial prefrontal cortex. A meta-analysis of the functional data highlighted a clustering of terms associated with semantic tasks at the end associated with better picture naming and category fluency (“sentence comprehension”, “verbal semantics” and “reading”). Similar to mode 1, therefore, there was a mapping between the pattern of behaviour for mode 2 (better performance on semantic tasks) and meta-analysis of the associated neural patterns (a relative preference for terms with semantic links).

Trade-offs between semantic and executive control predict detailed off-task mental time travel

Having determined two reliable neurocognitive modes, we next tested the hypothesis that these explained significant variance in the data collected in the experience sampling phase of the experiment. We conducted a multiple multivariate regression in which the average responses to the experience sampling questions were the dependent variables. We included scores for each individual for each mode of brain-behaviour co-variation as explanatory variables. Age, gender and mean frame displacement were included as nuisance covariates. This analysis revealed that the variation in mode 2 was associated with the experience sampling questions at the multivariate level (F(164, 13) = 2.87, p = 0.009, η2 = 0.19). No similar association was seen with mode 1 (F(164, 13) = 0.64, p = 0.82, η2 = 0.05). Figure 2B illustrates this multivariate association between mode 2 and reports of ongoing experience in the form of a word cloud in which the size and colour of the items reflect the strength and direction of the relationship (for further details please see full statistical report in supplemental material). The positive end of the thought pattern, which was linked to better tasks that rely broadly on executive function, was associated with being on-task. In contrast, the negative end of mode 2, that was linked to better performance on tasks that depended on semantic knowledge to generate task responses (see Fig. 2C), tended to emphasise a pattern of thoughts high on subjective detail, and that were unrelated to the task, and focused on time periods other than the present, which can be summarised as mental time travel during off-task thinking.

Figure 2
figure 2

The patterns in the experience sampling questions associated with mode 2. A The neurocognitive associations that constitute mode 2. B The multivariate relationship between mode 2 and patterns of ongoing thought displayed in the form of a word cloud in which the size and colour of the items reflect the strength and direction of the relationship. Individuals who tended to be better at tasks dependent on semantic knowledge to generate task responses tended to endorse patterns of thoughts high on subjective detail, and that were unrelated to the task, and focused on other periods (past and future). C Standardised task performance scores (y-axis) plotted alone the percentile of mode 2 score (x-axis). The task performance scores are the original measures used in SCCA (see “Cognitive tasks”). The mode 2 score is the addition of the SCCA component score of cognitive tasks and functional connectivity (see “Group level regression analysis”). The ribbon plot shows that mode 2 captures patterns of individual differences in the trade-offs between executive function and semantic knowledge. The shaded bars describe the 95% confidence intervals of the mean.

Full size image

The relationship between neurocognitive modes are stable across tasks conditions and days

Having determined an association between neurocognitive modes of variation and patterns of ongoing thought, we next examined how stable this relationship was across task context and day. In our experience sampling study, participants performed alternating task blocks of lower (0-back) and higher (1-back) working memory demands. If the relationship between the functional modes and patterns of ongoing thoughts are context-specific, then they could change across the conditions or days of the experiment. We examined whether similar patterns of thoughts emerged across these contexts by running separate multiple multivariate linear regressions on each task condition. The outcome of these analyses is presented in Fig. 3. In Fig. 3A,B, it can be seen that this produced similar patterns of experience in both tasks, indicating that the association with mode 2 was robust across task contexts (1-back: F(164, 13) = 2.97, p = 0.0006, η2 = 0.19; 0-back: F(164, 13) = 2.43, p = 0.0048, η2 = 0.16; see full statistical report in supplemental material). Importantly, the patterns produced by these analyses were generally highly correlated with each other, and with the pattern produced by the overall analyses (range of correlation across task contexts [0.93 0.98] with all p < 0.001). Next, we examined whether these relationships are consistent across different days of our experience sampling study. We carried out three multiple multivariate regressions in which average scores for each day were dependent variables. The results of this analysis is presented in Fig. 3C,D where it can be seen that the relationship remained relatively stable across the sessions of our experiment (Day 1: F(164, 13) = 1.91, p = 0.0327, η2 = 0.13; Day 2: F(161, 13) = 1.62, p = 0.0855, η2 = 0.12; Day 3: F(129, 13) = 2.93, p = 0.0009, η2 = 0.23; see full statistical report in supplemental material). As with the analysis of different task contexts, we found that the patterns of thoughts produced by the analysis on each day were consistent with each other and also with the pattern produced by the overall analyses (range of correlation among days [0.83 0.96] with all p < 0.001). It is important to note that the sample size for each day varies because some participants did not participate in all sessions.

Figure 3
figure 3

The stability of mode 2 in predicting thought patterns across task context and day. A Similar multivariate associations between mode 2 and the patterns of ongoing thought emerged in both by tasks conditions; B correlations between the patterns of ongoing thought observed in general and across both tasks; all p-value < 0.001; C similar patterns of ongoing thought were observed across each day of the study; D correlations between the patterns of ongoing thought observed in general and across each day of the study; all p-value < 0.001.

Full size image

Trade-offs between semantic and executive control and their relation to cortical structure

Our last set of analyses considers the relationship between the two modes and a more trait-like element of cortical organisation: whole-brain grey matter thickness. Cortical thickness is a relatively stable feature of an individual’s neuroanatomy and is related to genetic factors36,37, personality traits, and neuropsychiatric disorders (e.g. past studies38,39,40). In this study, we aimed to evaluate how the variation in brain organisation and task performance identified by our analysis was linked to this more stable aspect of brain organisation. If we found associations between the individual variation in a mode and individual differences in cortical grey matter structure, this would enhance our confidence that our CCA mode described a reasonably stable neurocognitive trait. Furthermore, any regions identified in this manner would be useful in future studies examining the neural architecture that underlies different aspects of ongoing thought.

To understand the association between variation in the two functional modes, and variation in the thickness of cortical grey matter, we conducted a regression in which a matrix describing thickness at each vertex for each individual were the dependent variables. The average score for each individual on each mode was included as explanatory variables. Age and gender were both entered as between participant explanatory variables. This initial model yielded no significant results.

Next, we conducted an exploratory analysis in which each individual’s score for the behavioural and neural aspects was entered in the same model as separate explanatory variables (a total of 4 explanatory variables). This analysis revealed significant associations (FWE-corrected p < 0.05, uncorrected-p < 0.0025) with both the behavioural and neural aspects of mode 2 and are presented in Fig. 4. The behavioural dimension, describing performance trade-offs between tasks tapping semantic and executive domains, was with variation in cortical thickness in a region of subgenual anterior cingulate (mode 2 CT: r = − 0.29, p < 0.001). Thickness in this region was higher for individuals better at semantic than executive tasks. In addition, the functional dimension was associated with cortical thickness in a region of motor cortex immediately posterior to the central sulcus (mode 2 FC: r = 0.26, p < 0.001). Thickness in this region was smaller towards the semantic end of this dimension.

Figure 4
figure 4

Exploratory cortical thickness analysis. We used a general linear model to explore whether individual variation in the canonical modes was also reflected in the structural organisation of the cortex. We found that the task component in mode 2 is related to the variation in a region of subgenual anterior cingulate among. Thickness in this region was higher for individuals better at semantic than executive tasks. The functional connectivity component of mode 2 was associated with cortical thickness in a region of motor cortex immediately posterior to the central sulcus. Thickness in this region was smaller towards the semantic end of this dimension.

Full size image

Summary

Using machine learning, we found that relatively better performance on tasks requiring the generation of responses based on semantic knowledge than executive control was linked to a pattern of functional organisation that was associated via meta-analysis with processes with semantic features (sentence processing, reading and verbal semantics). Features of this functional mode predicted significant variation in the types of thoughts that individuals experienced in the laboratory: performing better at tasks in which responses were guided by semantic representations, than those dependent on executive control, were associated with patterns of thought characterised by greater subjective detail and a focus on time periods other than the here and now. Finally, a cortical thickness analysis identified structural correlates of this functional mode, suggesting that the discovered neurocognitive trait has a potentially stable relationship to structural organisation.

Discussion

Our study aimed to understand why certain people spontaneously engage more in certain patterns of thoughts than do others. We focused on testing the prediction of contemporary accounts of ongoing thought, which assume that different aspects of ongoing experience are related to different underlying neurocognitive processes16,41. Accordingly, the ease with which an individual engages in particular forms of ongoing thought should be related to the degree to which their neurocognitive architecture emphasises the relevant underlying neurocognitive process. In particular, we were interested in testing the broad view that periods of imaginative thinking during states of mind-wandering rely more on the ability to self-generate information from long term memory than do thoughts focused on the present16.

Consistent with this view, we found that individuals who emphasise patterns of thoughts about other times and places with rich subjective detail, rather than the task in hand, show evidence of a trade-off towards expertise at semantic processing at the cost of executive control. Behaviourally, ‘detailed mental time travel’ was linked to better performance on tasks that rely on generating a response based on semantic knowledge rather than those that tap executive control. A meta-analysis of functional connectivity of the neural component of mode 2 suggested an association with patterns seen in tasks that rely on semantic processing (such as verbal semantics and sentence processing). These data provide converging evidence from both brain and behaviour, that the tendency to generate thoughts that extend beyond the current moment is linked to a more general ability to generate cognition based on existing semantic knowledge. Our finding, therefore, confirms predictions of theoretical accounts that assume that patterns of off-task thought are partly generated using information from long-term memory16. Our analysis also confirmed prior studies3 that tasks loading on executive control are linked to a tendency to focus on task relevant information. Together this has several implications for our understanding of the underlying mechanisms that give rise to different types of experience.

First, our study adds to a growing body of evidence showing that semantic processes play a key role in aspects of human imagination. Studies of patients with semantic dementia have shown that they have deficits in future thinking, as well as the inability to represent detailed scenes in imagination26. Moreover, dementia that results in atrophy to the posterior cingulate cortex impacts upon the intensity of mind-wandering episodes24 and our study highlights the connectivity of this region as important for patterns of mental time travel with vivid detail. Our study, therefore, builds on prior work that uses lesion based methods by showing that in a young healthy population individuals who perform better on task reliant on semantic processes, tend to report more detailed mental time travel. Together these studies provide converging support for a neural architecture that is implicated in tasks involving semantic processing as playing a role in imagination and in particular, content, level of detail, or perhaps both.

Second, our study has important methodological implications for contemporary accounts of ongoing thoughts. For example, work on mind-wandering has found that this pattern of experience has both costs and benefits in daily life. Deleterious aspects associated with excessive mind-wandering have been linked to problems in executive control and links to affective difficulties, while advantages have been linked to better performance on tasks that rely on the generation of information (creativity, semantic processing and self-reference)42. Our study suggests that patterns of reports that correspond broadly to contemporary views of the mind-wandering state (i.e. see review16) can be understood as the consequence of trade-offs between different underlying qualities of cognition. In this way our study highlights that questions regarding specific mental states can be understood as emergent properties from interactions within a more general component process architecture. One advantage of our approach is that it produces categories of experience in a data-driven manner, and so circumvents definitional questions regarding what constitutes a specific mental state2,43,44.

Although our study informs our understanding of how we use semantic knowledge to consider times and places other than the here and now, it nonetheless leaves several important questions open. For example, we used a battery of tasks that primarily operationalised self-generated processing through the lens of tasks tapping semantic processing and creativity. This decision was pragmatic: such tasks provide a reasonably direct indication of one aspect of self-generated processing because they require individuals to use conceptual knowledge to guide behaviour. It was also well aligned with the expertise of our lab. This design choice, however, places an important boundary condition on the interpretations that should be placed on our findings. Our study highlights a robust association between how we use knowledge of the world around us to generate behaviour and the patterns of thoughts we generate in imagination. As we did not collect other aspects of self-generated processing from the participants (such as autobiographical memory or emotional processing), it remains an open question what role individual differences in these processes play in patterns of ongoing thought. It will be important for future work to extend our methodological approach by using a broader range of tasks in order to deepen our understanding of how variation in the emotional, or episodic domain contribute to patterns of ongoing thoughts, and how similar or different these are to variation in semantic knowledge. It is also important to note that our study focused on patterns of ongoing through the lens of a trait. There are important aspects of patterns of ongoing thought that can only be properly understood when it is recorded simultaneously with ongoing neural activity45. It is important to consider, therefore, how these individual differences translate into momentary patterns of neural activity. In this context, it is worth noting that we have recently shown that that neural signals in both the dorsal left pre-frontal cortex, and the posterior cortex are linked to momentary differences in off-task content, and levels of subject detail respectively17. These regions are both highlighted by our analysis as members of the neural network linked to semantic processing, and to reports of experiences which emphasise off-task episodes with a high degree of subject detail (see also Kam and colleagues46 for a conceptually similar conclusion). This association provides reasonable grounds to suggest that it could be profitable for future work to explore how patterns of neurocognitive traits at rest relate to the momentary neural patterns observed when the same pattern of thought occurs. Finally, the sample size in our study did not allow us to fully validate our observations in an out of model sample. Although we used cross validation to ensure that parameters were able to predict out of sample data, and we successfully validated the canonical modes in a set of unseen variables (e.g. the experience sampling data) it would be an advantage for future studies to perform out of sample validation assessing the degree to which these findings generalise outside of the current group of participants.

In conclusion, our study suggests that the reason why certain people engage more in certain patterns of thoughts than do others is related to the emphasis their neurocognitive architecture places on different types of process. We found experiences that emphasise detailed mental time travel, are linked to neural indices indicative of relative expertise in tasks reliant on semantic processing. Our study, therefore, underlines the core role that the way we understand the world around us plays in what we construct using imagination. It is worth noting that that occurrence of imaginative thoughts are common across cultures47 and have links to both beneficial and detrimental aspects of wellbeing42. Accordingly, it seems possible that the structure and ease with which conceptual knowledge can be mobilised in imagination will be an important influence on how a ubiquitous aspect of daily life impacts upon a person’s happiness and success.

Methods

Participants

Two hundred and seven healthy participants were recruited from the University of York (132 females, 75 males; age range = 18–31 years, M = 20.21, SD = 2.36). This analysis included two data sets with some shared measurements and the same MRI protocol. Participants were right-handed native English speakers with normal or corrected-to-normal vision and no history of psychiatric or neurological illness. Participants underwent MRI scanning, completed a 1-h online questionnaire. Of the first cohort, participants attended three (165 participants; 99 females, 66 males; age range = 18–31 years, M = 20.43, SD = 2.63) 2-h behavioural testing sessions to complete a battery of cognitive tasks. The second cohort (42 participants; 33 females, 9 males; age range = 18–23 years, M = 19.79, SD = 1.37) underwent two 2-h behavioural testing sessions to complete a battery of cognitive tasks. The behavioural sessions took place within a week of the scan. Twenty-nine participants were excluded from the multivariate pattern analysis because they failed to complete all of the behavioural testing sessions or failed the imaging data quality check. In total, 178 participants (113 females, 65 males; age range = 18–25 years, M = 19.81, SD = 1.66) were included in the multivariate pattern analysis and the comparison with cognitive performance. Participants were rewarded with either a payment of £10 per hour or a commensurate amount of course credit. All participants provided written consent prior to the fMRI session and the first behavioural testing session. Approval for the study was obtained from the ethics committee of the University of York Department of Psychology and the University of York Neuroimaging Centre. All data acquisition methods used in this study were in accordance with international, national, and institutional guidelines.

MRI acquisition

Structural and functional data were acquired using a 3 T hdx excite MRI scanner (GE Healthcare, Little Chalfont, UK) utilising an eight-channel phased-array head coil tuned to 127.4 MHz at the York Neuroimaging Centre, University of York. Structural MRI acquisition in all participants was based on a T1-weighted 3-D fast-spoiled gradient-echo sequence repetition time (TR) = 7.8 s, echo time (TE) = minimum full, flip angle = 20°, matrix size = 256 × 256, 176 slices, voxel size = 1.13 × 1.13 × 1 mm3. Resting-state activity was recorded from the whole-brain using single-shot 2-d gradient-echo-planar imaging TR = 3 s, TE = minimum full, flip angle = 90°, matrix size = 64 × 64, 60 slices, voxel size =  3 × 3 ×  3 mm3, 180 volumes. Participants viewed a fixation cross for the duration of the 9-min fMRI resting-state scan. A fluid-attenuated inversion-recovery (FLAIR) scan with the same orientation as the functional scans were collected to improve co-registration between subject-specific structural and functional scans.

Resting-state data preprocessing

All preprocessing and denoising steps for the functional MRI data were carried out using the SPM software package (Version 12.0) and Conn functional connectivity toolbox (Version 17.f), based on the MATLAB platform (Version 17.a). The first three functional volumes were removed to achieve steady-state magnetisation. The remaining data were first corrected for motion using six degrees of freedom (x, y, z translations and rotations), and adjusted for differences in slice-time. Subsequently, the high-resolution structural images were co-registered to the mean functional image via rigid-body transformation, segmented into grey matter, white matter and cerebrospinal fluid probability maps, and all functional volumes were spatially normalized to Montreal Neurological Institute (MNI) space using the segmented images and a priori templates. This indirect procedure utilizes the unified segmentation normalization framework, which combines tissue segmentation, bias correction, and spatial normalization in a single unified model. No smoothing was employed, complying with recent studies that report the negative influence of this procedure on the construction of connectivity matrices analysis.

Moreover, a growing body of literature indicates the potential influence of participant motion inside the scanner on the subsequent estimates of functional connectivity. To ensure that motion and other artefacts did not confound our data, we have employed extensive motion-correction and denoising procedures, comparable to those reported in the literature. In addition to the removal of six realignment parameters and their second-order derivatives using the general linear model (GLM), a linear detrending term was applied as well as the CompCor method that removed five principal components of the signal from white matter and cerebrospinal fluid. Moreover, the volumes affected by motion were identified and scrubbed based on the conservative settings of motion greater than 0.5 mm and global signal changes larger than z = 3. Though recent reports suggest the ability of global signal regression to account for head motion, it is also known to introduce spurious anti-correlations and was thus not utilised in our analysis. Finally, a band-pass filter between 0.009 and 0.08 Hz was employed to focus on low-frequency fluctuations.

The Craddock connectivity-based parcellation48 was selected as the full brain parcellation. We used a set of 100 regions (K = 10) from the two-level spatial connectivity-based parcellation. The Craddock atlas clusters delineate spatially coherent regions that are more functionally homogenous, thus it demonstrated good dimension reduction ability in the context of resting-state functional connectivity analysis. The spatial connectivity-based parcellation focuses on similarity between functional connectivity maps48. Fully connected, undirected and weighted matrices of bivariate correlation coefficients (Pearson’s r) were constructed for each participant using the average BOLD signal time series obtained from all the 100 ROIs described above. The off-diagonal of each correlation matrix contained 4,950 unique measures of region-region connection strengths (i.e., the upper or lower triangle of the network covariance matrix). This approach provided a measure of connection strength of the whole-brain for each participant. Finally, Fisher’s r-to-z transformation was applied to each network covariance matrix.

Behavioural data

Cognitive tasks

We selected 9 cognitive tasks that are common across the two cohorts. The selected tasks measure cognitive functions that have been examined in previous mind-wandering literature, encompassing executive control (digit span, task switching task49), generation of information (unusual uses task50, verbal fluency task), semantic memory (semantics picture-word matching tasks51: relatedness judgement tasks and feature matching task52), episodic memory (paired-associate task53, four mountains task54), and fluid intelligence (Raven Advanced Progressive Matrices; RAPM55).

Thirteen cognitive scores were calculated from the selected tasks. Performance of the digit span task was represented as the average of digit span in the forward and backward recall conditions. The verbal fluency score is the contrast of the category condition and letter condition (category—letter). The previous research35 shows that category fluency is more dependent on semantic memory, while letter fluency is more executively demanding. Picture naming tasks, the four mountains tasks, RAMP were summarised with accuracy scores. The task switching measure provided two scores (a) flexibility (In the original study by Whitmer and colleagues49, contrast “switch cost”, smaller values indicates better ability to switch away from the previous condition. For the ease of interpretation, we reversed the scores and re-named the contrast as “flexibility”) as the ability to switch from a different condition and (b) inhibition as the ability to suppress information from the previous trial. The calculation of the task switching contrast can be found in the original study49. All the semantics related judgement tasks, feature matching task, and the paired-associate task were summarised using efficiency scores. The efficiency scores were calculated as reaction time divided by accuracy. A smaller score indicates better performance, thus the scores were reversed to ease the interpretation. In the semantics picture-word matching task51, a picture probe was matched either to an associated word (for strong, weak and word conditions) or an associated picture (in the picture condition) or to the name of the item at a superordinate or specific level. We calculated three contrasts based on the semantics modules tested: (a) strength (strong–weak), modality (picture–word), and (b) specificity (specific–general). All the scores were brought to a common scale by variance scaling to 1 and mean centring to 0 for the subsequent analysis.

Experience sampling

We assessed the contents of experience in the context of a simple task that manipulated working memory load using a block design (see prior published examples of this task7, 56). This task was performed at the beginning of each laboratory session to minimize the contribution of participant fatigue to these experiential measures. Measuring experience over multiple days provided us with a more comprehensive description of participants’ ongoing thought at the level of a trait than would have been possible in a single experimental session.

In both conditions, non-target trials involved the presentation of pairs of shapes appearing on the screen divided by a vertical line. The pairs could be a circle and a square, a circle and a triangle, or a square and a triangle for six possible pairs (two different left/right configurations for each). The pairs never had shapes of the same kind (e.g. A square and a square). In both tasks, following an unpredictable sequence of non-target trials, a target trial was presented in which participants had to make a manual response. The target was a small stimulus presented in either blue or red across conditions, with the colour counterbalanced across participants. In the 0-back condition, two shapes flanked the target and the target would be identical to one of the two flanking shapes. Participants had to indicate by pressing the appropriate button which shape matched the target shape. In the 1-back condition, the target was flanked by two question marks and participants had to respond depending on which side the target shape was on the prior trial. Responses were made using the left and right arrow keys. Fixation crosses presentation ranged from 1.3–1.7 s in steps of 0.05 s, non-targets were varied from 0.8–1.2 s in steps of 0.05 s. Targets always ranged from 2.1–2.5 s in steps of 0.05 s and a response from participants did not end the target presentation.

Multi-dimensional experience sampling (MDES) was used to describe experience during the 0-back/1-back task. This technique uses self-report to assess the contents of experiences on a number of dimensions. The thought probes first asked participants to rate their level of task focus (“My thoughts were focused on the task I was performing.”) on a sliding scale from 0 (completely off-task) to 1 (completely on task). Participants then answered 12 randomly presented questions regarding the content and form of their experience just before they were probed. These questions (described in Table 1) were based on prior studies adopting this approach to measure self-generated thought7. There was a 20% chance of a thought probe being presented instead of a target with a maximum of one thought probe per condition block of 0-back and 1-back. In each session, an average of 14.07 (SD = 3.30, range 6–25) MDES probes occurred; in the 0-back condition an average of 7.02 (SD = 2.36, range 2–14) MDES probes occurred and in the 1-back condition an average of 7.04 (SD = 2.24, range 1–15) occurred. In total, we sampled 7,006 examples of experience in this study. In the current analysis, we calculated the mean scores of each question across the three sessions for each participant. The MDES scores were first transformed into z-scores for mean centring and unit variance scaling. The scores described the average momentary experience in each dimension. We use this score in the multivariate pattern analysis later (Fig. 5).

Figure 5
figure 5

Analysis flowchart. The flowchart of the analysis pipeline. We first conducted SCCA to uncover the hidden structure that combines the task measures and the functional connectivity data. The SCCA model selection is detailed in “Model selection” and Fig. 6. The theoretical validity of latent variables was later examined by predicting the ongoing thought report. We also explore the associated cortical thickness change associated with the functional neurocognitive modes. For the cortical thickness analysis, please refer to “Cortical thickness analysis”. For the details of multiple multivariate regression, please see “Group level regression analysis”.

Full size image
Table 1 Multiple dimension experience sampling questions in 0-back/1-back task.
Full size table

Multivariate pattern analysis

Sparse canonical correlation analysis (SCCA)

We performed a sparse canonical correlation analysis (SCCA)30,31 on the functional connectomes and the cognitive tasks, to yield latent components that reflect multivariate patterns across neural organisation and cognition (For a similar application for experience sampling, see previous study29). SCCA maximised the linear correlation between the low-rank projections of two sets of multivariate data sets with a sparse model to regularise the decomposition solutions a process that helps maximise the interpretability of the results. The regularisation function of choice is L1 penalty, which produces sparse coefficients, meaning that the canonical vectors (i.e., translating from full variables to a data matrix’s low-rank components of variation) will contain a number of exactly zero elements. L1 regularisation conducted (a) feature selection (i.e., select only relevant components) and (b) model estimation (i.e., determine what combination of components best disentangles the neurocognitive relationship) in an identical process. This way we handle adverse behaviours of classical linear models in high-dimensional data. A reliable and robust open-source implementation of the SCCA method was retrieved as R package from CRAN (PMA, penalized multivariate analysis, version 1.0.928). The amount of L1 penalty for the functional connectomes and cognitive task performance were chosen by cross-validation. The procedure is described in the section below.

Before SCCA, we regressed age, sex, head motion indicated by mean frame-wise displacement53 out of both the connectivity and cognitive task data to ensure that these potential confounders did not drive results. The implementation of the confound removal method was retrieved from Python library Nilearn (https://nilearn.github.io/, version 0.3.1). As features that do not vary across subjects cannot be predictive of individual differences, we limited our analysis of connectivity data to the top 5% most variable connections, as measured by median absolute deviation, which is more robust against outliers than standard deviation57. Thus the input data to the subsequent analysis consist of 247 ROI-ROI connectivity and 13 cognitive task measures.

Model selection

The model selection process was conducted in two parts: L1 penalty coefficient selection and component selection. For the L1 penalty coefficient selection, we performed a grid search combined with cross-validation (CV) to avoid over-fitting58. Of each penalty pair on the search grid, fivefold cross-validation was performed to search for the best out-of-sample the rank-1 canonical correlation. We then decomposed the full dataset with the selected L1 penalty coefficients (see the top panel of Fig. 6). The K-Fold CV was conducted by the implementation in Python library scikit-learn (https://scikit-learn.org/stable/, version 0.18.2).

Figure 6
figure 6

Multivariate pattern analysis pipeline. Top: parameter tuning combined with k-fold cross-validation to search for the best sparsity constraint with maximal out-of-sample the rank-1 canonical correlation. The selected set of parameters was then used as a basis to recompute CCA on the full dataset. Bottom: Permutation test for mode selection.

Full size image

We performed SCCA with the optimal hyper-parameters on the full data set and saved all decompositions. A thousand permutations were applied to determine the mode(s) that occur above chance (see the bottom panel of Fig. 6). We constructed an empirical null distribution for the set of canonical components by holding the functional connectivity data in place and permuting the row order of self-reports data. The permutation scheme specifically disturbed the link between individual differences in the dataset. The emerging distribution of no-association effects in our analysis setup provided the basis for testing the robustness of the components in the hypothetical population. A p-value with family-wise error (FWE) correction was calculated based on the permutation results. Modes are accepted with a 5% level of significance. In the current analyses, we adopt the permutation test with the FWE-corrected p-value by Smith and colleagues59. All modes were compared to the first canonical correlation of the permuted sample. The low-rank components are more relevant than the rest, therefore we yield more conservative p-value by comparing to the first canonical correlation only.

Group level regression analysis

To determine how patterns of unconstrained neurocognitive activity related to performance on the ongoing thoughts, self-report experience summarised in three different ways (Overall average, average by days, and average by task), we conducted an independent statistical analysis on the identical subjects. The neuro-cognitive scores were calculated from averaging the canonical variates in the significant modes. A Type III multiple multivariate regression with Pillai’s trace test was applied to the data. Each of the latent components describing the neurocognitive mechanism from the SCCA was the independent variables, and the 13 measures from MDES were the dependent variables. We thus asked how well variability in the MDES items can be explained our robust components of brain-behaviour association across individuals. We hoped to describe the neuro-cognitive components by the linear combination of the self-report questions collected via MDES. The analysis was conducted in Python library Statsmodels (https://www.statsmodels.org/stable/index.html).

Cortical thickness analysis

Cortical thickness calculation

Freesurfer was used to estimate vertex-wise cortical thickness (5.3.0; https://surfer.nmr.mgh.harvard.edu). Briefly, the following processing steps were applied: intensity normalisation, removal of non-brain tissue, tissue classification and surface extraction. Cortical surfaces were visually inspected and corrected if necessary. Cortical thickness was calculated as the closest distance between the grey/white matter boundary and pial surface at each vertex across the entire cortex. A surface-based smoothing with a full-width at half maximum (FWHM) = 20 mm was applied. Surface alignment based on curvature to an average spherical representation, fsaverage5, was used to improve correspondence of measurement locations among subjects.

Cortical thickness analysis

We explore the relationship between the SCCA modes (i.e. their canonical variates) and cortical thickness to understand the structural effect of the neurocognitive components. The surfstat toolbox for Matlab60 was used for structural covariance network analysis, as in previous studies61,62. There is a well-established negative correlation between age and cortical thickness60 and gender also influences cortical thickness61. Consequently, these variables were included as covariates of no interest. The model fitted at a surface point i is shown as bellow:

$$\begin{aligned} {\text{Ti }} & = \beta 0 \, + \, \beta {1 }*{\text{ Sex }} + \, \beta {2 }*{\text{ Age }} + \, \beta {3 }*{\text{ Connectivity 1 }} \hfill \\ & \quad + \, \beta {4 }*{\text{ Connectivity 2 }} + \, \beta {5 }*{\text{ Task 1 }} + \, \beta {6 }*{\text{ Task 2}} \hfill \\ \end{aligned}$$

We determined significant clusters in this model using random field theory for nonisotropic images62 which controlled the Family-Wise Error rate at p < 0.05. We also explored the effect of the neurocognitive score.

$${\text{Ti }} = \, \beta 0 \, + \, \beta {1 }*{\text{ Sex }} + \, \beta {2 }*{\text{ Age }} + \, \beta {3 }*{\text{ neurocognitive 1 }} + \, \beta {4 }*{\text{ neurocognitive 2}}$$

Neurosynth meta-analysis

We used the neurosynth meta-analytic database (https://www.neurosynth.org) to assess topic terms associated with the functional connectivity profile and significant cortical thickness regions. For each region of interest map, the output of the analysis was a z statistic associated with the feature term. The terms were then visualised as word clouds. Feature terms were derived from the 50 topic term-based meta-analysis (v4; https://raw.githubusercontent.com/neurosynth/neurosynth-web/master/data/topics/keys/v4-topics-50.txt). The analysis method can be found on the official website (https://neurosynth.org/analyses/terms/). Of the 50, 25 were removed as “noise” terms or experiment paradigm related information, because they did not capture any coherent cognitive function, leaving 24 topic terms (see https://github.com/htwangtw/Wang2020_taskSCCA/blob/master/reports/neurosynth50v4.csv the index and labels for the selected 24 terms).