Introduction

The ability to extract pertinent visual information from our surroundings depends on the brain’s ability to aim, and account for, eye position1,2,3. Rapid eye movements (saccades) help gather new visual information by aligning the fovea with objects of interest, at the cost of disrupting visual stability and memory by displacing the retinal image relative to the world4,5. This necessitates some neural mechanism to continuously track both intrinsic object properties—cues to identity—and extrinsic spatial properties, like object location and orientation5,6,7,8. It is thought that during saccades, oculomotor signals are used to ‘update’ retinal location9,10,11,12,13,14,15 and other visual features5,7. Recently, it has been shown that parietal and occipital cortex are involved in transsaccadic memory of object orientation and spatial frequency, respectively16,17,18. However, in real world conditions, the brain must track intrinsic and extrinsic object properties simultaneously19,20.

Various structures in the occipito-parieto-frontal stream have been implicated in the spatial aspects of transsaccadic vision, notably the parietal and frontal eye fields9,10,21,22,23,24,25,26. Likewise, transcranial magnetic stimulation (TMS) over various occipital, parietal, and frontal sites disrupts transsaccadic updating of object orientation8,27,28,29. More recently, we employed functional magnetic resonance imaging (fMRI) adaptation30,31 to identify areas that are specifically involved in transsaccadic comparisons of object orientation. These experiments identified an area in the right supramarginal gyrus (SMG), anterolateral to the parietal eye fields, that showed transsaccadic (object) orientation processing for both perception and grasp16,17.

It has proven more difficult to localize the cortical mechanisms that update features related to object identity32. Modest amounts of feature remapping have been observed in monkey parietal eye fields33. A recent human study has shown that stimulus features (specifically, spatial frequency) can be decoded from whole-brain magnetoencephalography signals after an intervening saccade34. When we adapted our transsaccadic fMRI adaption paradigm to test spatial frequency, the main source of these modulations appeared to be superior-medial occipital cortex, i.e., cuneus18. While spatial updating (‘where’ information) has a clearer set of cortical correlates9,10,11,15,23,24,25,26,27, there is no consensus for transsaccadic updating of various low-level, intrinsic object properties (‘what’ information), nor their strict relationship (for further discussion of the processes underlying these two pieces of information and how they might be bound to one another, see Golomb, 2019 and Golomb & Mazer, 2021 for reviews)35,36. However, based on previous results and our earlier experiments16,17, we speculated that spatial and feature properties are combined in parietal and occipital cortex, showing a relative specialization for updating extrinsic attributes of objects versus cues to intrinsic object identity, respectively.

In our previous adaptation experiments16,17,18, we only examined transsaccadic changes in one feature at a time. However, as noted above, in real-world conditions, the brain must update multiple object features across saccades in order to track both object identity and its location / orientation. This, in turn, may require the ‘binding’ of multiple object features across saccades37,38,39. There is evidence that comparison between features evokes different mechanisms than the sum of the parts40, but it is not known how multiple features are retained, compared, and discriminated across saccades.

Here, we investigated the cortical mechanisms for transsaccadic perception of multiple object features, combining intrinsic identify-related features (such as shape and texture) and extrinsic spatial features (such as object location and orientation). To do this, we combined elements of our previous experiments16,17,18. Specifically, we used an fMRI paradigm that required participants to remember both object shape and orientation across a saccade and then discriminate which one of these had changed (Fig. 1). Our hypotheses were that (1) both SMG and cuneus would be modulated in this task, (2) SMG would show the saccade-object orientation interactions that we observed previously17, and (3) if cuneus is involved in object identity updating, our previous spatial frequency results should extend to the transsaccadic shape manipulation in the current task. To test this, we performed a bilateral region-of-interest (ROI) analysis, based on the sites obtained from our previous studies17,18. Analysis combining both saccade directions and fixations made it possible to focus on effects for eye movement and/or feature changes, avoiding location-only remapping effects16,17. Our results confirmed the expected transsaccadic feature modulations in both SMG and cuneus, and suggest that the latter participates in a saccade-dependent network spanning several cortical lobes.

Figure 1
figure 1

Experimental paradigm and criteria for transsaccadic feature modulation. An example trial is presented (initial left fixation, rectangle, 45º) with the four possible main conditions: (1) Fixation, Different Orientation, (2) Fixation, Different Shape, (3) Saccade, Different Orientation, or (4) Saccade, Different Shape. Each of the 19.5 s trials had three main phases: (1) Sensory/Encoding, during which the stimulus (one of the three possible objects: rectangle, barrel, or hourglass) was first presented at 45º to the left or right of vertical, while the participant was looking to the left or right of centre; (2) Change Detection, wherein the same object was presented at the orthogonal orientation (Different Orientation condition) or a different object at the same initial orientation (Different Shape condition) while participants fixated on the initial fixation cross (Fixation condition) or after they made a saccade (Saccade condition); and (3) Response, where participants made a button press response to make a judgment about whether the shape (‘I’) or the orientation (‘O’) of the object has changed across the two stimulus presentations.

Results

To identify areas involved in transsaccadic discrimination of object orientation versus shape, we employed a task (Fig. 1) where 21 participants briefly viewed an object with one of three shapes (a rectangle, a convex ‘barrel’, or a concave ‘hourglass’) at one of two orientations (45° clockwise or counterclockwise). They either maintained fixation to the left or right of this central object or made a saccade to the opposite side. Then, they re-viewed an object with either a different shape or different orientation from among the same options, and then indicated which feature had changed. 17 participants met our behavioural inclusion criteria for analysis of fMRI data collected during this task.

Based on previous experiments16,17,18, we expected both SMG and cuneus to be modulated in this task. To test this, we performed a hypothesis-driven region-of-interest (ROI) analysis on these sites (see next sections for details), and then performed a functional connectivity analysis to identify how saccade-related activity (if any) in these regions correlates with activity in other cortical areas. Our hypotheses were twofold: First, we anticipated that right SMG would show an interaction of eye movement and feature change, based on previous findings where this was shown specifically for object orientation changes17. Second, we expected to see a main effect for each factor, eye movement and feature change, in cuneus which was previously shown for another feature change – spatial frequency18.

Region-of-interest analysis (I): supramarginal gyrus

In a previous experiment, we found that an area in right SMG, slightly anterior and lateral to the classic parietal eye field (revealed by a saccade-fixation subtraction) showed transsaccadic repetition modulations for object orientation, i.e., different activation when saccades were preceded and then followed by the same or different object orientations16. More recently, we showed that right SMG shows an interaction between this effect and saccades, i.e., significantly more object orientation-related modulation during saccades compared to fixation17. Here, we predicted the same result in an ROI analysis based on the Talairach coordinates of peak right SMG activation in our previous study (Table 1) and the mirror left hemisphere site, forming a bilateral pair.

Table 1 Regions-of-interest and their Talairach coordinates, a priori selected for testing of saccade and feature modulations.

Figure 2a, left panel, shows the location of the right SMG site relative to visual field-specific areas (Left Visual Field–Right Visual Field Stimulation across all (fixation and saccade) trials; yellow) and saccade-specific areas (Saccade—Fixation trials, all conditions; magenta), superimposed on an ‘inflated brain’. (Note that this rendering is useful for showing full brain data, but causes some spatial distortions and cluster separation.) Our right SMG site was anterior to two magenta clusters that might be part of the parietal eye fields, but did not itself show significant independent modulations for saccade or visual field. A first, separate whole-brain analysis investigating effects related to initial object orientation effects (clockwise versus counterclockwise) revealed no significant clusters. A second whole-brain analysis that tested for effects driven by feature (i.e., shape) sensitivity at the initial fixation point (barrel versus hourglass) did not reveal significant clusters.

Figure 2
figure 2

Region-of-interest (ROI) analysis results for tests of eye movement and feature effects in right SMG. (a) Voxelwise maps (n = 17) from two contrasts used to determine simple eye movement (Saccade > Fixation; “Sacc > Fix”; fuchsia) and visual field (Left Visual Field > Right Visual Field; “LVF > RVF”; lime green) effects are overlaid onto inflated brain rendering of an example participant’s right hemisphere (lateral view). Saccade sensitivity is shown parietally in angular gyrus (AG), whereas visual field effects can be observed in occipital cortex (extending from inferior through middle to superior occipital gyrus; IOG, MOG, and SOG). These regions are posterior/medial to right SMG (orange). (b) B-weight differences depicted for right SMG in the bar graph on the left (error bars: SEMs) which show the significant interaction between eye movement- and feature-related changes across saccades (SO: Saccade, Orientation change condition; SS: Saccade, Shape change condition; FO: Fixation, Orientation change condition; FS: Fixation, Shape change condition). The interaction is shown in full in the bar graph on the right, which shows β-weights for each of the four conditions (error bars: SEMs).

To test our specific hypothesis, we created spheres (radius = 5 mm) around our two SMG coordinates, and then extracted β-weights from these ROIs. We then performed the same interaction test as in our previous study on each ROI, i.e., Saccade (Different – Same orientation) – Fixation (Different – Same orientation)18, except in this case ‘same orientation’ simultaneously corresponded to a shape change, thus: Saccade (Orientation change – Shape change) – Fixation (Orientation change – Shape change). SMG showed a significant hemispheric difference (F1,16 = 54.84, p = 1.5 × 10–6, η2 = 0.72), so we analyzed the left and right ROIs separately. RM-ANOVAs were then used to test our specific criteria in each of the ROIs. As predicted, right SMG passed this test (Fig. 2a, right panel) with a significant saccade – feature change interaction (F1,16 = 6.84, p = 0.019, η2 = 0.076). It is noteworthy, however, that the interaction was opposite to what we observed with orientation alone16,17. However, note that in our previous experiments participants only had to track one feature and detect if it changed or not,whereas in the current experiment participants had to track two features and detect which one changed, so one might not expect the results to translate exactly. Left SMG (not shown) did not show significant modulation for saccades (F1,16 = 0.39, p = 0.54, η2 = 0.010), feature (F1,16 = 0.048, p = 0.83, η2 = 8.5 × 10−4), or saccade—feature interactions (F1,16 = 1.46, p = 0.24, η2 = 0.025). The results observed in right SMG were not influenced by any condition-related imbalances in participant performance (eye movement: F1,16 = 2.88, p = 0.109, η2 = 0.048; feature change: F1,16 = 0.564, p = 0.46, η2 = 0.013; eye movement—feature interaction: F1,16 = 0.025, p = 0.88, η2 = 4.6 × 10−4). The mean performance for each condition was as follows (mean +/− standard deviation): (1) 98.5% +/− 1.9% for Fixation, Orientation change; (2) 98.9% +/− 2.2% for Fixation, Shape change; (3) 97.7% +/− 2.2% for Saccade, Orientation change; and (4) 98.2% + /− 2.5% for Saccade, Shape change.

Region-of-interest analysis (II): cuneus

In a previous experiment18, we found that right and left cuneus showed significant saccade modulations and right cuneus showed a significant transsaccadic feature modulation for spatial frequency (with left cuneus showing a similar trend). Neither area showed an interaction effect (perhaps because that occurs at some higher level). If this area plays a more general role in transsaccadic perception of object identity, we expected to see the same modulations for shape change in the current task.

To test this, we followed similar procedures and statistics as used for SMG. Figure 3a, left and middle panels, show locations of the cuneus ROIs relative to saccade and field-specific activity, respectively, superimposed on coronal slices. Both sites fall within a zone that contains both significant saccade and field-specific modulations. Figure 3b shows the results of our specific hypotheses tests. In this case, cuneus did not show significant hemispheric differences (p > 0.05), so data from the bilateral pair were pooled. As predicted, this bilateral pair showed a significant main effect of eye movement (| S-F | F1,16 = 32.55, p = 3.25 × 10−5, η2 = 0.386) and feature.

Figure 3
figure 3

Region-of-interest (ROI) analysis results for tests of eye movement and feature effects in bilateral cuneus. (a) Transverse slices (Talairach X-coordinates provided) for left and right cuneus (yellow) that show saccade (leftmost) and visual field (middle) sensitivity. (b) Leftmost: B-weights differences (error bars: SEMs) from left (LH) and right (RH) cuneus are plotted (left bar: S: Saccade condition; F: Fixation condition; right bar: O: Orientation change condition; S: Shape change condition) and show significant eye movement and feature effects, as was expected from similar results in another transsaccadic perception study18. Middle and rightmost: B-weights for each of the four main conditions in left and right cuneus, respectively. Significant effects of eye movement and feature changes are shown with an asterisk (SO: Saccade, Orientation change condition; SS: Saccade, Shape change condition; FO: Fixation, Orientation change condition; FS: Fixation, Shape change condition).

(|O–S | F1,16 = 11.68, p = 0.015, η2 = 0.011). In this case, the directionality (analogous to same-different shape) was the same as what we observed for spatial frequency in a previous study18. Further, as in that previous study, we did not find a saccade-feature interaction (F1,16 = 0.54, p = 0.47, η2 = 0.003).

Functional connectivity of cuneus with visual and sensorimotor regions

In the final step of our analysis, we performed a functional connectivity analysis based on our right cuneus ROI. To be consistent with our previous studies (and be feature agnostic), we based this analysis on the Saccade-Fixation subtraction17,18. This excluded SMG (since it did not show significant saccade modulations in this dataset). And, to be consistent with our previous study18, we chose the right cuneus as the seed region. We then defined a sphere (radius = 5 mm) around this site and performed a psychophysiological interaction (PPI) analysis. This should identify a ‘network’ of cortical regions with similar saccade-related signals in this task.

This analysis revealed significant connectivity with (right) occipital, occipitotemporal, parietal, and frontal cortex (Fig. 4, Table 2). Specifically, we observed significant functional connectivity with early-to-intermediate visual occipital (right calcarine sulcus, CalcS, and superior occipital gyrus, SOG, as well as bilateral lingual gyrus, LG), sensorimotor parietal (right superior parietal lobule, SPL), oculomotor (left medial superior frontal gyrus, SFG, and frontal eye field, FEF) and spatial coding (right inferior frontal sulcus, IFS) frontal cortex.

Figure 4
figure 4

Functional connectivity network involved in transsaccadic updating of object features (orientation, shape). Using a Saccade > Fixation contrast, a psychophysiological interaction (PPI) contrast revealed statistically significant (RFX GLM; FDR (q < 0.05)) functional connectivity between right cuneus and frontal (right inferior frontal sulcus, IFS, left superior frontal gyrus, SFG, and frontal eye field, FEF), parietal (bilateral superior parietal lobule, SPL), and occipital (right calcarine sulcus, CalcS, left superior occipital gyrus, SOG, and bilateral lingual gyrus, LG) regions. (Note that these correlations do not necessarily imply direct anatomical connections.) Left: Transverse slice through the average brain of all participants (n = 17) shows activation in left frontal (SFG, FEF) and right parietal (SPL) cortex in yellow which depicts correlation with the seed region, right Cu.

Table 2 Functional connectivity network regions and Talairach coordinates resulting from psychophysiological interaction (PPI) analysis with right cuneus (seed region).

Discussion

Our aim here was to investigate whether transsaccadic discrimination of multiple features relies on the same or different mechanisms as those for single features. Specifically, we tested if transsaccadic mechanisms related to intrinsic (shape) and extrinsic (orientation) object properties would combine and generalize to a task that involves discriminating object orientation versus shape changes. To do this, we tested for saccade and feature sensitivity in four regions that showed both these properties in previous transsaccadic tasks involving single features (SMG, cuneus)17,18. Of these regions, right SMG and bilateral cuneus showed the predicted patterns of saccade and feature sensitivity. Of these, cuneus showed main effects (of eye movement and feature change) and visual field sensitivity, whereas only SMG showed interactions, perhaps suggesting lower-level versus higher-level mechanisms. Further, saccade-related activity in cuneus correlated with activity throughout visual, control, and saccade-related regions of cerebral cortex.

Before comparing these results to our previous transsaccadic perception results, it should be noted that those studies used a simple fMRI adaptation design, where a single object feature was repeated or changed after an intervening saccade / fixation period30,31. In the current experiment, participants had to track two features and detect which one had changed, so there was no ‘no change’ condition. This introduces some complications to the interpretation. For example, for an object orientation change, activation may be due to the (suppression) changes in orientation, whereas for the shape change condition, it is possible that the effect is driven by the repetition (enhancement) of the object orientation. Future exploration of these subtle differences is required. Nevertheless, our overall results generally agree with our previous experiments and hypothesis that SMG and cuneus participate in a complex network for transsaccadic retention of object identity and orientation.

Monolithic versus feature-specific transsaccadic mechanisms

Although some theoretical discussions of transsaccadic feature perception have suggested a role for occipital cortex6,7,48,49, initial experimental studies emphasized the parietal cortex as the key hub8,16,17,33. However, considering the diversity and regionalization of visual functions30,50,51, and the prevalence of saccades in normal vision52,53, the notion that the combination of saccades and vision would be handled by one specialized mechanism seems naïve. As noted in the introduction, transsaccadic updating of object location is found throughout visual and visuomotor cortex5,9,34,54, whereas transsaccadic perception of object orientation, spatial frequency, and shape appears to involve parietal, medial occipital, and in some cases, lateral occipital cortex16,17,18. The current results are also consistent with a more distributed system, with feature-specific modules.

Role of SMG in transsaccadic perception

Our previous experiments6,16,17 consistently implicated right SMG in transsaccadic processing of object orientation (see those papers and the ‘networks’ section below for discussions of right hemispheric specialization). Specifically, (1) stimulation of a site near right SMG disrupted transsaccadic memory of object orientation6, (2) different subsections of SMG were modulated by object orientation change during fixation versus saccades16, and (3) SMG showed saccade-orientation interactions within the same region of interest17. Based on these previous results6,16,17, we hypothesized that SMG should also show saccade-feature interactions in the current multi-feature task.

As predicted, in the current experiment right SMG (but not cuneus) showed saccade-feature interactions in the shape vs. orientation comparison. It is noteworthy that objects contain both local line orientation information, which can convey information about object shape/identity55,56,57, and global line orientation information, which can be a cue about the spatial orientation of the entire object. Both our conditions (object shape vs. orientation change) contained local feature changes, so their effects might cancel in the comparison, whereas global processes related to object orientation versus shape should differentiate. Based on the literature16,17,58,59, it is likely that our SMG interaction results were driven by changes in object orientation, rather than shape change. Taken together with other studies of spatial updating for perception and motor control (e.g.,10,22,40,60,61), this suggests a general role for parietal cortex in the transsaccadic spatial updating of the extrinsic spatial properties of objects: location, orientation, and their interactions6,7,17.

Possible role of medial occipital cortex in transsaccadic perception

In this and our previous study on spatial frequency18, dorsomedial occipital cortex appears to play a role in the postsaccadic integration of spatial frequency and object shape. The cuneus sites described here likely fall within a region spanning visual areas V2 and V362. These areas are ideally suited to provide the rudimentary machinery for transsaccadic feature analysis6,7,63,64.

First, they are well-positioned to provide rapid, bottom-up operations on their anatomic inputs from V165,66,67. Second, they include the requisite machinery for feature orientation, spatial frequency, and early shape processing68,69,70. Third, there are indications that even these areas are able to store information in some sense64. Fourth, these areas – V1, V2, and V3 – are known to show saccade modulations consistent with remapping signals23,29,63,64. Finally, these areas have reciprocal connections with both ‘ventral stream’ object recognition areas71,72,73, and dorsal stream saccade and attention areas74,75,76. This feedback, in combination with V1/V2/V3 retinotopic topography (absent at higher levels), makes this area also ideally suited for integrating location and identity information across saccades6,7,48.

Thus, situated at both early and intermediate stages of object feature processing, with the capability to monitor several object feature changes and receive feedback signals for attention and saccades, medial occipital cortex is an excellent, additional candidate for transsaccadic feature perception. The presence of separate main effects for gaze and feature modulation in cuneus (Fig. 3), compared to the interaction in SMG, suggests that cuneus might have an equally important but earlier position in the visual streams for transsaccadic perception. While the strength of the effect is smaller for object shape changes (see Results) compared to our previous study on the modulation of spatial frequency18, this region remains a candidate region for focused feature (object shape) change remapping77. Our experiments suggest that this area might play a specific role in transsaccadic processing of lower-level intrinsic object features related to identity, presumably working in concert with higher level ‘ventral stream’ mechanisms like lateral occipital cortex and inferior parietal cortex20,78,79.

Combining mechanisms for extrinsic and intrinsic feature updating

As noted above, transsaccadic perception would be somewhat useless if we could not update and link both object identity and location / orientation across saccades50,80. There is evidence that comparison between features evokes different mechanisms than the sum of the parts47. Based on the common properties of spatial frequency and shape (as intrinsic object features related to object identity), we speculated that both cuneus and SMG would be modulated by transsaccadic shape changes. In the experiment, both cuneus and SMG showed the expected modulations, although the saccade-feature interaction had the opposite directionality17. This suggests that the individual mechanisms for transsaccadic (object) orientation and shape updating combined, although not necessarily in a linear fashion. What remains to be understood, is how these various signals are combined – the classic ‘binding problem’81,82. While cuneus and SMG were not linked in the same functional network in the current study (likely because SMG did not show strong enough saccade modulations), there is enough overlap between their functional networks in different studies (see next section) to suggest that this is a possible linking mechanism.

Transsaccadic networks: task-specificity

Considering the complexity of transsaccadic signals discussed above, it seems likely that these mechanisms can be better understood at a network level. In our previous study, where participants were required to update object orientation for grasp across saccades17, they recruited a cortical network that included each element required for the task, including a transsaccadic (object) orientation updater (SMG), saccade signals (FEF), and grasp motor areas (anterior intraparietal and superior parietal cortex). Likewise, in the current study, we identified a network that seems eminently suitable for our feature discrimination task (Fig. 4): calcarine sulcus for early feature processing62,70,83, lingual gyrus and superior occipital gyrus for mid-level shape recognition84,85, posterior parietal cortex for saccade/reach updating signals9,10,22,86, and oculomotor superior frontal gyrus (likely pre-supplementary motor area)/frontal eye field27 and spatial coding in inferior frontal sulcus43. The differences in these studies might thus simply be due to task and feature details.

Finally, although we used right cuneus as a seed region here, it is noteworthy that its activity predominantly correlated with activity in right occipital and parietal cortex, with oculomotor activation in left hemisphere. Right cortex also predominated in our previous regional and network analyses16,17,18. Consistent with this, transsaccadic memory is most sensitive to right parietal stimulation and brain damage6,40,87. This is generally consistent with the role of parietal cortex in spatial attention and cognition and suggests that impaired transsaccadic perception might contribute to better-known disorders like hemispatial neglect88.

Conclusion

Here, we used fMRI to determine the cortical correlates of transsaccadic perception in a task where participants had to remember and discriminate changes in object orientation versus shape after a saccade. Our current findings support specific roles of two regions in transsaccadic feature discrimination: SMG for object orientation and medial occipital cortex for shape. Taken together with the previous literature, these results suggest that the cortical mechanisms for transsaccadic perception are both feature- and task-dependent. Specifically, there seems to be relative specialization for updating of object location and orientation in parietal cortex9,16,17,18,63,64 versus transsaccadic perception of intrinsic object features in medial occipital cortex (here and18). These areas appear to work in conjunction for more complex tasks, apparently forming ad-hoc functional networks with saccade centers (providing the updating signal) and higher order recognition and/or motor areas, as required for the task.

Materials and methods

Participants

To determine the minimum number of participants required to attain adequate statistical power, we used the effect size from right SMG, which is closest to predicted SMG here, found in the most recent of a series of transsaccadic perception studies17. We performed a power analysis (G*Power) using this effect size (0.887), with the following parameters: (1) two-tailed t-tests for the difference of matched pair means, (2) a desired power value of 0.90, and (3) an alpha-value of 0.05. On this basis, the suggested number of participants was 13, though we wanted to ensure full desired power, so we recruited 21 participants for this study, of whom we analyzed data from 17 participants (see below). The expected power was 0.914.

Twenty-one participants with normal or corrected-to-normal vision from York University, Toronto, Canada took part in this study. One participant’s data were excluded, as this person did not complete the experiment, and three participants’ data were excluded due to poor behavioural performance (i.e., less than 80% behavioural accuracy across all runs), which left 17 participants’ data for all further analyses. Participants were compensated monetarily for their time. All participants were right-handed (mean age: 28.7 +/− 5.2, 12 females). All participants provided written informed consent and did not have any neurological disorders. We confirm that this experimental protocol involving human participants was approved by and in concordance with the guidelines of the York University Human Participants Review Subcommittee.

Experimental set-up and stimuli

Participants were first required to pass MRI screening. Upon successful completion of this step, participants were introduced to the task. Once they felt comfortable with the requirements of the task, they were asked to assume a supine position on the MRI table. Their head lay in a 64-channel head coil. A mount on the head coil was used to support an infrared eye-tracker (for the right eye) to ensure appropriate fixation/eye movements, as well as a mirror that reflected the image of the screen from inside the MRI bore. iViewX software (SensoMotoric Instruments) was used to record eye movements for offline verification of correct behavior (see exclusion criteria above). Participants were provided with a button box that was held with the right hand (index and middle fingers positioned over the first and second buttons, respectively). Button press responses were analyzed offline.

The experiment was conducted under conditions of complete darkness. A fixation cross was always visible (except during the presentation of the mask) to direct participants’ gaze as required by the task. The fixation cross could appear approximately 7° from either the left or right edge of the gray screen (45.1° × 25.1°) along the horizontal meridian. The black stimulus always appeared in the center of the screen and was either a: (1) rectangle, (2) barrel-shaped object, or (3) hourglass-shaped object (Fig. 1a). The dimensions of the rectangle were 12° × 6°; the other two objects had the same area as the rectangle.

General paradigm/procedure

Experiment

To determine the cortical correlates of transsaccadic perception of object orientation versus shape, we used an event-related fMRI design (Fig. 1). Each trial commenced with a fixation cross, presented either to the left or right center, for 7.5 s. Then, the central object (rectangle, barrel-shaped, or hourglass-shaped object) appeared for 3 s at +/− 45° from vertical. This was followed by a static noise mask for 0.75 s to avoid an afterimage. The fixation cross would then appear either at the same position as before the mask (Fixation condition) or at the other location (Saccade condition) for 0.75 s. The same object presented in a different orientation (Orientation change condition) or one of the other two different objects presented in the same orientation (Shape change condition) appeared in the center for 3 s. The object disappeared and the fixation cross was replaced by an instruction (“I or O?”) for 3 s, indicating that participants should indicate if the two objects presented in the trial changed in shape (first button, ‘I’) or in orientation (second button, ‘O’). Participants provided button press responses with index finger (indicating shape changes) or middle finger (indicating object orientation changes). This close proximity of representation of these two fingers in motor cortex89 makes it unlikely that this motor response confounded any transsaccadic feature changes in earlier visual cortex. Thus, there were four main conditions: (1) Fixation, Orientation change, (2) Fixation, Shape change, (3) Saccade, Orientation change, or (4) Saccade, Shape change. These trial types were randomly intermingled within a run (24 trials); there were eight runs in total. Each run began and ended with central fixation for 18 s to establish baseline measures.

Imaging parameters

We used a 3 T Siemens Magnetom Prisma Fit magnetic resonance imaging (MRI) scanner. Functional experimental data were acquired with an echo-planar imaging (EPI) sequence (repetition time [TR] = 1500 ms; echo time [TE] = 30 ms; flip angle [FA] = 78 degrees; field of view [FOV] = 220 mm × 220 mm, matrix size = 110 × 110 with an in-slice resolution of 2 mm × 2 mm; slice thickness = 2 mm, no gap). A total of 312 volumes of functional data (72 slices) were acquired in an ascending, interleaved manner for each of the eight runs. In addition to the functional scans, a T1-weighted anatomical reference volume was obtained using an MPRAGE sequence (TR = 2300 ms, TE = 2.26 ms; FA = 8 degrees; FOV = 256 mm × 256 mm; matrix size = 256 × 256; voxel size = 1 × 1 × 1 mm3). 192 slices were acquired per volume of anatomical data.

Analysis

Behavioural data

Eye movements and button presses were recorded throughout the experiment; the former were monitored during the experiment and analyzed offline to ensure appropriate fixation and saccade production. If participants made eye movements when not required to, the data associated with those trials were excluded from further analysis. In addition, data for trials where incorrect button presses were made were excluded from any additional analysis. On these bases, 148 trials were removed (out of 3264 trials, representing 4.5%; maximum four trials were removed across all participants, with a mode of four) for the 17 participants.

Functional imaging data

Functional data from each run for each participant were preprocessed (slice time correction: cubic spline, temporal filtering: < 2 cycles/run, and 3D motion correction: trilinear/sinc). Raw anatomical data were transformed to a Talairach template90. Functional data were co-registered via gradient-based affine alignment (translation, rotation, scale affine transformation). Finally, data were smoothed using a Gaussian kernel of 8 mm full width at half maximum. On the basis of missing predictors (more than 50% of trials with incorrect behavioral response), nine runs across all participants were removed from the overall random-effects general linear model (RFX GLM; i.e., 216 trials out of 3116 trials; 6.9%).

Four separate GLMs were generated for each participant for every run. The first set of GLMs contained a baseline predictor for the first 18 s of each run (Baseline). The beginning of each trial during which fixation occurred was modeled with a 7.5 s ‘Fixate’ predictor (period prior to the Sensory/Encoding phase). The remainder of each trial (12 s) was modeled with one of four main predictors: (1) Fixation, Orientation change, for an entire trial during which participants maintained fixation throughout while only the orientation of the same object changed; (2) Fixation, Shape change, for a trial during which participants maintained fixation while only the shape of the object changed; (3) Saccade, Orientation change, for a trial during which participants had to make a saccade while only the orientation of the object changed; and (4) Saccade, Shape change, for a trial where participants made a saccade while only the shape of the object changed. The first presentation of the stimulus was referred to as the Sensory/Encoding phase and the second presentation period was referred to as the Change Detection phase. These GLMs were created in BrainVoyager QX 20.6 (Brain Innovation), where each of the boxcar predictors was convolved with a haemodynamic response function (standard two-gamma function model)91. GLMs were additionally adjusted via the addition of a last, confound predictor (“Error”) to account for any errors made during trials due to unwanted eye movements or incorrect button responses.

The second set of GLMs were structured such that, in addition to the Baseline predictor (18 s) at the beginning and end of each run and the Fixate predictor for the beginning of each trial (7.5 s), the Sensory/Encoding phase was separated from the rest of the trial. Two predictors (3 s each) were used to model for visual field effects: (1) “LVF” for when the object appeared in the left visual field relative to fixation and (2) “RVF” for trials where the object appeared in the right visual field relative to fixation. The rest of the trial was modeled with “Fix” or “Sacc” predictors (9 s) to indicate if it was a Fixate or Saccade trial, respectively. An “Error” confound predictor was included to model trials where eye movement or button press errors had been made.

The third set of GLMs contained changes only to the predictors for the Sensory/Encoding phase, such that object orientation-sensitive responses during the first stimulus presentation could be tested. As such, GLMs were modeled exactly as in the second set of GLMs, except that two predictors (3 s each) modeled the orientation of the object upon first presentation during the Sensory/Encoding phase: (1) “45°” for objects that were initially presented at 45° and (2) “135°” for objects that were initially presented at 135° from horizontal. A confound predictor (“Error”) was also included for eye movement or button press errors.

The last set of GLMs were modified to determine whether there was an initial shape-related bias upon first stimulus presentation. Like in the previous two sets of GLMs, the Sensory/Encoding phase was modeled with one of three predictors (3 s each), one for each of the three object shapes: (1) “Rect” for when the rectangle was presented during this phase; (2) “Barrel” for the barrel-shaped object; and (3) “Hourglass” for the hourglass-shaped object. This was the only change and the other predictors were modeled like in the two previous sets of GLMs (Baseline, Fixate, and Fix/Sacc predictors). Finally, the “Error” predictor was included for erroneous eye movements and/or button presses.

Region-of-interest (ROI) analysis

A hypothesis-driven region-of-interest (ROI) analysis was performed based on Talairach coordinates obtained our previous studies17,18. Specifically, we used the previously identified coordinates of peak modulation for transsaccadic object orientation changes (right SMG) and spatial frequency changes (right cuneus), which we hypothesized would generalize to transsaccadic shape detection if cuneus plays a more general role in object identity. We then mirrored these coordinates laterally to obtain four ROIs (i.e., bilateral SMG and cuneus; Table 1). ROIs were created around these coordinates as spheres with 5-mm radius using BrainVoyager QX v2.8 (Brain Innovation). β-weights were then extracted and inspected for hemispheric differences with a two-tailed repeated-measures t-test. If a significant result was obtained, the β-weights from the left and right hemispheres were analyzed separately for each of the two criteria (see Results for details). On this basis, we pooled data only for cuneus. Within each of the ROIs, we ran a repeated-measures analysis-of-variance (RM-ANOVA) with two factors: Eye movement (2 levels: Fixation, Saccade) and Object feature (3 levels: Rectangle, Barrel, Hourglass). See Results and Table 1 for further information.

Whole-brain univariate analysis for saccade and visual field sensitivity.

We performed a whole-brain analysis to determine the presence of saccade sensitivity within the ROIs. For these univariate analyses, we applied four independent contrasts to test for four effects. First, we performed a whole-brain voxelwise Saccade > Fixation contrast to identify cortical regions that were modulated by saccades in this task (Figs. 2, 3). Second, we applied a Left > Right Visual Field contrast using the predictors for the Sensory/Encoding phase to find regions that are sensitive to perception of objects within the respective visual fields (Figs. 2, 3). To each of the resulting volumetric maps, we applied False Discovery Rate correction to account for multiple comparisons (q < 0.05) BrainVoyager QX v2.8, Brain Innovation).

Psychophysiological interaction

Our last goal was to identify the functional network for saccade-related updating during transsaccadic perception. In order to do this, we conducted psychophysiological interaction (PPI) analysis17,91,92,93 on data extracted from the cuneus peak site (Talairach coordinates: 6, − 79, 15). For optimal signal processing, we used a sphere of radius = 5 mm around this site. To perform this analysis, we used three predictors: (1) a physiological component (which is accounted for by z-normalized time-courses obtained from the seed region for each participant for each run); (2) a psychological component (task predictors, i.e. for the Saccade and Fixation conditions, were convolved with a hemodynamic response function), and (3) a psychophysiological interaction component (multiplication of seed region z-normalized time-courses with task model in a volume-by-volume manner). For the psychological component, the Saccade predictors were set to a value of ‘ + 1’, whereas the Fixation predictors were set to a value of ‘− 1’; all other predictors were set to a value of ‘0’ in order to determine the responses to Saccades as compared to Fixation. (For greater detail, please see O’Reilly et al.93 and17). We created single design matrices for each run for each of the 17 participants, which were ultimately included in an RFX GLM. Then, we used the third, psychophysiological interaction predictor to determine the regions that constitute the functional network that shows saccade-related modulations for feature updating. To the resulting volumetric map, we applied FDR correction (q < 0.05) (Fig. 4).