Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

## Abstract

Perceptual illusions help to understand how sensory signals are decoded in the brain. Here we report that the opposite approach is also applicable, i.e., results from decoding neural activity from monkey extrastriate visual cortex correctly predict a hitherto unknown perceptual illusion in humans. We record neural activity from monkey medial superior temporal (MST) and ventral intraparietal (VIP) area during presentation of self-motion stimuli and concurrent reflexive eye movements. A heading-decoder performs veridically during slow eye movements. During fast eye movements (saccades), however, the decoder erroneously reports compression of heading toward straight ahead. Functional equivalents of macaque areas MST and VIP have been identified in humans, implying a perceptual correlate (illusion) of this perisaccadic decoding error. Indeed, a behavioral experiment in humans shows that perceived heading is perisaccadically compressed toward the direction of gaze. Response properties of primate areas MST and VIP are consistent with being the substrate of the newly described visual illusion.

## Introduction

Visual perceptual illusions have been successfully employed to better understand how sensory signals are processed in the brain1. Often, such illusions allow to infer at what stage of the processing hierarchy certain computational steps take place, and resultant predictions can be validated experimentally. In the motion domain, examples include the perception of coherent motion2, the motion aftereffect3, and implied motion4 as encoded in cortical areas MT (the middle temporal area) and MST (the medial superior temporal area). In the spatial domain, many perceptual illusions are induced by fast ballistic eye movements, so-called saccades. Behavioral studies in humans have shown that briefly flashed stimuli that are presented in the temporal vicinity of saccades are mislocalized in a characteristic fashion. Dependent on the exact experimental condition, mislocalization can be described as a dynamic, global shift of space or as a compression of perceived space toward the saccade target5,6,7. Neurophysiological studies in macaque monkeys have identified neural correlates of both dynamic error patterns8,9,10,11,12.

In everyday life, visual motion and eye movements often occur concurrently. As an example, self-motion through an environment induces reflexive, optokinetic like eye movements, consisting of an alternation of smooth tracking phases and saccades13, 14. It was shown before that self-motion direction (heading) can be decoded from neural responses during fixed gaze15, 16. Accordingly, the question arises, if such decoding is also possible during eye movements. Previous studies have provided evidence for this idea by demonstrating that the neural tuning for heading in areas MST and VIP is invariant with respect to slow eye movements17,18,19. Such an eye movement invariance can be considered a prerequisite for the successful decoding of self-motion direction. Here we test this hypothesis and decode heading during smooth eye movements and during saccades.

We find that a linear heading-decoder performs veridically during slow eye movements. During saccades, however, the decoder incorrectly indicates compression of heading toward straight ahead. In a second experiment, we test whether these perisaccadic changes in the readout of the decoder have consequences on perception of human subjects. As predicted, perceived heading is perisaccadically compressed. A behavioral control experiment reveals compression to be directed toward the direction of gaze rather than the head- or body-midline. Response properties of primate areas MST and VIP are likely the neural substrate of this newly described visual illusion.

## Results

### Decoding of heading across eye movements

We recorded activity from 119 neurons in the medial superior temporal area (area MST) and the ventral intraparietal area (area VIP) of three macaque monkeys (Supplementary Table 1 for details). During the recording monkeys freely viewed long sequences of optic flow stimuli simulating self-motion across a ground plane in one of three directions (±30° and straight ahead, Fig. 1a).

Such visual stimuli induce reflexive, optokinetic-like eye movements, i.e., an alternation of slow tracking and fast (saccadic) eye movements. Figure 1b shows representative eye-movement traces in the middle and bottom row along with neural activity (top row). As a first step of our data analysis, we determined the onset times of each saccade during a given trial. The perisaccadic activity for a given self-motion direction was averaged in order to determine the response dynamics of a neuron under study. By definition, each response window represents neural activity during an alternation of slow-tracking eye movements (from −200 to 0 ms), followed by a saccade (from 0 to about 60 ms), followed again by slow-tracking eye movements (from about 60 to 450 ms). Figure 1c shows data from the same neuron from area MST, whose activity during a leftward heading was shown in Fig. 1b. For this neuron, mean discharges were lowest for self-motion to the left (left panel) and highest for self-motion to the right (right panel). For all three self-motion directions, neural discharges were rather constant during the slow eye-movement phases. Shortly after the saccades (indicated by the red vertical line in each panel), however, neural activity was briefly suppressed. Data from all neurons from areas MST and VIP were analyzed accordingly.

In a next step, we implemented an algorithm to decode self-motion direction from neural discharges (see Methods for details). In our decoding approach, firing rate directly translates into heading. We applied the decoder to the continuously recorded discharge of each neuron that showed a statistically significant tuning for heading (ANOVA on ranks, 2 df, P < 0.05, see Methods for details), i.e., 71/119 = 60% of the neurons. (Detailed cell numbers are given in Supplementary Table 1).

The decoding approach provided us with a time-resolved estimate of self-motion direction based on the discharges of a population of cells (n = 71). We determined heading from 200 ms before until 450 ms after the saccade in steps of 20 ms as the median value of the heading estimates from all neurons (Fig. 2a).

The colored symbols and solid lines indicate the decoded heading. Starting at the earliest time, i.e., 200 ms before the saccade, decoded heading for all three self-motion directions was close to veridical up to 100 ms before saccade onset. Then, however, it began to collapse. One-hundred ten milliseconds after saccade onset, compression toward straight ahead was at maximum, with decoded values being close to zero for all three headings. Thereafter, decoded values rapidly recovered and became close to veridical again. Maximum compression as indicated by the minimum of the standard deviation was observed 110 ms after saccade onset (Fig. 2b).

Given the results shown in Fig. 1c, we next investigated whether the compression of the representation of heading, as observed in the decoding of neural activity from monkey areas MST and VIP, could merely be a by-product of an overall perisaccadic reduction of neuronal excitability, i.e., saccadic suppression. Such saccadic suppression has been shown previously for areas MST and VIP20. Hence, we determined the time course of the average neural activity, aligned to saccade onset (Fig. 3a).

The time courses of saccadic suppression, as shown here, and the time course of compression of decoded heading as shown in Fig. 2b were indeed very similar, although not identical. Maximum suppression was observed 80 ms after saccade onset, i.e., 30 ms earlier than the maximum compression. This might be considered first evidence that a mere saccadic suppression cannot account for the observed compression of the decoded representation of heading.

To further understand the neural basis of the perisaccadic decoding error, we built a rather qualitative model. Here we considered a grand population of neurons from areas MST and VIP to be split up into two subpopulations: neurons that were tuned for leftward headings constituted “Subpopulation 1”. By definition, the slope m of the regression function fitted to the heading responses of these neurons was smaller than zero (m < 0). “Subpopulation 2” comprised those neurons that are tuned for rightward headings (m > 0). This procedure is graphically illustrated in Fig. 4a. Published data predict that from such a grand population of MST and/or VIP neurons, roughly the same number of neurons should fall into one of the two subpopulations15, 17, 21,22,23. Indeed, in our data set of n = 71 neurons, n Subpop1 = 38 neurons were tuned for leftward self-motion (m < 0), while n Subpop2 = 33 were tuned for rightward self-motion (m > 0).

For our decoding approach, we had employed linear regression as approximation for the real neurons’ response tunings. We applied this linear decoder also to our qualitative model. This is illustrated in Fig. 4b. Here we assumed presentation of a rightward self-motion stimulus. Accordingly, the activity of the neurons tuned for rightward heading is high (Subpopulation 2, light blue line), while the activity of neurons tuned for leftward heading is low (Subpopulation 1, pink line). The observed activity distribution of neurons from both subpopulations translates into a veridical heading estimate, i.e., self-motion to the right, as indicated graphically by the purple arrow.

This qualitative model led us to hypothesize the occurrence of selective suppression and release from inhibition in our data set. Accordingly, we analyzed in greater detail the individual heading responses of the neurons in our population. Indeed, we found evidence for a suppression of responses for one movement direction and a slight response enhancement for the opposite direction. Two examples are shown in Fig. 5.

In order to quantify this effect at the population level, we assigned the response of each neuron to leftward and rightward heading either to the congruent or incongruent condition, dependent on the neuron’s tuning for heading. Importantly, each neuron contributes to both conditions: leftward tuned neurons contribute to the congruent condition during leftward self-motion and also to the incongruent condition during rightward self-motion (vice versa for rightward tuned neurons). Then, we tested for a significant difference of the perisaccadic response modulations in the congruent and incongruent condition by comparing the neural activity in a fixed, post saccadic window – = 40–160 ms after saccade onset. In Fig. 5, in each panel, saccade onset is indicated by the vertical, solid line. The time window for analysis [40 ms 160 ms] is indicated by the pair of vertical red dashed lines in each panel.

This window was considered wide enough to capture both, suppressive and dis-inhibitory effects. We compared the neural activity in this time window with activity long after the saccade (300–450 ms after saccade onset) by computing the ratio of the two, defining a modulation index. Based on this definition, an index of 1.0 indicates no perisaccadic response modulation. An index smaller than 1.0 indicates response suppression, while an index >1.0 indicates response enhancement. Our prediction from the hypothesis detailed above was an index below 1.0 for the congruent condition and an index above 1.0 for the incongruent condition. This hypothesis was confirmed in our population data (n = 71 neurons, Fig. 6). The 95% confidence intervals as determined from bootstrapping were below 1.0 in the congruent condition and above 1.0 in the incongruent condition.

In a final step, we modeled the effect of saccadic suppression in a quantitative model. The advantage of this approach is that one can apply various forms of perisaccadic response modulation and determine, if and if so which form of suppression reproduces the effect of a compression of decoded heading. For this model, we created a set of n = 71 “virtual neurons”, i.e., time courses of neural activity. The activity levels were modeled after our real data set, i.e., activity distribution across the population was identical to our real data set, with additional noise. Importantly, model neurons could not have negative firing rates. Instead, all firing rates, which would have been negative, were set to zero. We simulated presentation of five self-motion directions (−30°, −15°, 0°, 15°, and 30°). These artificial neurons were linearly tuned to self-motion direction, with the distributions of strengths of their tunings modeled after our real data set. The results are shown in Fig. 7. Obviously, only selective suppression, i.e., suppression of the neurons in the congruent condition combined with a release from inhibition of the neurons in the incongruent condition (top right panel), but not global suppression triggers a compression of decoded heading similar to the one found from our real neurons (Fig. 2), with a maximum of suppression around 100 ms after saccade onset.

In summary, our analysis clearly shows for the first time that heading can be decoded robustly and reliably during smooth eye movements from neural activity in macaque areas MST and VIP. This is remarkable given the rather small sample size of only 71 neurons. Saccades, however, induce a systematic error of decoded heading toward straight ahead (red and green symbols and data curves in Fig. 2a). Since functional equivalents of monkey areas MST and VIP have been identified in humans (MST;25 VIP26), we asked whether the decoding from monkey neurophysiological data would correctly predict human behavior, i.e., whether or not humans perceive compressed heading across saccades. This result would be noteworthy for at least two reasons: firstly, neurophysiological and computational results from the animal model would have correctly predicted a hitherto undescribed visual perceptual illusion. Secondly, it would provide further and strong evidence for the similarity of visual motion processing in humans and non-human primates.

In our psychophysical study, subjects were presented brief (40 ms) sequences of visual stimuli simulating self-motion across a ground plane (Fig. 8). In separate blocks, subjects either kept central fixation or performed a visually guided 10° upward saccade along the vertical meridian.

Quantitatively, the temporal dynamics of perceived heading were also very consistent across all eight subjects: peak compression occurred between −10 and −1 ms relative to saccade onset, with a mean value of t = −8.25 ms, as shown in Fig. 9c. Given the similarity of perisaccadic heading compression as determined from our neurophysiological and psychophysical data, we employed temporal cross-correlation to determine the best overlap between the two time-courses. A shift of τ = −116 ms of the neurophysiological with respect to the psychophysical data resulted in the largest correlation coefficient (Fig. 9d). At first glance, it might appear surprising that the behavioral data lead the neurophysiological by 116 ms. Yet, this has to be expected because the value of tau corresponds to a visual response latency and is perfectly in line with values in higher visual cortical areas of the macaque27.

Previous studies on perisaccadic modulation of visual perception showed that response precision of behavioral responses did not decrease perisaccadically28. Accordingly, we were also interested in the precision of the decoded headings as well as of the behavioral responses of our subjects (Supplementary Fig. 2, with behavioral data from each subject shown in Supplementary Fig. 3). Precision did not modulate perisaccadically, neither for the decoded heading nor for the behavioral responses. This finding is in line with the above mentioned behavioral studies.

The neurophysiological data had provided evidence for an eye-centered reference frame of the compression effect. This leads to the hypothesis that the center of the compression should move with the eye but not the head, if both were changed independently from each other.

We decided to test this hypothesis in a further control experiment using a heading task instead of a forward/backward discrimination. Different from the main experiment, however, either (i) the subjects’ head was directed α = 15° to the left, while gaze was still straight ahead, or (ii) the head was straight ahead, but eyes were directed α = 15° to the left (Fig. 10). Clearly, we observed a perisaccadic shift of perceived heading toward the screen center in the first condition (head to the left, Fig. 10a), but a shift toward −15°, when the eyes were directed to the left (Fig. 10e). Hence, compression occurred in eye-centered coordinates toward the direction of gaze.

## Discussion

We have investigated the response properties of neurons in macaque extrastriate (area MST) and parietal (area VIP) cortex and developed a model to decode self-motion direction from neural discharges. This decoder performed veridically during fixation and slow eye movements, but it revealed a characteristic error in the temporal vicinity of saccades. This error pattern led us to hypothesize that perceived heading should be compromised by saccadic eye movements. A behavioral study in human observers confirmed this idea.

Visual perception is often far from being veridical. Instead, modulatory influences shape the processing and lead to a percept that can be significantly different from its physical counterpart32, 33. Eye movements are often the origin of such modulatory influences34. While eye-movement induced visual illusions lead to a (temporary) misperception of the outside world, they allow for a deeper understanding of the neural basis of visual processing in humans and in the animal model, i.e., the macaque monkey9, 10, 20, 35. In our current study, however, we took the opposite approach. We started off at the neural level of the macaque and correctly predicted a hitherto unknown visual illusion in humans. Hence, our physiological and behavioral results provide further strong support for the idea that areas MST and VIP are critically involved in the processing of self-motion information in both humans and monkeys. The functional similarities between the species extended from decoding neuronal responses in monkeys to a prediction for human self-motion processing that was confirmed behaviorally.

At first glance, our finding of a central bias appears at conflict with the results from refs. 36 and 37. Yet, given the complete different experimental settings in these previous and our current study, it might not be appropriate to put both results in contrast. Stimuli in our study were extremely short (40 ms), while those employed in the other two studies lasted beyond one second. It is known, e.g., from literature on localization of briefly flashed stimuli that these are perceived closer to the fovea than they actually were7, 40, 41. Yet, when targets are presented continuously, overestimation of perceived location has been reported42. In other words: in similar tasks, a bias can strictly depend on the experimental conditions.

References 36, 37 suggested that the observed centrifugal bias might result from a non-uniform distribution of heading preferences in primate areas MST and VIP, as described in the macaque among others, e.g., by Angelaki, DeAngelis and colleagues (MST16 VIP43). In their study on area MST, these authors have investigated accuracy and precision of decoded heading from discharges of monkey area MST and found a decoding bias toward the periphery. Indeed, long before and especially long after a saccade, also our decoding from areas MST and VIP revealed a (slight) centrifugal bias (Fig. 2). Accordingly, during slow eye movements, our data are in good agreement with previous results obtained during steady fixation.

In our data analysis, we have employed linear regression analysis for quantifying the neurons’ tuning for self-motion direction. This approach is in line with previous studies22. Other studies have employed sigmoidal15 or more complex tuning functions16 to model heading selectivity. In our study, we have tested a limited range of heading directions (±30°). This range did not allow to determine (i) response saturation values, as required for fitting a sigmoidal function, or (ii) local maxima or minima, as would have been necessary for fitting a rather complex tuning function. Hence, given a set of three data points within the central ±30º, a linear function appeared to be a well-motivated and plausible statistical model, which allowed quantitative modeling of the effects of saccades on the representation of heading information.

In a first behavioral control experiment subjects had to discriminate whether a brief self-motion stimulus simulated forward or backward self-motion. It is known from the literature that saccades impair the perception of visual motion29. Given that self-motion sequences in our study were extremely short (40 ms) we had to exclude the possibility that subjects simply were not able to perceive our stimuli due to saccadic suppression of the magnocellular processing of the visual system44. Obviously, the nature of this control experiment was different from our main experiment, in which subjects had to indicate their perceived self-motion direction. Nevertheless, the results of the control experiment clearly showed that subjects were not literally blind for the self-motion stimuli used in our experiment.

Our finding that the perception of heading from an optic flow field is compressed during a saccade is reminiscent of perisaccadic compression of perceived space, time, and numbers48. Like in these other perceptual illusions, the effect of a compressed perception of heading started well before the eyes started to move. This time course argues for an active visual process rather than a passive, purely visually induced effect. Similar conclusions have been drawn concerning other perisaccadic modulations of visual perception6, 20, 28, 49,50,51,52.

While the temporal dynamics of the compression effects are rather similar across various affected measures (among them space, time, and number), our results cannot be explained by these compressions. Instead, they add a further dimension, namely that of self-motion. Space, time, numbers, and motion are all fundamental representations within primate parietal cortex. As deduced from our model, a common mechanism for these perisaccadic illusions may consist of selective modulation of activity in these representations by saccades.

## Methods

### Monkey physiology

All procedures had been approved by the regional ethics committees (Regierungspräsidium Arnsberg; Regierungspräsidium Gießen) and were in accordance with the published guidelines on the use of animals in research (European Communities Council Directive 86/609/ECC). Experiments had previously been performed in three trained male macaque monkeys (C, H, and R)18, 19. We recorded neural activity from the medial superior temporal area (area MST) in two hemispheres of monkeys H and R and from the ventral intraparietal area (area VIP) in two hemispheres of monkeys C and H. Identification of areas was based on MR images, chamber position, electrode depth, preference for flow fields as well as large visual receptive fields. Eye movements were monitored with a search eye-coil system (Skalar Medical BV, Delft, the Netherlands), running at 500 Hz. Further details on recording procedures and animal preparation can be found in ref. 20.

Optic flow stimuli were generated in OpenGL with a frame-rate of 72 Hz. The stimuli consisted of large-field computer-generated sequences that were back projected via a video projector (Electrohome) onto a tangent screen, 48 cm in front of the monkey. The size of the projection covered the central 90° × 90° of the visual field. Optic flow sequences (15–20 s) simulated self-motion of a virtual observer over an extended horizontal plane covered with a texture pattern, located 37 cm below eye level (Fig. 1a). Trials simulated self-motion in one of the three directions: 30° to the left, straight ahead, and 30° to the right.

We recorded activities from 119 neurons in areas MST (n 1 = 64; 55 neurons from monkey H and 9 neurons from monkey R) and VIP (n 2 = 55; 36 neurons from monkey H and 19 neurons from monkey C) while the monkeys freely viewed long self-motion sequences, lasting between 15 and 20 s. In order to identify those neurons that are significantly tuned to heading, we split each trial in 1000 ms slices and computed the neuron’s activity for each temporal slice. Then, we employed an ANOVA on ranks with two degrees of freedom (2 df) on the responses in these temporal slices for the three different self-motion directions.

In primates, self-motion stimuli induce reflexive, optokinetic-like eye movements13, 14, 53. For further data analysis, we determined periods of slow tracking and saccades. Saccade onset was determined offline by a speed criterion (80°/s). Tracking phases in which eye position was outside the central 20° of the screen were excluded from further analysis to avoid possible contamination of neural activity by eye-position signals or visually induced screen-border effects. From the remaining data set, all tracking phases were temporally aligned to the onset of the preceding saccade. Average neural activity in 20 ms bins was determined across these tracking phases for each of the three self-motion directions.

During fixation, the tuning of neurons in areas MST15, 16 and VIP22 for forward-directed self-motion within the central ±45º can be approximated by 2D linear or sigmoidal functions. Given the sparse sampling of simulated horizontal forward self-motion in our experiment (−30°, 0°, and +30°), we used linear regression to approximate the cell’s response to heading:

$$y\left( t \right) = a \times x\left( t \right) + b$$

Here, “y(t)” represents the time resolved neural activity, “a” depicts the slope of the regression function, x(t) depicts the current heading direction, and b represents the intercept of the regression function, or mean activity of the neuron. We used a brief, 150 ms post-saccadic interval (300–450 ms) of each neuron’s response to the three simulated headings to compute the linear regression parameters.

We determined time-resolved decoded heading for each neuron for each of the three self-motion directions by relating activity in 20 ms bins to the linear regression, i.e., neural activity was directly translated into a heading estimate by solving the inverse equation

$$x\left( t \right) = \left( {y\left( t \right) - b} \right)/a$$

We employed bootstrapping (Matlab function bootci, with 1000 repetitions) to determine the time-resolved 95% confidence intervals of neuronal response modulations (Fig. 3a, b) and of the precision of heading-decoding and -perception (Supplementary Fig. 2).

For the quantitative model on the role of saccadic suppression on heading-decoding, we convoluted the time courses of the artificial neurons with the response profiles deduced from global suppression or selective suppression, respectively (Fig. 3). To this end, we normalized the time courses of global or selective suppression, so that the values >100 ms before and after the saccade were centered on 1.0. In the one case, i.e., global suppression, all time courses of neural activity were convoluted with this global suppression function. In the other case, we applied selective suppression, i.e., we convoluted neurons with the suppression function in case of a congruent stimulus condition (blue curve) and with the release-from-inhibition function in the incongruent stimulus condition (magenta curve).

### Human psychophysics

We recorded eye movements and behavioral data from eight human subjects (three female; 19–28 years). Experiments were approved by the local ethics committee at the Department of Psychology. Participants had normal or corrected to normal vision and gave their written informed consent prior to the experiments, which were in accordance with the Declaration of Helsinki.

In saccade trials, saccade onset was determined offline by a speed criterion (80°/s). All data (stimuli and eye movements) were aligned to saccade onset. For each subject, we first determined the time course of heading perception for each of the five self-motion directions. In line with previous studies7, compression of heading was defined as the standard deviation of the time courses of the five perceived headings normalized with respect to its average value up to 100 ms before and after the saccade. The time of maximum compression, i.e., the time point for which the standard deviation reached its minimum value, was determined for each subject.

In order to determine the best overlap for the two time-courses of heading compression as obtained from the neurophysiological (monkey) and behavioral (human) data, we employed a temporal cross correlation analysis. The temporal shift between the two time-courses resulting in the highest cross-correlation value was considered the neuronal processing latency.

Motion perception across saccades is impaired (saccadic suppression)29, 31. Nevertheless, even brief periods of visual motion can be perceived perisaccadically30. Accordingly, in a first control experiment, we aimed to determine whether or not this reduction of the perisaccadic perception of motion had a critical influence on our behavioral data. To this end, we presented to five of the eight subjects the same self-motion stimuli as before, but now simulating straight forward or backward self-motion, again with motion onsets between 200 ms before and 200 ms after saccade onset. In a 2-AFC task, subjects had to indicate whether they had perceived forward or backward self-motion.

In addition, we determined effect sizes of the perisaccadic response modulation, i.e., d′. We computed d′ as difference of the means of (i) discrimination performance as obtained >50 ms before or after saccade onset $${\bar x_{{\rm{pre\& post}}}}$$ and (ii) perisaccadic discrimination performance $${\bar x_{{\rm{peri}}}}$$ (i.e., onset of the self-motion stimuli <50 ms before or after saccade onset), divided by the combined standard deviation of these two activity patterns Stdcombined 55, 56.

$${d^{'}} = \frac{{{{\bar x}_{{\rm{pre\& post}}}} - {{\bar x}_{{\rm{peri}}}}}}{{{\rm{St}}{{\rm{d}}_{{\rm{Combined}}}}}},\,{\rm{with}}\,{\rm{St}}{{\rm{d}}_{{\rm{combined}}}} = \sqrt {\frac{{\left( {{n_1} - 1} \right){\rm{std}}_1^2 + \left( {{n_2} - 1} \right){\rm{std}}_2^2}}{{{n_1} + {n_2} - 2}}}$$

Here, n 1 and n 2 are the number of samples in the pre- and post-saccadic intervals (n 1) as well as in the perisaccadic interval (n 2). std1 and std2 are the standard deviations of these samples.

In a second control experiment, we aimed to determine the reference frame of the compression of perceived heading. Here, like in the main experiment, subjects performed a 10° upward saccade, and perisaccadically, we presented a 40 ms long self-motion sequence. Different from the main experiment, however, either (i) the subjects’ head was directed α = 15° to the left, while gaze was still straight ahead, or (ii) the head was straight ahead, but eyes were directed α = 15° to the left. This experiment tests four possibilities for the center of compression. If compression is directed to the head’s midline (variant A), one would expect a shift of the center of compression in the first case (head to the left), but not in the second (eyes to the left). If compression is directed toward the body midline (variant B) or the screen center (variant C) one would expect that the center of compression is always at the center of the screen. If compression is directed toward the center of gaze (variant D), one would expect compression toward the screen center when gaze was straight ahead and toward a position 15º to the left when gaze was at that position.

### Code availability

MATLAB code used for analysis is available from the corresponding author on reasonable request.

### Data availability

The data sets generated and analyzed for the current study are available from 10.5281/zenodo.837063.

## References

1. 1.

Jazayeri, M. & Movshon, J. A. A new perceptual illusion reveals mechanisms of sensory decoding. Nature 446, 912–915 (2007).

2. 2.

Pack, C. C. & Born, R. T. Temporal dynamics of a neural solution to the aperture problem in visual area MT of macaque brain. Nature 409, 1040–1042 (2001).

3. 3.

Tootell, R. B. et al. Visual motion aftereffect in human cortical area MT revealed by functional magnetic resonance imaging. Nature 375, 139–141 (1995).

4. 4.

Krekelberg, B., Dannenberg, S., Hoffmann, K.-P., Bremmer, F. & Ross, J. Neural correlates of implied motion. Nature 424, 674–677 (2003).

5. 5.

Cai, R. H., Pouget, A., Schlag-Rey, M. & Schlag, J. Perceived geometrical relationships affected by eye-movement signals. Nature 386, 601–604 (1997).

6. 6.

Ross, J., Morrone, M. C. & Burr, D. C. Compression of visual space before saccades. Nature 386, 598–601 (1997).

7. 7.

Lappe, M., Awater, H. & Krekelberg, B. Postsaccadic visual references generate presaccadic compression of space. Nature 403, 892–895 (2000).

8. 8.

Duhamel, J., Colby, C. L. & Goldberg, M. E. The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255, 90–92 (1992).

9. 9.

Krekelberg, B., Kubischik, M., Hoffmann, K.-P. & Bremmer, F. Neural correlates of visual localization and perisaccadic mislocalization. Neuron 37, 537–545 (2003).

10. 10.

Morris, A. P., Kubischik, M., Hoffmann, K.-P., Krekelberg, B. & Bremmer, F. Dynamics of eye-position signals in the dorsal visual system. Curr. Biol. 22, 173–179 (2012).

11. 11.

Zirnsak, M., Steinmetz, N. A., Noudoost, B., Xu, K. Z. & Moore, T. Visual space is compressed in prefrontal cortex before eye movements. Nature 507, 504–507 (2014).

12. 12.

Wang, X. et al. Perisaccadic receptive field expansion in the lateral intraparietal area. Neuron 90, 400–409 (2016).

13. 13.

Lappe, M., Pekel, M. & Hoffmann, K.-P. Optokinetic eye movements elicited by radial optic flow in the macaque monkey. J. Neurophysiol. 79, 1461–1480 (1998).

14. 14.

Angelaki, D. E. & Hess, B. J. Self-motion-induced eye movements: effects on visual acuity and navigation. Nat. Rev. Neurosci. 6, 966–976 (2005).

15. 15.

Lappe, M., Bremmer, F., Pekel, M., Thiele, A. & Hoffmann, K.-P. Optic flow processing in monkey STS: a theoretical and experimental approach. J. Neurosci. 16, 6265–6285 (1996).

16. 16.

Gu, Y., Fetsch, C. R., Adeyemo, B., DeAngelis, G. C. & Angelaki, D. E. Decoding of MSTd population activity accounts for variations in the precision of heading perception. Neuron 66, 596–609 (2010).

17. 17.

Maciokas, J. B. & Britten, K. H. Extrastriate area MST and parietal area VIP similarly represent forward headings. J. Neurophysiol. 104, 239–247 (2010).

18. 18.

Bremmer, F., Kubischik, M., Pekel, M., Hoffmann, K.-P. & Lappe, M. Visual selectivity for heading in monkey area MST. Exp. Brain. Res. 200, 51–60 (2010).

19. 19.

Kaminiarz, A., Schlack, A., Hoffmann, K.-P., Lappe, M. & Bremmer, F. Visual selectivity for heading in the macaque ventral intraparietal area. J. Neurophysiol. 112, 2470–2480 (2014).

20. 20.

Bremmer, F., Kubischik, M., Hoffmann, K.-P. & Krekelberg, B. Neural dynamics of saccadic suppression. J. Neurosci. 29, 12374–12383 (2009).

21. 21.

Duffy, C. J. & Wurtz, R. H. Sensitivity of MST neurons to optic flow stimuli. I. A continuum of response selectivity to large-field stimuli. J. Neurophysiol. 65, 1329–1345 (1991).

22. 22.

Bremmer, F., Duhamel, J., Ben Hamed, S. & Graf, W. Heading encoding in the macaque ventral intraparietal area (VIP). Eur. J. Neurosci. 16, 1554–1568 (2002).

23. 23.

Britten, K. H. Mechanisms of self-motion perception. Annu. Rev. Neurosci. 31, 389–410 (2008).

24. 24.

Ross, J. & Edwin Dickinson, J. Effects of adaptation to Glass pattern structure and to path of optic flow. Vision Res. 47, 2150–2155 (2007).

25. 25.

Morrone, M. C. et al. A cortical area that responds specifically to optic flow, revealed by fMRI. Nat. Neurosci. 3, 1322–1328 (2000).

26. 26.

Bremmer, F. et al. Polymodal motion processing in posterior parietal and premotor cortex: a human fMRI study strongly implies equivalencies between humans and monkeys. Neuron 29, 287–296 (2001).

27. 27.

Schmolesky, M. T. et al. Signal timing across the macaque visual system. J. Neurophysiol. 79, 3272–3278 (1998).

28. 28.

Morrone, M. C., Ross, J. & Burr, D. C. Saccadic eye movements cause compression of time as well as space. Nat. Neurosci. 8, 950–954 (2005).

29. 29.

Ilg, U. J. & Hoffmann, K.-P. Motion perception during saccades. Vision Res. 33, 211–220 (1993).

30. 30.

Castet, E. & Masson, G. S. Motion perception during saccadic eye movements. Nat. Neurosci. 3, 177–183 (2000).

31. 31.

Frost, A. & Niemeier, M. Suppression and reversal of motion perception around the time of the saccade. Front. Syst. Neurosci. 9, 143 (2015).

32. 32.

Kloosterman, N. A. et al. Top-down modulation in human visual cortex predicts the stability of a perceptual illusion. J. Neurophysiol. 113, 1063–1076 (2015).

33. 33.

Blake, R. & Logothetis, N. K. Visual competition. Nat. Rev. Neurosci. 3, 13–21 (2002).

34. 34.

Schlag, J. & Schlag-Rey, M. Through the eye, slowly: delays and localization errors in the visual system. Nat. Rev. Neurosci. 3, 191–215 (2002).

35. 35.

Dowiasch, S., Blohm, G. & Bremmer, F. Neural correlate of spatial (mis-)localization during smooth eye movements. Eur. J. Neurosci. 44, 1846–1855 (2016).

36. 36.

Cuturi, L. F. & MacNeilage, P. R. Systematic biases in human heading estimation. PLoS .ONE 8, e56862 (2013).

37. 37.

Crane, B. T. Direction specific biases in human visual and vestibular heading perception. PLoS ONE 7, e51383 (2012).

38. 38.

d’Avossa, G. & Kersten, D. Evidence in human subjects for independent coding of azimuth and elevation for direction of heading from optic flow. Vision Res. 36, 2915–2924 (1996).

39. 39.

Telford, L. & Howard, I. P. Role of optical flow field asymmetry in the perception of heading during linear motion. Percept. Psychophys. 58, 283–288 (1996).

40. 40.

Müsseler, J. et al. Relative mislocalization of briefly presented stimuli in the retinal periphery. Percept. Psychophys. 61, 1646–1661 (1999).

41. 41.

Kaminiarz, A., Krekelberg, B. & Bremmer, F. Localization of visual targets during optokinetic eye movements. Vision Res. 47, 869–878 (2007).

42. 42.

Bock, O. Localization of objects in the peripheral visual field. Behav. Brain Res. 56, 77–84 (1993).

43. 43.

Chen, A., DeAngelis, G. C. & Angelaki, D. E. Representation of vestibular and visual cues to self-motion in ventral intraparietal cortex. J. Neurosci. 31, 12036–12052 (2011).

44. 44.

Burr, D. C., Morrone, M. C. & Ross, J. Selective suppression of the magnocellular visual pathway during saccadic eye movements. Nature 371, 511–513 (1994).

45. 45.

Crane, B. T. Coordinates of human visual and inertial heading perception. PLoS ONE 10, e0135539 (2015).

46. 46.

Fetsch, C. R., Wang, S., Gu, Y., DeAngelis, G. C. & Angelaki, D. E. Spatial reference frames of visual, vestibular, and multimodal heading signals in the dorsal subdivision of the medial superior temporal area. J. Neurosci. 27, 700–712 (2007).

47. 47.

Chen, X., DeAngelis, G. C. & Angelaki, D. E. Eye-centered representation of optic flow tuning in the ventral intraparietal area. J. Neurosci. 33, 18574–18582 (2013).

48. 48.

Burr, D. C., Ross, J., Binda, P. & Morrone, M. C. Saccades compress space, time and number. Trends Cogn. Sci. 14, 528–533 (2010).

49. 49.

Hamker, F. H., Zirnsak, M., Calow, D. & Lappe, M. The peri-saccadic perception of objects and space. PLoS Comput. Biol. 4, e31 (2008).

50. 50.

Diamond, M. R., Ross, J. & Morrone, M. C. Extraretinal control of saccadic suppression. J. Neurosci. 20, 3449–3455 (2000).

51. 51.

Binda, P., Morrone, M. C. & Bremmer, F. Saccadic compression of symbolic numerical magnitude. PLoS ONE 7, e49587 (2012).

52. 52.

Knöll, J., Binda, P., Morrone, M. C. & Bremmer, F. Spatiotemporal profile of peri-saccadic contrast sensitivity. J. Vis. 11, 15 (2011).

53. 53.

Niemann, T., Lappe, M., Buscher, A. & Hoffmann, K.-P. Ocular responses to radial optic flow and single accelerated targets in humans. Vision Res. 39, 1359–1371 (1999).

54. 54.

Brainard, D. H. The psychophysics toolbox. Spat. Vis. 10, 433–436 (1997).

55. 55.

Cohen, J. Statistical Power Analysis for the Behavioral Sciences (L. Erlbaum Associates, 1988).

56. 56.

Hartung, J., Knapp, G. & Sinha, B. K. Statistical Meta-Analysis with Applications (Wiley, 2011).

57. 57.

Lappe, M., Bremmer, F. & Van Den Berg, A. V. V. Perception of self motion from visual flow. Trends Cogn. Sci. 3, 329–336 (1999).

## Acknowledgements

This work was supported by the Deutsche Forschungsgemeinschaft (DFG) Collaborative Research Center CRC/TRR-135 and Research Unit RU-1847. We thank Dirk Hofmann and Philipp Hesse for help with the behavioral experiments. We also thank the three reviewers for their valuable comments on the original version of the manuscript.

## Author information

Authors

### Contributions

F.B. and M.L. designed the research; Neurophysiological data had been recorded before18, 19; J.C. collected part of the behavioral data; F.B., J.C., and M.L. supervised the collection of the remaining behavioral data; F.B. created the decoder and the quantitative model and analyzed the neurophysiological and the behavioral data; and F.B., J.C., and M.L. wrote the paper.

### Corresponding author

Correspondence to Frank Bremmer.

## Ethics declarations

### Competing interests

The authors declare no competing financial interests.

Publisher's note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Rights and permissions

Reprints and Permissions

Bremmer, F., Churan, J. & Lappe, M. Heading representations in primates are compressed by saccades. Nat Commun 8, 920 (2017). https://doi.org/10.1038/s41467-017-01021-5

• Accepted:

• Published:

• ### Primate extrastriate cortical area MST: a gateway between sensation and cognition

• Benedict Wild
•  & Stefan Treue

Journal of Neurophysiology (2021)

• ### Perisaccadic encoding of temporal information in macaque area V4

• Jakob C. B. Schwenk
• , Steffen Klingenhoefer
• , Björn-Olaf Werner
• , Stefan Dowiasch
•  & Frank Bremmer

Journal of Neurophysiology (2021)

• ### Spatiotemporal Content of Saccade Transients

• Naghmeh Mostofi
• , Zhetuo Zhao
• , Janis Intoy
• , Marco Boi
• , Jonathan D. Victor
•  & Michele Rucci

Current Biology (2020)

• ### A Causal Role of Area hMST for Self-Motion Perception in Humans

• Constanze Schmitt
• , Bianca R Baltaretu
• , J Douglas Crawford
•  & Frank Bremmer

Cerebral Cortex Communications (2020)

• ### The role of the ventral intraparietal area (VIP/pVIP) in the perception of object-motion and self-motion

• David T. Field
• , Nicolò Biagi
•  & Laura A. Inman

NeuroImage (2020)