Main

We have discovered a visual illusion that is induced by sound: when a single visual flash is accompanied by multiple auditory beeps, the single flash is incorrectly perceived as multiple flashes. These results were obtained by flashing a uniform white disk (subtending 2 degrees at 5 degrees eccentricity) for a variable number of times (50 milliseconds apart) on a black background. Flashes were accompanied by a variable number of beeps, each spaced 57 milliseconds apart. Observers were asked to judge how many visual flashes were presented on each trial. The trials were randomized and each stimulus combination was run five times on eight naive observers.

Surprisingly, observers consistently and incorrectly reported seeing multiple flashes whenever a single flash was accompanied by more than one beep (Fig. 1a). Control conditions and catch trials ( Fig. 1 legend) indicate that the illusory flashing phenomenon is a perceptual illusion and is not due to the difficulty of the task, cognitive bias or other factors.

Figure 1: Illusory flashing.
figure 1

a, Perceived number of visual flashes by eight observers plotted as a function of the number of auditory beeps for a single flash. The number of perceived flashes did not increase linearly with the third and fourth beeps because they fell outside the optimal window of audiovisual integration, as revealed by our next experiment. b, Perceived number of flashes by eight observers plotted as a function of the actual number of flashes presented for trials with no sound (dashed line), and trials with single beeps corresponding to catch trials (grey line). Observers performed the task very well in the absence of sound (dashed line). The results of the catch trials (grey line) confirm that the observers' responses were not determined by their auditory percepts. The curve in a (for a single flash) is superimposed for comparison. Further details can be obtained from L.S.

Full size image

Figure 1b shows that observers' performance was the same, irrespective of whether a single flash was accompanied by two beeps, or two flashes by one or no beeps, suggesting that the illusory double flash is perceptually equivalent to the physical double flash. This was confirmed by the testimonies of the observers after the experiment. The performance of non-naive subjects (results not shown) indicated that the illusion persisted even when subjects were aware of the fact that the disk was actually flashed only once.

We next investigated the temporal properties of this illusion by varying the relative timing of visual and auditory stimuli. The illusory flashing effect declined from 70 milliseconds separation onwards. However, illusory flashing occurred as long as the beep and flash were within approximately 100 milliseconds of each other, which is consistent with the integration time of polysensory neurons in the brain1,2.

Our results indicate that the illusory flashing is caused by an alteration of visual perception by auditory stimuli. The modification of the visual percept by sound, however, was not categorical. It was rather selective, as sound did not have a fusing effect when multiple flashes were accompanied by a single beep. We suggest therefore that the direction of cross-modal interactions is partly dependent on the type of stimulus. Consistent with previous observations in other modalities3, we propose that the percept of a continuous stimulus in one modality is rendered more malleable by a discontinuous stimulus in another modality than vice versa.

The influence of auditory cues on visual perception has been demonstrated in other settings, in which perceived visual intensity is affected by the presence of an auditory stimulus4. This influence, however, is quantitative and does not alter the phenomenological quality of the percept. Others have shown that the perceived direction of ambiguous visual motion is influenced by auditory stimulation5.

Our results extend these previous findings by showing that visual perception can be qualitatively altered by sound even when the visual stimulus is not ambiguous. The conditions under which this alteration occurs — the stimulus configuration and the task — are very simple. The illusion is also surprisingly robust to variation in the many parameters we manipulated (including disk eccentricity and contrast, spatial disparity between sound and flash, shape and texture of the flashing pattern, flash and beep durations). The simplicity and robustness of the illusory flashing phenomenon indicate that it reflects a fundamental and widespread property of polysensory mechanisms in the brain.