Abstract
Auditory spatial perception is strongly affected by visual cues. For example, if auditory and visual stimuli are presented synchronously but from different positions, the auditory event is mislocated towards the locus of the visual stimulus—the ventriloquism effect1,2. This ‘visual capture’ also occurs in motion perception in which a static auditory stimulus appears to move with the visual moving object3,4. We investigated how the human perceptual system coordinates complementary inputs from auditory and visual senses. Here we show that an auditory aftereffect occurs from adaptation to visual motion in depth. After a few minutes of viewing a square moving in depth, a steady sound was perceived as changing loudness in the opposite direction. Adaptation to a combination of auditory and visual stimuli changing in a compatible direction increased the aftereffect and the effect of visual adaptation almost disappeared when the directions were opposite. On the other hand, listening to a sound changing in intensity did not affect the visual changing-size aftereffect. The results provide psychophysical evidence that, for processing of motion in depth, the auditory system responds to both auditory changing intensity and visual motion in depth.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Jack, C. E. & Thurlow, W. R. Effects of degree of visual association and angle of displacement on the “ventriloquism” effect. Percept. Motor Skill. 37, 967–979 (1973).
Bertelson, P. & Radeau, M. Cross-modal bias and perceptual fusion with auditory-visual spatial discordance. Percept. Psychophys. 29, 578–584 (1981).
Mateeff, S., Hohnsbein, J. & Noack, T. Dynamic visual capture: apparent auditory motion induced by a moving visual target. Perception 14, 721–727 (1985).
Kitajima, N. & Yamashita, Y. Dynamic capture of sound motion by light stimuli moving in three-dimensional space. Percept. Motor Skill. 89, 1139–1158 (1999).
Wade, N. J. A selective history of the study of visual motion aftereffects. Perception 23, 1111–1134 (1994).
Mather, G., Verstraten, F. & Anstis, S. The Motion Aftereffect: A Modern Perspective (MIT Press, Cambridge, Massachusetts, 1998).
Grantham, D. W. Motion aftereffects with horizontally moving sound sources in the free field. Percept. Psychophys. 45, 129–136 (1989).
Dong, C. J., Swindale, N. V., Zakarauskas, P., Hayward, V. & Cynader, M. S. The auditory motion aftereffect: its tuning and specificity in the spatial and frequency domains. Percept. Psychophys. 62, 1099–1111 (2000).
Shu, Z. J., Swindale, N. V. & Cynader, M. S. Spectral motion produces an auditory after-effect. Nature 364, 721–723 (1993).
Reinhardt-Rutland, A. H. & Anstis, S. Auditory adaptation to gradual rise or fall in intensity of a tone. Percept. Psychophys. 31, 63–67 (1982).
Reinhardt-Rutland, A. H. Increasing-loudness aftereffect following decreasing-intensity adaptation: spectral dependence in interotic and monotic testing. Perception 27, 473–482 (1998).
Kayahara, T. Aftereffect of adaptation to uni-direction frequency change: evidence for selective processing mechanism. Acoust. Sci. Technol. 22, 49–51 (2001).
Regan, D. & Beverley, K. I. Illusory motion in depth: aftereffect of adaptation to changing size. Vision Res. 18, 209–212 (1978).
Cumming, B. in Visual Detection of Motion (eds Smith, A. T. & Snowden, R. J.) 333–366 (Academic, London, 1994).
Ehrenstein, W. H. & Reinhardt-Rutland, A. H. A cross-modal aftereffect: auditory displacement following adaptation to visual motion. Percept. Motor Skill. 82, 23–26 (1996).
Welch, R. B. & Warren, D. H. in Handbook of Perception and Human Performance. Vol. 1 Sensory Processes and Perception (eds Boff, K. R., Kaufman, L. & Thomas, J. P.) Ch. 25 (Wiley, New York, 1986).
Blauert, J. Spatial Hearing: the Psychophysics of Human Sound Localization Revised edn (MIT Press, Cambridge, Massachusetts, 1997).
Calvert, G. A. et al. Activation of auditory cortex during silent lipreading. Science 276, 593–596 (1997).
Macaluso, E., Frith, C. D. & Driver, J. Modulation of human visual cortex by crossmodal spatial attention. Science 289, 1206–1208 (2000).
Lewis, J. W., Beauchamp, M. S. & DeYoe, E. A. A comparison of visual and auditory motion processing in human cerebral cortex. Cereb. Cortex 10, 873–888 (2000).
Driver, J. & Spence, C. Multisensory perception: beyond modularity and convergence. Curr. Biol. 10, R731–R735 (2000).
Calvert, G. A., Campbell, R. & Brammer, M. J. Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr. Biol. 10, 649–657 (2000).
Brainard, D. H. The psychophysics toolbox. Spatial Vision 10, 433–436 (1997).
Pelli, D. G. The Video Toolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision 10, 437–442 (1997).
Cornsweet, T. N. The staircase-method in psychophysics. Am. J. Psychol. 75, 485–491 (1962).
Acknowledgements
We thank D. Levi, H. Akutsu and D. Erickson for helpful comments and suggestions.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Rights and permissions
About this article
Cite this article
Kitagawa, N., Ichihara, S. Hearing visual motion in depth. Nature 416, 172–174 (2002). https://doi.org/10.1038/416172a
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1038/416172a
This article is cited by
-
Perceptual aftereffects of adiposity transfer from hands to whole bodies
Experimental Brain Research (2023)
-
The Audio-Corsi: an acoustic virtual reality-based technological solution for evaluating audio-spatial memory abilities
Journal on Multimodal User Interfaces (2022)
-
Congruent audio-visual stimulation during adaptation modulates the subsequently experienced visual motion aftereffect
Scientific Reports (2019)
-
The reference frame of the tilt aftereffect measured by differential Pavlovian conditioning
Scientific Reports (2017)
-
Concurrent talking in immersive virtual reality: on the dominance of visual speech cues
Scientific Reports (2017)
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.