Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Aero-tactile integration in speech perception

Nature volume 462, pages 502–504 (2009)Cite this article

Abstract

Visual information from a speaker’s face can enhance1 or interfere with2 accurate auditory perception. This integration of information across auditory and visual streams has been observed in functional imaging studies3,4, and has typically been attributed to the frequency and robustness with which perceivers jointly encounter event-specific information from these two modalities5. Adding the tactile modality has long been considered a crucial next step in understanding multisensory integration. However, previous studies have found an influence of tactile input on speech perception only under limited circumstances, either where perceivers were aware of the task6,7 or where they had received training to establish a cross-modal mapping8,9,10. Here we show that perceivers integrate naturalistic tactile information during auditory speech perception without previous training. Drawing on the observation that some speech sounds produce tiny bursts of aspiration (such as English ‘p’)11, we applied slight, inaudible air puffs on participants’ skin at one of two locations: the right hand or the neck. Syllables heard simultaneously with cutaneous air puffs were more likely to be heard as aspirated (for example, causing participants to mishear ‘b’ as ‘p’). These results demonstrate that perceivers integrate event-relevant tactile information in auditory perception in much the same way as they do visual information.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Rent or buy this article

Get just this article for as long as you need it

$39.95

Learn more

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Interaction graphs for the hand experiment with standard error bars.
Figure 2: Interaction graphs for the neck experiment with standard error bars.
Figure 3: Interaction graph for control experiment with standard error bars.

References

  1. Sumby, W. H. & Pollack, I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26, 212–215 (1954)

    Article  ADS  Google Scholar 

  2. McGurk, H. & MacDonald, J. W. Hearing lips and seeing voices. Nature 264, 746–748 (1976)

    Article  ADS  CAS  Google Scholar 

  3. Calvert, G. A. et al. Activation of auditory cortex during silent lipreading. Science 276, 593–596 (1997)

    Article  CAS  Google Scholar 

  4. Calvert, G. A. & Campbell, R. Reading speech from still and moving faces: the neural substrates of visible speech. J. Cogn. Neurosci. 15, 57–70 (2003)

    Article  Google Scholar 

  5. Diehl, R. L. & Kluender, K. R. On the objects of speech perception. Ecol. Psychol. 1, 121–144 (1989)

    Article  Google Scholar 

  6. Fowler, C. & Dekle, D. Listening with eye and hand: crossmodal contributions to speech perception. J. Exp. Psychol. Hum. Percept. Perform. 17, 816–828 (1991)

    Article  CAS  Google Scholar 

  7. Gick, B., Jóhannsdóttir, K. M., Gibraiel, D. & Mühlbauer, J. Tactile enhancement of auditory and visual speech perception in untrained perceivers. J. Acoust. Soc. Am. 123, EL72–EL76 (2008)

    Article  ADS  Google Scholar 

  8. Sparks, D. W., Kuhl, P. K., Edmonds, A. E. & Gray, G. P. Investigating the MESA (multipoint electrotactile speech aid): the transmission of segmental features of speech. J. Acoust. Soc. Am. 63, 246–257 (1978)

    Article  ADS  CAS  Google Scholar 

  9. Reed, C. M., Durlach, N. I., Braida, L. D. & Schultz, M. C. Analytic study of the Tadoma method: effects of hand position on segmental speech perception. J. Speech Hear. Res. 32, 921–929 (1989)

    Article  CAS  Google Scholar 

  10. Bernstein, L. E., Demorest, M. E., Coulter, D. C. & O’ Connell, M. P. Lipreading sentences with vibrotactile vocoders: performance of normal-hearing and hearing-impaired subjects. J. Acoust. Soc. Am. 90, 2971–2984 (1991)

    Article  ADS  CAS  Google Scholar 

  11. Derrick, D., Anderson, P., Gick, B. & Green, S. Characteristics of air puffs produced in English ‘pa’: data and simulation. J. Acoust. Soc. Am. 125, 2272–2281 (2009)

    Article  ADS  Google Scholar 

  12. Lisker, L. & Abramson, A. S. A cross-language study of voicing in initial stops: acoustical measurements. Word 20, 384–423 (1964)

    Article  Google Scholar 

  13. Mizobuchi, K. et al. Single unit responses of human cutaneous mechanoreceptors to air-puff stimulation. Clin. Neurophysiol. 111, 1577–1581 (2000)

    Article  CAS  Google Scholar 

  14. Sabah, N. H. Controlled stimulation for hair follicle receptors. J. Appl. Physiol. 36, 256–257 (1974)

    Article  ADS  CAS  Google Scholar 

  15. Sangras, R., Kwon, O. C. & Faeth, G. M. Self-preserving properties of unsteady round nonbuoyant turbulent starting jets and puffs in still fluids. J. Heat Transfer 124, 460–469 (2002)

    Article  Google Scholar 

  16. Diez, F. J., Sangras, R., Kwon, O. C. & Faeth, G. M. Erratum: “Self-preserving properties of unsteady round nonbuoyant turbulent starting jets and puffs in still fluids. ASME J. Heat Transfer, 124, 460–469 (2002)”. J. Heat Transfer 125, 204–205 (2003)

    Article  Google Scholar 

  17. Stevens, K. N. Acoustic Phonetics Ch. 7 (MIT Press, 1998)

    Google Scholar 

  18. Weinstein, S. in The Skin Sense (ed. Kenshalo, D. R.) 195–222 (Thomas, 1968)

    Google Scholar 

  19. Fu, K.-M. G. et al. Auditory cortical neurons respond to somatosensory stimulation. J. Neurosci. 23, 7510–7515 (2003)

    Article  CAS  Google Scholar 

  20. Ito, T., Tiede, M. & Ostry, D. J. Somatosensory function in speech perception. Proc. Natl Acad. Sci. USA 106, 1245–1248 (2009)

    Article  ADS  CAS  Google Scholar 

Download references

Acknowledgements

This research was funded by a Discovery Grant from the Natural Sciences and Engineering Council of Canada to B.G., and by a National Institutes of Health Grant DC-02717 to Haskins Laboratories. We acknowledge the contributions of Y. Ikegami for initial piloting, R. Lee for piloting tapping protocols, L. McLeod for help setting up the University of British Columbia system, G. Ramsay and J. Wang for building switch mechanisms, S. Renardy for organizing protocols, D. H. Whalen for discussion and students of the University of British Columbia’s Interdisciplinary Speech Research Laboratory for organizing participants and other assistance.

Author Contributions B.G. conceived and designed the experiment; D.D. designed and performed the data analysis.

Author information

Authors and Affiliations

  1. Department of Linguistics, University of British Columbia, Vancouver, British Columbia V6T 1Z4, Canada,

    Bryan Gick & Donald Derrick

  2. Haskins Laboratories, New Haven, Connecticut 06511-6695, USA ,

    Bryan Gick

Authors
  1. Bryan Gick
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Donald Derrick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bryan Gick.

Supplementary information

Supplementary Information

This file contains Supplementary Methods, Supplementary Figure 1 and Legend and Supplementary Tables 1-3. (PDF 1633 kb)

PowerPoint slides

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gick, B., Derrick, D. Aero-tactile integration in speech perception. Nature 462, 502–504 (2009). https://doi.org/10.1038/nature08572

Download citation

  • Received:

  • Accepted:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nature08572

This article is cited by

Comments

By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing