Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Predicting visual stimuli on the basis of activity in auditory cortices

Subjects

Abstract

Using multivariate pattern analysis of functional magnetic resonance imaging data, we found that the subjective experience of sound, in the absence of auditory stimulation, was associated with content-specific activity in early auditory cortices in humans. As subjects viewed sound-implying, but silent, visual stimuli, activity in auditory cortex differentiated among sounds related to various animals, musical instruments and objects. These results support the idea that early sensory cortex activity reflects perceptual experience, rather than sensory stimulation alone.

This is a preview of subscription content

Access options

Buy article

Get time limited or full article access on ReadCube.

$32.00

All prices are NET prices.

Figure 1: Classifier performance on pair-wise discriminations among individual stimuli.
Figure 2: Classifier performance on pair-wise discriminations among categories.
Figure 3: Classifier performance on categorical discriminations for visual, auditory and cross-modal classification.

References

  1. Formisano, E., De Martino, F., Bonte, M. & Goebel, R. Science 322, 970–973 (2008).

    CAS  Article  Google Scholar 

  2. Staeren, N., Renvall, H., De Martino, F., Goebel, R. & Formisano, E. Curr. Biol. 19, 498–502 (2009).

    CAS  Article  Google Scholar 

  3. Macknik, S.L. & Haglund, M.M. Proc. Natl. Acad. Sci. USA 96, 15208–15210 (1999).

    CAS  Article  Google Scholar 

  4. Watkins, S., Shams, L., Tanaka, S., Haynes, J.-D. & Rees, G. Neuroimage 31, 1247–1256 (2006).

    CAS  Article  Google Scholar 

  5. Chen, L.M., Friedman, R.M. & Roe, A.W. Science 302, 881–885 (2003).

    CAS  Article  Google Scholar 

  6. Blankenburg, F., Ruff, C.C., Deichmann, R., Rees, G. & Driver, J. PLoS Biol. 4, e69 (2006).

    Article  Google Scholar 

  7. Kosslyn, S.M., Thompson, W.L., Kim, I.J. & Alpert, N.M. Nature 378, 496–498 (1995).

    CAS  Article  Google Scholar 

  8. Slotnick, S.D., Thompson, W.L. & Kosslyn, S.M. Cereb. Cortex 15, 1570–1583 (2005).

    Article  Google Scholar 

  9. Thirion, B. et al. Neuroimage 33, 1104–1116 (2006).

    Article  Google Scholar 

  10. Serences, J.T., Ester, E.F., Vogel, E.K. & Awh, E. Psychol. Sci. 20, 207–214 (2009).

    Article  Google Scholar 

  11. Harrison, S.A. & Tong, F. Nature 458, 632–635 (2009).

    CAS  Article  Google Scholar 

  12. Zatorre, R.J. & Halpern, A.R. Neuron 47, 9–12 (2005).

    CAS  Article  Google Scholar 

  13. Kraemer, D.J.M., Macrae, C.N., Green, A.E. & Kelley, W.M. Nature 434, 158 (2005).

    CAS  Article  Google Scholar 

  14. Dierks, T. et al. Neuron 22, 615–621 (1999).

    CAS  Article  Google Scholar 

  15. Calvert, G.A. et al. Science 276, 593–596 (1997).

    CAS  Article  Google Scholar 

Download references

Acknowledgements

H.D. and A.D. are supported by the Mathers Foundation and the US National Institutes of Health (5P50NS019632-25).

Author information

Authors and Affiliations

Authors

Contributions

K.M. conceived and designed the study with input from the other authors, conducted functional magnetic resonance imaging (fMRI), participated in data analysis and wrote the manuscript together with A.D. J.T.K. advised on study design, conducted fMRI, performed the data analysis and wrote part of the supplementary information. R.E. prepared stimuli, wrote the stimulus presentation script, conducted fMRI, traced anatomical masks, participated in data analysis and conducted literature review. C.W. prepared stimuli, traced anatomical masks and conducted literature review. H.D. supervised the design of the anatomical masks and provided conceptual advice. A.D. supervised the project and advised K.M. on the preparation of the manuscript. All authors discussed the results and their implications and commented on the study and manuscript preparation at all stages.

Corresponding author

Correspondence to Antonio Damasio.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–8, Supplementary Methods, Supplementary Results and Supplementary Discussion (PDF 668 kb)

Supplementary Video 1

This clip displays a howling dog and serves as an example for the animal video clips. (MOV 1639 kb)

Supplementary Video 2

This clip displays a piano key being struck and serves as an example for the musical instrument video clips. (MOV 1273 kb)

Supplementary Video 3

This clip displays a handful of coins being dropped into a glass and serves as an example for the object video clips. (MOV 1231 kb)

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Meyer, K., Kaplan, J., Essex, R. et al. Predicting visual stimuli on the basis of activity in auditory cortices. Nat Neurosci 13, 667–668 (2010). https://doi.org/10.1038/nn.2533

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.2533

Further reading

Search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing