Article

Natural speech reveals the semantic maps that tile human cerebral cortex

Received:
Accepted:
Published online:

Abstract

The meaning of language is represented in regions of the cerebral cortex collectively known as the ‘semantic system’. However, little of the semantic system has been mapped comprehensively, and the semantic selectivity of most regions is unknown. Here we systematically map semantic selectivity across the cortex using voxel-wise modelling of functional MRI (fMRI) data collected while subjects listened to hours of narrative stories. We show that the semantic system is organized into intricate patterns that seem to be consistent across individuals. We then use a novel generative model to create a detailed semantic atlas. Our results suggest that most areas within the semantic system represent information about specific semantic domains, or groups of related concepts, and our atlas shows which domains are represented in each area. This study demonstrates that data-driven methods—commonplace in studies of human neuroanatomy and functional connectivity—provide a powerful and efficient means for mapping functional representations in the brain.

  • Subscribe to Nature for full access:

    $199

    Subscribe

Additional access options:

Already a subscriber?  Log in  now or  Register  for online access.

References

  1. 1.

    , , & Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. Cereb. Cortex 19, 2767–2796 (2009)

  2. 2.

    , , & Topographic mapping of a hierarchy of temporal receptive windows using a narrated story. J. Neurosci. 31, 2906–2915 (2011)

  3. 3.

    , & Segregating semantic and syntactic aspects of processing in the human brain: an fMRI investigation of different word types. Cereb. Cortex 10, 698–705 (2000)

  4. 4.

    & Retrieval of abstract semantics. Neuroimage 22, 164–170 (2004)

  5. 5.

    , , , & Distinct brain systems for processing concrete and abstract concepts. J. Cogn. Neurosci. 17, 905–917 (2005)

  6. 6.

    , , , & Concepts are more than percepts: the case of action verbs. J. Neurosci. 28, 11347–11353 (2008)

  7. 7.

    & People thinking about thinking people. The role of the temporo-parietal junction in “theory of mind”. Neuroimage 19, 1835–1842 (2003)

  8. 8.

    & Domain-specific knowledge systems in the brain the animate-inanimate distinction. J. Cogn. Neurosci. 10, 1–34 (1998)

  9. 9.

    , , & Functional neuroanatomy of the semantic system: divisible by what? J. Cogn. Neurosci. 10, 766–777 (1998)

  10. 10.

    , , & A neurosemantic theory of concrete noun representation based on the underlying brain codes. PLoS ONE 5, e8622 (2010)

  11. 11.

    The selective impairment of semantic memory. Q. J. Exp. Psychol. 27, 635–657 (1975)

  12. 12.

    et al. Predicting human brain activity associated with the meanings of nouns. Science 320, 1191–1195 (2008)

  13. 13.

    , , , & A neural basis for lexical retrieval. Nature 380, 499–505 (1996)

  14. 14.

    , , & A continuous semantic space describes the representation of thousands of object and action categories across the human brain. Neuron 76, 1210–1224 (2012)

  15. 15.

    et al. Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses. PLoS ONE 9, e112575 (2014)

  16. 16.

    , , , & Bayesian reconstruction of natural images from human brain activity. Neuron 63, 902–915 (2009)

  17. 17.

    et al. Reconstructing visual experiences from brain activity evoked by natural movies. Curr. Biol. 21, 1641–1646 (2011)

  18. 18.

    , , , & Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41, 391–407 (1990)

  19. 19.

    & Producing high-dimensional semantic spaces from lexical co-occurrence. Behav. Res. Methods Instrum. Comput. 28, 203–208 (1996)

  20. 20.

    & From frequency to meaning: vector space models of semantics. J. Artif. Intell. Res. 37, 141–188 (2010)

  21. 21.

    & The organisation of conceptual knowledge in the brain: the future’s past and some future directions. Cogn. Neuropsychol. 23, 13–38 (2006)

  22. 22.

    , , & PrAGMATiC: a probabilistic and generative model of areas tiling the cortex . Preprint at (2015)

  23. 23.

    , , , & Brodmann’s areas 17 and 18 brought into stereotaxic space—where and how variable? Neuroimage 11, 66–84 (2000)

  24. 24.

    , , , & New method for fMRI investigations of language: defining ROIs functionally in individual subjects. J. Neurophysiol. 104, 1177–1194 (2010)

  25. 25.

    Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771–1800 (2002)

  26. 26.

    , & The brain’s default network: anatomy, function, and relevance to disease. Ann. NY Acad. Sci . 1124, 1–38 (2008)

  27. 27.

    & Phoneme and word recognition in the auditory ventral stream. Proc. Natl Acad. Sci. USA 109, E505–E514 (2012)

  28. 28.

    Appearance isn’t everything: news on object representation in cortex. Neuron 55, 341–344 (2007)

  29. 29.

    , , & The neural code for written words: a proposal. Trends Cogn. Sci. 9, 335–341 (2005)

  30. 30.

    , & Interpreting fMRI data: maps, modules and dimensions. Nature Rev. Neurosci. 9, 123–135 (2008)

  31. 31.

    et al. Organization of the human inferior parietal lobule based on receptor architectonics. Cereb. Cortex 23, 615–628 (2013)

  32. 32.

    et al. Defining functional areas in individual human brains using resting functional connectivity MRI. Neuroimage 41, 45–57 (2008)

  33. 33.

    & Speaker identification on the SCOTUS corpus. Proc. Acoust. Preprint at (2008)

  34. 34.

    & Praat: doing phonetics by computer (University of Amsterdam, 2014)

  35. 35.

    & Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Statist. Soc. B 57, 289–300 (1995)

  36. 36.

    Guide to NumPy (Brigham Young University, 2006)

  37. 37.

    , & SciPy: Open source scientific tools for Python (SciPy, 2001)

  38. 38.

    , , & Pycortex: an interactive surface visualizer for fMRI. Front. Neuroinform . 9, 23 (2015)

Download references

Acknowledgements

This work was supported by grants from the National Science Foundation (NSF; IIS1208203), the National Eye Institute (EY019684), and from the Center for Science of Information (CSoI), an NSF Science and Technology Center, under grant agreement CCF-0939370. A.G.H. was also supported by the William Orr Dingwall Neurolinguistics Fellowship. We thank J. Sohl-Dickstein and K. Crane for technical discussions about PrAGMATiC, J. Nguyen for assistance transcribing and aligning stimuli, B. Griffin for segmenting and flattening cortical surfaces, and N. Bilenko, J. Gao, M. Lescroart and A. Nunez-Elizalde for general comments and discussions.

Author information

Affiliations

  1. Helen Wills Neuroscience Institute, University of California, Berkeley, California 94720, USA

    • Alexander G. Huth
    • , Thomas L. Griffiths
    • , Frédéric E. Theunissen
    •  & Jack L. Gallant
  2. Department of Psychology, University of California, Berkeley, California 94720, USA

    • Wendy A. de Heer
    • , Thomas L. Griffiths
    • , Frédéric E. Theunissen
    •  & Jack L. Gallant

Authors

  1. Search for Alexander G. Huth in:

  2. Search for Wendy A. de Heer in:

  3. Search for Thomas L. Griffiths in:

  4. Search for Frédéric E. Theunissen in:

  5. Search for Jack L. Gallant in:

Contributions

All authors helped conceive and design the experiment. W.A.d.H. and A.G.H. selected and annotated stimuli and collected fMRI data. A.G.H. analysed the data. A.G.H. and T.L.G. designed the PrAGMATiC generative model. A.G.H. and J.L.G. wrote the paper. J.L.G. contributed to all aspects of the project.

Competing interests

The authors declare no competing financial interests.

Corresponding author

Correspondence to Jack L. Gallant.

Extended data

Supplementary information

PDF files

  1. 1.

    Supplementary Information

    This file contains Supplementary Data, Supplementary methods, Supplementary Tables 1-3 and Supplementary References.