Article | Published:

A unified model of human semantic knowledge and its disorders

Nature Human Behaviour volume 1, Article number: 0039 (2017) | Download Citation

Abstract

How is knowledge about the meanings of words and objects represented in the human brain? Current theories embrace two radically different proposals: either distinct cortical systems have evolved to represent different kinds of things, or knowledge for all kinds is encoded within a single domain-general network. Neither view explains the full scope of relevant evidence from neuroimaging and neuropsychology. Here we propose that graded category-specificity emerges in some components of the semantic network through joint effects of learning and network connectivity. We test the proposal by measuring connectivity amongst cortical regions implicated in semantic representation, then simulating healthy and disordered semantic processing in a deep neural network whose architecture mirrors this structure. The resulting neuro-computational model explains the full complement of neuroimaging and patient evidence adduced in support of both domain-specific and domain-general approaches, reconciling long-standing disputes about the nature and origins of this uniquely human cognitive faculty.

Access optionsAccess options

Rent or Buy article

Get time limited or full article access on ReadCube.

from$8.99

All prices are NET prices.

References

  1. 1.

    et al. Predicting brain activation patterns associated with individual lexical concepts based on five sensory-motor attributes. Neuropsychologia 76, 17–26 (2015).

  2. 2.

    & Domain-specific knowledge systems in the brain: the animate–inanimate distinction. J. Cogn. Neurosci. 10, 1–34 (1998).

  3. 3.

    & The organization of conceptual knowledge: the evidence from category-specific semantic deficits. Trends Cogn. Sci. 7, 354–361 (2003).

  4. 4.

    , & Where do you know what you know? The representation of semantic knowledge in the human brain. Nat. Rev. Neurosci 8, 976–987 (2007).

  5. 5.

    et al. The structure and deterioration of semantic memory: a computational and neuropsychological investigation. Psychol. Rev. 111, 205–235 (2004).

  6. 6.

    , , & Conceptual structure and the structure of concepts: a distributed account of category-specific deficits. Brain Lang. 75, 195–231 (2000).

  7. 7.

    & Revisiting domain-general accounts of category specificity in mind and brain. Wiley Interdiscip. Rev. Cogn. Sci. 5, 327–44 (2014).

  8. 8.

    Graded modality-specific specialisation in semantics: a computational account of optic aphasia. Cogn. Neuropsychol. 19, 603–639 (2002).

  9. 9.

    , , , & Category-specific organization in the human brain does not require visual experience. Neuron 63, 397–405 (2009).

  10. 10.

    & Category specific semantic impairments. Brain 107, 829–854 (1984).

  11. 11.

    & Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). J. Exp. Psychol. Gen. 132, 163–201 (2003).

  12. 12.

    & A model of emergent category-specific activation in the posterior fusiform gyrus of sighted and congenitally blind populations. J. Cogn. Neurosci. 27, 1981–1999 (2015).

  13. 13.

    , & Category-specific versus category-general semantic impairment induced by transcranial magnetic stimulation. Curr. Biol. 20, 964–968 (2010).

  14. 14.

    et al. Neural constraints on learning. Nature 512, 423–426 (2014).

  15. 15.

    et al. Functionally defined white matter reveals segregated pathways in human ventral temporal cortex associated with category-specific processing. Neuron 85, 216–227 (2015).

  16. 16.

    & What drives the organization of object knowledge in the brain? Trends Cogn. Sci. 15, 97–103 (2011).

  17. 17.

    & Complementary neural representations for faces and words: a computational exploration. Cogn. Neuropsychol. 28, 251–275 (2011).

  18. 18.

    & Semantic memory and the brain: structure and processes. Curr. Opin. Neurobiol. 11, 194–201 (2001).

  19. 19.

    et al. Action-related properties shape object representations in the ventral stream. Neuron 55, 507–520 (2007).

  20. 20.

    & The cortical organization of speech processing. Nat. Rev. Neurosci. 8, 393–402 (2007).

  21. 21.

    , & Actions speak louder than functions: the importance of manipulability and action in tool representation. J. Cogn. Neurosci. 15, 30–46 (2003).

  22. 22.

    & Category-specific neural processing for naming pictures of animals and naming pictures of tools: an ALE meta-analysis. Neuropsychologia 48, 409–418 (2010).

  23. 23.

    , & Anterior temporal lobes mediate semantic representation: mimicking semantic dementia by using rTMS in normal participants. Proc. Natl Acad. Sci. USA 104, 20137–20141 (2007).

  24. 24.

    et al. Atrophy, hypometabolism and white matter abnormalities in semantic dementia tell a coherent story. Brain 134, 2025–2035 (2011).

  25. 25.

    & Category-specific neural processing for naming pictures of animals and naming pictures of tools: an ALE meta-analysis. Neuropsychologia 48, 409 (2010).

  26. 26.

    et al. Category-specific activations during word generation reflect experiential sensorimotor modalities. Neuroimage 48, 717–725 (2009).

  27. 27.

    et al. Differences in functional magnetic resonance imaging activation by category in a visual confrontation naming task. J. Neuroimaging 11, 165–170 (2001).

  28. 28.

    et al. The neural basis for category-specific knowledge: an fMRI study. Neuroimage 15, 936–948 (2002).

  29. 29.

    , , , & Discrete cortical regions associated with knowledge of color and knowledge of action. Science 270, 102–105 (1995).

  30. 30.

    et al. Do semantic categories activate distinct cortical regions? Evidence for a distributed neural semantic system. Cogn. Neuropsychol. 20, 541–559 (2003).

  31. 31.

    , , , & A neural basis for lexical retrieval. Nature 380, 499–505 (1996).

  32. 32.

    , , , & The effects of semantic category and knowledge type on lexical-semantic access: a PET study. Neuroimage 8, 350–359 (1998).

  33. 33.

    , , & Semantic relevance explains category effects in medial fusiform gyri. Neuroimage 30, 992–1002 (2006).

  34. 34.

    , , , & Action selectivity in parietal and temporal cortex. Brain Res. Cogn. Brain Res. 25, 641 (2005).

  35. 35.

    , , & Can segregation within the semantic system account for category-specific deficits? Brain 125, 2067–2080 (2002).

  36. 36.

    & A common network in the left cerebral hemisphere represents planning of tool use pantomimes and familiar intransitive gestures at the hand-independent level. Cereb. Cortex 19, 2396–2410 (2009).

  37. 37.

    et al. Category-specific semantic memory: converging evidence from bold fMRI and Alzheimer’s disease. Neuroimage 68, 263–274 (2013).

  38. 38.

    , , , & Different brain activation patterns during production of animals versus artefacts: a PET activation study on category-specific processing. Cogn. Brain Res. 13, 95–99 (2002).

  39. 39.

    et al. Distinctions between manipulation and function knowledge of objects: evidence from functional magnetic resonance imaging. Cogn. Brain Res. 23, 361–373 (2005).

  40. 40.

    Category differences in brain activation studies: where do they come from? Proc. R. Soc. Lond. B 267, 1253–1258 (2000).

  41. 41.

    , & When action turns into words. Activation of motor-based knowledge during categorization of manipulable objects. J. Cogn. Neurosci. 14, 1230–1239 (2002).

  42. 42.

    , , , & Fusiform activation to animals is driven by the process, not the stimulus. J. Cogn. Neurosci. 17, 434–445 (2005).

  43. 43.

    , , , & Distinct cortical pathways for processing tool versus animal sounds. J. Neurosci. 25, 5148–5158 (2005).

  44. 44.

    et al. The different neural correlates of action and functional knowledge in semantic memory: an fMRI study. Cereb. Cortex 18, 740–751 (2008).

  45. 45.

    , & Shared and dissociated cortical regions for object and letter processing. Cogn. Brain Res. 17, 56–67 (2003).

  46. 46.

    et al. Brain activity related to integrative processes in visual object recognition: bottom-up integration and the modulatory influence of stored knowledge. Neuropsychologia 40, 1254–1267 (2002).

  47. 47.

    , , & Categorization and category effects in normal object recognition: a PET study. Neuropsychologia 38, 1693–1703 (2000).

  48. 48.

    , & Category-related activation for written words in the posterior fusiform is task specific. Neuropsychologia 43, 69–74 (2005).

  49. 49.

    , , & Two distinct neural mechanisms for category-selective responses. Cereb. Cortex 16, 437–445 (2006).

  50. 50.

    , , & Dissociable brain regions process object meaning and object structure during picture naming. Neuropsychologia 40, 174–186 (2002).

  51. 51.

    , , & Premotor cortex activation during observation and naming of familiar tools. Neuroimage 6, 231–236 (1997).

  52. 52.

    , & Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects. Nat. Neurosci. 2, 913–919 (1999).

  53. 53.

    & Representation of manipulable man-made objects in the dorsal stream. Neuroimage 12, 478–484 (2000).

  54. 54.

    , & Perceptual knowledge retrieval activates sensory brain regions. J. Neurosci. 26, 4917–4921 (2006).

  55. 55.

    et al. Functional MRI mapping of category-specific sites associated with naming of famous faces, animals and man-made objects. Neurosci. Bull. 27, 307–318 (2011).

  56. 56.

    , & Category learning increases discriminability of relevant object dimensions in visual cortex. Cereb. Cortex 23, 814–823 (2013).

  57. 57.

    , , & Neural correlates of category-specific knowledge. Nature 379, 649–652 (1996).

  58. 58.

    et al. Word and picture matching: a PET study of semantic category effects. Neuropsychologia 37, 293–306 (1999).

  59. 59.

    , , , & Graspable objects grab attention when the potential for action is recognized. Nat. Neurosci. 6, 421–427 (2003).

  60. 60.

    , & Structural similarity and category-specificity: a refined account. Neuropsychologia 42, 1543–1553 (2004).

  61. 61.

    & Brain mechanisms of perceiving tools and imagining tool use acts: a functional MRI study. Neuropsychologia 49, 1863–1869 (2011).

  62. 62.

    et al. Naming of animals and tools: a functional magnetic resonance imaging study of categorical differences in the human brain areas commonly used for naming visually presented objects. Neurosci. Lett. 296, 33 (2000).

  63. 63.

    , & The representation of tools in left parietal cortex is independent of visual experience. Psychol. Sci. 21, 764–771 (2010).

  64. 64.

    & Neural representations of graspable objects: are tools special? Cogn. Brain Res. 22, 457–469 (2005).

  65. 65.

    , & The representation of tool and non-tool object information in the human intraparietal sulcus. J. Neurophysiol. 109, 2883–2896 (2013).

  66. 66.

    et al. Visual and semantic processing of living things and artifacts: an fMRI study. J. Cogn. Neurosci. 22, 554–570 (2010).

  67. 67.

    , & Experience-dependent modulation of category-related cortical activity. Cereb. Cortex 12, 545–551 (2002).

  68. 68.

    , , & Differential activity for animals and manipulable objects in the anterior temporal lobes. J. Cogn. Neurosci. 23, 2059–2067 (2011).

  69. 69.

    & A functional neuroimaging study of the variables that generate category-specific object processing differences. Brain 122, 943–962 (1999).

  70. 70.

    , , , & Effects of noun–verb homonymy on the neural correlates of naming concrete entities and actions. Brain Lang. 92, 288–299 (2005).

  71. 71.

    , & Premotor and prefrontal correlates of category-related lexical retrieval. Neuroimage 7, 232–243 (1998).

  72. 72.

    , , , & Activation likelihood estimation meta-analysis revisited. Neuroimage 59, 2349–2361 (2012).

  73. 73.

    The representation of object concepts in the brain. Annu. Rev. Psychol. 58, 25–45 (2007).

  74. 74.

    , , & Lichtheim 2: synthesizing aphasia and the neural basis of language in a neurocomputational model of the dual dorsal-ventral language pathways. Neuron 72, 385–96 (2011).

  75. 75.

    , & Semantic processing in the anterior temporal lobes: a meta-analysis of the functional neuroimaging literature. J. Cogn. Neurosci. 22, 1083–1094 (2010).

  76. 76.

    et al. Semantic dementia and fluent primary progressive aphasia: two sides of the same coin? Brain 129, 3066–3080 (2006).

  77. 77.

    et al. Direct exploration of the role of the ventral anterior temporal lobe in semantic memory: cortical stimulation and local field potential evidence from subdural grid electrodes. Cereb. Cortex 25, 3802–3817 (2015).

  78. 78.

    , & Neural basis of category-specific semantic deficits for living things: evidence from semantic dementia, HSVE and a neural network model. Brain 130, 1127–1137 (2007).

  79. 79.

    & Two action systems in the human brain. Brain Lang. 127, 222–229 (2013).

  80. 80.

    & Features, objects, action: the cognitive neuropsychology of visual object processing, 1984–2004. Cogn. Neuropsychol. 23, 156–183 (2006).

  81. 81.

    & Fusion and fission of cognitive functions in the human parietal cortex. Cereb. Cortex 25, 3547–3560 (2015).

  82. 82.

    , , & The structural connectivity of higher order association cortices reflects human functional brain networks. Cortex (2016).

  83. 83.

    , , , & Distortion correction for diffusion-weighted MRI tractography and fMRI in the temporal lobes. Hum. Brain Mapp. 31, 1570–1587 (2010).

  84. 84.

    , , , & The ventral and inferolateral aspects of the anterior temporal lobe are crucial in semantic memory: evidence from a novel direct comparison of distortion-corrected fMRI, rTMS, and semantic dementia. Cereb. Cortex 20, 2728–2738 (2010).

  85. 85.

    , , , & The ventral visual pathway: an expanded neural framework for the processing of object quality. Trends Cogn. Sci. 17, 26–49 (2013).

  86. 86.

    , & Convergent connectivity and graded specialization in the rostral human temporal lobe as revealed by diffusion-weighted imaging probabilistic tractography. J. Cogn. Neurosci. 24, 1998–2014 (2012).

  87. 87.

    & Fiber Pathways of the Brain (Oxford Univ. Press, 2009).

  88. 88.

    , & Transport for language south of the Sylvian fissure: the routes and history of the main tracts and stations in the ventral language network. Cortex 69, 141–151 (2015).

  89. 89.

    , , & Typical neural representations of action verbs develop without vision. Cereb. Cortex 22, 286–293 (2012).

  90. 90.

    & Category specific access dysphasia. Brain 106, 859–878 (1983).

  91. 91.

    & The impact of synaptic depression following brain damage: a connectionist account of ‘access/refractory’ and ‘degraded-store’ semantic impairments. Cogn. Affect. Behav. Neurosci. 2, 187–213 (2002).

  92. 92.

    et al. Temporal lobe lesions and semantic impairment: a comparison of herpes simplex virus encephalitis and semantic dementia. Brain 130, 1138–1147 (2007).

  93. 93.

    , , & Naming manipulable objects: anatomy of a category specific effect in left temporal tumours. Neuropsychologia 48, 1583–1597 (2010).

  94. 94.

    Exploring the Link Between Visual Impairment and Pure Alexia. PhD thesis, Univ. Manchester (2009).

  95. 95.

    & Category specificity in mind and brain? Behav. Brain Sci. 24, 497–509 (2001).

  96. 96.

    , & Semantic category dissociations: a longitudinal study of two cases. Cortex 33, 441–461 (1997).

  97. 97.

    et al. Recovery from herpes simplex encephalitis: selective impairment of specific semantic categories with neuroradiological correlation. J. Neurol. Neurosurg. Psychiatry 51, 1284–1293 (1988).

  98. 98.

    , , , & Neural systems behind word and concept retrieval. Cognition 92, 179–229 (2004).

  99. 99.

    & Complementary neural representations for faces and words: a computational exploration. Cogn. Neuropsychol. 28, 251–275 (2011).

  100. 100.

    & A computational model of semantic memory impairment: modality-specificity and emergent category-specificity. J. Exp. Psychol. Gen. 120, 339–357 (1991).

  101. 101.

    & Semantic retrieval, mnemonic control, and prefrontal cortex. Behav. Cogn. Neurosci. Rev. 1, 206–218 (2002).

  102. 102.

    , & Deep learning. Nature 521, 436–444 (2015).

  103. 103.

    , & in Parallel Distributed Processing: Explorations in the Microstructure of Cognition Vol. 1 (eds Rumelhart, D. E., McClelland, J. L. & the PDP Research Group) 3–44 (MIT Press, 1986).

  104. 104.

    , , , & Meta-analyses of object naming: effect of baseline. Hum. Brain Mapp. 25, 70–82 (2005).

  105. 105.

    et al. Minimizing within-experiment and within-group effects in activation likelihood estimation meta-analyses. Hum. Brain Mapp. 33, 1–13 (2012).

  106. 106.

    The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97–113 (1971).

  107. 107.

    et al. Coordinate-based activation likelihood estimation meta-analysis of neuroimaging data: a random-effects approach based on empirical estimates of spatial uncertainty. Hum. Brain Mapp. 30, 2907–2926 (2009).

  108. 108.

    , , , & The variation of function across the human insula mirrors its patterns of structural connectivity: evidence from in vivo probabilistic tractography. Neuroimage 59, 3514–3521 (2012).

  109. 109.

    & Differential contributions of bilateral ventral anterior temporal lobe and left anterior superior temporal gyrus to semantic processes. J. Cogn. Neurosci. 23, 3121–3131 (2011).

  110. 110.

    , & Category-specific versus category-general semantic impairment induced by transcranial magnetic stimulation. Curr. Biol. 20, 964–968 (2010).

  111. 111.

    & Probabilistic anatomical connectivity derived from the microscopic persistent angular structure of cerebral tissue. Phil. Trans. R. Soc. B 360, 893–902 (2005).

  112. 112.

    LENS: The Light, Efficient Network Simulator. Technical Report CMU-CS-99-164 (Carnegie Mellon Univ., 1999).

  113. 113.

    , & Learning Representations by Back-Propagating Errors (MIT Press, 1988).

  114. 114.

    & Lost in semantic space: a multi-modal, non-verbal assessment of feature knowledge in semantic dementia. Brain 129, 1152–1163 (2006).

  115. 115.

    , & The interaction of object form and object meaning in the identification performance of a patient with category-specific visual agnosia. Cogn. Neuropsychol. 14, 1085–1130 (1997).

  116. 116.

    , , & Object identification deficits in dementia of the Alzheimer type: combined effects of semantic and visual proximity. J. Int. Neuropsychol. Soc. 5, 330–345 (1999).

  117. 117.

    & Simulating consistency effects and individual differences in nonword naming: a comparison of current models. J. Mem. Lang. 54, 145–160 (2006).

Download references

Acknowledgements

This research was supported by a programme grant from the Medical Research Council (MRC, UK, MR/J004146/1) to M.A.L.R. and by a University Fellowship from UW-Madison to L.C. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. We thank L. Cloutman for assisting with the tractography analysis and R. Ishibashi for assisting with the ALE analysis.

Author information

Affiliations

  1. Department of Psychology, University of Wisconsin-Madison, 1202 West Johnson Street, Madison, Wisconsin 53705, USA

    • Lang Chen
    •  & Timothy T. Rogers
  2. Stanford Cognitive and Systems Neuroscience Laboratory, 1070 Arastradero Road Suite 220, Palo Alto, California 94304, USA

    • Lang Chen
  3. Neuroscience and Aphasia Research Unit (NARU), Division of Neuroscience and Experimental Psychology, School of Biological Sciences, University of Manchester, M13 9PL, UK

    • Matthew A. Lambon Ralph

Authors

  1. Search for Lang Chen in:

  2. Search for Matthew A. Lambon Ralph in:

  3. Search for Timothy T. Rogers in:

Contributions

All authors contributed to the entire process of this project, including project planning, experiment work, data analysis and writing the paper.

Competing interests

The authors declare no competing interests.

Corresponding authors

Correspondence to Lang Chen or Matthew A. Lambon Ralph or Timothy T. Rogers.

Supplementary information

PDF files

  1. 1.

    Supplementary Information

    Supplementary Figures 1–8, Supplementary tables 1–7, Supplementary Discussion, Supplementary Methods, Supplementary References

About this article

Publication history

Received

Accepted

Published

DOI

https://doi.org/10.1038/s41562-016-0039