Abstract
Language and social cognition are traditionally studied as separate cognitive domains, yet accumulative studies reveal overlapping neural correlates at the left ventral temporoparietal junction (vTPJ) and the left lateral anterior temporal lobe (lATL), which have been attributed to sentence processing and social concept activation. We propose a common cognitive component underlying both effects: social-semantic working memory. We confirmed two key predictions of our hypothesis using functional MRI. First, the left vTPJ and lATL showed sensitivity to sentences only when the sentences conveyed social meaning; second, these regions showed persistent social-semantic-selective activity after the linguistic stimuli disappeared. We additionally found that both regions were sensitive to the socialness of non-linguistic stimuli and were more tightly connected with the social-semantic-processing areas than with the sentence-processing areas. The converging evidence indicates the social-semantic working-memory function of the left vTPJ and lATL and challenges the general-semantic and/or syntactic accounts for the neural activity of these regions.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
All data that support the findings of this study are available from Psychological Science Bank (https://doi.org/10.57760/sciencedb.psych.00138).
Code availability
Custom code that supports the findings of this study is available from Psychological Science Bank (https://doi.org/10.57760/sciencedb.psych.00138).
References
de Villiers, J. The interface of language and theory of mind. Lingua 117, 1858–1878 (2007).
Richardson, H. et al. Reduced neural selectivity for mental states in deaf children with delayed exposure to sign language. Nat. Commun. 11, 3246 (2020).
Scott, S. K. From speech and talkers to the social world: the neural processing of human spoken language. Science 366, 58–62 (2019).
Dunbar, R. I. M. Gossip in evolutionary perspective. Rev. Gen. Psychol. 8, 100–110 (2004).
Bzdok, D. et al. Left inferior parietal lobe engagement in social cognition and language. Neurosci. Biobehav. Rev. 68, 319–334 (2016).
Mar, R. A. The neural bases of social cognition and story comprehension. Annu. Rev. Psychol. 62, 103–134 (2011).
Mellem, M. S., Jasmin, K. M., Peng, C. & Martin, A. Sentence processing in anterior superior temporal cortex shows a social–emotional bias. Neuropsychologia 89, 217–224 (2016).
Diveica, V., Koldewyn, K. & Binney, R. J. Establishing a role of the semantic control network in social cognitive processing: a meta-analysis of functional neuroimaging studies. NeuroImage 245, 118702 (2021).
Schurz, M. et al. Toward a hierarchical model of social cognition: a neuroimaging meta-analysis and integrative review of empathy and theory of mind. Psychol. Bull. 147, 293–327 (2021).
Binney, R. J. & Ramsey, R. Social semantics: the role of conceptual knowledge and cognitive control in a neurobiological model of the social brain. Neurosci. Biobehav. Rev. 112, 28–38 (2020).
Pexman, P. M., Diveica, V. & Binney, R. J. Social semantics: the organization and grounding of abstract concepts. Phil. Trans. R. Soc. B 378, 20210363 (2022).
Arioli, M., Gianelli, C. & Canessa, N. Neural representation of social concepts: a coordinate-based meta-analysis of fMRI studies. Brain Imaging Behav. 15, 1912–1921 (2021).
Lin, N. et al. Dissociating the neural correlates of the sociality and plausibility effects in simple conceptual combination. Brain Struct. Funct. 225, 995–1008 (2020).
Zhang, G., Hung, J. & Lin, N. Coexistence of the social semantic effect and non-semantic effect in the default mode network. Brain Struct. Funct. 228, 321–339 (2023).
Zahn, R. et al. Social concepts are represented in the superior anterior temporal cortex. Proc. Natl Acad. Sci. USA 104, 6430–6435 (2007).
Tamir, D. I., Thornton, M. A., Contreras, J. M. & Mitchell, J. P. Neural evidence that three dimensions organize mental state representation: rationality, social impact, and valence. Proc. Natl Acad. Sci. USA 113, 194–199 (2016).
Contreras, J. M., Banaji, M. R. & Mitchell, J. P. Dissociable neural correlates of stereotypes and other forms of semantic knowledge. Soc. Cogn. Affect. Neurosci. 7, 764–770 (2012).
Saxe, R. & Wexler, A. Making sense of another mind: the role of the right temporo-parietal junction. Neuropsychologia 43, 1391–1399 (2005).
Lin, N., Bi, Y., Zhao, Y., Luo, C. & Li, X. The theory-of-mind network in support of action verb comprehension: evidence from an fMRI study. Brain Lang. 141, 1–10 (2015).
Lin, N. et al. Fine subdivisions of the semantic network supporting social and sensory–motor semantic processing. Cereb. Cortex 28, 2699–2710 (2018).
Spunt, R. P., Kemmerer, D. & Adolphs, R. The neural basis of conceptualizing the same action at different levels of abstraction. Soc. Cogn. Affect. Neurosci. 11, 1141–1151 (2016).
Lin, N. et al. Coin, telephone, and handcuffs: neural correlates of social knowledge of inanimate objects. Neuropsychologia 133, 107187 (2019).
Dronkers, N. F. et al. Lesion analysis of the brain areas involved in language comprehension. Cognition 92, 145–177 (2004).
Fedorenko, E., Behr, M. K. & Kanwisher, N. Functional specificity for high-level linguistic processing in the human brain. Proc. Natl Acad. Sci. USA 108, 16428–16433 (2011).
Pallier, C., Devauchelle, A.-D. & Dehaene, S. Cortical representation of the constituent structure of sentences. Proc. Natl Acad. Sci. USA 108, 2522–2527 (2011).
Malik-Moraleda, S. et al. An investigation across 45 languages and 12 language families reveals a universal language network. Nat. Neurosci. 25, 1014–1019 (2022).
Zaccarella, E., Schell, M. & Friederici, A. D. Reviewing the functional basis of the syntactic Merge mechanism for language: a coordinate-based activation likelihood estimation meta-analysis. Neurosci. Biobehav. Rev. 80, 646–656 (2017).
Fedorenko, E. & Thompson-Schill, S. L. Reworking the language network. Trends Cogn. Sci. 18, 120–126 (2014).
Labache, L. et al. A SENtence Supramodal Areas AtlaS (SENSAAS) based on multiple task-induced activation mapping and graph analysis of intrinsic connectivity in 144 healthy right-handers. Brain Struct. Funct. 224, 859–882 (2019).
Matchin, W., Brodbeck, C., Hammerly, C. & Lau, E. The temporal dynamics of structure and content in sentence comprehension: evidence from fMRI-constrained MEG. Hum. Brain Mapp. 40, 663–678 (2019).
Humphries, C., Binder, J. R., Medler, D. A. & Liebenthal, E. Syntactic and semantic modulation of neural activity during auditory sentence comprehension. J. Cogn. Neurosci. 18, 665–679 (2006).
Price, C. J. The anatomy of language: a review of 100 fMRI studies published in 2009. Ann. N. Y. Acad. Sci. 1191, 62–88 (2010).
Christophel, T. B., Klink, P. C., Spitzer, B., Roelfsema, P. R. & Haynes, J.-D. The distributed nature of working memory. Trends Cogn. Sci. 21, 111–124 (2017).
D’Esposito, M. & Postle, B. R. The cognitive neuroscience of working memory. Annu. Rev. Psychol. 66, 115–142 (2015).
Fuster, J. M. Network memory. Trends Neurosci. 20, 451–459 (1997).
Postle, B. R. Working memory as an emergent property of the mind and brain. Neuroscience 139, 23–38 (2006).
Cowan, N. Attention and Memory: An Integrated Framework (Oxford Univ. Press, 1998); https://doi.org/10.1093/acprof:oso/9780195119107.001.0001
Potter, M. C., Kroll, J. F., Yachzel, B., Carpenter, E. & Sherman, J. Pictures in sentences: understanding without words. J. Exp. Psychol. Gen. 115, 281–294 (1986).
Potter, M. C. Very short-term conceptual memory. Mem. Cogn. 21, 156–161 (1993).
Potter, M. Conceptual short term memory in perception and thought. Front. Psychol. 3, 113 (2012).
Fedorenko, E., Hsieh, P.-J., Nieto-Castañón, A., Whitfield-Gabrieli, S. & Kanwisher, N. New method for fMRI investigations of language: defining ROIs functionally in individual subjects. J. Neurophysiol. 104, 1177–1194 (2010).
Zhang, G., Xu, Y., Zhang, M., Wang, S. & Lin, N. The brain network in support of social semantic accumulation. Soc. Cogn. Affect. Neurosci. 16, 393–405 (2021).
Kuhnke, P. et al. The role of the angular gyrus in semantic cognition: a synthesis of five functional neuroimaging studies. Brain Struct. Funct. 228, 273–291 (2023).
Townsend, J. T. & Ashby, F. G. Stochastic Modeling of Elementary Psychological Processes (Cambridge Univ. Press, 1983).
Paunov, A. M. et al. Differential tracking of linguistic vs. mental state content in naturalistic stimuli by language and theory of mind (ToM) brain networks. Neurobiol. Lang. 3, 413–440 (2022).
Hagoort, P. & Indefrey, P. The neurobiology of language beyond single words. Annu. Rev. Neurosci. 37, 347–362 (2014).
Heard, M. & Lee, Y. S. Shared neural resources of rhythm and syntax: an ALE meta-analysis. Neuropsychologia 137, 107284 (2020).
Sreenivasan, K. K. & D’Esposito, M. The what, where and how of delay activity. Nat. Rev. Neurosci. 20, 466–481 (2019).
Postle, B. R. The cognitive neuroscience of visual short-term memory. Cogn. Control 1, 40–46 (2015).
Manoach, D. S. et al. Prefrontal cortex fMRI signal changes are correlated with working memory load. NeuroReport 8, 545–549 (1997).
Rottschy, C. et al. Modelling neural correlates of working memory: a coordinate-based meta-analysis. NeuroImage 60, 830–846 (2012).
Druzgal, T. J. & D’Esposito, M. Activity in fusiform face area modulated as a function of working memory load. Cogn. Brain Res. 10, 355–364 (2001).
Meyer, M. L., Taylor, S. E. & Lieberman, M. D. Social working memory and its distinctive link to social cognitive ability: an fMRI study. Soc. Cogn. Affect. Neurosci. 10, 1338–1347 (2015).
Xu, Y. & Chun, M. M. Dissociable neural mechanisms supporting visual short-term memory for objects. Nature 440, 91–95 (2006).
Martin, R. C., Wu, D., Freedman, M., Jackson, E. F. & Lesch, M. An event-related fMRI investigation of phonological versus semantic short-term memory. J. Neurolinguistics 16, 341–360 (2003).
Song, J.-H. & Jiang, Y. Visual working memory for simple and complex features: an fMRI study. NeuroImage 30, 963–972 (2006).
Thornton, M. A. & Conway, A. R. A. Working memory for social information: chunking or domain-specific buffer? NeuroImage 70, 233–239 (2013).
Zhao, Y., Kuai, S., Zanto, T. P. & Ku, Y. Neural correlates underlying the precision of visual working memory. Neuroscience 425, 301–311 (2020).
Amft, M. et al. Definition and characterization of an extended social–affective default network. Brain Struct. Funct. 220, 1031–1049 (2015).
Wang, Y. et al. A large-scale structural and functional connectome of social mentalizing. NeuroImage 236, 118115 (2021).
Oosterhof, N. N. & Todorov, A. The functional basis of face evaluation. Proc. Natl Acad. Sci. USA 105, 11087–11092 (2008).
Olson, I. R., McCoy, D., Klobusicky, E. & Ross, L. A. Social cognition and the anterior temporal lobes: a review and theoretical framework. Soc. Cogn. Affect. Neurosci. 8, 123–133 (2013).
Dunbar, R. I. M., Marriott, A. & Duncan, N. D. C. Human conversational behavior. Hum. Nat. 8, 231–246 (1997).
Branco, P., Seixas, D. & Castro, S. L. Mapping language with resting-state functional magnetic resonance imaging: a study on the functional profile of the language network. Hum. Brain Mapp. 41, 545–560 (2020).
Bedny, M., Pascual-Leone, A., Dodell-Feder, D., Fedorenko, E. & Saxe, R. Language processing in the occipital cortex of congenitally blind adults. Proc. Natl Acad. Sci. USA 108, 4429–4434 (2011).
Ferstl, E. C., Neumann, J., Bogler, C. & von Cramon, D. Y. The extended language network: a meta-analysis of neuroimaging studies on text comprehension. Hum. Brain Mapp. 29, 581–593 (2008).
Hagoort, P. The neurobiology of language beyond single-word processing. Science 366, 55–58 (2019).
Lin, N. et al. Neural correlates of three cognitive processes involved in theory of mind and discourse comprehension. Cogn. Affect. Behav. Neurosci. 18, 273–283 (2018).
Schurz, M., Radua, J., Aichhorn, M., Richlan, F. & Perner, J. Fractionating theory of mind: a meta-analysis of functional brain imaging studies. Neurosci. Biobehav. Rev. 42, 9–34 (2014).
Feng, W., Yu, H. & Zhou, X. Understanding particularized and generalized conversational implicatures: is theory-of-mind necessary? Brain Lang. 212, 104878 (2021).
Lewis, P. A., Rezaie, R., Brown, R., Roberts, N. & Dunbar, R. I. M. Ventromedial prefrontal volume predicts understanding of others and social network size. NeuroImage 57, 1624–1629 (2011).
Samson, D., Apperly, I. A., Chiavarino, C. & Humphreys, G. W. Left temporoparietal junction is necessary for representing someone else’s belief. Nat. Neurosci. 7, 499–500 (2004).
Wang, Y. et al. Dynamic neural architecture for social knowledge retrieval. Proc. Natl Acad. Sci. USA 114, E3305–E3314 (2017).
Rapp, A. M., Mutschler, D. E. & Erb, M. Where in the brain is nonliteral language? A coordinate-based meta-analysis of functional magnetic resonance imaging studies. NeuroImage 63, 600–610 (2012).
Yang, H. & Bi, Y. From words to phrases: neural basis of social event semantic composition. Brain Struct. Funct. 227, 1683–1695 (2022).
Wang, X., Wang, B. & Bi, Y. Close yet independent: dissociation of social from valence and abstract semantic dimensions in the left anterior temporal lobe. Hum. Brain Mapp. 40, 4759–4776 (2019).
Dingemanse, M. et al. Beyond single-mindedness: a figure–ground reversal for the cognitive sciences. Cogn. Sci. 47, e13230 (2023).
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C. & Wager, T. D. Large-scale automated synthesis of human functional neuroimaging data. Nat. Methods 8, 665–670 (2011).
Redcay, E., Velnoskey, K. R. & Rowe, M. L. Perceived communicative intent in gesture and language modulates the superior temporal sulcus. Hum. Brain Mapp. 37, 3444–3461 (2016).
Weisberg, J., Hubbard, A. L. & Emmorey, K. Multimodal integration of spontaneously produced representational co-speech gestures: an fMRI study. Lang. Cogn. Neurosci. 32, 158–174 (2017).
Hassabis, D. et al. Imagine all the people: how the brain creates and uses personality models to predict behavior. Cereb. Cortex 24, 1979–1987 (2014).
Van Overwalle, F., Ma, N. & Baetens, K. Nice or nerdy? The neural representation of social and competence traits. Soc. Neurosci. 11, 567–578 (2016).
Thornton, M. A. & Mitchell, J. P. Theories of person perception predict patterns of neural activity during mentalizing. Cereb. Cortex 28, 3505–3520 (2018).
Martin, R. C., Ding, J., Hamilton, A. C. & Schnur, T. T. Working memory capacities neurally dissociate: evidence from acute stroke. Cereb. Cortex Commun. 2, tgab005 (2021).
Yue, Q., Martin, R. C., Hamilton, A. C. & Rose, N. S. Non-perceptual regions in the left inferior parietal lobe support phonological short-term memory: evidence for a buffer account? Cereb. Cortex 29, 1398–1413 (2019).
Yue, Q. & Martin, R. C. Maintaining verbal short-term memory representations in non-perceptual parietal regions. Cortex 138, 72–89 (2021).
Meyer, M. L. & Collier, E. Theory of minds: managing mental state inferences in working memory is associated with the dorsomedial subsystem of the default network and social integration. Soc. Cogn. Affect. Neurosci. 15, 63–73 (2020).
Bemis, D. K. & Pylkkänen, L. Simple composition: a magnetoencephalography investigation into the comprehension of minimal linguistic phrases. J. Neurosci. 31, 2801 (2011).
Bemis, D. K. & Pylkkänen, L. Basic linguistic composition recruits the left anterior temporal lobe and left angular gyrus during both listening and reading. Cereb. Cortex 23, 1859–1873 (2013).
Graves, W. W. et al. Correspondence between cognitive and neural representations for phonology, orthography, and semantics in supramarginal compared to angular gyrus. Brain Struct. Funct. 228, 255–271 (2023).
Hung, J., Wang, X., Wang, X. & Bi, Y. Functional subdivisions in the anterior temporal lobes: a large scale meta-analytic investigation. Neurosci. Biobehav. Rev. 115, 134–145 (2020).
Huth, A. G., de Heer, W. A., Griffiths, T. L., Theunissen, F. E. & Gallant, J. L. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature 532, 453–458 (2016).
Seghier, M. L. The angular gyrus: multiple functions and multiple subdivisions. Neuroscientist 19, 43–61 (2013).
Price, A. R., Bonner, M. F., Peelle, J. E. & Grossman, M. Converging evidence for the neuroanatomic basis of combinatorial semantics in the angular gyrus. J. Neurosci. 35, 3276–3284 (2015).
Schell, M., Zaccarella, E. & Friederici, A. D. Differential cortical contribution of syntax and semantics: an fMRI study on two-word phrasal processing. Cortex 96, 105–120 (2017).
Chinese Linguistic Data Consortium (Tsinghua University, State Key Laboratory of Intelligent Technology and Systems, and Chinese Academy of Sciences, Institute of Automation, 2003).
Lin, N., Yu, X., Zhao, Y. & Zhang, M. Functional anatomy of recognition of Chinese multi-character words: convergent evidence from effects of transposable nonwords, lexicality, and word frequency. PLoS ONE 11, e0149583 (2016).
Vernon, R. J. W., Sutherland, C. A. M., Young, A. W. & Hartley, T. Modeling first impressions from highly variable facial images. Proc. Natl Acad. Sci. USA 111, E3353–E3361 (2014).
Gao, W. et al. The CAS-PEAL large-scale Chinese face database and baseline evaluations. IEEE Trans. Syst. Man Cybern. A Syst. Hum. 38, 149–161 (2008).
The CAS-PEAL Face Database (ICT-ISVISION Joint Research & Development Laboratory for Face Recognition, 2008); http://www.jdl.link/peal/home.htm
Free Stock Video Footage (Videvo Team, 2021); https://www.videvo.net/free-stock-footage/
Yan, C. & Zang, Y. DPARSF: a MATLAB toolbox for ‘pipeline’ data analysis of resting-state fMRI. Front. Syst. Neurosci. 4, 13 (2010).
Yan, C.-G., Wang, X.-D., Zuo, X.-N. & Zang, Y.-F. DPABI: Data Processing & Analysis for (Resting-State) Brain Imaging. Neuroinformatics 14, 339–351 (2016).
Ashburner, J. & Friston, K. J. Unified segmentation. NeuroImage 26, 839–851 (2005).
Ashburner, J. A fast diffeomorphic image registration algorithm. NeuroImage 38, 95–113 (2007).
Friston, K. J., Williams, S., Howard, R., Frackowiak, R. S. J. & Turner, R. Movement-related effects in fMRI time-series. Magn. Reson. Med. 35, 346–355 (1996).
Yan, C.-G. et al. A comprehensive assessment of regional variation in the impact of head micromovements on functional connectomics. NeuroImage 76, 183–201 (2013).
Blank, I. A. & Fedorenko, E. No evidence for differences among language regions in their temporal receptive windows. NeuroImage 219, 116925 (2020).
Bates, D., Maechler, M., Bolker, B. & Walker, S. lme4: Linear Mixed-Effects Models using 'Eigen' and S4 (version 1.1-30) R package (2007).
R Core Team. R: A Language and Environment for Statistical Computing (version 4.2.1) R Foundation for Statistical Computing (2020).
Morey, R. & Rouder, J. N. BayesFactor. R package v.0.9.12-4.3 (2015).
Rouder, J. N., Morey, R. D., Speckman, P. L. & Province, J. M. Default Bayes factors for ANOVA designs. J. Math. Psychol. 56, 356–374 (2012).
Friston, K. J., Harrison, L. & Penny, W. Dynamic causal modelling. NeuroImage 19, 1273–1302 (2003).
Meyer, D. et al. E1071: Misc Functions of the Department of Statistics, TU Wien (version 1.7-11) R package (2022).
Cortes, C. & Vapnik, V. Support-vector networks. Mach. Learn. 20, 273–297 (1995).
Acknowledgements
This research was supported by grants from the National Natural Science Foundation of China (grant no. 31871105 to N.L.); the Scientific Foundation of the Institute of Psychology, Chinese Academy of Sciences (grant no. E2CX3625CX to X.W. and N.L.); the Scientific Foundation of the Institute of Psychology, Chinese Academy of Sciences (grant no. E1CX4725CX to X.W.); the National Science and Technology Innovation 2030 Major Program (grant no. 2021ZD0204104 to Y.B.); and the National Natural Science Foundation of China (grant no. 31925020 to Y.B.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the paper. Experiment 4 in this paper used the CAS-PEAL-R1 face database collected under the sponsor of the Chinese National Hi-Tech Program and ISVISION Tech. Co. Ltd. We thank X. Wang, H. Yang, H. Wen and W. Zhou for assistance in performing the MVPA and DCM analysis.
Author information
Authors and Affiliations
Contributions
G.Z. and N.L. conceived the study. G.Z., N.L. and W.S. developed the methods. G.Z. performed the investigation and the data analysis. N.L. supervised the work. G.Z. and N.L. wrote the initial draft. All authors reviewed and edited the paper.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Human Behaviour thanks Moritz Wurm, David Kemmerer and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary Information
Supplementary Figures 1–5, Tables 1–26 and description of the methods and results of the location-based analyses of Neurosynth and the methods and results of the behavioural analysis.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, G., Xu, Y., Wang, X. et al. A social-semantic working-memory account for two canonical language areas. Nat Hum Behav 7, 1980–1997 (2023). https://doi.org/10.1038/s41562-023-01704-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41562-023-01704-8