How is knowledge about word meaning represented in the mental lexicon? Current computational models infer word meanings from lexical co-occurrence patterns. They learn to represent words as vectors in a multidimensional space, wherein words that are used in more similar linguistic contexts—that is, are more semantically related—are located closer together. However, whereas inter-word proximity captures only overall relatedness, human judgements are highly context dependent. For example, dolphins and alligators are similar in size but differ in dangerousness. Here, we use a domain-general method to extract context-dependent relationships from word embeddings: ‘semantic projection’ of word-vectors onto lines that represent features such as size (the line connecting the words ‘small’ and ‘big’) or danger (‘safe’ to ‘dangerous’), analogous to ‘mental scales’. This method recovers human judgements across various object categories and properties. Thus, the geometry of word embeddings explicitly represents a wealth of context-dependent world knowledge.
This is a preview of subscription content
Subscribe to Nature+
Get immediate online access to the entire Nature family of 50+ journals
Subscribe to Journal
Get full journal access for 1 year
only $9.92 per issue
All prices are NET prices.
VAT will be added later in the checkout.
Tax calculation will be finalised during checkout.
Get time limited or full article access on ReadCube.
All prices are NET prices.
Marr, D. in Vision: A Computational Investigation into the Human Representation and Processing of Visual Information (ed. Marr, D.) 8–38 (MIT Press, 1982).
Goldberg, A. E. Constructions: A Construction Grammar Approach to Argument Structure (Univ. of Chicago Press, 1995).
Jackendoff, R. Foundation of Language: Brain, Meaning, Grammar, Evolution (Oxford Univ. Press, 2002).
Murphy, G. The Big Book of Concepts (MIT Press, 2004).
Jackendoff, R. A User’s Guide to Thought and Meaning (Oxford Univ. Press, 2012).
Steinberg, D. D. & Jakobovits, L. A. Semantics: An Interdisciplinary Reader in Philosophy, Linguistics and Psychology. (Cambridge Univ. Press, 1971).
Richards, M. M. in Language Development, Vol. 1: Syntax and Semantics Vol. 1 (ed. S. Kuczaj) 365–396 (Routledge, 1982).
Pinker, S. & Levin, B. Lexical and Conceptual Semantics (MIT Press, 1991).
Pustejovsky, J. Semantics and the Lexicon Vol. 49 (Springer, 2012).
Quillian, M. R. Semantic Memory. PhD thesis, Carnegie Intitute of Technology (1966).
Tulving, E. in Organization of Memory Vol. 1 (eds Tulving E. & Donaldson W.) 381–403 (Academic, 1972).
Gleitman, L. & Papafragou, A. in The Oxford Handbook of Cognitive Psychology (ed D. Resiberg) 255–275 (Oxford Univ. Press, 2013).
Jackendoff, R. Parts and boundaries. Cognition 41, 9–45 (1991).
Smith, N. J. & Levy, R. The effect of word predictability on reading time is logarithmic. Cognition 128, 302–319 (2013).
Skarabela, B., Ota, M., O’Connor, R. & Arnon, I. ‘Clap your hands’ or ‘take your hands’? One-year-olds distinguish between frequent and infrequent multiword phrases. Cognition 211, 104612 (2021).
Monsalve, I. F., Frank, S. L. & Vigliocco, G. in Proc. 13th Conference of the European Chapter of the Association for Computational Linguistics, 398–408 (Association for Computational Linguistics, 2012).
Frank, S. & Thompson, R. Early effects of word surprisal on pupil size during reading. In Proc. 34th Annual Conference of the Cognitive Science Society Vol. 34 (eds Miyake, N. et al.) 1554–1559 (Cognitive Science Society, 2012).
Willems, R. M., Frank, S. L., Nijhof, A. D., Hagoort, P. & Van den Bosch, A. Prediction during natural language comprehension. Cereb. Cortex 26, 2506–2516 (2015).
McDonald, S. & Ramscar, M. Testing the distributional hypothesis: the influence of context on judgements of semantic similarity. In Proc. 23rd Annual Conference of the Cognitive Science Society https://escholarship.org/uc/item/6959p7b0 (2001).
Ellis, N. C. & Simpson-Vlach, R. Formulaic language in native speakers: triangulating psycholinguistics, corpus linguistics, and education. Corpus Linguist. Linguistic Theory 5, 61–78 (2009).
Louwerse, M. M. Embodied relations are encoded in language. Psychonomic Bull. Rev. 15, 838–844 (2008).
De Saussure, F. Course in General Linguistics (Columbia Univ. Press, 2011).
Wittgenstein, L. Philosophical Investigations. §114–115 (Wiley-Blackwell, 2010).
Harris, Z. S. Distributional structure. Word 10, 146–162 (1954).
Firth, J. R. in Studies in Linguistic Analysis Special volume of the Philological Society (ed. Firth, J. R.) 1–31 (Blackwell, 1957).
Miller, G. A. & Charles, W. G. Contextual correlates of semantic similarity. Lang. Cogn. Process. 6, 1–28 (1991).
Sahlgren, M. The distributional hypothesis. Ital. J. Disabil. Stud. 20, 33–53 (2008).
Bengio, Y., Ducharme, R., Vincent, P. & Jauvin, C. A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003).
Huang, E. H., Socher, R., Manning, C. D. & Ng, A. Y. in Proc. 50th Annual Meeting of the Association for Computational Linguistics Vol. 1: Long Papers, 873–882 (Association for Computational Linguistics, 2012).
Collobert, R. et al. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011).
Lenci, A. Distributional semantics in linguistic and cognitive research. Ital. J. Ling. 20, 1–31 (2008).
Erk, K. Vector space models of word meaning and phrase meaning: a survey. Lang. Linguist. Compass 6, 635–653 (2012).
Clark, S. in Handbook of Contemporary Semantics (eds Lappin S. & Fox C.) 493–522 (Blackwell, 2015).
Turney, P. D. & Pantel, P. From frequency to meaning: vector space models of semantics. J. Artif. Intell. Res. 37, 141–188 (2010).
Baroni, M., Dinu, G. & Kruszewski, G. in Proc. 52nd Annual Meeting of the Association for Computational Linguistics Vol. 1: Long Papers, 238–247 (Association for Computational Linguistics, 2014).
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. & Dean, J. in Proc. 26th International Conference on Neural Information Processing Systems, 3111–3119 (Curran Associates, Inc., 2013).
Pennington, J., Socher, R. & Manning, C. in Proc. 2014 Conference on Empirical Methods in Natural Language Processing, 1532–1543 (Association for Computational Linguistics, 2014).
Pereira, F., Gershman, S., Ritter, S. & Botvinick, M. A comparative evaluation of off-the-shelf distributed semantic representations for modelling behavioural data. Cogn. Neuropsychol. 33, 175–190 (2016).
Levy, O. & Goldberg, Y. in Advances in Neural Information Processing Systems, 2177–2185 (Curran Associates, Inc., 2014).
Lu, H., Wu, Y. N. & Holyoak, K. J. Emergence of analogy from relation learning. Proc. Natl Acad. Sci. U. S. A. 116, 4176–4181 (2019).
Rogers, A., Drozd, A. & Li, B. in Proc. 6th Joint Conference on Lexical and Computational Semantics, 135–148 (Association for Computational Linguistics, 2017).
Peterson, J. C., Chen, D. & Griffiths, T. L. Parallelograms revisited: exploring the limitations of vector space models for simple analogies. Cognition 205, 104440 (2020).
Osgood, C. E. The nature and measurement of meaning. Psychol. Bull. 49, 197 (1952).
Osgood, C. E. Semantic differential technique in the comparative study of cultures. Am. Anthropol. 66, 171–200 (1964).
Kozima, H. & Ito, A. Context-sensitive measurement of word distance by adaptive scaling of a semantic space. In Proc. RANLP-95, 161–168 (John Benjamins Publishing Company, 1995).
Bates, D., Mächler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. J. Stat. Softw. 67, 1–48 (2015).
Peters, M. E. et al. in Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2227–2237 (Association for Computational Linguistics, 2018).
Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. BERT: pre-training of deep bidirectional transformers for language understanding. In Proc. 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Vol. 1: Long and Short Papers, 4171–4186 (Association for Computational Linguistics, 2019).
Vaswani, A. et al. Attention is all you need. In Proc. Advances in Neural Information Processing Systems 30 (NIPS 2017), 5998–6008 (Curran Associates, Inc., 2017).
Huebner, P. A. & Willits, J. A. Structured semantic knowledge can emerge automatically from predicting word sequences in child-directed speech. Front. Psychol. 9, 133 (2018).
Unger, L. & Fisher, A. V. The emergence of richly organized semantic knowledge from simple statistics: a synthetic review. Dev. Rev. 60, 100949 (2021).
Caliskan, A., Bryson, J. J. & Narayanan, A. Semantics derived automatically from language corpora contain human-like biases. Science 356, 183–186 (2017).
Lewis, M. & Lupyan, G. Gender stereotypes are reflected in the distributional structure of 25 languages. Nat. Hum. Behav. 4, 1021–1028 (2020).
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V. & Kalai, A. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Proc. 30th International Conference on Neural Information Processing Systems (NIPS 2016), 4356–4364 (Curran Associates, Inc., 2016).
Kozlowski, A. C., Taddy, M. & Evans, J. A. The geometry of culture: analyzing the meanings of class through word embeddings. Am. Sociol. Rev. 84, 905–949 (2019).
Tshitoyan, V. et al. Unsupervised word embeddings capture latent knowledge from materials science literature. Nature 571, 95–98 (2019).
Johns, B. T. & Jones, M. N. Perceptual inference through global lexical similarity. Top. Cog. Sci. 4, 103–120 (2012).
Herbelot, A. & Vecchi, E. M. in Proc. 2015 Conference on Empirical Methods in Natural Language Processing, 22–32 (Association for Computational Linguistics, 2015).
Gupta, A., Boleda, G., Baroni, M. & Padó, S. in Proc. 2015 Conference on Empirical Methods in Natural Language Processing, 12–21 (Association for Computational Linguistics, 2015).
Utsumi, A. Exploring what is encoded in distributional word vectors: a neurobiologically motivated analysis. Cogn. Sci. 44, e12844 (2020).
Ichien, N., Lu, H. & Holyoak, K. J. Predicting patterns of similarity among abstract semantic relations. J. Exp. Psychol. Learning Memory Cogn. https://doi.org/10.1037/xlm0001010 (2021).
Iordan, M. C., Giallanza, T., Ellis, C. T., Beckage, N. M. & Cohen, J. D. Context matters: recovering human semantic structure from machine learning analysis of large-scale text corpora. Cogn. Sci. 46, e13085 (2022).
Laurence, S. & Margolis, E. in Concepts: Core Readings (eds Laurence, S. & Margolis, E.) 3–81 (MIT Press, 1999).
Markman, A. B. Knowledge Representation (Lawrence Erlbaum, 2013).
Mahon, B. Z. & Hickok, G. Arguments about the nature of concepts: symbols, embodiment, and beyond. Psychon. Bull. Rev. 23, 941–958 (2016).
Yee, E., Jones, M. & McRae, K. in Stevens’ Handbook of Experimental Psychology, Memory and Cognitive Processes Vol. 2 (ed Wixted J.) (Wiley, 2014).
Rosch, E. & Mervis, C. B. Family resemblances: studies in the internal structure of categories. Cogn. Psychol. 7, 573–605 (1975).
Smith, E. E. & Medin, D. L. Categories and Concepts (Harvard Univ. Press, 1981).
Rumelhart, D. & Ortony, A. in Schooling and the Acquisition of Knowledge (eds Anderson R. C., Spiro R. J., & Montague W. E.) 99–135 (Lawrence Erlbaum, 1977).
Gopnik, A., Meltzoff, A. N. & Bryant, P. Words, Thoughts, and Theories, Vol. 1 (MIT Press, 1997).
Gopnik, A. in Chomsky and His Critics (eds Antony L. & Hornstein N.) 238–254 (Blackwell, 2003).
Medin, D. L. & Schaffer, M. M. Context theory of classification learning. Psych. Rev. 85, 207 (1978).
Poesio, M. & Almuhareb, A. in Proc. Association for Computational Linguistics SIGLEX Workshop on Deep Lexical Acquisition, 18–27 (Association for Computational Linguistics, 2005).
Barbu, E. in Proc. ESSLLI Workshop on Distributional Lexical Semantics, 9–16 (Association for Logic, Language and Information, 2008).
Baroni, M. & Lenci, A. in Proc. Workshop on Geometrical Models of Natural Language Semantics, 1–8 (Association for Computational Linguistics, 2009).
Baroni, M., Murphy, B., Barbu, E. & Poesio, M. Strudel: a corpus-based semantic model based on properties and types. Cogn. Sci. 34, 222–254 (2010).
Rubinstein, D., Levi, E., Schwartz, R. & Rappoport, A. in Proc. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing Vol. 2: Short Papers, 726–730 (Association for Computational Linguistics, 2015).
Kelly, C., Devereux, B. & Korhonen, A. Automatic extraction of property norm‐like data from large text corpora. Cogn. Sci. 38, 638–682 (2014).
Lupyan, G. & Lewis, M. From words-as-mappings to words-as-cues: the role of language in semantic knowledge. Lang. Cogn. Neurosci. 34, 1319–1337 (2019).
Rumelhart, D. E. in Metaphor and Thought (ed. Andrew Ortony) 71–82 (Cambridge Univ. Press, 1979).
Landauer, T. K. & Dumais, S. T. A solution to Plato’s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychol. Rev. 104, 211–240 (1997).
Elman, J. L. An alternative view of the mental lexicon. Trends Cogn. Sci. 8, 301–306 (2004).
Elman, J. L. On the meaning of words and dinosaur bones: lexical knowledge without a lexicon. Cogn. Sci. 33, 547–582 (2009).
Lupyan, G. & Bergen, B. How language programs the mind. Top. Cogn. Sci. 8, 408–424 (2016).
Clifton, C., Frazier, L. & Connine, C. Lexical expectations in sentence comprehension. J. Verbal Learn. Verbal Behav. 23, 696–708 (1984).
MacDonald, M. C., Pearlmutter, N. J. & Seidenberg, M. S. The lexical nature of syntactic ambiguity resolution. Psychol. Rev. 101, 676–703 (1994).
Trueswell, J. C., Tanenhaus, M. K. & Garnsey, S. M. Semantic influences on parsing: use of thematic role information in syntactic ambiguity resolution. J. Mem. Lang. 33, 285–318 (1994).
Garnsey, S. M., Pearlmutter, N. J., Myers, E. & Lotocky, M. A. The contributions of verb bias and plausibility to the comprehension of temporarily ambiguous sentences. J. Mem. Lang. 37, 58–93 (1997).
Hale, J. in Proc. Second Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies, 1–8 (Association for Computational Linguistics, 2001).
Traxler, M. J., Morris, R. K. & Seely, R. E. Processing subject and object relative clauses: evidence from eye movements. J. Mem. Lang. 47, 69–90 (2002).
Gennari, S. P. & MacDonald, M. C. Semantic indeterminacy in object relative clauses. J. Mem. Lang. 58, 161–187 (2008).
Levy, R. P. Expectation-based syntactic comprehension. Cognition 106, 1126–1177 (2008).
Marmor, G. S. Age at onset of blindness and the development of the semantics of color names. J. Exp. Child Psych. 25, 267–278 (1978).
Landau, B. & Gleitman, L. R. Language and Experience: Evidence from the Blind Child, Vol. 8 (Harvard Univ. Press, 2009).
Shepard, R. N. & Cooper, L. A. Representation of colors in the blind, color-blind, and normally sighted. Psychol. Sci. 3, 97–104 (1992).
Noppeney, U., Friston, K. J. & Price, C. J. Effects of visual deprivation on the organization of the semantic system. Brain 126, 1620–1627 (2003).
Bedny, M., Caramazza, A., Pascual-Leone, A. & Saxe, R. Typical neural representations of action verbs develop without vision. Cereb. Cortex 22, 286–293 (2011).
Bedny, M., Koster-Hale, J., Elli, G., Yazzolino, L. & Saxe, R. There’s more to “sparkle” than meets the eye: knowledge of vision and light verbs among congenitally blind and sighted individuals. Cognition 189, 105–115 (2019).
Louwerse, M. & Connell, L. A taste of words: linguistic context and perceptual simulation predict the modality of words. Cogn. Sci. 35, 381–398 (2011).
Baroni, M. & Lenci, A. Concepts and properties in word spaces. Ital. J. Ling. 20, 55–88 (2008).
Andrews, M., Vigliocco, G. & Vinson, D. Integrating experiential and distributional data to learn semantic representations. Psych. Rev. 116, 463–498 (2009).
Riordan, B. & Jones, M. N. Redundancy in perceptual and linguistic experience: comparing feature‐based and distributional models of semantic representation. Top. Cogn. Sci. 3, 303–345 (2011).
Hill, F., Reichart, R. & Korhonen, A. Simlex-999: eEvaluating semantic models with (genuine) similarity estimation. Comput. Linguist. 41, 665–695 (2015).
Kim, J. S., Elli, G. V. & Bedny, M. Knowledge of animal appearance among sighted and blind adults. Proc. Natl Acad. Sci. U. S. A. 116, 11213–11222 (2019).
Kim, J. S., Elli, G. V. & Bedny, M. Reply to Ostarek et al.: Language, but not co-occurrence statistics, is useful for learning animal appearance. Proc. Natl Acad. Sci. U. S. A. 116, 21974–21975 (2019).
Kim, J. S., Elli, G. V. & Bedny, M. Reply to Lewis et al.: Inference is key to learning appearance from language, for humans and distributional semantic models alike. Proc. Natl Acad. Sci. U. S. A. 116, 19239–19240 (2019).
Ostarek, M., Van Paridon, J. & Montero-Melis, G. Sighted people’s language is not helpful for blind individuals’ acquisition of typical animal colors. Proc. Natl Acad. Sci. U. S. A. 116, 21972–21973 (2019).
Lewis, M., Zettersten, M. & Lupyan, G. Distributional semantics as a source of visual knowledge. Proc. Natl Acad. Sci. U. S. A. 116, 19237–19238 (2019).
Majid, A. et al. Differential coding of perception in the world’s languages. Proc. Natl Acad. Sci. U. S. A. 115, 11369–11376 (2018).
Clark, E. V. in Cognitive Development and the Acquisition of Language (ed. Moskowitz B. A.) 223–260 (Academic, 1973).
Binder, J. R. et al. Toward a brain-based componential semantic representation. Cogn. Neuropsychol. 33, 130–174 (2016).
Barsalou, L. W. & Sewell, D. R. Contrasting the representation of scripts and categories. J. Mem. Lang. 24, 646–665 (1985).
Tanaka, J. W. & Taylor, M. Object categories and expertise: is the basic level in the eye of the beholder? Cogn. Psych. 23, 457–482 (1991).
Baroni, M. & Zamparelli, R. in Proc. 2010 Conference on Empirical Methods in Natural Language Processing, 1183–1193 (Association for Computational Linguistics, 2010).
Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S. & Dean, J. in NeurIPS (prev. NIPS), 3111–3119 (Curran Associates, Inc., 2013).
Mahowald, K., Isola, P., Fedorenko, E., Gibson, E. & Oliva, A. Memorable words are monogamous: the role of synonymy and homonymy in word recognition memory. Preprint at PsyArxiv https://psyarxiv.com/p6kv9/ (2018).
Paivio, A., Yuille, J. C. & Madigan, S. A. Concreteness, imagery, and meaningfulness values for 925 nouns. J. Exp. Psychol. 76, 1–25 (1968).
Battig, W. F. & Montague, W. E. Category norms of verbal items in 56 categories A replication and extension of the Connecticut category norms. J. Exp. Psychol. 80, 1–46 (1969).
Cree, G. S. & McRae, K. Analyzing the factors underlying the structure and computation of the meaning of chipmunk, cherry, chisel, cheese, and cello (and many other such concrete nouns). J. Exp. Psychol. Gen. 132, 163–201 (2003).
McRae, K., Cree, G. S., Seidenberg, M. S. & McNorgan, C. Semantic feature production norms for a large set of living and nonliving things. Behav. Res. Methods 37, 547–559 (2005).
Pereira, F., Botvinick, M. & Detre, G. Using Wikipedia to learn semantic feature representations of concrete concepts in neuroimaging experiments. Artif. Intell. 194, 240–252 (2013).
Nelson, D. L., McEvoy, C. L. & Schreiber, T. A. The University of South Florida free association, rhyme, and word fragment norms. Behav. Res. Methods 36, 402–407 (2004).
Brysbaert, M. & New, B. Moving beyond Kučera and Francis: a critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Beh. Res. Methods 41, 977–990 (2009).
Buhrmester, M., Kwang, T. & Gosling, S. D. Amazon’s Mechanical Turk: a new source of inexpensive, yet high-quality, data? Perspect. Psychol. Sci. 6, 3–5 (2011).
Silver, N. C. & Dunlap, W. P. Averaging correlation coefficients: should Fisher’s z transformation be used? J. Appl. Psychol. 72, 146 (1987).
Benjamini, Y. & Yekutieli, D. The control of the false discovery rate in multiple testing under dependency. Ann. Stat. 29, 1165–1188 (2001).
F.P. was supported by the National Institute of Mental Health Intramural Research Program ZIC-MH002968. E.F. was supported by R01 awards DC016607 and DC016950, U01 award NS6945189 and funds from the McGovern Institute for Brain Research, the Department of Brain and Cognitive Sciences and the Simons Center for the Social Brain. This work was partially supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via Air Force Research Laboratory (AFRL), under contract FA8650-14-C-7358 (to I.A.B., F.P. and E.F.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript. The authors are responsible for all aspects of the study, and views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of ODNI, IARPA, AFRL or the US Government. The US Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation thereon.
The authors declare no competing interests.
Peer review information
Nature Human Behaviour thanks Gary Lupyan, Katrin Erk and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Conventions are the same as in Fig. 3 in the manuscript. Descriptive statistics across all tested pairs: (a) Pearson’s correlation: med = 0.41 (CI95 = 0.29-0.50, IQR = 0.26-0.57), adjusted med = 0.44 (CI95 = 0.35-0.53, IQR = 0.29-0.60). (b) OCp: med = 64% (CI95 = 61-68%, IQR = 57-73%), adjusted med = 73% (CI95 = 70-78%, IQR = 67-81%).
Conventions are the same as in Fig. 3 in the manuscript. Descriptive statistics across all tested pairs: (a) Pearson’s correlation: med = 0.33 (CI95 = 0.27-0.40, IQR = 0.22-0.44), adjusted med = 0.35 (CI95 = 0.28-0.43, IQR = 0.24-0.47). (b) OCp: med = 62% (CI95 = 57-65%, IQR = 55-67%), adjusted med = 71% (CI95 = 65-74%, IQR = 63-78%).
Conventions are the same as in Fig. 3 in the manuscript. Descriptive statistics across all tested pairs: (a) Pearson’s correlation: med = 0.26 (CI95 = 0.20-0.36, IQR = 0.14-0.43), adjusted med = 0.31 (CI95 = 0.21-0.41, IQR = 0.15-0.45). (b) OCp: med = 59% (CI95 = 57-63%, IQR = 55-66%), adjusted med = 70% (CI95 = 65-73%, IQR = 63-76%).
Conventions are the same as in Fig. 3 in the manuscript. Descriptive statistics across all tested pairs: (a) Pearson’s correlation: med = 0.42 (CI95 = 0.35-0.47, IQR = 0.20-0.54), adjusted med = 0.44 (CI95 = 0.40-0.50, IQR = 0.25-0.57). (b) OCp: med = 65% (CI95 = 62-67%, IQR = 56-72%), adjusted med = 74% (CI95 = 72-76%, IQR = 67-80%).
Extended Data Fig. 9 Evaluating how well different word embeddings capture conceptual category structure.
Each matrix shows Pearson’s correlations between all pairs of word vectors for all items used in our study, grouped by category (indicated on the y-axis), for a different embedding. Color corresponds to correlation strength, with dark blue corresponding to -1 and red corresponding to 1. Qualitatively, all three embeddings capture categorical structure, as is evidenced by the block-diagonal structure of the correlation matrix. Nonetheless, ELMo appears to generate highly similar vectors for words sharing a category (the diagonal blocks are colored in strong red), indicating a poorer ability to distinguish among within-category items, compared to the other two embeddings. In contrast, BERT appears to separate items from across different categories more poorly than the other two embeddings (the color differences between the diagonal blocks and the rest of the matrix are somewhat weak).
About this article
Cite this article
Grand, G., Blank, I.A., Pereira, F. et al. Semantic projection recovers rich human knowledge of multiple object features from word embeddings. Nat Hum Behav (2022). https://doi.org/10.1038/s41562-022-01316-8