Abstract
Human communication combines language with gesture. Gesture contributes to the uniquely human ability to communicate about an infinite number of ideas in an efficient way and to generate representations that are useful for thinking. Gesture and language can be distinguished by distinct underlying modes of thinking and by gradations of conventionalization and the transparency of form–meaning relations. However, it is not always possible or useful to draw a sharp line between gesture and language. In this Review, we first describe how speakers and signers produce facial, manual, and body gestures. Then, we describe how representational gesture encodes information, considering the constraints from properties of languages, and how speakers and signers orchestrate language and gesture. Next, we review how gesture production shapes thinking for both signers and speakers, and consider gesture comprehension and how the meaning of gestures is integrated with language. We conclude with suggestions for further exploration of gesture as a critical expression of the human mind.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 digital issues and online access to articles
$59.00 per year
only $4.92 per issue
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Prices may be subject to local taxes which are calculated during checkout


References
Chomsky, N. Three factors in language design. Linguist. Inq. 36, 1–22 (2005).
Everaert, M. B. H. et al. What is language and how could it have evolved? Trends Cogn. Sci. 21, 569–571 (2017).
Levinson, S. C. Space in Language and Cognition: Exploration in Cognitive Diversity (Cambridge Univ. Press, 2003).
Whorf, B. L. Language, Thought, and Reality (MIT Press, 1956).
Hockett, C. The origin of speech. Sci. Am. 203, 89–97 (1960).
Tattersall, I. The Origin of The Human Capacity (American Museum of Natural History, 1998).
Kendon, A. in The Relation Between Verbal and Nonverbal Communication (ed. Key, M. R.) 207–227 (Mouton, 1980).
Kendon, A. Gesture: Visible Action as Utterance (Cambridge Univ. Press, 2004). This book discusses the range of ways in which language and gesture work as an integrated multimodal utterance in communication and reviews the history of gesture research.
McNeill, D. So you think gestures are nonverbal. Psychol. Rev. 92, 350–371 (1985).
McNeill, D. Hand and Mind (Univ. Chicago Press, 1992). This book covers a broad range of issues concerning the relationship between gesture and speech, and proposes that gesture and language form a processing unit for thinking and communicating.
McNeill, D. Gesture and Thought (Univ. Chicago Press, 2005).
Perniss, P. Why we should study multimodal language. Front. Psychol. 9, 01109 (2018).
Mondada, L. Challenges of multimodality: language and the body in social interaction. J. Socioling. 20, 336–366 (2016).
Streeck, J. Gesture as communication. I: its coordination with gaze and speech. Commun. Monogr. 60, 275–299 (1993).
Streeck, J. Gesturecraft: The Manufacture of Meaning (John Benjamins, 2009). This anthropological work reveals how co-speech gestures and other bodily actions create meaning in interaction.
Vigliocco, G., Perniss, P., Thompson, R. L. & Vinson, D. (eds) Language as a Multimodal Phenomenon: Implications for Language Learning, Processing and Evolution (Royal Society, 2014).
Stivers, T. & Sidnell, J. Multimodal communication. Semiotica 156, 1–20 (2005).
Liszkowski, U., Schäfer, M., Carpenter, M. & Tomasello, M. Prelinguistic infants, but not chimpanzees, communicate about absent entities. Psychol. Sci. 20, 654–660 (2009).
Kita, S., Alibali, M. W. & Chu, M. How do gestures influence thinking and speaking? The gesture-for-conceptualization hypothesis. Psychol. Rev. 124, 245–266 (2017). This paper proposes that representational gestures influence thinking by activating, packaging, manipulating and exploring spatio-motoric information.
Caselli, N. K., Sehyr, Z. S., Cohen-Goldberg, A. M. & Emmorey, K. ASL-LEX: a lexical database of American Sign Language. Behav. Res. Methods 49, 784–801 (2017).
Sehyr, Z. S., Caselli, N., Cohen-Goldberg, A. M. & Emmorey, K. The ASL-LEX 2.0 Project: a database of lexical and phonological properties for 2,723 signs in American Sign Language. J. Deaf Stud. Deaf Educ. 26, 263–277 (2021).
Bavelas, J. B., Chovil, N., Lawrie, D. A. & Wade, A. Interactive gestures. Discourse Process. 15, 269–189 (1992). This study experimentally shows that there are gestures whose main function is to coordinate interaction.
Efron, D. Gesture, Race, and Culture (Mouton, 1972).
Ekman, P. & Friesen, W. V. The repertoire of nonverbal behavioral categories: origins, usage, and coding. Semiotica 1, 49–98 (1969).
Cienki, A. & Müller, C. Metaphor and Gesture Vol. 3 (John Benjamins, 2008).
Kita, S. Pointing: Where Language, Culture, and Cognition Meet (Lawrence Erlbaum, 2003).
Kita, S. in Language and Gesture (ed. David, M. N.) 162–185 (Cambridge Univ. Press, 2000).
Morris, D., Collett, P., Marsh, P. & O’Shaughnessy, M. Gestures, Their Origins and Distribution (Stein and Day, 1979).
Kendon, A. Gestures as illocutionary and discourse structure markers in Southern Italian conversation. J. Prag. 23, 247–279 (1995).
Müller, C. How recurrent gestures mean: conventionalized contexts-of-use and embodied motivation. Gesture 16, 277–304 (2017).
Emmorey, K. in Gesture, Speech, and Sign (eds Messing, L. & Campbell, R.) 133–158 (Oxford Univ. Press, 1999). The author argues that signers do indeed gesture, but points out ways in which speakers and signers differ in their gesture production.
Goldin-Meadow, S. & Brentari, D. Gesture, sign, and language: the coming of age of sign language and gesture studies. Behav. Brain Sci. 40, e46 (2017). This article provides a review of sign language and gesture studies and proposes a hard distinction between sign and gesture.
Leonard, T. & Cummins, F. The temporal relation between beat gestures and speech. Lang. Cogn. Process. 26, 1457–1471 (2011).
Emmorey, K. How to distinguish gesture from sign: new technology is not the answer. Behav. Brain Sci. 40, e54 (2017).
Sandler, W. Symbiotic symbolization by hand and mouth in sign language. Semiotica 2009, 241–275 (2009).
Emmorey, K. & Herzig, M. in Perspectives on Classifier Constructions in Sign Languages (ed. Emmorey, K.) 231–256 (Psychology Press, 2003).
Duncan, S. Gesture in signing: a case study from Taiwan Sign Language. Lang. Linguist. 6, 279–318 (2005).
Lu, J., Emmorey, K. & Goldin-Meadow, S. in The 12th International Symposium on Iconicity in Language and Literature (Lund Univ., 2019).
Okrent, A. in Modality and Structure in Signed and Spoken Languages (eds Meier, R., Cormier, K. & Quinto-Pozos, D.) 175–198 (Cambridge Univ. Press, 2002).
Lillo-Martin, D. & Meier, R. P. On the linguistic status of ‘agreement’ in sign languages. Theor. Linguist. 37, 95–141 (2011).
Liddell, S. K. Grammar, Gesture, and Meaning in American Sign Language (Cambridge Univ. Press, 2003). This book describes how some signs can meaningfully point towards things or locations in the environment and argues for an integration of grammar and gesture in sign languages.
Fenlon, J., Cooperrider, K., Keane, J., Brentari, D. & Goldin-Meadow, S. Comparing sign language and gesture: insights from pointing. Glossa 4, 2 (2019).
Emmorey, K. Environmentally-coupled signs and gestures. J. Cogn. 4, 39 (2021).
Goodwin, C. in Pointing: Where Language, Cognition, and Culture Meet (ed. Kita, S.) 217–241 (Lawrence Erlbaum, 2003).
Goodwin, C. in Gesture and the Dynamic Dimension of Language (eds Duncan, S., Cassell, J. & Levey, E.) 195–212 (John Benjamins, 2007).
Haviland, J. B. in Pointing: Where Language, Cognition, and Culture Meet (ed. Kita, S.) 139–169 (Lawrence Erlbaum, 2003).
Fenlon, J., Schembri, A. & Cormier, K. Modification of indicating verbs in British Sign Language: a corpus-based study. Language 94, 84–118 (2018).
Mathur, G. & Rathman, C. in Sign Language: An International Handbook (eds Pfau, R., Steinbach, M. & Woll, B.) 136–157 (De Gruyter Mouton, 2012).
Wilbur, R. B. The point of agreement: changing how we think about sign language, gesture, and agreement. Sign Lang. Linguist. 16, 221–258 (2013).
Kusters, A. & Sahasrabudhe, S. Language ideologies on the difference between gesture and sign. Lang. Commun. 60, 44–63 (2018).
Kendon, A. Some reflections on the relationship between ‘gesture’ and ‘sign’. Gesture 8, 348–366 (2008).
Paivio, A. Dual coding theory: retrospect and current status. Can. J. Psychol. 45, 255–287 (1991).
Baddeley, A. Working Memory (Oxford Univ. Press, 1986).
Kita, S. & Özyürek, A. What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: evidence for an interface representation of spatial thinking and speaking. J. Mem. Lang. 48, 16–32 (2003). This study demonstrates cross-linguistic differences in the way co-speech representational gestures depict events and proposes a theory of how the contents of speech and gesture are coordinated.
de Ruiter, J. P. in Language and Gesture (ed. McNeill, D.) 284–311 (Univ. Chicago Press, 2000).
Krauss, R. M., Chen, Y. & Gottesman, R. F. in Language and Gesture (ed. David, M. N.) 261–283 (Cambridge Univ. Press, 2000).
Hostetter, A. B. & Alibali, M. W. Visible embodiment: gesture as simulated action. Psychon. Bull. Rev. 15, 495–514 (2008). This paper proposes a theory that co-speech gesture and language are both products of simulation of perceptual motor processes.
Hostetter, A. B. & Alibali, M. W. Gesture as simulated action: revisiting the framework. Psychonom. Bull. Rev. 26, 721–752 (2019).
Kita, S. Two-dimensional semantic analysis of Japanese mimetics. Linguistics 35, 379–415 (1997).
Kendon, A. in Cross-cultural Perspectives in Nonverbal Communication (ed. Fernando, P.) 131–141 (C. J. Hogrefe, 1988).
de Saussure, F. Course In General Linguistics [trans. Harris, R.] (Open Court, 1983).
Enfield, N. J. ‘Lip pointing’: a discussion of form and function with reference to data from Laos. Gesture 1, 185–212 (2001).
Sherzer, J. in Proceedings 1st Annual Symposium about Language and Society — Austin (SALSA I) (eds Queen, R. & Barrett, R.) 196–211 (Univ. Texas, 1993).
Wilkins, D. in Pointing: Where Language, Culture, and Cognition Meet (ed. Kita, S.) 171–215 (Lawrence Erlbaum, 2003).
Cooperrider, K. & Núñez, R. Nose-pointing: notes on a facial gesture of Papua New Guinea. Gesture 12, 103–129 (2012).
McNeill, D. Recurrent gestures: how the mental reflects the social. Gesture 17, 229–244 (2018).
Ladwig, S. H. in Body–Language–Communication: An International Handbook on Multimodality in Human Interaction Vol. 2 (eds Cornelia, M. et al.) 1558–1574 (De Gruyter Mouton, 2014).
Will, I. Recurrent Gestures of Hausa Speakers (Brill, 2021).
Klima, E. S. & Bellugi, U. The Signs of Language (Harvard Univ. Press, 1979). This classic book provides a discussion of (among other things) iconicity in sign languages, the distinction between mimes and signs, and the nature of poetic expression in sign languages.
Sehyr, Z. S. & Emmorey, K. The perceived mapping between form and meaning in American Sign Language depends on linguistic knowledge and task: evidence from iconicity and transparency judgments. Lang. Cogn. 11, 208–234 (2019).
Blasi, D. E., Wichmann, S., Hammarström, H., Stadler, P. F. & Christiansen, M. H. Sound–meaning association biases evidenced across thousands of languages. Proc. Natl Acad. Sci. USA 113, 10818–10823 (2016).
Dingemanse, M., Schuerman, W., Reinisch, E., Tufvesson, S. & Mitterer, H. What sound symbolism can and cannot do: testing the iconicity of ideophones from five languages. Language 92, e117–e133 (2016).
Iwasaki, N., Vinson, D. P. & Vigliocco, G. in Applying Theory and Research To Learning Japanese as a Foreign Language (ed. Masahiko, M.) 2–19 (Cambridge Scholars, 2007).
Perniss, P., Thompson, R. L. & Vigliocco, G. Iconicity as a general property of language: evidence from spoken and signed languages. Front. Psychol. 1, 227 (2010).
Capirci, O., Bonsignori, C. & Di Renzo, A. Signed languages: a triangular semiotic dimension. Front. Psychol. 12, 802911 (2022).
Voeltz, F. K. E. & Kilian-Hatz, C. Ideophones (John Benjamins, 2001).
Willems, R. M., Özyürek, A. & Hagoort, P. Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language. Neuroimage 47, 1992–2004 (2009).
Sandler, W. Redefining multimodality. Front. Commun. https://doi.org/10.3389/fcomm.2021.758993 (2022).
Müller, C. Gesture and sign: cataclysmic break or dynamic relations? Front. Psychol. 9, 1651 (2018).
Ferrara, L. & Hodges, G. Language as description, indication, and depiction. Front. Psychol. 9, 716 (2018). The authors advocate for a linguistic framework that does not make a distinction between gesture and language.
Krahmer, E. & Swerts, M. Effect of visual beats on prosodic prominence: acoustic analyses, auditory perception, and visual perception. J. Mem. Lang. 57, 396–414 (2007).
Gluhareva, D. & Prieto, P. Training with rhythmic beat gestures benefits L2 pronunciation in discourse-demanding situations. Lang. Teach. Res. 21, 609–631 (2016).
Vilà i Giménez, I. Non-referential Beat Gestures as a Window onto the Development of Children’s Narrative Abilities. Thesis, Universitat Pompeu Fabra (2020).
Gunter, T. C. & Bach, P. Communicating hands: ERPs elicited by meaningful symbolic hand postures. Neurosci. Lett. 372, 52–56 (2004).
Levelt, W. J. M., Richardson, G. & La Heij, W. Pointing and voicing in deictic expressions. J. Mem. Lang. 24, 133–164 (1985). This study is a foundational work on cognitive processes underlying speech–gesture synchronization.
de Ruiter, J. P. in Why Gesture?: How the Hands Function in Speaking, Thinking and Communicating (eds Church, R. B., Alibali, M. W. & Kelly, S. D.) 59–75 (John Benjamins, 2017).
Kita, S. Language and Thought Interface: A Study of Spontaneous Gestures and Japanese Mimetics (Univ. Chicago, 1993).
Chieffi, S., Secchi, C. & Gentilucci, M. Deictic word and gesture production: their interaction. Behav. Brain Res. 203, 200–206 (2009).
Kopp, S. & Bergmann, K. in The Handbook of Multimodal-Multisensor Interfaces: Foundations, User Modeling, and Common Modality Combinations Vol. 1 (eds Oviatt, S. et al.) 239–276 (ACM Books/Morgan & Claypool, 2017).
Gullberg, M. & Narasimhan, B. What gestures reveal about how semantic distinctions develop in Dutch children’s placement verbs. Cogn. Linguist. 21, 239–262 (2010).
Majid, A., Bowerman, M., Kita, S., Haun, D. B. M. & Levinson, S. C. L. Can language restructure cognition? The case of space. Trends Cogn. Sci. 8, 108–114 (2004).
Brown, A. & Gullberg, M. Bidirectional crosslinguistic influence in L1–L2 encoding of manner in speech and gesture: a study of Japanese speakers of English. Student Soc. Landsc. A 30, 225–251 (2008).
Gullberg, M. Reconstructing verb meaning in a second language: how English speakers of L2 Dutch talk and gesture about placement. Annu. Rev. Cogn. Linguist. 7, 221–244 (2009).
So, W. C., Kita, S. & Goldin-Meadow, S. Using the hands to keep track of who does what to whom: gesture and speech go hand-in-hand. Cogn. Sci. 33, 115–125 (2009).
Özyürek, A. et al. Development of cross-linguistic variation in speech and gesture: motion events in English and Turkish. Dev. Psychol. 44, 1040–1054 (2008).
Özyürek, A., Kita, S., Allen, S., Furman, R. & Brown, A. How does linguistic framing of events influence co-speech gestures? Insights from crosslinguistic variations and similarities. Gesture 5, 219–240 (2005).
Özçalışkan, Ş., Lucero, C. & Goldin-Meadow, S. Is seeing gesture necessary to gesture like a native speaker? Psychol. Sci. 27, 737–747 (2016).
Özçalışkan, Ş., Lucero, C. & Goldin-Meadow, S. Does language shape silent gesture? Cognition 148, 10–18 (2016).
Kashiwadate, K., Yasuda, T., Fujita, K., Kita, S. & Kobayashi, H. Syntactic structure influences speech–gesture synchronization. Lett. Evol. Behav. Sci. 11, 10–14 (2020).
Fritz, I., Kita, S., Littlemore, J. & Krott, A. Information packaging in speech shapes information packaging in gesture: the role of speech planning units in the coordination of speech–gesture production. J. Mem. Lang. 104, 56–69 (2019).
Parrill, F., Bergen, B. K. & Lichtenstein, P. V. Grammatical aspect, gesture, and conceptualization: using co-speech gesture to reveal event representations. Cogn. Linguist. 24, 135–158 (2013).
Furman, R., Küntay, A. C. & Özyürek, A. Early language-specificity of children’s event encoding in speech and gesture: evidence from caused motion in Turkish. Lang. Cogn. Neurosci. 29, 620–634 (2014).
Lewandowski, W. & Özçalışkan, Ş. How event perspective influences speech and co-speech gestures about motion. J. Prag. 128, 22–29 (2018).
Wessel-Tolvig, B. & Paggio, P. Revisiting the thinking-for-speaking hypothesis: speech and gesture representation of motion in Danish and Italian. J. Prag. 99, 39–61 (2016).
Bock, J. K. Toward a cognitive psychology of syntax: information processing contributions to sentence formulation. Psychol. Rev. 89, 1–47 (1982).
Talmy, L. in Language Typology and Syntactic Description. Volume III: Grammatical Categories and the Lexicon (ed. Timothy, S.) 57–149 (Cambridge Univ. Press, 1985).
Kita, S. et al. Relations between syntactic encoding and co-speech gestures: implications for a model of speech and gesture production. Lang. Cogn. Process. 22, 1212–1236 (2007).
Akhavan, N., Nozari, N. & Göksun, T. Expression of motion events in Farsi. Lang. Cogn. Neurosci. 32, 792–804 (2017).
Duncan, S. D. Gesture, verb aspect, and the nature of iconic imagery in natural discourse. Gesture 2, 183–206 (2002).
Cienki, A. & Iriskhanova, O. K. in Human Cognitive Processing (John Benjamins, 2018).
Hinnell, J. The multimodal marking of aspect: the case of five periphrastic auxiliary constructions in North American English. Cogn. Linguist. 29, 773–806 (2018).
Trujillo, J., Özyürek, A., Holler, J. & Drijvers, L. Speakers exhibit a multimodal Lombard effect in noise. Sci. Rep. 11, 16721 (2021).
Cohen, A. A. The communicative functions of hand illustrators. J. Commun. 27, 54–63 (1977).
Alibali, M. W., Heath, D. C. & Myers, H. J. Effects of visibility between speaker and listener on gesture production: some gestures are meant to be seen. J. Mem. Lang. 44, 169–188 (2001).
Bavelas, J., Gerwing, J., Sutton, C. & Prevost, D. Gesturing on the telephone: independent effects of dialogue and visibility. J. Mem. Lang. 58, 495–520 (2008).
Cohen, A. A. & Harrison, R. P. Intentionality in the use of hand illustrators in face-to-face communication situations. J. Pers. Soc. Psychol. 28, 276–279 (1973).
Krauss, R. M., Dushay, R. A., Chen, Y. & Rauscher, F. The communicative value of conversational hand gestures. J. Exp. Soc. Psychol. 31, 533–552 (1995).
Emmorey, K. & Casey, S. Gesture, thought and spatial language. Gesture 1, 35–50 (2001).
Barker, J. Production and comprehension of Audience Design Behaviours in Co-speech Gesture. Thesis, Univ. Warwick (2022).
Hoetjes, M., Krahmer, E. & Swerts, M. On what happens in gesture when communication is unsuccessful. Speech Commun. 72, 160–175 (2015).
Gullberg, M. Handling discourse: gestures, reference tracking, and communication strategies in early L2. Lang. Learn. 56, 155–196 (2006).
Naeve, S. L., Siegel, G. M. & Clay, J. L. Modifications in sign under conditions of impeded visibility. J. Speech. Lang. Hear. Res. 35, 1272–1280 (1992).
Emmorey, K., Gertsberg, N., Korpics, F. & Wright, C. E. The influence of visual feedback and register changes on sign language production: a kinematic study with deaf signers. Appl. Psycholinguist. 30, 187–203 (2009).
Adams, T. W. Gesture in Foreigner Talk (Univ. Pennsylvania, 1998).
Tellier, M., Stam, G. & Ghio, A. Handling language: how future language teachers adapt their gestures to their interlocutor. Gesture 20, 30–62 (2021).
Holler, J. & Beattie, G. Pragmatic aspects of representational gestures: do speakers use them to clarify verbal ambiguity with the listener? Gesture 3, 127–154 (2003).
Bangerter, A. Using pointing and describing to achieve joint focus of attention in dialogue. Psychol. Sci. 15, 415–419 (2004).
Holmström, I. Teaching a language in another modality: a case study from Swedish Sign Language L2 instruction. J. Lang. Teach. Res. 10, 659–672 (2019).
Perniss, P., Lu, J. C., Morgan, G. & Vigliocco, G. Mapping language to the world: the role of iconicity in the sign language input. Dev. Sci. 21, e12551 (2018).
Kelly, S., Byrne, K. & Holler, J. Raising the ante of communication: evidence for enhanced gesture use in high stakes situations. Information 2, 579–593 (2011).
Hilliard, C., O’Neal, E., Plumert, J. & Cook, S. W. Mothers modulate their gesture independently of their speech. Cognition 140, 89–94 (2015).
Galati, A. & Brennan, S. E. Speakers adapt gestures to addressees’ knowledge: implications for models of co-speech gesture. Lang. Cogn. Neurosci. 29, 435–451 (2013).
Gerwing, J. & Bavelas, J. Linguistic influences on gesture’s form. Gesture 4, 157–195 (2004).
Hoetjes, M., Koolen, R., Goudbeek, M., Krahmer, E. & Swerts, M. Reduction in gesture during the production of repeated references. J. Mem. Lang. 79–80, 1–17 (2015).
Jacobs, N. & Garnham, A. The role of conversational hand gestures in a narrative task. J. Mem. Lang. 56, 291–303 (2007).
Schubotz, L., Özyürek, A. & Holler, J. Age-related differences in multimodal recipient design: younger, but not older adults, adapt speech and co-speech gestures to common ground. Lang. Cogn. Neurosci. 34, 254–271 (2019).
Hilliard, C. & Cook, S. W. Bridging gaps in common ground: speakers design their gestures for their listeners. J. Exp. Psychol. Learn. Mem. Cogn. 42, 91–103 (2016).
Peeters, D., Chu, M., Holler, J., Hagoort, P. & Özyürek, A. Electrophysiological and kinematic correlates of communicative intent in the planning and production of pointing gestures and speech. J. Cogn. Neurosci. 27, 2352–2368 (2015).
Holler, J. & Wilkin, K. An experimental investigation of how addressee feedback affects co-speech gestures accompanying speakers’ responses. J. Prag. 43, 3522–3536 (2011).
Enfield, N. J., Kita, S. & de Ruiter, J. P. Primary and secondary pragmatic functions of pointing gestures. J. Prag. 39, 1722–1741 (2007).
Slonimska, A., Özyürek, A. & Capirci, O. The role of iconicity and simultaneity for efficient communication: the case of Italian Sign Language (LIS). Cognition 200, 104246 (2020).
Slonimska, A., ÖZyÜRek, A. & Capirci, O. Using depiction for efficient communication in LIS (Italian Sign Language). Lang. Cogn. 13, 367–396 (2021).
Dudis, P. G. Body partitioning and real-space blends. Cogn. Linguist. 15, 223–238 (2004).
Hostetter, A. B. When do gestures communicate? A meta-analysis. Psychol. Bull. 137, 297–315 (2011). This paper reviews factors that facilitate information uptake from co-speech gesture by the recipient.
Feyereisen, P. & Havard, I. Mental imagery and production of hand gestures while speaking in younger and older adults. J. Nonverb. Behav. 23, 153–171 (1999).
Rauscher, F. H., Krauss, R. M. & Chen, Y. Gesture, speech, and lexical access: the role of lexical movements in speech production. Psychol. Sci. 7, 226–230 (1996).
Hostetter, A. B. & Alibali, M. W. Language, gesture, action! A test of the Gesture as Simulated Action framework. J. Mem. Lang. 63, 245–257 (2010).
Pine, K. J., Gurney, D. J. & Fletcher, B. The semantic specificity hypothesis: when gestures do not depend upon the presence of a listener. J. Nonverb. Behav. 34, 169–178 (2010).
Masson-Carro, I., Goudbeek, M. & Krahmer, E. How what we see and what we know influence iconic gesture production. J. Nonverb. Behav. 41, 367–394 (2017).
Chu, M. & Kita, S. Co-thought and co-speech gestures are generated by the same action generation process. J. Exp. Psychol. Learn. Mem. Cogn. 42, 257–270 (2016).
Rimé, B., Schiaratura, L., Hupet, M. & Ghysselinckx, A. Effects of relative immobilization on the speaker’s nonverbal behavior and on the dialogue imagery level. Motiv. Emot. 8, 311–325 (1984).
Alibali, M. W., Spencer, R. C., Knox, L. & Kita, S. Spontaneous gestures influence strategy choices in problem solving. Psychol. Sci. 22, 1138–1144 (2011).
Wilson, M. & Emmorey, K. in Signed Languages: Discoveries from International Research (eds Dively, V., Metzger, M., Taub, S. & Baer, A. M.) 91–99 (Gallaudet Univ. Press, 2001).
Mol, L. & Kita, S. in Proceedings 34th Annual Conference of The Cognitive Science Society (eds Miyake, N., Peebles, D. & Cooper, R. P.) 761–766 (Cognitive Science Society, 2012).
Kirk, E. & Lewis, C. Gesture facilitates children’s creative thinking. Psychol. Sci. 28, 225–232 (2017).
Church, R. B. & Goldin-Meadow, S. The mismatch between gesture and speech as an index of transitional knowledge. Cognition 23, 43–71 (1986). In this study on gesture’s role in cognitive development, the semantic discrepancy between speech and co-speech gesture indicates that the underlying knowledge is unstable and malleable.
Perry, M., Church, R. B. & Goldinmeadow, S. Transitional knowledge in the acquisition of concepts. Cogn. Dev. 3, 359–400 (1988).
Goldin-Meadow, S., Shield, A., Lenzen, D., Herzig, M. & Padden, C. The gestures ASL signers use tell us when they are ready to learn math. Cognition 123, 448–453 (2012). This study with deaf signing children provides key evidence that gestures are used to explore ideas that are different from linguistically expressed ideas.
Broaders, S. C., Cook, S. W., Mitchell, Z. & Goldin-Meadow, S. Making children gesture brings out implicit knowledge and leads to learning. J. Exp. Psychol. Gen. 136, 539–550 (2007).
Vilà-Giménez, I., Igualada, A. & Prieto, P. Observing storytellers who use rhythmic beat gestures improves children’s narrative discourse performance. Dev. Psychol. 55, 250–262 (2019).
Bosker, H. R. & Peeters, D. Beat gestures influence which speech sounds you hear. Proc. R. Soc. B 288, 20202419 (2021).
Llanes-Coromina, J., Vilà-Giménez, I., Kushch, O., Borràs-Comes, J. & Prieto, P. Beat gestures help preschoolers recall and comprehend discourse information. J. Exp. Child Psychol. 172, 168–188 (2018).
Cocks, N., Sautin, L., Kita, S., Morgan, G. & Zlotowitz, S. Gesture and speech integration: an exploratory study of a case of a man with aphasia. Int. J. Lang. Commun. Disord. 44, 795–804 (2009).
Sekine, K., Sowden, H. & Kita, S. The development of the ability to semantically integrate information in speech and iconic gesture in comprehension. Cogn. Sci. 39, 1855–1880 (2015).
Kelly, S. D., Barr, D. J., Church, R. B. & Lynch, K. Offering a hand to pragmatic understanding: the role of speech and gesture in comprehension and memory. J. Mem. Lang. 40, 577–592 (1999).
Broaders, S. C. & Goldin-Meadow, S. Truth is at hand: how gesture adds information during investigative interviews. Psychol. Sci. 21, 623–628 (2010).
Kirk, E., Gurney, D., Edwards, R. & Dodimead, C. Handmade memories: the robustness of the gestural misinformation effect in children’s eyewitness interviews. J. Nonverb. Behav. 39, 259–273 (2015).
Gurney, D. J., Pine, K. J. & Wiseman, R. The gestural misinformation effect: skewing eyewitness testimony through gesture. Am. J. Psychol. 126, 301–314 (2013).
Gurney, D. J., Ellis, L. R. & Vardon-Hynard, E. The saliency of gestural misinformation in the perception of a violent crime. Psychol. Crime. Law 22, 651–665 (2016).
Aussems, S. & Kita, S. Seeing iconic gesture promotes first- and second-order verb generalization in preschoolers. Child Dev. 92, 124–141 (2021).
Aussems, S., Mumford, K. H. & Kita, S. Prior experience with unlabeled actions promotes 3-year-old children’s verb learning. J. Exp. Psychol. Gen. 151, 246–262 (2022).
Mumford, K. H. & Kita, S. Children use gesture to interpret novel verb meanings. Child Dev. 85, 1181–1189 (2014).
Vogt, S. & Kauschke, C. Observing iconic gestures enhances word learning in typically developing children and children with specific language impairment. J. Child Lang. 44, 1458–1484 (2017).
Vogt, S. S. & Kauschke, C. With some help from others’ hands: iconic gesture helps semantic learning in children with specific language impairment. J. Speech Lang. Hear. Res. 60, 3213–3225 (2017).
Wakefield, E. M., Hall, C., James, K. H. & Goldin-Meadow, S. Gesture for generalization: gesture facilitates flexible learning of words for actions on objects. Dev. Sci. 21, e12656 (2018).
Goodrich, W. & Hudson Kam, C. L. Co-speech gesture as input in verb learning. Dev. Sci. 12, 81–87 (2009).
McGregor, K. K., Rohlfing, K. J., Bean, A. & Marschner, E. Gesture as a support for word learning: the case of under. J. Child Lang. 36, 807–828 (2009).
Tellier, M. The effect of gestures on second language memorisation by young children. Gesture 8, 219–235 (2008).
Gunter, T. C., Weinbrenner, J. E. D. & Holle, H. Inconsistent use of gesture space during abstract pointing impairs language comprehension. Front. Psychol. 6, 80 (2015).
Sekine, K. & Kita, S. The listener automatically uses spatial story representations from the speaker’s cohesive gestures when processing subsequent sentences without gestures. Acta Psychol. 179, 89–95 (2017).
Hinnell, J. & Parrill, F. Gesture influences resolution of ambiguous statements of neutral and moral preferences. Front. Psychol. 11, 587129 (2020).
Meier, R. P. & Lilo-Martin, D. The points of language. Humanamente 24, 151–176 (2013).
Engberg-Pedersen, E. Space in Danish Sign Language: The Semantics and Morphosyntax of The Use of Space in a Visual Language (Signum, 1993).
Lakoff, G. & Johnson, M. Metaphor as We Live By (Univ. Chicago Press, 1980).
Sekine, K. & Kita, S. Development of multimodal discourse comprehension: cohesive use of space by gestures. Lang. Cogn. Neurosci. 30, 1245–1258 (2015).
Maricchiolo, F., Gnisci, A., Bonaiuto, M. & Ficca, G. Effects of different types of hand gestures in persuasive speech on receivers’ evaluations. Lang. Cogn. Process. 24, 239–266 (2009).
Peters, J. & Hoetjes, M. The effect of gesture on persuasive speech. Proc. Interspeech https://doi.org/10.21437/Interspeech.2017-194 (2017).
Wakefield, E. M., Novack, M. A., Congdon, E. L. & Howard, L. H. Individual differences in gesture interpretation predict children’s propensity to pick a gesturer as a good informant. J. Exp. Child Psychol. 205, 105069 (2021).
Goldin-Meadow, S. & Singer, M. A. From children’s hands to adults’ ears: gesture’s role in the learning process. Dev. Psychol. 39, 509–520 (2003).
Bauman, D. H. L., Nelson, J. L. & Rose, H. Signing the Body Poetic: Essay on American Sign Language Literature (Univ. California Press, 2006).
Sutton-Spence, R. & Kaneko, M. Introducing Sign Language Literature: Folklore and Creativity (Red Globe Press, 2017).
Holler, J., Kendrick, K. H. & Levinson, S. C. Processing language in face-to-face conversation: questions with gestures get faster responses. Psychon. Bull. Rev. 25, 1900–1908 (2018).
Trujillo, J. P., Levinson, S. C. & Holler, J. Human–Computer Interaction: Design and User Experience Case Studies (ed. Kurosu, M.) 643–657 (Springer, 2021).
Duncan, S. Some signals and rules for taking speaking turns in conversations. J. Pers. Soc. Psychol. 23, 283–292 (1972).
Schegloff, E. A. in Structure of Social Action: Studies in Conversation Analysis (eds Atkinson, J. M. & Heritage, J.) 266–296 (Cambridge Univ. Press, 1984).
Morrel-Samuels, P. & Krauss, R. M. Word familiarity predicts temporal asynchrony of hand gestures and speech. J. Exp. Psychol. Learn. Mem. Cogn. 18, 615–622 (1992).
de Vos, C., Casillas, M., Uittenbogert, T., Crasborn, O. & Levinson, S. C. Predicting conversational turns: signers’ and nonsigners’ sensitivity to language-specific and globally accessible cues. Language 98, 35–62 (2022).
Baker, C. in On the Other Hand: New Perspectives on American Sign Language (ed. Friedmann, L.) 218–236 (Academic, 1977).
de Vos, C., Torreira, F. & Levinson, S. C. Turn-timing in signed conversations: coordinating stroke-to-stroke turn boundaries. Front. Psychol. 6, 268 (2015).
Girard-Groeber, S. The management of turn transition in signed interaction through the lens of overlaps. Front. Psychol. 6, 741 (2015).
Obermeier, C., Kelly, S. D. & Gunter, T. C. A speaker’s gesture style can affect language comprehension: ERP evidence from gesture–speech integration. Soc. Cogn. Affect. Neurosci. 10, 1236–1243 (2015).
Gunter, T. C. & Weinbrenner, J. E. D. When to take a gesture seriously: on how we use and prioritize communicative cues. J. Cogn. Neurosci. 29, 1355–1367 (2017).
Kelly, S. D., Özyürek, A. & Maris, E. Two sides of the same coin: speech and gesture mutually interact to enhance comprehension. Psychol. Sci. 21, 260–267 (2010). This study shows that recipients automatically integrate speech and co-speech gesture.
Kelly, S. D., Creigh, P. & Bartolotti, J. Integrating speech and iconic gestures in a Stroop-like task: evidence for automatic processing. J. Cogn. Neurosci. 22, 683–694 (2010).
Kelly, S. D., Ward, S., Creigh, P. & Bartolotti, J. An intentional stance modulates the integration of gesture and speech during comprehension. Brain Lang. 101, 222 (2007).
Özer, D. & Göksun, T. Visual-spatial and verbal abilities differentially affect processing of gestural vs. spoken expressions. Lang. Cogn. Neurosci. 35, 896–914 (2020).
Wu, Y. C. & Coulson, S. Iconic gestures facilitate discourse comprehension in individuals with superior immediate memory for body configurations. Psychol. Sci. 26, 1717–1727 (2015).
McNeill, D. in Pointing: Where Language, Cognition, and Culture Meet (ed. Kita, S.) 293–306 (Lawrence Erlbaum, 2003).
Liddell, S. K. Spatial representations in discourse: comparing spoken and signed language. Lingua 98, 145–167 (1996).
Emmorey, K., Norman, F. & O’Grady, L. The activation of spatial antecedents from overt pronouns in american sign language. Lang. Cogn. Process. 6, 207–228 (1991).
Emmorey, K. & Lillo-martin, D. Processing spatial anaphora: referent reactivation with overt and null pronouns in American Sign Language. Lang. Cogn. Process. 10, 631–653 (1995).
Wienholz, A. et al. Pointing to the right side? An ERP study on anaphora resolution in German Sign Language. PLoS ONE 13, e0204223 (2018).
Fritz, I. How Gesture and Speech Interact During Production and Comprehension. Thesis, Univ. Birmingham (2017).
Habets, B., Kita, S., Shao, Z., Özyürek, A. & Hagoort, P. The role of synchrony and ambiguity in speech–gesture integration during comprehension. J. Cogn. Neurosci. 23, 1845–1854 (2011).
Obermeier, C., Holle, H. & Gunter, T. C. What iconic gesture fragments reveal about Gesture–Speech integration: when synchrony is lost, memory can help. J. Cogn. Neurosci. 23, 1648–1663 (2011).
Krauss, R. M., Morrel-Samuels, P. & Colasante, C. Do conversational hand gestures communicate? J. Pers. Soc. Psychol. 61, 743–754 (1991).
Fritz, I., Kita, S., Littlemore, J. & Krott, A. Multimodal language processing: how preceding discourse constrains gesture interpretation and affects gesture integration when gestures do not synchronise with semantic affiliates. J. Mem. Lang. 117, 104191 (2021).
Drijvers, L., Vaitonytė, J. & Özyürek, A. Degree of language experience modulates visual attention to visible speech and iconic gestures during clear and degraded speech comprehension. Cogn. Sci. 43, e12789 (2019).
Drijvers, L. & Özyürek, A. Non-native listeners benefit less from gestures and visible speech than native listeners during degraded speech comprehension. Lang. Speech 63, 209–220 (2020).
Holle, H., Obleser, J., Rueschemeyer, S.-A. & Gunter, T. C. Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions. Neuroimage 49, 875–884 (2010).
Slobin, D. I. in Proceedings 13th Annual Meeting of Berkeley Linguistics Society (Berkeley Linguistics Society, 1987).
Sherzer, J. Verbal and nonverbal deixis: the pointed lip gesture among the San Blas Cuna. Lang. Soc. 2, 117–131 (1973).
Ergin, R., Senghas, A., Jackendoff, R. & Gleitman, L. Structural cues for symmetry, asymmetry, and non-symmetry in Central Taurus Sign Language. Sign Lang. Linguist. 23, 171–207 (2020).
Sandler, W. & Lillo-Martin, D. Sign Language And Linguistic Universals (Cambridge Univ. Press, 2006).
Corina, D. P. et al. Dissociation between linguistic and nonlinguistic gestural systems: a case for compositionality. Brain Lang. 43, 414–447 (1992).
Marshall, J., Atkinson, J., Smulovitch, E., Thacker, A. & Woll, B. Aphasia in a user of British Sign Language: dissociation between sign and gesture. Cogn. Neuropsychol. 21, 537–554 (2004).
Baynton, D. C. Forbidden Signs (Univ. Chicago Press, 1996).
Bauman, H., Rose, H. & Nelson, J. Signing the body poetic: Essays on American Sign Language Literature (Cambridge Univ. Press, 2006).
Özer, D. & Göksun, T. Gesture use and processing: a review on individual differences in cognitive resources. Front. Psychol. 11, 573555 (2020).
Smithson, L. & Nicoladis, E. Verbal memory resources predict iconic gesture use among monolinguals and bilinguals. Bilingualism 16, 934–944 (2013).
Chu, M., Meyer, A., Foulkes, L. & Kita, S. Individual differences in frequency and saliency of speech-accompanying gestures: the role of cognitive abilities and empathy. J. Exp. Psychol. Gen. 143, 694–709 (2014).
Gillespie, M., James, A. N., Federmeier, K. D. & Watson, D. G. Verbal working memory predicts co-speech gesture: evidence from individual differences. Cognition 132, 174–180 (2014).
Hostetter, A. B. & Alibali, M. W. Cognitive skills and gesture–speech redundancy formulation difficulty or communicative strategy? Gesture 11, 40–60 (2011).
Hostetter, A. B. & Alibali, M. W. Raise your hand if you’re spatial: relations between verbal and spatial skills and gesture production. Gesture 7, 73–95 (2007).
Pyers, J. E., Magid, R., Gollan, T. H. & Emmorey, K. Gesture helps, only if you need it: inhibiting gesture reduces tip-of-the-tongue resolution for those with weak short-term memory. Cogn. Sci. 45, e12914 (2021).
Hostetter, A. B. & Potthoff, A. L. Effects of personality and social situation on representational gesture production. Gesture 12, 62–83 (2012).
Kimura, D. Manual activity during speaking — II. Left-handers. Neuropsychologia 11, 51–55 (1973).
Argyriou, P., Mohr, C. & Kita, S. Hand matters: left-hand gestures enhance metaphor explanation. J. Exp. Psychol. Learn. Mem. Cogn. 43, 874–886 (2017).
Aldugom, M., Fenn, K. M. & Cook, S. W. Gesture during math instruction specifically benefits learners with high visuospatial working memory capacity. Cogn. Res. Princ. Implic. 5, 27 (2020).
Momsen, J., Gordon, J., Wu, Y. C. & Coulson, S. Event related spectral perturbations of gesture congruity: visuospatial resources are recruited for multimodal discourse comprehension. Brain Lang. 216, 104916 (2021).
Author information
Authors and Affiliations
Contributions
Both authors contributed substantially to discussion of the content. Both authors wrote the article. S.K. led the writing on co-speech gesturing and K.E. led the writing on co-sign gesturing. Both authors reviewed and/or edited the manuscript before submission.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Reviews Psychology thanks Tilbe Goksun, Jennifer Hinnell, Wendy Sandler and Adam Schembri for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Kita, S., Emmorey, K. Gesture links language and cognition for spoken and signed languages. Nat Rev Psychol 2, 407–420 (2023). https://doi.org/10.1038/s44159-023-00186-9
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s44159-023-00186-9