Metaphor and gesture Some implications for psychology Raymond W. Gibbs, Jr. 1. Metaphor and gesture: Some implications for psychology The new work on metaphor and gesture, as seen in the chapters in this volume, represents a major advance in the study of metaphor, and the psychology of human thought and communication. It comes as no surprise that people produce metaphoric gestures given their frequent use of metaphor in speech and writing. Nevertheless, the emerging literature on metaphor and gesture adds a significant source of evidence in support of the idea that abstract concepts are understood and communicated metaphorically. As several authors note in this volume, some cognitive scientists have raised skeptical questions about the possibility of metaphoric concepts because of the heavy reliance on linguistic evidence in making arguments for the existence of conceptual metaphor (e.g., Glucksberg, 2001; Murphy, 1996). These scholars argue that speaking about something metaphorically should not be seen as reliable data as to whether people actually think metaphorically, and urge that nonlinguistic evidence be gathered to support any claims about the metaphoric nature of ordinary thought. Indeed, the various kinds of research showing the prominence of metaphor in gesture in this volume not only provides nonlinguistic data skeptical scientists demand to support the possibility of metaphoric thought, but also shows how fundamental aspects of abstract thought are rooted in, and expressed by, recurring patterns of bodily experience and action. Most generally, the various explorations of metaphor in gesture are consistent with the “embodiment premise” as a research strategy in cognitive science (Gibbs, 2006a, p. 9): People’s subjective, felt experiences of their bodies in action provide part of the fundamental grounding for language and thought. Cognition is what occurs when the body engages the physical, cultural world and must be studied in terms of the dynamical interactions between people and the environment. Human language and thought emerge from recurring patterns of embodied activity that constrain ongoing intelligent behavior. We must not assume cognition to be purely internal,

3rd proofs

 Raymond W. Gibbs, Jr



Metaphor and gesture 

frequently studied gesture in experimental situations, yet claims about the relationship between metaphoric gesture and thought demand more explicit criteria for defining, counting, and categorizing metaphoric gestures and the circumstances under which they are enacted, as well as how often they are produced. Although linguistic research offers some tentative answers to some of the above questions, studies in which a scholar selectively identifies certain metaphoric gestures for analysis and discussion must be complemented by larger empirical studies that provide comparative, quantitative data on metaphoric gesture production.

symbolic, computational, and disembodied, but seek out the gross and detailed ways that language and thought are inextricably shaped by embodied action.

Thus, the empirical study of metaphoric gesture, across languages and cultural contexts, aims to explore the “gross and detailed ways that language and thought are inextricably shaped by embodied action” (ibid.). Moreover, the empirical demonstrations of metaphoric gesture certainly reflect the embodied quality of abstract thought, but also illustrate how “cognition is what occurs when the body engages the physical, cultural world and must be studied in terms of the dynamical interaction between people and the environment” (ibid.). Under this view, metaphoric gestures are not simply the outward manifestation of an inner, symbolic, disembodied idea, but exhibit the dynamic creation, and recreation, of metaphoric thought in the bodily act of online communication. The following suggests a few directions for future research on metaphoric gesture in the broad discipline of Psychology that takes off from these themes.

3. Relationship between speech and gesture The growing interest in gesture over the past twenty year has provided the basis for serious debates among psychologists on the relationship between speech and gesture. At present, there are several ways of characterizing the speech-gesture relationship (Gibbs, 2006a; Iverson & Thelen, 1999). First, speech and gesture are conceived as separate communicative systems, which occasionally become linked due to the cognitive demands associated with speech production (Butterworth & Hadar, 1989). For example, gesture serves to support speech production activities by compensating when speech is temporarily disrupted (e.g., by coughing) or when speakers are unable to put their thoughts into words. But gestures do not influence underlying speech production processes. A second view holds that there are deep cognitive linkages between speech and gesture, presumably located at the physiological encoding stage (i.e., the stage at which words’ forms must be accessed from lexical memory) (Krauss, 1998). Thus, gestures are especially useful when speakers experience difficulty retrieving words, because the production of gesture activates relevant spatio-dynamic features of the concept (i.e., thought) that a speaker had in mind. The link between speech and gesture is limited, then, to a particular stage of speech production. Third, speech and gesture are sometimes assumed to be communicative activities that are grounded in common thought processes (Iverson & Thelen, 1999; McNeill, 2005). Under this view, speech and gesture have a strong reciprocal relationship through the entire process of speech production, ranging from phonological encoding up through producing syntax, semantics, and discourse. Even though speech and gesture may communicate different aspects of people’s thoughts, the tight coupling of these activities suggests that any disruption in one (e.g., gesture) will have negative effects on the other (e.g., speech). Gesture appears to play a critical role in the conceptualizing and planning of messages. Research on metaphoric gestures may provide important evidence relevant to these theoretical possibilities. Of course, metaphoric gesture may be helpful to

2. Some needed empirical facts Psychologists like to think in quantitative ways and often, for better or worse, like to see numbers of some sort before they evaluate any behavioral phenomenon or theory. The growing literature on metaphor and gesture is primarily centered on linguistic research, much of which is concerned with the analysis of different case studies. But to more firmly establish the psychological reality of metaphoric gesture, there is a need for some basic information. What are the different kinds of metaphoric gestures? What are the specific, reliable criteria that distinguish a metaphoric gesture from one that is metonymic, or more generally, iconic? How often do people use metaphoric gestures compared to various other gestures? How often do metaphoric gestures accompany metaphoric and non-metaphoric speech? In what contexts are metaphoric gestures found most frequently? Do people gesture metaphorically when speaking with no observers, such as when talking on the telephone? Do children produce metaphoric gestures and what is the developmental progression in the use of these enactments? What are the cross-linguistic and cross-cultural similarities in the kind and amount of metaphoric gestures enacted? How often do people create novel metaphoric gestures? Are there regularities in the source domains recruited in conventional and novel metaphoric gestures? Do people mix their metaphoric gestures like they sometimes do with their verbal metaphors? This list of questions reflects some of the likely concerns that psychologists would have for empirical research on metaphoric gestures. Psychologists have

3rd proofs

 Raymond W. Gibbs, Jr



Metaphor and gesture 

Does observing a metaphoric gesture make a speaker’s accompanying metaphoric utterance easier to understand and remember? There is good reason to suspect that this may be the case. First, psychological studies indicate that the accuracy of hearing spoken sentences is twice as high when the sentences are presented combined with gestures than when they are not (cf. Gibbs, 1999). Second, research in neuropsychology demonstrates that observing another person’s simple actions activates relevant motor areas of the brain (i.e., “mirror neurons”), as if the observer were doing the very action being seen (Decety & Grezes, 1999). In an extended way, observing a metaphoric gesture may activate appropriate motor regions of the brain that could be linked to the embodied source domains of many metaphoric concepts (Gallese & Lakoff, 2005), which may also facilitate the embodied simulation launched to comprehend a speaker’s communicative message when saying something like “I just couldn’t grasp that concept.” Empirical research is needed to study this possibility in more detail. Of course, the temporal coordination between the production of a gesture and the production of a specific lexical item or phrase will also need to be systematically explored to see under what conditions observing metaphoric gestures facilitates speech understanding, particularly compared to cases where literal gestures accompanying literal speech (e.g., gesture before words, gesture simultaneous to words, gesture immediately after words). Furthermore, the differences between metaphoric and metonymic gestures must also be studied. One possibility here is that metonymic gestures facilitate both production and understanding of individual lexical items, while metaphoric gestures best enhance people’s understanding of abstract concepts that are not as conventionally associated with individual lexical items.

speakers when they are unable to convey their thoughts in words, and therefore may assist lexical retrieval, yet also reflect common metaphoric thought processes in the same way that many aspects of metaphoric speech does. If some metaphoric gestures reflect common metaphoric mappings, similar to those motivating different words and phrases, then producing these gestures should facilitate metaphoric speech production given these gestures’ fundamental role in conceptualization. Thus, gesturing a grasping motion with one hand may both reflect some natural conceptualization of the idea of a concept, but may also help a speaker verbally articulate the idea of “grasping a concept” as in “I just couldn’t grasp that concept.” The physical gesture of grasping, or attempting to grasp, which is related to the conceptual metaphors of ideas/concepts are physical objects and understanding is grasping, sets up the easy formulation of the conventional statement “I just couldn’t grasp that concept” as metaphorically meaning, “I just couldn’t understand that concept.” In this view, gestures that may be related to parts of conventional metaphoric concepts help people understand metaphoric messages that have embodied source domains. There are no experimental studies that explicitly show whether metaphoric gestures facilitate the formulation of metaphoric utterances. But there is psycholinguistic evidence demonstrating that when people make a hand gesture or body movement, such as a hand grasping motion, appropriate to the meaning of a metaphoric phrase, such as “grasp the concept,” they are subsequently quicker to comprehend the phrase than if they had previously made some other gesture (e.g., a pushing away gesture) (Gibbs, 2006b). Furthermore, people’s speeded understanding of metaphoric phrases is facilitated when they first simply imagined making the appropriate gesture before seeing the phrase. These experimental results suggest that engaging in body movements associated certain metaphoric ideas (i.e., an actual grasping motion related to the possibility that concepts can be metaphorically conceptualized as physical entities which can be grasped, fumbled, dropped, etc.) enhances the simulations that people create to form a metaphoric understanding of abstract notions related to these gestures. This same set of studies also shows the grasping gesture, to take one of many examples in the experiments, does not simply activate a literal word “grasp” as people associated a variety of words with this action. But the gesture is metaphorically meaningful precisely because people think, once more, of concepts as things that can be grasped. People therefore appear to create embodied simulations of speakers’ messages that involve moment-by-moment “what must it be like” processes that make use of ongoing tactile-kinesthetic experiences such as grasping gestures. Seeing metaphoric gestures may enhance the creation of simulations when people encounter language that is abstract, or refer to actions that are physically impossible to perform (i.e, “grasp the concept,” “throw out an idea”).

4. Analyzability of metaphoric gestures Gesture scholars differ as to whether metaphoric gestures are compositional or not (Calbris, 1990; Kendon & Versante, 2003; McNeill, 1992; Müller, 1998). However, psycholinguistic studies of verbal metaphors suggest that even highly conventional kinds of metaphors, such as idioms motivated by metaphor, are analyzable to varying extents (Gibbs, 1994). Listeners comprehend metaphoric idioms (e.g., “John popped the question to Mary”), for example, through analysis of how each word makes independent contributions to the phrase’s overall figurative meaning, although this analysis links parts of metaphoric expressions with metaphoric, and not literal, meanings. This work implies that even the most clichéd metaphoric phrases are not understood through simple retrieval of their meanings stored in a phrasal, mental lexicon.

3rd proofs

 Raymond W. Gibbs, Jr



Metaphor and gesture 

5. Coordination between participants in talk/gesture exchanges

Are metaphoric gestures understood as wholes, through retrieval of their meanings from some gestural lexicon or might these be interpreted compositionally with parts of each gesture referring meaningfully to parts of their overall metaphoric messages? One way of answering this question is to present observers with metaphoric gestures and their typical metaphoric meanings and ask to what extent different parts of the temporal articulation of each gesture are associated with parts of its metaphoric meaning. More complex studies could present gestures with appropriate metaphoric speech and ask observers/listeners to judge, more once, the extent to which aspects of the gesture correspond with parts of the metaphoric speech (both in terms of the words spoken and the overall figurative meaning conveyed). Similar to conventional metaphoric speech, people may alter parts of their metaphoric gestures to convey metaphoric meanings in novel ways. Novel gestures need not be completely new, but may be slightly creative instantiations of conventional articulations, which possibly communicate additional cognitive effects, compared to typical metaphoric gestures. The extent to which people gesture in these slightly novel ways, and whether people actually infer different cognitive effects, or meanings, when observing these novel instantiations, are also empirical questions worthy of empirical study. This last issue raises a more general challenge for future psychological studies on metaphor and gesture. Both linguists and psychologists studying metaphoric language understanding typically describe what is understood in rather crude terms, often simply noting that the metaphoric meaning of some statement can be roughly paraphrased in a manner that is different from the meaning of a literal expression. Yet a closer look at what people really infer when hearing metaphoric language reveals that even highly conventional expressions, like metaphoric idioms, are understood as communicating complex figurative meanings that are not easily paraphrased (Gibbs, 1994). Novel metaphoric utterances often express even greater degrees of aesthetic and interpersonal meaning. One challenge for metaphor scholars of both speech and gesture is to empirically examine the kinds of meanings, or cognitive effects, people draw from seeing metaphoric gestures, both with and without accompanying speech. As mentioned earlier, metaphoric gestures with speech are likely not just communicating redundant information, but should, in many cases, express something different. Determining what those meanings are, and the conditions under which they are best inferred, should be a significant topic of study in both linguistics and psychology.

It is important to note that observing a metaphoric gesture is not necessarily just an added piece of information to ease understanding of a speaker’s linguistic, metaphoric message. Instead, a person’s metaphoric gesturing is part of the ensemble of meaning that is communicated by the speaker’s overall bodily actions, which can include voice, hands, and other parts of the body in action. More accurately, a metaphoric gesture is part of the dynamic interaction between speakers that enables interpersonal coordination and intersubjectivity. One concern is whether metaphoric gestures are indeed understood as intentional communications. People appear to have two different capacities for displaying information (Gibbs, 1999). Information “given” is information intentionally emitted by a person and recognized by another in the manner intended by the actor. On the other hand, information “given-off ” is information interpreted for meaning by another person even if it had not been intended to convey that meaning. Thus, when Jack spies on Sally, he can draw various meaningful inferences about what Sally might be doing even though Sally’s gestures and actions are only providing information that is given-off, not intentionally given to Jack. Studies show that people can differentiate, even without speech, between gestures that are intended to convey meaning and gestures that only seem to emphasize what a speaker is saying (Feyereisen, van de Wiele, & Dubois, 1988). This conclusion may be extended to suggest that metaphoric gestures, facial expressions, and so on, are also specifically produced to be understood as part of a person’s overall communicative intentions and must be recognized as such for successful interpersonal interactions to occur (see Müller, this volume). Although there is no experimental evidence to support the above conjecture, as of yet, there is a growing literature showing that people appear to specifically design their gestures for intentional purposes. Studies show, for example, that during causal conversation between first and second language speakers of Japanese, first language speakers will often complete statements using gestures when they believe that their addressees, the second language users, may not be able to understand the words spoken (Mori & Hayaski, 2006). Second language users observed these “embodied completions” and showed their understanding of what was intended by continuing the talk and focusing on the meaning of what was gestured, and not just the words spoken. It appears, then, that speakers’ awareness of their co-participants’ linguistic background fosters the use of gestures to enhance intersubjectivity during talk exchanges. One possibility is that speakers will employ metaphoric gestures particularly in cases where there are concerns over whether the addressee will be able to understand a specific metaphoric statement, with these metaphoric gestures sometimes serving as embodied completions of what speakers were attempting to verbally articulate. Thus, metaphoric gestures surely reflect

3rd proofs

 Raymond W. Gibbs, Jr



Metaphor and gesture 

(veracity factor). In both conditions, participants were then accused of lying by the interviewer and asked to repeat their account a second time (suspicion factor). Interestingly, lying was associated with a decrease in deictic gestures, but with a significant increase in metaphoric gestures. These findings suggests that the increased use of metaphoric gestures, at least in some situations, compared to some previously established baseline, may indicate a speaker’s attempt to deceive others. Whether a conversationalist can detect variations in the amount and kind (i.e., metaphoric) of gesturing in real-world contexts is not yet established, but this topic is clearly worthy of additional psychological research with implications for theories of interpersonal communication.

“growth points” in which a speaker dynamically reconfigures what is said and gesturally enacted to meet the demands of the social context (McNeill, 2005). More generally, metaphoric gestures are not just part of the meaning expressed and understood by conversationalists, but a major part of how speakers and listeners ground their joint understandings. For instance, a person may gesture with palms up, as if offering, to suggest that some message being communicated by another is obvious. Many other instances of face-to-face interaction are structured by the conduit metaphor such as when a listener uses subtle hand and finger movements to suggest that the speaker should continue speaking as if the message is a physical entity being transferred from one person to another. When discussing difficult topics, speakers will sometimes rapidly wave their hands to indicate the complexity of what they are trying to communicate, or wave their hands in front of their faces as if to ward off the multiple messages being rapidly hurled at them. Speakers may place their arms out beside themselves and act as if they are pushing alternately against different forces to indicate a struggle to maintain balance in instances where they are experiencing difficulty with the burden of what is being communicated. People will even display metaphoric facial expressions to communicate their metaphoric reaction to a speaker’s message, such as when a listener displays a disgust face to suggest displeasure with what a speaker is saying as if the speaker were presenting rotten food in front of the listener’s nose. These observations on the prominence of metaphoric gestures in face-to-face communications are indirectly supported by research on human and computerized embodied agent interactions, where there are major attempts to get computerized figures to gesture appropriately during talk with humans (Cassell et al., 2001; Yan, 2000). There is, as yet, not much explicit attention paid to the appropriate production of metaphoric gestures by embodied computerized agents, although some of the protocols available of human-computer verbal interactions suggest that some agents may be doing so already to some degree (see Cassell et al., 2001). There may be important positive benefits of incorporating appropriate metaphoric gestures in human-computer interaction, a possibility that is also ripe for future empirical study in psychology and cognitive science. Finally, the frequently repeated wisdom in social psychology that people’s body actions are often more truthful than their words spoken raises one interesting question about metaphoric gestures. Might people use metaphoric gestures more often when they are trying to conceal something, or avoid responsibility for the impact of what they are communicating? A recent study suggests this may be the case (Letiza et al., 2006). Hand gestures were experimentally studied during truth telling and deception, and in situations with either weak or strong suspicion. Participants in this study were interviewed twice about the possession of an object. In one interview they were asked to lie and in the other asked to tell the truth

6. Moving beyond the hands Most of the work on metaphoric gesture has focused on how the hands work to communicate metaphoric ideas. But, as noted above, people move other body parts, and even their whole bodies, to express metaphoric messages. Even further, people sometimes perform whole body actions that clearly represent metaphoric ideas. During the World Cup football/soccer matches in 1994, players who had just scored a goal were seen mimicking a person pulling a gun out of a holster, shooting the gun forward, and replacing it back in a holster, to convey the message that scoring a goal was metaphorically like slaying an opponent in a gun battle or duel. Another European football player once, after scoring a goal, ran the white end-line, kneeled down and pretended to snort the white substance there, as if he were ingesting a drug like cocaine. This action is perhaps doubly metaphoric in that not only is scoring a football goal metaphorically like experiencing the rush of ingesting a powerful drug, but the idea of “scoring” in football is linked with “scoring” some drug. Other simple, widely known metaphoric actions include cutting one’s throat and hanging oneself by a rope, both of which are especially salient gestures because they refer, like the previous examples, to taboo topics (i.e., killing someone, doing illegal drugs, killing oneself), which is perhaps why these gestures can be enacted without one verbally saying what one really means. Metaphoric gestures like these are parodies of well known body routines and convey a rich set of meanings that would be impossible to communicate via words, especially in cases where the audience is unlikely to hear any words spoken, as in a large, noisy football stadium.

3rd proofs

 Raymond W. Gibbs, Jr



7. Conclusion

Metaphor and gesture  Gibbs, R. W. (2006a). Embodiment and cognitive science. New York: Cambridge University Press. Gibbs, R. W. (2006b). Embodied simulation in metaphor interpretation. Mind & Language, 21, 434–458. Glucksberg, S. (2001). Understanding figurative language: From metaphor to idioms. New York: Oxford University Press. Iverson, J. & E. Thelen (1999). Hand, mouth, and brain: The dynamic emergence of speech and gesture. Journal of Consciousness Studies, 11–12, 19–40. Kendon, A. & L. Versante (2003). Pointing by hand in Neapolitan. In S. Kita (Ed.), Pointing: Where language culture and cognition meet (pp. 109–137). Hillsdale, N.J.: Lawrence Erlbaum. Krauss, R. (1998). Why do we gesture when we speak? Current Directions in Psychological Science, 7, 54–60. Letizia C., F. Maricchiolo, M. Bonaiuto, A. Vrij, & S. Mann (2006). The impact of deception and suspicion on different hand movements. Journal of Nonverbal Behavior, 30, 1–9. McNeill, D. (1992). Hand and mind. Chicago: University of Chicago Press. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. Mori, J. & M. Hayashi (2006). The achievement of intersubjectivity through embodied completions: a study of interaction between first and second language speakers. Applied Linguistics, 27, 195–219. Murphy, G. (1996). On metaphoric representations. Cognition, 60, 173–204. Müller, C. (1998). Redebegleitende Gesten: Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berlin Verlag Arno Spitz. Yan, H. (2000). Paired speech and gesture generation in embodied conversational agents. MA thesis, MIT Media Laboratory, Cambridge, MA.

Making and understanding metaphoric gestures are psychological events, within cultural contexts, and must be studied in a systematic way that acknowledges several major factors that likely shape production and understanding of these actions. These include (1) the participants (e.g., their background abilities, interests, beliefs, motivations), (2) the participants’ immediate discourse goals (e.g., fill in for missing words, communicate something different from what was spoken, add redundancy to the spoken message, be provocative), (3) the participants’ criteria for production and understanding (e.g., to express an exact vs. approximate meaning, to generally understand, to remember how something was expressed), and (4) the stimuli (e.g., conventional or novel metaphoric gesture, whether it accompanies speech and the temporal relation with that speech). Metaphoric gesture production and understanding will depend on the complex interaction of these different factors, which should cause psychologists, and others, to be cautious in the generalizations they make from the data collected and explained. But the ground-breaking body of work seen in this volume testifies to some of the many empirical possibilities on human thought and communication processes that cry out for new empirical study. At the very least, the research described in this volume suggests the increasing possibility for fruitful interdisciplinary collaboration between linguistics and psychology on the topic of metaphor. References Butterworth, B. & U. Hadar (1989). Gesture, speech and computational stages: A reply to McNeill. Psychological Review, 96, 168–174. Calbris, G. (1990). The semiotics of French gestures. Bloomington, IN: Indiana University Press. Cassell, J., T. Bickmore, L. Campbell, H. Vihjammsseon, & H. Yan (2001). More than just a pretty face. Conversational protocols and the affordances of embodiment. Knowledge-Based Systems, 14, 55–64. Decety, J. & J. Grezes (1999). Neural mechanisms subserving the perception of human action. Trends in Cognitive Sciences, 3, 172–178. Feyereisen, P., M. van de Wiele, & F. Dubois (1988). The meaning of gestures: What can be understood without speech. Cahiers de Psychologie Cognitives, 8, 3–25. Gallese, V. & G. Lakoff (2005). The brain’s concepts. The role of sensory-motor system in conceptual knowledge. Cognitive Neuropsychology, 22, 455–479. Gibbs, R. W. (1994). The poetics of mind: Figurative thought, language, and understanding. New York: Cambridge University Press. Gibbs, R. W. (1999). Intentions in the experience of meaning. New York: Cambridge University Press.

3rd proofs

Gesture Studies (GS)

Metaphor and Gesture

Gesture Studies aims to publish book-length publications on all aspects of gesture. These include, for instance, the relationship between gesture and speech; the role gesture may play in social interaction; gesture and cognition; the development of gesture in children; the processes by which spontaneously created gestures may become transformed into codified forms; the relationship between gesture and sign; biological studies of gesture, including the place of gesture in language evolution; and gesture in human-machine interaction. Volumes in this peer-reviewed series may be collected volumes, monographs, or reference books, in the English language.

Edited by

Alan Cienki Vrije Universiteit Amsterdam

Cornelia Müller Europa-Universität Viadrina

Editors Adam Kendon

University of Pennsylvania, Philadelphia

Cornelia Müller

European University Viadrina, Frankfurt/Oder

Volume 3

John Benjamins Publishing Company

Metaphor and Gesture Edited by Alan Cienki and Cornelia Müller

Amsterdam / Philadelphia

3rd proofs