This volume is the first to offer an overview on metaphor and gesture -- a new multi-disciplinary area of research. This book presents a snapshot of the state of the art in these converging fields, offering research papers as well as commentaries from multiple perspectives.

This dissertation is a study of metaphor in usage: metaphor in language, metaphor in gesture, and how they interact. Gesture provides a route to study both the cognition associated with language and the domain-generality of cognitive processes. While English speakers may be producing metaphoric manner verbs due to the lexicalization patterns of their language, are they necessarily thinking in terms of metaphoric manner? This is difficult to judge when looking at language alone. To answer this question, we turn to metaphoric gesture. Metaphoric gestures, in which the gesture represents the source domain of a conceptual metaphor, are well-known but under-studied (Cienki and Müller, 2008b). Iconic gestures conveying information about a motion event are known to interact with the syntactic and semantic structure of speech; speakers of languages that express manner of motion in the verb gesture differently than speakers of languages that primarily express path of motion in the verb. Metaphoric usages of motion in language - prices falling, hopes rising, time flying - also interact with the grammatical patterns of language. However, we know little about how metaphoric motion in gesture interacts with grammar. In Part One of the dissertation, I focus on metaphor in language. In Chapter 2 I propose to represent metaphors as a complex network of frames, mappings, and bindings as implemented in the MetaNet Metaphor Repository (Dodge et al., 2015). This advances the representation of conceptual metaphors to a level that interfaces more accurately with representations of frames and constructions in FrameNet (Ruppenhofer et al., 2006) and Embodied Construction Grammar (Bergen and Chang, 2005). In turn, this enables the detailed analysis of metaphors and metaphor systems, as exemplified by the Location Event Structure Metaphor (Lakoff and Johnson, 1999) case study in Chapter 3. This corpus-based study is one of the first to make use of the MetaNet method for large-scale automatic metaphor identification and annotation. This approach reveals not only how the metaphor system is evoked in language, but further illustrates the conceptual structure of the metaphor. I demonstrate that although English, as a satellite-framed language, privileges manner in its lexicalization of motion events, metaphoric English motion backgrounds manner and foregrounds path. The foregrounding of path information in linguistic realization of metaphoric motion runs counter to the privileging of manner in English lexicalization patterns. This finding lays the groundwork for the investigation of the same metaphor system in gesture. In Part Two, I focus on metaphor in co-speech gesture. I investigate metaphoric motion-evoking metaphoric gestures using two complementary approaches. Chapter 4 uses a corpus approach; I analyze a parallel corpus of video gesture data in which speakers use a motion verb either literally or metaphorically in their speech while producing a co-expressive representational gesture. To analyze the corpora, I develop a set of annotation guidelines and then demonstrate the benefits of taking an image-schematic approach to gesture analysis. I argue that the image schema is the most appropriate level of structure in analyzing the form and meaning of metaphoric gestures. Results of this image schema analysis suggest that, reflecting the English language data in Chapter 3, these metaphoric gestures emphasize path and do not represent the manner of motion. Chapter 5 is the first study to take an experimental approach to metaphoric gesture that uses non-metaphoric stimuli. Participants were given short stories about state change, such as prices decreasing or grades improving, to read and re-tell to a friend; half of the stimuli contained metaphoric language and half did not. Results from this study demonstrated the viability of this methodology in eliciting both metaphoric speech and gesture, and supported those of Chapter 4. I find that speakers are more likely produce metaphoric gestures if they are also producing metaphoric language - even if the gesture evokes a different metaphor than the speech does. I unify my analyses of metaphoric motion in speech and gesture in a multi-modal Embodied Construction Grammar analysis of both co-expressive and complementary metaphoric co-speech gestures. I represent both the meaning and form of the gesture and the meaning and form of the speech including frame structure, argument structure, and metaphoric structure. This analysis provides the first formal representation of a multi-modal utterance in a construction grammar and an innovative approach to the unification of the construction of multi-modal meaning.

This volume is the first to offer an overview on metaphor and gesture — a new multi-disciplinary area of research. Scholars of metaphor have been paying increasing attention to spontaneous gestures with speech; meanwhile, researchers in gesture studies have been focussing on the abstract ideas which receive physical representation through metaphors when speakers gesture. This book presents a snapshot of the state of the art in these converging fields, offering research papers as well as commentaries from multiple perspectives. In addition to conceptual metaphor theory it includes different theoretical approaches to semiotics, and the methods used range from controlled experimentation, to cognitive ethnography, to lexical semantic analysis. The use of metaphor in gesture is shown to reflect idiosyncracies of thought in the moment of speaking as well as structural, cultural, and interactional patterns. The series of commentaries discusses the potential importance of studying metaphor and gesture from the perspectives of such fields as anthropology, cognitive linguistics, conversation analysis, psychology, and semiotics.

This book constitutes the proceedings of the 14th International Conference on Intelligent Virtual Agents, IVA 2014, held in Boston, MA, USA, in August 2014. The 14 full and 24 short papers presented were carefully reviewed and selected from 78 submissions. In addition, the volume includes 25 demo and poster papers which were on display during the conference. The papers cover many aspects of intelligent virtual agent theory and application with a special focus on their use in healthcare.

Summarizing her pioneering work on the semiotic analysis of gestures in conversational settings, Geneviève Calbris offers a comprehensive account of her unique perspective on the relationship between gesture, speech, and thought. She highlights the various functions of gesture and especially shows how various gestural signs can be created in the same gesture by analogical links between physical and semantic elements. Originating in our world experience via mimetic and metonymic processes, these analogical links are activated by contexts of use and thus lead to a diverse range of semantic constructions rather as, from the components of a Meccano kit, many different objects can be assembled. By (re)presenting perceptual schemata that mediate between the concrete and the abstract, gesture may frequently anticipate verbal formulation. Arguing for gesture as a symbolic system in its own right that interfaces with thought and speech production, Calbris’ book brings a challenging new perspective to gesture studies and will be seminal for generations of gesture researchers.

In cognitive metaphor theory, metaphor is a conceptual and experiential process that structures our world. This book, taking an applied linguistic view on CMT, examines how metaphor is manifested in speech and via gestures by music teachers in classrooms where Mandarin Chinese is the main language employed. Thirteen music sessions by, and interviews with, six teachers in six junior high schools in Taiwan constitute the data. The three-stage analysis focuses on the nature of verbal and gestural metaphors, the relations between verbal and gestural metaphors, their functions, and the educational implications. The study represents an original and exploratory empirical contribution to the field, and the results further support CMT by providing empirical data on how native speakers of Chinese express metaphor via the two modalities. It also contributes to metaphor identification procedures for identifying metaphorically-used words in Chinese, and coding metaphoric gestures.