This book provides an extended exploration of the multimodal analysis of spatial (three-dimensional) texts of the built environment, culminating in a holistic approach termed Spatial Discourse Analysis (SpDA). Based on existing frameworks of multimodal analysis, this book applies, adapts, and extends these frameworks to spatial texts. The authors argue that choices in spatial design create meanings about what we perceive and how we can or should behave within spatial texts, influence how we feel in and about those spaces, and enable these texts to function as coherent wholes. Importantly, a spatial text, once built, is also a resource which is then used, and an essential aspect of understanding these texts is to consider what users themselves contribute to the meaning potential of these texts. The book takes the metafunctional approach familiar from Systemic-Functional Linguistics (SFL) and foregrounds each metafunction in turn (textual, interpersonal, experiential, and logical), in relation to the detailed analysis of a particular spatial text.

This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many disciplines and in each of the multimodal domains addressed are provided. The authors adopt a problem-oriented perspective throughout, showing how an appropriate foundation for understanding multimodality as a phenomenon can be used to derive strong methodological guidance for analysis as well as supporting the adoption and combination of appropriate theoretical tools. Theoretical positions found in the literature are consequently always related back to the purposes of analysis rather than being promoted as valuable in their own right. By these means the book establishes the necessary theoretical foundations to engage productively with today’s increasingly complex combinations of multimodal artefacts and performances of all kinds.

This book brings together the work of leading theorist, Theo van Leeuwen, on typography, colour, texture, sound and movement, and shows how they are used to communicate identity, both corporate and individual. The book provides a detailed approach to analysing the key elements of multimodal style, and shows how these can be applied to a wide range of domains, including typography, product design, architecture, and animation films. Combining sociological insights into contemporary forms of identity with multimodal approaches to analysing how these identities are expressed, the text is richly illustrated with examples from fashion, the built environment, logos, modern art and more. With sample analyses, this user-friendly text provides clear methods for analysis and creative strategies for the practice of multimodal communication. Providing an invaluable toolkit to analysing the key elements of multimodal design and the way they work together, this book is essential reading for students, teachers and researchers in the field of multimodal communication, whether in communication studies, linguistics, design studies, media studies or the arts.

Multimodality in Writing attempts to generate and apply new theories, disciplines and methods to account for semiotic processes in texts and during text production. It thus showcases new directions in multimodal research and theorizing writing practices from a multimodal perspective. It explores texts, producers of texts, and readers of texts. It also focuses on teaching multimodal text production and writing pedagogy from different domains and disciplines, such as rhetoric and writing composition, architecture, mathematics, film-making, science and the newsroom. Multimodality in Writing explores the kinds of methodological approaches that can augment social semiotic approaches to analyzing and teaching writing, including rhetoric, Systemic Functional Linguistics, ethnographic approaches, and genre pedagogy. Much of the research shows how the regularities of modes and interest of sign makers are socially shaped to realize convention. Because of this, the approaches are strongly underpinned by social and cultural theories of representation and communication.

Multimodality’s popularity as a semiotic approach has not resulted in a common voice yet. Its conceptual anchoring as well as its empirical applications often remain localized and disparate, and ideas of a theory of multimodality are heterogeneous and uncoordinated. For the field to move ahead, it must achieve a more mature status of reflection, mutual support, and interaction with regard to both past and future directions. The red thread across the disciplines reflected in this book is a common goal of capturing the mechanisms of synergetic knowledge construction and transmission using diverse forms of expressions, i.e., multimodality. The collection of chapters brought together in the book reflects both a diversity of disciplines and common interests and challenges, thereby establishing an excellent roadmap for the future. The contributions revisit and redefine theoretical concepts or empirical analyses, which are crucial to the study of multimodality from various perspectives, with a view towards evolving issues of multimodal analysis. With this, the book aims at repositioning the field as a well-grounded scientific discipline with significant implications for future communication research in many fields of study.

During the last decade, cell phones with multimodal interfaces based on combined new media have become the dominant computer interface worldwide. Multimodal interfaces support mobility and expand the expressive power of human input to computers. They have shifted the fulcrum of human-computer interaction much closer to the human. This book explains the foundation of human-centered multimodal interaction and interface design, based on the cognitive and neurosciences, as well as the major benefits of multimodal interfaces for human cognition and performance. It describes the data-intensive methodologies used to envision, prototype, and evaluate new multimodal interfaces. From a system development viewpoint, this book outlines major approaches for multimodal signal processing, fusion, architectures, and techniques for robustly interpreting users' meaning. Multimodal interfaces have been commercialized extensively for field and mobile applications during the last decade. Research also is growing rapidly in areas like multimodal data analytics, affect recognition, accessible interfaces, embedded and robotic interfaces, machine learning and new hybrid processing approaches, and similar topics. The expansion of multimodal interfaces is part of the long-term evolution of more expressively powerful input to computers, a trend that will substantially improve support for human cognition and performance. Table of Contents: Preface: Intended Audience and Teaching with this Book / Acknowledgments / Introduction / Definition and Typre of Multimodal Interface / History of Paradigm Shift from Graphical to Multimodal Interfaces / Aims and Advantages of Multimodal Interfaces / Evolutionary, Neuroscience, and Cognitive Foundations of Multimodal Interfaces / Theoretical Foundations of Multimodal Interfaces / Human-Centered Design of Multimodal Interfaces / Multimodal Signal Processing, Fusion, and Architectures / Multimodal Language, Semantic Processing, and Multimodal Integration / Commercialization of Multimodal Interfaces / Emerging Multimodal Research Areas, and Applications / Beyond Multimodality: Designing More Expressively Powerful Interfaces / Conclusions and Future Directions / Bibliography / Author Biographies

The “smart mobile” has become an essential and inseparable part of our lives. This powerful tool enables us to perform multi-tasks in different modalities of voice, text, gesture, etc. The user plays an important role in the mode of operation, so multimodal interaction provides the user with new complex multiple modalities of interfacing with a system, such as speech, touch, type and more. The book will discuss the new world of mobile multimodality, focusing on innovative technologies and design which create a state-of-the-art user interface. It will examine the practical challenges entailed in meeting commercial deployment goals, and offer new approaches to the designing such interfaces. A multimodal interface for mobile devices requires the integration of several recognition technologies together with sophisticated user interface and distinct tools for input and output of data. The book will address the challenge of designing devices in a synergetic fashion which does not burden the user or to create a technological overload.