This volume advances the data-based study of multimodal artefacts and performances by showcasing methods and results from the latest endeavors in empirical multimodal research, representing a vibrant international and interdisciplinary research community. The collated chapters identify and seek to inspire novel, mixed-method approaches to investigate meaning-making mechanisms in current communicative artifacts, designs, and contexts; while attending to their immersive, aesthetic, and ideological dimensions. Each contribution details innovative aspects of empirical multimodality research, offering insights into challenges evolving from quantitative approaches, particular corpus work, results from eye-tracking and psychological experiments, and analyses of dynamic interactive experiences. The approaches and results presented foreground the inherent multidisciplinary nature and implications of multimodality, renegotiating concepts across linguistics, media studies, (social) semiotics, game studies, and design. With this, the volume will inform both current and future developments in theory, methods, and transdisciplinary contexts and become a landmark reference for anyone interested in the empirical study of multimodality.

This volume advances the data-based study of multimodal artefacts and performances by showcasing methods and results from the latest endeavors in empirical multimodal research, representing a vibrant international and interdisciplinary research community. The collated chapters identify and seek to inspire novel, mixed-method approaches to investigate meaning-making mechanisms in current communicative artifacts, designs, and contexts; while attending to their immersive, aesthetic, and ideological dimensions. Each contribution details innovative aspects of empirical multimodality research, offering insights into challenges evolving from quantitative approaches, particular corpus work, results from eye-tracking and psychological experiments, and analyses of dynamic interactive experiences. The approaches and results presented foreground the inherent multidisciplinary nature and implications of multimodality, renegotiating concepts across linguistics, media studies, (social) semiotics, game studies, and design. With this, the volume will inform both current and future developments in theory, methods, and transdisciplinary contexts and become a landmark reference for anyone interested in the empirical study of multimodality.

Multimodality has most recently been described no longer as a research field or discipline on its own, but rather as a “stage of development within a field” (Bateman 2022a, 49). The realization that (1) many different fields and disciplines now enter their own multimodal phase with new interest in multimodal phenomena and that (2) these disciplines all commit to the development of multimodality research with their own theoretical principles and methodological tools, brings with it not only an immense breadth of potential analytical objects, but also many new meta-methodological issues. “We need to find ways of ‘combining’ insights from the variously imported theoretical and methodological backgrounds brought along by previous non-multimodal stages of any contributing disciplines” (Bateman 2022a, 49). At the same time, the search for a meta-methodology for multimodal analyses is pushed further by the recent trend towards more empirical approaches to multimodal phenomena and the development and use of larger multimodal corpora that just as well require theoretical and methodological refinements. “We need to develop ways of strengthening claims with robustly applicable methods which nevertheless remain firmly anchored theoretically” (Bateman 2022b, 64). For a productive handling of these issues, disciplinary triangulation and finding a ‘common language’ or metalanguage (Maton & Chen 2016) for an ‘integrationist interdisciplinarity’ (van Leeuwen 2005) are the greatest challenges in contemporary multimodality research (Bateman 2022a). Also, there is a need for reconceptualizing the practice of analysis by making available large-scale corpora and broader and more complex empirical setups to fully process the ‘move from theory to data,’ and to substantiate long-lasting theoretical and methodological hypotheses (Pflaeging et al. 2021). For this project, we see these challenges productively as “a multimodal task from the ground up,” as John Bateman (2022b, 64) has phrased it in one of his most recent papers. This Research Topic will address this task by convening the most recent theoretical, methodological, practical, and empirical developments within contemporary multimodality research. The aim is to gain new insights in • the metalanguages or external languages that are currently being developed for multimodal analysis in many different research fields and disciplines, e.g., in pedagogy, literary theory, cultural studies, design, argumentation theory, computer science, and (experimental) psychology; • newest results from data collection methods and multimodal corpus analyses that expand the current quantitative work by, e.g., applying existing theories and methods to larger datasets, or exploring the newest communication technologies. We are particularly interested in seeing how works addressing these aspects contribute to finding ways of productive triangulation and integration for and within a meta-methodology for multimodality research. This Research Topic aims to bring together scholars from a variety of disciplines interested in multimodality research to review, explore, and advance the contributions that John Bateman, as one of the key figures in multimodality research, has made to both theory- and method-building as well as to the driving forward of multimodal empirical and corpus analyses. We welcome contributions that, for example, • critically address the theoretical and methodological advancements that John Bateman has made with regard to the notions of semiotic mode, discourse semantics, genre, textuality, etc.; • apply one of the many approaches that John Bateman has developed for the empirical analysis of multimodal artefacts (e.g., the GeM model for page-based documents, his work on multimodal film and audio-visual analysis, and the discourse semantics and/or annotation approach to visual narratives) to larger corpora or currently newly developing communicative situations; • expand on one of the abovementioned aspects with new ideas and insights from disciplines that have not yet been included in multimodality research.

This edited volume brings together work in the field of empirical comics research. Drawing on computer and cognitive science, psychology and art history, linguistics and literary studies, each chapter presents innovative methods and establishes the practical and theoretical motivations for the quantitative study of comics, manga, and graphic novels. Individual chapters focus on corpus studies, the potential of crowdsourcing for comics research, annotation and narrative analysis, cognitive processing and reception studies. This volume opens up new perspectives for the study of visual narrative, making it a key reference for anyone interested in the scientific study of art and literature as well as the digital humanities.

This volume addresses some of the most important conceptual, methodological, and empirical challenges and opportunities with which the sister disciplines of semiotics and discourse analysis are mutually confronted in the context of considering new avenues of cross-disciplinary application to distinctive branding research streams. In continuation of the collective volume 'Handbook of Brand Semiotics' (Kassel University Press, 2015), which sought to consolidate relevant scholarship and to identify the main territories that have been established at the cross-roads between branding and semiotic research, the current 'Advances in Brand Semiotics & Discourse Analysis' aims at accomplishing further strides in critical areas, such as the exigency for reconsidering the aptness of existing semiotic theories in the face of the radically shifting co-creative landscape of digital branding, the benefits of systematically micro-analyzing brand communities’ discourses by drawing on CAQDAS programs, the combination of big data analytics with discourse theory in corpus analysis, and the epistemological issues that emerge while combining discourse analysis with time-hallowed marketing qualitative and quantitative research methods. At the same time, the volume hosts a resourceful blend of empirical studies and novel conceptual frameworks in burgeoning streams, such as place, heritage, culinary, personal, and political branding.

This timely book will guide researchers on how to apply qualitative research methods to explore English-medium instruction (EMI) issues, such as classroom interactions, teachers’ and students’ perceptions on language and pedagogical challenges, and stakeholders’ views on the implementation of EMI. Each chapter focuses on a specific type of qualitative research methodology, beginning with an overview of the research and the method used, before presenting a unique case study. Chapters will also identify the process that EMI researchers went through to conduct their research, the key dilemmas they faced, and focus particularly on the methodological issues they encountered. By exploring these issues and providing up-to-date insights in contexts across the globe, this book informs theory or the lack thereof, underlying research into the phenomenon of EMI. This text will be indispensable for researchers who want to learn and acquire skills in conducting qualitative research in EMI, as well as undergraduate and postgraduate students reading in the fields of applied linguistics and language education.

This textbook provides the first foundational introduction to the practice of analysing multimodality, covering the full breadth of media and situations in which multimodality needs to be a concern. Readers learn via use cases how to approach any multimodal situation and to derive their own specifically tailored sets of methods for conducting and evaluating analyses. Extensive references and critical discussion of existing approaches from many disciplines and in each of the multimodal domains addressed are provided. The authors adopt a problem-oriented perspective throughout, showing how an appropriate foundation for understanding multimodality as a phenomenon can be used to derive strong methodological guidance for analysis as well as supporting the adoption and combination of appropriate theoretical tools. Theoretical positions found in the literature are consequently always related back to the purposes of analysis rather than being promoted as valuable in their own right. By these means the book establishes the necessary theoretical foundations to engage productively with today’s increasingly complex combinations of multimodal artefacts and performances of all kinds.

Designing Learning for Multimodal Literacy addresses the need to design learning for multimodal literacy in a world that is increasingly saturated with print and digital media. In the current age, communication and interactions on social media are seldom made with language alone but are often accompanied with emojis, images, and videos, making meanings multimodally. Young people, including children, are also increasingly active in making videos of themselves, their ideas, and their experiences as part of their out-of-school literacy activities. In particular, for language teachers, the present shifts in our world require that teachers re-examine what they teach and how they can meaningfully and effectively teach the students in their classes today. At 8 years old, Alden created his own rap music video and shared it with the world. He wrote his own lyrics and set it against the music he remixed and meshed from a music download site. Alden is in your classroom today. As his teacher, what would you teach him? How would you engage him? Alden, and children like him, is the inspiration for why the authors have written this book. The changing times and changing learners place a demand on educators to continually reflect on what and how teachers are teaching their students – to ensure that learning in school remains relevant, relatable, and prepares them for the world of the future. Lim’s book outlines how teachers can design learning for multimodal literacy. It is a result of a collaboration between an educational researcher and a curriculum developer, and offers practical resources for practitioners but also design principles and considerations based on practice with a range of students to inform and inspire academics and postgraduate students. It is poised to contribute to the global conversation and interest on how educators can reflect on the zeitgeist of the digital age and design learning for multimodal literacy.

This book develops a new framework for describing the structure of multimodal documents: how language, image, layout and other modes of communication work together to convey meaning. Building on recent research in multimodal analysis, functional linguistics and information design, the book examines the textual, visual, and spatial aspects of page-based multimodal documents and employs an analytical model to describe and interpret their structure using the concepts of semiotic modes, medium and genre. To demonstrate and test this approach, the study performs a systematic, longitudinal analysis of a corpus of multimodal documents within a single genre: an extensively annotated corpus of tourist brochures produced between 1967-2008. The book provides multimodal discourse analysts with methodological tools to draw empirically-based conclusions about multimodal documents, and will be a valuable resource for researchers planning to develop and study multimodal corpora.