Javier Valenzuela

Toward an infrastructure for data-driven multimodal communication research

Abstract Research into the multimodal dimensions of human communication faces a set of distinctive methodological challenges. Collecting the datasets is resource-intensive, analysis often lacks peer validation, and the absence of shared datasets makes it difficult to develop standards. External validity is hampered by small datasets, yet large datasets are intractable. Red Hen Lab spearheads an international infrastructure for data-driven multimodal communication research, facilitating an integrated cross-disciplinary workflow. Linguists, communication scholars, statisticians, and computer scientists work together to develop research questions, annotate training sets, and develop pattern discovery and machine learning tools that handle vast collections of multimodal data, beyond the dreams of previous researchers. This infrastructure makes it possible for researchers at multiple sites to work in real-time in transdisciplinary teams. We review the vision, progress, and prospects of this research consortium.

Timelines and multimodal constructions: Facing new challenges

Abstract There is little doubt that spatial information underlies a great deal of our processing of temporal information. Research on the ways in which timelines are associated with specific grammatical constructions has just started, and has brought forward new challenges for construction grammar. A true multimodal construction grammar cannot be developed using the same tools and mechanisms applied so far, but will require instead a fresh approach and a careful rethinking of the whole communicative process. It is our belief than in this transition from a narrow conception of linguistic communication to a broader and more complete view of communication “in the wild”, multimodal construction grammar will play an important role.