Max M. Louwerse

Toward a Taxonomy of a Set of Discourse Markers in Dialog: A Theoretical and Computational Linguistic Account

Discourse markers are verbal and nonverbal devices that mark transition points in communication. They presumably facilitate the construction of a mental representation of the events described by the discourse. A taxonomy of these relational markers is one important beginning in investigations of language use. Although several taxonomies of coherence relations have been proposed for monolog, only a few have been proposed for dialog. This article argues that discourse markers are important in language use because they operate at different levels of the dialog. What these levels are and how markers function is discussed by amalgamating 2 leading theories of language use. Based on this theory, a taxonomy of between-turn coherence relations in dialog is presented and several issues that arise out of constructing such a taxonomy are discussed. By sampling a large number of discourse markers from a corpus and substituting each marker for all other markers, this extensive substitution test could determine whether hyponymous, hypernymous, and synonymous relations existed between the markers from this corpus of dialogs. Evidence is presented for clustering this set of discourse markers into four categories: direction, polarity, acknowledgment, and emphatics.

Reproducing affective norms with lexical co-occurrence statistics: Predicting valence, arousal, and dominance.

Human ratings of valence, arousal, and dominance are frequently used to study the cognitive mechanisms of emotional attention, word recognition, and numerous other phenomena in which emotions are hypothesized to play an important role. Collecting such norms from human raters is expensive and time consuming. As a result, affective norms are available for only a small number of English words, are not available for proper nouns in English, and are sparse in other languages. This paper investigated whether affective ratings can be predicted from length, contextual diversity, co-occurrences with words of known valence, and orthographic similarity to words of known valence, providing an algorithm for estimating affective ratings for larger and different datasets. Our bootstrapped ratings achieved correlations with human ratings on valence, arousal, and dominance that are on par with previously reported correlations across gender, age, education and language boundaries. We release these bootstrapped norms for 23,495 English words.

The Effects of Personal Involvement in Narrative Discourse

Over the last several decades, the study of discourse processes has moved from the complementary efforts characteristic of multidisciplinary research to the explicitly integrative focus of interdisciplinary research (Graesser, Gernsbacher, & Goldman, 2003; Louwerse & Van Peer, 2002). Organizations like the International Society for the Empirical Study of Literature (IGEL), the International Association of Empirical Aesthetics, and the Poetics and Linguistics Association have supported the methodological and conceptual merger of areas such as literary studies, psychology, linguistics, and education. As is evident in this special issue, research concerning personal involvement in narrative discourse has benefitted from these developments. Across disciplinary boundaries, a variety of terms have been used to identify personal involvement in narrative presentations. Readers may become captured by a literary text, moviegoers may become entranced by a cinematic narrative, members of an audience may be moved by a dramatic performance, and so on. Despite terminological diversity, something common seems at stake: Often during narrative encounters, feeling becomes fluid, comprehension seems multifaceted, and DISCOURSE PROCESSES, 38(2), 169–172 Copyright

Design for Collaboration in Mixed Reality: Technical Challenges and Solutions

One of the key challenges in the rapid technological advance of Virtual Reality (VR) and Mixed Reality (MR) concerns the design of collaborative experiences. VR systems do not readily support team collaboration because they tend to focus on individual experiences and do not easily facilitate naturalistic collaboration. MR environments provide solutions for collaborative experiences, but establishing smooth communication between hardware components and software modules faces a major hurdle. This paper presents the background to and main challenges of an ongoing project on collaboration in an MR lab, aiming to design a serious team collaboration game. To this end, we utilized a common game engine to engineer a cost-effective solution that would make the game playable in a configuration operated by WorldViz and Volfoni equipment. Evaluation of various solutions in the development process found a Unity 3D Cluster Rendering Beta solution to be the most cost-effective and successful.

Estimating valence from the sound of a word: Computational, experimental, and cross-linguistic evidence.

It is assumed linguistic symbols must be grounded in perceptual information to attain meaning, because the sound of a word in a language has an arbitrary relation with its referent. This paper demonstrates that a strong arbitrariness claim should be reconsidered. In a computational study, we showed that one phonological feature (nasals in the beginning of a word) predicted negative valence in three European languages (English, Dutch, and German) and positive valence in Chinese. In three experiments, we tested whether participants used this feature in estimating the valence of a word. In Experiment 1, Chinese and Dutch participants rated the valence of written valence-neutral words, with Chinese participants rating the nasal-first neutral-valence words more positive and the Dutch participants rating nasal-first neutral-valence words more negative. In Experiment 2, Chinese (and Dutch) participants rated the valence of Dutch (and Chinese) written valence-neutral words without being able to understand the meaning of these words. The patterns replicated the valence patterns from Experiment 1. When the written words from Experiment 2 were transformed into spoken words, results in Experiment 3 again showed that participants estimated the valence of words on the basis of the sound of the word. The computational study and psycholinguistic experiments indicated that language users can bootstrap meaning from the sound of a word.

The Bursts and Lulls of Multimodal Interaction: Temporal Distributions of Behavior Reveal Differences between Verbal and Non-Verbal Communication.

Recent studies of naturalistic face-to-face communication have demonstrated coordination patterns such as the temporal matching of verbal and non-verbal behavior, which provides evidence for the proposal that verbal and non-verbal communicative control derives from one system. In this study, we argue that the observed relationship between verbal and non-verbal behaviors depends on the level of analysis. In a reanalysis of a corpus of naturalistic multimodal communication (Louwerse, Dale, Bard, & Jeuniaux, ), we focus on measuring the temporal patterns of specific communicative behaviors in terms of their burstiness. We examined burstiness estimates across different roles of the speaker and different communicative modalities. We observed more burstiness for verbal versus non-verbal channels, and for more versus less informative language subchannels. Using this new method for analyzing temporal patterns in communicative behaviors, we show that there is a complex relationship between verbal and non-verbal channels. We propose a temporal heterogeneity hypothesis to explainxa0how the language system adapts to the demands of dialog.

AMELIO: Evaluating the Team-building Potential of a Mixed Reality Escape Room Game

The authors investigate the potential of Mixed Reality (MR) games for team building and assessment. The AMELIO game was designed for a highly immersive MR lab. The game is a multi-player team challenge based on the concept of an escape room, staged in a space colony emergency situation. An explorative empirical pre-post measurement study was carried out to establish whether playing AMELIO influences team cohesiveness. Ten teams of three played AMELIO and filled out pre- and post-game questionnaires with validated measurements of team cohesiveness and mediating factors related to team composition, game experience and team dynamics. The findings show a positive and significant increase in team cohesiveness, with stronger effects for teams with lower pre-game familiarity. In terms of game experience and team dynamics, audio aesthetics and empathy proved to be significant mediating factors. This aids game validation and improvement, and understanding and guiding the team building process.

Archaeology Through Computational Linguistics: Inscription Statistics Predict Excavation Sites of Indus Valley Artifacts.

Computational techniques comparing co-occurrences of city names in texts allow the relative longitudes and latitudes of cities to be estimated algorithmically. However, these techniques have not been applied to estimate the provenance of artifacts with unknown origins. Here, we estimate the geographic origin of artifacts from the Indus Valley Civilization, applying methods commonly used in cognitive science to the Indus script. We show that these methods can accurately predict the relative locations of archeological sites on the basis of artifacts of known provenance, and we further apply these techniques to determine the most probable excavation sites of four sealings of unknown provenance. These findings suggest that inscription statistics reflect historical interactions among locations in the Indus Valley region, and they illustrate how computational methods can help localize inscribed archeological artifacts of unknown origin. The success of this method offers opportunities for the cognitive sciences in general and for computational anthropology specifically.

Knowing the Meaning of a Word by the Linguistic and Perceptual Company It Keeps

Debates on meaning and cognition suggest that an embodied cognition account is exclusive of a symbolic cognition account. Decades of research in the cognitive sciences have, however, shown that these accounts are not at all mutually exclusive. Acknowledging cognition is both symbolic and embodied generates more relevant questions that propel, rather than divide, the cognitive sciences: questions such as how computational symbolic findings map onto experimental embodied findings, and under what conditions cognition is relatively more symbolic or embodied in nature. The current paper revisits the Symbol Interdependency Hypothesis, which argues that language encodes perceptual information and that language users rely on these language statistics in cognitive processes. It argues that the claim that words are abstract, amodal, and arbitrary symbols and therefore must always be grounded to become meaningful is an oversimplification of the language system. Instead, language has evolved such that it maps onto the perceptual system, whereby language users rely on language statistics, which allow for bootstrapping meaning also when grounding is limited.

Estimating Emotions Through Language Statistics and Embodied Cognition

Recent research has suggested that language processing activates perceptual simulations. We have demonstrated that findings that have been attributed to an embodied cognition account can also be explained by language statistics, because language encodes perceptual information. We investigated whether comprehension of emotion words can be explained by an embodied cognition or a language statistics account. A corpus linguistic study comparing emotions words showed that words denoting the same emotions (happy–delighted) co-occur more frequently than different emotions (happy–angry). These findings were used in two experiments in which participants read same-emotion and different-emotion sentence pairs. Sentence pairs with different emotions yielded longer RTs than sentences with the same emotions both in a cognitive task tailored toward linguistic representations and a task tailored toward embodied representations. These findings contribute to a growing body of literature that demonstrates that language processing does not always rely solely on perceptual simulation.