Malte F. Jung

Glance: rapidly coding behavioral video with the crowd

Behavioral researchers spend considerable amount of time coding video data to systematically extract meaning from subtle human actions and emotions. In this paper, we present Glance, a tool that allows researchers to rapidly query, sample, and analyze large video datasets for behavioral events that are hard to detect automatically. Glance takes advantage of the parallelism available in paid online crowds to interpret natural language queries and then aggregates responses in a summary view of the video data. Glance provides analysts with rapid responses when initially exploring a dataset, and reliable codings when refining an analysis. Our experiments show that Glance can code nearly 50 minutes of video in 5 minutes by recruiting over 60 workers simultaneously, and can get initial feedback to analysts in under 10 seconds for most clips. We present and compare new methods for accurately aggregating the input of multiple workers marking the spans of events in video data, and for measuring the quality of their coding in real-time before a baseline is established by measuring the variance between workers. Glances rapid responses to natural language queries, feedback regarding question ambiguity and anomalies in the data, and ability to build on prior context in followup queries allow users to have a conversation-like interaction with their data - opening up new possibilities for naturally exploring video data.

Mindless computing: designing technologies to subtly influence behavior

Persuasive technologies aim to influence users behaviors. In order to be effective, many of the persuasive technologies developed so far relies on users motivation and ability, which is highly variable and often the reason behind the failure of such technology. In this paper, we present the concept of Mindless Computing, which is a new approach to persuasive technology design. Mindless Computing leverages theories and concepts from psychology and behavioral economics into the design of technologies for behavior change. We show through a systematic review that most of the current persuasive technologies do not utilize the fast and automatic mental processes for behavioral change and there is an opportunity for persuasive technology designers to develop systems that are less reliant on users motivation and ability. We describe two examples of mindless technologies and present pilot studies with encouraging results. Finally, we discuss design guidelines and considerations for developing this type of persuasive technology.

EmotionCheck: leveraging bodily signals and false feedback to regulate our emotions

In this paper we demonstrate that it is possible to help individuals regulate their emotions with mobile interventions that leverage the way we naturally react to our bodily signals. Previous studies demonstrate that the awareness of our bodily signals, such as our heart rate, directly influences the way we feel. By leveraging these findings we designed a wearable device to regulate users anxiety by providing a false feedback of a slow heart rate. The results of an experiment with 67 participants show that the device kept the anxiety of the individuals in low levels when compared to the control group and the other conditions. We discuss the implications of our findings and present some promising directions for designing and developing this type of intervention for emotion regulation.

Coupling Interactions and Performance: Predicting Team Performance from Thin Slices of Conflict

Do teams show stable conflict interaction patterns that predict their performance hours, weeks, or even months in advance? Two studies demonstrate that two of the same patterns of emotional interaction dynamics that distinguish functional from dysfunctional marriages also distinguish high from low-performance design teams in the field, up to 6 months in advance, with up to 91% accuracy, and based on just 15minutes of interaction data: Group Affective Balance, the balance of positive to negative affect during an interaction, and Hostile Affect, the expression of a set of specific negative behaviors were both found as predictors of team performance. The research also contributes a novel method to obtain a representative sample of a teams conflict interaction. Implications for our understanding of design work in teams and for the design of groupware and feedback intervention systems are discussed.

Affective Grounding in Human-Robot Interaction

Participating in interaction requires not only coordination on content and process, as previously proposed, but also on affect. The term affective grounding is introduced to refer to the coordination of affect in interaction with the purpose of building shared understanding about what behavior can be exhibited, and how behavior is interpreted emotionally and responded to. Affective Ground is achieved when interactants have reached shared understanding about how behavior should be interpreted emotionally. The paper contributes a review and critique of current perspectives on emotion in HRI. Further it outlines how research on emotion in HRI can benefit from taking an affective grounding perspective and outlines implications for the design of robots capable of participating in the coordination on affect in interaction.

Human Expectations of Social Robots

A key assumption that drives much of HRI research is that human robot collaboration can be improved by advancing a robots capabilities. We argue that this assumption has potentially negative implications, as increasing social capabilities in robots can produce an expectations gap where humans develop unrealistically high expectations of social robots due to generalization from human mental models. By conducting two studies with 674 participants, we examine how people develop and adjust mental models of robots. We find that both a robots physical appearance and its behavior influence how we form these models. This suggests it is possible for a robot to unintentionally manipulate a human into building an inaccurate mental model of its overall abilities simply by displaying a few capabilities that humans possess, such as speaking and turn-taking. We conclude that this expectations gap, if not corrected for, could ironically result in less effective collaborations as robot capabilities improve.

Crystallize: An Immersive, Collaborative Game for Second Language Learning

Learning a second language is challenging. Becoming fluent requires learning contextual information about how language should be used as well as word meanings and grammar. The majority of existing language learning applications provide only thin context around content. In this paper, we present Crystallize, a collaborative 3D game that provides rich context along with scaffolded learning and engaging gameplay mechanics. Players collaborate through joint tasks, or quests. We present a user study with 42 participants that examined the impact of low and high levels of task interdependence on language learning experience and outcomes. We found that requiring players to help each other led to improved collaborative partner interactions, learning outcomes, and gameplay. A detailed analysis of the chat-logs further revealed that changes in task interdependence affected learning behaviors.

Robots in Groups and Teams

Over the last decade, the idea that robots could become an integral part of groups and teams has developed from a promising vision into a reality. Robots are increasingly designed to interact with groups and teams of people, yet most human-robot interaction research still focuses on a single humans interacting with a single robot. The goal for the workshop is therefore to advance research in computer supported cooperative work (CSCW) and human robot interaction (HRI) by raising awareness for the social and technical challenges that surround the placement of robots within work-groups and teams. The workshop will be organized around three central questions: (1) How do robots shape the dynamics of groups and teams in existing settings? (2) How does a robots behavior shape how humans interact with each other in dyads and in larger groups and teams? (3) How can robots improve the performance of work groups and teams by acting on social processes? These core issues will be covered across a set of presentations that initiate in- depth discussions around each question to improve the quality of and support the growth of research in the CSCW community that focuses on the intersection of robots, groups, and teams.

Have your Cake and Eat it Too: Foreign Language Learning with a Crowdsourced Video Captioning System

Learning from captioned foreign language videos is highly effective, but the availability of such videos is limited. By using speech-to-text technology to generate partially correct transcripts as a starting point, we see an opportunity for learners to build accurate foreign language captions while learning at the same time. We present a system where learners correct captions using automatic transcription and machine-generated suggested alternative words for scaffolding. In a lab study of 49 participants, we found that compared to watching the video with accurate caption, learning and quality of experience were not significantly impaired by the secondary caption correction task using interface designs either with or without scaffolding from speech-to-text generated alternative words. Nevertheless, aggregating corrections reduced word error rate from 19% to 5.5% without scaffolding from suggested-alternatives, and 1.8% with scaffolding. Feedback from participants suggest that emphasizing the learning community contribution aspect is important for motivating learners and reducing frustration.

Participatory materials: having a reflective conversation with an artifact in the making

Designing and building mechatronic systems has gradually ceased to be the domain of only highly trained professionals and has become broadly accessible. Drawing from a notion of designing as a conversation with the materials of the situation we built an artifact that could actively engage in its own making by embedding a Wizard of Oz operated animated agent into an Arduino prototyping platform. In a 2x2 between-participants Wizard of Oz laboratory experiment with (N=68) high-school students we specifically examined how this prototyping agents expression of interest affected perceptions of the agent and learning outcomes dependent on the embodiment of the agent as embedded in the prototyping material itself or as an external entity. We found evidence that embedding an agent into the prototyping material can positively influence learning processes and outcomes while not harming perceptions of the agent.