Winslow Burleson

Affect-aware tutors: recognising and responding to student affect

Theories and technologies are needed to understand and integrate the knowledge of student affect (e.g., frustration, motivation and self-confidence) into learning models. Our goals are to redress the cognitive versus affective imbalance in teaching systems, develop tools that model student affect and build tutors that elicit, measure and respond to student affect. This article describes our broad approach towards this goal and our three main objectives: develop tools for affect recognition, interventions in response to student affect, and emotionally animated agents.

PaperPhone: understanding the use of bend gestures in mobile devices with flexible electronic paper displays

Flexible displays potentially allow for interaction styles that resemble those used in paper documents. Bending the display, e.g., to page forward, shows particular promise as an interaction technique. In this paper, we present an evaluation of the effectiveness of various bend gestures in executing a set of tasks with a flexible display. We discuss a study in which users designed bend gestures for common computing actions deployed on a smartphone-inspired flexible E Ink prototype called PaperPhone. We collected a total of 87 bend gesture pairs from ten participants and their appropriateness over twenty actions in five applications. We identified six most frequently used bend gesture pairs out of 24 unique pairs. Results show users preferred bend gestures and bend gesture pairs that were conceptually simpler, e.g., along one axis, and less physically demanding. There was a strong agreement among participants to use the same three pairs in applications: (1) side of display, up/down (2) top corner, up/down (3) bottom corner, up/down. For actions with a strong directional cue, we found strong consensus on the polarity of the bend gestures (e.g., navigating left is performed with an upwards bend gesture, navigating right, downwards). This implies that bend gestures that take directional cues into account are likely more natural to users.

Developing creativity, motivation, and self-actualization with learning systems

Developing learning experiences that facilitate self-actualization and creativity is among the most important goals of our society in preparation for the future. To facilitate deep understanding of a new concept, to facilitate learning, learners must have the opportunity to develop multiple and flexible perspectives. The process of becoming an expert involves failure, as well as the ability to understand failure and the motivation to move onward. Meta-cognitive awareness and personal strategies can play a role in developing an individuals ability to persevere through failure, and combat other diluting influences. Awareness and reflective technologies can be instrumental in developing a meta-cognitive ability to make conscious and unconscious decisions about engagement that will ultimately enhance learning, expertise, creativity, and self-actualization. This paper will review diverse perspectives from psychology, engineering, education, and computer science to present opportunities to enhance creativity, motivation, and self-actualization in learning systems.

The effect of motivational learning companions on low achieving students and students with disabilities

We report the results of a randomized controlled evaluation of the effectiveness of pedagogical agents as providers of affective feedback. These digital learning companions were embedded in an intelligent tutoring system for mathematics, and were used by approximately one hundred students in two public high schools. Students in the control group did not receive the learning companions. Results indicate that low-achieving students—one third of whom have learning disabilities—had higher affective needs than their higher-achieving peers; they initially considered math problem-solving more frustrating, less exciting, and felt more anxious when solving math problems. However, after they interacted with affective pedagogical agents, low-achieving students improved their affective outcomes, e.g., reported reduced frustration and anxiety.

The Impact of Animated Pedagogical Agents on Girls' and Boys' Emotions, Attitudes, Behaviors and Learning

We report on the reactions of males and female students to the presence of animated pedagogical agents that provided emotional and motivational support. One hundred high school students used agents embedded in an Intelligent Tutoring System for Mathematics and randomized controlled evaluations compared students with and without learning companions. The results indicate that affective pedagogical agents improve affective outcomes of students in general and particularly so for female students, who reported being more frustrated and less confident while solving math problems prior to using the tutoring system. We discuss issues of incorporating gender into user models and of generating responses tailored to gender.

An analysis of students' gaming behaviors in an intelligent tutoring system: predictors and impacts

Students who exploit properties of an instructional system to make progress while avoiding learning are said to be “gaming” the system. In order to investigate what causes gaming and how it impacts students, we analyzed log data from two Intelligent Tutoring Systems (ITS). The primary analyses focused on six college physics classes using the Andes ITS for homework and test preparation, starting with the research question: What is a better predictor of gaming, problem or student? To address this question, we developed a computational gaming detector for automatically labeling the Andes data, and applied several data mining techniques, including machine learning of Bayesian network parameters. Contrary to some prior findings, the analyses indicated that student was a better predictor of gaming than problem. This result was surprising, so we tested and confirmed it with log data from a second ITS (the Algebra Cognitive Tutor) and population (high school students). Given that student was more predictive of gaming than problem, subsequent analyses focused on how students gamed and in turn benefited (or not) from instructional features of the environment, as well as how gaming in general influenced problem solving and learning outcomes.

ABE: An Agent-Based Software Architecture for a Multimodal Emotion Recognition Framework

The computers ability to recognize human emotional states given physiological signals is gaining in popularity to create empathetic systems such as learning environments, health care systems and videogames. Despite that, there are few frameworks, libraries, architectures, or software tools, which allow systems developers to easily integrate emotion recognition into their software projects. The work reported here offers a first step to fill this gap in the lack of frameworks and models, addressing: (a) the modeling of an agent-driven component-based architecture for multimodal emotion recognition, called ABE, and (b) the use of ABE to implement a multimodal emotion recognition framework to support third-party systems becoming empathetic systems.

Embodiment, Multimodality, and Composition: Convergent Themes across HCI and Education for Mixed-Reality Learning Environments

We present concurrent theoretical work from HCI and Education that reveals a convergence of trends focused on the importance of three themes: embodiment, multimodality, and composition. We argue that there is great potential for truly transformative work that aligns HCI and Education research, and posit that there is an important opportunity to advance this effort through the full integration of the three themes into a theoretical and technological framework for learning. We present our own work in this regard, introducing the Situated Multimedia Arts Learning Lab (SMALLab). SMALLab is a mixed-reality environment where students collaborate and interact with sonic and visual media through full-body, 3D movements in an open physical space. SMALLab emphasizes human-to-human interaction within a multimodal, computational context. We present a recent case study that documents the development of a new SMALLab learning scenario, a collaborative student participation framework, a student-centered curriculum, and a three-day teaching experiment for seventy-two earth science students. Participating students demonstrated significant learning gains as a result of the treatment. We conclude that our theoretical and technological framework can be broadly applied in the realization of mixed reality, student-centered learning environments.

Evaluation of a meta-tutor for constructing models of dynamic systems

Abstract Modelling is an important skill to acquire, but it is not an easy one for students to learn. Existing instructional technology has had limited success in teaching modelling. We have applied a recently developed technology, meta-tutoring, to address the important problem of teaching model construction. More specifically, we have developed and evaluated a system that has two parts, a tutor and a meta-tutor. The tutor is a simple step-based tutoring system that can give correct/incorrect feedback on students steps and can demonstrate steps for students when asked. Because deep modelling requires difficult analyses of the quantitative relationships in a given system, we expected, and found, that students tended to avoid deep modelling by abusing the tutors help. In order to increase the frequency of deep modelling, we added a meta-tutor that coached students to follow a learning strategy that decomposed the overall modelling problem into a series of “atomic” modelling problems. We conducted three experiments to test the effectiveness of the meta-tutor. The results indicate that students who studied with meta-tutor did indeed engage in more deep modelling practices. However, when the meta-tutor and tutor were turned off, students tended to revert to shallow modelling. Thus, the next stage of the research is to add an affective agent that will try to persuade students to persist in using the taught strategies even when the meta-tutoring and tutoring have ceased.

A tangible programming tool for creation of context-aware applications

End-user programming tools, if properly designed, have the potential to empower end-users to create context-aware applications tailored to their own needs and lives, in order to help them break bad habits and change their behaviors. In this work, we present GALLAG Strip, an easy to use mobile and tangible tool that allows users to create context-aware applications without the need of programming experience. It enables programming by physical demonstration of envisioned interactions with the same sensors and objects that users will later encounter in their finished application. After an initial pilot to verify the usability of GALLAG Strip, we conducted a user study to evaluate the effects of tangible programming in terms of ease of use, engagement, and facilitation of the ideation process. We found that tangibility has both benefits and drawbacks, and suggest a mixed tangible and non-tangible approach for better user experience.