Ivon Arroyo

Affect-aware tutors: recognising and responding to student affect

Theories and technologies are needed to understand and integrate the knowledge of student affect (e.g., frustration, motivation and self-confidence) into learning models. Our goals are to redress the cognitive versus affective imbalance in teaching systems, develop tools that model student affect and build tutors that elicit, measure and respond to student affect. This article describes our broad approach towards this goal and our three main objectives: develop tools for affect recognition, interventions in response to student affect, and emotionally animated agents.

Toward Measuring and Maintaining the Zone of Proximal Development in Adaptive Instructional Systems

Intelligent tutoring Systems (ITSs) adapt content and activities with the goals of being both effective and efficient instructional environments. They have goals for students to be challenged and guided in an optimal way-- without being too overwhelmed with difficult material or too bored with easy or repetitive material. We propose a particular definition of the zone of proximal development (ZPD) as a general way to describe what all ITSs try to do, and we propose a foundational analysis of instructional adaptivity, student modeling, and system evaluation in terms of the ZPD. We give an operational definition of the ZPD and give an example of its use, and summarize how instructional methods such as scaffolding can be used to maintain ZPD-learning. We also explain how our definition of the ZPD can lead to a more complete model for efficient and effective instruction than common mastery learning criteria.

The effect of motivational learning companions on low achieving students and students with disabilities

We report the results of a randomized controlled evaluation of the effectiveness of pedagogical agents as providers of affective feedback. These digital learning companions were embedded in an intelligent tutoring system for mathematics, and were used by approximately one hundred students in two public high schools. Students in the control group did not receive the learning companions. Results indicate that low-achieving students—one third of whom have learning disabilities—had higher affective needs than their higher-achieving peers; they initially considered math problem-solving more frustrating, less exciting, and felt more anxious when solving math problems. However, after they interacted with affective pedagogical agents, low-achieving students improved their affective outcomes, e.g., reported reduced frustration and anxiety.

Web-Based Intelligent Multimedia Tutoring for High Stakes Achievement Tests

We describe Wayang Outpost, a web-based ITS for the Math section of the Scholastic Aptitude Test (SAT). It has several distinctive features: help with multimedia animations and sound, problems embedded in narrative and fantasy contexts, alternative teaching strategies for students of different mental rotation abilities and memory retrieval speeds. Our work on adding intelligence for adaptivity is described. Evaluations prove that students learn with the tutor, but learning depends on the interaction of teaching strategies and cognitive abilities. A new adaptive tutor is being built based on evaluation results; surveys results and students’ log files analyses.

A Multimedia Adaptive Tutoring System for Mathematics that Addresses Cognition, Metacognition and Affect

This article describes research results based on multiple years of experimentation and real-world experience with an adaptive tutoring system named Wayang Outpost. The system represents a novel adaptive learning technology that has shown successful outcomes with thousands of students, and provided teachers with valuable information about students’ mathematics performance. We define progress in three areas: improved student cognition, engagement, and affect, and we attribute this improvement to specific components and interventions that are inherently affective, cognitive, and metacognitive in nature. For instance, improved student cognitive outcomes have been measured with pre-post tests and state standardized tests, and achieved due to personalization of content and math fluency training. Improved student engagement was achieved by supporting students’ metacognition and motivation via affective learning companions and progress reports, measured via records of student gaming of the system. Student affect within the tutor was measured through sensors and student self-reports, and supported through affective learning companions and progress reports. Collectively, these studies elucidate a suite of effective strategies to support advanced personalized learning via an intelligent adaptive tutor that can be tailored to the individual needs, emotions, cognitive states, and metacognitive skills of learners.

The Impact of Animated Pedagogical Agents on Girls' and Boys' Emotions, Attitudes, Behaviors and Learning

We report on the reactions of males and female students to the presence of animated pedagogical agents that provided emotional and motivational support. One hundred high school students used agents embedded in an Intelligent Tutoring System for Mathematics and randomized controlled evaluations compared students with and without learning companions. The results indicate that affective pedagogical agents improve affective outcomes of students in general and particularly so for female students, who reported being more frustrated and less confident while solving math problems prior to using the tutoring system. We discuss issues of incorporating gender into user models and of generating responses tailored to gender.

Inferring Unobservable Learning Variables from Students’ Help Seeking Behavior

Results of an evaluation of students’ attitudes and their relationship to student behaviors within a tutoring system are presented. Starting from a correlation analysis that integrates survey-collected student attitudes, learning variables, and behaviors while using the tutor, we constructed a Bayesian Network that infers attitudes and perceptions towards help and the tutoring system.

AgentX: Using Reinforcement Learning to Improve the Effectiveness of Intelligent Tutoring Systems

Reinforcement Learning (RL) can be used to train an agent to comply with the needs of a student using an intelligent tutoring system. In this paper, we introduce a method of increasing efficiency by way of customization of the hints provided by a tutoring system, by applying techniques from RL to gain knowledge about the usefulness of hints leading to the exclusion or introduction of other helpful hints. Students are clustered into learning levels and can influence the agents method of selecting actions in each state in their cluster of affect. In addition, students can change learning levels based on their performance within the tutoring system and continue to affect the entire student population. The RL agent, AgentX, then uses the cluster information to create one optimal policy for all students in the cluster and begin to customize the help given to the cluster based on that optimal policy.

Actionable Affective Processing for Automatic Tutor Interventions

Once a tutoring system is able to detect students’ emotions, it is not obvious how to change the tutor’s behavior to leverage this emotion detection for the benefit of the student. For instance, if students state that they are excited, then providing harder problems may be appropriate in one case, while providing actions to calm them down so that they can better focus may be the best response in other cases. Both the cognitive and emotional states are important when choosing the tutor’s actions. The purpose of this chapter is to describe the elements necessary for a tutoring system that makes appropriate actions based on a detected affective state. This is broken down into three parts. First we describe several methods for emotion detection. Then we present a study using Wayang Outpost, our math tutor, using sensors to detect the student’s emotion and taking actions based on that emotion. Then we discuss potential actions for the detected emotions. We conclude with future steps needed to improve the actions of tutoring systems in general.

Improving math learning through intelligent tutoring and basic skills training

We studied the effectiveness of a math fact fluency tool integrated with an intelligent tutor as a means to improve student performance in math standardized tests. The study evaluated the impact of Math Facts Retrieval Training (MFRT) on 250 middle school students and analyzed the main effects of the training by itself and also as a supplement to the Wayang Tutoring System on easy and hard items of the test. Efficacy data shows improved student performance on tests and positive impact on mathematics learning. We also report on interaction effects of MFRT with student gender and incoming math ability.