Jason M. Harley

Using Trace Data to Examine the Complex Roles of Cognitive, Metacognitive, and Emotional Self-Regulatory Processes During Learning with Multi-agent Systems

This chapter emphasizes the importance of using multi-channel trace data to examine the complex roles of cognitive, affective, and metacognitive (CAM) self-regulatory processes deployed by students during learning with multi-agent systems. We argue that tracing these processes as they unfold in real-time is key to understanding how they contribute both individually and together to learning and problem solving. In this chapter we describe MetaTutor (a multi-agent, intelligent hypermedia system) and how it can be used to facilitate learning of complex biological topics and as a research tool to examine the role of CAM processes used by learners. Following a description of the theoretical perspective and underlying assumptions of self-regulated learning (SRL) as an event, we provide empirical evidence from five different trace data, including concurrent think-alouds, eye-tracking, note taking and drawing, log-files, and facial recognition, to exemplify how these diverse sources of data help understand the complexity of CAM processes and their relation to learning. Lastly, we provide implications for future research of advanced leaning technologies (ALTs) that focus on examining the role of CAM processes during SRL with these powerful, yet challenging, technological environments.

Inferring Learning from Gaze Data during Interaction with an Environment to Support Self-Regulated Learning

In this paper, we explore the potential of gaze data as a source of information to predict learning as students interact with MetaTutor, an ITS that scaffolds self-regulated learning. Using data from 47 college students, we show that a classifier using a variety of gaze features achieves considerable accuracy in predicting student learning after seeing gaze data from the complete interaction. We also show promising results on the classifier ability to detect learning in real-time during interaction.

Examining the predictive relationship between personality and emotion traits and students' agent-directed emotions: towards emotionally-adaptive agent-based learning environments

The current study examined the relationships between learners’ (

An Integrated Emotion-Aware Framework for Intelligent Tutoring Systems

N = 123

Developing Emotion-Aware, Advanced Learning Technologies: A Taxonomy of Approaches and Features

N=123) personality traits, the emotions they typically experience while studying (trait studying emotions), and the emotions they reported experiencing as a result of interacting with four pedagogical agents (agent-directed emotions) in MetaTutor, an advanced multi-agent learning environment. Overall, significant relationships between a subset of trait emotions (trait anger, trait anxiety) and personality traits (agreeableness, conscientiousness, and neuroticism) were found for four agent-directed emotions (enjoyment, pride, boredom, and neutral) though the relationships differed between pedagogical agents. These results demonstrate that some trait emotions and personality traits can be used to predict learners’ emotions directed toward specific pedagogical agents (with different roles). Results provide suggestions for adapting pedagogical agents to support learners’ (with certain characteristics; e.g., high in neuroticism or agreeableness) experience of adaptive emotions (e.g., enjoyment) and minimize their experience on non-adaptive emotions (e.g., boredom). Such an approach presents a scalable and easily implementable method for creating emotionally-adaptive, agent-based learning environments, and improving learner-pedagogical agent interactions in order to support learning.

Examining the Predictive Relationship Between Personality and Emotion Traits and Learners’ Agent-Direct Emotions

In this study we aligned and compared self-report and on-line emotions data on 67 college students’ emotions at five different points in time over the course of their interactions with MetaTutor. Self-reported emotion data as well as facial expression data were converged and analyzed. Results across channels revealed that neutral and positively-valenced basic and learner-centered emotional states represented the majority of emotional states experienced with MetaTutor. The self-report results revealed a decline in the intensity of positively-valenced and neutral states across the learning session. The facial expression results revealed a substantial decrease in the number of learners’ with neutral facial expressions from time one to time two, but a fairly stable pattern for the remainder of the session, with participants who experienced other basic emotional states, transitioning back to a state of neutral between self-reports. Agreement between channels was 75.6%.

Examining the Relationship Between Performance Feedback and Emotions in Diagnostic Reasoning: Toward a Predictive Framework for Emotional Support

This conceptual paper integrates empirical studies and existing conceptual work describing emotion regulation strategies deployed in intelligent tutoring systems and advances an integrated framework for the development and evaluation of emotion-aware systems.

Examining learners' emotional responses to virtual pedagogical agents' tutoring strategies

This review provides a contemporary, critical survey of the interdisciplinary methods used in research with computer-based learning environments (CBLEs) to measure learners’ emotions, including: facial expression coding, body posture and physiological measurement devices, log-file data, and self-report measures. Novel insights are provided and important issues pertaining to emotion measurement are revisited in light of new research, including: theoretical and analytical considerations; the effectiveness of multimethod (e.g., multimodal) approaches; and the potential use of different types of data for informing emotionally supportive CBLEs. Recommendations are also shared to assist researchers in improving their emotion measurement methodologies. This chapter was written with the educational psychology community in mind to help support a growing interest in alternative and complimentary methods to self-report measures. As such, this chapter stands to not only add to the existing interdisciplinary discussion on emotion measurement methods, but also to expand it to potentially new participants.