Roger Azevedo

Defining and Measuring Engagement and Learning in Science: Conceptual, Theoretical, Methodological, and Analytical Issues

Engagement is one of the most widely misused and overgeneralized constructs found in the educational, learning, instructional, and psychological sciences. The articles in this special issue represent a wide range of traditions and highlight several key conceptual, theoretical, methodological, and analytical issues related to defining and measuring engagement. All the approaches exemplified by the contributors show different ways of conceptualizing and measuring engagement and demonstrate the strengths and weaknesses of each method to significantly augment our current understanding of engagement. Despite the numerous issues raised by the authors of this special issue and in my commentary, I argue that focusing on process data will lead to advances in models, theory, methods, analytical techniques, and ultimately instructional recommendations for learning contexts that effectively engage students.

Examining the predictive relationship between personality and emotion traits and students' agent-directed emotions: towards emotionally-adaptive agent-based learning environments

The current study examined the relationships between learners’ (

Using multi-channel data with multi-level modeling to assess in-game performance during gameplay with Crystal Island

N = 123

Using Data Visualizations to Foster Emotion Regulation During Self-Regulated Learning with Advanced Learning Technologies

N=123) personality traits, the emotions they typically experience while studying (trait studying emotions), and the emotions they reported experiencing as a result of interacting with four pedagogical agents (agent-directed emotions) in MetaTutor, an advanced multi-agent learning environment. Overall, significant relationships between a subset of trait emotions (trait anger, trait anxiety) and personality traits (agreeableness, conscientiousness, and neuroticism) were found for four agent-directed emotions (enjoyment, pride, boredom, and neutral) though the relationships differed between pedagogical agents. These results demonstrate that some trait emotions and personality traits can be used to predict learners’ emotions directed toward specific pedagogical agents (with different roles). Results provide suggestions for adapting pedagogical agents to support learners’ (with certain characteristics; e.g., high in neuroticism or agreeableness) experience of adaptive emotions (e.g., enjoyment) and minimize their experience on non-adaptive emotions (e.g., boredom). Such an approach presents a scalable and easily implementable method for creating emotionally-adaptive, agent-based learning environments, and improving learner-pedagogical agent interactions in order to support learning.

Examining the Predictive Relationship Between Personality and Emotion Traits and Learners’ Agent-Direct Emotions

Recent years have seen a growing recognition of the role that affect plays in learning. Because game-based learning environments elicit a wide range of student affective states, affect-enhanced student modeling for game-based learning holds considerable promise. This paper introduces an affect-enhanced student modeling framework that leverages facial expression tracking for game-based learning. The affect-enhanced student modeling framework was used to generate predictive models of student learning and student engagement for students who interacted with CRYSTAL ISLAND, a game-based learning environment for microbiology education. Findings from the study reveal that the affect-enhanced student models significantly outperform baseline predictive student models that utilize the same gameplay traces but do not use facial expression tracking. The study also found that models based on individual facial action coding units are more effective than composite emotion models. The findings suggest that introducing facial expression tracking can improve the accuracy of student models, both for predicting student learning gains and also for predicting student engagement.

The Impact of Student Individual Differences and Visual Attention to Pedagogical Agents During Learning with MetaTutor

Game-based learning environments (GBLEs) have been touted as the solution for failing educational outcomes. In this study, we address some of these major issues by using multi-level modeling with data from eye movements and log files to examine the cognitive and metacognitive self-regulatory processes used by 50 college students as they read books and completed the associated in-game assessments (concept matrices) while playing the Crystal Island game-based learning environment. Results revealed that participants who read fewer books in total, but read each of them more frequently, and who had low proportions of fixations on books and concept matrices exhibited the strongest performance. Results stress the importance of assessing quality vs. quantity during gameplay, such that it is important to read books in-depth (i.e., quality), compared to reading books once (i.e., quantity). Implications for these findings involve designing adaptive GBLEs that scaffold participants based on their trace data, such that we can model efficient behaviors that lead to successful performance.

Is More Agency Better? The Impact of Student Agency on Game-Based Learning

This chapter provides a synthesis of several research methods used by interdisciplinary researchers to investigate emotions in advanced learning technologies. More specifically, the authors: (1) critique self-report measures used to investigate emotions; (2) briefly describe Scherer’s (2009) model as particularly relevant for investigating emotions due to its complex appraisal system; (3) describe three process-oriented methods (electrodermal activity, facial expressions, and eye-tracking) currently used by interdisciplinary researchers to detect, identify, and classify affective states during learning and briefly highlight strengths and weaknesses of each method; and (4) present major conceptual, methodological, and analytical issues related to investigating emotions during learning with advanced learning technologies.

Impact of Individual Differences on Affective Reactions to Pedagogical Agents Scaffolding

Emotions play a critical role during learning and problem solving with advanced learning technologies. However, learners typically do not accurately monitor and regulate their emotions and may therefore not learn as much, disengage from the task, and not optimize their learning of the instructional material. Despite their importance, relatively few attempts have been made to understand learners’ emotional monitoring and regulation during learning with advanced learning technologies by using data visualizations of their own (and others’) cognitive, affective, metacognitive, and motivational (CAMM) self-regulated learning (SRL) processes to potentially foster their emotion regulation during learning with advanced learning technologies. We present a theoretically-based and empirically-driven conceptual framework that addresses emotion regulation by proposing the use of visualizations of one’s and others’ CAMM-SRL multichannel data (e.g., cognitive strategy use, metacognitive monitoring accuracy, facial expressions of emotions, physiological arousal, eye-movement behaviors, etc.) to facilitate learners’ monitoring and regulation of their emotions during learning with advanced learning technologies. We use examples from several of our laboratory and classroom studies to illustrate a possible mapping between theoretical assumptions, emotion-regulation strategies, and the types of data visualizations that can be used to enhance and scaffold learners’ emotion regulation, including key processes such as emotion flexibility, emotion adaptivity, and emotion efficacy. We conclude with future directions that can lead to a systematic interdisciplinary research agenda that addresses outstanding emotion regulation-related issues by integrating models, theories, methods, and analytical techniques for the areas of cognitive, learning, and affective sciences, human computer interaction, data visualization, big data, data mining, data science, learning analytics, open learner models, and SRL.