Bradford W. Mott

Modeling self-efficacy in intelligent tutoring systems: An inductive approach

Self-efficacy is an individual’s belief about her ability to perform well in a given situation. Because self-efficacious students are effective learners, endowing intelligent tutoring systems with the ability to diagnose self-efficacy could lead to improved pedagogy. Self-efficacy is influenced by (and influences) affective state. Thus, physiological data might be used to predict a student’s level of self-efficacy. This article investigates an inductive approach to automatically constructing models of self-efficacy that can be used at runtime to inform pedagogical decisions. It reports on two complementary empirical studies. In the first study, two families of self-efficacy models were induced: a static self-efficacy model, learned solely from pre-test (non-intrusively collected) data, and a dynamic self-efficacy model, learned from both pre-test data as well as runtime physiological data collected with a biofeedback apparatus. In the second empirical study, a similar experimental design was applied to an interactive narrative-centered learning environment. Self-efficacy models were induced from combinations of static and dynamic information, including pre-test data, physiological data, and observations of student behavior in the learning environment. The highest performing induced naïve Bayes models correctly classified 85.2% of instances in the first empirical study and 82.1% of instances in the second empirical study. The highest performing decision tree models correctly classified 86.9% of instances in the first study and 87.3% of instances in the second study.

U-director: a decision-theoretic narrative planning architecture for storytelling environments

Recent years have seen significant growth in work on interactive storytelling environments. A key challenge posed by these environments is narrative planning, in which a director agent orchestrates all of the events in a storyworld to create an optimal experience for a user, who is herself an active participant in the unfolding story. To create effective stories, the director agent must cope with the tasks inherent uncertainty, including uncertainty about the users intentions and the absence of a complete theory of narrative. Director agents must be efficient so they can operate in real time. In this paper, we present U-Director, a decision-theoretic narrative planning architecture that dynamically models narrative objectives (e.g., plot progress, narrative flow), storyworld state (e.g., plot focus), and user state (e.g., goals, beliefs) with a dynamic decision network that continually selects storyworld actions to maximize narrative utility on an ongoing basis. The U-DIRECTOR architecture has been implemented in a narrative planner for Crystal Island, an interactive storyworld in which users play the role of a medical detective solving a science mystery. Preliminary evaluations suggest that the U-DIRECTOR architecture satisfies the real-time constraints of interactive environments and creates engaging narrative experiences.

Problem Solving and Game-Based Learning: Effects of Middle Grade Students' Hypothesis Testing Strategies on Learning Outcomes

Targeted as a highly desired skill for contemporary work and life, problem solving is central to game-based learning research. In this study, middle grade students achieved significant learning gains from gameplay interactions that required solving a science mystery based on microbiology content. Student trace data results indicated that effective exploration and navigation of the hypothesis space within a science problem-solving task was predictive of student science content learning and in-game performance. Students who selected a higher proportion of appropriate hypotheses demonstrated greater learning gains and completed more in-game goals. Students providing correct explanations for hypothesis selection completed more in-game goals; however, providing the correct explanation for hypothesis selection did not account for greater learning gains. From the analysis, we concluded that hypothesis testing strategies play a central role in game-based learning environments that involve problem-solving tasks, thereby demonstrating strong connections to science content learning and in-game performance.

An integrated scenario management strategy

Scenarios have proven effective for eliciting, describing and validating software requirements; however, scenario management continues to be a significant challenge to practitioners. One reason for this difficulty is that the number of possible relations among scenarios grows exponentially with the number of scenarios. If these relations are formalized, they can be more easily identified and supported. To provide this support, we extend the benefits of project-wide glossaries with two complementary approaches. The first approach employs shared scenario elements to identify and maintain common episodes among scenarios. The resulting episodes impose consistency across related scenarios and provide a way to visualize their interdependencies. The second approach quantifies similarity between scenarios. The resulting similarity measures serve as heuristics for finding duplicate scenarios, scenarios needing further elaboration, and scenarios which have not yet been identified yielding valuable information about how well the scenarios provide coverage of the requirements. These two approaches, integrated with a scenario database, project glossaries, configuration management, and coverage analysis, form the basis of a useful and effective strategy for scenario management and evolution.

Modeling learner affect with theoretically grounded dynamic bayesian networks

Evidence of the strong relationship between learning and emotion has fueled recent work in modeling affective states in intelligent tutoring systems. Many of these models are based on general models of affect without a specific focus on learner emotions. This paper presents work that investigates the benefits of using theoretical models of learner emotions to guide the development of Bayesian networks for prediction of student affect. Predictive models are empirically learned from data acquired from 260 students interacting with the game-based learning environment, CRYSTAL ISLAND. Results indicate the benefits of using theoretical models of learner emotions to inform predictive models. The most successful model, a dynamic Bayesian network, also highlights the importance of temporal information in predicting learner emotions. This work demonstrates the benefits of basing predictive models of learner emotions on theoretical foundations and has implications for how these models may be used to validate theoretical models of emotion.

Narrative-Centered tutorial planning for inquiry-based learning environments

Recent years have seen growing interest in narrative-centered learning environments. Leveraging the inherent structure of narrative, narrative-centered learning environments offer significant potential for inquiry-based learning in which students actively participate in engaging story-based problem-solving. A key challenge posed by narrative-centered learning is orchestrating all of the events in the unfolding story to motivate students and promote effective learning. In this paper we present a narrative-centered tutorial planning architecture that integrates narrative planning and pedagogical control. The architecture continually constructs and updates narrative plans to support the hypothesis-generation-testing cycles that form the basis for inquiry-based learning. It is being used to implement a prototype narrative-centered inquiry-based learning environment for the domain of microbiology. The planner dynamically balances narrative and pedagogical goals while at the same time satisfying the real-time constraints of highly interactive learning environments.

Designing game-based learning environments for elementary science education: A narrative-centered learning perspective

Game-based learning environments hold significant promise for STEM education, yet they are enormously complex. Crystal Island: Uncharted Discovery, is a game-based learning environment designed for upper elementary science education that has been under development in our laboratory for the past four years. This article discusses curricular and narrative interaction design requirements, presents the design of the Crystal Island learning environment, and describes its evolution through a series of pilots and field tests. Additionally, a classroom integration study was conducted to initiate a shift towards ecological validity. Results indicated that Crystal Island produced significant learning gains on both science content and problem-solving measures. Importantly, gains were consistent for gender across studies. This finding is key in light of past studies that revealed disproportionate participation by boys within game-based learning environments.

Individual differences in gameplay and learning: a narrative-centered learning perspective

Narrative-centered learning environments are an important class of educational games that situate learning within rich story contexts. The work presented in this paper investigates individual differences in gameplay and learning during student interactions with a narrative-centered learning environment, Crystal Island. Findings reveal striking differences between high- and low-achieving science students in problem-solving effectiveness, attention to particular gameplay elements, learning gains and engagement ratings. High-achieving science students tended to demonstrate greater problem-solving efficiency, reported higher levels of interest and presence in the narrative environment, and demonstrated an increased focus on information gathering and information organization gameplay activities. Lower-achieving microbiology students gravitated toward novel gameplay elements, such as conversations with non-player characters and the use of laboratory testing equipment. The findings have implications for the design of broadly effective gameplay activities for narrative-centered learning environments, as well as investigations of scaffolding techniques to promote effective problem solving, improved learning outcomes and sustained engagement for all students.

Dialogue Management for Conversational Case-Based Reasoning

Two key objectives of conversational case-based reasoning (CCBR) systems are (1) eliciting case facts in a manner that minimizes the user’s burden in terms of resources such as time, information cost, and cognitive load, and (2) integrating CBR with other problem solving modalities. This paper proposes an architecture that addresses both these goals by integrating CBR with a discourse-oriented dialogue engine. The dialogue engine determines when CBR or other problem-solving techniques are needed to achieve pending discourse goals. Conversely, the CBR component has the full resources of a dialogue engine to handle topic changes, interruptions, clarification questions by either the user or the system, and other speech acts that arise in problem-solving dialogues.

Understanding and Predicting Student Self-Regulated Learning Strategies in Game-Based Learning Environments

Self-regulated learning behaviors such as goal setting and monitoring have been found to be crucial to students’ success in computer-based learning environments. Consequently, understanding students’ self-regulated learning behavior has been the subject of increasing attention. Unfortunately, monitoring these behaviors in real-time has proven challenging. This paper presents an initial investigation into self-regulated learning in a game-based learning environment. Evidence of goal setting and monitoring behaviors is examined through students’ text-based responses to update their ‘status’ in an in-game social network. Students are then classified into SRL-use categories. This article describes the methodology used to classify students and discusses analyses demonstrating the learning and gameplay behaviors across students in different SRL-use categories. Finally, machine learning models capable of predicting these classes early in students’ interaction are presented.