Angela Z. Wagner

Adapting to when students game an intelligent tutoring system

It has been found in recent years that many students who use intelligent tutoring systems game the system, attempting to succeed in the educational environment by exploiting properties of the system rather than by learning the material and trying to use that knowledge to answer correctly. In this paper, we introduce a system which gives a gaming student supplementary exercises focused on exactly the material the student bypassed by gaming, and which also expresses negative emotion to gaming students through an animated agent. Students using this system engage in less gaming, and students who receive many supplemental exercises have considerably better learning than is associated with gaming in the control condition or prior studies.

Contextual slip and prediction of student performance after use of an intelligent tutor

Intelligent tutoring systems that utilize Bayesian Knowledge Tracing have achieved the ability to accurately predict student performance not only within the intelligent tutoring system, but on paper post-tests outside of the system Recent work has suggested that contextual estimation of student guessing and slipping leads to better prediction within the tutoring software (Baker, Corbett, & Aleven, 2008a, 2008b) However, it is not yet clear whether this new variant on knowledge tracing is effective at predicting the latent student knowledge that leads to successful post-test performance In this paper, we compare the Contextual-Guess-and-Slip variant on Bayesian Knowledge Tracing to classical four-parameter Bayesian Knowledge Tracing and the Individual Difference Weights variant of Bayesian Knowledge Tracing (Corbett & Anderson, 1995), investigating how well each model variant predicts post-test performance We also test other ways to utilize contextual estimation of slipping within the tutor in post-test prediction, and discuss hypotheses for why slipping during tutor use is a significant predictor of post-test performance, even after Bayesian Knowledge Tracing estimates are controlled for.

Student Question-Asking Patterns in an Intelligent Algebra Tutor

Cognitive Tutors are proven effective learning environments, but are still not as effective as one-on-one human tutoring. We describe an environment (ALPS) designed to engage students in question-asking during problem solving. ALPS integrates Cognitive Tutors with Synthetic Interview (SI) technology, allowing students to type free-form questions and receive pre-recorded video clip answers. We performed a Wizard-of-Oz study to evaluate the feasibility of ALPS and to design the question-and-answer database for the SI. In the study, a human tutor played the SI’s role, reading the students’ typed questions and answering over an audio/video channel. We examine the rate at which students ask questions, the content of the questions, and the events that stimulate questions. We found that students ask questions in this paradigm at a promising rate, but there is a need for further work in encouraging them to ask deeper questions that may improve knowledge encoding and learning.

Automated detection of proactive remediation by teachers in reasoning mind classrooms

Among the most important tasks of the teacher in a classroom using the Reasoning Mind blended learning system is proactive remediation: dynamically planned interventions conducted by the teacher with one or more students. While there are several examples of detectors of student behavior within an online learning environment, most have focused on behaviors occurring fully within the context of the system, and on student behaviors. In contrast, proactive remediation is a teacher-driven activity that occurs outside of the system, and its occurrence is not necessarily related to the students current task within the Reasoning Mind system. We present a sensor-free detector of proactive remediation, which is able to distinguish these activities from other behaviors involving idle time, such as on-task conversation related to immediate learning activities and off-task behavior.

Estimating Individual Differences for Student Modeling in Intelligent Tutors from Reading and Pretest Data

Past studies have shown that Bayesian Knowledge Tracing BKT can predict student performance and implement Cognitive Mastery successfully. Standard BKT individualizes parameter estimates for skills, also referred to as knowledge components KCs, but not for students. Studies deriving individual student parameters from the data logs of student tutor performance have shown improvements to the standard BKT model fits, and result in different practice recommendations for students. This study investigates whether individual student parameters, specifically individual difference weights IDWs [1], can be derived from student activities prior to tutor use. We find that student performance measures in reading instructional text and in a conceptual knowledge pretest can be employed to predict IDWs. Further, we find that a model incorporating these predicted IDWs performs well, in terms of model fit and learning efficiency, when compared to a standard BKT model and a model with best-fitting IDWs derived from tutor performance.

Differential Impact of Learning Activities Designed to Support Robust Learning in the Genetics Cognitive Tutor

This paper describes two types of Conceptually Grounded Learning Activities designed to foster more robust learning in the Genetics Cognitive Tutor: interleaved worked examples and genetic-process reasoning scaffolds. We report three empirical studies that evaluate the impact of these learning activities on three diverse genetics problem-solving topics in the tutor. We found that interleaved worked examples yielded less basic-skill learning than conventional problem solving, unlike many prior ITS studies of worked examples. We also found preliminary evidence that scaffolded reasoning tasks in conjunction with conventional problem solving leads to more robust understanding than conventional problem solving alone. Implications for the use of contextually grounded learning activities are discussed.

Modeling Students' Natural Language Explanations

Intelligent tutoring systems have achieved demonstrable success in supporting formal problem solving. More recently such systems have begun incorporating student explanations of problem solutions. Typically, these natural language explanations are entered with menus, but some ITSs accept open-ended typed inputs. Typed inputs require more work by both developers and students and evaluations of the added value for learning outcomes has been mixed. This paper examines whether typed input can yield more accurate student modeling than menu-based input. This paper examines the application of Knowledge Tracing student modeling to natural language inputs and examines the standard Knowledge Tracing definition of errors. The analyses indicate that typed explanations can yield more predictive models of student test performance than menu-based explanations and that focusing on semantic errors can further improve predictive accuracy.

Exploring Learner Model Differences Between Students

Bayesian Knowledge Tracing (BKT) has been employed successfully in intelligent learning environments to individualize curriculum sequencing and help messages. Standard BKT employs four parameters, which are estimated separately for individual knowledge components, but not for individual students. Studies have shown that individualizing the parameter estimates for students based on existing data logs improves goodness of fit and leads to substantially different practice recommendations. This study investigates how well BKT parameters in a tutor lesson can be individualized ahead of time, based on learners’ prior activities, including reading text and completing prior tutor lessons. We find that directly applying best-fitting individualized parameter estimates from prior tutor lessons does not appreciably improve BKT goodness of fit for a later tutor lesson, but that individual differences in the later lesson can be effectively predicted from measures of learners’ behaviors in reading text and in completing the prior tutor lessons.

The Social Role of Technical Personnel in the Deployment of Intelligent Tutoring Systems

We present a model – developed using Contextual Inquiry – of how prototype intelligent tutors are deployed into classrooms, focusing on how field technical personnel can serve as vital conduits for information and negotiation between ITS researchers and school personnel such as teachers and principals.