Benjamin A. MacLaren

Contextual slip and prediction of student performance after use of an intelligent tutor

Intelligent tutoring systems that utilize Bayesian Knowledge Tracing have achieved the ability to accurately predict student performance not only within the intelligent tutoring system, but on paper post-tests outside of the system Recent work has suggested that contextual estimation of student guessing and slipping leads to better prediction within the tutoring software (Baker, Corbett, & Aleven, 2008a, 2008b) However, it is not yet clear whether this new variant on knowledge tracing is effective at predicting the latent student knowledge that leads to successful post-test performance In this paper, we compare the Contextual-Guess-and-Slip variant on Bayesian Knowledge Tracing to classical four-parameter Bayesian Knowledge Tracing and the Individual Difference Weights variant of Bayesian Knowledge Tracing (Corbett & Anderson, 1995), investigating how well each model variant predicts post-test performance We also test other ways to utilize contextual estimation of slipping within the tutor in post-test prediction, and discuss hypotheses for why slipping during tutor use is a significant predictor of post-test performance, even after Bayesian Knowledge Tracing estimates are controlled for.

When and Why Does Mastery Learning Work: Instructional Experiments with ACT-R SimStudents

Research in machine learning is making it possible for instructional developers to perform formative evaluations of different curricula using simulated students (VanLehn, Ohlsson & Nason, 1993). Experiments using simulated students can help clarify issues of instructional design, such as when a complex skill can be better learned by being broken into components. This paper describes two formative evaluations using simulated students that shed light on the potential benefits and limitations of mastery learning. Using an ACT-R based cognitive model (Anderson & Lebiere, 1998) we show that while mastery learning can contribute to success in some cases (Corbett & Anderson, 1995), it may actually impede learning in others. Mastery learning was crucial to learning success in an experiment comparing a traditional early algebra curriculum to a novel one presenting verbal problems first. However, in a second experiment, an instructional manipulation that contradicts mastery learning led to greater success than one consistent with it. In that experiment learning was better when more difficult problems were inserted earlier in the instructional sequence. Such problems are more difficult not because they have more components but because they cannot be successfully solved using shallow procedures that work on easier problems.

Estimating Individual Differences for Student Modeling in Intelligent Tutors from Reading and Pretest Data

Past studies have shown that Bayesian Knowledge Tracing BKT can predict student performance and implement Cognitive Mastery successfully. Standard BKT individualizes parameter estimates for skills, also referred to as knowledge components KCs, but not for students. Studies deriving individual student parameters from the data logs of student tutor performance have shown improvements to the standard BKT model fits, and result in different practice recommendations for students. This study investigates whether individual student parameters, specifically individual difference weights IDWs [1], can be derived from student activities prior to tutor use. We find that student performance measures in reading instructional text and in a conceptual knowledge pretest can be employed to predict IDWs. Further, we find that a model incorporating these predicted IDWs performs well, in terms of model fit and learning efficiency, when compared to a standard BKT model and a model with best-fitting IDWs derived from tutor performance.

Differential Impact of Learning Activities Designed to Support Robust Learning in the Genetics Cognitive Tutor

This paper describes two types of Conceptually Grounded Learning Activities designed to foster more robust learning in the Genetics Cognitive Tutor: interleaved worked examples and genetic-process reasoning scaffolds. We report three empirical studies that evaluate the impact of these learning activities on three diverse genetics problem-solving topics in the tutor. We found that interleaved worked examples yielded less basic-skill learning than conventional problem solving, unlike many prior ITS studies of worked examples. We also found preliminary evidence that scaffolded reasoning tasks in conjunction with conventional problem solving leads to more robust understanding than conventional problem solving alone. Implications for the use of contextually grounded learning activities are discussed.

Exploring Learner Model Differences Between Students

Bayesian Knowledge Tracing (BKT) has been employed successfully in intelligent learning environments to individualize curriculum sequencing and help messages. Standard BKT employs four parameters, which are estimated separately for individual knowledge components, but not for individual students. Studies have shown that individualizing the parameter estimates for students based on existing data logs improves goodness of fit and leads to substantially different practice recommendations. This study investigates how well BKT parameters in a tutor lesson can be individualized ahead of time, based on learners’ prior activities, including reading text and completing prior tutor lessons. We find that directly applying best-fitting individualized parameter estimates from prior tutor lessons does not appreciably improve BKT goodness of fit for a later tutor lesson, but that individual differences in the later lesson can be effectively predicted from measures of learners’ behaviors in reading text and in completing the prior tutor lessons.