Kenneth R. Koedinger

An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor

Recent studies have shown that self-explanation is an effective metacognitive strategy, but how can it be leveraged to improve students’ learning in actual classrooms? How do instructional treatments that emphasizes self-explanation affect students’ learning, as compared to other instructional treatments? We investigated whether self-explanation can be scaffolded effectively in a classroom environment using a Cognitive Tutor, which is intelligent instructional software that supports guided learning by doing. In two classroom experiments, we found that students who explained their steps during problem-solving practice with a Cognitive Tutor learned with greater understanding compared to students who did not explain steps. The explainers better explained their solutions steps and were more successful on transfer problems. We interpret these results as follows: By engaging in explanation, students acquired better-integrated visual and verbal declarative knowledge and acquired less shallow procedural knowledge. The research demonstrates that the benefits of self-explanation can be achieved in a relatively simple computer-based approach that scales well for classroom use.

The knowledge-learning-instruction framework: bridging the science-practice chasm to enhance robust student learning.

Despite the accumulation of substantial cognitive science research relevant to education, there remains confusion and controversy in the application of research to educational practice. In support of a more systematic approach, we describe the Knowledge-Learning-Instruction (KLI) framework. KLI promotes the emergence of instructional principles of high potential for generality, while explicitly identifying constraints of and opportunities for detailed analysis of the knowledge students may acquire in courses. Drawing on research across domains of science, math, and language learning, we illustrate the analyses of knowledge, learning, and instructional events that the KLI framework affords. We present a set of three coordinated taxonomies of knowledge, learning, and instruction. For example, we identify three broad classes of learning events (LEs): (a) memory and fluency processes, (b) induction and refinement processes, and (c) understanding and sense-making processes, and we show how these can lead to different knowledge changes and constraints on optimal instructional choices.

Cognitive Tutor: Applied research in mathematics education

For 25 years, we have been working to build cognitive models of mathematics, which have become a basis for middle- and high-school curricula. We discuss the theoretical background of this approach and evidence that the resulting curricula are more effective than other approaches to instruction. We also discuss how embedding a well specified theory in our instructional software allows us to dynamically evaluate the effectiveness of our instruction at a more detailed level than was previously possible. The current widespread use of the software is allowing us to test hypotheses across large numbers of students. We believe that this will lead to new approaches both to understanding mathematical cognition and to improving instruction.

Learning factors analysis – a general method for cognitive model evaluation and improvement

A cognitive model is a set of production rules or skills encoded in intelligent tutors to model how students solve problems. It is usually generated by brainstorming and iterative refinement between subject experts, cognitive scientists and programmers. In this paper we propose a semi-automated method for improving a cognitive model called Learning Factors Analysis that combines a statistical model, human expertise and a combinatorial search. We use this method to evaluate an existing cognitive model and to generate and evaluate alternative models. We present improved cognitive models and make suggestions for improving the intelligent tutor based on those models.

Predictive human performance modeling made easy

Although engineering models of user behavior have enjoyed a rich history in HCI, they have yet to have a widespread impact due to the complexities of the modeling process. In this paper we describe a development system in which designers generate predictive cognitive models of user behavior simply by demonstrating tasks on HTML mock-ups of new interfaces. Keystroke-Level Models are produced automatically using new rules for placing mental operators, then implemented in the ACT-R cognitive architecture. They interact with the mock-up through integrated perceptual and motor modules, generating behavior that is automatically quantified and easily examined. Using a query-entry user interface as an example [19], we demonstrate that this new system enables more rapid development of predictive models, with more accurate results, than previously published models of these tasks.

Adapting to when students game an intelligent tutoring system

It has been found in recent years that many students who use intelligent tutoring systems game the system, attempting to succeed in the educational environment by exploiting properties of the system rather than by learning the material and trying to use that knowledge to answer correctly. In this paper, we introduce a system which gives a gaming student supplementary exercises focused on exactly the material the student bypassed by gaming, and which also expresses negative emotion to gaming students through an animated agent. Students using this system engage in less gaming, and students who receive many supplemental exercises have considerably better learning than is associated with gaming in the control condition or prior studies.

Teachers' and Researchers' Beliefs About the Development of Algebraic Reasoning

Mathematics teachers and educational researchers ordered arithmetic and algebra problems according to their predicted problem-solving difficulty for students. Predictions deviated systematically from algebra students’ performances but closely matched a view implicit in textbooks. Analysis of students’ problem-solving strategies indicates specific ways that students’ algebraic reasoning differs from that predicted by most teachers and researchers in the sample and portrayed in common textbooks. The Symbol Precedence Model of development of algebraic reasoning, in which symbolic problem solving precedes verbal problem solving and arithmetic skills strictly precede algebraic skills, was contrasted with the Verbal Precedence Model of development, which provided a better quantitative fit of students’ performance data. Implications of the findings for student and teacher cognition and for algebra instruction are discussed.

The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains

Intelligent Tutoring Systems have been shown to be effective in a number of domains, but they remain hard to build, with estimates of 200-300 hours of development per hour of instruction. Two goals of the Cognitive Tutor Authoring Tools (CTAT) project are to (a) make tutor development more efficient for both programmers and non-programmers and (b) produce scientific evidence indicating which tool features lead to improved efficiency. CTAT supports development of two types of tutors, Cognitive Tutors and Example-Tracing Tutors, which represent different trade-offs in terms of ease of authoring and generality. In preliminary small-scale controlled experiments involving basic Cognitive Tutor development tasks, we found efficiency gains due to CTAT of 1.4 to 2 times faster. We expect that continued development of CTAT, informed by repeated evaluations involving increasingly complex authoring tasks, will lead to further efficiency gains.

A comparative analysis of cognitive tutoring and constraint-based modeling

Numerous approaches to student modeling have been proposed since the inception of the field more than three decades ago. What the field is lacking completely is comparative analyses of different student modeling approaches. In this paper we compare Cognitive Tutoring to Constraint-Based Modeling (CBM). We present our experiences in implementing a database design tutor using both methodologies and highlight their strengths and weaknesses. We compare their characteristics and argue the differences are often more apparent than real: for specific domains one approach may be favoured over the other, making them viable complementary methods for supporting learning.

Developing a generalizable detector of when students game the system

Some students, when working in interactive learning environments, attempt to “game the system”, attempting to succeed in the environment by exploiting properties of the system rather than by learning the material and trying to use that knowledge to answer correctly. In this paper, we present a system that can accurately detect whether a student is gaming the system, within a Cognitive Tutor mathematics curricula. Our detector also distinguishes between two distinct types of gaming which are associated with different learning outcomes. We explore this detector’s generalizability, and find that it transfers successfully to both new students and new tutor lessons.