Vincent Aleven

An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor

Recent studies have shown that self-explanation is an effective metacognitive strategy, but how can it be leveraged to improve students’ learning in actual classrooms? How do instructional treatments that emphasizes self-explanation affect students’ learning, as compared to other instructional treatments? We investigated whether self-explanation can be scaffolded effectively in a classroom environment using a Cognitive Tutor, which is intelligent instructional software that supports guided learning by doing. In two classroom experiments, we found that students who explained their steps during problem-solving practice with a Cognitive Tutor learned with greater understanding compared to students who did not explain steps. The explainers better explained their solutions steps and were more successful on transfer problems. We interpret these results as follows: By engaging in explanation, students acquired better-integrated visual and verbal declarative knowledge and acquired less shallow procedural knowledge. The research demonstrates that the benefits of self-explanation can be achieved in a relatively simple computer-based approach that scales well for classroom use.

Help Seeking and Help Design in Interactive Learning Environments

Many interactive learning environments (ILEs) offer on-demand help, intended to positively influence learning. Recent studies report evidence that although effective help-seeking behavior in ILEs is related to better learning outcomes, learners are not using help facilities effectively. This selective review (a) examines theoretical perspectives on the role of on-demand help in ILEs, (b) reviews literature on the relations between help seeking and learning in ILEs, and (c) identifies reasons for the lack of effective help use. We review the effect of system-related factors, of student-related factors, and of interactions between these factors. The interaction between metacognitive skills and cognitive factors is important for appropriate help seeking, as are a potentially large space of system-related factors as well as interactions among learner- and system-related factors. We suggest directions for future research.

The Beginning of a Beautiful Friendship? Intelligent Tutoring Systems and MOOCs

A key challenge in ITS research and development is to support tutoring at scale, for example by embedding tutors in MOOCs. An obstacle to at-scale deployment is that ITS architectures tend to be complex, not easily deployed in browsers without significant server-side processing, and not easily embedded in a learning management system (LMS). We present a case study in which a widely used ITS authoring tool suite, CTAT/TutorShop, was modified so that tutors can be embedded in MOOCs. Specifically, the inner loop (the example-tracing tutor engine) was moved to the client by reimplementing it in JavaScript, and the tutors were made compatible with the LTI e-learning standard. The feasibility of this general approach to ITS/MOOC integration was demonstrated with simple tutors in an edX MOOC “Data Analytics and Learning.”

The cognitive tutor authoring tools (CTAT): preliminary evaluation of efficiency gains

Intelligent Tutoring Systems have been shown to be effective in a number of domains, but they remain hard to build, with estimates of 200-300 hours of development per hour of instruction. Two goals of the Cognitive Tutor Authoring Tools (CTAT) project are to (a) make tutor development more efficient for both programmers and non-programmers and (b) produce scientific evidence indicating which tool features lead to improved efficiency. CTAT supports development of two types of tutors, Cognitive Tutors and Example-Tracing Tutors, which represent different trade-offs in terms of ease of authoring and generality. In preliminary small-scale controlled experiments involving basic Cognitive Tutor development tasks, we found efficiency gains due to CTAT of 1.4 to 2 times faster. We expect that continued development of CTAT, informed by repeated evaluations involving increasingly complex authoring tasks, will lead to further efficiency gains.

International Handbook of Metacognition and Learning Technologies

Integrating all aspects of the fields of metacognition and learning technologies, this book describes features of the learning technologies and how they have been designed to study and support metacognitive processing and self-regulated learning.

The worked-example effect: Not an artefact of lousy control conditions

Recently it has been argued that the worked-example effect, as postulated by Cognitive Load Theory, might only occur when compared to unsupported problem-solving, but not when compared to well-supported or tutored problem-solving as instantiated, for example, in Cognitive Tutors. In two experiments, we compared a standard Cognitive Tutor with a version that was enriched with faded worked examples. In Experiment 1, students in the example condition needed less learning time to acquire a comparable amount of procedural skills and conceptual understanding. In Experiment 2, the efficiency advantage was replicated. In addition, students in the example condition acquired a deeper conceptual understanding. The present findings demonstrate that the worked-example effect is indeed robust and can be found even when compared to well-supported learning by problem-solving.

Automated, Unobtrusive, Action-by-Action Assessment of Self-Regulation During Learning With an Intelligent Tutoring System

Assessment of students’ self-regulated learning (SRL) requires a method for evaluating whether observed actions are appropriate acts of self-regulation in the specific learning context in which they occur. We review research that has resulted in an automated method for context-sensitive assessment of a specific SRL strategy, help seeking while working with an intelligent tutoring system. The method relies on a computer-executable model of the targeted SRL strategy. The method was validated by showing that it converges with other measures of help seeking. Automated feedback on help seeking driven by this method led to a lasting improvement in students’ help-seeking behavior, although not in domain-specific learning. The method is unobtrusive, is temporally fine-grained, and can be applied on a large scale and over extended periods. The approach could be applied to other SRL strategies besides help seeking.

Toward a Framework for the Analysis and Design of Educational Games

We describe and illustrate the beginnings of a general framework for the design and analysis of educational games. Our students have used it to analyze existing educational games and to create prototype educational games. The framework is built on existing components: a method for precisely specifying educational objectives, a framework for relating a game’s mechanics, dynamics, and aesthetics, and principles for instructional design grounded in empirical research in the learning sciences. The power of the framework comes from the components themselves, as well as from considering these components in concert and making connections between them. The framework coordinates the many levels at which an educational game must succeed in order to be effective. We illustrate the framework by using it to analyze Zombie Division and to generate some redesign ideas for this game.

Evaluating the Effectiveness of a Tutorial Dialogue System for Self-Explanation

Previous research has shown that self-explanation can be supported effectively in an intelligent tutoring system by simple means such as menus. We now focus on the hypothesis that natural language dialogue is an even more effective way to support self-explanation. We have developed the Geometry Explanation Tutor, which helps students to state explanations of their problem-solving steps in their own words. In a classroom study involving 71 advanced students, we found that students who explained problem-solving steps in a dialogue with the tutor did not learn better overall than students who explained by means of a menu, but did learn better to state explanations. Second, examining a subset of 700 student explanations, students who received higher-quality feedback from the system made greater progress in their dialogues and learned more, providing some measure of confidence that progress is a useful intermediate variable to guide further system development. Finally, students who tended to reference specific problem elements in their explanations, rather than state a general problem-solving principle, had lower learning gains than other students. Such explanations may be indicative of an earlier developmental level.

Worked examples and tutored problem solving: redundant or synergistic forms of support?

The current research investigates a combination of two instructional approaches, tutored problem solving and worked examples. Tutored problem solving with automated tutors has proven to be an effective instructional method. Worked-out examples have been shown to be an effective complement to untutored problem solving, but it is largely unknown whether they are an effective complement to tutored problem solving. Further, while computer-based learning environments offer the possibility of adaptively transitioning from examples to problems while tailoring to an individual learner, the effectiveness of such machine-adapted example fading is largely unstudied. To address these research questions, one lab and one classroom experiment were conducted. Both studies compared a standard Cognitive Tutor with two example-enhanced Cognitive Tutors, in which the fading of worked-out examples occurred either in a fixed way or adaptively. Results indicate that the adaptive fading of worked-out examples leads to higher transfer performance on delayed posttests than the other two methods.