Brett van de Sande

An analysis of students' gaming behaviors in an intelligent tutoring system: predictors and impacts

Students who exploit properties of an instructional system to make progress while avoiding learning are said to be “gaming” the system. In order to investigate what causes gaming and how it impacts students, we analyzed log data from two Intelligent Tutoring Systems (ITS). The primary analyses focused on six college physics classes using the Andes ITS for homework and test preparation, starting with the research question: What is a better predictor of gaming, problem or student? To address this question, we developed a computational gaming detector for automatically labeling the Andes data, and applied several data mining techniques, including machine learning of Bayesian network parameters. Contrary to some prior findings, the analyses indicated that student was a better predictor of gaming than problem. This result was surprising, so we tested and confirmed it with log data from a second ITS (the Algebra Cognitive Tutor) and population (high school students). Given that student was more predictive of gaming than problem, subsequent analyses focused on how students gamed and in turn benefited (or not) from instructional features of the environment, as well as how gaming in general influenced problem solving and learning outcomes.

The Andes Physics Tutoring System: An Experiment in Freedom

The Andes physics tutoring system is an experiment in student freedom. It allows students to solve a physics problem in virtually any legal way. This means that Andes must recognize an extremely large number of possible steps occurring in an extraordinarily large number of possible orders. Such freedom raises several research questions. (1) How can Andes solve the technical challenge of understanding student’s behavior in such a wide-open context? (2) How can Andes give pedagogically useful help and guidance? In particular, how can it guide students who are floundering without curtailing the freedom of students who are not floundering? (3) Will Andes be effective in getting students in real classrooms to learn physics? (4) What does it take to scale up Andes and disseminate it widely? The Andes project, which began in the mid 1990’s, has achieved workable solutions to the first three goals: Andes can understand student behavior; It provides pedagogical help similar to that of human experts; Most importantly, Andes causes large, reliable learning gains compared to control classes taught with convention, paper-based instruction. This chapter summarizes the first three results and discusses our progress on the fourth goal, scale-up.

Learning How to Construct Models of Dynamic Systems: An Initial Evaluation of the Dragoon Intelligent Tutoring System

Constructing models of dynamic systems is an important skill in both mathematics and science instruction. However, it has proved difficult to teach. Dragoon is an intelligent tutoring system intended to quickly and effectively teach this important skill. This paper describes Dragoon and an evaluation of it. The evaluation randomly assigned students in a university class to either Dragoon or baseline instruction that used Dragoon as an editor only. Among students who did use their systems, the tutored students scored reliably higher (p < .021, d = 1.06) on the post-test than the students who used only the conventional editor-based instruction.

The design and development of the dragoon intelligent tutoring system for model construction: lessons learned

ABSTRACT This paper describes Dragoon, a simple intelligent tutoring system which teaches the construction of models of dynamic systems. Modelling is one of seven practices dictated in two new sets of educational standards in the U.S.A., and Dragoon is one of the first systems for teaching model construction for dynamic systems. Dragoon can be classified as a step-based tutoring system that uses example-tracing, an explicit pedagogical policy and an open learner model. Dragoon can also be used for computer-supported collaborative learning, and provides tools for classroom orchestration. This paper describes the features, user interfaces, and architecture of Dragoon; compares and contrasts Dragoon with other intelligent tutoring systems; and presents a brief overview of formative and summative evaluations of Dragoon in both high school and college classes. Of four summative evaluations, three found that students who used Dragoon learned more about the target system than students who did equivalent work without Dragoon.

Collaborative Dialog While Studying Worked-out Examples

Self-explaining is a beneficial learning strategy for studying worked-out examples because it either supplies missing information through the generation of inferences or because it provides a mechanism for repairing flawed mental models. Although self-explanation is generated with the purpose of helping the individual, is it also helpful to produce explanations in a collaborative setting? Can individuals help each other infer missing information or repair their flawed mental models collaboratively? To find out, we coded the dialog from dyads collaboratively studying examples and contrasted it with individuals studying examples alone. The results suggest that dyads were more likely to attempt to reconcile the examples with their attempted solutions, and avoid shallow processing of examples through paraphrasing.