Abigail S. Gertner

Using Bayesian Networks to Manage Uncertainty in Student Modeling

When a tutoring system aims to provide students with interactive help, it needs to know what knowledge the student has and what goals the student is currently trying to achieve. That is, it must do both assessment and plan recognition. These modeling tasks involve a high level of uncertainty when students are allowed to follow various lines of reasoning and are not required to show all their reasoning explicitly. We use Bayesian networks as a comprehensive, sound formalism to handle this uncertainty. Using Bayesian networks, we have devised the probabilistic student models for Andes, a tutoring system for Newtonian physics whose philosophy is to maximize student initiative and freedom during the pedagogical interaction. Andes’ models provide long-term knowledge assessment, plan recognition, and prediction of students’ actions during problem solving, as well as assessment of students’ knowledge and understanding as students read and explain worked out examples. In this paper, we describe the basic mechanisms that allow Andes’ student models to soundly perform assessment and plan recognition, as well as the Bayesian network solutions to issues that arose in scaling up the model to a full-scale, field evaluated application. We also summarize the results of several evaluations of Andes which provide evidence on the accuracy of its student models.

Student Modeling from Conversational Test Data: A Bayesian Approach Without Priors

Although conventional tests are often used for determining a students overall competence, they are seldom used for determining a finegrained model. However, this problem does arise occasionally, such as when a conventional test is used to initialize the student model of an ITS. Existing psychometric techniques for solving this problem are intractable. Straightforward Bayesian techniques are also inapplicable because they depend too strongly on the priors, which are often not available. Our solution is to base the assessment on the difference between the prior and posterior probabilities. If the test data raise the posterior probability of mastery of a piece of knowledge even slightly above its prior probability, then that is interpreted as evidence that the student has mastered that piece of knowledge. Evaluation of this technique with artificial students indicates that it can deliver highly accurate assessments.

Andes: A Coached Problem Solving Environment for Physics

Andes is an Intelligent Tutoring System for introductory college physics. The fundamental principles underlying the design of Andes are: (1) encourage the student to construct new knowledge by providing hints that require them to derive most of the solution on their own, (2) facilitate transfer from the system by making the interface as much like a piece of paper as possible, (3) give immediate feedback after each action to maximize the opportunities for learning and minimize the amount of time spent going down wrong paths, and (4) give the student flexibility in the order in which actions are performed, and allow them to skip steps when appropriate. This paper gives an overview of Andes, focusing on the overall architecture and the students experience using the system.

TraumaTIQ: online decision support for trauma management

TraumAID, a rule-based expert system combined with a planner, helps physicians manage patients with severe injuries. To improve communication of TraumAIDs plans to physicians, the authors created TraumTIQ (Trauma criTIQuing), an interface that examines and critiques a physicians intended actions.

Collaborative Discourse Theory as a Foundation for Tutorial Dialogue

Research on intelligent tutoring systems has not leveraged general models of collaborative discourse, even though tutoring is inherently collaborative. Similarly, research on collaborative discourse theory has rarely addressed tutorial issues, even though teaching and learning are important components of collaboration. We help bridge the gap between these two related research threads by presenting a tutorial agent, called Paco, that we built using a domain-independent collaboration manager, called Collagen. Our primary contribution is to show how a variety of tutorial behaviors can be expressed as rules for generating candidate discourse acts in the framework of collaborative discourse theory.

Providing Feedback to Equation Entries in an Intelligent Tutoring System for Physics

Andes, an intelligent tutoring system for Newtonian physics, provides an environment for students to solve quantitative physics problems. Andes provides immediate correct/incorrect feedback to each student entry during problem solving. When a student enters an equation, Andes must (1) determine quickly whether that equation is correct, and (2) provide helpful feedback indicating what is wrong with the students entry. To address the former, we match student equations against a pregenerated list of correct equations. To address the latter, we use the pre-generated equations to infer what equation the student may have been trying to enter, and generate hints based on the discrepancies. This paper describes the representation of equations and the procedures Andes uses to perform these tasks.

Incorporating tutorial strategies into an intelligent assistant

Computer tutors and intelligent software assistants have traditionally been thought of as different kinds of systems. However tutors and assistants share many properties. We have incorporated tutorial strategies into an intelligent assistant based on the COLLAGEN architecture. We are working on an agent, named Triton, which teaches and helps users with the graphical user interface of an air travel planning system. We found that the collaborative model underlying COLLAGEN is an excellent foundation for both an assistant and a tutor, and that both modes of interaction can be implemented in the same system with different parameter settings.

Plan Recognition and Evaluation for On-line Critiquing

The Traum-AID system is a tool for assisting physicians during the management of patients with severe injuries. Originally, Traum-AID was conceived as a rule-based expert system combined with a planner. After this architecture had been implemented, we began to face the issue of how Traum-AID could communicate its plans to physicians in order to influence their behavior and have a positive effect on patient outcome. This paper describes Trauma-TIQ – the critiquing interface for Traum-AID – which examines the actions the physician intends to carry out and produces a critique in response to those intentions. Trauma-TIQs two main components are a plan recognizer that uses the context of the case to disambiguate plans, and a plan evaluator that identifies errors and calculates their significance in order to determine an appropriate response. Unlike previously developed reminder systems, Trauma-TIQ evaluates the physicians proposed plan and attempts to intervene before problems occur. And unlike previous critiquing systems, it is able to provide ongoing decision support during the planning and delivery of care. In the context of time-critical patient management it is, therefore, a more appropriate means of interaction.

Dialogue complexity with portability?: research directions for the information state approach

We review existing types of dialogue managers (DMs), and propose that the Information State (IS) approach may allow both complexity of dialogue and ease of portability. We discuss implementational drawbacks of the only existing IS DM, and describe our work underway to develop a new DM resolving those drawbacks.

On-line assurance in the initial definitive management of multiple trauma: evaluating system potential

The TraumAID system has been designed to provide on-line decision support throughout the initial definitive management of injured patients. Here we describe its retrospective evaluation and the use we subsequently made of judges comments on the validation data to evaluate TraumaTIQ, a new critiquing interface for TraumAID, investigating the question of whether, with timely recording of information, a system could produce commentary in line with that of human experts. Our results show that (1) comparable commentary can be produced, and (2) validation studies, which take great time and effort to conduct, can produce useful data beyond their original design goals.