Chun Wai Liew

Reasoning about Systems of Physics Equations

Many problems in introductory Physics require the student to enter a system of algebraic equations as the answer. Tutoring systems must be able to understand the students submission before they can generate useful feedback. This paper presents an approach that accepts from the student a system of equations describing the physics of the problem and checks to see if it is correct. When it is not, the students equation set is analyzed vis-a-vis one or more correct sets of equations, known physics concepts, and algebraic transformations. During this analysis credit-blame assignment is performed to identify one of several types of errors including 1) algebraic errors, 2) one or more omitted physics concepts, 3) incorrect instances of a required physics concept, and 4) use of an inappropriate physics concept. Experimental data collected from an introductory physics class is summarized and discussed vis-a-vis other methods. Results indicate that the techniques applied are effective at localizing most errors but that more work is needed to distinguish between algebraic and conceptual errors.

DETERMINING THE DIMENSIONS OF VARIABLES IN PHYSICS ALGEBRAIC EQUATIONS

This paper describes work on methods that evaluate algebraic solutions to word problems in physics. Many current tutoring systems rely on substantial scaffolding and consequently require students to completely describe every variable used in the solution. A heuristic, based on constraint propagation, capable of inferring the description of variables (i.e., the possible dimensions and physics concepts) is shown to be highly reliable on three real world data sets, one covering a few problems with a small number of student answers and two others covering a large class of problems (~100) with a large number of student answers (~11,000). The heuristic uniquely determines the dimensions of all the variables in 91–92% of the equation sets. By asking the student for dimension information about one variable, an additional 3% of the sets can be determined. An ITS tutoring system can use this heuristic to reason about a students answers even when the scaffolding and context are removed.

Knowledge Based Mechanisms for Tutoring Systems in Science and Engineering

In science and engineering courses, students are often presented a situation for which they are asked to identify the relevant principles and to instantiate them as a set of equations. For an ITS to determine the correctness and relevance of the students answer and generate effective feedback, it must map the student variables and equations onto the physical properties and concepts that are relevant to the situation. The space of possible mappings of variables and equations is extremely large. Domain independent techniques by themselves are unable to overcome the complexity hurdles. This paper describes how an ITS can use constraint propagation and algebraic techniques combined with domain and problem-specific knowledge to solve the mapping problem with systems of algebraic equations. The techniques described in this paper have been implemented in the PHYSICS_TUTOR tutoring system and evaluated on three data sets that contain submissions from students in several introductory Physics courses.

Determining What the Student Understands - Assessment in an Unscaffolded Environment

Assessment of skills and process knowledge is difficult and quite different from assessing knowledge of content. Many assessment systems use either multiple choice questions or other frameworks that provide a significant amount of scaffolding and this can influence the results. One reason for this is that they are easy to administer and the answers can be automatically graded. This paper describes an assessment tool that does not provide scaffolding (and therefore hints) and yet is able to automatically grade the free form answers through the use of domain knowledge heuristics. The tool has been developed for a tutoring system in the domain of red black trees (a data structure in computer science) and has been evaluated on three semesters of students in a computer science course.

Using Student Logs to Build Bayesian Models of Student Knowledge and Skills

Recent works on Intelligent Tutoring Systems have focused on more complicated knowledge domains, which pose challenges in automated assessment of student performance. In particular, while the system can log every user action and keep track of the student’s solution state, it is unable to determine the hidden intermediate steps leading to such state or what the student is trying to achieve. In this paper, we show that this information can be acquired through data mining, along with the type, frequency and context of errors that students made. Our technique has been implemented as part of the student model in a tutor that teaches red-black trees. The system was evaluated on three semesters of student data. Analysis of the results shows that the proposed framework of error analysis can help the system in predicting student performance with good accuracy and the instructor in determining difficulties that students encounter, both individually and collectively as a class.

Assessing Student Answers to Balanced Tree Problems

Problems in the domain of balanced binary tree operations usually involve the students constructing a sequence of transformations to insert or delete a value. An Intelligent Tutoring System (ITS) in this area must be able to perform.