Michael Lipschultz

Pilot Test of a Natural-Language Tutoring System for Physics That Simulates the Highly Interactive Nature of Human Tutoring

This poster describes Rimac, a natural-language tutoring system that engages students in dialogues that address physics concepts and principles, after students have solved quantitative physics problems. We summarize our approach to deriving decision rules that simulate the highly interactive nature of human tutoring, and describe a pilot test that compares two versions of Rimac: an experimental version that deliberately executes these decision rules within a Knowledge Construction Dialogue (KCD) framework, and a control KCD system that does not intentionally execute these rules.

Correcting scientific knowledge in a general-purpose ontology

General-purpose ontologies (e.g WordNet) are convenient, but they are not always scientifically valid We draw on techniques from semantic class learning to improve the scientific validity of WordNets physics forces hyponym (IS-A) hierarchy for use in an intelligent tutoring system We demonstrate the promise of a web-based approach which gathers web statistics used to relabel the forces as scientifically valid or scientifically invalid Our results greatly improve the F1 for predicting scientific invalidity, with small improvements in F1 for predicting scientific validity and in overall accuracy compared to the WordNet baseline.

Modeling Student Benefit from Illustrations and Graphs

We examine a corpus of physics tutorial dialogues between a computer tutor and students. Either graphs or illustrations were displayed during the dialogues. In this work, stepwise linear regression, augmented to remove unwanted terms, is used to build models that identify situations when each graphic may aid learning. Our experimental results show that grouping students by pretest score, then by gender produces a model that significantly outperforms the baseline.

Illustrations or Graphs: Some Students Benefit from One over the Other

We examine whether there are differences between students regarding the utility of learning from visual representations (illustrations or graphs) within the context of a typed natural language-based conceptual physics tutoring system. Showing half of the students only illustrations and the other half only graphs, we found that novices benefited from illustrations, whereas non-novices showed no difference.

Interactive Event: The Rimac Tutor - A Simulation of the Highly Interactive Nature of Human Tutorial Dialogue

Rimac is a natural-language intelligent tutoring system that engages students in dialogues that address physics concepts and principles, after they have solved quantitative physics problems. Much research has been devoted to identifying features of tutorial dialogue that can explain its effectiveness (e.g., [1]), so that these features can be simulated in natural-language tutoring systems. One hypothesis is that the highly interactive nature of tutoring itself promotes learning. Several studies indicate that our understanding of interactivty needs refinement because it cannot be defined simply by the amount of interaction nor the granularity of the interaction but must also take into consideration how well the interaction is carried out (e.g., [2]). This need for refinement suggests that we should more closely examine the linguistic mechanisms evident in tutorial dialogue. Towards this end, we first identified which of a subset of co-constructed discourse relations correlate with learning and operationalized our findings with a set of nine decision rules which we implemented in Rimac [3]. To test for causality, we are conducting pilot tests that compare learning outcomes for two versions of Rimac: an experimental version that deliberately executes the nine decision rules within a Knowledge Construction Dialogue (KCD) framework, and a control KCD system that does not intentionally execute these rules. In this interactive demo, participants will experience the two versions of the system that students have been using in high school classrooms during pilot testing. Students first take a pre-test, and then complete a homework assignment in which they solve four quantitative physics problems. In a subsequent class, they then use the Rimac system and finally during the next class meeting take a post-test. When working with the Rimac system, students are asked to first view a brief video that describes how to solve a homework problem and then are engaged in a reflective dialogue about that problem. See [4] for a more detailed description of the pilot study and planned analyses. Demo participants will have the opportunity to experience exactly what the students experience when working with Rimac. They will see the video and engage in a reflective dialogue about that problem with the highly interactive

Predicting semantic changes in abstraction in tutor responses to students

Post-problem reflective tutorial dialogues between human tutors and students are examined to predict when the tutor changed the level of abstraction from the students preceding turn (i.e., used more general terms or more specific terms); such changes correlate with learning. Prior work examined lexical changes in abstraction. In this work, we consider semantic changes. Since we are interested in developing a fully-automatic computer-based tutor, we use only automatically-extractable features (e.g., percent of domain words in student turn) or features available in a tutoring system (e.g., correctness). We find patterns that predict tutor changes in abstraction better than a majority class baseline. Generalisation is best-predicted using student and reflection features. Specification is best-predicted using student and problem features.