Pamela W. Jordan

The Architecture of Why2-Atlas: A Coach for Qualitative Physics Essay Writing

The Why2-Atlas system teaches qualitative physics by having students write paragraph-long explanations of simple mechanical phenomena. The tutor uses deep syntactic analysis and abductive theorem proving to convert the students essay to a proof. The proof formalizes not only what was said, but the likely beliefs behind what was said. This allows the tutor to uncover misconceptions as well as to detect missing correct parts of the explanation. If the tutor finds such a flaw in the essay, it conducts a dialogue intended to remedy the missing or misconceived beliefs, then asks the student to correct the essay. It often takes several iterations of essay correction and dialogue to get the student to produce an acceptable explanation. Pilot subjects have been run, and an evaluation is in progress. After explaining the research questions that the system addresses, the bulk of the paper describes the systems architecture and operation.

Learning Content Selection Rules for Generating Object Descriptions in Dialogue

A fundamental requirement of any task-oriented dialogue system is the ability to generate object descriptions that refer to objects in the task domain. The subproblem of content selection for object descriptions in task-oriented dialogue has been the focus of much previous work and a large number of models have been proposed. In this paper, we use the annotated COCONUT corpus of task-oriented design dialogues to develop feature sets based on Dale and Reiters (1995) incremental model, Brennan and Clarks (1996) conceptual pact model, and Jordans (2000b) intentional influences model, and use these feature sets in a machine learning experiment to automatically learn a model of content selection for object descriptions. Since Dale and Reiters model requires a representation of discourse structure, the corpus annotations are used to derive a representation based on Grosz and Sidners (1986) theory of the intentional structure of discourse, as well as two very simple representations of discourse structure based purely on recency. We then apply the rule-induction program RIPPER to train and test the content selection component of an object description generator on a set of 393 object descriptions from the corpus. To our knowledge, this is the first reported experiment of a trainable content selection component for object description generation in dialogue. Three separate content selection models that are based on the three theoretical models, all independently achieve accuracies significantly above the MAJORITY CLASS baseline (17%) on unseen test data, with the intentional influences model (42.4%) performing significantly better than either the incremental model (30.4%) or the conceptual pact model (28.9%). But the best performing models combine all the feature sets, achieving accuracies near 60%. Surprisingly, a simple recency-based representation of discourse structure does as well as one based on intentional structure. To our knowledge, this is also the first empirical comparison of a representation of Grosz and Sidners model of discourse structure with a simpler model for any generation task.

A Survey of Current Paradigms in Machine Translation

Abstract This paper is a survey of the current machine translation research in the US, Europe and Japan. A short history of machine translation is presented first, followed by an overview of the current research work. Representative examples of a wide range of different approaches adopted by machine translation researchers are presented. These are described in detail along with a discussion of the practicalities of scaling up these approaches for operational environments. In support of this discussion, issues in, and techniques for, evaluating machine translation systems are addressed.

The agreement process

In this paper, we investigate the empirical correlates of the agreement process. Informally, the agreement process is the dialog process by which collaborators achieve joint commitment on a joint action. We propose a specific instantiation of the agreement process, derived from our theoretical model, that integrates the IRMA framework for rational problem solving (Bratman, Israel & Pollack, 1988) with Clarks (1992, 1996) work on language as a collaborative activity; and from the characteristics of our task, a simple design problem (furnishing a two-room apartment) in which knowledge is equally distributed among agents, and needs to be shared. The main contribution of our paper is an empirical study of some of the components of the agreement process. We first discuss why we believe the findings from our corpus of computer-mediated dialogs are applicable to human?human collaborative dialogs in general. We then present our theoretical model, and apply it to make predictions about the components of the agreement process. We focus on how information is exchanged in order to arrive at a proposal, and on what constitutes a proposal and its acceptance/rejection. Our corpus study makes use of features of both the dialog and the domain reasoning situation, and led us to discover that the notion of commitment is more useful to model the agreement process than that of acceptance/rejection, as it more closely relates to the unfolding of negotiation.

Empirically evaluating the application of reinforcement learning to the induction of effective and adaptive pedagogical strategies

For many forms of e-learning environments, the system’s behavior can be viewed as a sequential decision process wherein, at each discrete step, the system is responsible for selecting the next action to take. Pedagogical strategies are policies to decide the next system action when there are multiple ones available. In this project we present a Reinforcement Learning (RL) approach for inducing effective pedagogical strategies and empirical evaluations of the induced strategies. This paper addresses the technical challenges in applying RL to Cordillera, a Natural Language Tutoring System teaching students introductory college physics. The algorithm chosen for this project is a model-based RL approach, Policy Iteration, and the training corpus for the RL approach is an exploratory corpus, which was collected by letting the system make random decisions when interacting with real students. Overall, our results show that by using a rather small training corpus, the RL-induced strategies indeed measurably improved the effectiveness of Cordillera in that the RL-induced policies improved students’ learning gains significantly.

An Empirical Investigation of Proposals in Collaborative Dialogues

We describe a corpus-based investigation of proposals in dialogue. First, we describe our DRI compliant coding scheme and report our inter-coder reliability results. Next, we test several hypotheses about what constitutes a well-formed proposal.

Combining Competing Language Understanding Approaches in an Intelligent Tutoring System

When implementing a tutoring system that attempts a deep understanding of students’ natural language explanations, there are three basic approaches to choose between; symbolic, in which sentence strings are parsed using a lexicon and grammar; statistical, in which a corpus is used to train a text classifier; and hybrid, in which rich, symbolically produced features supplement statistical training. Because each type of approach requires different amounts of domain knowledge preparation and provides different quality output for the same input, we describe a method for heuristically combining multiple natural language understanding approaches in an attempt to use each to its best advantage. We explore two basic models for combining approaches in the context of a tutoring system; one where heuristics select the first satisficing representation and another in which heuristics select the highest ranked representation.

Learning attribute selections for non-pronominal expressions

A fundamental function of any task-oriented dialogue system is the ability to generate nominal expressions that describe objects in the task domain. In this paper, we report results from using machine learning to train and test a nominal-expression generator on a set of 393 nominal descriptions from the COCONUT corpus of task-oriented design dialogues. Results show that we can achieve a 50% match to human performance as opposed to a 16% baseline for just guessing the most frequent type of nominal expression in the COCONUT corpus. To our surprise our results indicate that many of the central features of previously proposed selection models did not improve the performance of the learned nominal-expression generator.

Software storming: combining rapid prototyping and knowledge engineering

A method for rapidly producing highly functional prototypes is described. This method, called software storming, involves experts in the initial design and implementation of a system during an intense development effort that combines knowledge engineering with the latest advances in software-development technology and workstation hardware The experiment described is a test of tools and techniques developed in the field of artificial intelligence as well as the first step in the development of software storming. Using this approach, the authors developed a software prototype which possesses significantly more functionality than a standard prototype, in less than four months.<<ETX>>

Coping with ambiguity in a large-scale machine translation system

In an interlingual knowledge-based machine translation system, ambignuity arises when the source language analyzer produces more than one interlingua expression for a source sentence. This can have a negative impact on translation quality, since a target sentence may be produced from an unintended meaning. In this paper we describe the methods used in the KANT machine translation system to reduce or eliminate ambiguity in a large-scale application domain. We also test these methods on a large corpus of test sentences, in order to illustrate how the different disambiguation methods reduce the average number of parses per sentence.