Gabriel Illouz

Follow-up question handling in the imix and ritel systems: A comparative study

One of the basic topics of question answering (QA) dialogue systems is how follow-up questions should be interpreted by a QA system. In this paper, we shall discuss our experience with the IMIX and Ritel systems, for both of which a follow-up question handling scheme has been developed, and corpora have been collected. These two systems are each others opposites in many respects: IMIX is multimodal, non-factoid, black-box QA, while Ritel is speech, factoid, keyword-based QA. Nevertheless, we will show that they are quite comparable, and that it is fruitful to examine the similarities and differences. We shall look at how the systems are composed, and how real, non-expert, users interact with the systems. We shall also provide comparisons with systems from the literature where possible, and indicate where open issues lie and in what areas existing systems may be improved. We conclude that most systems have a common architecture with a set of common subtasks, in particular detecting follow-up questions and finding referents for them. We characterise these tasks using the typical techniques used for performing them, and data from our corpora. We also identify a special type of follow-up question, the discourse question, which is asked when the user is trying to understand an answer, and propose some basic methods for handling it.

Terminological variants for document selection and question/answer matching

Answering precise questions requires applying Natural Language techniques in order to locate the answers inside retrieved documents. The QALC system, presented in this paper, participated to the Question Answering track of the TREC8 and TREC9 evaluations. QALC exploits an analysis of documents based on the search for multi-word terms and their variations. These indexes are used to select a minimal number of documents to be processed and to give indices when comparing question and sentence representations. This comparison also takes advantage of a question analysis module and recognition of numeric and named entities in the documents.

PEAS, the first instantiation of a comparative framework for evaluating parsers of French

This paper presents PEAS, the first comparative evaluation framework for parsers of French whose annotation formalism allows the annotation of both constituents and functional relations. A test corpus containing an assortment of different text types has been built and part of it has been manually annotated. Precision/Recall and crossing brackets metrics will be adapted to our formalism and applied to the parses produced by one parser from academia and another one from industry in order to validate the framework.

Analogical Reasoning for Natural to Formal Language Transfer

In this paper, we explore the use of proportional analogy reasoning to address the problem of language transfer, that is the production of an utterance in a defined target language, given another in the source language. The application we focused on is the transfer of requests in a natural language (French) into commands in a formal language (bash). We introduce a hybrid approach which parallelly uses formal analogy in the language domain and in the more general word sequences domain in order to separately tackle linguistic and parametric variations. On the observation that parametric variations are the most unpredictable, we also apply analogy to automatically extend the available data by generating new requests from the existing ones. Our approach is language independent, it was designed with the help of analogical reasoning to be as generic as possible.

Answering french questions in english by exploiting results from several sources of information

Our bilingual QA system MUSCLEF, is based on QALC, the monolingual system with which we have participated in the previous TREC, where our best results were obtained when we combined the results of several searches. First, QALC searched a reliable document collection for answers, and second the WEB. We kept this strategy for CLEF, returning two runs. In the first one, we modified QALC so as to handle multilinguality by translating the terms identified in the question. In the second run, we combined the results of the first run with those obtained by first translating the question, then applying the full QALC strategy i.e. searching both the collection and the WEB. The final evaluation confirms the fact that the best results are obtained by combining different sources of information.

Incremental Learning from Scratch Using Analogical Reasoning

This paper explores the application of formal analogical reasoning to incremental machine learning. The applicative context is the design of an operational assistant. The specific learning task that is focused on is the transfer from requests in natural language to commands in programming language. This work explores two questions for applying analogy in incremental learning situations: How does formal analogical reasoning behave in incremental learning situation? How do the conditions on the learning sequence influence the performance? To address these issues, an experimental setup is proposed in which multiple users are simulated. The knowledge transfer from one to the other is studied. Moreover, we discuss the influence of the order in which examples are presented to the system on the learning process.

A Mapping-Based Approach for General Formal Human Computer Interaction Using Natural Language

We consider the problem of mapping natural language written utterances expressing operational instructions 1 to formal language expressions, applied to French and the R programming language. Developing a learning operational assistant requires the means to train and evaluate it, that is, a baseline system able to interact with the user. After presenting the guidelines of our work, we propose a model to represent the problem and discuss the fit of direct mapping methods to our task. Finally, we show that, while not resulting in excellent scores, a simple approach seems to be sufficient to provide a baseline for an interactive learning system.