Arnaud Grappy

Lexical validation of answers in Question Answering

Question answering (QA) aims at retrieving precise information from a large collection of documents, typically the Web. Different techniques can be used to find relevant information, and to compare these techniques, it is important to evaluate question answering systems. The objective of an Answer Validation task is to estimate the correctness of an answer returned by a QA system for a question, according to the text snippet given to support it. In this article, we present a lexical strategy for deciding if the snippets justify the answers, based on our own question answering system. We discuss our results, and show the possible extensions of our strategy.

Selecting Answers to Questions from Web Documents by a Robust Validation Process

Question answering (QA) systems aim at finding answers to question posed in natural language using a collection of documents. When the collection is extracted from the Web, the structure and style of the texts are quite different from those of newspaper articles. We developed a QA system based on an answer validation process able to handle Web specificity. A large number of candidate answers are extracted from short passages in order to be validated according to question and passages characteristics. The validation module is based on a machine learning approach. It takes into account criteria characterizing both passage and answer relevance at surface, lexical, syntactic and semantic levels to deal with different types of texts. We present and compare results obtained for factual questions posed on a Web and on a newspaper collection. We show that our system outperforms a baseline by up to 48% in MRR.

Validation du type de la réponse dans un système de questions réponses

Les systemes de questions reponses recherchent la reponse a une question posee en langue naturelle dans un ensemble de documents. Certaines questions attendent une reponse d’un certain type, explicite dans la question. La methode presentee dans cet article verifie que la reponse renvoyee correspond bien au type cherche. Pour cela elle suit une approche par apprentissage automatique en utilisant trois types de criteres. Les premiers sont statistiques et fondes sur la frequence d’apparition de la reponse avec le type dans un ensemble de documents. Les seconds relevent de la reconnaissance des entites nommees et les derniers utilisent l’encyclopedie Wikipedia. L’evaluation globale, 80 % de resultats corrects, montre l’interet de la methode.

Towards an Automatic Validation of Answers in Question Answering

Question answering (QA) aims at retrieving precise information from a large collection of documents. Different techniques can be used to find relevant information, and to compare these techniques, it is important to evaluate QA systems. The objective of an Answer Validation task is thus to judge the correctness of an answer returned by a QA system for a question, according to the text snippet given to support it. We participated in such a task in 2006. In this article, we present our strategy for deciding if the snippets justify the answers: a strategy based on our own QA system, comparing the answers it returned with the answer to judge. We discuss our results, then we point out the difficulties of this task.