Deepak Ravichandran

Learning surface text patterns for a Question Answering System

In this paper we explore the power of surface text patterns for open-domain question answering systems. In order to obtain an optimal set of patterns, we have developed a method for learning such patterns automatically. A tagged corpus is built from the Internet in a bootstrapping process by providing a few hand-crafted examples of each question type to Altavista. Patterns are then automatically extracted from the returned documents and standardized. We calculate the precision of each pattern, and the average precision for each question type. These patterns are then applied to find answers to new questions. Using the TREC-10 question set, we report results for two cases: answers determined from the TREC-10 corpus and from the web.

Randomized Algorithms and NLP: Using Locality Sensitive Hash Functions for High Speed Noun Clustering

In this paper, we explore the power of randomized algorithm to address the challenge of working with very large amounts of data. We apply these algorithms to generate noun similarity lists from 70 million pages. We reduce the running time from quadratic to practically linear in the number of elements to be computed.

Towards terascale knowledge acquisition

Although vast amounts of textual data are freely available, many NLP algorithms exploit only a minute percentage of it. In this paper, we study the challenges of working at the terascale. We present an algorithm, designed for the teraxale, for mining is-a relations that achieves similar performance to a state-of-the-art linguistically-rich method. We focus on the accuracy of these two systems as a function of processing time and corpus size.

Toward semantics-based answer pinpointing

We describe the treatment of questions (Question-Answer Typology, question parsing, and results) in the Weblcopedia question answering system.

Statistical QA - Classifier vs. Re-ranker: What's the difference?

In this paper, we show that we can obtain a good baseline performance for Question Answering (QA) by using only 4 simple features. Using these features, we contrast two approaches used for a Maximum Entropy based QA system. We view the QA problem as a classification problem and as a re-ranking problem. Our results indicate that the QA system viewed as a re-ranker clearly outperforms the QA system used as a classifier. Both systems are trained using the same data.

Automatic derivation of surface text patterns for a maximum entropy based question answering system

In this paper we investigate the use of surface text patterns for a Maximum Entropy based Question Answering (QA) system. These text patterns are collected automatically in an unsupervised fashion using a collection of trivia question and answer pairs as seeds. These patterns are used to generate features for a statistical question answering system. We report our results on the TREC-10 question set.

Using knowledge to facilitate factoid answer pinpointing

In order to answer factoid questions, the Webclopedia QA system employs a range of knowledge resources. These include a QA Typology with answer patterns, WordNet, information about typical numerical answer ranges, and semantic relations identified by a robust parser, to filter out likely-looking but wrong candidate answers. This paper describes the knowledge resources and their impact on system performance.

How To Select An Answer String

Given a question Q and a sentence/paragraph SP that is likely to contain the answer to Q, an answer selection module is supposed to select the “exact” answer sub-string A ⊂ SP. We study three distinct approaches to solving this problem: one approach uses algorithms that rely on rich knowledge bases and sophisticated syntactic/semantic processing; one approach uses patterns that are learned in an unsupervised manner from the web, using computational biology-inspired alignment algorithms; and one approach uses statistical noisy-channel algorithms similar to those used in machine translation. We assess the strengths and weaknesses of these three approaches and show how they can be combined using a maximum entropy-based framework.