Munirathnam Srikanth

Biterm language models for document retrieval

Statistical Language Models(LM) have been used in many natural language processing tasks including speech recognition and machine translation [5, 2]. Recently language models have been explored as a framework for information retrieval [9, 4, 7, 1, 6]. The basic idea is to view each document to have its own language model and model querying as a generative process. Documents are ranked based on the probability of their language model generating the given query. Since documents are fixed entities in information retrieval, language models for documents suffer from sparse data problem. Smoothed unigram models have been used to demonstrate better performance of language models against vector space or probabilistic retrieval models for document retrieval. Song and Croft [10] proposed a general language model that combined bigram language models with Good-Turing estimate and corpus-based smoothing of unigram probabilities. Improved performance was observed with combined bigram language models. The language models explored for information retrieval mimic those used for speech recognition. Specifically, in the bigram model a document d represented as word sequence w1, w2, · · · , wn is modeled as

Exploiting syntactic structure of queries in a language modeling approach to IR

Natural Language Processing (NLP) techniques have been explored to enhance the performance of Information Retrieval (IR) methods with varied results. Most efforts in using NLP techniques have been to identify better index terms for representing documents. This use in the indexing phase of IR has implicit effect on retrieval performance. However, the explicit use of NLP techniques during the retrieval or information seeking phase has been restricted to interactive or dialogue systems. Recent advances in IR are based on using Statistical Language Models (SLM) to represent documents and ranking them based on their model generating a given user query. This paper presents a novel method for using NLP techniques on user queries, specifically, a syntactic parse of a query, in the statistical language modeling approach to IR. In the proposed method, named Concept Language Models, a query is viewed as a sequence of concepts and a concept as a sequence terms. The paper presents different approximations to estimate the concept and term probabilities and compute the query likelihood estimate for documents. Some empirical results on TREC test collections comparing Concept Language Models with smoothed N-gram language models are presented.

Incorporating query term dependencies in language models for document retrieval

Recent advances in Information Retrieval are based on using Statistical Language Models (SLM) for representing documents and evaluating their relevance to user queries [6, 3, 4]. Language Modeling (LM) has been explored in many natural language tasks including machine translation and speech recognition [1]. In LM approach to document retrieval, each document, D, is viewed to have its own language model, MD. Given a query, Q, documents are ranked based on the probability, P (Q|MD), of their language model generating the query. While the LM approach to information retrieval has been motivated from different perspectives [3, 4], most experiments have used smoothed unigram language models that assume term independence for estimating document language models. N-gram, specifically, bigram language models that capture context provided by the previous word(s) perform better than unigram models [7]. Biterm language models [8] that ignore the word order constraint in bigram language models have been shown to perform better than bigram models. However, word order constraint cannot always be relaxed since a blind venetian is not a venetian blind. Term dependencies can be measured using their co-occurrence statistics. Nallapati and Allan [5] represent term dependencies in a sentence using a maximum spanning tree and generate a sentence tree language model for the story link detection task in TDT. Syntactic parse of user queries can provide clues for when the word order constraint can be relaxed. Syn-

Extracting exact answers to questions based on structural links

This paper presents a novel approach to extracting phrase-level answers in a question answering system. This approach uses structural support provided by an integrated Natural Language Processing (NLP) and Information Extraction (IE) system. Both questions and the sentence-level candidate answer strings are parsed by this NLP/IE system into binary dependency structures. Phrase-level answer extraction is modelled by comparing the structural similarity involving the question-phrase and the candidate answer-phrase.There are two types of structural support. The first type involves predefined, specific entity associations such as Affiliation, Position, Age for a person entity. If a question asks about one of these associations, the answer-phrase can be determined as long as the system decodes such pre-defined dependency links correctly, despite the syntactic difference used in expressions between the question and used in expressions between the question and the candidate answer string. The second type involves generic grammatical relationships such as V-S (verb-subject), V-O (verb-object).Preliminary experimental results show an improvement in both precision and recall in extracting phrase-level answers, compared with a baseline system which only uses Named Entity constraints. The proposed methods are particularly effective in cases where the question-phrase does not correspond to a known named entity type and in cases where there are multiple candidate answer-phrases satisfying the named entity constraints.

Computation of the MinMax Measure

The MinMax measure of information, defined by Kapur, Baciu and Kesavan [6], is a quantitative measure of the information contained in a given set of moment constraints. It is based on both maximum and minimum entropy. Computational difficulties in the determination of minimum entropy probability distributions (MinEPD) have inhibited exploration of the full potential of minimum entropy and, hence, the MinMax measure. Initial attempts to solve the minimum entropy problem were directed towards finding analytical solutions for some specific set of constraints. Here, we present a numerical solution to the general minimum entropy problem and discuss the significance of minimum entropy and the MinMax measure. Some numerical examples are given for illustration.