Mounia Lalmas

A survey on the use of relevance feedback for information access systems

Users of online search engines often find it difficult to express their need for information in the form of a query. However, if the user can identify examples of the kind of documents they require then they can employ a technique known as relevance feedback. Relevance feedback covers a range of techniques intended to improve a users query and facilitate retrieval of information relevant to a users information need. In this paper we survey relevance feedback techniques. We study both automatic techniques, in which the system modifies the users query, and interactive techniques, in which the user has control over query modification. We also consider specific interfaces to relevance feedback systems and characteristics of searchers that can affect the use and success of relevance feedback systems.

“Is this document relevant?…probably”: a survey of probabilistic models in information retrieval

This article surveys probablistic approaches to modeling information retrieval. The basic concepts of probabilistic approaches to information retrieval are outlined and the principles and assumptions upon which the approaches are based are presented. The various models proposed in the development of IR are described, classified, and compared using a common formalism. New approaches that constitute the basis of future research are described.

Sound and complete relevance assessment for XML retrieval

In information retrieval research, comparing retrieval approaches requires test collections consisting of documents, user requests and relevance assessments. Obtaining relevance assessments that are as sound and complete as possible is crucial for the comparison of retrieval approaches. In XML retrieval, the problem of obtaining sound and complete relevance assessments is further complicated by the structural relationships between retrieval results. A major difference between XML retrieval and flat document retrieval is that the relevance of elements (the retrievable units) is not independent of that of related elements. This has major consequences for the gathering of relevance assessments. This article describes investigations into the creation of sound and complete relevance assessments for the evaluation of content-oriented XML retrieval as carried out at INEX, the evaluation campaign for XML retrieval. The campaign, now in its seventh year, has had three substantially different approaches to gather assessments and has finally settled on a highlighting method for marking relevant passages within documents—even though the objective is to collect assessments at element level. The different methods of gathering assessments at INEX are discussed and contrasted. The highlighting method is shown to be the most reliable of the methods.

Models of user engagement

Our research goal is to provide a better understanding of how users engage with online services, and how to measure this engagement. We should not speak of one main approach to measure user engagement --- e.g. through one fixed set of metrics --- because engagement depends on the online services at hand. Instead, we should be talking of models of user engagement. As a first step, we analysed a number of online services, and show that it is possible to derive effectively simple models of user engagement, for example, accounting for user types and temporal aspects. This paper provides initial insights into engagement patterns, allowing for a better understanding of the important characteristics of how users repeatedly interact with a service or group of services.

Dempster-Shafer's theory of evidence applied to structured documents: modelling uncertainty

Documents ojlen display a structure determined by the author, e.g., several chapters, each with several sub-chapters and so on. Taking into account the structure of a document allows the retrieval process to focus on those parts of the documents that are most relevant to an information need. Chiaramella et al advanced a model for indexing and retrieving structured documents. Their aim was to express the model within a framework based on formal logics with associated theories. They developed the logical formalism of the model. This paper adds to this model a theory of uncertainty, the Dempster-Shafer theory of evidence. It is shown that the theory provides a rule, the Dempster’s combination rule, that a[lows the expression of the uncertainty with respect to parts of a document, and that is compatible with the iogica{ model developed by Chiaramella et al.

The overlap problem in content-oriented XML retrieval evaluation

Within the INitiative for the Evaluation of XML Retrieval(INEX) a number of metrics to evaluate the effectiveness of content-oriented XML retrieval approaches were developed. Although these metrics provide a solution towards addressing the problem of overlapping result elements, they do not consider the problem of overlapping reference components within the recall-base, thus leading to skewed effectiveness scores. We propose alternative metrics that aim to provide a solution to both overlap issues.

INEX 2005 evaluation measures

This paper describes the official measures of retrieval effectiveness employed in INEX 2005: the eXtended Cumulated Gain (XCG) measures. In addition, results of correlation analysis are reported, examining the correlation between the employed quantisation functions and the different measures for the INEX 2005 ad-hoc tasks.

Logical models in information retrieval: introduction and overview

The use of logic to model the information retrieval process has become an established research area. Nevertheless, many people in the information retrieval community do not yet appreciate the work performed in this area, mainly because they do not understand logical formalisms, and hence cannot see the connection between logic and information retrieval. This paper aims at resolving the problem. It introduces the formalisms used in logical models for information retrieval, shows the use of logic to build the models, and presents a brief overview of some of the current logical models in information retrieval.

Overview of INEX 2005

Since 2002, INEX has been working towards the goal of establishing an infrastructure, in the form of a large XML test collection and appropriate scoring methods, for the evaluation of content-oriented XML retrieval systems. This paper provides an overview of the work carried out as part of INEX 2005.

Incorporating user search behavior into relevance feedback

In this paper, we present five user experiments on incorporating behavioral information into the relevance feedback process. In particular, we concentrate on ranking terms for query expansion and selecting new terms to add to the users query. Our experiments are an attempt to widen the evidence used for relevance feedback from simply the relevant documents to include information on how users are searching. We show that this information can lead to more successful relevance feedback techniques. We also show that the presentation of relevance feedback to the user is important in the success of relevance feedback.