Cornelis Joost van Rijsbergen

Probabilistic models of information retrieval based on measuring the divergence from randomness

We introduce and create a framework for deriving probabilistic models of Information Retrieval. The models are nonparametric models of IR obtained in the language model approach. We derive term-weighting models by measuring the divergence of the actual term distribution from that obtained under a random process. Among the random processes we study the binomial distribution and Bose--Einstein statistics. We define two types of term frequency normalization for tuning term weights in the document--query matching process. The first normalization assumes that documents have the same length and measures the information gain with the observed term once it has been accepted as a good descriptor of the observed document. The second normalization is related to the document length and to other statistics. These two normalization methods are applied to the basic models in succession to obtain weighting formulae. Results show that our framework produces different nonparametric models forming baseline alternatives to the standard tf-idf model.

“Is this document relevant?…probably”: a survey of probabilistic models in information retrieval

This article surveys probablistic approaches to modeling information retrieval. The basic concepts of probabilistic approaches to information retrieval are outlined and the principles and assumptions upon which the approaches are based are presented. The various models proposed in the development of IR are described, classified, and compared using a common formalism. New approaches that constitute the basis of future research are described.

The potential and actual effectiveness of interactive query expansion

In query expansion, terms from a source such as relevance feedback are added to the query. This often improves retrieval effectiveness but results are variable across queries. In interactive query expansion (IQE) the automatically-derived terms are instead offered as suggestions to the searcher, who decides which to add. There is little evidence of whether IQE is likely to be effective over multiple iterations in a large scale retrieval context, or whether inexperienced users can achieve this effectiveness in practice. These experiments address these two questions. A small but significant improvement in potential retrieval effectiveness is found. This is consistent across a range of topics. Inexperienced users’ term selections consistently fail to improve on automatic query expansion, however. It is concluded that interactive query expansion has good potential, particular y for term sources that are porer than relevance feedback. But it may be difficult for searchers to realise this potential without experience or training in term selection and free-text search strategies.

An adaptive technique for content-based image retrieval

We discuss an adaptive approach towards Content-Based Image Retrieval. It is based on the Ostensive Model of developing information needs—a special kind of relevance feedback model that learns from implicit user feedback and adds a temporal notion to relevance. The ostensive approach supports content-assisted browsing through visualising the interaction by adding user-selected images to a browsing path, which ends with a set of system recommendations. The suggestions are based on an adaptive query learning scheme, in which the query is learnt from previously selected images. Our approach is an adaptation of the original Ostensive Model based on textual features only, to include content-based features to characterise images. In the proposed scheme textual and colour features are combined using the Dempster-Shafer theory of evidence combination. Results from a user-centred, work-task oriented evaluation show that the ostensive interface is preferred over a traditional interface with manual query facilities. This is due to its ability to adapt to the users need, its intuitiveness and the fluid way in which it operates. Studying and comparing the nature of the underlying information need, it emerges that our approach elicits changes in the users need based on the interaction, and is successful in adapting the retrieval to match the changes. In addition, a preliminary study of the retrieval performance of the ostensive relevance feedback scheme shows that it can outperform a standard relevance feedback strategy in terms of image recall in category search.

Eraser Lattices and Semantic Contents

A novel way to define Quantum like measurements for text is through transformations called Selective Erasers. When applied to text, an Eraser acts like a filter and preserves part of the information of the document (tokens surrounding a central term) and erases the rest. In this paper, we describe how inclusion relations between Erasers can be used to construct an Eraser Lattice for relevant content. It is posited that given a new piece of text, the application of elements of the Eraser Lattice, will result in the destruction or preservation of the content depending on the relevancy of the document. The paper provides the theoretical derivations required to perform such transformations, along with some example applications, before outlining directions and challenges of future work.

A Model for Adaptive Information Retrieval

The paper presents a network model that can be used toproduce conceptual and logical schemas for Information Retrievalapplications. The model has interesting adaptability characteristicsand can be instantiated in various effective ways. The paper alsoreports the results of an experimental investigation into theeffectiveness of implementing associative and adaptive retrieval onthe proposed model by means of Neural Networks. The implementationmakes use of the learning and generalisation capabilities of theBackpropagation learning algorithm to build up and use applicationdomain knowledge in a sub-symbolic form. The knowledge is acquiredfrom examples of queries and relevant documents. Three differentlearning strategies are introduced, their performance is analysed andcompared with the performance of a traditional Information Retrievalsystem.

HMM Model Selection Issues for Soccer Video

There has been a concerted effort from the Video Retrieval community to develop tools that automate the annotation process of Sports video. In this paper, we provide an in-depth investigation into three Hidden Markov Model (HMM) selection approaches. Where HMM, a popular indexing framework, is often applied in a ad hoc manner. We investigate what effect, if any, poor HMM selection can have on future indexing performance when classifying specific audio content. Audio is a rich source of information that can provide an effective alternative to high dimensional visual or motion based features. As a case study, we also illustrate how a superior HMM framework optimised using a Bayesian HMM selection strategy, can both segment and then classify Soccer video, yielding promising results.

Mathematical, logical, and formal methods in information retrieval: an introduction to the special issue

Research on the use of mathematical, logical, and formal methods, has been central to Information Retrieval research for a long time. Research in this area is important not only because it helps enhancing retrieval effectiveness, but also because it helps clarifying the underlying concepts of Information Retrieval. In this article we outline some of the major aspects of the subject, and summarize the papers of this special issue with respect to how they relate to these aspects. We conclude by highlighting some directions of future research, which are needed to better understand the formal characteristics of Information Retrieval.