Jana Urban

An adaptive technique for content-based image retrieval

We discuss an adaptive approach towards Content-Based Image Retrieval. It is based on the Ostensive Model of developing information needs—a special kind of relevance feedback model that learns from implicit user feedback and adds a temporal notion to relevance. The ostensive approach supports content-assisted browsing through visualising the interaction by adding user-selected images to a browsing path, which ends with a set of system recommendations. The suggestions are based on an adaptive query learning scheme, in which the query is learnt from previously selected images. Our approach is an adaptation of the original Ostensive Model based on textual features only, to include content-based features to characterise images. In the proposed scheme textual and colour features are combined using the Dempster-Shafer theory of evidence combination. Results from a user-centred, work-task oriented evaluation show that the ostensive interface is preferred over a traditional interface with manual query facilities. This is due to its ability to adapt to the users need, its intuitiveness and the fluid way in which it operates. Studying and comparing the nature of the underlying information need, it emerges that our approach elicits changes in the users need based on the interaction, and is successful in adapting the retrieval to match the changes. In addition, a preliminary study of the retrieval performance of the ostensive relevance feedback scheme shows that it can outperform a standard relevance feedback strategy in terms of image recall in category search.

Adaptive image retrieval using a Graph model for semantic feature integration

The variety of features available to represent multimedia data constitutes a rich pool of information. However, the plethora of data poses a challenge in terms of feature selection and integration for effective retrieval. Moreover, to further improve effectiveness, the retrieval model should ideally incorporate context-dependent feature representations to allow for retrieval on a higher semantic level. In this paper we present a retrieval model and learning framework for the purpose of interactive information retrieval. We describe how semantic relations between multimedia objects based on user interaction can be learnt and then integrated with visual and textual features into a unified framework. The framework models both feature similarities and semantic relations in a single graph. Querying in this model is implemented using the theory of random walks. In addition, we present ideas to implement short-term learning from relevance feedback. Systematic experimental results validate the effectiveness of the proposed approach for image retrieval. However, the model is not restricted to the image domain and could easily be employed for retrieving multimedia data (and even a combination of different domains, eg images, audio and text documents).

Simulated Testing of an Adaptive Multimedia Information Retrieval System

The Semantic Gap is considered to be a bottleneck in image and video retrieval. One way to increase the communication between user and system is to take advantage of the users action with a system, e.g. to infer the relevance or otherwise of a video shot viewed by the user. In this paper we introduce a novel video retrieval system and propose a model of implicit information for interpreting the users actions with the interface. The assumptions on which this model was created are then analysed in an experiment using simulated users based on relevance judgements to compare results of explicit and implicit retrieval cycles. Our model seems to enhance retrieval results. Results are presented and discussed in the final section.

EGO: A Personalized Multimedia Management and Retrieval Tool

The problems of content‐based image retrieval (CBIR) systems can be attributed to the semantic gap between the low‐level data representation and the high‐level concepts the user associates with images, on the one hand, and the time‐varying and often vague nature of the underlying information need, on the other. These problems can be addressed by improving the interaction between the user and the system. In this article, we sketch the development of CBIR interfaces and introduce our view on how to solve some of the problems these interfaces present. To address the semantic gap and long‐term multifaceted information needs, we propose a “retrieval in context” system, EGO. EGO is a tool for the management of image collections, supporting the user through personalization and adaptation. We will describe how it learns from the users personal organization, allowing it to recommend relevant images to the user. The recommendation algorithm is described, which is based on relevance feedback techniques. Additionally, we provide results of a performance analysis of the recommendation system and of a preliminary user study.

Evidence combination for multi-point query learning in content-based image retrieval

In multipoint query learning a number of query representatives are selected based on the positive feedback samples. The similarity score to a multipoint query is obtained from merging the individual scores. In this paper, we investigate three different combination strategies and present a comparative evaluation of their performance. Results show that the performance of multipoint queries relies heavily on the right choice of settings for the fusion. Unlike previous results, suggesting that multipoint queries generally perform better than a single query representation, our evaluation results do not allow such an overall conclusion. Instead our study points to the type of queries for which query expansion is better suited than a single query, and vice versa.

Evaluating a workspace's usefulness for image retrieval

Image searching is a creative process. We have proposed a novel image retrieval system that supports creative search sessions by allowing the user to organise their search results on a workspace. The workspace’s usefulness is evaluated in a task-oriented and user-centred comparative experiment, involving design professionals and several types of realistic search tasks. In particular, we focus on its effect on task conceptualisation and query formulation. A traditional relevance feedback system serves as a baseline. The results of this study show that the workspace is more useful in terms of both of the above aspects and that the proposed approach leads to a more effective and enjoyable search experience. This paper also highlights the influence of tasks on the users’ search and organisation strategy.

Attention-based video summarisation in rushes collection

This paper presents the framework of a general video summarisation system on the rushes collection, which formalises the summarisation process as an 0-1 Knapsack optimisation problem. Three stages are included, namely content analysis, content selection and summary composition. Content analysis is the pre-processing step, consisting of shot segmentation, feature extraction, raw video discrimination and shot clustering. Content selection weights the importance of video segments by an attention model. A greedy approximation approach is employed in the composition of summary video with the cost function, which balances the video importance gain and the duration cost. The average content coverage achieved on the rushes test collection is about 29%, while the average qualification score on readability is 3.13 with the redundancy credit at 4.08.

Can a workspace help to overcome the query formulation problem in image retrieval

We have proposed a novel image retrieval system that incorporates a workspace where users can organise their search results. A task-oriented and user-centred experiment has been devised involving design professionals and several types of realistic search tasks. We study the workspaces effect on two aspects: task conceptualisation and query formulation. A traditional relevance feedback system serves as baseline. The results of this study show that the workspace is more useful with respect to both of the above aspects. The proposed approach leads to a more effective and enjoyable search experience.

Exploring results organisation for image searching

An explorative study of an image retrieval interface with respect to the support it offers the user to organise their search results is presented. The evaluation, involving design professionals performing practical and relevant tasks, shows that the proposed approach succeeds in encouraging the user to conceptualise their tasks better.

AIR 2006: First International Workshop on Adaptive Information Retrieval

Adaptive Information Retrieval (IR) systems are designed to optimize retrieval effectiveness and user interaction in an underlying search environment. In this article, we report on a workshop held at Glasgow, UK, in October 2006. The workshop program consisted of three invited talks, four provocative presentations, and a panel discussion. In addition, 15 poster presentations enriched the workshop themes. Important pointers to further development of adaptive IR systems and their evaluation were suggested.