Michael Ortega

Relevance feedback: a power tool for interactive content-based image retrieval

Content-based image retrieval (CBIR) has become one of the most active research areas in the past few years. Many visual feature representations have been explored and many systems built. While these research efforts establish the basis of CBIR, the usefulness of the proposed approaches is limited. Specifically, these efforts have relatively ignored two distinct characteristics of CBIR systems: (1) the gap between high-level concepts and low-level features, and (2) the subjectivity of human perception of visual content. This paper proposes a relevance feedback based interactive retrieval approach, which effectively takes into account the above two characteristics in CBIR. During the retrieval process, the users high-level query and perception subjectivity are captured by dynamically updated weights based on the users feedback. The experimental results over more than 70000 images show that the proposed approach greatly reduces the users effort of composing a query, and captures the users information need more precisely.

Supporting similarity queries in MARS

To address the emerging needs of applications that require access to and retrieval of multimedia objects, we are developing the Multimedia Analysis and Retrieval System (MARS) in our group at the University of Illinois [13]. In this paper, we concentrate on the retrieval subsystem of MARS and its support for content-based queries over image databases. Content-based retrieval techniques have been extensively studied for textual documents in the area of automatic information retrieval [24, 21. This paper describes how these techniques can be adapted for ranked retried over image databases. Specifically, we discuss the ranking and retrieval algorithms developed in MARS based on the Boolean retrievaI model and describe the results of our experiments that demonstrate the effectiveness of the developed model for image retrieval.

Supporting ranked Boolean similarity queries in MARS

To address the emerging needs of applications that require access to and retrieval of multimedia objects, we are developing the Multimedia Analysis and Retrieval System (MARS). In this paper, we concentrate on the retrieval subsystem of MARS and its support for content-based queries over image databases. Content-based retrieval techniques have been extensively studied for textual documents in the area of automatic information retrieval. This paper describes how these techniques can be adapted for ranked retrieval over image databases. Specifically, we discuss the ranking and retrieval algorithms developed in MARS based on the Boolean retrieval model and describe the results of our experiments that demonstrate the effectiveness of the developed model for image retrieval.

A relevance feedback architecture for content-based multimedia information retrieval systems

Content-based multimedia information retrieval (MIR) has become one of the most active research areas in the past few years. Many retrieval approaches based on extracting and representing visual properties of multimedia data have been developed. While these approaches establish the viability of MIR based on visual features, techniques for incorporating human expertise directly during the query process to improve retrieval performance have not drawn enough attention. To address this limitation, this paper introduces a human-computer interaction based approach to MIR in which the user guides the system during retrieval using relevance feedback. Our experiments show that the retrieval performance improves significantly by incorporating humans in the retrieval process

Query reformulation for content based multimedia retrieval in MARS

Unlike traditional database management systems, in content-based multimedia retrieval databases, it is difficult for users to express their exact information need directly in a precise query. A typical interface allows users to express their information need using examples of objects similar to the ones they wish to retrieve. Such a user interface, however, requires mechanisms to learn the query representation from the examples. In this paper, we describe the query refinement framework implemented in the Multimedia Analysis and Retrieval System (MARS) for learning query representations using relevance feedback. The proposed framework uses a query expansion approach towards modifying the query representation in which relevant objects are added to the query. Furthermore, query reweighting techniques are used to adjust similarity functions.

Similarity Search Using Multiple Examples in MARS

Unlike traditional database management systems, in multimedia databases that support content-based retrieval over multimedia objects, it is difficult for users to express their exact information need directly in the form of a precise query. At ypical interface supported by content-based retrieval systems allows users to express their query in the form of examples of objects similar to the ones they wish to retrieve. Such a user interface, however, requires mechanisms to learn the query representation from the examples provided by the user. In our previous work, we proposed a query refinement mechanism in which a query representation is modified by adding new relevant examples based on user feedback. In this paper, we describe query processing mechanisms that can efficiently support query expansion using multidimensional index structures.

Efficient evaluation of relevance feedback for multidimensional all-pairs retrieval

New retrieval applications support flexible comparison for all-pairs best match operations based on a notion of similarity or distance. The distance between items is determined by some arbitrary distance function. Users that pose queries may change their definition of the distance metric as they progress. The distance metric change may be explicit or implicit in an application (e.g., using relevance feedback). Recomputing from scratch the results with the new distance metric is wasteful. In this paper, we present an efficient approach to recomputing the all-pairs best match (join) operation using the new distance metric by re-using the work already carried out for the old distance metric. Our approach reduces significantly the work required to compute the new result, as compared to a naive re-evaluation.