Mohammed Lamine Kherfi

Image Retrieval from the World Wide Web: Issues, Techniques, and Systems

With the explosive growth of the World Wide Web, the public is gaining access to massive amounts of information. However, locating needed and relevant information remains a difficult task, whether the information is textual or visual. Text search engines have existed for some years now and have achieved a certain degree of success. However, despite the large number of images available on the Web, image search engines are still rare. In this article, we show that in order to allow people to profit from all this visual information, there is a need to develop tools that help them to locate the needed images with good precision in a reasonable time, and that such tools are useful for many applications and purposes. The article surveys the main characteristics of the existing systems most often cited in the literature, such as ImageRover, WebSeek, Diogenes, and Atlas WISE. It then examines the various issues related to the design and implementation of a Web image search engine, such as data gathering and digestion, indexing, query specification, retrieval and similarity, Web coverage, and performance evaluation. A general discussion is given for each of these issues, with examples of the ways they are addressed by existing engines, and 130 related references are given. Some concluding remarks and directions for future research are also presented.

Relevance feedback for CBIR: a new approach based on probabilistic feature weighting with positive and negative examples

In content-based image retrieval, understanding the users needs is a challenging task that requires integrating him in the process of retrieval. Relevance feedback (RF) has proven to be an effective tool for taking the users judgement into account. In this paper, we present a new RF framework based on a feature selection algorithm that nicely combines the advantages of a probabilistic formulation with those of using both the positive example (PE) and the negative example (NE). Through interaction with the user, our algorithm learns the importance he assigns to image features, and then applies the results obtained to define similarity measures that correspond better to his judgement. The use of the NE allows images undesired by the user to be discarded, thereby improving retrieval accuracy. As for the probabilistic formulation of the problem, it presents a multitude of advantages and opens the door to more modeling possibilities that achieve a good feature selection. It makes it possible to cluster the query data into classes, choose the probability law that best models each class, model missing data, and support queries with multiple PE and/or NE classes. The basic principle of our algorithm is to assign more importance to features with a high likelihood and those which distinguish well between PE classes and NE classes. The proposed algorithm was validated separately and in image retrieval context, and the experiments show that it performs a good feature selection and contributes to improving retrieval effectiveness.

Combining positive and negative examples in relevance feedback for content-based image retrieval

Abstract In this paper, we address some issues related to the combination of positive and negative examples to improve the efficiency of image retrieval. We start by analyzing the relevance of the negative example and how it can be interpreted and utilized to mitigate certain problems in image retrieval, such as noise, miss, the page zero problem and feature selection. Then we propose a new relevance feedback approach that uses the positive example (PE) to perform generalization and the negative example (NE) to perform specialization. In this approach, a query containing both PE and NE is processed in two steps. The first step considers the PE alone, in order to reduce the set of images participating in retrieval to a more homogeneous subset. Then, the second step considers both PE and NE and acts on the images retained in the first step. Mathematically, relevance feedback is formulated as an optimization of the intra and inter variances of the PE and NE. The proposed relevance feedback algorithm was implemented in our image retrieval system, which we tested on a collection of more than 10,000 images. The experimental results show how the NE as considered in our model can contribute in improving the relevance of the images retrieved.

Learning from negative example in relevance feedback for content-based image retrieval

In this paper, we address some issues related to the combination of positive and negative examples to perform more efficient image retrieval. We analyze the relevance of negative example and how it can be interpreted. Then we propose a new relevance feedback model that integrates both positive and negative examples. First, a query is formulated using positive example, then negative example is used to refine the systems response. Mathematically, relevance feedback is formulated as an optimization of intra and inter variances of positive and negative examples.

Combining visual features with semantics for a more effective image retrieval

We present a new framework which tries to improve the effectiveness of CBIR by integrating semantic concepts extracted from text. Our model is inspired from the VSM model developed in information retrieval. We represent each image in our collection with a vector of probabilities linking it to the different keywords. In addition to the semantic content of images, these probabilities capture the users preference in each step of relevance feedback. The obtained features are then combined with visual ones in retrieval phase. Evaluation carried out on more than 10,000 images shows that this considerably improves retrieval effectiveness.

Image Collection Organization and Its Application to Indexing, Browsing, Summarization, and Semantic Retrieval

In this paper, we present a new framework for organizing image collections into structures that can be used for indexing, browsing, retrieval and summarization. Instead of using tree-based techniques which are not suitable for images, we develop a new solution that is specifically designed for image collections. We consider both low-level image content and high-level semantics in an attempt to alleviate the semantic gap encountered by many systems. The fact that our model is based on a probabilistic framework makes it possible to combine it in a natural way with probabilistic techniques developed recently for image retrieval. The structure our model generates is applied for four purposes. The first is to provide retrieval module with an index, which allows it to improve retrieval time and accuracy, while the second is to provide users with a hierarchical browsing catalog that allows them to navigate the image collection by subject. This represents an additional step towards facilitating human-computer interaction in the context of image retrieval and navigation. The third aim is to provide users with a summarization of the general content of each class in the collection, and the fourth is a retrieval mechanism. Related issues such as relevance feedback and feature selection are also addressed. The experiments at the end of the paper show that the proposed framework yields some significant improvements

Image retrieval based on feature weighting and relevance feedback

We present a relevance feedback model for CBIR, based on a feature weighting algorithm. The proposed model uses positive and negative items selected by the user to learn the importance of image features, then applies the obtained weights to define similarity measures corresponding to the users perception. The basic principle of this work is to give more importance to features with a high likelihood and those which separate well between positive example (PE) classes and negative example (NE) classes. The proposed algorithm was validated separately and in the image retrieval context, and the experiments show that it contributes in improving retrieval effectiveness.

Review of Human-Computer Interaction Issues in Image Retrieval

Image retrieval is an active area of research, which is growing very rapidly. Indeed, stimulated by the rapid growth in storage capacity and processing speed, the number of images in electronic collections and the World Wide Web has considerably increased over the last few years. However, with this abundance of information, people are continuously looking for tools that help them find the image(s) they are looking for within a reasonable amount of time. These tools are image retrieval engines. When using an image retrieval engine, the user is continuously interacting with the machine. First, he1 uses the system’s interface to formulate a query that expresses his needs. Second, he provides feedback about the retrieved results at each search iteration. This allows the engine to provide more accurate results by using relevance feedback (RF) techniques. Third, he may be asked to assign a goodness score or weight to each image retrieved, which helps evaluating the system’s performance. In this chapter, we will review the main interactions between human and the machine in the context of image retrieval. We will address several issues, including: Query formulation: • How the user expresses his needs and what he is looking for • The different ways the query can be formulated: keywords-based, sentence-based, query by example image, query by sketch, query by feature values, composite queries, etc. • Query by region of interest (ROI) vs. global query. • Queries with positive example only vs. queries with both positive and negative examples. • Page zero problem: finding a good image to initiate a retrieval session. Relevance feedback: we will try to answer questions like: • Why do systems use relevance feedback? • How can the user express his needs during the relevance feedback process • How this information is exploited by the system to perform operations like feature selection or the identification of the sought image. 1 Note that the masculine gender has been used strictly to facilitate reading, and is to be understood to include the feminine. Advances in Human-Computer Interaction 216 • The different families of RF techniques. • Relevance feedback with retrieval memory, i.e., taking into account the value of old iteration queries when constructing the new one. • Whether it is useful for the system to create user profiles, and the challenges it has to face. • The number of RF iterations required to obtain satisfactory results. Viewing retrieval results: • Existing viewing techniques: 2D linear presentation, 3D-based presentation, etc. • Different ways the resulting images may be ordered and presented to the user: similarity-ordered, time-ordered, event-ordered, etc. Evaluation of the retrieval performance by the user: • How the user can express his satisfaction/dissatisfaction about the retrieved images • What about the ground truth in image retrieval evaluation? • System response time and its influence on user satisfaction. • The ease of use of the system’s interface. Other issues: • User’s needs: He may be looking for a specific image, for images that meet a given need (e.g. illustrate a concept) or simply browsing the collection looking for potentially “good” images.