Sabrina Tollari

Enhancement of Textual Images Classification Using Segmented Visual Contents for Image Search Engine

This paper deals with the use of the dependencies between the textual indexation of an image (a set of keywords) and its visual indexation (colour and shape features). Experiments are realized on a corpus of photographs of a press agency (EDITING) and on another corpus of animals and landscape photographs (COREL). Both are manually indexed by keywords. Keywords of the news photos are extracted from a hierarchically structured thesaurus. Keywords of Corel corpus are semantically linked using WordNet database. A semantic clustering of the photos is constructed from their textual indexation. We use two different visual segmentation schemes. One is based on areas of interest, the other one on blobs of homogenous colour. Both segmentation schemes are used to evaluate the performance of a content-based image retrieval system combining textual and visual descriptions. Results of visuo-textual classifications show an improvement of 50% against classification using only textual information. Finally, we show how to apply this system in order to enhance a web image search engine. To this purpose, we illustrate a method allowing selecting only accurate images resulting from a textual query.

Learning optimal visual features from Web sampling in online image retrieval

Linear discriminant analysis (LDA) to improve a Web images retrieval system. Our work takes place in the official European ImagEVAL 2006 campaign evaluation. The task consists to retrieve Web images using both textual (Web pages) and visual information. Our visual features integrate subband entropy profile, usual mean and color standard deviation. A simple weighted norm fusion is done with standard tf-idf Web page text analysis. Our model is the second best model of the ImagEVAL task2. We show how, sampling online image sets from the Web, one can estimate by approximated Fisher criterion an optimal visual feature subsets for some query concepts and then enhance their mean average precision by 50%. We discuss on the fact that some concept may not so nicely be enhanced, but that in average, this optimization reduces by 10 the visual dimension, without any MAP degradation, yielding to a significant CPU cost reduction.

LDA Versus MMD Approximation on Mislabeled Images for Dependant Selection of Visual Features and Their Heterogeneity

We propose first to generate new visual features based on entropy measure (heterogeneity), and then we address the question of feature selection in the context of mislabeled images for automatic image classification. We compare two methods of word dependant feature selection on mislabeled images: approximation of linear discriminant analysis (ALDA) and approximation of maximum marginal diversity (AMMD). A hierarchical ascendant classification (HAC) is trained and tested using full or reduced visual space. Experiments are conducted on 10 K Corel images with 52 keywords, 40 visual features (U) and 40 new heterogeneity features (H). Compared to HAC on all U features, we measure a classification gain of 56% and in the same time a reduction of 92% of the number of features using a simple late fusion of U and H

Using tree of concepts and hierarchical reordering for diversity in image retrieval

Current search engines return relevant results, but often the retrieved items are similar. Moreover, the first images tend to hide all the available richness. In this paper, we propose not only to show how to increase the diversity, but also how to address the hierarchical nature of the diversity. We propose innovative image ordering strategies based on an agglomerative hierarchical clustering (ARC). Furthermore, we introduce a novel approach for exploiting richer description resources, such as a “tree of concepts”. The different approaches are compared on a highly relevant and manually annotated benchmark: the Xilopix benchmark; and on the, more general but less adapted, ImageClef2008 benchmark. Any of the proposed approaches increase the diversity (CR20) compared to search engines standard outputs and outperform an average random shuffling (baseline). Discussion for each individual novelty is presented. In particular it is show that a hierarchical exploitation of the results of the ARC increases the diversity in all cases.

Approximation of linear discriminant analysis for word dependent visual features selection

To automatically determine a set of keywords that describes the content of a given image is a difficult problem, because of (i) the huge dimension number of the visual space and (ii) the unsolved object segmentation problem. Therefore, in order to solve matter (i), we present a novel method based on an Approximation of Linear Discriminant Analysis (ALDA) from the theoretical and practical point of view. Application of ALDA is more generic than usual LDA because it doesn’t require explicit class labelling of each training sample, and however allows efficient estimation of the visual features discrimination power. This is particularly interesting because of (ii) and the expensive manually object segmentation and labelling tasks on large visual database. In first step of ALDA, for each word wk, the train set is split in two, according if images are labelled or not by wk. Then, under weak assumptions, we show theoretically that Between and Within variances of these two sets are giving good estimates of the best discriminative features for wk. Experimentations are conducted on COREL database, showing an efficient word adaptive feature selection, and a great enhancement (+37%) of an image Hierarchical Ascendant Classification (HAC) for which ALDA saves also computational cost reducing by 90% the visual features space.

Sélection adaptative des dimensions de l'indexation visuelle d'images mal annotées en fonction du mot recherché

The construction of generic visual oncept models is difficult, because the great image databases are not well labeled. This invalidates the traditional optimization methods for the high dimensional visual space. In order to enhance image search and index system we propose novel visual featues based on entropy analysis, and two methods to reduce the features space allowing to estimate the most discriminant visual features for a given keyword. We approximate LDA or MMD on real misslabled image databases. Then, we use a non supervised clustering algorithm to build visual clusters, using all the features of the visual space, or several subspaces made up with the most discriminant features depending of each keyword. Results on COREL show classification enhancement of 69 % while reducing the number of dimensions by 79 %. The collections size impacts for our methods are discussed.