Matthieu Cord

Pooling in image representation: The visual codeword point of view

In this work, we propose BossaNova, a novel representation for content-based concept detection in images and videos, which enriches the Bag-of-Words model. Relying on the quantization of highly discriminant local descriptors by a codebook, and the aggregation of those quantized descriptors into a single pooled feature vector, the Bag-of-Words model has emerged as the most promising approach for concept detection on visual documents. BossaNova enhances that representation by keeping a histogram of distances between the descriptors found in the image and those in the codebook, preserving thus important information about the distribution of the local descriptors around each codeword. Contrarily to other approaches found in the literature, the non-parametric histogram representation is compact and simple to compute. BossaNova compares well with the state-of-the-art in several standard datasets: MIRFLICKR, ImageCLEF 2011, PASCAL VOC 2007 and 15-Scenes, even without using complex combinations of different local descriptors. It also complements well the cutting-edge Fisher Vector descriptors, showing even better results when employed in combination with them. BossaNova also shows good results in the challenging real-world application of pornography detection.

Active Learning Methods for Interactive Image Retrieval

Active learning methods have been considered with increased interest in the statistical learning community. Initially developed within a classification framework, a lot of extension are now being proposed to handle multimedia applications. This paper provides algorithms within a statistical framework to extend active learning for online content-based image retrieval (CBIR). The classification framework is presented with experiments to compare several powerful classification techniques in this information retrieval context. Focusing on interactive methods, active learning strategy is then described. The limitations of this approach for CBIR are emphasized before presenting our new active selection process RETIN. First, as any active method is sensitive to the boundary estimation between classes, the RETIN strategy carries out a boundary correction to make the retrieval process more robust. Second, the criterion of generalization error to optimize the active learning selection is modified to better represent the CBIR objective of database ranking. Third, a batch processing of images is proposed. Our strategy leads to a fast and efficient active learning scheme to retrieve sets of online images (query concept). Experiments on large databases show that the RETIN method performs well in comparison to several other active strategies.

Building Detection and Reconstruction from Mid- and High-Resolution Aerial Imagery

In this paper, we discuss methods for building detection and reconstruction from aerial imagery. These methods are intended for the analysis of urban and suburban areas and have been applied to images of different resolutions (between 1 m and 10 cm per pixel). Various algorithms for image matching have been investigated, including hierarchical processing and new correlation schemes that have interesting properties for building recognition and building feature grouping. Cooperative combination of 2-D (monocular) and 3-D (stereoscopic) information allows the complete representation of the observed scene and particularly the detection of man-made raised structures such as buildings. A performance evaluation on simulation-based images has been considered in comparison with the corresponding ground truth reference. Our work illustrates that mid-resolution methods cannot be directly applied to high-resolution images. Classical algorithms must be adapted and new techniques have been defined to carry out dense urban area reconstruction.

T-HOG: An effective gradient-based descriptor for single line text regions

We discuss the use of histogram of oriented gradients (HOG) descriptors as an effective tool for text description and recognition. Specifically, we propose a HOG-based texture descriptor (T-HOG) that uses a partition of the image into overlapping horizontal cells with gradual boundaries, to characterize single-line texts in outdoor scenes. The input of our algorithm is a rectangular image presumed to contain a single line of text in Roman-like characters. The output is a relatively short descriptor that provides an effective input to an SVM classifier. Extensive experiments show that the T-HOG is more accurate than Dalal and Triggss original HOG-based classifier, for any descriptor size. In addition, we show that the T-HOG is an effective tool for text/non-text discrimination and can be used in various text detection applications. In particular, combining T-HOG with a permissive bottom-up text detector is shown to outperform state-of-the-art text detection systems in two major publicly available databases.

RETIN: A Content-Based Image Indexing and Retrieval System

Abstract:This paper presents RETIN, a new system for automatic image indexing and interactive content-based image retrieval. The most original aspect of our work rests on the distance computation and its adjustment by relevance feedback. First, during an offline stage, the indexes are computed from attribute vectors associated with image pixels. The feature spaces are partitioned through an unsupervised classification, and then, thanks to these partitions, statistical distributions are processed for each image. During the online use of the system, the user makes an iconic request, i.e. he brings an example of the type of image he is looking for. The query may be global or partial, since the user can reduce his request to a region of interest. The comparison between the query distribution and that of every image in the collection is carried out by using a weighted dissimilarity function which manages the use of several attributes. The results of the search are then refined by means of relevance feedback, which tunes the weights of the dissimilarity measure via user interaction. Experiments are then performed on large databases and statistical quality assessment shows the good properties of RETIN for digital image retrieval. The evaluation also shows that relevance feedback brings flexibility and robustness to the search.

Stochastic exploration and active learning for image retrieval

Abstract This paper deals with content-based image retrieval. When the user is looking for large categories, statistical classification techniques are efficient as soon as the training set is large enough. We introduce a two-step—exploration, classification—interactive strategy designed for category retrieval. The first step aims at getting a useful initial training set for the classification step. A stochastic image selection process is used instead of the usual strategy based on a similarity score ranking. This process is dedicated to explore the database in order to collect examples as various as possible of the searched category. The second step aims at providing the best classification between relevant and irrelevant images. Based on SVM, the classification applies an active learning strategy through user interaction. A quality assessment is carried out on the ANN and COREL databases in order to compare and validate our approach.

A comparison of active classification methods for content-based image retrieval

This paper deals with content-based image indexing and category retrieval in general databases. Statistical learning approaches have been recently introduced in CBIR. Labelled images are considered as training data in learning strategy based on classification process. We introduce an active learning strategy to select the most difficult images to classify with only few training data. Experimentations are carried out on the COREL database. We compare seven classification strategies to evaluate the active learning contribution in CBIR.

Long-term similarity learning in content-based image retrieval

This paper presents a new learning technique for the similarity model refinement in CBIR systems. We propose a whole retrieval strategy based on a new relevance feedback scheme and on a long-term similarity learning algorithm which uses feedback information of previous sessions. We introduce this technique as the simple evolution of the short-term relevance feedback approach into a long-term similarity learning, without additional need of user interaction. Our algorithm is validated via a quality assessment realized on a heterogeneous database of 1,200 color images.

BOSSA: Extended bow formalism for image classification

In image classification, the most powerful statistical learning approaches are based on the Bag-of-Words paradigm. In this article, we propose an extension of this formalism. Considering the Bag-of-Features, dictionary coding and pooling steps, we propose to focus on the pooling step. Instead of using the classical sum or max pooling strategies, we introduced a density function-based pooling strategy. This flexible formalism allows us to better represent the links between dictionary codewords and local descriptors in the resulting image signature. We evaluate our approach in two very challenging tasks of video and image classification, involving very high level semantic categories with large and nuanced visual diversity.

Extended Coding and Pooling in the HMAX Model

This paper presents an extension of the HMAX model, a neural network model for image classification. The HMAX model can be described as a four-level architecture, with the first level consisting of multiscale and multiorientation local filters. We introduce two main contributions to this model. First, we improve the way the local filters at the first level are integrated into more complex filters at the last level, providing a flexible description of object regions and combining local information of multiple scales and orientations. These new filters are discriminative and yet invariant, two key aspects of visual classification. We evaluate their discriminative power and their level of invariance to geometrical transformations on a synthetic image set. Second, we introduce a multiresolution spatial pooling. This pooling encodes both local and global spatial information to produce discriminative image signatures. Classification results are reported on three image data sets: Caltech101, Caltech256, and fifteen scenes. We show significant improvements over previous architectures using a similar framework.