Cécile Barat

Discriminative Color Descriptors

Color description is a challenging task because of large variations in RGB values which occur due to scene accidental events, such as shadows, shading, specularities, illuminant color changes, and changes in viewing geometry. Traditionally, this challenge has been addressed by capturing the variations in physics-based models, and deriving invariants for the undesired variations. The drawback of this approach is that sets of distinguishable colors in the original color space are mapped to the same value in the photometric invariant space. This results in a drop of discriminative power of the color description. In this paper we take an information theoretic approach to color description. We cluster color values together based on their discriminative power in a classification problem. The clustering has the explicit objective to minimize the drop of mutual information of the final representation. We show that such a color description automatically learns a certain degree of photometric invariance. We also show that a universal color representation, which is based on other data sets than the one at hand, can obtain competing performance. Experiments show that the proposed descriptor outperforms existing photometric invariants. Furthermore, we show that combined with shape description these color descriptors obtain excellent results on four challenging datasets, namely, PASCAL VOC 2007, Flowers-102, Stanford dogs-120 and Birds-200.

Scale-adaptive detection and local characterization of edges based on wavelet transform

In this paper, we present an edge detection and characterization method based on wavelet transform. This method relies on a modelization of contours as smoothed singularities of three particular types (transition, peak and line). Using the wavelet transform modulus maxima lines of the edge models, position and descriptive parameters of each edge point can be inferred. Indeed, on the one hand, the proposed algorithm allows to detect and locate edges at a locally adapted scale. On the other hand, it is able to identify the type of each detected edge point and to measure both its amplitude and smoothness degree. The latter parameters represent, respectively, the contrast and the blur level of the edge point. Evaluation of the method is performed on both synthetic and real images. Synthetic data are used to investigate the influence of different factors and the sensitivity to noise, whereas real images allow to highlight the performance and interests of the method. In particular, we point out that the measured smoothness degree provides a cue to recover depth from defocused images or a cue to diffusion measurements in images of cloud structures. Moreover, from an indoor scene, we demonstrate the relevance of type identification for segmentation purposes.

Pattern matching using morphological probing

In this paper, we introduce two new morphological transforms for pattern matching in gray scale images. They rely on a profiling approach and are defined in the context of mathematical morphology. The first transform allows to detect all occurrences of a single pattern in an image, which justifies the name SOMP (single object matching using probing). It is shown to have the properties of a metric and therefore returns a measure of similarity between the search image and the reference pattern. Other properties relative to noise and computation time are highlighted. The second transform MOMP (multiple objects matching using probing) offers the ability to locate multiple patterns simultaneously. It is particularly suited to the detection of objects varying in size and with noisy distortion. Some results are presented for both transforms.

String representations and distances in deep Convolutional Neural Networks for image classification

Recent advances in image classification mostly rely on the use of powerful local features combined with an adapted image representation. Although Convolutional Neural Network (CNN) features learned from ImageNet were shown to be generic and very efficient, they still lack of flexibility to take into account variations in the spatial layout of visual elements. In this paper, we investigate the use of structural representations on top of pretrained CNN features to improve image classification. Images are represented as strings of CNN features. Similarities between such representations are computed using two new edit distance variants adapted to the image classification domain. Our algorithms have been implemented and tested on several challenging datasets, 15Scenes, Caltech101, Pascal VOC 2007 and MIT indoor. The results show that our idea of using structural string representations and distances clearly improves the classification performance over standard approaches based on CNN and SVM with linear kernel, as well as other recognized methods of the literature. HighlightsA structural representation of images on top of CNN features is proposed.Images are represented as strings to integrate spatial relationships.We introduce tailored string edit distances to compare images represented as strings.Experiments show that our structural approach is more powerful than existing ones.It also outperforms state-of-the-art CNN-based classification methods.

Spatial orientations of visual word pairs to improve Bag-of-Visual-Words model.

This paper presents a novel approach to incorporate spatial information in the bag-of-visual-words model for category level and scene classification. In the traditional bag-of-visual-words model, feature vectors are histograms of visual words. This representation is appearance based and does not contain any information regarding the arrangement of the visual words in the 2D image space. In this framework, we present a simple and effi- cient way to infuse spatial information. Particularly, we are interested in explicit global relationships among the spatial positions of visual words. Therefore, we take advantage of the orientation of the segments formed by Pairs of Identical visual Words (PIW). An evenly distributed normalized histogram of angles of PIW is computed. Histograms pro- duced by each word type constitute a powerful description of intra type visual words relationships. Experiments on challenging datasets demonstrate that our method is com- petitive with the concurrent ones. We also show that, our method provides important complementary information to the spatial pyramid matching and can improve the overall performance.

Spatial histograms of soft pairwise similar patches to improve the bag-of-visual-words model

A new approach to improve image representation for category level classification.We encode pairwise relative spatial information of patches in the bag of word model.A simple approach complementary to the Spatial Pyramid Representation (SPR).Can be combined with SPR and outperforms other existing spatial methods.Experimental validation of the approach is shown on 5 challenging datasets. In the context of category level scene classification, the bag-of-visual-words model (BoVW) is widely used for image representation. This model is appearance based and does not contain any information regarding the arrangement of the visual words in the 2D image space. To overcome this problem, recent approaches try to capture information about either the absolute or the relative spatial location of visual words. In the first category, the so-called Spatial Pyramid Representation (SPR) is very popular thanks to its simplicity and good results. Alternatively, adding information about occurrences of relative spatial configurations of visual words was proven to be effective but at the cost of higher computational complexity, specifically when relative distance and angles are taken into account. In this paper, we introduce a novel way to incorporate both distance and angle information in the BoVW representation. The novelty is first to provide a computationally efficient representation adding relative spatial information between visual words and second to use a soft pairwise voting scheme based on the distance in the descriptor space. Experiments on challenging data sets MSRC-2, 15Scene, Caltech101, Caltech256 and Pascal VOC 2007 demonstrate that our method outperforms or is competitive with concurrent ones. We also show that it provides important complementary information to the spatial pyramid matching and can improve the overall performance.

Fisher Linear Discriminant Analysis for text-image combination in multimedia information retrieval

With multimedia information retrieval, combining different modalities - text, image, audio or video provides additional information and generally improves the overall system performance. For this purpose, the linear combination method is presented as simple, flexible and effective. However, it requires to choose the weight assigned to each modality. This issue is still an open problem and is addressed in this paper.Our approach, based on Fisher Linear Discriminant Analysis, aims to learn these weights for multimedia documents composed of text and images. Text and images are both represented with the classical bag-of-words model. Our method was tested over the ImageCLEF datasets 2008 and 2009. Results demonstrate that our combination approach not only outperforms the use of the single textual modality but provides a nearly optimal learning of the weights with an efficient computation. Moreover, it is pointed out that the method allows to combine more than two modalities without increasing the complexity and thus the computing time. HighlightsWe model text and image documents with bag-of-words approach.We Fisher LDA for learning weights assigned to each modality.We experiment our model on ImageCLEF datasets 2008 and 2009.Our model outperforms the use of the single textual modality.Our method provides a nearly optimal learning with an efficient computation.

Virtual double-sided image probing: A unifying framework for non-linear grayscale pattern matching

This paper focuses on non-linear pattern matching transforms based on mathematical morphology for gray level image processing. Our contribution is on two fronts. First, we unify the existing and a priori unconnected approaches to this problem by establishing their theoretical links with topology. Setting them within the same context allows to highlight their differences and similarities, and to derive new variants. Second, we develop the concept of virtual double-sided image probing (VDIP), a broad framework for non-linear pattern matching in grayscale images. VDIP extends our work on the multiple object matching using probing (MOMP) transform we previously defined to locate multiple grayscale patterns simultaneously. We show that available methods as well as the topological approach can be generalized within the VDIP framework. They can be formulated as particular variants of a general transform designed for virtual probing. Furthermore, a morphological metric, called SVDIP (single VDIP), is deduced from the VDIP concept. Some results are presented and compared with those obtained with classical methods.

Line pattern segmentation using morphological probing

In this paper, we propose a new algorithm for line pattern segmentation. It allows to recover line structures of various widths and intensities and removes structures other than line structures, effects of non-uniformity of the background level and noise. This algorithm is composed of two stages : line enhancement and line extraction. Firstly, line enhancement is carried out using a pattern matching method, relying on a profiling approach and defined with mathematical morphology operators. This transform allows to detect multiple objects of varying size or shape simultaneously using a unique template. For line segmentation, this template is divided into three parallel flat segments. Two lateral segments are used to probe the line from above while a central one probes the line from below. Several parameters describe the template and allow to adapt to various applications. Secondly, line extraction is performed by thinning and thresholding. Results on synthetic line patterns images as well as on real biomedical images are presented.

UJM at ImageCLEFwiki 2008

This paper reports our multimedia information retrieval experiments carried out for the ImageCLEF track (ImageCLEFwiki[10]). We propose a new multimedia model combining textual and/or visual information which enables to perform textual, visual, or multimedia queries. We experiment the model on ImageCLEF data and we compare the results obtained using the different modalities. Our multimedia document model is based on a vector of textual and visual terms. Textual terms correspond to textual words while the visual ones are computed using local colour features. We obtain good results using only the textual part and we show that the visual information is useful in some particular cases.