Jean Martinet

Toward a higher-level visual representation for content-based image retrieval

Having effective methods to access the desired images is essential nowadays with the availability of a huge amount of digital images. The proposed approach is based on an analogy between content-based image retrieval and text retrieval. The aim of the approach is to build a meaningful mid-level representation of images to be used later on for matching between a query image and other images in the desired database. The approach is based firstly on constructing different visual words using local patch extraction and fusion of descriptors. Secondly, we introduce a new method using multilayer pLSA to eliminate the noisiest words generated by the vocabulary building process. Thirdly, a new spatial weighting scheme is introduced that consists of weighting visual words according to the probability of each visual word to belong to each of the n Gaussian. Finally, we construct visual phrases from groups of visual words that are involved in strong association rules. Experimental results show that our approach outperforms the results of traditional image retrieval techniques.

Semantics for intelligent delivery of multimedia content

This paper describes a new generic metadata model, called CAM Metamodel, that merges altogether information about content, services, physical and technical environment in order to enable homogenous delivery and consumption of content. We introduce a metadata model that covers all these aspects and which can be easily extended so as to absorb new types of models and standards. We ensure this flexibility by introducing an abstract metamodel, which defines structured archetypes for metadata and metadata containers. The metamodel is the foundation for the technical metadata specification. We also introduce new structures in the abstract and core metamodels supporting the management of distributed community created metadata.

A model for weighting image objects in home photographs

The paper presents a contribution to image indexing consisting in a weighting model for visible objects -- or image objects -- in home photographs. To improve its effectiveness this weighting model has been designed according to human perception criteria about what is estimated as important in photographs. Four basic hypotheses related to human perception are presented, and their validity is estimated as compared to actual observations from a user study. Finally a formal definition of this weighting model is presented and its consistence with the user study is evaluated.

Unsupervised face identification in TV content using audio-visual sources

Our goal is to automatically identify faces in TV content without pre-defined dictionary of identities. Most of methods are based on identity detection (from OCR and ASR) and require a propagation strategy based on visual clusterings. In TV content, people appear with many variation making the clustering very difficult. In this case, identifying speakers can be a reliable link to identify faces. In this work, we propose to combine reliable unsupervised face and speaker identification systems through talking-faces detection in order to improve face identification results. First, OCR and ASR results are combined to extract locally the identities. Then, the reliable visual associations are used to propagate those identities locally. The reliable identified faces are used as unsupervised models to identify similar faces. Finally speaker identities are propagated to the faces in case of lip activity detection. Experiments performed on the REPERE database show an improvement of the recall of +5% compared to the baseline, without degrading the precision.

Intelligent pixels of interest selection with application to facial expression recognition using multilayer perceptron

This paper presents an automatic way to discover pixels in a face image that improves the facial expression recognition results. Main contribution of our study is to provide a practical method to improve classification performance of classifiers by selecting best pixels of interest. Our method exhaustively searches for the best and worst feature window position from a set of face images among all possible combinations using MLP. Then, it creates a non-rectangular emotion mask for feature selection in supervised facial expression recognition problem. It eliminates irrelevant data and improves the classification performance using backward feature elimination. Experimental studies on GENKI, JAFFE and FERET databases showed that the proposed system improves the classification results by selecting the best pixels of interest.

Boosting gender recognition performance with a fuzzy inference system

We used both inner and outer face cues.External cues improve classification performance for gender recognition.FIS framework improves classification results when combined with SVM.Unconstrained databases provide better results than that of constrained databases.We obtained 93.35% accuracy on Groups/LFW cross-database test. In this paper, we propose a novel gender recognition framework based on a fuzzy inference system (FIS). Our main objective is to study the gain brought by FIS in presence of various visual sensors (e.g., hair, mustache, inner face). We use inner and outer facial features to extract input variables. First, we define the fuzzy statements and then we generate a knowledge base composed of a set of rules over the linguistic variables including hair volume, mustache and a vision-sensor. Hair volume and mustache information are obtained from Part Labels subset of Labeled Faces in the Wild (LFW) database and vision-sensor is obtained from a pixel-intensity based SVM+RBF classifier trained on different databases including Feret, Groups and GENKI-4K. Cross-database test experiments on LFW database showed that the proposed method provides better accuracy than optimized SVM+RBF only classification. We also showed that FIS increases the inter-class variability by decreasing false negatives (FN) and false positives (FP) using expert knowledge. Our experimental results yield an average accuracy of 93.35% using Groups/LFW test, while the SVM performance baseline yields 91.25% accuracy.

A new spatial weighting scheme for bag-of-visual-words

In this paper, we develop a novel image representation method which is based firstly on constructing visual words based on a local patch extraction and a fusion of descriptors. The spatial constitution of an image is represented with a mixture of n Gaussians in the feature space. The new spatial weighting scheme consists in weighting visual words according to the probability of each visual word belongs to each of the n Gaussians. Experimental results show that the proposed approach integrated to an image retrieval system, outperforms the results of traditional image retrieval techniques.

A relational vector space model using an advanced weighting scheme for image retrieval

In this paper, we lay out a relational approach for indexing and retrieving photographs from a collection. The increase of digital image acquisition devices, combined with the growth of the World Wide Web, requires the development of information retrieval (IR) models and systems that provide fast access to images searched by users in databases. The aim of our work is to develop an IR model suited to images, integrating rich semantics for representing this visual data and user queries, which can also be applied to large corpora. Our proposal merges the vector space model of IR - widely tested in textual IR - with the conceptual graph (CG) formalism, based on the use of star graphs (i.e. elementary CGs made up of a single relation connected to some concepts representing image objects). A novel weighting scheme for star graphs, based on image objects size, position, and image heterogeneity is outlined. We show that integrating relations into the vector space model through star graphs increases the systems precision, and that the results are comparable to those from graph projection systems, and also that they shorten processing time for user queries.

A Study of Intra-Modal Association Rules for Visual Modality Representation

This paper presents a study of the use of association rules in document representation. It consists of an approach for building a compact and meaningful representation of low level features in the documents of a multimedia database. The proposed method is based on the discovery and the study of intra-modal association rules between individual objects in the context of a spatial-temporal window. The discovered relations are used to build mid-level objects capturing the most frequently occurring patterns in the database, possibly closer to high level concepts. We show the results of an evaluation assessing the quality of the representation with regards to its compactness.

DLBP: A novel descriptor for depth image based face recognition

This paper presents a novel descriptor for face depth images, generalizing the well-known Local Binary Pattern (LBP), in order to enhance its discriminative power for smooth depth images. The proposed descriptor is based on detecting shape patterns from face surfaces and enables accurate and fast description of shape variation in depth images. It is in the same form as conventional LBP, so patterns can be readily combined to form joint histograms to represent depth faces. The descriptor is computationally very simple, rapid and it is totally training-free. When we associate our descriptor in a face recognition scheme based on nearest neighbor classifier, it shows its discriminative power in depth based face recognition comparing to the conventional LBP and other extensions proposed for 3D face recognition. Many experiments are conducted on different databases in order to evaluate our method.