Benoit Huet

Line pattern retrieval using relational histograms

This paper presents a new compact shape representation for retrieving line-patterns from large databases. The basic idea is to exploit both geometric attributes and structural information to construct a shape histogram. We realize this goal by computing the N-nearest neighbor graph for the lines-segments for each pattern. The edges of the neighborhood graphs are used to gate contributions to a two-dimensional pairwise geometric histogram. Shapes are indexed by searching for the line-pattern that maximizes the cross correlation of the normalized histogram bin-contents. We evaluate the new method on a database containing over 2,500 line-patterns each composed of hundreds of lines.

Finding media illustrating events

We present a method combining semantic inferencing and visual analysis for finding automatically media (photos and videos) illustrating events. We report on experiments validating our heuristic for mining media sharing platforms and large event directories in order to mutually enrich the descriptions of the content they host. Our overall goal is to design a web-based environment that allows users to explore and select events, to inspect associated media, and to discover meaningful, surprising or entertaining connections between events, media and people participating in events. We present a large dataset composed of semantic descriptions of events, photos and videos interlinked with the larger Linked Open Data cloud and we show the benefits of using semantic web technologies for integrating multimedia metadata.

Automatic video summarization

In this paper, we present a new approach for the automatic construction of video summaries. We introduce the Simulated User Principle to evaluate the quality of a video summary in a way which is automatic, yet related to user perception. We present experimental results to support our ideas.

Relational object recognition from large structural libraries

This paper presents a probabilistic similarity measure for object recognition from large libraries of line-patterns. We commence from a structural pattern representation which uses a nearest neighbour graph to establish the adjacency ofline-segments. Associated with each pair ofline-segments connected in this way is a vector ofEuclidean invariant relative angle and distance ratio attributes. The relational similarity measure uses robust error kernels to compare sets ofpairwise attributes on the edges ofa nearest neighbour graph. We use the relational similarity measure in a series ofrecognition experiments which involve a library ofover 2500 line-patterns. A sensitivity study reveals that the method is capable ofdelivering a recognition accuracy of94%. A comparative study reveals that the method is most e2ective when either a Gaussian kernel or Huber’s robust kernel is used to weight the attribute relations. Moreover, the method consistently outperforms the standard and the quantile Hausdor2 distance. ? 2002 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.

Toward emotion indexing of multimedia excerpts

Multimedia indexing is about developing techniques allowing people to effectively find media. Content-based methods become necessary when dealing with large databases. Current technology allows exploring the emotional space which is known to carry very interesting semantic information. In this paper we state the need for an integrated method which extracts reliable affective information and attaches this semantic information to the medium itself. We describe SAMMI [1], a framework explicitly designed to fulfill this need and we present a list of possible applications pointing out the advantages that the emotional information can bring about. Finally, different scenarios are considered for the recognition of the emotions which involve different modalities, feature sets, fusion algorithms, and result optimization methods such as temporal averaging or thresholding.

Relational histograms for shape indexing

This paper is concerned with the retrieval of images from large databases based on their shape similarity to a query image. Our approach is based on two dimensional histograms that encode both the local and global geometric properties of the shapes. The pairwise attributes are the directed segment relative angle and directed relative position. The novelty of the proposed approach is to simultaneously use the relational and structural constraints, derived from an adjacency graph, to gate histogram contributions. We investigate the retrieval capabilities of the method for various queries. We also investigate the robustness of the method to segmentation errors. We conclude that a relational histogram of pairwise segment attributes presents a very efficient way of indexing into large databases. The optimal configuration is obtained when the local features are constructed from six neighbouring segments pairs. Moreover, a sensitivity analysis reveals that segmentation errors do not affect the retrieval performances.

Shape recognition from large image libraries by inexact graph matching

Abstract This paper describes a graph-matching technique for recognising line-pattern shapes in large image databases. The methodological contribution of the paper is to develop a Bayesian matching algorithm that uses edge-consistency and node attribute similarity. This information is used to determine the a posteriori probability of a query graph for each of the candidate matches in the database. The node feature-vectors are constructed by computing normalised histograms of pairwise geometric attributes. Attribute similarity is assessed by computing the Bhattacharyya distance between the histograms. Recognition is realised by selecting the candidate from the database which has the largest a posteriori probability.

Heterogeneous features and model selection for event-based media classification

With the rapid development of social media sites, a lot of user generated content is being shared in the Web, leading to new challenges for traditional media retrieval techniques. An event describes the happening at a specific time and place in real-world, and it is one of the most important cues for people to recall past memories. The reminder value of an event makes it extremely helpful in organizing human life. Thus, organizing media by events has recently drawn much attention within the multimedia research community. In this paper, we focus on two fundamental problems related to event based social media analysis: the study of feature importance for modeling the relation between events and media, and how to deal with missing and erroneous metadata often present in social media data. These issues are studied within an event-based media classification framework. Different learning approaches are employed to train the event models on different features. We find, through experiments on a large set of events, that the best discriminant features are tags, spatial and temporal feature. We address the missing value problem by extending the feature with an extra attribute to indicate if the values are missing. Promising results are achieved demonstrating the effectiveness of the proposed method.

Evidence Theory-Based Multimodal Emotion Recognition

Automatic recognition of human affective states is still a largely unexplored and challenging topic. Even more issues arise when dealing with variable quality of the inputs or aiming for real-time, unconstrained, and person independent scenarios. In this paper, we explore audio-visual multimodal emotion recognition. We present SAMMI, a framework designed to extract real-time emotion appraisals from non-prototypical, person independent, facial expressions and vocal prosody. Different probabilistic method for fusion are compared and evaluated with a novel fusion technique called NNET. Results shows that NNET can improve the recognition score (CR + ) of about 19% and the mean average precision of about 30% with respect to the best unimodal system.

Neural network combining classifier based on dempster-shafer theory for semantic indexing in video content

Classification is a major task in many applications and in particular for automatic semantic-based video content indexing and retrieval. In this paper, we focus on the challenging task of classifier output fusion. It is a necessary step to efficiently estimate the semantic content of video shots from multiple cues. We propose to fuse the numeric information provided by multiple classifiers in the framework of evidence logic. For this purpose, an improved version of RBF network based on Evidence Theory (NN-ET) is proposed. Experiments are conducted in the framework of TrecVid high level feature extraction task that consists of ordering shots with respect to their relevance to a given semantic class.