Valérie Gouet-Brunet

Robust voting algorithm based on labels of behavior for video copy detection

This paper presents an efficient approach for copies detection in a large videos archive consisting of several hundred of hours. The video content indexing method consists of extracting the dynamic behavior on the local description of interest points and further on the estimation of their trajectories along the video sequence. Analyzing the low-level description obtained allows to highlight trends of behaviors and then to assign a label of behavior to each local descriptor. Such an indexing approach has several interesting properties: it provides a rich, compact and generic description, while labels of behavior provide a high-level description of the video content. Here, we focus on video Content Based Copy Detection (CBCD). Copy detection is problematic as similarity search problem but with prominent differences. To be efficient, it requires a dedicated on-line retrieval method based on a specific voting function. This voting function must be robust to signal transformations and discriminating versus high similarities which are not copies. The method we propose in this paper is a dedicated on-line retrieval method based on a combination of the different dynamic contexts computed during the off-line indexing. A spatio-temporal registration based on the relevant combination of detected labels is then applied. This approach is evaluated using a huge video database of 300 hours with different video tests. The method is compared to a state-of-the art technique in the same conditions. We illustrate that taking labels into account in the specific voting process reduces false alarms significantly and drastically improves the precision.

Visual word spatial arrangement for image retrieval and classification

We present word spatial arrangement (WSA), an approach to represent the spatial arrangement of visual words under the bag-of-visual-words model. It lies in a simple idea which encodes the relative position of visual words by splitting the image space into quadrants using each detected point as origin. WSA generates compact feature vectors and is flexible for being used for image retrieval and classification, for working with hard or soft assignment, requiring no pre/post processing for spatial verification. Experiments in the retrieval scenario show the superiority of WSA in relation to Spatial Pyramids. Experiments in the classification scenario show a reasonable compromise between those methods, with Spatial Pyramids generating larger feature vectors, while WSA provides adequate performance with much more compact features. As WSA encodes only the spatial information of visual words and not their frequency of occurrence, the results indicate the importance of such information for visual categorization. HighlightsSpatial arrangement of visual words (WSA) for image retrieval and classification.WSA generates vectors more compact than the traditional spatial pooling methods.WSA outperforms Spatial Pyramids in the retrieval scenario.WSA presents adequate performance in the classification scenario.

Object recognition and segmentation in videos by connecting heterogeneous visual features

We present an approach for model-free and instance-level object recognition and segmentation in cluttered scenes, based on heterogeneous visual features. The first contribution of this work addresses the description of the visual appearance of objects, by proposing the joint use of complementary features of different natures: on the one hand, a set of local descriptors based on interest points that have well-known interesting properties; on the other hand, a global descriptor based on a snake, providing a high-level description of the object shape. Our second contribution consists in efficiently structuring and connecting the visual features obtained, making possible the use of global descriptors without prior segmentation/detection. Our approach is compared to a classic one based on local descriptors only and is evaluated for video surveillance purposes over sequences involving 20 objects. We show that recognition is improved, and provides precise object segmentation, even with large occlusions. A real scenario of application to video surveillance of truck traffic validates the relevance of the approach.

Embedding spatial information into image content description for scene retrieval

This article presents @D-TSR, an image content representation describing the spatial layout with triangular relationships of visual entities, which can be symbolic objects or low-level visual features. A semi-local implementation of @D-TSR is also proposed, making the description robust to viewpoint changes. We evaluate @D-TSR for image retrieval under the query-by-example paradigm, on contents represented with interest points in a bag-of-features model: it improves state-of-the-art techniques, in terms of retrieval quality as well as of execution time, and is scalable. Finally, its effectiveness is evaluated on a topical scenario dedicated to scene retrieval in datasets of city landmarks.

HiPeR : Hierarchical progressive exact retrieval in multi dimensional spaces

In this article, we are interested in accelerating similarity search in high dimensional vector spaces. The presented approach, called HiPeR, is based on a hierarchy of sub- spaces and indexes: it performs nearest neighbors search across spaces of different dimensions, by beginning with the lowest dimensions up to the highest ones, with the aim of minimizing the effects of the curse of dimensionality. HiPeR significantly accelerates exact retrieval even with the best indexes, and also allows for progressive retrieval, i.e. the possibility to provide results to the user progressively with refinements until satisfaction. Scanning the hierarchy can be done according to several strategies. We propose and evaluate two heuristics: the first one supposes an a priori knowledge on the data-set distribution, while the second chooses the most interesting levels at run time. HiPeR is evaluated for range queries on 3 real data-sets varying from 500,000 vectors to 4 millions.

A survey on Visual-Based Localization: On the benefit of heterogeneous data

Abstract We are surrounded by plenty of information about our environment. From these multiple sources, numerous data could be extracted: set of images, 3D model, coloured points cloud... When classical localization devices failed ( e.g. GPS sensor in cluttered environments), aforementioned data could be used within a localization framework. This is called Visual Based Localization (VBL). Due to numerous data types that can be collected from a scene, VBL encompasses a large amount of different methods. This paper presents a survey about recent methods that localize a visual acquisition system according to a known environment. We start by categorizing VBL methods into two distinct families: indirect and direct localization systems. As the localization environment is almost always dynamic, we pay special attention to methods designed to handle appearances changes occurring in a scene. Thereafter, we highlight methods exploiting heterogeneous types of data. Finally, we conclude the paper with a discussion on promising trends that could permit to a localization system to reach high precision pose estimation within an area as large as possible.

Evaluation of strategies for multiple sphere queries with local image descriptors

In this paper, we are interested in the fast retrieval, in a large collection of points in high-dimensional space, of points close to a set of m query points (a multiple query): we want to efficiently find the sequence Ai,iε1,m} where Ai is the set of points within a sphere of center query point pi,iε{1,m} and radius ε (a sphere query). It has been argued that beyond a rather small dimension (d ⩾ 10) for such sphere queries as well as for other similarity queries, sequentially scanning the collection of points is faster than crossing a tree structure indexing the collection (the so-called curse of dimensionality phenomenon). Our first contribution is to experimentally assess whether the curse of dimensionality is reached with various points distributions. We compare the performance of a single sphere query when the collection is indexed by a tree structure (an SR-tree in our experiments) to that of a sequential scan. The second objective of this paper is to propose and evaluate several algorithms for multiple queries in a collection of points indexed by a tree structure. We compare the performance of these algorithms to that of a naive one consisting in sequentially running the m queries. This study is applied to content-based image retrieval where images are described by local descriptors based on points of interest. Such descriptors involve a relatively small dimension (8 to 30) justifying that the collection of points be indexed by a tree structure; similarity search with local descriptors implies multiple sphere queries that are usually time expensive, justifying the proposal of new strategies.

A fast algorithm for music search by similarity in large databases based on modified Symetrized Kullback Leibler Divergence

State of the art on music similarity search is based on the pairwise comparison of statistical models representing audio features. The comparison is often obtained by the Symetrized Kullback-Leibler Divergence (SKLD). When dealing with very large databases (over one million items), usual search by similarity algorithms — sequential or exhaustive search — cannot be used. In these cases, optimized search strategies such as the M-tree reduces the search time but requires the dissimilarity measure to be a metric. Unfortunately, this is not the case of the SKLD. In this paper, we propose and successfully test on a large-scale a modification of the Symetrized Kullback-Leibler Divergence which allows to use it as a metric.

Large scale disk-based metric indexing structure for approximate information retrieval by content

In order to achieve large scalability, indexing structures are usually distributed to incorporate more of expensive main memory during the query processing. In this paper, an indexing structure, that does not suffer from a performance degradation by its transition from main memory storage to hard drive, is proposed. The high efficiency of the index is achieved using a very effective pruning based on precomputed distances and so called locality phenomenon which substantially diminishes the number of retrieved candidates. The trade-offs for the large scalability are, firstly, the approximation and, secondly, longer query times, yet both are still bearable enough for recent multimedia content-based search systems, proved by an evaluation using visual and audio data and both metric and semi-metric distance functions. The tuning of the indexs parameters based on the analysis of the particulars data intrinsic dimensionality is also discussed.

The Many Facets of Progressive Retrieval for CBIR

Recently, progressive retrieval has been advocated as an alternate solution to multidimensional indexes or approximate techniques, in order to accelerate similarity search of points in multidimensional spaces. The principle of progressive search is to offer a first subset of the answers to the user during retrieval. If this subset satisfies the users needs retrieval stops. Otherwise search resumes, and after a number of steps the exact answer set is returned to the user. Such a process is justified by the fact that in a large number of applications it is more interesting to rapidly bring first approximate answer sets rather than waiting for a long time the exact answer set. The contribution of this paper is a first typology of existing techniques for progressive retrieval. We survey a variety of methods designed for image retrieval although some of them apply to a general database browsing context which goes beyond CBIR. We also include techniques not designed for but that can easily be adapted to progressive retrieval.