Olivier Buisson

Random maximum margin hashing

Following the success of hashing methods for multidimensional indexing, more and more works are interested in embedding visual feature space in compact hash codes. Such approaches are not an alternative to using index structures but a complementary way to reduce both the memory usage and the distance computation cost. Several data dependent hash functions have notably been proposed to closely fit data distribution and provide better selectivity than usual random projections such as LSH. However, improvements occur only for relatively small hash code sizes up to 64 or 128 bits. As discussed in the paper, this is mainly due to the lack of independence between the produced hash functions. We introduce a new hash function family that attempts to solve this issue in any kernel space. Rather than boosting the collision probability of close points, our method focus on data scattering. By training purely random splits of the data, regardless the closeness of the training samples, it is indeed possible to generate consistently more independent hash functions. On the other side, the use of large margin classifiers allows to maintain good generalization performances. Experiments show that our new Random Maximum Margin Hashing scheme (RMMH) outperforms four state-of-the-art hashing methods, notably in kernel spaces.

A posteriori multi-probe locality sensitive hashing

Efficient high-dimensional similarity search structures are essential for building scalable content-based search systems on feature-rich multimedia data. In the last decade, Locality Sensitive Hashing (LSH) has been proposed as indexing technique for approximate similarity search. Among the most recent variations of LSH, multi-probe LSH techniques have been proved to overcome the overlinear space cost drawback of common LSH. Multi-probe LSH is built on the well-known LSH technique, but it intelligently probes multiple buckets that are likely to contain query results in a hash table. Our method is inspired by previous work on probabilistic similarity search structures and improves upon recent theoretical work on multi-probe and query adaptive LSH. Whereas these methods are based on likelihood criteria that a given bucket contains query results, we define a more reliable a posteriori model taking account some prior about the queries and the searched objects. This prior knowledge allows a better quality control of the search and a more accurate selection of the most probable buckets. We implemented a nearest neighbors search based on this paradigm and performed experiments on different real visual features datasets. We show that our a posteriori scheme outperforms other multi-probe LSH while offering a better quality control. Comparisons to the basic LSH technique show that our method allows consistent improvements both in space and time efficiency.

Logo retrieval with a contrario visual query expansion

This paper presents a new content-based retrieval framework applied to logo retrieval in large natural image collections. The first contribution is a new challenging dataset, called BelgaLogos, which was created in collaboration with professionals of a press agency, in order to evaluate logo retrieval technologies in real-world scenarios. The second and main contribution is a new visual query expansion method using an a contrario thresholding strategy in order to improve the accuracy of expanded query images. Whereas previous methods based on the same paradigm used a purely hand tuned fixed threshold, we provide a fully adaptive method enhancing both genericity and effectiveness. This new technique is evaluated on both OxfordBuilding dataset and our new BelgaLogos dataset.

Detection and removal of line scratches in motion picture films

Line scratches are common degradations in motion picture films. This paper presents an efficient method for line scratches detection strengthened by a Kalman filter. A new interpolation technique, dealing with both low and high frequencies (i.e. film grain) around the line artifacts, is investigated to achieve a nearby invisible reconstruction of damaged areas. Our line scratches detection and removal techniques have been validated on several film sequences.

Reconstruction of degraded image sequences. Application to film restoration

Abstract A suitable detection–reconstruction approach is proposed for removing impulsive distortion and other types of deterioration from degraded image sequences. The main application that has motivated this work is the problem of digital film restoration for the movie industry, which has only very recently been explored. Line artifacts, which are prominent degradations in motion picture films, are also considered here. The detection procedure consists of two steps. First, a morphological filter provides impulsive distortions and line scratch candidates. Unlike impulsive distortions, which appear randomly in an image, line artifacts persist in nearby or the same location across several frames. Furthermore, the detection process is complicated by the fact that lines occur as natural part in interesting scenes. Therefore, we add a validation step for separating possible line defects from false detections. It consists in tracking the potential line artifacts over the frames using a Kalman filter. An interpolation technique, dealing with both low and high frequencies around the detected deteriorations, is investigated to achieve a nearly invisible reconstruction of damaged areas.

Deterioration detection for digital film restoration

This paper presents a robust technique to detect local deteriorations of old cinematographic films. This method relies on spatio-temporal information and combines two different detectors: a morphological detector which uses spatial properties of deteriorations, and a dynamic detector based on motion estimation techniques. Our deterioration detector has been validated on several film sequences and turned out to be a powerful tool for digital film restoration.

Scalable mining of small visual objects

This paper presents a scalable method for automatically discovering frequent visual objects in large multimedia collections even if their size is very small. It first formally revisits the problem of mining or discovering such objects, and then generalizes two kinds of existing methods for probing candidate object seeds: weighted adaptive sampling and hashing-based methods. The idea is that the collision frequencies obtained with hashing-based methods can actually be converted into a prior probability density function given as input to a weighted adaptive sampling algorithm. This allows for an evaluation of any hashing scheme effectiveness in a more generalized way, and a comparison with other priors, e.g. guided by visual saliency concerns. We then introduce a new hashing strategy, working first at the visual level, and then at the geometric level. This strategy allows us to integrate weak geometric constraints into the hashing phase itself and not only neighborhood constraints as in previous works. Experiments conducted on a new dataset introduced in this paper will show that using this new hashing-based prior allows a drastic reduction of the number of tentative probes required to discover small objects instantiated several times in a large dataset.

Consistent visual words mining with adaptive sampling

State-of-the-art large-scale object retrieval systems usually combine efficient Bag-of-Words indexing models with a spatial verification re-ranking stage to improve query performance. In this paper we propose to directly discover spatially verified visual words as a batch process. Contrary to previous related methods based on feature sets hashing or clustering, we suggest not trading recall for efficiency by sticking on an accurate two-stage matching strategy. The problem then rather becomes a sampling issue: how to effectively and efficiently select relevant query regions while minimizing the number of tentative probes? We therefore introduce an adaptive weighted sampling scheme, starting with some prior distribution and iteratively converging to unvisited regions. Interestingly, the proposed paradigm is generalizable to any input prior distribution, including specific visual concept detectors or efficient hashing-based methods. We show in the experiments that the proposed method allows to discover highly interpretable visual words while providing excellent recall and image representativity.

Motion compensated film restoration

Abstract. Motion picture films are susceptible to local degradations such as dust spots. Other deteriorations are global such as intensity and spatial jitter. It is obvious that motion needs to be compensated for before the detection/correction of such local and dynamic defects. Therefore, we propose a hierarchical motion estimation method ideally suited for high resolution film sequences. This recursive block-based motion estimator relies on an adaptive search strategy and Radon projections to improve processing speed. The localization of dust particles then becomes straightforward. Thus, it is achieved by simple inter-frame differences between the current image and motion compensated successive and preceding frames. However, the detection of spatial and intensity jitter requires a specific process taking advantage of the high temporal correlation in the image sequence. In this paper, we present our motion compensation-based algorithms for removing dust spots, spatial and intensity jitter in degraded motion pictures. Experimental results are presented showing the usefulness of our motion estimator for film restoration at reasonable computational costs.

Video exploration: from multimedia content analysis to interactive visualization

This paper presents 3 interfaces to access video contents. The stream explorer allows to explore and to segment video streams. The video explorer shows a synthetic view of structured TV programmes. The collection explorer proposes cartographies of large video collections. Based on visual and textual automatic processing, proximities and redundancies are analyzed, allowing the emergence of different levels of structure. This is made possible thanks to the volume of data considered: 7 channels during 100 days, ie 16000 hours or 20 Millions key frames. These three tools allow efficient exploration of video contents at different levels of interest: image, shot and sequence, programme and collection level.