Olivier Barnich

ViBe: A Universal Background Subtraction Algorithm for Video Sequences

This paper presents a technique for motion detection that incorporates several innovative mechanisms. For example, our proposed technique stores, for each pixel, a set of values taken in the past at the same location or in the neighborhood. It then compares this set to the current pixel value in order to determine whether that pixel belongs to the background, and adapts the model by choosing randomly which values to substitute from the background model. This approach differs from those based upon the classical belief that the oldest values should be replaced first. Finally, when the pixel is found to be part of the background, its value is propagated into the background model of a neighboring pixel. We describe our method in full details (including pseudo-code and the parameter values used) and compare it to other background subtraction techniques. Efficiency figures show that our method outperforms recent and proven state-of-the-art methods in terms of both computation speed and detection rate. We also analyze the performance of a downscaled version of our algorithm to the absolute minimum of one comparison and one byte of memory per pixel. It appears that even such a simplified version of our algorithm performs better than mainstream techniques.

ViBE: A powerful random technique to estimate the background in video sequences

Background subtraction is a crucial step in many automatic video content analysis applications. While numerous acceptable techniques have been proposed so far for background extraction, there is still a need to produce more efficient algorithms in terms of adaptability to multiple environments, noise resilience, and computation efficiency. In this paper, we present a powerful method for background extraction that improves in accuracy and reduces the computational load. The main innovation concerns the use of a random policy to select values to build a samples-based estimation of the background. To our knowledge, it is the first time that a random aggregation is used in the field of background extraction. In addition we propose a novel policy that propagates information between neighboring pixels of an image. Experiment detailed in this paper show how our method improves on other widely used techniques, and how it outperforms these techniques for noisy images.

Combining Color, Depth, and Motion for Video Segmentation

This paper presents an innovative method to interpret the content of a video scene using a depth camera. Cameras that provide distance instead of color information are part of a promising young technology but they come with many difficulties: noisy signals, small resolution, and ambiguities, to cite a few. By taking advantage of the robustness to noise of a recent background subtraction algorithm, our method is able to extract useful information from the depth signals. We further enhance the robustness of the algorithm by combining this information with that of an RGB camera. In our experiments, we demonstrate this increased robustness and conclude by showing a practical example of an immersive application taking advantage of our algorithm.

Frontal-view gait recognition by intra- and inter-frame rectangle size distribution

Current trends seem to accredit gait as a sensible biometric feature for human identification, at least in a multimodal system. In addition to being a robust feature, gait is hard to fake and requires no cooperation from the user. As in many video systems, the recognition confidence relies on the angle of view of the camera and on the illumination conditions, inducing a sensitivity to operational conditions that one may wish to lower. In this paper we present an efficient approach capable of recognizing people in frontal-view video sequences. The approach uses an intra-frame description of silhouettes which consists of a set of rectangles that will fit into any closed silhouette. A dynamic, inter-frame, dimension is then added by aggregating the size distributions of these rectangles over multiple successive frames. For each new frame, the inter-frame gait signature is updated and used to estimate the identity of the person detected in the scene. Finally, in order to smooth the decision on the identity, a majority vote is applied to previous results. In the final part of this article, we provide experimental results and discuss the accuracy of the classification for our own database of 21 known persons, and for a public database of 25 persons.

Robust analysis of silhouettes by morphological size distributions

We address the topic of real-time analysis and recognition of silhouettes. The method that we propose first produces object features obtained by a new type of morphological operators, which can be seen as an extension of existing granulometric filters, and then insert them into a tailored classification scheme. Intuitively, given a binary segmented image, our operator produces the set of all the largest rectangles that can be wedged inside any connected component of the image. The latter are obtained by a standard background subtraction technique and morphological filtering. To classify connected components into one of the known object categories, the rectangles of a connected component are submitted to a machine learning algorithm called EXtremely RAndomized trees (Extra-trees). The machine learning algorithm is fed with a static database of silhouettes that contains both positive and negative instances. The whole process, including image processing and rectangle classification, is carried out in real-time. Finally we evaluate our approach on one of todays hot topics: the detection of human silhouettes. We discuss experimental results and show that our method is stable and computationally effective. Therefore, we assess that algorithms like ours introduce new ways for the detection of humans in video sequences.

An evaluation of pixel-based methods for the detection of floating objects on the sea surface

Ship-based automatic detection of small floating objects on an agitated sea surface remains a hard problem. Our main concern is the detection of floating mines, which proved a real threat to shipping in confined waterways during the first Gulf War, but applications include salvaging, search-and-rescue operation, perimeter, or harbour defense. Detection in infrared (IR) is challenging because a rough sea is seen as a dynamic background of moving objects with size order, shape, and temperature similar to those of the floating mine. In this paper we have applied a selection of background subtraction algorithms to the problem, and we show that the recent algorithms such as ViBe and behaviour subtraction, which take into account spatial and temporal correlations within the dynamic scene, significantly outperform the more conventional parametric techniques, with only little prior assumptions about the physical properties of the scene.

Design of statistical measures for the assessment of image segmentation schemes

Image segmentation is discussed for years in numerous papers, but assessing its quality is mainly dealt with in recent works. Quality assessment is a primary concern for anyone working towards better segmentation tools. It both helps to objectively improve segmentation techniques and to compare performances with respect to other similar algorithms. In this paper we use a statistical framework to propose statistical measures capable to describe the performances of a segmentation scheme. All the measures rely on a ground-truth segmentation map that is supposed to be known and that serves as a reference when qualifying the results of any segmentation tool. We derive the analytical expression of several transition probabilities and show how to calculate them. An important conclusion from our study, often overlooked, is that performances can be content dependent, which means that one should adapt a measure to the content of an image.

A platform for the fast interpretation of movements and localization of users in 3D applications driven by a range camera

Interactivity is one of the key challenges for immersive applications like gaming. Manufacturers have been working towards interfaces that are driven by a device (e.g. a Wiimote) or interfaces that are controlled by a camera with a subsequent computer vision module. Both approaches have unique advantages, but they do not permit to localize users in the scene with an appropriate accuracy. Therefore, we propose to use both a range camera and accurate range sensors to enable the interpretation of movements. This paper describes a platform that uses a range camera to acquire the silhouettes of users, regardless of illumination, and to improve the pose recovery with range information after some image processing steps. In addition, to circumvent the difficult process of calibration required to map range values to physical distances, we complete the system with several range laser sensors. These sensors are located in a horizontal plane, and measure distances up to a few centimeters. We combine all these measurements to obtain a localization map, used to locate users in the scene at a negligible computational cost. Our method fills a gap in 3D applications that require absolute positions.

A Virtual Curtain for the Detection of Humans and Access Control

Biometrics has become a popular field for the development of techniques that aim at recognizing humans based upon one or more intrinsic physical or behavioral traits. In particular, many solutions dedicated to access control integrate biometric features like fingerprinting or face recognition.

Design of a morphological moving object signature and application to human identification

Many computer vision systems try to infer semantic information about a video scene content by looking at the time series of the silhouettes of the moving objects. This paper proposes a new inter-frame feature set (signature) based on piecewise surfacic descriptions of binary silhouettes. It captures the dynamics of moving objects and compacts it into a robust set of features suitable for classification. To assess its ability to represent motion information, we use it to build a complete gait recognition algorithm that we test on a database of 21 different subjects. To highlight the efficiency of our signature, we use frontal views instead of side views of persons, which is less discussed in literature and is considered to be harder as the movement of legs is not visible. In that context, the high recognition rates obtained (over 95% of correct identifications) proves that our signature is appropriate to describe moving objects.