Guillaume-Alexandre Bilodeau

SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity

Foreground/background segmentation via change detection in video sequences is often used as a stepping stone in high-level analytics and applications. Despite the wide variety of methods that have been proposed for this problem, none has been able to fully address the complex nature of dynamic scenes in real surveillance tasks. In this paper, we present a universal pixel-level segmentation method that relies on spatiotemporal binary features as well as color information to detect changes. This allows camouflaged foreground objects to be detected more easily while most illumination variations are ignored. Besides, instead of using manually set, frame-wide constants to dictate model sensitivity and adaptation speed, we use pixel-level feedback loops to dynamically adjust our methods internal parameters without user intervention. These adjustments are based on the continuous monitoring of model fidelity and local segmentation noise levels. This new approach enables us to outperform all 32 previously tested state-of-the-art methods on the 2012 and 2014 versions of the ChangeDetection.net dataset in terms of overall F-Measure. The use of local binary image descriptors for pixel-level modeling also facilitates high-speed parallel implementations: our own version, which used no low-level or architecture-specific instruction, reached real-time processing speed on a midlevel desktop CPU. A complete C++ implementation based on OpenCV is available online.

Flexible Background Subtraction with Self-Balanced Local Sensitivity

Most background subtraction approaches offer decent results in baseline scenarios, but adaptive and flexible solutions are still uncommon as many require scenario-specific parameter tuning to achieve optimal performance. In this paper, we introduce a new strategy to tackle this problem that focuses on balancing the inner workings of a non-parametric model based on pixel-level feedback loops. Pixels are modeled using a spatiotemporal feature descriptor for increased sensitivity. Using the video sequences and ground truth annotations of the 2012 and 2014 CVPR Change Detection Workshops, we demonstrate that our approach outperforms all previously ranked methods in the original dataset while achieving good results in the most recent one.

An iterative integrated framework for thermal-visible image registration, sensor fusion, and people tracking for video surveillance applications

In this work, we propose a new integrated framework that addresses the problems of thermal-visible video registration, sensor fusion, and people tracking for far-range videos. The video registration is based on a RANSAC trajectory-to-trajectory matching, which estimates an affine transformation matrix that maximizes the overlapping of thermal and visible foreground pixels. Sensor fusion uses the aligned images to compute sum-rule silhouettes, and then constructs thermal-visible object models. Finally, multiple object tracking uses blobs constructed in sensor fusion to output the trajectories. Results demonstrate the advantage of our proposed framework in obtaining better results for both image registration and tracking than separate image registration and tracking methods.

Improving background subtraction using Local Binary Similarity Patterns

Most of the recently published background subtraction methods can still be classified as pixel-based, as most of their analysis is still only done using pixel-by-pixel comparisons. Few others might be regarded as spatial-based (or even spatiotemporal-based) methods, as they take into account the neighborhood of each analyzed pixel. Although the latter types can be viewed as improvements in many cases, most of the methods that have been proposed so far suffer in complexity, processing speed, and/or versatility when compared to their simpler pixel-based counterparts. In this paper, we present an adaptive background subtraction method, derived from the low-cost and highly efficient ViBe method, which uses a spatiotemporal binary similarity descriptor instead of simply relying on pixel intensities as its core component. We then test this method on multiple video sequences and show that by only replacing the core component of a pixel-based method it is possible to dramatically improve its overall performance while keeping memory usage, complexity and speed at acceptable levels for online applications.

A Multiscale Region-Based Motion Detection and Background Subtraction Algorithm

This paper presents a region-based method for background subtraction. It relies on color histograms, texture information, and successive division of candidate rectangular image regions to model the background and detect motion. Our proposed algorithm uses this principle and combines it with Gaussian Mixture background modeling to produce a new method which outperforms the classic Gaussian Mixture background subtraction method. Our method has the advantages of filtering noise during image differentiation and providing a selectable level of detail for the contour of the moving shapes. The algorithm is tested on various video sequences and is shown to outperform state-of-the-art background subtraction methods.

Change Detection in Feature Space Using Local Binary Similarity Patterns

In general, the problem of change detection is studied in color space. Most proposed methods aim at dynamically finding the best color thresholds to detect moving objects against a background model. Background models are often complex to handle noise affecting pixels. Because the pixels are considered individually, some changes cannot be detected because it involves groups of pixels and some individual pixels may have the same appearance as the background. To solve this problem, we propose to formulate the problem of background subtraction in feature space. Instead of comparing the color of pixels in the current image with colors in a background model, features in the current image are compared with features in the background model. The use of a feature at each pixel position allows accounting for change affecting groups of pixels, and at the same time adds robustness to local perturbations. With the advent of binary feature descriptors such as BRISK or FREAK, it is now possible to use features in various applications at low computational cost. We thus propose to perform background subtraction with a small binary descriptor that we named Local Binary Similarity Patterns (LBSP). We show that this descriptor outperforms color, and that a simple background subtractor using LBSP outperforms many sophisticated state of the art methods in baseline scenarios.

Adaptive Fuzzy Particle Filter Tracker for a PTZ Camera in an IP Surveillance System

We propose an adaptive fuzzy particle filter (PF) (AFPF) method adapted to general object tracking with an IP pan-tilt-zoom (PTZ) camera. PF samples are weighted using fuzzy membership functions and are applied to geometric and appearance features. In our PF, targets are modeled and tracked based on sampling around predicted positions obtained by a position predictor and moving regions detected by optical flow. Sample features are scored based on fuzzy rules. In this paper, we apply the AFPF to a human-tracking application in an IP PTZ surveillance system. Results show that our system has good target-detection precision (>; 93.9%), low track fragmentation, and a high processing rate, and the target is almost always located within one-sixth of the image diameter from the image center.

Monitoring of Medication Intake Using a Camera System

This paper presents a computer vision system for monitoring medication intake in the context of home care services. We use a method based on color and shape to detect the body parts and the medication bottles. Color is used for skin detection, and the shape is used to distinguish the face from the hands and to differentiate bottles of medicine. To track these objects, we use a method based on color histograms, Hu moments, and edges. For the recognition of medication intake, we use a Petri network and event recognition. Our method has an accuracy of more than 75% and allows the detection of the medication intake in various scenarios where the user is cooperative.

Local self-similarity-based registration of human ROIs in pairs of stereo thermal-visible videos

For several years, mutual information (MI) has been the classic multimodal similarity measure. The robustness of MI is closely restricted by the choice of MI window sizes. For unsupervised human monitoring applications, obtaining appropriate MI window sizes for computing MI in videos with multiple people in different sizes and different levels of occlusion is problematic. In this work, we apply local self-similarity (LSS) as a dense multimodal similarity metric and show its adequacy and strengths compared to MI for a human ROIs registration. We also propose an LSS-based registration of thermal-visible stereo videos that addresses the problem of multiple people and occlusions in the scene. Our method improves the accuracy of the state-of-the-art disparity voting (DV) correspondence algorithm by proposing a motion segmentation step that approximates depth segments in an image and enables assigning disparity to each depth segment using larger matching window while keeping registration accuracy. We demonstrate that our registration method outperforms the recent state-of-the-art MI-based stereo registration for several realistic close-range indoor thermal-visible stereo videos of multiple people. Highlights? We propose a Local self-similarity (LSS)-based multimodal correspondence measure. ? We study Comparatively MI and LSS for thermal-visible stereo matching. ? We propose an LSS-based registration method for human monitoring. ? LSS-based registration is more accurate compared to MI-based registration.

Urban Tracker: Multiple object tracking in urban mixed traffic

In this paper, we study the problem of detecting and tracking multiple objects of various types in outdoor urban traffic scenes. This problem is especially challenging due to the large variation of road user appearances. To handle that variation, our system uses background subtraction to detect moving objects. In order to build the object tracks, an object model is built and updated through time inside a state machine using feature points and spatial information. When an occlusion occurs between multiple objects, the positions of feature points at previous observations are used to estimate the positions and sizes of the individual occluded objects. Our Urban Tracker algorithm is validated on four outdoor urban videos involving mixed traffic that includes pedestrians, cars, large vehicles, etc. Our method compares favorably to a current state of the art feature-based tracker for urban traffic scenes on pedestrians and mixed traffic.