El-hadi Zahzah

Decomposition into low-rank plus additive matrices for background/foreground separation

Background/foreground separation is the first step in video surveillance system to detect moving objects. Recent research on problem formulations based on decomposition into low-rank plus sparse matrices shows a suitable framework to separate moving objects from the background. The most representative problem formulation is the Robust Principal Component Analysis (RPCA) solved via Principal Component Pursuit (PCP) which decomposes a data matrix into a low-rank matrix and a sparse matrix. However, similar robust implicit or explicit decompositions can be made in the following problem formulations: Robust Non-negative Matrix Factorization (RNMF), Robust Matrix Completion (RMC), Robust Subspace Recovery (RSR), Robust Subspace Tracking (RST) and Robust Low-Rank Minimization (RLRM). The main goal of these similar problem formulations is to obtain explicitly or implicitly a decomposition into low-rank matrix plus additive matrices. These formulation problems differ from the implicit or explicit decomposition, the loss function, the optimization problem and the solvers. As the problem formulation can be NP-hard in its original formulation, and it can be convex or not following the constraints and the loss functions used, the key challenges concern the design of efficient relaxed models and solvers which have to be with iterations as few as possible, and as efficient as possible. In the application of background/foreground separation, constraints inherent to the specificities of the background and the foreground as the temporal and spatial properties need to be taken into account in the design of the problem formulation. Practically, the background sequence is then modeled by a low-rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. Although, many efforts have been made to develop methods for the decomposition into low-rank plus additive matrices that perform visually well in foreground detection with reducing their computational cost, no algorithm today seems to emerge and to be able to simultaneously address all the key challenges that accompany real-world videos. This is due, in part, to the absence of a rigorous quantitative evaluation with synthetic and realistic large-scale dataset with accurate ground truth providing a balanced coverage of the range of challenges present in the real world. In this context, this work aims to initiate a rigorous and comprehensive review of the similar problem formulations in robust subspace learning and tracking based on decomposition into low-rank plus additive matrices for testing and ranking existing algorithms for background/foreground separation. For this, we first provide a preliminary review of the recent developments in the different problem formulations which allows us to define a unified view that we called Decomposition into Low-rank plus Additive Matrices (DLAM). Then, we examine carefully each method in each robust subspace learning/tracking frameworks with their decomposition, their loss functions, their optimization problem and their solvers. Furthermore, we investigate if incremental algorithms and real-time implementations can be achieved for background/foreground separation. Finally, experimental results on a large-scale dataset called Background Models Challenge (BMC 2012) show the comparative performance of 32 different robust subspace learning/tracking methods.

Foreground detection based on low-rank and block-sparse matrix decomposition

Foreground detection is the first step in video surveillance system to detect moving objects. Principal Components Analysis (PCA) shows a nice framework to separate moving objects from the background but without a mechanism of robust analysis, the moving objects may be absorbed into the background model. This drawback can be solved by recent researches on Robust Principal Component Analysis (RPCA). The background sequence is then modeled by a low rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. In this paper, we propose to use a RPCA method based on low-rank and block-sparse matrix decomposition to achieve foreground detection. This decomposition enforces the low-rankness of the background and the block-sparsity aspect of the foreground. Experimental results on different datasets show the pertinence of the proposed approach.

Numeric and symbolic data fusion: a soft computing approach to remote sensing images analysis

Abstract An expert system approach for image classification according to expert knowledge about best sites for vegetation classes is described. Uncertainty management is solved by a certainty factor approach. The numerical and symbolic data fusion is viewed as an updating process. The fusion approach is then described. A neural classifier applied to image data is the first source. A set of fuzzy neural networks representing expert knowledge constitutes the second source. A conjunctive combination based on evidence theory is applied. Finally, a possibility theory-based pooling aggregation rule is presented. These three approaches are applied to a vegetation classification problem.

Foreground detection via robust low rank matrix decomposition including spatio-temporal constraint

Foreground detection is the first step in video surveillance system to detect moving objects. Robust Principal Components Analysis (RPCA) shows a nice framework to separate moving objects from the background. The background sequence is then modeled by a low rank subspace that can gradually change over time, while the moving foreground objects constitute the correlated sparse outliers. In this paper, we propose to use a low-rank matrix factorization with IRLS scheme (Iteratively reweighted least squares) and to address in the minimization process the spatial connexity and the temporal sparseness of moving objects (e.g. outliers). Experimental results on the BMC 2012 datasets show the pertinence of the proposed approach.

Incremental and Multi-feature Tensor Subspace Learning Applied for Background Modeling and Subtraction

Background subtraction (BS) is the art of separating moving objects from their background. The Background Modeling (BM) is one of the main steps of the BS process. Several subspace learning (SL) algorithms based on matrix and tensor tools have been used to perform the BM of the scenes. However, several SL algorithms work on a batch process increasing memory consumption when data size is very large. Moreover, these algorithms are not suitable for streaming data when the full size of the data is unknown. In this work, we propose an incremental tensor subspace learning that uses only a small part of the entire data and updates the low-rank model incrementally when new data arrive. In addition, the multi-feature model allows us to build a robust low-rank background model of the scene. Experimental results shows that the proposed method achieves interesting results for background subtraction task.

Comparison of Matrix Completion Algorithms for Background Initialization in Videos

Background model initialization is commonly the first step of the background subtraction process. In practice, several challenges appear and perturb this process such as dynamic background, bootstrapping, illumination changes, noise image, etc. In this context, this work aims to investigate the background model initialization as a matrix completion problem. Thus, we consider the image sequence (or video) as a partially observed matrix. First, a simple joint motion-detection and frame-selection operation is done. The redundant frames are eliminated, and the moving regions are represented by zeros in our observation matrix. The second stage involves evaluating nine popular matrix completion algorithms with the Scene Background Initialization (SBI) data set, and analyze them with respect to the background model challenges. The experimental results show the good performance of LRGeomCG [17] method over its direct competitors.

On the improvement of combined fuzzy topological and directional relations information

Concept of combined extraction of topological and directional relations information developed by Zahzah et al. [1] by employing the Allens temporal relations in 1D spatial domain was improved by Matsakis and Nikitenko [2]. This latter algorithm has high computational complexity due to its limitations of object approximation and segment fuzzification. In this paper, fuzzy Allen relations are used to define the fuzzy topological and directional relations information between different objects. Some extended results of Salamat and Zahzah [3] are discussed. Polygonal object approximation allows us to use fuzzy operators and this approach reduces computational complexity of the method for computing the combined topological and directional relations. To validate the method, some experiments are tested giving satisfactory and promising results. Affine transformation are depicted, these properties will be helpful for using the method in other areas of image analysis such as object tracking.

Matrix and tensor completion algorithms for background model initialization: A comparative evaluation

Abstract Background model initialization is commonly the first step of the background subtraction process. In practice, several challenges appear and perturb this process, such as dynamic background, bootstrapping, illumination changes, noise image, etc. In this context, we investigate the background model initialization as a reconstruction problem from missing data. This problem can be formulated as a matrix or tensor completion task where the image sequence (or video) is revealed as partially observed data. In this paper, the missing entries are induced from the moving regions through a simple joint motion-detection and frame-selection operation. The redundant frames are eliminated, and the moving regions are represented by zeros in our observation model. The second stage involves evaluating twenty-three state-of-the-art algorithms comprising of thirteen matrix completion and ten tensor completion algorithms. These algorithms aim to recover the low-rank component (or background model) from partially observed data. The Scene Background Initialization data set was selected in order to evaluate this proposal with respect to the background model challenges. Our experimental results show the good performance of LRGeomCG method over its direct competitors.

Online Stochastic Tensor Decomposition for Background Subtraction in Multispectral Video Sequences

Background subtraction is an important task for visual surveillance systems. However, this task becomes more complex when the data size grows since the real-world scenario requires larger data to be processed in a more efficient way, and in some cases, in a continuous manner. Until now, most of background subtraction algorithms were designed for mono or trichromatic cameras within the visible spectrum or near infrared part. Recent advances in multispectral imaging technologies give the possibility to record multispectral videos for video surveillance applications. Due to the specific nature of these data, many of the bands within multispectral images are often strongly correlated. In addition, processing multispectral images with hundreds of bands can be computationally burdensome. In order to address these major difficulties of multispectral imaging for video surveillance, this paper propose an online stochastic framework for tensor decomposition of multispectral video sequences (OSTD). First, the experimental evaluations on synthetic generated data show the robustness of the OSTD with other state of the art approaches then, we apply the same idea on seven multispectral video bands to show that only RGB features are not sufficient to tackle color saturation, illumination variations and shadows problem, but the addition of six visible spectral bands together with one near infrared spectra provides a better background/foreground separation.

Double-constrained RPCA based on saliency maps for foreground detection in automated maritime surveillance

The development of automated video-surveillance applications for maritime environment is a very difficult task due to the complexity of the scenes: moving water, waves, etc. The motion of the objects of interest (i.e. ships or boats) can be mixed with the dynamic behavior of the background (non-regular patterns). In this paper, a double-constrained Robust Principal Component Analysis (RPCA), named SCM-RPCA (Shape and Confidence Map-based RPCA), is proposed to improve the object foreground detection in maritime scenes. The sparse component is constrained by shape and confidence maps both extracted from spatial saliency maps. The experimental results in the UCSD and MarDT data sets indicate a better enhancement of the object foreground mask when compared with some related RPCA methods.