yuan Li

Statistical modeling of complex backgrounds for foreground object detection

This paper addresses the problem of background modeling for foreground object detection in complex environments. A Bayesian framework that incorporates spectral, spatial, and temporal features to characterize the background appearance is proposed. Under this framework, the background is represented by the most significant and frequent features, i.e., the principal features , at each pixel. A Bayes decision rule is derived for background and foreground classification based on the statistics of principal features. Principal feature representation for both the static and dynamic background pixels is investigated. A novel learning method is proposed to adapt to both gradual and sudden once-off background changes. The convergence of the learning process is analyzed and a formula to select a proper learning rate is derived. Under the proposed framework, a novel algorithm for detecting foreground objects from complex environments is then established. It consists of change detection, change classification, foreground segmentation, and background maintenance. Experiments were conducted on image sequences containing targets of interest in a variety of environments, e.g., offices, public buildings, subway stations, campuses, parking lots, airports, and sidewalks. Good results of foreground detection were obtained. Quantitative evaluation and comparison with the existing method show that the proposed method provides much improved results.

Foreground object detection from videos containing complex background

This paper proposes a novel method for detection and segmentation of foreground objects from a video which contains both stationary and moving background objects and undergoes both gradual and sudden once-off changes. A Bayes decision rule for classification of background and foreground from selected feature vectors is formulated. Under this rule, different types of background objects will be classified from foreground objects by choosing a proper feature vector. The stationary background object is described by the color feature, and the moving background object is represented by the color co-occurrence feature. Foreground objects are extracted by fusing the classification results from both stationary and moving pixels. Learning strategies for the gradual and sudden once-off background changes are proposed to adapt to various changes in background through the video. The convergence of the learning process is proved and a formula to select a proper learning rate is also derived. Experiments have shown promising results in extracting foreground objects from many complex backgrounds including wavering tree branches, flickering screens and water surfaces, moving escalators, opening and closing doors, switching lights and shadows of moving objects.

On pixel count based crowd density estimation for visual surveillance

Surveillance systems for public security are going beyond the conventional CCTV. A new generation of systems relies on image processing and computer vision techniques, deliver more ready-to-use information, and provide assistance for early detection of unusual events. Crowd density is a useful source of information because unusual crowdedness is often related to unusual events. Previous works on crowd density estimation either ignore perspective distortion or perform the correction based on incorrect formulation. Also there is no investigation on whether the geometric correction derived for the ground plane can be applied to human objects standing upright to the plane. This paper derives the relation for geometric correction for the ground plane and proves formally that it can be directly applied to all the foreground pixels. We also propose a very efficient implementation because it is important for a real-time application. Finally a time-adaptive criterion for unusual crowdedness detection is described.

Foreground object detection in changing background based on color co-occurrence statistics

This paper proposes a novel method for detecting foreground objects in nonstationary complex environments containing moving background objects. We derive a Bayes decision rule for classification of background and foreground changes based on inter-frame color co-occurrence statistics. An approach to store and fast retrieve color co-occurrence statistics is also established In the proposed method, foreground objects are detected in two steps. First, both foreground and background changes are extracted using background subtraction and temporal differencing. The frequent background changes are then recognized using the Bayes decision rule based on the learned color co-occurrence statistics. Both short-term and longterm strategies to learn the frequent background changes are proposed Experiments have shown promising results in detecting foreground objects from video containing wavering tree branches and flickering screens/water surface. The proposed method has shown better performance as compared with two existing methods.

An Efficient Sequential Approach to Tracking Multiple Objects Through Crowds for Real-Time Intelligent CCTV Systems

Efficiency and robustness are the two most important issues for multiobject tracking algorithms in real-time intelligent video surveillance systems. We propose a novel 2.5-D approach to real-time multiobject tracking in crowds, which is formulated as a maximum a posteriori estimation problem and is approximated through an assignment step and a location step. Observing that the occluding object is usually less affected by the occluded objects, sequential solutions for the assignment and the location are derived. A novel dominant color histogram (DCH) is proposed as an efficient object model. The DCH can be regarded as a generalized color histogram, where dominant colors are selected based on a given distance measure. Comparing with conventional color histograms, the DCH only requires a few color components (31 on average). Furthermore, our theoretical analysis and evaluation on real data have shown that DCHs are robust to illumination changes. Using the DCH, efficient implementations of sequential solutions for the assignment and location steps are proposed. The assignment step includes the estimation of the depth order for the objects in a dispersing group, one-by-one assignment, and feature exclusion from the group representation. The location step includes the depth-order estimation for the objects in a new group, the two-phase mean-shift location, and the exclusion of tracked objects from the new position in the group. Multiobject tracking results and evaluation from public data sets are presented. Experiments on image sequences captured from crowded public environments have shown good tracking results, where about 90% of the objects have been successfully tracked with the correct identification numbers by the proposed method. Our results and evaluation have indicated that the method is efficient and robust for tracking multiple objects ( ges 3) in complex occlusion for real-world surveillance scenarios.

Principal color representation for tracking persons

This paper proposes a novel method for tracking persons based on the principal colors of human objects. First, an efficient human object representation method, principal color representation (PCR), is proposed. Asymmetric similarity measures are then proposed based on the principal color representation. These asymmetric similarity measures could be used to evaluate the matching between two individuals as well as visual evident of an individual in a group. An efficient algorithm for tracking persons as individuals or in groups is then described. The method has been tested using image sequences containing multiple moving persons frequently gathering and separating. Our test results have shown that proposed method has successfully tracked both persons as individuals or in groups, and is robust to illumination changes.

Adaptive background subtraction based on feedback from fuzzy classification

Background subtraction is an important issue for achieving effective foreground object detection in video surveillance. Background subtraction requires the timely updating of a background model to gradual illumination changes as well as the significant changes in the background. It is also essential that foreground objects have little impact on the updating of the background. Based on our change-type categories, we propose an adaptive background subtraction method where a two-strategy-based background maintenance is introduced to adapt to different types of changes by using feedback from change segmentation and region classification. The work mainly contributes to the following issues: 1. propose a change segmentation method that detects change regions as well as provides spatiotemporal information about the changes by using fuzzy techniques; 2. propose a fuzzy reasoning method to classify background and foreground changes at the object level; and 3. propose a new method for adaptive background maintenance based on the feedback from pixel-level to object-level processing that is able to avoid tradeoff in the updating rate. Experiments on indoor and outdoor video scenes are conducted and results show that the proposed method adapts well to various background changes without absorbing foreground objects. Comparisons with an existing method using a constant learning rate show an improved performance.

Efficient adaptive background subtraction based on multi-resolution background modelling and updating

Adaptive background subtraction (ABS) is a fundamental step for foreground object detection in many real-time video surveillance systems. In many ABS methods, a pixel-based statistical model is used for the background and each pixel is updated online to adapt to various background changes. As a result, heavy computation and memory consumption are required. In this paper, we propose an efficient methodology for implementation of ABS algorithms based on multiresolution background modelling and sequential sampling for updating background. Experiments and quantitative evaluation are conducted on two open data sets (PETS2001 and PETS2006) and scenarios captured in some public places, and some results are included. Our results have shown that the proposed method requires a significant reduction in memory and CPU usage, meanwhile maintaining a similar foreground segmentation performance as compared with the corresponding single resolution methods.

Region-Based Statistical Background Modeling for Foreground Object Segmentation

This paper proposes a novel region-based scheme for dynamically modeling time-evolving statistics of video background, leading to an effective segmentation of foreground moving objects for a video surveillance system. In (L. Li et al., 2004) statistical-based video surveillance systems employ a Bayes decision rule for classifying foreground and background changes in individual pixels. Although principal feature representations significantly reduce the size of tables of statistics, pixel-wise maintenance remains a challenge due to the computations and memory requirement. The proposed region-based scheme, which is an extension of the above method, replaces pixel-based statistics by region-based statistics through introducing dynamic background region (or pixel) merging and splitting. Simulations have been performed to several outdoor and indoor image sequences, and results have shown a significant reduction of memory requirements for tables of statistics while maintaining relatively good quality in foreground segmented video objects.

Adaptive background subtraction with multiple feedbacks for video surveillance

Background subtraction is the first step for video surveillance. Existing methods almost all update their background models with a constant learning rate, which makes them not adaptive to some complex situations, e.g., crowded scenes or objects staying for a long time. In this paper, a novel framework which integrates both positive and negative feedbacks to control the learning rate is proposed. The negative feedback comes from background contextual analysis and the positive feedback comes from the foreground region analysis. Two descriptors of global contextual features are proposed and the visibility measures of background regions are derived based on contextual descriptors. Spatial-temporal features of the foreground regions are exploited. Fusing both positive and negative feedbacks, suitable strategy of background updating for specified surveillance task can be implemented. Three strategies for short-term, selective and long-term surveillance have been implemented and tested. Improved results compared with conventional background subtraction have been obtained.