Sen-ching S. Cheung

Robust techniques for background subtraction in urban traffic video

Identifying moving objects from a video sequence is a fundamental and critical task in many computer-vision applications. A common approach is to perform background subtraction, which identifies moving objects from the portion of a video frame that differs significantly from a background model. There are many challenges in developing a good background subtraction algorithm. First, it must be robust against changes in illumination. Second, it should avoid detecting non-stationary background objects such as swinging leaves, rain, snow, and shadow cast by moving objects. Finally, its internal background model should react quickly to changes in background such as starting and stopping of vehicles. In this paper, we compare various background subtraction algorithms for detecting moving vehicles and pedestrians in urban traffic video sequences. We consider approaches varying from simple techniques such as frame differencing and adaptive median filtering, to more sophisticated probabilistic modeling techniques. While complicated techniques often produce superior performance, our experiments show that simple techniques such as adaptive median filtering can produce good results with much lower computational complexity.

Efficient video similarity measurement with video signature

The proliferation of video content on the Web makes similarity detection an indispensable tool in Web data management, searching, and navigation. We propose a number of algorithms to efficiently measure video similarity. We define video as a set of frames, which are represented as high dimensional vectors in a feature space. Our goal is to measure ideal video similarity (IVS), defined as the percentage of clusters of similar frames shared between two video sequences. Since IVS is too complex to be deployed in large database applications, we approximate it with Voronoi video similarity (VVS), defined as the volume of the intersection between Voronoi cells of similar clusters. We propose a class of randomized algorithms to estimate VVS by first summarizing each video with a small set of its sampled frames, called the video signature (ViSig), and then calculating the distances between corresponding frames from the two ViSigs. By generating samples with a probability distribution that describes the video statistics, and ranking them based upon their likelihood of making an error in the estimation, we show analytically that ViSig can provide an unbiased estimate of IVS. Experimental results on a large dataset of Web video and a set of MPEG-7 test sequences with artificially generated similar versions are provided to demonstrate the retrieval performance of our proposed techniques.

Robust background subtraction with foreground validation for urban traffic video

Identifying moving objects in a video sequence is a fundamental and critical task in many computer-vision applications. Background subtraction techniques are commonly used to separate foreground moving objects from the background. Most background subtraction techniques assume a single rate of adaptation, which is inadequate for complex scenes such as a traffic intersection where objects are moving at different and varying speeds. In this paper, we propose a foreground validation algorithm that first builds a foreground mask using a slow-adapting Kalman filter, and then validates individual foreground pixels by a simple moving object model built using both the foreground and background statistics as well as the frame difference. Ground-truth experiments with urban traffic sequences show that our proposed algorithm significantly improves upon results using only Kalman filter or frame-differencing, and outperforms other techniques based on mixture of Gaussians, median filter, and approximated median filter.

Hiding privacy information in video surveillance system

This paper proposes a detailed framework of storing privacy information in surveillance video as a watermark. Authorized personnel is not only removed from the surveillance video as in J. Wickramasuriya et al. (2004) but also embedded into the video itself, which can only be retrieved with a secrete key. A perceptual-model-based compressed domain video watermarking scheme is proposed to deal with the huge payload problem in the proposed surveillance system. A signature is also embedded into the header of the video as in M. Pramateftakis et al. (2004) for authentication. Simulation results have shown that the proposed algorithm can embed all the privacy information into the video without affecting its visual quality. As a result, the proposed video surveillance system can monitor the unauthorized persons in a restricted environment, protect the privacy of the authorized persons but, at the same time, allow the privacy information to be revealed in a secure and reliable way.

Layer Depth Denoising and Completion for Structured-Light RGB-D Cameras

The recent popularity of structured-light depth sensors has enabled many new applications from gesture-based user interface to 3D reconstructions. The quality of the depth measurements of these systems, however, is far from perfect. Some depth values can have significant errors, while others can be missing altogether. The uncertainty in depth measurements among these sensors can significantly degrade the performance of any subsequent vision processing. In this paper, we propose a novel probabilistic model to capture various types of uncertainties in the depth measurement process among structured-light systems. The key to our model is the use of depth layers to account for the differences between foreground objects and background scene, the missing depth value phenomenon, and the correlation between color and depth channels. The depth layer labeling is solved as a maximum a-posteriori estimation problem, and a Markov Random Field attuned to the uncertainty in measurements is used to spatially smooth the labeling process. Using the depth-layer labels, we propose a depth correction and completion algorithm that outperforms other techniques in the literature.

Fast similarity search and clustering of video sequences on the world-wide-web

We define similar video content as video sequences with almost identical content but possibly compressed at different qualities, reformatted to different sizes and frame-rates, undergone minor editing in either spatial or temporal domain, or summarized into keyframe sequences. Building a search engine to identify such similar content in the World-Wide Web requires: 1) robust video similarity measurements; 2) fast similarity search techniques on large databases; and 3) intuitive organization of search results. In a previous paper, we proposed a randomized technique called the video signature (ViSig) method for video similarity measurement. In this paper, we focus on the remaining two issues by proposing a feature extraction scheme for fast similarity search, and a clustering algorithm for identification of similar clusters. Similar to many other content-based methods, the ViSig method uses high-dimensional feature vectors to represent video. To warrant a fast response time for similarity searches on high dimensional vectors, we propose a novel nonlinear feature extraction scheme on arbitrary metric spaces that combines the triangle inequality with the classical Principal Component Analysis (PCA). We show experimentally that the proposed technique outperforms PCA, Fastmap, Triangle-Inequality Pruning, and Haar wavelet on signature data. To further improve retrieval performance, and provide better organization of similarity search results, we introduce a new graph-theoretical clustering algorithm on large databases of signatures. This algorithm treats all signatures as an abstract threshold graph, where the distance threshold is determined based on local data statistics. Similar clusters are then identified as highly connected regions in the graph. By measuring the retrieval performance against a ground-truth set, we show that our proposed algorithm outperforms simple thresholding, single-link and complete-link hierarchical clustering techniques.

Multimedia streaming using multiple TCP connections

As broadband Internet becomes widely available, multimedia applications over the Internet become increasingly popular. However, packet loss, delay, and time-varying bandwidth of the Internet have remained the major problems for multimedia streaming applications. As such, a number of approaches, including network infrastructure and protocol, source and channel coding have been proposed to either overcome or alleviate these drawbacks of the Internet. In this paper, we propose the MultiTCP system, a receiver-driven, TCP-based system for multimedia streaming over the Internet. Our proposed algorithm aims at providing resilience against SHORT TERM insufficient bandwidth by using MULTIPLE TCP connections for the same application. Furthermore, our proposed system enables the application to achieve and control the desired sending rate during congested periods, which cannot be achieved using traditional TCP. Finally, our proposed system is implemented at the application layer, and hence, no kernel modification to TCP is necessary. We analyze the proposed system, and present simulation results to demonstrate its advantages over the traditional single TCP based approach.

Video Streaming with Network Coding

Recent years have witnessed an explosive growth in multimedia streaming applications over the Internet. Notably, Content Delivery Networks (CDN) and Peer-to-Peer (P2P) networks have emerged as two effective paradigms for delivering multimedia contents over the Internet. One salient feature shared between these two networks is the inherent support for path diversity streaming where a receiver receives multiple streams simultaneously on different network paths as a result of having multiple senders. In this paper, we propose a network coding framework for efficient video streaming in CDNs and P2P networks in which, multiple servers/peers are employed to simultaneously stream a video to a single receiver. We show that network coding techniques can (a) eliminate the need for tight synchronization between the senders, (b) be integrated easily with TCP, and (c) reduce server’s storage in CDN settings. Importantly, we propose the Hierarchical Network Coding (HNC) technique to be used with scalable video bit stream to combat bandwidth fluctuation on the Internet. Simulations demonstrate that under certain scenarios, our proposed network coding techniques can result in bandwidth saving up to 60% over the traditional schemes.

Optimal Camera Network Configurations for Visual Tagging

Proper placement of cameras in a distributed smart camera network is an important design problem. Not only does it determine the coverage of the surveillance, but it also has a direct impact on the appearance of objects in the cameras which dictates the performance of all subsequent computer vision tasks. In this paper, we propose a generic camera placement model based on the visibility of objects at different cameras. Our motivation stems from the need of identifying and locating objects with distinctive visual features or ldquotags.rdquo This is a very common goal in computer vision with applications ranging from identifying soccer players by their jersey numbers to locating and recognizing faces of individuals. Our proposed framework places no restriction on the visual classification tasks. It incorporates realistic camera models, self occlusion of tags, and occlusion by other moving objects. It is also flexible enough to handle arbitrary-shaped three-dimensional environments. Using this framework, two novel binary integer programming (BIP) algorithms are proposed to find the optimal camera placement for ldquovisual taggingrdquo and a greedy implementation is developed to cope with the complexity of BIP. Extensive performance analysis is performed using Monte Carlo simulations, virtual environment simulations, and real-world experiments. We also demonstrate the usefulness of visual tagging through robust individual identification and obfuscation across multiple camera views for privacy protection.

Peer-to-Peer Streaming with Hierarchical Network Coding

In recent years, content delivery networks (CDN) and Peer-to-Peer (P2P) networks have emerged as two effective paradigms for delivering multimedia contents over the Internet. An important feature in CDN and P2P networks is the data redundancy across multiple servers/peers which enables efficient media delivery. In this paper, we propose a network coding framework for efficient media streaming in either content delivery networks or P2P networks in which, multiple servers/peers are employed to simultaneously stream a video to a single receiver. Unlike previous multi-sender schemes, we show that network coding technique can (a) reduce the redundancy storage, (b) eliminate the need for tight synchronization between the senders, and (c) be integrated easily with TCP. Furthermore, we propose the Hierarchical Network Coding (HNC) technique to be used with scalable video bit stream to combat bandwidth fluctuation on the Internet. Simulation results demonstrate that our proposed scheme can result in bandwidth saving up to 40% for many cases over the traditional schemes.