Prakash Ishwar

Changedetection.net: A new change detection benchmark dataset

Change detection is one of the most commonly encountered low-level tasks in computer vision and video processing. A plethora of algorithms have been developed to date, yet no widely accepted, realistic, large-scale video dataset exists for benchmarking different methods. Presented here is a unique change detection benchmark dataset consisting of nearly 90,000 frames in 31 video sequences representing 6 categories selected to cover a wide range of challenges in 2 modalities (color and thermal IR). A distinguishing characteristic of this dataset is that each frame is meticulously annotated for ground-truth foreground, background, and shadow area boundaries - an effort that goes much beyond a simple binary label denoting the presence of change. This enables objective and precise quantitative comparison and ranking of change detection algorithms. This paper presents and discusses various aspects of the new dataset, quantitative performance metrics used, and comparative results for over a dozen previous and new change detection algorithms. The dataset, evaluation tools, and algorithm rankings are available to the public on a website1 and will be updated with feedback from academia and industry in the future.

On compressing encrypted data

When it is desired to transmit redundant data over an insecure and bandwidth-constrained channel, it is customary to first compress the data and then encrypt it. In this paper, we investigate the novelty of reversing the order of these steps, i.e., first encrypting and then compressing, without compromising either the compression efficiency or the information-theoretic security. Although counter-intuitive, we show surprisingly that, through the use of coding with side information principles, this reversal of order is indeed possible in some settings of interest without loss of either optimal coding efficiency or perfect secrecy. We show that in certain scenarios our scheme requires no more randomness in the encryption key than the conventional system where compression precedes encryption. In addition to proving the theoretical feasibility of this reversal of operations, we also describe a system which implements compression of encrypted data.

CDnet 2014: An Expanded Change Detection Benchmark Dataset

Change detection is one of the most important lowlevel tasks in video analytics. In 2012, we introduced the changedetection.net (CDnet) benchmark, a video dataset devoted to the evalaution of change and motion detection approaches. Here, we present the latest release of the CDnet dataset, which includes 22 additional videos (70; 000 pixel-wise annotated frames) spanning 5 new categories that incorporate challenges encountered in many surveillance settings. We describe these categories in detail and provide an overview of the results of more than a dozen methods submitted to the IEEE Change DetectionWorkshop 2014. We highlight strengths and weaknesses of these methods and identify remaining issues in change detection.

Distributed video coding in wireless sensor networks

This paper addresses the important aspect of compressing and transmitting video signals generated by wireless broadband networks while heeding the architectural demands imposed by these networks in terms of energy constraints as well as the channel uncertainty related to the wireless communication medium. Driven by the need to develop light, robust, energy-efficient, and low delay video delivery schemes, a distributed video coding based framework dubbed PRISM is introduced. PRISM addresses the wireless video sensor network requirements far more effectively than current state-of-the-art standards like MPEG. This paper focuses on the case of a single video camera and use it as a platform to describe the theoretical principles and practical aspects underlying distributed video coding.

Action Recognition Using Sparse Representation on Covariance Manifolds of Optical Flow

A novel approach to action recognition in video based onthe analysis of optical flow is presented. Properties of opticalflow useful for action recognition are captured usingonly the empirical covariance matrix of a bag of featuressuch as flow velocity, gradient, and divergence. The featurecovariance matrix is a low-dimensional representationof video dynamics that belongs to a Riemannian manifold.The Riemannian manifold of covariance matrices is transformedinto the vector space of symmetric matrices underthe matrix logarithm mapping. The log-covariance matrixof a test action segment is approximated by a sparse linearcombination of the log-covariance matrices of training actionsegments using a linear program and the coefficients ofthe sparse linear representation are used to recognize actions.This approach based on the unique blend of a logcovariance-descriptor and a sparse linear representation istested on the Weizmann and KTH datasets. The proposedapproach attains leave-one-out cross validation scores of94.4% correct classification rate for the Weizmann datasetand 98.5% for the KTH dataset. Furthermore, the methodis computationally efficient and easy to implement.

A gesture-driven computer interface using Kinect

Automatic recognition of human actions from video has been studied for many years. Although still very difficult in uncontrolled scenarios, it has been successful in more restricted settings (e.g., fixed viewpoint, no occlusions) with recognition rates approaching 100%. However, the best-performing methods are complex and computationally-demanding and thus not well-suited for real-time deployments. This paper proposes to leverage the Kinect camera for close-range gesture recognition using two methods. Both methods use feature vectors that are derived from the skeleton model provided by the Kinect SDK in real-time. Although both methods perform nearest-neighbor classification, one method does this in the space of features using the Euclidean distance metric, while the other method does this in the space of feature covariances using a log-Euclidean metric. Both methods recognize 8 hand gestures in real time achieving correct-classification rates of over 99% on a dataset of 20 subjects but the method based on Euclidean distance requires feature-vector collections to be of the same size, is sensitive to temporal misalignment, and has higher computation and storage requirements.

Image saliency: From intrinsic to extrinsic context

We propose a novel framework for automatic saliency estimation in natural images. We consider saliency to be an anomaly with respect to a given context that can be global or local. In the case of global context, we estimate saliency in the whole image relative to a large dictionary of images. Unlike in some prior methods, this dictionary is not annotated, i.e., saliency is assumed unknown. In the case of local context, we partition the image into patches and estimate saliency in each patch relative to a large dictionary of un-annotated patches from the rest of the image. We propose a unified framework that applies to both cases in three steps. First, given an input (image or patch) we extract k nearest neighbors from the dictionary. Then, we geometrically warp each neighbor to match the input. Finally, we derive the saliency map from the mean absolute error between the input and all its warped neighbors. This algorithm is not only easy to implement but also outperforms state-of-the-art methods.

Some Results on Distributed Source Coding for Interactive Function Computation

A two-terminal interactive distributed source coding problem with alternating messages for function computation at both locations is studied. For any number of messages, a computable characterization of the rate region is provided in terms of single-letter information measures. While interaction is useless in terms of the minimum sum-rate for lossless source reproduction at one or both locations, the gains can be arbitrarily large for function computation even when the sources are independent. For a class of sources and functions, interaction is shown to be useless, even with infinite messages, when a function has to be computed at only one location, but is shown to be useful, if functions have to be computed at both locations. For computing the Boolean AND function of two independent Bernoulli sources at both locations, an achievable infinite-message sum-rate with infinitesimal-rate messages is derived in terms of a 2-D definite integral and a rate-allocation curve. The benefit of interaction is highlighted in multiterminal function computation problem through examples. For networks with a star topology, multiple rounds of interactive coding is shown to decrease the scaling law of the total network rate by an order of magnitude as the network grows.

Towards a theory for video coding using distributed compression principles

This paper presents an information-theoretic study of video codecs that are based on the principle of source coding with side information at the decoder. In contrast to the classical Wyner-Ziv side-information source coding problem (1976), in this work we address the situation where the source and side-information are connected through a state of nature that is unknown to both the encoder and the decoder. We dub this framework as source encoding with side-information under ambiguous state of nature (SEASON). Our objective is to compare the achievable rate-distortion (R/D) performance of conventional video codecs designed under the motion-compensated predictive coding (MCPC) framework and video codecs designed under the SEASON framework. Our analysis shows that under appropriate motion models and for Gaussian displaced frame difference (DFD) statistics, the R/D performance of a classical MCPC-based video codec is matched by that of our proposed SEASON-based video codec, with the hitter being characterized by the novel concept of moving the motion compensation task from the encoder to the decoder.

Action Recognition From Video Using Feature Covariance Matrices

We propose a general framework for fast and accurate recognition of actions in video using empirical covariance matrices of features. A dense set of spatio-temporal feature vectors are computed from video to provide a localized description of the action, and subsequently aggregated in an empirical covariance matrix to compactly represent the action. Two supervised learning methods for action recognition are developed using feature covariance matrices. Common to both methods is the transformation of the classification problem in the closed convex cone of covariance matrices into an equivalent problem in the vector space of symmetric matrices via the matrix logarithm. The first method applies nearest-neighbor classification using a suitable Riemannian metric for covariance matrices. The second method approximates the logarithm of a query covariance matrix by a sparse linear combination of the logarithms of training covariance matrices. The action label is then determined from the sparse coefficients. Both methods achieve state-of-the-art classification performance on several datasets, and are robust to action variability, viewpoint changes, and low object resolution. The proposed framework is conceptually simple and has low storage and computational requirements making it attractive for real-time implementation.