Ahmed Tashrif Kamal

Tracking and Activity Recognition Through Consensus in Distributed Camera Networks

Camera networks are being deployed for various applications like security and surveillance, disaster response and environmental modeling. However, there is little automated processing of the data. Moreover, most methods for multicamera analysis are centralized schemes that require the data to be present at a central server. In many applications, this is prohibitively expensive, both technically and economically. In this paper, we investigate distributed scene analysis algorithms by leveraging upon concepts of consensus that have been studied in the context of multiagent systems, but have had little applications in video analysis. Each camera estimates certain parameters based upon its own sensed data which is then shared locally with the neighboring cameras in an iterative fashion, and a final estimate is arrived at in the network using consensus algorithms. We specifically focus on two basic problems - tracking and activity recognition. For multitarget tracking in a distributed camera network, we show how the Kalman-Consensus algorithm can be adapted to take into account the directional nature of video sensors and the network topology. For the activity recognition problem, we derive a probabilistic consensus scheme that combines the similarity scores of neighboring cameras to come up with a probability for each action at the network level. Thorough experimental results are shown on real data along with a quantitative analysis.

Information Weighted Consensus Filters and Their Application in Distributed Camera Networks

Due to their high fault-tolerance and scalability to large networks, consensus-based distributed algorithms have recently gained immense popularity in the sensor networks community. Large-scale camera networks are a special case. In a consensus-based state estimation framework, multiple neighboring nodes iteratively communicate with each other, exchanging their own local information about each targets state with the goal of converging to a single state estimate over the entire network. However, the state estimation problem becomes challenging when some nodes have limited observability of the state. In addition, the consensus estimate is suboptimal when the cross-covariances between the individual state estimates across different nodes are not incorporated in the distributed estimation framework. The cross-covariance is usually neglected because the computational and bandwidth requirements for its computation become unscalable for a large network. These limitations can be overcome by noting that, as the state estimates at different nodes converge, the information at each node becomes correlated. This fact can be utilized to compute the optimal estimate by proper weighting of the prior state and measurement information. Motivated by this idea, we propose information-weighted consensus algorithms for distributed maximum a posteriori parameter estimation, and their extension to the information-weighted consensus filter (ICF) for state estimation. We compare the performance of the ICF with existing consensus algorithms analytically, as well as experimentally by considering the scenario of a distributed camera network under various operating conditions.

Distributed Camera Networks

Over the past decade, large-scale camera networks have become increasingly prevalent in a wide range of applications, such as security and surveillance, disaster response, and environmental modeling. In many applications, bandwidth constraints, security concerns, and difficulty in storing and analyzing large amounts of data centrally at a single location necessitate the development of distributed camera network architectures. Thus, the development of distributed scene-analysis algorithms has received much attention lately. However, the performance of these algorithms often suffers because of the inability to effectively acquire the desired images, especially when the targets are dispersed over a wide field of view (FOV). In this article, we show how to develop an end-to-end framework for integrated sensing and analysis in a distributed camera network so as to maximize various scene-understanding performance criteria (e.g., tracking accuracy, best shot, and image resolution).

A Generalized Kalman Consensus Filter for wide-area video networks

Distributed analysis of video captured by a large network of cameras has received significant attention lately. Tracking moving targets is one of the most fundamental tasks in this regard and the well-known Kalman Consensus Filter (KCF) has been applied to this problem. However, existing solutions do not consider the specific characteristics of video sensor networks, which are necessary for robustness across various application scenarios. Cameras are directional sensors with limited sensing range (field-of-view), and thus, targets are often not observed by many of the cameras. The network may also be spread over a wide area, preventing direct communication between all of the cameras. This limited field-of-view, combined with sparse communication and coverage topologies, motivates us to propose modifications to the traditional KCF framework. Specifically, we consider the covariance matrices of the state estimates of the neighbors and compute a weighted average consensus estimate at each node. Also, the update at each node is computed in two steps, first towards the weighted consensus estimate and then towards the final Kalman measurement update. This leads us to propose a Generalized KCF herein. Experimental results clearly show the advantage of the GKCF compared to the KCF in the considered application scenario.

VideoWeb Dataset for Multi-camera Activities and Non-verbal Communication

Human-activity recognition is one of the most challenging problems in computer vision. Researchers from around the world have tried to solve this problem and have come a long way in recognizing simple motions and atomic activities. As the computer vision community heads toward fully recognizing human activities, a challenging and labeled dataset is needed. To respond to that need, we collected a dataset of realistic scenarios in a multi-camera network environment (VideoWeb) involving multiple persons performing dozens of different repetitive and non-repetitive activities. This chapter describes the details of the dataset. We believe that this VideoWeb Activities dataset is unique and it is one of the most challenging datasets available today. The dataset is publicly available online at http://vwdata.ee.ucr.edu/ along with the data annotation.

Information weighted consensus

Consensus-based distributed estimation schemes are becoming increasingly popular in sensor networks due to their scalability and fault tolerance capabilities. In a consensus-based state estimation framework, multiple neighboring nodes iteratively communicate with each other, exchanging their own local estimates of a targets state with the goal of converging to a single state estimate over the entire network. However, the state estimation problem becomes challenging when a node has limited observability of the state. In addition, the consensus estimate is sub-optimal when the cross-covariances between the individual state estimates across different nodes are not incorporated in the distributed estimation framework. The cross-covariance is usually neglected because the computational and bandwidth requirements for its computation grow exponentially with the number of nodes. These limitations can be overcome by noting that, as the state estimates at different nodes converge, the information at each node becomes redundant. This fact can be utilized to compute the optimal estimate by proper weighting of the prior state and measurement information. Motivated by this idea, we propose information-weighted consensus algorithms for distributed maximum a posteriori parameter estimates, and their extension to the information-weighted consensus filter (ICF) for state estimation. We show both theoretically and experimentally that the proposed methods asymptotically approach the optimal centralized performance. Simulation results show that ICF is robust even when the optimality conditions are not met and has low communication requirements.

Distributed Multi-Target Tracking and Data Association in Vision Networks

Distributed algorithms have recently gained immense popularity. With regards to computer vision applications, distributed multi-target tracking in a camera network is a fundamental problem. The goal is for all cameras to have accurate state estimates for all targets. Distributed estimation algorithms work by exchanging information between sensors that are communication neighbors. Vision-based distributed multi-target state estimation has at least two characteristics that distinguishes it from other applications. First, cameras are directional sensors and often neighboring sensors may not be sensing the same targets, i.e., they are naive with respect to that target. Second, in the presence of clutter and multiple targets, each camera must solve a data association problem. This paper presents an information-weighted, consensus-based, distributed multi-target tracking algorithm referred to as the Multi-target Information Consensus (MTIC) algorithm that is designed to address both the naivety and the data association problems. It converges to the centralized minimum mean square error estimate. The proposed MTIC algorithm and its extensions to non-linear camera models, termed as the Extended MTIC (EMTIC), are robust to false measurements and limited resources like power, bandwidth and the real-time operational requirements. Simulation and experimental analysis are provided to support the theoretical results.

Belief consensus for distributed action recognition

In this work, we consider a camera network where processing is distributed across the cameras. Our goal is to recognize actions of multiple targets consistently observed over the entire network. To obtain consistent and better results we need to properly fuse the action scores from multiple cameras. There have been multiple works on distributed tracking and distributed data association for multiple targets in a camera network. We can use the data association results and tracking confidence scores to improve the action recognition results. We propose a consensus based framework for solving this problem in an integrated manner and with a completely distributed camera network architecture. We propose a novel method for weighting the action scores based on tracking confidences and show how the cameras can reach a consensus about the action of a target using belief consensus. We show real life experiments and performance metrics with multiple cameras and targets.

Consensus-based distributed estimation in camera networks

Distributed algorithms in the sensors networks community usually require each sensor to have its own measurement. In practice, this constraint can not always be met. For example, in a camera network, all cameras might not observe a particular target as cameras are directional sensors and have a limited field-of-view (FOV). Moreover, different sensors might provide different quality measures related to different elements of the measurement vector depending on various factors as directionality, occlusion etc. This requires the designing of a new type of distributed algorithm that considers the quality and/or absence of measurements. In this paper, we present a distributed algorithm to compute the maximum likelihood estimate of the state of a target viewed by the network of cameras, taking into account the above-mentioned factors. We provide step-by-step derivation along with theoretical guarantee of optimality and convergence of the method. Experimental results are provided to show the performance of the proposed algorithm.

Vector field analysis for motion pattern identification in video

Identification of motion patterns in video is an important problem because it is the first step towards analysis of complex multi-person behaviors to obtain long-term interaction models. In this paper, we will present a flow based technique to identify spatio-temporal motion patterns in a multi-object video. We use the Helmholtz decomposition of optical flow and compute singular points corresponding to component fields. We will show that the optical flow can be used to identify regions which correspond to different moving entities in the video. The singular points in these regions capture the characteristics of the field around them and can be used to identify these regions. This representation would provide us with a framework to analyze activities of individual entities in the scene as well as the global interactions between them. We demonstrate our algorithm on a dataset composed of multi-object videos recorded in a realistic environment.