Emil Eriksson

Real-Time Distributed Visual Feature Extraction from Video in Sensor Networks

Enabling visual sensor networks to perform visual analysis tasks in real-time is challenging due to the computational complexity of detecting and extracting visual features. A promising approach to address this challenge is to distribute the detection and the extraction of local features among the sensor nodes, in which case the time to complete the visual analysis of an image is a function of the number of features found and of the distribution of the features in the image. In this paper we formulate the minimization of the time needed to complete the distributed visual analysis for a video sequence subject to a mean average precision requirement as a stochastic optimization problem. We propose a solution based on two composite predictors that reconstruct randomly missing data, and use a quantile-based linear approximation of the feature distribution and time series analysis methods. The composite predictors allow us to compute an approximate optimal solution through linear programming. We use two surveillance videos to evaluate the proposed algorithms, and show that prediction is essential for controlling the completion time. The results show that the last value predictor together with regular quantile-based distribution approximation provide a low complexity solution with very good performance.

Prediction-based load control and balancing for feature extraction in visual sensor networks

We consider controlling and balancing the processing load in a visual sensor network (VSN) used for detecting local features, such as BRISK. We formulate a prediction problem with random missing data, and propose two regression-based algorithms for data reconstruction. Numerical results illustrate the performance of the proposed algorithms, and show that backward regression combined with the last value predictor can be used for controlling and balancing the processing load in VSNs with good performance.

Predictive Distributed Visual Analysis for Video in Wireless Sensor Networks

We consider the problem of performing distributed visual analysis for a video sequence in a visual sensor network that contains sensor nodes dedicated to processing. Visual analysis requires the detection and extraction of visual features from the images, and thus the time to complete the analysis depends on the number and on the spatial distribution of the features, both of which are unknown before performing the detection. In this paper, we formulate the minimization of the time needed to complete the distributed visual analysis for a video sequence subject to a mean average precision requirement as a stochastic optimization problem. We propose a solution based on two composite predictors that reconstruct randomly missing data, on quantile-based linear approximation of feature distribution and on time series analysis methods. The composite predictors allow us to compute an approximate optimal solution through linear programming. We use two surveillance video traces to evaluate the proposed algorithms, and show that prediction is essential for minimizing the completion time, even if the wireless channel conditions vary and introduce significant randomness. The results show that the last value predictor together with regular quantile-based distribution approximation provide a low complexity solution with very good performance.

Algorithms for distributed feature extraction in multi-camera visual sensor networks

Real-time visual analysis tasks, like tracking and recognition, require swift execution of computationally intensive algorithms. Enabling visual sensor networks to perform such tasks can be achieved by augmenting the sensor network with processing nodes and distributing the computational burden among several nodes, in a way that the cameras contend for the processing nodes while trying to minimize their completion times. In this paper, we formulate the problem of minimizing the completion time of all camera sensors as an optimization problem. We propose algorithms for fully distributed optimization, analyze the existence of equilibrium allocations, and evaluate their performance. Simulation results show that distributed optimization can provide good performance despite limited information availability at low computational complexity, but the predictable and stable performance is often not provided by the algorithm that provides lowest average completion time.

Enabling visual analysis in wireless sensor networks

This demo showcases some of the results obtained by the GreenEyes project, whose main objective is to enable visual analysis on resource-constrained multimedia sensor networks. The demo features a multi-hop visual sensor network operated by BeagleBones Linux computers with IEEE 802.15.4 communication capabilities, and capable of recognizing and tracking objects according to two different visual paradigms. In the traditional compress-then-analyze (CTA) paradigm, JPEG compressed images are transmitted through the network from a camera node to a central controller, where the analysis takes place. In the alternative analyze-then-compress (ATC) paradigm, the camera node extracts and compresses local binary visual features from the acquired images (either locally or in a distributed fashion) and transmits them to the central controller, where they are used to perform object recognition/tracking. We show that, in a bandwidth constrained scenario, the latter paradigm allows to reach better results in terms of application frame rates, still ensuring excellent analysis performance.

Efficient Distribution of Visual Processing Tasks in Multi-camera Visual Sensor Networks

Multi-camera visual sensor networks (VSNs) require large computational resources in order to perform visual analysis in real-time. One way to match the computational needs is to augment the VSN with dedicated processing nodes that do in-network processing, but this requires careful allocation of loads from the sensor nodes in order to ensure low processing times. In this paper we formulate the problem of load allocation and completion time minimization in a VSN as an optimization problem. We propose a distributed algorithm for load allocation, and evaluate its performance in terms of completion time and convergence compared to a Greedy algorithm. Simulations show that the proposed algorithm converges faster, but at the cost of increased completion times. Nonetheless, combined with appropriate coordination, the proposed algorithm achieves low completion times at low complexity.

Coordinating Distributed Algorithms for Feature Extraction Offloading in Multi-Camera Visual Sensor Networks

Real-time visual analysis tasks, like tracking and recognition, require swift execution of computationally intensive algorithms. Visual sensor networks could be enabled to perform such tasks by allowing the camera nodes to offload their computational load to nearby processing nodes. In this paper, we address the problem of minimizing the completion time of multiple camera sensors that share the transmission and the processing resources of multiple processing nodes for computation offloading. We show that the problem is NP-hard, and propose a combination of central coordination and distributed optimization with limited signaling among the camera sensors as a solution. We analyze the existence of equilibrium allocations for the distributed algorithms, evaluate the effect of the network topology and of the video characteristics on the algorithms’ performance, and assess the benefits of central coordination. Our results demonstrate that with sufficient information available, distributed optimization can provide low completion times, moreover predictable and stable performance can be achieved with additional, sparse central coordination.

Radio and Computational Resource Management for Fog Computing Enabled Wireless Camera Networks

We consider the problem of assigning communication and computing resources of a fog computing system to visual sensors that may observe various scenes from multiple viewing angles. We formulate the Multi-View Assignment Problem (MVAP) as a quadratic mixed-integer problem, and show that it is NP-hard. We propose a polynomial time

GreenEyes : Networked energy-aware visual analysis

4

Demo: Enabling Image Analysis Tasks in Visual Sensor Networks

-approximation based on a transformation of MVAP to a General Assignment Problem with dependent profits in which items are sets of sensors with an overlapping field of view, and based on a reduction of the set of items to be assigned. We show that the reduction of the set of items does not affect the solution of the problem if it results in the dominating set of items. Extensive numerical results show that the proposed algorithm performs close to optimal for small systems, performs well even if the reduction finds an approximately dominating set, and scales well with the number of sensors in the system.