Tobias Senst

Robust Local Optical Flow for Feature Tracking

This paper is motivated by the problem of local motion estimation via robust regression with linear models. In order to increase the robustness of the motion estimates, we propose a novel robust local optical flow approach based on a modified Hampel estimator. We show the deficiencies of the least squares estimator used by the standard Kanade-Lucas-Tomasi (KLT) tracker when the assumptions made by Lucas-Kanade are violated. We propose a strategy to adapt the window sizes to cope with the generalized aperture problem. Finally, we evaluate our method on the Middlebury and MIT dataset and show that the algorithm provides excellent feature tracking performance with only slightly increased computational complexity compared to KLT. To facilitate further development, the presented algorithm can be downloaded from http://www.nue.tu-berlin.de/menue/forschung/projekte/rlof.

Detecting people carrying objects based on an optical flow motion model

Detecting people carrying objects is a commonly formulated problem as a first step to monitor interactions between people and objects. Recent work relies on a precise foreground object segmentation, which is often difficult to achieve in video surveillance sequences due to a bad contrast of the foreground objects with the scene background, abrupt changing light conditions and small camera vibrations. In order to cope with these difficulties we propose an approach based on motion statistics. Therefore we use a Gaussian mixture motion model (GMMM) and, based on that model, we define a novel speed and direction independent motion descriptor in order to detect carried baggage as those regions not fitting in the motion description model of an average walking person. The system was tested with the public dataset PETS2006 and a more challenging dataset including abrupt lighting changes and bad color contrast and compared with existing systems, showing very promissing results.

Detection of static objects for the task of video surveillance

Detecting static objects in video sequences has a high relevance in many surveillance scenarios like airports and railwaystations. In this paper we propose a system for the detection of static objects in crowded scenes that, based on the detection of two background models learning at different rates, classifies pixels with the help of a finite-state machine. The background is modelled by two mixtures of Gaussians with identical parameters except for the learning rate. The state machine provides the meaning for the interpretation of the results obtained from background subtraction and can be used to incorporate additional information cues, obtaining thus a flexible system specially suitable for real-life applications. The system was built in our surveillance application and successfully validated with several public datasets.

A Lagrangian framework for video analytics

The extraction of motion patterns from image sequences based on the optical flow methodology is an important and timely topic among visual multi media applications. In this work we will present a novel framework that combines the optical flow methodology from image processing with methods developed for the Lagrangian analysis of time-dependent vector fields. The Lagrangian approach has been proven to be a valuable and powerful tool to capture the complex dynamic motion behavior within unsteady vector fields. To come up with a compact and applicable framework, this paper will provide concepts on how to compute trajectory-based Lagrangian measures in series of optical flow fields, a set of basic measures to capture the essence of the motion behavior within the image, and a compact hierarchical, feature-based description of the resulting motion features. The resulting framework will bee shown to be suitable for an automated image analysis as well as compact visual analysis of image sequences in its spatio-temporal context. We show its applicability for the task of motion feature description and extraction on different temporal scales, crowd motion analysis, and automated detection of abnormal events within video sequences.

Detecting People Carrying Objects Utilizing Lagrangian Dynamics

The availability of dense motion information in computer vision domain allows for the effective application of Lagrangian techniques that have their origin in fluid flow analysis and dynamical systems theory. A well established technique that has been proven to be useful in image-based crowd analysis are Finite Time Lyapunov Exponents (FTLE). Based on this, we present a method to detect people carrying object and describe a methodology how to apply established flow field methods onto the problem of describing individuals. Further, we reinterpret Lagrangian features in relation to the underlying motion process and show their applicability towards the appearance modeling of pedestrians. This definition allows to increase performance of state-of-the-art methods and is shown to be robust under varying parameter settings and different optical flow extraction approaches.

Robust modified L 2 local optical flow estimation and feature tracking

This paper describes a robust method for the local optical flow estimation and the KLT feature tracking performed on the GPU. Therefore we present an estimator based on the L2 norm with robust characteristics. In order to increase the robustness at discontinuities we propose a strategy to adapt the used region size. The GPU implementation of our approach achieves real-time (>25 fps) performance for High Definition (HD) video sequences while tracking several thousands of points. The benefit of the suggested enhancement is illustrated on the Middlebury optical flow benchmark.

Human action recognition using Lagrangian descriptors

Human action recognition requires the description of complex motion patterns in image sequences. In general, these patterns span varying temporal scales. In this context, Lagrangian methods have proven to be valuable for crowd analysis tasks such as crowd segmentation. In this paper, we show that, besides their potential in describing large scale motion patterns, Lagrangian methods are also well suited to model complex individual human activities over variable time intervals. We use Finite Time Lyapunov Exponents and time-normalized arc length measures in a linear SVM classification scheme. We evaluated our method on the Weizmann and KTH datasets. The results demonstrate that our approach is promising and that human action recognition performance is improved by fusing Lagrangian measures.

II-LK – a real-time implementation for sparse optical flow

In this paper we present an approach to speed up the computation of sparse optical flow fields by means of integral images and provide implementation details. Proposing a modification of the Lucas-Kanade energy functional allows us to use integral images and thus to speed up the method notably while affecting only slightly the quality of the computed optical flow. The approach is combined with an efficient scanline algorithm to reduce the computation of integral images to those areas where there are features to be tracked. The proposed method can speed up current surveillance algorithms used for scene description and crowd analysis.

Robust local optical flow estimation using bilinear equations for sparse motion estimation

This article presents a theoretical framework to decrease the computation effort of the Robust Local Optical Flow method which is based on the Lucas Kanade method. We show mathematically, how to transform the iterative scheme of the feature tracker into a system of bilinear equations and thus estimate the motion vectors directly by analyzing its zeros. Furthermore, we show that it is possible to parallelise our approach efficiently on a GPU, thus, outperforming the current OpenCV-OpenCL implementation of the pyramidal Lucas Kanade method in terms of runtime and accuracy. Finally, an evaluation is given for the Middlebury Optical Flow and the KITTI datasets.

On building decentralized wide-area surveillance networks based on ONVIF

In this paper we present a decentralized surveillance network composed of IP video cameras, analysis devices and a central node which collects information and displays it in a 3D model of the complete area. The exchange of information between all components in the surveillance network takes place according to the ONVIF specification, therefore ensuring interoperability between products complying with the specification and flexibility regarding the integration of new devices and services. The collected information is displayed in a 3D model of the surveilled area, therefore providing a comfortable overview of the activity in large environments and offering the user an intuitive way to eventually interact with network devices.