Ákos Utasi

A 3-D marked point process model for multi-view people detection

In this paper we introduce a probabilistic approach on multiple person localization using multiple calibrated camera views. People present in the scene are approximated by a population of cylinder objects in the 3-D world coordinate system, which is a realization of a Marked Point Process. The observation model is based on the projection of the pixels of the obtained motion masks in the different camera images to the ground plane and to other parallel planes with different height. The proposed pixel-level feature is based on physical properties of the 2-D image formation process and can accurately localize the leg position on the ground plane and estimate the height of the people, even if the area of interest is only a part of the scene, meanwhile silhouettes from irrelevant outside motions may significantly overlap with the monitored region in some of the camera views. We introduce an energy function, which contains a data term calculated from the extracted features and a geometrical constraint term modeling the distance between two people. The final configuration results (location and height) are obtained by an iterative stochastic energy optimization process, called the Multiple Birth and Death dynamics. The proposed approached is compared to a recent state-of-the-art technique in a publicly available dataset and its advantages are quantitatively demonstrated.

Multi-view people surveillance using 3D information

In this paper we introduce a novel surveillance system, which uses 3D information extracted from multiple cameras to detect, track and re-identify people. The detection method is based on a 3D Marked Point Process model using two pixel-level features extracted from multi-plane projections of binary foreground masks, and uses a stochastic optimization framework to estimate the position and the height of each person. We apply a rule based Kalman-filter tracking on the detection results to find the object-to-object correspondence between consecutive time steps. Finally, a 3D body model based long-term tracking module connects broken tracks and is also used to re-identify people.

A Bayesian Approach on People Localization in Multicamera Systems

In this paper, we introduce a Bayesian approach on multiple people localization in multicamera systems. First, pixel-level features are extracted, which are based on physical properties of the 2-D image formation process, and provide information about the head and leg positions of the pedestrians, distinguishing standing and walking people, respectively. Then, features from the multiple camera views are fused to create evidence for the location and height of people in the ground plane. This evidence accurately estimates the leg position even if either the area of interest is only a part of the scene or the overlap ratio of the silhouettes from irrelevant outside motions with the monitored area is significant. Using this information, we create a 3-D object configuration model in the real world. We also utilize a prior geometrical constraint, which describes the possible interactions between two pedestrians. To approximate the position of the people, we use a population of 3-D cylinder objects, which is realized by a marked point process. The final configuration results are obtained by an iterative stochastic energy optimization algorithm. The proposed approach is evaluated on two publicly available datasets, and compared to a recent state-of-the-art technique. To obtain relevant quantitative test results, a 3-D ground truth annotation of the real pedestrian locations is prepared, while two different error metrics and various parameter settings are proposed and evaluated showing the advantages of our proposed model.

Detection of unusual optical flow patterns by multilevel hidden Markov models

The analysis of motion information is one of the main tools for the understanding of complex behaviors in video. However, due to the quality of the optical flow of low-cost surveillance camera systems and the complexity of motion, new robust image-processing methods are required to generate reliable higher-level information. In our novel approach there is no need for tracking objects (vehicles, pedestrians) in order to recognize anomalous motion, but dense optical flow information is used to construct mixtures of Gaussians, which are analyzed temporally. We create a multilevel model, where low-level states of non-overlapping image regions are modeled by continuous hidden Markov models (HMMs). From low-level HMMs we compose high-level HMMs to analyze the occurrence of the low-level states. The processing of large numbers of data in traditional HMMs can result in a precision problem due to the multiplication of low probability values. Thus, besides introducing new motion models, we incorporate a scaling technique into the mathematical model of HMMs to avoid precision problems and to get an effective tool for the analysis of large numbers of motion vectors. We illustrate the use of our models with real-life traffic videos.

A multi-view annotation tool for people detection evaluation

In this paper we introduce a novel multi-view annotation tool for generating 3D ground truth data of the real location of people in the scene. The proposed tool allows the user to accurately select the ground occupancy of people by aligning an oriented rectangle on the ground plane. In addition, the height of the people can also be adjusted. In order to achieve precise ground truth data the user is aided by the video frames of multiple synchronized and calibrated cameras. Finally, the 3D annotation data can be easily converted to 2D image positions using the available calibration matrices. One key advantage of the proposed technique is that different methods can be compared against each other, whether they estimate the real world ground position of people or the 2D position on the camera images. Therefore, we defined two different error metrics, which quantitatively evaluate the estimated positions. We used the proposed tool to annotate two publicly available datasets, and evaluated the metrics on two state of the art algorithms.

Intelligent Multi Sensor Fusion System for Advanced Situation Awareness in Urban Environments

This paper presents a distributed multi sensor data processing and fusion system providing sophisticated surveillance capabilities in the urban environment. The system enables visual/non-visual event detection, situation assessment, and semantic event-based reasoning for force protection and civil surveillance applications. The novelties lie in the high level system view approach, not only concentrating on data fusion methodologies per se, but rather on a holistic view of sensor data fusion that provides both lower (sensor) level and higher level (semantic) fusion. At the same time, we concentrate on easy and quick extensibility with new sensors and processing capabilities. The system also makes provisions for visualizing and processing space-time alerts from sensor detections up to high level alerts based on rule-based semantic reasoning over sensor data and fusion events. The proposed architecture has been validated in a number of different synthetic and live urban scenarios.

VISRET - a content based annotation, retrieval and visualization toolchain

This paper presents a system for content-based video retrieval, with a complete toolchain for annotation, indexing, retrieval and visualization of imported data. The system contains around 20 feature descriptors, a modular infrastructure for descriptor addition and indexing, a web-based search interface and an easy-to-use query-annotation-result visualization module. The features that make this system differ from others is the support of all the steps of the retrieval chain, the modular support for standard MPEG-7 and custom descriptors, and the easy-to-use tools for query formulation and retrieval visualization. The intended use cases of the system are content- and annotation-based retrieval applications, ranging from community video portals to indexing of image, video, judicial, and other multimedia databases.

Shape-and-motion-fused multiple flying target recognition and tracking

This paper presents an automatic approach for camera/image based detection, recognition and tracking of flying objects (planes, missiles, etc.). The method detects appearing objects, and recognizes re-appearing targets. It uses a feature-based statistical modeling approach (e.g. HMM) for motion-based recognition, and an image feature (e.g. shape) based indexed database of pre-trained object classes, suitable for recognition on known and alerting on unknown objects. The method can be used for detection of flying objects, recognition of the same object category through multiple views/cameras and signal on unusual motions and shape appearances.

HMM-based unusual motion detection without tracking

We propose novel pixel dense modeling of motion of urban traffic in noisy environments with the help of multidimensional Gaussian Mixture Models (GMMs) and Hidden Markov Models (HMMs). In our approach there is no need for object tracking in order to detect anomalous motion or to model and visualize the fluctuation of traffic. We propose a new scaling method introduced into the HMM to get a robust tool for the analysis of hundreds of motion vector samples at a time. We show the use of our model with a photorealistic video synthetized from real life recordings.

Multi-camera people localization and height estimation using multiple birth-and-death dynamics

This paper presents a novel tool for localizing people in multicamera environment using calibrated cameras. Additionally, we will estimate the height of each person in the scene. Currently, the presented method uses the human body silhouettes as input, but it can be easily modified to process other widely used object (e.g. head, leg, body) detection results. In the first step we project all the pixels of the silhouettes to the ground plane and to other parallel planes with different height. Then we extract our features, which are based on the physical properties of the 2-D image formation. The final configuration results (location and height) are obtained by an iterative stochastic optimization process, namely the multiple birth-and-death dynamics framework.