José Luis Landabaso

Shape from inconsistent silhouette

Shape from silhouette (SfS) is the general term used to refer to the techniques that obtain a volume estimate from a set of binary images. In a first step, a number of images are taken from different positions around the scene of interest. Later, each image is segmented to produce binary masks, also called silhouettes, to delimit the objects of interest. Finally, the volume estimate is obtained as the maximal one which yields the silhouettes. The set of silhouettes is usually considered to be consistent which means that there exists at least one volume which completely explains them. However, silhouettes are normally inconsistent due to inaccurate calibration or erroneous silhouette extraction techniques. In spite of that, SfS techniques reconstruct only that part of the volume which projects consistently in all the silhouettes, leaving the rest unreconstructed. In this paper, we extend the idea of SfS to be used with sets of inconsistent silhouettes. We propose a fast technique for estimating that part of the volume which projects inconsistently and propose a criteria for classifying it by minimizing the probability of miss-classification taking into account the 2D error detection probabilities of the silhouettes. A number of theoretical and empirical results are given, showing that the proposed method reduces the reconstruction error.

Segmentation and tracking of static and moving objects in video surveillance scenarios

In this paper we present a real-time object tracking system for monocular video sequences with static camera. The workflow is based on a pixel-based foreground detection system followed by foreground object tracking. The foreground detection method performs the segmentation in three levels: Moving Foreground, Static Foreground and Background level. The tracking uses the foreground segmentation for identifying the tracked objects, but minimizes the reliance on the foreground segmentation, using a modified Mean Shift tracking algorithm. Combining this tracking system with the Multi-Level foreground segmentation, we have improved the tracking results using the classification in static or moving objects. The system solves successfully a high percentage of the moving objects occlusions, and most of the occlusions between static and moving objects.

Depth estimation based on multiview matching with depth/color segmentation and memory efficient Belief Propagation

3D technologies are becoming the more and more relevant in recent years. Visual communications, as well as image and video analysis, benefit in great manner from spatial information such as depth for various applications. Highly accurate visual depth estimation often involves complex optimization algorithms in order to fit proper estimation models to data. From a stereo/multiview matching perspective, local and global algorithms exist. Commonly, the latter are more complex and accurate, as data models are used to take the global structure into account. Belief Propagation has proven to be a good global algorithmic framework for depth estimation. By means of an iterative procedure, it is able to regularize, according to set of local smoothness and geometry constrains, an initial estimation of depth by a local approach such as simple block matching. However, information transfer from iteration to iteration by means of message passing can be excessively demanding in terms of memory bandwidth and usage. In this paper, a new Belief Propagation based algorithm with multiview matching with depth/color segmentation is proposed together with a strategy for message passing compression. Experimental results show the algorithm to be competitive with best performing ones in the state of the art, while reducing by a factor 10 the memory usage, with marginal loss in performance, of a typical Belief Propagation strategy.

Emotion recognition based on MPEG-4 Facial Animation Parameters

In this paper a facial expression recognition system is presented. The system is based on the modelling of the expressions by means of Hidden Markov Models. The observations used to create the models are the MPEG-4 standardized Facial Animation Parameters (FAPs). The FAPs of a video sequence are first extracted and then analyzed using semi-continuous HMM. The system shows good performance for distinguishing isolated expressions and can also be used, with lower accuracy, to extract the expressions in long video sequences where speech is mixed with silence frames.

Robust Tracking and Object Classification Towards Automated Video Surveillance

This paper addresses some of the key issues in computer vision that contribute to the technical advances and system realisation for automated visual events analysis in video surveillance applications. The objectives are to robustly segment and track multiple objects in the cluttered dynamic scene, and, if required, further classify the objects into several categories, e.g. single person, group of people or car. There are two major contributions being presented. First, an effective scheme is proposed for accurate cast shadows / highlights removal with error corrections based on conditional morphological reconstruction. Second, a temporal template-based robust tracking scheme is introduced, taking account of multiple characteristic features (velocity, shape, colour) of a 2D object appearance simultaneously in accordance with their respective variances. Extensive experiments on video sequences of variety real-world scenarios are conducted, showing very promising tracking performance, and the results on PETS2001 sequences are illustrated.

A Unified Framework for Consistent 2-D/3-D Foreground Object Detection

This paper addresses 2-D and 3-D active entity detection in video scenes. Active entities are the foreground parts in a stationary background scene and they typically correspond to the regions of interest in many applications such as video surveillance, object and person tracking, and suspicious object detection, among others. We present a novel framework that permits obtaining 2-D and 3-D active entities as an inter-dependent probabilistic procedure. In the process of creating this framework, a study has been conducted to explore ways to generalize existing activity detection techniques to a Bayesian form. A new Bayesian 3-D activity detection technique has been developed. The Bayesian framework gives a unified manner to interact between the planar and the volumetric detection tasks and helps to prevent the propagation of noisy pixel observations to the 3-D space. However, when large systematic errors occur in the 2-D detection level, a different approach has to be taken to correct them. We use a new 3-D foreground detection scheme that is able to correct errors in 2-D planar detections by checking the consistency between 3-D foreground detections and the set of corresponding 2-D foreground regions.

UPC audio, video and multimodal person tracking systems in the clear evaluation campaign

Reliable measures of person positions are needed for computational perception of human activities taking place in a smart-room environment. In this work, we present the Person Tracking systems developed at UPC for audio, video and audio-video modalities in the context of the EU funded CHIL project research activities. The aim of the designed systems, and particularly of the new contributions proposed, is to deal robustly in both single and multiperson localization tasks independently on the environmental conditions. Besides the technology description, experimental results conducted for the CLEAR evaluation workshop are also reported.

Reconstruction of 3D Shapes Considering Inconsistent 2D Silhouettes

The visual hull is defined as the intersection of the visual cones formed by the back-projection of C 2D silhouettes into the 3D space. The set of 2D silhouettes is consistent if there exists at least one volume which exactly explains them. Shape from silhouette (SfS) is the general term used to refer to the techniques employed to obtain a volume from silhouettes, which are considered to be consistent. In this paper we extend the idea of SfS to be used with sets of inconsistent silhouettes resulting from inaccurate calibration and erroneous 2D silhouette extraction techniques. The method presented detects and corrects errors in the silhouettes based on the consistency principle, implying an unbiased treatment of false alarms and misses in 2D.

A global probabilistic framework for the foreground, background and shadow classification task

Over the years, many works have been published on the two-dimensional foreground segmentation task, describing different methods that treat to extract that part of the scene containing active entities. In most of the cases, the stochastic background process for each pixel is modeled first, and then the foreground pixels are classified as an exception to the model or using maximum a posteriori (MAP) or maximum likelihood (ML). The shadow is usually removed in a later stage and salt and pepper noise is treated with connected component analysis or mathematical morphology. In this paper, we propose a global method that classifies each pixel by finding the best possible class (foreground, background, shadow) examining the image globally. A Markov Random Field is used to represent the dependencies between all the pixels and classes and the global optimal solution is approximated with the Belief Propagation algorithm. The method can extend most local methods and increase their accuracy. In addition, this approach brings a probabilistic justification of the classification problem and it avoids the use of additional post-processing techniques.

Multi-view depth estimation based on visual-hull enhanced Hybrid Recursive Matching for 3D video conference systems

This paper discusses the problem of high quality depth map estimation for real-time systems. Our work is based on the European FP7 project 3DPresence which aims to build a multi-view and multiuser 3D videoconferencing system. Based on new multi-view auto-stereoscopic display technology the remote conferees will be rendered as an integral part of a three dimensional virtual shared environment. In order to create the related views for the 3D displays as well as to virtually correct the eye contact problem robust depth maps are required. For this purpose, in this paper we will discuss the fusion of two competing approaches which have, from a camera configuration point of view, contrary to each other properties. Namely, we will combine the volumetric Visual Hull (VH) approach with the stereo matching based Hybrid Recursive Matching (HRM) to a new method which benefits from the advantages of both techniques and discards their weak points.