Luigi Bagnato

Optical flow and depth from motion for omnidirectional images using a TV-L1 variational framework on graphs

This paper deals with the problem of efficiently computing the optical flow of image sequences acquired by omnidirectional (nearly full field of view) cameras. We formulate the problem in the natural spherical geometry associated with these devices and extend a recent TV-L1 variational formulation for computing the optical flow [1]. The discretization of differential operators occurring in this formulation turns out to be an extremely sensitive point, in particular for the TV part of our algorithm. We show that these difficulties can be very efficiently overcome using a graph-based formulation of TV denoising, which we solve by introducing a graph version of Chambolles algorithm [2]. A slight modification of the original framework allows us to solve the depth from motion problem using the same techniques. In both cases, our graph-based algorithms provide computationally efficient solutions and significantly outperform naive implementations based on direct discretization of the operators, or on neglecting the influence of geometry.

A Variational Framework for Structure from Motion in Omnidirectional Image Sequences

We address the problem of depth and ego-motion estimation from omnidirectional images. We propose a correspondence-free structure-from-motion problem for sequences of images mapped on the 2-sphere. A novel graph-based variational framework is first proposed for depth estimation between pairs of images. The estimation is cast as a TV-L1 optimization problem that is solved by a fast graph-based algorithm. The ego-motion is then estimated directly from the depth information without explicit computation of the optical flow. Both problems are finally addressed together in an iterative algorithm that alternates between depth and ego-motion estimation for fast computation of 3D information from motion in image sequences. Experimental results demonstrate the effective performance of the proposed algorithm for 3D reconstruction from synthetic and natural omnidirectional images.

Hardware implementation of an omnidirectional camerawith real-time 3D imaging capability

A novel hardware implementation of an omnidirectional image sensor is presented which is capable of acquiring and processing 3D image sequences in real time. The system consists of a hemispherical arrangement of a large number of CMOS imagers, connecting to a layered arrangement of a high-end FPGA platform that is responsible data framing and image processing. The hardware platform in charge of real-time processing the 3.8 Gb/s data which is generated by the cameras is presented, and a first application of the system consisting of omnidirectional image acquisition is demonstrated.

Plenoptic spherical sampling

We present a novel plenoptic sampling scheme that permits an efficient representation of the full light ray field in a space limited by a convex closed surface. We show that a convenient way to sample the light ray field around an observer consists in using a discrete set of perspective imagers with overlapping field-of-views that are distributed on a closed convex surface and looking along the normal to the surface. Taking inspiration from the vision system of flying insects, we choose to constrain the cameras on a sphere of finite radius. Building on spectral analysis we propose a sampling scheme that permits to reconstruct the spherical light field without aliasing. We validate our framework through experiments in a synthetic environment and we show that our constructive sampling scheme permits to effectively reconstruct the light field without artifacts.

Robust infants face tracking using active appearance models: a mixed-state condensation approach

In this paper a new extension of the CONDENSATION algorithm, with application to infants face tracking, will be introduced. In this work we address the problem of tracking a face and its features in baby video sequences. A mixed state particle filtering scheme is proposed, where the distribution of observations is derived from an active appearance model. The mixed state approach combines several dynamic models in order to account for different occlusion situations. Experiments on real video show that the proposed approach augments the tracker robustness to occlusions while maintaining the computational time competitive.

Foreground silhouette extraction robust to sudden changes of background appearance

Vision-based background subtraction algorithms model the intensity variation across time to classify a pixel as foreground. Unfortunately, such algorithms are sensitive to appearance changes of the background such as sudden changes of illumination or when videos are projected in the background. In this work, we propose an algorithm to extract foreground silhouettes without modeling the intensity variation across time. Using a camera pair, the stereo mismatch is processed to produce a dense disparity based on a Total Variation (TV) framework. Experimental results show that with sudden changes of background appearance, our proposed TV disparity-based extraction outperforms intensity-based algorithms and existing stereo-based approaches based on temporal depth variation and stereo mismatch.

Plenoptic based super-resolution for omnidirectional image sequences

This paper addresses the reconstruction of high resolution omnidirectional images from a low resolution video acquired by an omnidirectional camera moving in a static scene. In order to exploit the additional information provided by the side images in the video sequence, the ego-motion of the camera must be accurately estimated in a first step. The reconstruction can then be modeled as a plenoptic sampling problem that has to encompass the change of viewpoint between each position of the omnidirectional sensor and the specific discretization of the real scene observed from each position. We formulate this problem as an ill-posed inverse problem that incorporates a regularization term based on a Total Variation (TV) prior. A graph variational formulation is used in order to ease the representation of omnidirectional data and to adapt the discretization of differential operators to the omnidirectional geometry. Experimental results on synthetic images demonstrate the relevance of this approach and its superiority compared to standard super-resolution using a single image.