Jean-Yves Bouguet

A real time system for robust 3D voxel reconstruction of human motions

We present a multi-PC/camera system that can perform 3D reconstruction and ellipsoid fitting of moving humans in real time. The system consists of five cameras. Each camera is connected to a PC which locally extracts the silhouettes of the moving person in the image captured by the camera. The five silhouette images are then sent, via local network, to a host computer to perform 3D voxel-based reconstruction by an algorithm called SPOT. Ellipsoids are then used to fit the reconstructed data. By using a simple and user-friendly interface, the user can display and observe, in real time and from any view-point, the 3D models of the moving human body. With a rate of higher than 15 frames per second, the system is able to capture non-intrusively, a sequence of human motions.

Model-based face tracking for view-independent facial expression recognition

Facial expression recognition is necessary for designing any realistic human-machine interfaces. Previous published facial expression recognition systems achieve good recognition rates, but most of them perform well only when the user faces the camera and does not change his 3D head pose. We propose a new method for robust, view-independent recognition of facial expressions that does not make this assumption. The system uses a novel 3D model-based tracker to extract simultaneously and robustly the pose and shape of the face at every frame of a monocular video sequence. There are two main contributions. First, we demonstrate that the 3D information extracted through 3D tracking enables robust facial expression recognition in spite of large rotational and translational head movements (up to 90 degrees in head rotation). Second, we show that the Support Vector Machine is a suitable engine for robust classification. Recognition rates as high as 91 percent are achieved at classifying 5 distinct dynamic facial motions (neutral, opening/closing mouth, smile, raising eyebrow).

A data-driven model for monocular face tracking

This paper describes a two-stage system for 3D tracking of pose and deformation of the human face in monocular image sequences without the use of special markers. The first stage of the system learns the space of all possible facial deformations by applying principal component analysis on real stereo tracking data. The resulting model approximates any generic shape as a linear combination of shape basis vectors. The second stage of the system uses this low-complexity deformable model for simultaneous tracking of pose and deformation of the face from a single image sequence. This stage is known as model-based monocular tracking. There are three main contributions of this paper. First we demonstrate that a data-driven approach for model construction is suitable for tracking non rigid objects and offers an elegant and practical alternative to the task of manual construction of models using 3D scanners or CAD modelers. Second, we show that such a method exhibits good tracking accuracy (errors less than 5 mm) and robustness characteristics. Third, we demonstrate that our system exhibits very promising generalization properties in enabling tracking of multiple persons with the same 3D model.

3D Photography Using Shadows in Dual-Space Geometry

A simple and inexpensive approach for extracting the three-dimensional shape of objects is presented. It is based on ‘weak structured lighting’. It requires very little hardware besides the camera: a light source (a desk-lamp or the sun), a stick and a checkerboard. The object, illuminated by the light source, is placed on a stage composed of a ground plane and a back plane; the camera faces the object. The user moves the stick in front of the light source, casting a moving shadow on the scene. The 3D shape of the object is extracted from the spatial and temporal location of the observed shadow. Experimental results are presented on five different scenes (indoor with a desk lamp and outdoor with the sun) demonstrating that the error in reconstructing the surface is less than 0.5% of the size of the object. A mathematical formalism is proposed that simplifies the notation and keep the algebra compact. A real-time implementation of the system is also presented.

Visual navigation using a single camera

We assess the usefulness of monocular recursive motion estimation techniques for vehicle navigation in the absence of a model for the environment. For this purpose we extend a recently proposed recursive motion estimator, the Essential filter, to handle scale estimation. We examine experimentally the accuracy with which the motion and position of the vehicle may be computed on an 8000 frame indoors sequence. The issues of sampling time frequency and number of necessary features in the environment are addressed systematically.<<ETX>>

What do planar shadows tell about scene geometry

A method for reconstructing 3D scene geometry from a set of projected shadows is presented. It is composed of two stages. First, the scene geometry is retrieved up to three scalar unknowns using only the information contained in the observed shadow edges on the image plane. Then, the three remaining unknowns are computed making use of the known depths at three points. This technique improves upon previous results in that it does not require the presence of a reference plane in the background. A mathematical analysis is presented using dual-space geometry, a formalism that provides adequate tools to carry out all the derivations in a compact and intuitive manner. A linear algorithm based on singular value decomposition (SVD) is presented leading to a closed form solution for reconstruction.