Marc Lievin

Nonlinear color space and spatiotemporal MRF for hierarchical segmentation of face features in video

This paper deals with the low-level joint processing of color and motion for robust face analysis within a feature-based approach. To gain robustness and contrast under unsupervised viewing conditions, a nonlinear color transform relevant for hue segmentation is derived from a logarithmic model. A hierarchical segmentation scheme is based on Markov random field modeling, that combines hue and motion detection within a spatiotemporal neighborhood. Relevant face regions are segmented without parameter tuning. The accuracy of the label fields enables not only face detection and tracking but also geometrical measurements on facial feature edges, such as lips or eyes. Results are shown both on typical test sequences and on various sequences acquired from micro- or mobile-cameras. The efficiency of the method makes it suitable for real-time applications aiming at audiovisual communication in unsupervised environments.

Automatic lip tracking: Bayesian segmentation and active contours in a cooperative scheme

An algorithm for speakers lip contour extraction is presented in this paper. A color video sequence of the speakers face is acquired under natural lighting conditions and without any particular make-up. First, a logarithmic color transform is performed from RGB to HI (hue, intensity) color space. A Bayesian approach segments the mouth area using Markov random field modelling. Motion is combined with red hue lip information into a spatiotemporal neighbourhood. Simultaneously, a region of interest and relevant boundary points are automatically extracted. Next, an active contour using spatially varying coefficients is initialised with the results of the preprocessing stage. Finally, an accurate lip shape with inner and outer borders is obtained with good quality results in this challenging situation.

Spatiotemporal MRF approach to video segmentation: application to motion detection and lip segmentation

In this paper, a spatiotemporal strategy for image sequence analysis is proposed: a video sequence is processed as a 3-D data batch instead of a series of 2-D images. Applying this approach to motion detection, a 3-D Markovian model associated with a spatiotemporal relaxation is defined. Using a 3-D neighbourhood of pixels for modelling spatiotemporal interactions, robust results are obtained for detecting moving objects in noisy sequences or in the case of overlapping motion. In order to improve the performance to detect poorly-textured objects or very slow motion, the algorithm is integrated in a spatiotemporal multiresolution scheme. The data pyramid is built by using 3-D low-pass filtering and 3-D subsampling. Robust results for synthetic and real-world outdoor image sequences are reported. This approach is also applied successfully to speakers lip segmentation in image sequences, for audiovisual telecommunication.

Unsupervised lip segmentation under natural conditions

An unsupervised algorithm for speakers lip segmentation is presented. A color video sequence of the speakers face is acquired, under natural lighting conditions and without any particular make-up. First, a logarithmic color transform is performed from the RGB to HI (hue, intensity) color space and sequence dependant parameters are evaluated. Second, a statistical approach using Markov random field modeling segment the mouth shape using the red hue predominant region and motion in a spatiotemporal neighborhood. Simultaneously, a region of interest (ROI) is automatically extracted. Third, the speakers lip shape is extracted from the final hue field with good quality results in this challenging situation.

On the use of entropy power for threshold selection

This paper deals with an entropic approach as unsupervised thresholding technique for image processing, in order to extract a relevant binary information from noisy data. It is dedicated to situations where a signal of relatively high energy is localized in the image whereas the noise is spread over the entire image. The method is based on the computation of the entropy power of the information source, as defined by Shannon. The threshold used for binarization is proportional to the entropic deviation of the observation source. The performance of the approach is illustrated by two classical image preprocessing tasks, namely motion detection and edge detection. The evaluation set contains both synthetic data and real-world image sequences.

Towards robust lip tracking

An algorithm for the automatic extraction of speakers lips in video sequences is presented. Our goal is to extract minimum face feature parameters, vital for audio-visual communication, in adverse conditions and at a very low bit rate coding. Our method uses spatial (region and contour) and temporal (similarity function) information from luminance and hue components. A new literal inverse of the active contours stiffness matrix is introduced. This ensures a fast and accurate convergence of active contours towards lip boundaries. The use of the Kanade-Lucas tracking algorithm with our point extraction method leads to an automatic, fast and robust initialization of Snakes. The significant robustness enhancement and related computational cost decrease allow processing approaching real time.

Lip features automatic extraction

An algorithm for speakers lip segmentation and features extraction is presented. A color video sequence of speakers face is acquired, under natural lighting conditions and without any particular make-up. First, a logarithmic color transform is performed from the RGB to HI (hue, intensity) color space. Second, a statistical approach using Markov random field modeling determines the red hue prevailing region and motion in a spatiotemporal neighborhood. Third, the final label field is used to extract ROI (region of interest) and geometrical features.

Stereoscopic augmented reality system for computer-assisted surgery.

Abstract A first architecture for an augmented reality system in computer-assisted surgery is presented in this paper. Like in “X-ray vision” systems, a stereoscopic overlay is visually superimposed on the patient. The main purpose of our approach is user-friendliness for the surgeon: no additive wearing equipment is required. Registration, rigid body location and 3D volume computation are proven to respect real-time processing, thanks to an optical navigation system and our integrated software framework. Studies are undertaken to replace our actual monitor display by an upcoming holographic screen.

A hierarchical segmentation algorithm for face analysis. Application to lipreading

A hierarchical algorithm for face analysis is presented. A color video sequence of the speakers face is acquired under natural lighting conditions and without any particular make-up. The application aims at providing geometrical features of the face for scalable video transmission when no specific model of the speaker face is assumed. First, a logarithmic hue transform is performed from RGB to HI (hue, intensity) color space. Next, a Markov random field modeling regularizes motion and hue information within a spatiotemporal neighborhood. The hierarchical segmentation labels the different areas of the face. Results are shown on the lower part of the face and compared with a standard color segmentation algorithm (fuzzy c-means). A speakers lip shape with inner and outer borders is extracted from the final labeling and used to initialize an active contour stage.

Nerves - level sets for interactive 3D segmentation of nerve channels

In this paper, we present a novel method for 3D segmentation of the nerve channels in the human mandible that contain the nervus alveolaris inferior. The technique utilizes geodesic active surfaces that are implemented with level sets. The method consists of two steps: (i) After defining two points, which denote the entry and exit of the nerve channel, a connecting path of minimal action inside the channel is calculated. This calculation is driven by the gray values in proximity to the two defined points inside the channel, (ii) Using this path as an initial configuration, an active surface evolves until the inner borders of the channel are reached. Since this initial configuration is located very close to the borders of the channel, a propagation term is not necessary in this model.