Patrice Delmas

Automatic snakes for robust lip boundaries extraction

Active contours or snakes are widely used in object segmentation for their ability to integrate feature extraction and pixel candidate linking in a single energy minimizing process. But the sensitivity to parameters values and initialization is also a widely known problem. The performance of snakes can be enhanced by better initialization close to the desired solution. We present a fine mouth region of interest (ROI) extraction using gray level image and corresponding gradient information. We link this technique with an original snake method. The automatic snakes use spatially varying coefficients to remain along its evolution in a mouth-like shape. Our experimentations on a large image database prove its robustness regarding speakers change of the ROI mouth extraction and automatic snakes algorithms. The main application of our algorithms is video-conferencing.

Towards robust lip tracking

An algorithm for the automatic extraction of speakers lips in video sequences is presented. Our goal is to extract minimum face feature parameters, vital for audio-visual communication, in adverse conditions and at a very low bit rate coding. Our method uses spatial (region and contour) and temporal (similarity function) information from luminance and hue components. A new literal inverse of the active contours stiffness matrix is introduced. This ensures a fast and accurate convergence of active contours towards lip boundaries. The use of the Kanade-Lucas tracking algorithm with our point extraction method leads to an automatic, fast and robust initialization of Snakes. The significant robustness enhancement and related computational cost decrease allow processing approaching real time.

Diagnostic radiograph based 3D bone reconstruction framework: Application to the femur

Three dimensional (3D) visualization of anatomy plays an important role in image guided orthopedic surgery and ultimately motivates minimally invasive procedures. However, direct 3D imaging modalities such as Computed Tomography (CT) are restricted to a minority of complex orthopedic procedures. Thus the diagnostics and planning of many interventions still rely on two dimensional (2D) radiographic images, where the surgeon has to mentally visualize the anatomy of interest. The purpose of this paper is to apply and validate a bi-planar 3D reconstruction methodology driven by prominent bony anatomy edges and contours identified on orthogonal radiographs. The results obtained through the proposed methodology are benchmarked against 3D CT scan data to assess the accuracy of reconstruction. The human femur has been used as the anatomy of interest throughout the paper. The novelty of this methodology is that it not only involves the outer contours of the bony anatomy in the reconstruction but also several key interior edges identifiable on radiographic images. Hence, this framework is not simply limited to long bones, but is generally applicable to a multitude of other bony anatomies as illustrated in the results section.

Active Contour Based Segmentation of 3D Surfaces

Algorithms incorporating 3D information have proven to be superior to purely 2D approaches in many areas of computer vision including face biometrics and recognition. Still, the range of methods for feature extraction from 3D surfaces is limited. Very popular in 2D image analysis, active contours have been generalized to curved surfaces only recently. Current implementations require a global surface parametrisation. We show that a balloon force cannot be included properly in existing methods, making them unsuitable for applications with noisy data. To overcome this drawback we propose a new algorithm for evolving geodesic active contours on implicit surfaces. We also introduce a new narrowband scheme which results in linear computational complexity. The performance of our model is illustrated on various real and synthetic 3D surfaces.

Multi-Kinect scene reconstruction: Calibration and depth inconsistencies

We investigated calibration procedures of multiple Kinect sensors simultaneously. Through standard calibration algorithms our multi-Kinect system is accurately registered. We propose and implemented a multi-Kinect system to seamlessly render a scene from a wide range of angles. Such a system is capable of real-time operation. We also investigated the problem of inconsistent depth measurement between different Kinect units, and arrived at the same conclusion as the state-of-the-art regarding the characteristics of depth measurement errors in the Kinect sensor. In order to compensate these errors for live acquisition and display of multi-Kinect systems, we introduced an offline ICP calibration of multiple Kinect data. Our experimental results provide a robust way to properly align multiple Kinect data.

A practical comparison between Zhang's and Tsai's calibration approaches

With the rise of affordable processing power and off-the-shelf apparatus supporting 3D imaging, there is a growing need for reliable and fast calibration tools, enabling timely accurate data gathering. When confronted with a choice of camera calibration tools, Zhangs and Tsais are not only the most cited, but also the most widely available solutions. Zhangs calibration is often chosen by default, based on the assumption that it is more accurate. However, it typically involves extensive manual data gathering when compared to the Tsai approach. Here, we demonstrate that there is no significant accuracy gain between Tsais or Zhangs approach in terms of stereo matching, given the variety of readily available 3D devices tested. Further to this, the trade-off between measurement accuracy compared to setup and data acquisition time is decisively in favour of Tsai. This paper also covers a new algorithm for the extraction of points from images of checkboards attached to calibration objects.

Comparison of Active Structure Lighting Mono and Stereo Camera Systems: Application to 3D Face Acquisition

In this paper, we compare three structured lighting techniques (gray code, gray code shift and stripe boundary) using mono or stereo camera systems to assess 3D reconstruction for human faces. The calibration technique specific to each system is described. The different algorithms were assessed quantitatively and qualitatively with respect to benchmark data obtained from a high precision 3D scanner. We demonstrate that although a mono camera system coupled to a LCD projector runs faster and at lower cost, the stereo camera system aided by structure lighting generates more accurate results

Web-based on-line computational stereo vision

A design of an on-line Java applet that allows the user to upload images being close to a stereo pair and reconstruct a depth map of a depicted 3D scene is described. It is accessible to a wide range of users and can serve as a tool that allows the Internet users to learn and explore the research domain of computational stereo vision. The images are automatically rectified into an epipolar stereo pair after several corresponding points were interactively specified. The rectified pair is converted into an anaglyphic image of the scene, or a dense depth map is reconstructed from this pair using available or even user-submitted stereo algorithms. After the depth map is computed, the user can display arbitrary 3D views of the scene based on the depth and textural information. The implemented Web-community sharing port allows the users to upload and share their own solutions. The current version of this applet is temporarily stored at http://www.cs.auckland.ac.nz/~mngu012.

An evaluation of three popular computer vision approaches for 3-d face synthesis

We have evaluated three computer approaches to 3-D reconstruction – passive computational binocular stereo and active structured lighting and photometric stereo – in regard to human face reconstruction for modelling virtual humans. An integrated experimental environment simultaneously acquired images for 3-D reconstruction and data from a 3-D scanner which provided an accurate ground truth. Our goal was to determine whether today’s computer vision approaches are accurate and fast enough for practical 3-D facial reconstruction applications. We showed that the combination of structured lighting with symmetric dynamic programming stereo has good prospects with reasonable processing time and accuracy.

High-Order MGRF Models for Contrast/Offset Invariant Texture Retrieval

Local ordinal signal relations, such as local binary or ternary patterns (LBP/LTP), and their statistics are promising texture descriptors due to their invariance to frequent in practice spatially variant contrast / offset deviations that preserve image appearance. This paper extends conventional LBP/LTP-based classifiers towards learning, rather than prescribing characteristic shapes, sizes, and numbers of such patterns in order to facilitate accurate image query based texture retrieval. The proposed learning and retrieval framework models a query image as a sample from a high-order Markov-Gibbs random field (MGRF) and uses the patterns learned and their training statistics to classify candidate images in a certain database and retrieve samples, which are similar to the query texture. Analytical approximations of the model parameters guide selecting characteristic patterns of a given order, the higher order patterns being learned on the basis of the already found lower order ones. Comparative experiments on four texture databases confirmed that the learned models with multiple high-order (from the 3rd to 8th order) LTPs, often just from the 4th order, consistently outperform the conventional prescribed 8th-order fixed-shape LBP/LTPs.