Faysal Boughorbel

Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition

AbstractThis paper describes a new software-based registration and fusion of visible and thermal infrared (IR) image data for face recognition in challenging operating environments that involve illumination variations. The combined use of visible and thermal IR imaging sensors offers a viable means for improving the performance of face recognition techniques based on a single imaging modality. Despite successes in indoor access control applications, imaging in the visible spectrum demonstrates difficulties in recognizing the faces in varying illumination conditions. Thermal IR sensors measure energy radiations from the object, which is less sensitive to illumination changes, and are even operable in darkness. However, thermal images do not provide high-resolution data. Data fusion of visible and thermal images can produce face images robust to illumination variations. However, thermal face images with eyeglasses may fail to provide useful information around the eyes since glass blocks a large portion of thermal energy. In this paper, eyeglass regions are detected using an ellipse fitting method, and replaced with eye template patterns to preserve the details useful for face recognition in the fused image. Software registration of images replaces a special-purpose imaging sensor assembly and produces co-registered image pairs at a reasonable cost for large-scale deployment. Face recognition techniques using visible, thermal IR, and data-fused visible-thermal images are compared using a commercial face recognition software (FaceIt®) and two visible-thermal face image databases (the NIST/Equinox and the UTK-IRIS databases). The proposed multiscale data-fusion technique improved the recognition accuracy under a wide range of illumination changes. Experimental results showed that the eyeglass replacement increased the number of correct first match subjects by 85% (NIST/Equinox) and 67% (UTK-IRIS).

A new method for the registration of three-dimensional point-sets: The Gaussian Fields framework

In this paper, we present a 3D automatic registration method based on Gaussian Fields and energy minimization. A continuously differentiable energy function is defined, which is convex in a large neighborhood of the alignment parameters. We show that the size of the region of convergence can be significantly extended reducing the need for close initialization and overcoming local convergence problems of the standard Iterative Closest Point (ICP) algorithms. Moreover, the Gaussian criterion can be applied with linear computational complexity using Fast Gauss Transform methods. Experimental evaluation of the technique using synthetic and real datasets demonstrates the usefulness as well as the limits of the approach.

Laser ranging and video imaging for bin picking

This paper describes an imaging system that was developed to aid industrial bin picking tasks. The purpose of this system was to provide accurate 3D models of parts and objects in the bin, so that precise grasping operations could be performed. The technology described here is based on two types of sensors: range mapping scanners and video cameras. The geometry of bin contents was reconstructed from range maps and modeled using superquadric representations, providing location and parts surface information that can be employed to guide the robotic arm. Texture was also provided by the video streams and applied to the recovered models. The system is expected to improve the accuracy and efficiency of bin sorting and represents a step toward full automation.

Gaussian fields: a new criterion for 3D rigid registration

Abstract This paper introduces a new and simple criterion for rigid registration based on Gaussian fields. The criterion is always differentiable and convex in a large neighborhood of the alignment parameters; allowing for the use of well-proven optimization techniques. Using this method we can extend the size of the region of convergence so that no close initialization is needed, thus overcoming local convergence problems of Iterative Closest Point algorithms. Furthermore, the Gaussian energy function can be evaluated with linear complexity using the fast Gauss transform, which permits efficient implementation of the registration algorithm. Experimental analysis on real-world data sets shows the usefulness and points the limits of the approach.

A New Method for Automatic 3D Face Registration

In view of today’s security concerns, 3D face reconstruction and recognition has gained a significant position in computer vision research. Depth information of a 3D face can be used to solve the problems of illumination and pose variation associated with face recognition. Registration is an integral part of any reconstruction process and hence we focus on the problem of automatic registration of 3D face point sets through a criterion based on Gaussian fields. The method defines a simple energy function, which is always differentiable and convex in a large neighborhood of the alignment parameters; allowing for the use of powerful standard optimization techniques. The new method overcomes the necessity of close initialization, which is required by Iterative Closest Point algorithm. Moreover, the use of the Fast Gauss Transform reduces the computational complexity of the registration algorithm.

Gaussian energy functions for registration without correspondences

A new criterion based on Gaussian fields is introduced and applied to the task of automatic rigid registration of point-sets. The method defines a simple energy function, which is always differentiable and convex in a large neighborhood of the alignment parameters; allowing for the use of powerful standard optimization techniques. We show that the size of the region of convergence can be extended so that no close initialization is needed, thus overcoming local convergence problems of iterative closest point algorithms. Furthermore, the Gaussian energy function can be evaluated with the linear complexity using the fast Gauss transform, which permits efficient implementation of the registration algorithm. Analysis through several experimental results on real world datasets shows the practicality and points out the limits of the approach.

Adaptive Filters for Depth from Stereo and Occlusion Detection

In this paper we present two novel techniques developed in the context of the stereo to multi-view conversion research at Philips in support of the introduction of stereoscopic and auto-stereoscopic. First, we show that we can use a relatively simple filtering approach, based on the recently popular bilateral filters, to address the correspondence problem, which is at the heart of depth and motion estimation. The proposed recursive filter uses Gaussian kernels to filter best matches and to incorporate image-based constraints. It iteratively refines the depth values starting from a random initialization and converges in a limited number of iterations to a time-stable high-quality depth map. The second contribution of the paper is an occlusion detection method that uses robust filtering for the detection of occlusion that is primarily based on the analysis of the variation of the matching metric used in the disparity estimation process. The basic underlying ideas behind the occlusion detection method are (1) that occluded areas are highly likely to be located near image boundaries (where luminance or color changes abruptly), and (2) occluded regions are characterized by a large decrease in the quality of the matching metric across these boundaries. The two algorithms were tested on real-world stereoscopic video content showing promising results.

A New Multiple-Windows Depth from Stereo Algorithm for 3D Displays

The research presented in this paper focuses on the conversion of stereoscopic video material into an image + depth format suitable for rendering on the multiview auto-stereoscopic displays of Philips. The recent interest shown in the movie industry for 3D significantly increased the availability of stereo material. In this context the conversion from stereo to the input formats of 3D displays becomes an important task. In this paper we present a stereo algorithm that uses multiple footprints generating several depth candidates for each image pixel. We characterize the various matching windows and we devise a robust strategy for extracting high quality estimates from the resulting depth candidates. The proposed algorithm is based on a surface filtering method that employs, simultaneously, the available depth estimates in a small local neighborhood while ensuring correct depth discontinuities by the inclusion of image constraints. The resulting high-quality image-aligned depth maps proved an excellent match with our 3D displays.

Automatic registration of 3D datasets using Gaussian fields

In this paper we introduce a new 3D automatic registration method based on Gaussian fields and energy minimization. The method defines a simple C/sup /spl infin// energy function, which is convex in a large neighborhood of the alignment parameters; allowing for the use of powerful standard optimization techniques. We show that the size of the region of convergence can be significantly extended reducing the need for close initialization and overcoming local convergence problems of the standard iterative closest point (ICP) algorithms. Furthermore, the Gaussian criterion can be evaluated with linear computational complexity using fast Gauss transform methods, allowing for an efficient implementation of the registration algorithm. Experimental analysis of the technique using real world datasets shows the usefulness as well as the limits of the approach.

A new multi-sensor registration technique for three-dimensional scene modeling with application to unmanned vehicle mobility enhancement

The focus of this paper is on the reconstruction of 3D representations of real world scenes and objects using multiple sensors with, as one of its main applications, the enhancement of the autonomy and mobility of unmanned vehicles. The sensors considered are primarily range acquisition devices (such as laser scanners and stereo systems) that allow the recovery of 3D geometry. One of the most important technical challenges that we are addressing is the registration task in both its multi-modal and single modality aspects. Our work is based on a unified approach that formulates the correspondence problem as an optimization task. In this context we developed a criterion that can be used for 3D free-form shape registration. The new criterion is derived from simple Boolean matching principles by approximation and relaxation. Technically, one of the main advantages of the proposed approach is convexity in the neighborhood of the alignment parameters and continuous differentiability, allowing for the use of standard gradient-based optimization techniques. The proposed algorithm allows also for a significant automation of the scene modeling task by reducing the intervention of human operators in the tedious image registration task. Furthermore, we show that the criterion can be computed in linear time complexity which permits the fast implementation critical in many applications of autonomous mobile platforms.