Shafik Huq

Occlusion filling in stereo

Disparity maps, occlusions, and occlusion filling results on the Middlebury College test images Map, Venus, and Tsukuba (from top to bottom). Occlusions are filled using SLS linear interpolation model.Display Omitted Highlights? Comprehensive study of different occlusions and their origins. ? Discussion of ambiguities in occlusion filling. ? Achieve higher accuracy in occlusion filling applying color homogeneity. ? Probabilistic definition of homogeneity to overcome user-defined thresholds. A number of stereo matching algorithms have been developed in the last few years, which also have successfully detected occlusions in stereo images. These algorithms typically fall short of a systematic study of occlusions; they predominantly emphasize matching and regard occlusion filling as a secondary operation. Filling occlusions, however, is useful in many applications such as image-based rendering where 3D models are desired to be as complete as possible. In this paper, we study occlusions in a systematic way and propose two algorithms to fill occlusions reliably by applying statistical modeling, visibility constraints, and scene constraints. We introduce a probabilistic, model-based filling order of the occluded points to maintain consistency in filling. Furthermore, we show how an ambiguity in the interpolation of the disparity value of an occluded point can safely be avoided using color homogeneity when the points neighborhood consists of multiple scene surfaces. We perform a comparative study and show that statistically, the new algorithms deliver good quality results compared to existing algorithms.

Fusion of visual, thermal, and range as a solution to illumination and pose restrictions in face recognition

This paper summarizes the various components of face recognition research conducted at the IRIS Lab. First, fusion of visual and thermal infrared (IR) images for robust face recognition is discussed. Two techniques are implemented: data fusion and decision fusion. With the knowledge that eyeglasses block the emission of thermal energy, an algorithm is designed to detect and replace eyeglasses with an eye template in thermal images. A commercial face recognition software (FaceIt/spl reg/) is used in the evaluation of the various fusion algorithms. Comparison results show that fusion-based face recognition outperforms individual visual or thermal face recognizers under illumination variations and facial expressions. Efforts in the 3D arena are also described. Results of high resolution stereo-based 3D reconstruction of faces are shown and analyzed, in a first approach, then in a second approach, a warping technique is applied to overlay color and thermal textures on 3D mannequin head models, obtained using a laser range scanner.

Stereo-based 3D Face Modeling using Annealing in Local Energy Minimization

In the past few years, 3D face modeling has gained significant attention. Reliable modeling of the face is necessary for good performance of a 3D assisted face recognition system. In this paper, we model 3D human face from binocular stereo images. The stereo matching problem has been formulated as an energy minimization problem that progressively propagates depth from reliable regions to unreliable regions by an annealing scheme. The concept of smooth 2D grid is used to enable regularizing the final depth solution. In the energy equation, area based matching is used for the data term. The smoothness term relies on annealing. The 2D grid facilitates the estimation of the smoothness parameter, from a correlation profile obtained from neighborhood of the grid node. It is evident from 3D modeling results that the proposed algorithm performs well on images of human faces.

Stereo matching with energy-minimizing snake grid for 3D face modeling

An energy minimizing snake algorithm that runs over a grid is designed and used to reconstruct high resolution 3D human faces from pairs of stereo images. The accuracy of reconstructed 3D data from stereo depends highly on how well stereo correspondences are established during the feature matching step. Establishing stereo correspondences on human faces is often ill posed and hard to achieve because of uniform texture, slow changes in depth, occlusion, and lack of gradient. We designed an energy minimizing algorithm that accurately finds correspondences on face images despite the aforementioned characteristics. The algorithm helps establish stereo correspondences unambiguously by applying a coarse-to-fine energy minimizing snake in grid format and yields a high resolution reconstruction at nearly every point of the image. Initially, the grid is stabilized using matches at a few selected high confidence edge points. The grid then gradually and consistently spreads over the low gradient regions of the image to reveal the accurate depths of object points. The grid applies its internal energy to approximate mismatches in occluded and noisy regions and to maintain smoothness of the reconstructed surfaces. The grid works in such a way that with every increment in reconstruction resolution, less time is required to establish correspondences. The snake used the curvature of the grid and gradient of image regions to automatically select its energy parameters and approximate the unmatched points using matched points from previous iterations, which also accelerates the overall matching process. The algorithm has been applied for the reconstruction of 3D human faces, and experimental results demonstrate the effectiveness and accuracy of the reconstruction.

A Survey on 3D Modeling of Human Faces for Face Recognition

In its quest for more reliability and higher recognition rates the face recognition community has been focusing more and more on 3D based recognition. Depth information adds another dimension to facial features and provides ways to minimize the effects of pose and illumination variations for achieving greater recognition accuracy. This chapter reviews, therefore, the major techniques for 3D face modeling, the first step in any 3D assisted face recognition system. The reviewed techniques are laser range scans, 3D from structured light projection, stereo vision, morphing, shape from motion, shape from space carving, and shape from shading. Concepts, accuracy, feasibility, and limitations of these techniques and their effectiveness for 3D face recognition are discussed.

Efficient BP stereo with automatic paramemeter estimation

In this paper, we propose a series of techniques to enhance the computational performance of existing Belief Propagation (BP) based stereo matching that relies on automatic estimation of the Markov random field (MRF) parameters. First, we show how convergence in matching can be achieved faster than with the existing message comparison technique by skipping comparisons in early inferences. Second, assuming that a stereo pair is captured with identical cameras, we apply a hypothesis called noise equivalence to pre-estimate the likelihood parameters and thus, avoid costly nested inferences to reduce the computational time. The likelihood parameters and intensity information are used for accelerated message propagation in image regions lacking gradients. Third, the prior model parameters are estimated with a combination of maximum likelihood (ML) estimation and disparity gradient constraint to further reduce the computational time. Supporting experiments for the proposed algorithms show encouraging results on ground truth test images.

3D nanovision for the inspection of micro-electro-mechanical systems

Micro-electro-mechanical systems (MEMS) are found in area applications such as the automotive industry, the aviation industry, the semiconductor industry, the medical field, and various other fields where miniaturization is taking over. The accurate measurement of features on the surface of MEMS is an important tool for the assessment and monitoring of product quality. Presented here are the algorithms and results of 3D model reconstructions of MEMS devices using a variety of microscopic sensors. These sensors include an atomic force microscope, a scanning electron microscope, and a laser scanning confocal microscope. MEMS devices with micron-size features were first scanned with these microscopes. 3D models were then built and visualized using methods specific to each microscope. This allows for the models use in applications such as inspection, study of wear and tear, behavior, and reaction of such systems to pressure, heat, and friction.