Mongi A. Abidi

Recent advances in visual and infrared face recognition: a review

Face recognition is a rapidly growing research area due to increasing demands for security in commercial and law enforcement applications. This paper provides an up-to-date review of research efforts in face recognition techniques based on two-dimensional (2D) images in the visual and infrared (IR) spectra. Face recognition systems based on visual images have reached a significant level of maturity with some practical success. However, the performance of visual face recognition may degrade under poor illumination conditions or for subjects of various skin colors. IR imagery represents a viable alternative to visible imaging in the search for a robust and practical identification system. While visual face recognition systems perform relatively reliably under controlled illumination conditions, thermal IR face recognition systems are advantageous when there is no control over illumination or for detecting disguised faces. Face recognition using 3D images is another active area of face recognition, which provides robust face recognition with changes in pose. Recent research has also demonstrated that the fusion of different imaging modalities and spectral components can improve the overall performance of face recognition.

Gray-level grouping (GLG): an automatic method for optimized image contrast Enhancement-part I: the basic method

Contrast enhancement has an important role in image processing applications. Conventional contrast enhancement techniques either often fail to produce satisfactory results for a broad variety of low-contrast images, or cannot be automatically applied to different images, because their parameters must be specified manually to produce a satisfactory result for a given image. This paper describes a new automatic method for contrast enhancement. The basic procedure is to first group the histogram components of a low-contrast image into a proper number of bins according to a selected criterion, then redistribute these bins uniformly over the grayscale, and finally ungroup the previously grouped gray-levels. Accordingly, this new technique is named gray-level grouping (GLG). GLG not only produces results superior to conventional contrast enhancement techniques, but is also fully automatic in most circumstances, and is applicable to a broad variety of images. An extension of GLG, selective GLG (SGLG), and its variations will be discussed in Part II of this paper. SGLG selectively groups and ungroups histogram components to achieve specific application purposes, such as eliminating background noise, enhancing a specific segment of the histogram, and so on. The extension of GLG to color images will also be discussed in Part II.

Detection and classification of edges in color images

Up to now, most of the color edge detection methods are monochromatic-based techniques, which produce, in general, better than when traditional gray-value techniques are applied. In this overview, we focus mainly on vector-valued techniques because it is easy to understand how to apply common edge detection schemes to every color component. Opposed to this, vector-valued techniques are new and different. The second part of the article addresses the topic of edge classification. While edges are often classified into step edges and ramp edges, we address the topic of physical edge classification based on their origin into shadow edges, reflectance edges, orientation edges, occlusion edges, and specular edges. In the rest of this article we discuss various vector-valued techniques for detecting discontinuities in color images. Then operators are presented based on vector order statistics, followed by presentation by examples of a couple of results of color edge detection. We then discuss different approaches to a physical classification of edges by their origin.

A new efficient and direct solution for pose estimation using quadrangular targets: algorithm and evaluation

Pose estimation is an important operation for many vision tasks. In this paper, the authors propose an algorithm for pose estimation based on the volume measurement of tetrahedra composed of feature-point triplets extracted from an arbitrary quadrangular target and the lens center of the vision system. The inputs to this algorithm are the six distances joining all feature pairs and the image coordinates of the quadrangular target. The outputs of this algorithm are the effective focal length of the vision system, the interior orientation parameters of the target, the exterior orientation parameters of the camera with respect to an arbitrary coordinate system if the target coordinates are known in this frame, and the final pose of the camera. The authors have also developed a shape restoration technique which is applied prior to pose recovery in order to reduce the effects of inaccuracies caused by image projection. An evaluation of the method has shown that this pose estimation technique is accurate and robust. Because it is based on a unique and closed form solution, its speed makes it a potential candidate for solving a variety of landmark-based tracking problems. >

Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition

AbstractThis paper describes a new software-based registration and fusion of visible and thermal infrared (IR) image data for face recognition in challenging operating environments that involve illumination variations. The combined use of visible and thermal IR imaging sensors offers a viable means for improving the performance of face recognition techniques based on a single imaging modality. Despite successes in indoor access control applications, imaging in the visible spectrum demonstrates difficulties in recognizing the faces in varying illumination conditions. Thermal IR sensors measure energy radiations from the object, which is less sensitive to illumination changes, and are even operable in darkness. However, thermal images do not provide high-resolution data. Data fusion of visible and thermal images can produce face images robust to illumination variations. However, thermal face images with eyeglasses may fail to provide useful information around the eyes since glass blocks a large portion of thermal energy. In this paper, eyeglass regions are detected using an ellipse fitting method, and replaced with eye template patterns to preserve the details useful for face recognition in the fused image. Software registration of images replaces a special-purpose imaging sensor assembly and produces co-registered image pairs at a reasonable cost for large-scale deployment. Face recognition techniques using visible, thermal IR, and data-fused visible-thermal images are compared using a commercial face recognition software (FaceIt®) and two visible-thermal face image databases (the NIST/Equinox and the UTK-IRIS databases). The proposed multiscale data-fusion technique improved the recognition accuracy under a wide range of illumination changes. Experimental results showed that the eyeglass replacement increased the number of correct first match subjects by 85% (NIST/Equinox) and 67% (UTK-IRIS).

Normal vector voting: crease detection and curvature estimation on large, noisy meshes

This paper describes a robust method for crease detection and curvature estimation on large, noisy triangle meshes. We assume that these meshes are approximations of piecewise-smooth surfaces derived from range or medical imaging systems and thus may exhibit measurement or even registration noise. The proposed algorithm, which we call normal vector voting, uses an ensemble of triangles in the geodesic neighborhood of a vertex-instead of its simple umbrella neighborhood-to estimate the orientation and curvature of the original surface at that point. With the orientation information, we designate a vertex as either lying on a smooth surface, following a crease discontinuity, or having no preferred orientation. For vertices on a smooth surface, the curvature estimation yields both principal curvatures and principal directions while for vertices on a discontinuity we estimate only the curvature along the crease. The last case for no preferred orientation occurs when three or more surfaces meet to form a corner or when surface noise is too large and sampling density is insufficient to determine orientation accurately. To demonstrate the capabilities of the method, we present results for both synthetic and real data and compare these results to the G. Taubin (1995, in Proceedings of the Fifth International Conference on Computer Vision, pp. 902-907) algorithm. Additionally, we show practical results for several large mesh data sets that are the motivation for this algorithm.

Fusion of Visual and Thermal Signatures with Eyeglass Removal for Robust Face Recognition

This paper describes a fusion of visual and thermal infrared (IR) images for robust face recognition. Two types of fusion methods are discussed: data fusion and decision fusion. Data fusion produces an illumination-invariant face image by adaptively integrating registered visual and thermal face images. Decision fusion combines matching scores of individual face recognition modules. In the data fusion process, eyeglasses, which block thermal energy, are detected from thermal images and replaced with an eye template. Three fusion-based face recognition techniques are implemented and tested: Data fusion of visual and thermal images (Df), Decision fusion with highest matching score (Fh), and Decision fusion with average matching score (Fa). A commercial face recognition software FaceIt® is used as an individual recognition module. Comparison results show that fusion-based face recognition techniques outperformed individual visual and thermal face recognizers under illumination variations and facial expressions.

Surface matching by 3D point's fingerprint

This paper proposes a new efficient surface representation method for the application of surface matching. We generate a feature carrier for the surface point, which is a set of 2D contours that are the projection of geodesic circles onto the tangent plane. The carrier is named points fingerprint because its pattern is similar to human fingerprint and discriminating for each point. Each points fingerprint carries the information of the normal variation along geodesic circles. Corresponding points on surfaces from different views are found by comparing fingerprints of the points. This representation scheme includes more local geometry information than some previous works that only use one contour as the feature carrier. It is not histogram based so that it is able to carry more features to improve comparison accuracy. To speed up the matching, we use a novel candidate point selection method based on the shape irregularity of the projected local geodesic circle. The points fingerprint is successfully used to register both synthetic and real 2 1/2 data.

Image Fusion and Enhancement via Empirical Mode Decomposition

In this paper, we describe a novel technique for image fusion and enhancement, using Empirical Mode Decomposition (EMD). EMD is a non-parametric data-driven analysis tool that decomposes non-linear non-stationary signals into Intrinsic Mode Functions (IMFs). In this method, we decompose images, rather than signals, from different imaging modalities into their IMFs. Fusion is performed at the decomposition level and the fused IMFs are reconstructed to realize the fused image. We have devised weighting schemes which emphasize features from both modalities by decreasing the mutual information between IMFs, thereby increasing the information and visual content of the fused image. We demonstrate how the proposed method improves the interpretive information of the input images, by comparing it with widely used fusion schemes. Apart from comparing our method with some advanced techniques, we have also evaluated our method against pixelby-pixel averaging, a comparison, which incidentally, is not common in the literature.

Shape analysis algorithm based on information theory

In this paper, we describe an algorithm to measure the shape complexity for discrete approximations of planar curves in 2D images and manifold surfaces for 3D triangle meshes. We base our algorithm on shape curvature, and thus we compute shape information as the entropy of curvature. We present definitions to estimate curvature for both discrete curves and surfaces and then formulate our theory of shape information from these definitions. We demonstrate our algorithm with experimental results.