Amar A. El-Sallam

3-D Face Recognition Using Curvelet Local Features

In this letter, we present a robust single modality feature-based algorithm for 3-D face recognition. The proposed algorithm exploits Curvelet transform not only to detect salient points on the face but also to build multi-scale local surface descriptors that can capture highly distinctive rotation/displacement invariant local features around the detected keypoints. This approach is shown to provide robust and accurate recognition under varying illumination conditions and facial expressions. Using the well-known and challenging FRGC v2 dataset, we report a superior performance compared to other algorithms, with a 97.83% verification rate for probes with all facial expressions.

A Curvelet-based approach for textured 3D face recognition

In this paper, we present a fully automated multimodal Curvelet-based approach for textured 3D face recognition. The proposed approach relies on a novel multimodal keypoint detector capable of repeatably identifying keypoints on textured 3D face surfaces. Unique local surface descriptors are then constructed around each detected keypoint by integrating Curvelet elements of different orientations, resulting in highly descriptive rotation invariant features. Unlike previously reported Curvelet-based face recognition algorithms which extract global features from textured faces only, our algorithm extracts both texture and 3D local features. In addition, this is achieved across a number of frequency bands to achieve robust and accurate recognition under varying illumination conditions and facial expressions. The proposed algorithm was evaluated using three well-known and challenging datasets, namely FRGC v2, BU-3DFE and Bosphorus datasets. Reported results show superior performance compared to prior art, with 99.2%, 95.1% and 91% verification rates at 0.001 FAR for FRGC v2, BU-3DFE and Bosphorus datasets, respectively. HighlightsIdentifying distinctive keypoints on textured 3D face surfaces rich with features.These keypoints are identified in the Curvelet domain across mid-frequency bands.The repeatability of these keypoints is high in both neutral and nonneutral faces.Building local surface descriptors around the keypoints in the Curvelet domain.Reported results show superior performance on three datasets, namely FRGC, BU-3DFE and Bosphorus, compared to prior art.

3D-Div: A novel local surface descriptor for feature matching and pairwise range image registration

This paper presents a novel local surface descriptor, called 3D-Div. The proposed descriptor is based on the concept of 3D vector fields divergence, extensively used in electromagnetic theory. To generate a 3D-Div descriptor of a 3D surface, a keypoint is first extracted on the 3D surface, then a local patch of a certain size is selected around that keypoint. A Local Reference Frame (LRF) is then constructed at the keypoint using all points forming the patch. A normalized 3D vector field is then computed at each point in the patch and referenced with LRF vectors. The 3D-Div descriptors are finally generated as the divergence of the reoriented 3D vector field. We tested our proposed descriptor on the low resolution Washington RGB-D (Kinect) object dataset. Performance was evaluated for the tasks of feature matching and pairwise range image registration. Experimental results showed that the proposed 3D-Div is 88% more computationally efficient and 47% more accurate than commonly used Spin Image (SI) descriptors.

A Novel Local Surface Description for Automatic 3D Object Recognition in Low Resolution Cluttered Scenes

Local surface description is a critical stage for feature matching and recognition tasks. This paper presents a rotation invariant local surface descriptor, called 3D-Div. The proposed descriptor is based on the concept of 3D vector fields divergence, extensively used in electromagnetic theory. To generate a 3D-Div descriptor of a 3D surface, a local surface patch is parameterized around a randomly selected 3D point at a fixed scale. A unique Local Reference Frame (LRF) is then constructed at that 3D point using all the neighboring points forming the patch. A normalized 3D vector field is then computed at each point in the patch and referenced with LRF vectors. The 3D-Div descriptor is finally generated as the divergence of the reoriented 3D vector field. We tested our proposed descriptor on the challenging low resolution Washington RGB-D (Kinect) object dataset, for the task of automatic 3D object recognition. Reported experimental results show that 3D-Div based recognition achieves 93% accuracy as compared to 85% for existing state-of-the-art depth kernel descriptors [2].

Spectral-Based Blind Image Restoration Method for Thin TOMBO Imagers

With the recent advances in microelectronic fabrication technology, it has now become now possible to fabricate thin imagers, less than half a millimeter thick. Dubbed TOMBO (an acronym for Thin Observation Module by Bound Optics), a thin camera-on-a-chip integrates micro-optics and photo-sensing elements, together with advanced processing circuitry, all on a single silicon chip. Modeled after the compound-eye found in insects and many other arthropods, the TOMBO imager simultaneously captures a mosaic of low resolution images. In this paper, we describe and analyze a novel spectral-based blind algorithm that enables the restoration of a high resolution image from the captured low resolution images. The proposed blind restoration method does not require prior information about the imaging system nor the original scene. Furthermore, it alleviates the need for conventional de-shading and rearrangement processing techniques. Experimental results demonstrate that the proposed method can restore images for Signal-to-Noise Energy Ratio (SNER) lower than 3 dB.

An RGB-D based image set classification for robust face recognition from Kinect data

The paper proposes a method for robust face recognition from low quality Kinect acquired images which have a wide range of variations in head pose, illumination, facial expressions, sunglass disguise and occlusions by hand. Multiple Kinect images of a person are considered as an image set and face recognition from these images is formulated as an RGB-D image set classification problem. The Kinect acquired raw depth data is used for pose estimation and an automatic cropping of the face region. Based upon the estimated poses, the face images of a set are divided into multiple image subsets. An efficient block based covariance matrix representation is proposed to model images in an image subset on Riemannian manifold (Lie group). For classification, SVM models are separately learnt for each image subset on the Lie group of Riemannian manifold and a fusion strategy is introduced to combine results from all image subsets. The proposed technique has been evaluated on a combination of three large data sets containing over 35,000 RGB-D images under challenging conditions. The proposed RGB-D based image set classification incurs low computational cost and achieves an identification rate as high as 99.5%.

Clustering of video-patches on Grassmannian manifold for facial expression recognition from 3D videos

This paper presents a fully automatic system which exploits the dynamics of 3D videos and is capable of recognizing six basic facial expressions. Local video-patches of variable lengths are extracted from different locations of the training videos and represented as points on the Grass-mannian manifold. An efficient spectral clustering based algorithm is used to separately cluster points for each of the six expression classes. The resulting cluster centers are matched with the points of a test video and a voting based strategy is used to decide about the expression class of the test video. The proposed system is tested on the largest publicly available 3D video database, BU4DFE. The experimental results show that the system achieves a very high classification accuracy and outperforms the current state of the art algorithms for facial expression recognition from 3D videos.

Evaluation of Spatiotemporal Detectors and Descriptors for Facial Expression Recognition

Local spatiotemporal detectors and descriptors have recently become very popular for video analysis in many applications. They do not require any preprocessing steps and are invariant to spatial and temporal scales. Despite their computational simplicity, they have not been evaluated and tested for video analysis of facial data. This paper considers two space-time detectors and four descriptors and uses bag of features framework for human facial expression recognition on BU_4DFE data set. A comparison of local spatiotemporal features with other non-spatiotemporal published techniques on the same data set is also given. Unlike spatiotemporal features, these techniques involve time consuming and computationally intensive preprocessing steps like manual initialization and tracking of facial points. Our results show that despite being totally automatic and not requiring any user intervention, local spacetime features provide promising and comparable performance for facial expression recognition on BU_4DFE data set.

Robust pose invariant shape-based hand recognition

This paper presents a novel technique for hand shape and appearance based personal identification and verification. It has two major building blocks. A segmentation block presents robust and fully automatic algorithms which are able to accurately segment the hands palm and fingers irrespective of colour contrast between the fosreground and background. They achieve a consistent representation of the fingers and the palm regardless of their pose/orientation or the spaces between the fingers. In the feature extraction/matching block, the iterative closest point (ICP) algorithm is employed to align the images. Both shape and appearance based features are extracted and comparatively assessed. The modified Hausdorff distance and independent component analysis (ICA) algorithms are used for shape and appearance analysis. Identification and verification were performed using fusion strategies upon the similarity scores of the fingers and the palm. Experimental results show the proposed system exhibits an accuracy of over 98% in hand recognition and verification in a database consisting of 500 different subjects.

A low cost 3D markerless system for the reconstruction of athletic techniques

We present a low cost markerless system for the optimization of athlete performance in sports such as pole vault, jumping and javelin throw. The system uses a number of calibrated cameras to capture a video of an athlete from different viewpoints. The athletes body is then segmented from the background in each video frame. The silhouettes of the segmented body are then reprojected to reconstruct an estimate of the 3D body shape of the athlete, known as the visual hull (VH). The VH is tracked over a number of frames in real testing trials. A template combining a high resolution 3D scan and a 2D mass scan is then aligned with the VH in each frame. A set of motion analysis parameters such as the take-off data are finally estimated from the aligned template and compared with the ones obtained using a gold standard marker-based system, namely the Vicon. The proposed system was tested in real-time trials and was able to provide comparable results to the Vicon system.