Steven Cadavid

A framework for automated measurement of the intensity of non-posed Facial Action Units

This paper presents a framework to automatically measure the intensity of naturally occurring facial actions. Naturalistic expressions are non-posed spontaneous actions. The facial action coding system (FACS) is the gold standard technique for describing facial expressions, which are parsed as comprehensive, nonoverlapping action units (Aus). AUs have intensities ranging from absent to maximal on a six-point metric (i.e., 0 to 5). Despite the efforts in recognizing the presence of non-posed action units, measuring their intensity has not been studied comprehensively. In this paper, we develop a framework to measure the intensity of AU12 (lip corner puller) and AU6 (cheek raising) in videos captured from infant-mother live face-to-face communications. The AU12 and AU6 are the most challenging case of infants expressions (e.g., low facial texture in infants face). One of the problems in facial image analysis is the large dimensionality of the visual data. Our approach for solving this problem is to utilize the spectral regression technique to project high dimensionality facial images into a low dimensionality space. Represented facial images in the low dimensional space are utilized to train support vector machine classifiers to predict the intensity of action units. Analysis of 18 minutes of captured video of non-posed facial expressions of several infants and mothers shows significant agreement between a human FACS coder and our approach, which makes it an efficient approach for automated measurement of the intensity of non-posed facial action units.

3-D Ear Modeling and Recognition From Video Sequences Using Shape From Shading

We describe a novel approach for 3-D ear biometrics using video. A series of frames is extracted from a video clip and the region of interest in each frame is independently reconstructed in 3-D using shape from shading. The resulting 3-D models are then registered using the iterative closest point algorithm. We iteratively consider each model in the series as a reference model and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3-D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3-D model and is subsequently enrolled in the database. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips. The results (95.0% rank-1 recognition rate and 3.3% equal error rate) indicate that the proposed approach can produce recognition rates comparable to systems that use 3-D range data. To the best of our knowledge, we are the first to develop a 3-D ear biometric system that obtains a 3-D ear structure from a video sequence.

Histograms of Categorized Shapes for 3D ear detection

We introduce a novel shape-based feature set, termed the Histograms of Categorized Shapes (HCS), for robust Three-Dimensional (3D) object recognition. By adopting the sliding window approach and a linear Support Vector Machine (SVM) classifier, the efficacy of the HCS feature is assessed on a 3D ear detection task. Experimental results demonstrate that the approach achieves a perfect detection rate, i.e., a 100% detection rate with a 0% false positive rate, on a validation set consisting of 142 range profile images from the University of Notre Dame (UND) 3D ear biometrics database. It is to the best of our knowledge that the detection rate achieved here outperforms those reported in the literature for the given dataset. The proposed detector is also extremely efficient in both training and detection due to the simplicity of the feature extraction and speed of the classification process, suggesting that the method is suitable for practical use in 3D ear biométrie applications.

An Efficient 3-D Ear Recognition System Employing Local and Holistic Features

We present a complete three-dimensional (3-D) ear recognition system combining local and holistic features in a computationally efficient manner. The system is comprised of four primary components, namely: 1) ear image segmentation; 2) local feature extraction and matching; 3) holistic feature extraction and matching; and 4) a fusion framework combining local and holistic features at the match score level. For the segmentation component, we introduce a novel shape-based feature set, termed the Histograms of Indexed Shapes (HIS), to localize a rectangular region containing the ear. For the local feature extraction and representation component, we extend the HIS feature descriptor to an object-centered 3-D shape descriptor, the Surface Patch Histogram of Indexed Shapes (SPHIS), for local ear surface representation and matching. For the holistic matching component, we introduce a voxelization scheme for holistic ear representation from which an efficient, voxel-wise comparison of gallery-probe model pairs can be made. The match scores obtained from both the local and holistic matching components are fused to generate the final match scores. Experimental results conducted on the University of Notre Dame (UND) collection G dataset, containing range images of 415 subjects yielded a rank-one recognition rate of 98.3% and an equal error rate of 1.7%. These results demonstrate that the proposed approach outperforms state-of-the-art 3-D ear biometric systems. Additionally, the method is considerably more efficient compared to the state-of-the-art because it employs a sparse set of features rather than using the dense model.

A computationally efficient approach to 3D ear recognition employing local and holistic features

We present a complete, Three-Dimensional (3D) object recognition system combining local and holistic features in a computationally efficient manner. An evaluation of the proposed system is conducted on a 3D ear recognition task. The ear provides a challenging case study because of its high degree of inter-subject similarity. In this work, we focus primarily on the local and holistic feature extraction and matching components, as well as the fusion framework used to combine these features at the match score level. Experimental results conducted on the University of Notre Dame (UND) collection G dataset, containing range images of 415 subjects, yielded a rank-one recognition rate of 98.6% and an equal error rate of 1.6%. These results demonstrate that the proposed system outperforms state-of-the-art 3D ear biometric systems.

Exploiting visual quasi-periodicity for real-time chewing event detection using active appearance models and support vector machines

Steady increases in healthcare costs and obesity have inspired recent studies into cost-effective, assistive systems capable of monitoring dietary habits. Few researchers, though, have investigated the use of video as a means of monitoring dietary activities. Video possesses several inherent qualities, such as passive acquisition, that merits its analysis as an input modality for such an application. To this end, we propose a method to automatically detect chewing events in surveillance video of a subject. Firstly, an Active Appearance Model (AAM) is used to track a subject’s face across the video sequence. It is observed that the variations in the AAM parameters across chewing events demonstrate a distinct periodicity. We utilize this property to discriminate between chewing and non-chewing facial actions such as talking. A feature representation is constructed by applying spectral analysis to a temporal window of model parameter values. The estimated power spectra subsequently undergo non-linear dimensionality reduction. The low-dimensional embedding of the power spectra are employed to train a binary Support Vector Machine classifier to detect chewing events. To emulate the gradual onset and offset of chewing, smoothness is imposed over the class predictions of neighboring video frames in order to deter abrupt changes in the class labels. Experiments are conducted on a dataset consisting of 37 subjects performing each of five actions, namely, open- and closed-mouth chewing, clutter faces, talking, and still face. Experimental results yielded a cross-validated percentage agreement of 93.0%, indicating that the proposed system provides an efficient approach to automated chewing detection.

Human Identification based on 3D Ear Models

Two 3D ear recognition systems using structure from motion (SFM) and shape from shading (SFS) techniques, respectively, are explored. Segmentation of the ear region is performed using interpolation of ridges and ravines identified in each frame in a video sequence. For the SFM system, salient features are tracked across the video sequence and are reconstructed in 3D using a factorization method. Reconstructed points located within the valid ear region are stored as the ear model. The dataset used consists of video sequences for 48 subjects. Each test model is optimally aligned to the database models using a combination of geometric transformations which result in a minimal partial Hausdorff distance. For the SFS system, the ear structure is recovered by using reflectance and illumination properties of the scene. Shape matching is performed via iterative closest point. Based on our results, we conclude that both structure from motion and shape from shading are viable approaches for 3D ear recognition from video sequences.

Exploiting color SIFT features for 2D ear recognition

In this paper, we present a robust method for 2D ear recognition using color SIFT features. Firstly, we extend the Scale Invariant Feature Transform (SIFT) algorithm originally performed on the intensity channel [1] to the RGB color channels to maximize the robustness of the SIFT feature descriptor. Secondly, a feature matching algorithm for ear recognition is proposed by fusion of the features extracted from the different color channels. Experiments conducted on the University of Notre Dame (UND) and the West Virginia University (WVU) ear biometric datasets indicate that our method can achieve better recognition rates than the state-of-the-art methods applied on the same datasets.

Multi-modal ear and face modeling and recognition

In this paper we describe a multi-modal ear and face biometric system. The system is comprised of two components: a 3D ear recognition component and a 2D face recognition component. For the 3D ear recognition, a series of frames is extracted from a video clip and the region of interest (i.e., ear) in each frame is independently reconstructed in 3D using Shape From Shading. The resulting 3D models are then registered using the iterative closest point algorithm. We iteratively consider each model in the series as a reference model and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3D model and is subsequently enrolled in the database. For the 2D face recognition, a set of facial landmarks is extracted from frontal facial images using the Active Shape Model. Then, the response of facial images to a series of Gabor filters at the locations of facial landmarks are calculated. The Gabor features (attributes) are stored in the database as the face model for recognition. The similarity between the Gabor features of a probe facial image and the reference models are utilized to determine the best match. The match scores of the ear recognition and face recognition modalities are fused to boost the overall recognition rate of the system. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips (images). As a result, a rank-one identification rate of 100% was achieved using the weighted sum technique for fusion.

3D ear modeling and recognition from video sequences using shape from shading

We describe a novel approach for 3D ear biometrics using video. A series of frames are extracted from a video clip and the region-of-interest (ROI) in each frame is independently reconstructed in 3D using Shape from Shading (SFS). The resulting 3D models are then registered using the Iterative Closest Point (ICP) algorithm. We iteratively consider each model in the series as a reference and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3D model and is subsequently enrolled in the database. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips. The results (95.0% rank-1 recognition rate) indicate that the proposed approach can produce recognition rates comparable to systems that use 3D range data. To the best of our knowledge, we are the first to develop a 3D ear biometric system that obtains 3D ear structure from a video sequence.