Y. V. Venkatesh

Facial expression recognition using radial encoding of local Gabor features and classifier synthesis

Primarily motivated by some characteristics of the human visual cortex (HVC), we propose a new facial expression recognition scheme, involving a statistical synthesis of hierarchical classifiers. In this scheme, the input images of the database are first subjected to local, multi-scale Gabor-filter operations, and then the resulting Gabor decompositions are encoded using radial grids, imitating the topographical map-structure of the HVC. The codes are fed to local classifiers to produce global features, representing facial expressions. Experimental results show that such a hybrid combination of the HVC structure with a hierarchical classifier significantly improves expression recognition accuracy when applied to wide-ranging databases in comparison with the results in the literature. Furthermore, the proposed system is not only robust to corrupted data and missing information, but can also be generalized to cross-database expression recognition.

Gini Index as Sparsity Measure for Signal Reconstruction from Compressive Samples

Sparsity is a fundamental concept in compressive sampling of signals/images, which is commonly measured using the l0 norm, even though, in practice, the l1 or the lp ( 0 <; p <; 1) (pseudo-) norm is preferred. In this paper, we explore the use of the Gini index (GI), of a discrete signal, as a more effective measure of its sparsity for a significantly improved performance in its reconstruction from compressive samples. We also successfully incorporate the GI into a stochastic optimization algorithm for signal reconstruction from compressive samples and illustrate our approach with both synthetic and real signals/images.

Human Facial Expression Recognition using a 3D Morphable Model

We propose a novel approach to the detection and classification of human facial expressions using a morphable 3D model. We acquire the various expressions of an individual using a face scanner that produces textured 3D meshes using stereoscopic reconstruction. A morphable expression model (MEM), that incorporates emotion-dependent face variations in terms of morphing parameters, is then computed by establishing correspondence among the emotive faces. These morphing parameters are used for emotion recognition and classification. We demonstrate that the different facial expressions correspond to distinct clusters in the expression space.

Understanding gestures with systematic variations in movement dynamics

Sign language communication includes not only lexical sign gestures but also grammatical processes which represent inflections through systematic variations in sign appearance. We present a new approach to analyse these inflections by modelling the systematic variations as parallel channels of information with independent feature sets. A Bayesian network framework is used to combine the channel outputs and infer both the basic lexical meaning and inflection categories. Experiments using a simulated vocabulary of six basic signs and five different inflections (a total of 20 distinct gestures) obtained from multiple test subjects yielded 85.0% recognition accuracy. We also propose an adaptation scheme to extend a trained system to recognize gestures from a new person by using only a small set of data from the new person. This scheme yielded 88.5% recognition accuracy for the new person while the unadapted system yielded only 52.6% accuracy.

On the simultaneous recognition of identity and expression from BU-3DFE datasets

We propose a new linear model, based on resampling 3D face meshes to convert them to 3D matrices, to recognize identity and expression simultaneously. This contrasts with bilinear models currently used with 3D meshes. The matrices are amenable to algebraic operations for facial data analysis and synthesis. Facial emotion is represented as a linear combination of its identity and expression using principal components extracted from training data as neutral-to-emotion deformations. The linear model is applicable to other mesh data with pose variations after correction using recently available techniques. The proposed approach avoids the problem of correspondence between pairs of persons neutral and emotion meshes for estimating facial deformations used as features. Identity and expression recognition accuracies, obtained by representing resampled matrices as linear combinations of composite depth-color (gray) PCs, are better than the results in the literature on both simultaneous identity-expression using bilinear models and expression-only recognition using deformable models, facial action codes, distances between pairs of annotated facial points as features and others. The proposed framework can also be used to generate synthetic matrices displaying a wide array of natural and mixed emotions for any chosen identity. A byproduct is the result that second-order deformations as features do not seem to perform as effectively as first-order deformations for identity and expression recognition.

Resampling Approach to Facial Expression Recognition Using 3D Meshes

We propose a novel strategy, based on resampling of 3D meshes, to recognize facial expressions. This entails conversion of the existing irregular 3D mesh structure in the database to a uniformly sampled 3D matrix structure. An important consequence of this operation is that the classical correspondence problem can be dispensed with. In the present paper, in order to demonstrate the feasibility of the proposed strategy, we employ only spectral flow matrices as features to recognize facial expressions. Experimental results are presented, along with suggestions for possible refinements to the strategy to improve classification accuracy.

Medical image reconstruction from sparse samples using Simultaneous Perturbation Stochastic Optimization

Concerning medical images, which are known to have sparsity in either the spatial (or its derivative), DFT, DCT or curvelet domain, we propose a new approach for reconstruction from sparse samples, based on Simultaneous Perturbation Stochastic Optimization (SPSA) to minimize a nonconvex ℓp-norm for 0 < p < 1. The value of p chosen is such as to achieve as close an approximation to ℓ0-norm as is computationally feasible. This approach is distinct from the homotopy-theoretic and hard-thresholding techniques of recent literature for ℓ0- and ℓp-norm minimization. For lack of space, our illustrations are limited to only one each of synthetic and real images.

Deciphering layered meaning in gestures

Signs produced by gestures (such as in American Sign Language) can have a basic meaning coupled with additional meanings that are layered over the basic meaning of the sign. These layered meanings are conveyed by temporal and spatial modification of the basic form of the gesture movement. The work reported in this paper seeks to recognize temporal and spatial modifiers of hand movement and integrates them with recognition of the basic meaning of the sign. To this end, a Bayesian network framework is explored with a simulated vocabulary of 4 basic signs which give rise to 14 different combinations of basic meanings and layered meanings. Recognition accuracies of up to 88.2% were obtained.

Improving PMVS Algorithm for 3D Scene Reconstruction from Sparse Stereo Pairs

3D scene reconstruction resulting from a limited number of stereo pairs captured by a 3D camera is a nontrivial and challenging task even for current state-of-the-art multi-view stereo (MVS) reconstruction algorithms. It also has many application potentials in related techniques, such as robotics, virtual reality, video games, and 3D animation. In this paper, we analyze the performance of the PMVS (Patch-based Multi-View Stereo software) for scene reconstruction from stereo pairs of scenes captured by a simple 3D camera. We demonstrate that when applied to a limited number of stereo pairs, PMVS is inadequate for 3D scene reconstruction and discuss new strategies to overcome these limitations to improve 3D reconstruction. The proposed Canny edge feature-based PMVS algorithm is shown to produce better reconstruction results. We also discuss further enhancements using dense feature matching and disparity map-based stereo reconstruction.

Modeling layered meaning with gesture parameters

Signs produced by gestures (such as in American Sign Language) can have a basic meaning coupled with additional meanings that are like layers added to the basic meaning of the sign. These layered meanings are conveyed by Systematic temporal and spatial modification of the basic form of the gesture. The work reported in this paper seeks to recognize temporal and spatial modifiers of hand movement and integrates them with the recognition of the basic meaning of the sign. To this end, a Bayesian network framework is explored with a simulated vocabulary of 4 basic signs which give rise to 14 different combinations of basic meanings and layered meanings. In this paper we approached the problem of deciphering layered meanings by drawing analogies to the gesture parameters in Parametric HMM which represent systematic spatial modifications to gesture movement. Various Bayesian network structures were compared for recognizing the signs with layered meanings. The best performing network yielded 85.5% accuracy.