John Ghent

Non-Linear Approaches for the Classification of Facial Expressions at Varying Degrees of Intensity

The research discussed in this paper documents a comparative analysis of two nonlinear dimensionality reduction techniques for the classification of facial expressions at varying degrees of intensity. These nonlinear dimensionality reduction techniques are Kernel Principal Component Analysis (KPCA) and Locally Linear Embedding (LLE). The approaches presented in this paper employ psychological tools, computer vision techniques and machine learning algorithms. In this paper we concentrate on comparing the performance of these two techniques when combined with Support Vector Machines (SVMs) at the task of classifying facial expressions across the full expression intensity range from near-neutral to extreme facial expression. Receiver Operating Characteristic (ROC) curve analysis is employed as a means of comprehensively comparing the results of these techniques.

Investigating the dynamics of facial expression

This paper is concerned with capturing the dynamics of facial expression. The dynamics of facial expression can be described as the intensity and timing of a facial expression and its formation. To achieve this we developed a technique that can accurately classify and differentiate between subtle and similar expressions, involving the lower face. This is achieved by using Local Linear Embedding (LLE) to reduce the dimensionality of the dataset and applying Support Vector Machines (SVMs) to classify expressions. We then extended this technique to estimate the dynamics of facial expression formation in terms of intensity and timing.

Photo-realistic facial expression synthesis

This paper details a procedure for generating a function which maps an image of a neutral face to one depicting a desired expression independent of age, sex, or skin colour. Facial expression synthesis is a growing and relatively new domain within computer vision. One of the fundamental problems when trying to produce accurate expression synthesis in previous approaches is the lack of a consistent method for measuring expression. This inhibits the generation of a universal mapping function. This paper advances this domain by the introduction of the Facial Expression Shape Model (FESM) and the Facial Expression Texture Model (FETM). These are statistical models of facial expression based on anatomical analysis of expression called the Facial Action Coding System (FACS). The FESM and the FETM allow for the generation of a universal mapping function. These models provide a robust means for upholding the rules of the FACS and are flexible enough to describe subjects that are not present during the training phase. We use these models in conjunction with several Artificial Neural Networks (ANN) to generate photo-realistic images of facial expressions.

Modelling, Classification and Synthesis of Facial Expressions

The field of computer vision endeavours to develop automatic approaches to the interpretation of images from the real world. Over the past number of decades researchers within this field have created systems specifically for the automatic analysis of facial expression. The most successful of these approaches draw on the tools from behavioural science. In this chapter we examine facial expression analysis from both a behavioural science and a computer vision perspective. First we will provide details of the principal approach used in behavioural science to analyze facial expressions. This will include an overview of the evolution of facial expression analysis, where we introduce the field of facial expression analysis with Darwin’s initial findings (Darwin, 1872). We then go on to show how his findings were confirmed nearly 100 years later by Ekman et al. (Ekman et al., 1969). Following on from this we provide details of recent works investigating the appearance and dynamics of facial expressions.

Holistic facial expression classification

This paper details a procedure for classifying facial expressions. This is a growing and relatively new type of problem within computer vision. One of the fundamental problems when classifying facial expressions in previous approaches is the lack of a consistent method of measuring expression. This paper solves this problem by the computation of the Facial Expression Shape Model (FESM). This statistical model of facial expression is based on an anatomical analysis of facial expression called the Facial Action Coding System (FACS). We use the term Action Unit (AU) to describe a movement of one or more muscles of the face and all expressions can be described using the AUs described by FACS. The shape model is calculated by marking the face with 122 landmark points. We use Principal Component Analysis (PCA) to analyse how the landmark points move with respect to each other and to lower the dimensionality of the problem. Using the FESM in conjunction with Support Vector Machines (SVM) we classify facial expressions. SVMs are a powerful machine learning technique based on optimisation theory. This project is largely concerned with statistical models, machine learning techniques and psychological tools used in the classification of facial expression. This holistic approach to expression classification provides a means for a level of interaction with a computer that is a significant step forward in human-computer interaction.