Matteo Sorci

Modelling human perception of static facial expressions

A recent internet based survey of over 35,000 samples has shown that when different human observers are asked to assign labels to static human facial expressions, different individuals categorize differently the same image. This fact results in a lack of an unique ground-truth, an assumption held by the large majority of existing models for classification. This is especially true for highly ambiguous expressions, especially in the lack of a dynamic context. In this paper we propose to address this shortcoming by the use of discrete choice models (DCM) to describe the choice a human observer is faced to when assigning labels to static facial expressions. Different models of increasing complexity are specified to capture the causal effect between features of an image and its associated expression, using several combinations of different measurements. The sets of measurements we used are largely inspired by FACS but also introduce some new ideas, specific to a static framework. These models are calibrated using maximum likelihood techniques and they are compared with each other using a likelihood ratio test, in order to test for significance in the improvement resulting from adding supplemental features. Through a cross-validation procedure we assess the validity of our approach against overfitting and we provide a comparison with an alternative model based on Neural Networks for benchmark purposes.

Improved local binary pattern based action unit detection using morphological and bilateral filters

Automatic facial action unit (AU) detection in videos is the key ingredient to all systems that utilize a subject face for either interaction or analysis purposes. With the ever growing range of possible applications, achieving a high accuracy in the simplest possible manner gains even more importance. In this paper, we present new features obtained by applying local binary patterns to images processed by morphological and bilateral filters. We use as features the variations of these patterns between the expressive and neutral faces, and show that we can gain a considerable amount of accuracy increase by simply applying these fundamental image processing tools and choosing the right way of representing the patterns. We also use these features in conjunction with additional features based on facial point geometrical relations between frames and achieve detection rates higher than methods previously proposed, using a small number of features and basic support vector machine classification.

Robust infants face tracking using active appearance models: a mixed-state condensation approach

In this paper a new extension of the CONDENSATION algorithm, with application to infants face tracking, will be introduced. In this work we address the problem of tracking a face and its features in baby video sequences. A mixed state particle filtering scheme is proposed, where the distribution of observations is derived from an active appearance model. The mixed state approach combines several dynamic models in order to account for different occlusion situations. Experiments on real video show that the proposed approach augments the tracker robustness to occlusions while maintaining the computational time competitive.

Capturing human perception of facial expressions by discrete choice modelling

Facial expression recognition by human observers is affected by subjective components. Indeed there is no ground truth. We have developed Discrete Choice Models (DCM) to capture the human perception of facial expressions. In a first step, the static case is treated, that is modelling perception of facial images. Image information is extracted using a computer vision tool called Active Appearance Model (AAM). DCMs attributes are based on the Facial Action Coding System (FACS), Expression Descriptive Units (EDUs) and outputs of AAM. Some behavioural data have been collected using an Internet survey, where respondents are asked to label facial images from the Cohn– Kanade database with expressions. Different models were estimated by likelihood maximization using the obtained data. In a second step, the proposed static discrete choice framework is extended to the dynamic case, which considers facial video instead of images. The model theory is described and another Internet survey is currently conducted in order to obtain expressions labels on videos. In this second Internet survey, videos come from the Cohn–Kanade database and the Facial Expressions and Emotions Database (FEED).