Joseph C. Hager

Classifying facial actions

The Facial Action Coding System (FACS) [23] is an objective method for quantifying facial movement in terms of component actions. This system is widely used in behavioral investigations of emotion, cognitive processes, and social interaction. The coding is presently performed by highly trained human experts. This paper explores and compares techniques for automatically recognizing facial actions in sequences of images. These techniques include analysis of facial motion through estimation of optical flow; holistic spatial analysis, such as principal component analysis, independent component analysis, local feature analysis, and linear discriminant analysis; and methods based on the outputs of local filters, such as Gabor wavelet representations and local principal components. Performance of these systems is compared to naive and expert human subjects. Best performances were obtained using the Gabor wavelet representation and the independent component representation, both of which achieved 96 percent accuracy for classifying 12 facial actions of the upper and lower face. The results provide converging evidence for the importance of using local filters, high spatial frequencies, and statistical independence for classifying facial actions.

Measuring facial expressions by computer image analysis

Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System (Ekman & Friesen, 1978) is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. Three approaches were compared: holistic spatial analysis, explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system that classified six upper facial actions with 91% accuracy. The hybrid system outperformed human nonexperts on this task and performed as well as highly trained experts. An automated system would make facial expression measurement more widely accessible as a research tool in behavioral science and investigations of the neural substrates of emotion.

Long-distance of transmission of facial affect signals☆

Abstract This study examined the distance at which certain facial expression can transmit affect messages. A man and a woman assumed—facial expressions that were selected carefully to represent six affects. These expressions were shown in still photographs and in live portrayals to 49 observers who composed four groups which were 30, 35, 40, and 45 meters away from the stimuli. Photographs and live portrayals produced comparable results. Every observer was able to label the expressions accurately although accuracy declined as distance increased. Extrapolation from the data suggested that some messages may be sent far beyond the distances used in this study. These results raise important issues about the transmission of facial signals over distance and suggest that the face is a long-distance transmitter of affect signals.