Facial Expression Recognition By Using a Disentangled Identity-Invariant Expression Representation

Abstract

Facial Expression Recognition (FER) is a challenging task because many factors of variation such as pose, illumination, and identity-specific attributes are entangled with the expression information in an expressive face image. Recent works show that the performance of a FER algorithm can be improved by disentangling the expression information from identity features. In this paper, we present Transfer-based Expression Recognition Generative Adversarial Network (TER-G AN) that combines the effectiveness of a novel feature disentanglement technique with the concept of identity-invariant expression representation learning for FER. More specifically, TER-GAN learns a disentangled expression representation by extracting expression features from one image and transferring the expression information to the identity of another image. To improve the feature disentanglement process and to learn an identity-invariant expression representation, we introduce a novel expression consistency loss and an identity consistency loss that exploit expression and identity information from both real and synthetic images. We evaluated the performance of our proposed facial expression recognition technique by employing four public facial expression databases, CK+, Oulu-CASIA, MMI, BU-3DFE. The experimental results show the effectiveness of the proposed technique.