Facial expression recognition based on facial part attention mechanism

Abstract

Abstract. In the context of facial expression recognition (FER), a lot of methods focus on feature extraction of the whole facial image, but sometimes it is equally important to concentrate on local area features that embody stronger emotions. This paper proposes an FER method based on the facial part attention mechanism (FPA). We adopt the attention mechanism to extract emotional rich local area features, which are complementary to the whole facial features for better FER. First, this paper proposes a cluster-based facial landmarks selection method to select facial landmarks that have commonality and best reflect local area emotions. Then, we design an FPA convolutional neural network, which consists of two parts. The first part is the object network, which is used to extract the whole facial features; the second part is the part attention network, and the attention component is used to extract the local part features. Finally, the two parts are merged to train the final classifier. The method was tested on the Real-World Affective Faces database and the recognition accuracy of 87.26% was obtained, which proved the effectiveness of the method.