The Facial Expression Recognition Method Based on Image Fusion and CNN

Abstract

Abstract Facial expression recognition (FER) is an important task in the field of human-computer interaction. However, the traditional facial expression recognition task needs to be based on the hand-crafted features, and the feature extraction method is single; the facial expression recognition task based on deep learning cannot extract local texture features of the image and loss more information. Therefore, a facial expression recognition method based on image fusion and convolution neural network (FERFC) is proposed in this paper. Which fused the facial expression images after extracted by the local binary pattern (LBP) with the original images. It can effectively improved the utilization of images. Firstly, some image pre-processing approaches are used in this paper, such as data augmentation, face detection and data normalization. Secondly, the images of local texture features extracted by the LBP and the original images are fused in this step. Finally, the task of facial expression features learning and classification is completed by convolution neural network (CNN). The results show that the method proposed in this paper can accomplish the facial expression recognition task accurately. The recognition rate of reference database ‘Jaffe’, ‘CK+’ and ‘FER2013’ is 91.9%, 95.6% and 75.9%. The results show that the FERFC has significant advantages than traditional facial expression recognition. At the same time, the number of training samples is small, the FERFC still has obvious advantages and a higher robustness.