Katsuhiko Mori

Subject independent facial expression recognition with robust face detection using a convolutional neural network

Reliable detection of ordinary facial expressions (e.g. smile) despite the variability among individuals as well as face appearance is an important step toward the realization of perceptual user interface with autonomous perception of persons. We describe a rule-based algorithm for robust facial expression recognition combined with robust face detection using a convolutional neural network. In this study, we address the problem of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expression. The result shows reliable detection of smiles with recognition rate of 97.6% for 5600 still images of more than 10 subjects. The proposed algorithm demonstrated the ability to discriminate smiling from talking based on the saliency score obtained from voting visual cues. To the best of our knowledge, it is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance.

Convolutional spiking neural network model for robust face detection

We propose a convolutional spiking neural network (CSNN) model with population coding for robust face detection. The basic structure of the network includes hierarchically alternating layers for feature detection and feature pooling. The proposed model implements hierarchical template matching by temporal integration of structured pulse packet. The packet signal represents some intermediate or complex visual feature (e.g., a pair of line segments, corners, eye, nose, etc.) that constitutes a face model. The output pulse of a feature pooling neuron represents some local feature (e.g., line segments). Introducing a population coding scheme in the CSNN architecture, we show how the biologically inspired model attains invariance to changes in size and position of face and ensures the efficiency of face detection.

Convolutional Spiking Neural Network for Robust Object Detection with Population Code Using Structured Pulse Packets

We propose a convolutional spiking neural network (CSNN) model with population coding for robust object (e.g., face) detection. Basic structure of the network involves hierarchically alternating layers for feature detection and feature pooling. The proposed model implements hierarchical template matching by temporal integration of structured pulse packet. The packet signal represents some intermediate or complex visual feature (e.g., a pair of line segments, corners, eye, nose, etc.) that constitutes a face model. The output pulse of a feature pooling neuron represents some local feature (e.g., end-stop, blob, eye, etc.). Introducing a population coding scheme in the CSNN architecture, we show how the biologically inspired model attains invariance to changes in size and position of face and ensures the efficiency of face detection.

Facial expression recognition combined with robust face detection in a convolutional neural network

Reliable detection of ordinary facial expressions (e.g., smile) despite the variability among individuals as well as face appearance is an important step toward the realization of perceptual user interface and the next generation imaging system with autonomous perception of persons. We describe a robust facial expression recognition system using the result of face detection by a convolutional neural network and rule-based processing. In this study, we address the problem of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expression. The result shows reliable detection of smiles with recognition rate of 97.6% for 5600 still images of more than 10 subjects. The proposed algorithm demonstrated the ability to discriminate smiling from talking based on the saliency score in the proposed algorithm. To the best of our knowledge, it is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance.

Face Recognition Using SVM Combined with CNN for Face Detection

We propose a model for face recognition using a support vector machine being fed with a feature vector generated from outputs in several modules in bottom as well as intermediate layers of convolutional neural network (CNN) trained for face detection. The feature vector is composed of a set of local output distributions from feature detecting modules in the face detecting CNN. The set of local areas are automatically selected around facial components (e.g., eyes, moth, nose, etc.) detected by the CNN. Local areas for intermediate level features are defined so that information on spatial arrangement of facial components is implicitly included as output distribution from facial component detecting modules. Results demonstrate highly efficient and robust performance both in face recognition and in detection as well.