Masakazu Matsugu

Subject independent facial expression recognition with robust face detection using a convolutional neural network

Reliable detection of ordinary facial expressions (e.g. smile) despite the variability among individuals as well as face appearance is an important step toward the realization of perceptual user interface with autonomous perception of persons. We describe a rule-based algorithm for robust facial expression recognition combined with robust face detection using a convolutional neural network. In this study, we address the problem of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expression. The result shows reliable detection of smiles with recognition rate of 97.6% for 5600 still images of more than 10 subjects. The proposed algorithm demonstrated the ability to discriminate smiling from talking based on the saliency score obtained from voting visual cues. To the best of our knowledge, it is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance.

Convolutional spiking neural network model for robust face detection

We propose a convolutional spiking neural network (CSNN) model with population coding for robust face detection. The basic structure of the network includes hierarchically alternating layers for feature detection and feature pooling. The proposed model implements hierarchical template matching by temporal integration of structured pulse packet. The packet signal represents some intermediate or complex visual feature (e.g., a pair of line segments, corners, eye, nose, etc.) that constitutes a face model. The output pulse of a feature pooling neuron represents some local feature (e.g., line segments). Introducing a population coding scheme in the CSNN architecture, we show how the biologically inspired model attains invariance to changes in size and position of face and ensures the efficiency of face detection.

Entrainment, instability, quasi-periodicity, and chaos in a compound neural oscillator.

We studied the dynamical behavior of a class of compound central pattern generator (CPG) models consisting of a simple neural network oscillator driven by both constant and periodic inputs of varying amplitudes, frequencies, and phases. We focused on a specific oscillator composed of two mutually inhibiting types of neuron (inspiratory and expiratory neurons) that may be considered as a minimal model of the mammalian respiratory rhythm generator. The simulation results demonstrated how a simple CPG model— with a minimum number of neurons and mild nonlinearities— may reproduce a host of complex dynamical behaviors under various periodic inputs. In particular, the network oscillated spontaneously only when both neurons received adequate and proportionate constant excitations. In the presence of a periodic source, the spontaneous rhythm was overriden by an entrained oscillation of varying forms depending on the nature of the source. Stable entrained oscillations were inducible by two types of inputs: (1) anti-phase periodic inputs with alternating agonist-antagonist drives to both neurons and (2) a single periodic drive to only one of the neurons. In-phase inputs, which exert periodic drives of similar magnitude and phase relationships to both neurons, resulted in varying disruptions of the entrained oscillations including magnitude attenuation, harmonic and phase distortions, and quasi-periodic interference. In the absence of significant phasic feedback, chaotic motion developed only when the CPG was driven by multiple periodic inputs. Apneic episodes with repetitive alternation of active (intrinsic oscillation) and inactive (cessation of oscillation) states developed when the network was driven by a moderate periodic input of low frequency. %and amplitudes of intermediate strength, Similar results were demonstrated in other, more complex oscillator models (that is, half-center oscillator and three-phase respiratory network model). These theoretical results may have important implications in elucidating the mechanisms of rhythmogenesis in the mature and developing respiratory CPG as well as other compound CPGs in mammalian and invertebrate nervous systems.

A VLSI convolutional neural network for image recognition using merged/mixed analog-digital architecture

Hierarchical convolutional neural networks represent a well-known robust image-recognition model. In order to apply this model to robot vision or various intelligent vision systems, its VLSI implementation with high performance and low power consumption is required. This paper proposes a VLSI convolutional network architecture using a hybrid approach composed of pulse-width modulation (PWM) and digital circuits. We call this approach merged/mixed analog-digital architecture. The VLSI chip includes PWM neuron circuits, PWM/digital converters, digital adder-subtracters, and digital memory. We have designed and fabricated a VLSI chip by using a 0.35 μm CMOS process. The VLSI chip can perform 6-bit precision convolution calculations for an image of 100 × 100 pixels with a receptive field area of up to 20 × 20 pixels within 5 ms, which means a performance of 2 GOPS. Power consumption of PWM neuron circuits was measured to be 20 mW. We have verified successful operations using a fabricated VLSI chip.

Convolutional Spiking Neural Network for Robust Object Detection with Population Code Using Structured Pulse Packets

We propose a convolutional spiking neural network (CSNN) model with population coding for robust object (e.g., face) detection. Basic structure of the network involves hierarchically alternating layers for feature detection and feature pooling. The proposed model implements hierarchical template matching by temporal integration of structured pulse packet. The packet signal represents some intermediate or complex visual feature (e.g., a pair of line segments, corners, eye, nose, etc.) that constitutes a face model. The output pulse of a feature pooling neuron represents some local feature (e.g., end-stop, blob, eye, etc.). Introducing a population coding scheme in the CSNN architecture, we show how the biologically inspired model attains invariance to changes in size and position of face and ensures the efficiency of face detection.

An image filtering processor for face/object recognition using merged/mixed analog-digital architecture

This paper proposes an image-filtering processor LSI based on a hybrid approach using pulse-width modulation (PWM) and digital circuits. The LSI has been designed for implementing convolutional neural networks with a very large convolution-kernel size. The LSI designed using a 0.35 /spl mu/m CMOS performs 6-bit precision convolutions for an image of 80/spl times/80 pixels with a kernel size of up to 51/spl times/51 pixels within 8.2 ms. All operations for the fabricated LSI have been successfully verified. The power consumption estimated from SPICE simulation is 280 mW.

Hierarchical pulse-coupled neural network model with temporal coding and emergent feature binding mechanism

We propose a convolutional-type, spiking neural network model with explicit timing structure of pulse trains (pulse packet) used for encoding/decoding local visual features. The pulse phase modulating (PPM) synapses function as feature encoders that reflect an internal representation of higher class feature in terms of spike timing. PPM synapses together with a local bus that transmits the structured pulse packet signals form convergent connections to a feature detecting neuron. Distributed, local timing neurons are introduced for an event-driven, stable, and accurate control of the pulse packet signals propagated in the hierarchical, synchronously spiking network.

Unsupervised Feature Selection for Multi-class Object Detection Using Convolutional Neural Networks

Convolutional Neural Networks (CNN) have proven to be useful tools for object detection and object recognition. They act like feature extractor and classifier at the same time. In this study we present an unsupervised feature selection procedure for constructing a training set for the CNN and analyze in detail the learnt receptive fields. We then introduce, for the first time, a figural alphabet to be used for low-level feature detection with CNN. This alphabet turned out to be useful in detecting a vocabulary set of intermediate level features and considerably reduces the complexity of the CNN. Moreover we propose an optimal high-level feature selection procedure and apply this to the challenging problem of car detection. We demonstrate promising results for multi-class object detection using obtained figural alphabet to detect considerably different categories of objects (e.g., faces and cars).

Facial expression recognition combined with robust face detection in a convolutional neural network

Reliable detection of ordinary facial expressions (e.g., smile) despite the variability among individuals as well as face appearance is an important step toward the realization of perceptual user interface and the next generation imaging system with autonomous perception of persons. We describe a robust facial expression recognition system using the result of face detection by a convolutional neural network and rule-based processing. In this study, we address the problem of subject independence as well as translation, rotation, and scale invariance in the recognition of facial expression. The result shows reliable detection of smiles with recognition rate of 97.6% for 5600 still images of more than 10 subjects. The proposed algorithm demonstrated the ability to discriminate smiling from talking based on the saliency score in the proposed algorithm. To the best of our knowledge, it is the first facial expression recognition model with the property of subject independence combined with robustness to variability in facial appearance.

Face Tracking Active Vision System with Saccadic and Smooth Pursuit

- People detection and tracking is very important and fundamental functionality in surveillance and human-robot interaction. This paper presents a robotic vision module with rapid eye movements for fast face tracking. Face tracking is initiated by robust face detection followed with color histogram-based matching and linear prediction of face position. Rapid and smooth eye movements (i.e. panning and tilting) are embodied by linear and hybrid (i.e. velocity-based FF + position based FB) control scheme as well as powerful yet widely used DC motor. Experimental results demonstrated ultra-fast and extremely smooth pursuit of moving face, at 6 deg/40 ms (approximately 10 m/sec at the distance of 3 m), with abrupt change in motion direction (e.g., jumping), with sustained robustness to the change of pose (e.g., from frontal face to nearly back of head).