Yusuke Mitarai

Convolutional Spiking Neural Network for Robust Object Detection with Population Code Using Structured Pulse Packets

We propose a convolutional spiking neural network (CSNN) model with population coding for robust object (e.g., face) detection. Basic structure of the network involves hierarchically alternating layers for feature detection and feature pooling. The proposed model implements hierarchical template matching by temporal integration of structured pulse packet. The packet signal represents some intermediate or complex visual feature (e.g., a pair of line segments, corners, eye, nose, etc.) that constitutes a face model. The output pulse of a feature pooling neuron represents some local feature (e.g., end-stop, blob, eye, etc.). Introducing a population coding scheme in the CSNN architecture, we show how the biologically inspired model attains invariance to changes in size and position of face and ensures the efficiency of face detection.

Visual code-sentences: a new video representation based on image descriptor sequences

We present a new descriptor-sequence model for action recognition that enhances discriminative power in the spatio-temporal context, while maintaining robustness against background clutter as well as variability in inter-/intra-person behavior. We extend the framework of Dense Trajectories based activity recognition (Wang et al., 2011) and introduce a pool of dynamic Bayesian networks (e.g., multiple HMMs) with histogram descriptors as codebooks of composite action categories represented at respective key points. The entire codebooks bound with spatio-temporal interest points constitute intermediate feature representation as basis for generic action categories. This representation scheme is intended to serve as visual code-sentences which subsume a rich vocabulary of basis action categories. Through extensive experiments using KTH, UCF Sports, and Hollywood2 datasets, we demonstrate some improvements over the state-of-the-art methods.