Terumasa Aoki

Motion dense sampling and component clustering for action recognition

In this paper, we propose Motion Dense Sampling (MDS) for action recognition, which detects very informative interest points from video frames. MDS has three advantages compared to other existing methods. The first advantage is that MDS detects only interest points which belong to action regions of all regions of a video frame. The second one is that it can detect the constant number of points even when the size of action region in an image drastically changes. The Third one is that MDS enables to describe scale invariant features by computing sampling scale for each frame based on the size of action regions. Thus, our method detects much more informative interest points from videos unlike other methods. We also propose Category Clustering and Component Clustering, which generate the very effective codebook for action recognition. Experimental results show a significant improvement over existing methods on YouTube dataset. Our method achieves 87.5 % accuracy for video classification by using only one descriptor.

Moment Symmetry: A novel method for interest point detection to match blurred and non-blurred images

To match a pair of images, currently lots of local features can be used. Many of them show good performances for image changes such as rotation, resizing, and occlusion etc. However, matching between a blurred (especially, motion-blurred) image and a non-blurred image is still a challenging task although it is required for many image/video applications. Local feature matching is usually composed of three steps: interest point detection, feature description for each interest point, and matching these features. Unfortunately interest point detection is not robust for strong blur in most existing methods. In this paper, we present Moment Symmetry (MS) to solve this problem. MS is a new concept of symmetry against traditional symmetry. By using MS, we can extract the same interest points from a blurred image and a non-blurred image in a simple way. Experimental results show MS outperforms existing methods as a novel interest point detector.

3D object retrieval system using skewness database

This paper presents a novel 3D object retrieval system from a single query image. A 3D object retrieval system is one of the most difficult systems to archive. It will be much easier if users can input 3D model of the object and use it as a query to the retrieval system. Unfortunately, creating a 3D model from an object is not an easy task. Usually a 2D rendered image of the object (photos, sketches, etc.) that needs to be retrieved is used and the system will find matching (or nearly matching) objects based on the query image. This will take a high computational cost since every image in the database must be compared with the query image. By using the proposed method, not only that every image in the database need not to be compared with the query image, but also the horizontal and vertical angles of a 3D object can be inferred immediately. Each 3D object image inside the database will be represented by a set of cubic spline function. We also use the skewness value as a feature when constructing the database. In order to retrieve the data, we use a simple thresholding method. The experimental results show that the proposed method has a good accuracy in retrieving the information within the databases.

Automatic Image Colorization based on Feature Lines

Automatic image colorization is one of the attractive research topics in image processing. The most crucial task in this field is how to design an algorithm to define appropriate color from the reference image(s) for propagating to the target image. In other words, we need to determine whether two pixels in reference and target images have similar color. In previous methods, many approaches have been introduced mostly based on local feature matching algorithms. However, they still have some defects as well as time-consuming. In this paper, we will present a novel automatic image colorization method based on Feature Lines. Feature Lines is our new concept, which enhances the concept of Color Lines. It represents the distribution of each pixel feature vector as being elongated around the lines so that we are able to assemble the similar feature pixels into one feature line. By introducing this new technique, pixel matching between reference and target images performs precisely. The experimental achievements show our proposed method achieves smoother, evener and more natural color assignment than the previous methods.

BROAFERENCE: A Prototype of an Emotion-based TV Quality Rating System

Whether a television (TV) program is good or not is generally judged by its TV rating, which indicates the audience size. However, it is obvious that a TV rating does not necessarily guarantee the quality of the program. In this chapter, we describe the architecture and experimental results of BROAFERENCE, which is an emotion-based TV quality rating system that makes it possible to judge the quality of a TV program by measuring emotional information from the TV audience. BROAFERENCE utilizes automatic detection of facial expressions and gaze information via a camera observing the TV program’s audience in order to collect emotional cues about the impact of a certain program as well as measuring the attention of viewers.

Global Distribution Adjustment and Nonlinear Feature Transformation for Automatic Colorization

Automatic colorization is generally classified into two groups: propagation-based methods and reference-based methods. In reference-based automatic colorization methods, color image(s) are used as reference(s) to reconstruct original color of a gray target image. The most important task here is to find the best matching pairs for all pixels between reference and target images in order to transfer color information from reference to target pixels. A lot of attractive local feature-based image matching methods have already been developed for the last two decades. Unfortunately, as far as we know, there are no optimal matching methods for automatic colorization because the requirements for pixel matching in automatic colorization are wholly different from those for traditional image matching. To design an efficient matching algorithm for automatic colorization, clustering pixel with low computational cost and generating descriptive feature vector are the most important challenges to be solved. In this paper, we present a novel method to address these two problems. In particular, our work concentrates on solving the second problem (designing a descriptive feature vector); namely, we will discuss how to learn a descriptive texture feature using scaled sparse texture feature combining with a nonlinear transformation to construct an optimal feature descriptor. Our experimental results show our proposed method outperforms the state-of-the-art methods in terms of robustness for color reconstruction for automatic colorization applications.

Improving the performance of non-rigid 3D shape recovery by points classification

The goal of Non-Rigid Structure from Motion (NRSfM) is to recover 3D shapes of a deformable object from a monocular video sequence. Procrustean Normal Distribution (PND) is one of the best algorithms for NRSfM. It uses Generalized Procrustes Analysis (GPA) model to accomplish this task. But the biggest problem of this method is that just a few non-rigid points in 2D observations can largely affect the reconstruction performance. We believe that PND can achieve better reconstruction performance by eliminating the affection of these points. In this paper, we present a novel reconstruction method to solve this problem. We present two solutions to simply classify the points into non-rigid and nearly rigid points. After that, we use EM algorithm of PND to recover 3D structure again for nearly rigid points. Experimental results show that the proposed method outperforms the existing state-of-the-art algorithms.

A blur-invariant local feature for motion blurred image matching

Image matching between a blurred (caused by camera motion, out of focus, etc.) image and a non-blurred image is a critical task for many image/video applications. However, most of the existing local feature schemes fail to achieve this work. This paper presents a blur-invariant descriptor and a novel local feature scheme including the descriptor and the interest point detector based on moment symmetry – the authors’ previous work. The descriptor is based on a new concept - center peak moment-like element (CPME) which is robust to blur and boundary effect. Then by constructing CPMEs, the descriptor is also distinctive and suitable for image matching. Experimental results show our scheme outperforms state of the art methods for blurred image matching

A Convex Approach for Non-rigid Structure from Motion Via Sparse Representation.

This paper presents a convex solution for simultaneously recovering 3D non-rigid structures and camera motions from 2D image sequences based on sparse representation. Most existing methods rely on low rank assumption. However, it will lead to poor reconstruction for objects with strong local deformation. Also, when camera motion is unknown, there is no convex solution for non-rigid structure from motion (NRSfM). In order to solve this problem, we estimate non-rigid structures by sparse representation. In this paper, we estimate camera motions through a sparse spectral-norm minimization approach, and then a fast l1-norm minimization algorithm is introduced to reconstruct 3D structures. Both of them are convex, therefore, our method gives a global optimum. Our method can handle objects with strong local deformation and also doesn’t need low rank prior. Experimental results show that our method achieves state-of-the-art reconstruction performance on CMU benchmark dataset.

Skeleton Extraction in Cluttered Image Based on Delaunay Triangulation

Conventional skeleton extraction method usually limits its applicability due to the close curve or complete boundary constraint. Unfortunately in cluttered image or natural image, the close curve or complete boundary constraint happens all the time. Therefore a skeleton extraction algorithm with an incomplete boundary condition is proposed. The proposed method is based on the Delaunay triangulation of some sampled points from the input image. By using the proposed method, the incomplete boundary condition can be relaxed to some extends.