Yoshinori Niwa

Focus of attention for face and hand gesture recognition using multiple cameras

We propose a multi-camera system that can track multiple human faces and hands as well as focus on face and hand gestures for recognition. Our current system consists of four cameras. Two fixed cameras are used as a stereo system to estimate face and hand positions. The stereo camera detects faces and hands by a standard skin color method we propose. The distances of the targets are then estimated. Next to track multiple targets, we estimate the positions and sizes of targets between consecutive frames. The other two cameras perform tracking of such targets as faces and hands. If a target is not the appropriate size for recognition, the tracking cameras acquire its zoomed image. Since our system has two tracking cameras, it can track two targets at the same time. To recognize faces and hand gestures, we propose four directional features by using linear discriminant analysis. Using our system, we experimented on human position estimation, multiple face tracking, and face and hand gesture recognition. These experiments showed that our system could estimate human position with the stereo camera and track multiple targets by using target positions and sizes even if the persons overlapped with each other. In addition, our system could recognize faces and hand gestures by using the four directional features.

Comparison of local plane fitting methods for range data

In this research, we introduce a reasonable noise model for range data which is obtained by a laser radar range finder, and derive two simple approximate solutions of optimal local plane fitting the range data under the noise model. We compare our methods with general least-squares based methods, such as Z-function fitting, the eigenvalue method, the maximum likelihood estimation method, and the renormalization method, an iterative method to obtain the optimal fitting of planes of range data under the noise model. All the methods are compared and evaluated using both synthetic range data and real range data with ground truth. From the experimental evaluation results, the proposed methods are shown to be effective, and the general least-squares-based methods are shown to be unsuitable for the assumed noise model.

Age and gender estimation from facial image processing

We are researching about an age and gender estimation based on wrinkle texture and color of facial images. A preliminary questionnaire (enquete) examination on how effectively the facial images could be used for gender and age estimations was executed by using 300 different faces and 21 examinees. Wrinkles appearing in the face and the shape and size of the facial parts are selected to model the age and gender estimation in this research basing on this enquete. Basing on this preliminary consideration, an image-processing algorithm for wrinkle modeling, was proposed. In addition, a method for making relationships between facial images and their keywords was proposed by using the latent semantic indexing. An efficient interface for displaying the relationships among keywords and facial images has been introduced with 2D/3D-feature space defined by the indexing.

Acquisition of three-dimensional information in a real environment by using the Stereo Omni-directional System (SOS)

In the field of virtual reality technology, it is an important issue to construct the real 3-D virtual environment of the actual event. Furthermore various systems that can represent these environments have been proposed. This paper proposes a method to acquire 3-D information of a dynamic real environment in all directions by using a stereo omnidirectional system (SOS), which we developed. This system can acquire all direction color images and stereo image pairs in real time. All cameras on this system operate simultaneously, and all images can be obtained at the same time. From the stereo image pairs, depth maps can be calculated easily with high accuracy. In the depth map, the moving region in the real environment can be extracted stably. A virtual environment of a real environment can be constructed separately for static and dynamic environment purposes.

Face and head detection for a real-time surveillance system

This paper describes a face and head detection method for a real-time surveillance system. Since there is no guarantee that surveillance cameras can capture frontal face or full-body of human, face and head detection has an advantage for the practical use. Proposed method employs four directional features and linear discriminant analysis. It can detect face and head simultaneously, and reduce computation cost. In the experiments, comparison of two classifiers and evaluation of proposed human detection method were performed using still images and video scenes. The results showed that the performances of two classifiers were almost equivalent. Thus, the classifier labeled face samples to one class was better in terms of computation cost. In the human detection experiment, the results were 87.2% (48/55) for human detection rate, and 83.6% (832/995) for reliability of detection. The proposed detection method was implemented on a PC and run at approximately over 10 fps for VGA input with motion detection.

Moving object detection by mobile Stereo Omni-directional System (SOS) using spherical depth image

Moving object detection with a mobile image sensor is an important task for mobile surveillance systems running in real environments. In this paper, we propose a novel method to effectively solve this problem by using a Stereo Omni-directional System (SOS), which can obtain both color and depth images of the environment in real time with a complete spherical field of view. Taking advantage of the SOS that the frame-out problem never occurs, we develop a method to detect the regions of moving objects stably under arbitrary movement and pose change of the SOS, by using the spherical depth image sequence obtained by the SOS. The method first predicts the depth image for the current time from that obtained at the previous time and the ego-motion of the SOS, and then detects moving objects by comparing the predicted depth image with the actual one obtained at the current time.

Real-time face and head detection using four directional features

Real-time human detection is an important part in a surveillance system using computer vision. In this paper, a real-time face and head detection method is proposed for such human detection. The method has an advantage of detecting peoples who are not facing a camera, by detecting their heads. It employs four directional features (FDF) and linear discriminant analysis in order to save computation cost for scanning and classification. Since FDF represents edge directional information in low resolution, it is resistive to changes in lighting conditions and scales. The proposed method was evaluated through an experiment using 26 video scenes. The results of experiment were 83.6% (46/55) for human detection rate, and 84.6% (1048/1239) for reliability of detection. The human detection system implemented with the proposed method runs on a PC at approximately over 10 fps for VGA input with motion detection.

Development of a stereo omnidirectional imaging system (SOS)

We proposed a stereo omnidirectional imaging system, named SOS. The system can take all directional views and depth maps from an observation point, in real-time, by using a component constructed of sixty cameras. The cameras are separated into twenty stereo vision units. Each unit is composed of three cameras that are arranged horizontally and vertically. These units are arranged on each face of a regular icosahedron and each stereo vision unit is arranged so that it does not obstruct the view of other units. As a result, we succeeded in reducing the size of the entire sensor, securing the base line length to generate the depth value with enough precision to allow the simultaneous acquisition of the color images and depth maps omnidirectionally in three dimensions. In this paper the characteristics of our proposed system and an example image acquired from the system are then shown.

Event Detection for a Visual Surveillance System Using Stereo Omni-directional System

In this paper, by using a stereo omni-directional system, we propose an automatic surveillance system for detecting the events in which a person enters or leaves a room and of an object appearing or disappearing. It is important for a video surveillance system to detect events automatically and offer effective information. A stereo omni-directional system captures all directional color and disparity images from an observation point simultaneously, in real-time. Using background and frame subtraction methods, our system detects the foreground pixels, analyzes them and estimates the status of human and object regions.

Estimation of human motion from multiple cameras for gesture recognition

We have been researching human sensing technologies based on computer vision for the realization of the Percept-Room as an intelligent environment. This room offers several services such as the control of electrical appliances by use of gestures. A more caring technology is strongly, expected for providing more human-friendly interfaces. As a first step, the recognition of human motion in the Percept-Room is an important task. We propose a method for extracting humans and then hand gestures from multichannel motion images captured in the Percept-Room. The human position and the rising hand gestures are estimated by integrating the silhouettes from multiple cameras by a background image subtraction and a frame subtraction. We describe the proposed methods and the experimental results obtained by use of a PC system implemented in the Percept-Room.