Makoto Hirose

Reconstruction of realistic 3D surface model and 3D animation from range images obtained by real time 3D measurement system

We have developed a new type of 3D measurement system which enabled one to obtain successive 3D range data at video rate with an error within /spl plusmn/0.3%. In this paper, we reconstruct realistic colored 3D surface model and 3D animation of the moving target from range images obtained by our 3D measurement system. We synthesize video images with the range images obtained by our system. A video camera is fixed on our 3D measurement system and takes video images synchronizing with the 3D measurement by it. Then, the measured 3D points are mapped onto the respective video images by means of the coordinate system transformation from the coordinate system of the 3D measurement system to that of the video camera and perspective transformation. Thus, the color and brightness of the corresponding pixel are attributed to the mapped measured 3D point. Finally, the realistic colored 3D image and animation are reconstructed through the texture mapping technique.

Automatic registration of multi-view range images measured under free condition

Describes a method for automatic registration of real-time serial range images obtained free from any artificial restriction. As the first step, serial range images are captured by a real-time 3-D measurement system developed in our laboratory. Then, four cross points between four real and/or virtual planes extracted from an extended Gaussian-image of the target are estimated in each range image and four corresponding points between two consecutive range images are determined from thus estimated cross points and extended Gaussian-images, automatically. Next, affine transformation matrices, which give the mutual position relationship between two consecutive range images, are derived in order from the thus determined four corresponding points. Concatenating the thus derived affine transformation matrices in order, registration of all serial range images are performed by transforming them to one coordinate system of an arbitrarily chosen range image. As a result, automatic registration of time sequential range images is realized with high accuracy and low computational cost.

3D gesture recognition from serial range image

In this research, the recognition of gesture in 3D space is examined by using serial range images obtained by a real-time 3D measurement system developed in our laboratory. Using this system, it is possible to obtain time sequences of range, intensity and color data for a moving object in real-time without assigning markers to the targets. At first, gestures are tracked in 2D space by calculating 2D flow vectors at each points using an ordinal optical flow estimation method, based on time sequences of the intensity data. Then, location of each point after 2D movement is detected on the x-y plane using thus obtained 2D flow vectors. Depth information of each point after movement is then obtained from the range data and 3D flow vectors are assigned to each point. Time sequences of thus obtained 3D flow vectors allow us to track the 3D movement of the target. So, based on time sequences of 3D flow vectors of the targets, it is possible to classify the movement of the targets using continuous DP matching technique. This tracking of 3D movement using time sequences of 3D flow vectors may be applicable for a robust gesture recognition system.