Kouta Fujimura

Motion Tracking of Deformable Objects by Active Contour Models Using Multiscale Dynamic Programming

Abstract This paper describes a method of motion tracking and analysis of a deformable object in time-varying images. The approach is based upon modeling the boundary of the object as a new active contour model defined by an energy functional in terms of both intra- and interframe constraints, and then tracking the object boundary in consecutive image frames by minimizing the energy functional using dynamic programming in multiple scales. In order to increase the efficiency of the method, dynamic programming is carried out by using variable neighbors for each model point; that is, we define coarse neighbors at a coarse scale, and finer ones at a finer scale. The proposed motion tracking method has been successfully applied to the behavioral analysis of a slug ( Limax flavus ) in the biological study of learning and memory formation in slugs, and also to the problem of tracking the boundary of the left ventricle of the heart from time-varying ultrasonic echocardiographic images. In the former application, motion analysis is further carried out by tracking characteristic points which are detected as local curvature extrema along the object contour. In the latter application, the temporal change of the apparent area of target object region is computed in order to quantify cardiac motion.

3D modeling system of human face and full 3D facial caricaturing

Proposes a method for modeling a 3D face from the 2D facial images captured from the surrounding 2D cameras by which the color texture and surface shape information of the face are synchronously measured. A 3D facial caricaturing method is proposed by using the 3D (the polygon data) face model. An automatic method for extracting regions of the facial parts is proposed, and the feature points are extracted from those regions. We propose a mesh model composed of 44 feature points and 82 meshes to cover a head. To generate the caricature from this polygon data the individuality feature is defined in value by the difference of the feature points between the input face and the mean face, which was defined from the average of many input faces.

A flexible 3D modeling system based on combining shape-from-silhouette with light-sectioning algorithm

In this paper we present a flexible modeling system for obtaining the texture-mapped 3D geometric model. The modeling system uses an algorithm combining shape-from-silhouette with light-sectioning. In the algorithm, at first, a rough shape model is obtained by shape-from-silhouette method almost automatically. Next, concavities and complex parts on the object surface are obtained by light-sectioning method with manual scanning. For applying light-sectioning method to volume data, we propose volumetric light-sectioning algorithm. Then our modeling system can realize easy and accurate generation of 3D geometric model.

Multicamera 3D modeling system to digitize human head and body

A multi-camera 3D modeling system to digitize a human head and body is presented in this paper. The main features of this system are as follows: 1) Fast capturing: Both of texture images and pattern images can be taken within a few seconds using multiple digital still cameras which are set around the target human. Slide projectors are also set to provide a color line patterned light on the target for pattern image capturing, 2) Realistic Shape and Texture: The whole shape and photo-realistic textures of the human head including hair can be digitized at a time on a personal computer, and 3) Hybrid Algorithm: Our modeling algorithm is based on a hybrid method where the Shape-from-Silhouette technique and the Active-Stereo technique are combined. In the first step, the rough shape of the target is estimated in a voxel space using our Extended Shape-from-Silhouette method. In the next step, the shape is refined based on the depth-map data that is calculated using a multi-camera active stereo method. This combination makes up for the shortcomings of each method. Our system has been applied to the digitizing several Japanese people using sixteen cameras for texture image capturing and twelve cameras and two projectors for pattern image capturing. Its capturing time is approximately three seconds and calculation time is about 15-20 minutes on a personal computer with the Pentium-III processor (600MHz) and 512MB memory to digitize the whole shape as well as the texture of the human head and body.

CyberModeler: a compact 3D scanner based on monoscopic camera

A 3D scanner based on a monoscopic camera is presented in this paper. The scanner, CyberModeler, is capable of reconstructing 3D shapes as well as obtaining texture information of a target object form multiple camera views. The CyberModeler is a compact, easy-to-use, and inexpensive system because it does not require any special ray such as laser. Several new techniques are featured in the CyberModeler: voting-based Shape-from-Silhouette, calibration using the Hough method, and texture acquisition using an energy minimization technique. These lead to robust use in real-world environments as well as wide applicability to viewing applications. Our experiments showed that not only was the quality of the models generated high enough for viewing but also the modeling speed was at an acceptable practical level.

Motion analysis of nonrigid objects by active contour models using multiscale images

This paper considers the approach to dynamic image processing, which is one of the important problems in the future medical image processing. The tracking of the ojbect and the analysis of the motion are discussed for the dynamic images of a nonrigid object with smooth shape, motion and deformation, which is the case in most medical images. This approach is based on an active contour model defined by an energy function in terms of both intra- and interframe constraints for the contour of the object. The contour of the target object is extracted and tracked by minimizing the energy function using multiscale dynamic programming and the motion is analyzed. The dynamic programming in multiscale proposed in this paper is to adjust the search neighborhood of the dynamic programming according to the scale. The coarse or fine neighborhood is defined for the coarse and fine scales, respectively, and the nergy is minimized starting from the coarse scale and shifting to the fine scale. By this scheme, a large motion an deformation of the object can be handled. The proposed motion tracking method has been applied successfully to the dynamic image in the “behavioral analysis of a slug aiming at the analysis of the neural mechanism of learning and memory formation in slugs,” as well as to dynamic echocardiographic images. In the first application, the positive maximum of the curvature along the contour is extracted in the motion analysis as a characteristic point invariant to the deformation of the object. Then the shift of that point is traced. By this approach, the rough motion of the object can be estimated.

Motion tracking of deformable objects based on energy minimization using multiscale dynamic programming

Describes a method for tracking a deformable object in a time-varying image sequence and analyzing the motion of the object. The approach is based upon modeling the boundary of the object as a new active contour model defined by an energy functional in terms of intra- and inter-frame constraints, and then finding the object boundary in the temporal images by minimizing the energy functional using dynamic programming in multiple scales. Motion analysis is carried out by analyzing some characteristic points which are extracted as local curvature extrema along the contour. The proposed method has been successfully applied to the analysis of a slug (Limax flavus) in motion with smooth deformation, in the study of the mechanisms of learning and memory formation in slugs.<<ETX>>

Improved 3D head reconstruction system based on combining shape-from-silhouette with two-stage stereo algorithm

In this paper, we present a quality improvement algorithm for the system, which models a human head. We have already proposed and applied the hybrid algorithm combining shape-from-silhouette with active stereo to our product of 3D head reconstruction system. Our system is characterized by the ability of the reconstruction of the whole shape and texture of the human head, even the black hair parts, while other existing systems cannot do it. Feature-based stereo algorithm adopted into our current system is the fast and robust approach. However, the depth data may be sparse and the accuracy depends on how accurate the edge detection is. On the other hand, area-based stereo algorithms generally provide dense depth data and can apply subpixel estimation, but it is comparatively time-consuming and may be inaccurate influenced by the difference of how to reflect the object in each image. To overcome these problems for improving our system practicably, we propose a novel two-stage stereo algorithm. In our algorithm, first, we adopt feature-based approach to get the depth data robustly. Next, an interpolation of the depth data is performed to predict the depth data at the unestimated pixels on the edge. Finally, applying area-based approach and subpixel estimation refine the depth data. In this paper, we describe our hybrid modeling algorithm and two-stage stereo algorithm with some experimental results.

Handheld camera 3D modeling system using multiple reference panels

A novel 3D modeling system in which a target object is easily captured and modeled by using a hand-held camera with several reference panels is presented in this paper. The reference panels are designed to be able to obtain the camera position and discriminate between each other. A conventional 3D modeling system using a reference panel has several restrictions regarding the target object, specifically the size and its location. Our system uses multiple reference panels, which are set around the target object to remove these restrictions. The main features of this system are as follows: 1) The whole shape and photo-realistic textures of the target object can be digitized based on several still images or a movie captured by using a hand-held camera; as well as each location of the camera that can be calculated using the reference panels. 2) Our system can be provided as a software product only. That means there are no special requirements for hardware; even the reference panels , because they can be printed from image files or software. 3) This system can be applied to digitize a larger object. In the experiments, we developed and used an interactive region selection tool to detect the silhouette on each image instead of using the chroma -keying method. We have tested our system with a toy object. The calculation time is about 10 minutes (except for the capturing the images and extracting the silhouette by using our tool) on a personal computer with a Pentium-III processor (600MHz) and 320MB memory. However, it depends on how complex the images are and how many images you use. Our future plan is to evaluate the system with various kind of objects, specifically, large ones in outdoor environments.