Masahide Kaneko

Design and implementation of SPIDER—A transportable image processing software package

Abstract SPIDER is a general-purpose image processing software package which consists of over 400 FORTRAN IV subroutines for various image processing algorithms and several utility programs for managing them. The package was developed for the benefit of extensive interchange and accumulation of programs among research groups. Thus, high transportability of software is emphasized above all in its design concept. In effect, all the image processing subroutines are implemented to be completely free of I/O work such as file access or driving peripheral image devices. The specifications of SPIDER programs also regulate the style of comments in source programs and documentation for the users manual. SPIDER may also be very useful as a research tool in other scientific disciplines as well as integrating fundamental algorithms in the image processing community. The design concepts, specifications, and contents of SPIDER are described.

3D visual data compression based on ray-space projection

As 2D image communication systems come into use widely, 3D imaging technology enhancing the reality of visual communication is getting to be considered as a promising next-generation medium that can revolutionize information systems. To date, 3D image communication has not been discussed at a comprehensive level because several kinds of promising 3D display technologies are still making rapid progress. Considering such a situation, this paper introduces the concept of the integrated 3D visual communication system. The key feature in this new concept is a display-independent neutral representation of visual data. The flexibility of this concept will promote the progress of 3D image communication systems before the 3D display technology reaches maturity. In this paper, for this purpose, ray-based approach is examined. In the present representation method, the whole ray data is equally treated as a set of orthogonal views of the scene objects. The advantage of this approach is to allow the synthesis of any perspective view by gathering appropriate ray data from the set of orthogonal views independently of any geometric representation. A real-time progressive transmission method has been also examined. The experimental results show how the present representation method could be applied to the next-generation 3D image communication system.

3D segmentation of multiview images based on disparity estimation

This paper presents new methods for partitioning a set of multi-view images into 3-D regions corresponding to objects in the scene in order to parse raw multi-view data into a 3-D region based structured representation. For this purpose, color, position, and disparity information at each pixel are incorporated as an attributes vector into the segmentation process. We propose three methods, all of which are based on K-means clustering algorithm. The first method is sensitive to the estimation error of disparity at each pixel, as it is formulated assuming that the estimated disparity is accurate. We solve this problem in the second method by prohibiting estimated disparity from being used for calculating the distance between attributes vectors. Finally, a third method is proposed to reduce the calculation cost of the segmentation process. As each 3-D region has one-to-one correspondence to an object or surface in the scene, 3-D region based structured representation of multi-view images is useful and powerful for data compression, view interpolation, structure recovery, and so on. The experimental results show the potential applicability of the method to the next-generation 3-D image communication system.

Structural motion segmentation for compact image sequence representation

This paper addresses a problem of extraction of the structural motion information for compact image sequence representation. In order to extract a meaningful scene structure from image sequence, global motion and region shape of moving objects are taken into consideration. Firstly, intraframe segmentation is carried out with edges that are detected from zero-crossings of a wavelet transform, and local motions are estimated using a gradient-based method. Moving regions are then extracted using the local motion based on the intraframe segmentation. Secondly, moving regions are roughly separated into the region of the moving objects based on probabilistic clustering with mixture models using the optical flow and the image intensity for each region of the intraframe segmentation. Motion segmentation can finally be obtained by iterated estimation of affine motion parameters and region reassignment according to a criterion using Gauss-Newton iterative optimization algorithm.

Interactive model-based coding for face metaphor user interface in network communications

The model-based coding is a new semantic-based coding which utilizes a common knowledge database being both at a transmitter and its receiver. We have proposed the interactive model-based coding for the face metaphor user interface applications such as multimedia e-mail, WWW and agents. Three points are addressed in this paper. (l)We have constructed a framework of interactive model-based coding tool in network communications environment. (2)We show that motion parameter description for face expressions should have both CGbased synthesis and natural image-based synthesis. The corresponding descriptions by relative and absolute way are necessary for sufficient face metaphor expressions. (3)We have developed the encoder and decoder. The motion parameters are transmitted from experimental server to a receiver. The decoded motion picture can be demonstrated both by a videotape and by a PC on-line browsing through the internet.

Fractal-like video coding with weighted summation

We introduce in the present paper a fast 3D fractal encoding method based on tree compression and achieving good compression ratio at very low bit rate. The algorithm is also open to transposition on the space of wavelet coefficients.

Command coding of moving images with interactive user's operations

This article relates to interactive command coding with active participation by the user in the coding process through his interactive operations, and structuralization of a moving image according to its substance. This coding process enables a low rate transmission with the preferential transmission of the substance of the original image, which is performed by using the users interactive operations for extracting characteristics of objects relating to their shapes, texture, and motions, and by appropriately assigning a detailed description-based or simplified description-based technique, depending on the substance of the original image. In addition, it enables easy browsing for capturing outlines on the decoding side, and easy application even to nonlinear image editing, by modifying the motion-related parameters and also modifying the feature description technique to another technique. Concrete examples including test examples are given to explain this approach.

Interactive operation on 3D real images using ray description

In the field of 3D visual communications, a method of description of 3D images that would be neutral (independent) with respect to specific techniques for 3D input–output is desirable. In this context, the authors have proposed the concept of “ray-space” as a ray-based description method for 3D image data. In this paper, the concept of “ray-space” is employed to offer a new method for simultaneous description of 3D objects and space, as well as a technique for interactive manipulation in virtual space built using real images. Synthesis of images as seen from an arbitrary viewpoint in real space means extraction of ray data on a curved surface described by some basic formula in ray-space, using information about the viewpoint location. In this study, real-time processing is pursued; for this purpose, a reference table is employed for efficient extraction of ray data. Experiments were carried out concerning virtual manipulation of real objects in near-real time by means of overwriting ray-space data with ray data of the 3D object under consideration. The experiments confirmed that the proposed method offered efficient description of 3D space data and object manipulation.

Representation of a moving image sequence with skewed planes and its application to video coding

This paper proposes a new coding method using structural features such as shape and motion in a moving image so that they are described as plane motion in a 3D space. In this method, a layered representation (a moving image is described on a 2D projection plane) is extended by using a setting representation so as to handle 3D data. In the setting representation, 3D parameters which describe the orientation of a plane and the motion of a rigid body are needed for the regions that move coherently on the projection plane. Parameters can be estimated stably by applying a conventional 3D parameter estimation algorithm (which uses the square-error criterion based on the assumption of equal intensity between adjacent frames) to multiple frames. This paper also proposes a method of integration of the rigid-body motion of multiple regions so that the motion of images can be described more efficiently. The proposed method is compared with the layered representation using 2D data, and a conventional waveform-based method. The results show the effectiveness of the proposed method.