Hiromasa Nakatani

An error analysis for surface orientation from vanishing points

Given two parallel lines in a plane, their images formed by central projection onto the viewing plane intersect at a vanishing point, which provides a constraint on the orientation of the plane. Two such independent constraints define a vanishing line and thereby determine the orientation of the plane uniquely. In order to effectively use this information, an estimate of its accuracy must be incorporated. In the approach suggested, line directions and surface normal vectors are represented as points on a Gaussian sphere, and the error bounds are computed as regions on the sphere. Information from multiple vanishing points and surfaces is combined by intersecting the corresponding regions. In addition, constraints based on real-world knowledge can be introduced to improve the accuracy. Some experimental results on natural images re presented to illustrate this method. >

Three-dimensional measurement endoscope system with virtual rulers

Conventional endoscopic images do not provide quantitative 3-D information. We present an endoscope system that can measure the size and position of an object in real time. Our endoscope contains four laser beam sources and a camera. The procedural steps for 3-D measurements are as follows. First, to obtain the function that maps 2-D coordinates of an image point to its 3-D coordinates in 3-D space, we observe a standard chart with the endoscope lens and determine the correspondence between the image and object height. In addition to the mapping, this function can correct barrel-shaped distortion of endoscopic images. The system detects laser spots on an object surface automatically using a template matching method, and maps the 2-D coordinates of the laser spots to the 3-D coordinates by the triangulation method. Then the system calculates the magnification ratio on the object plane, which is perpendicular to the optical axis and passes the laser spot, so that the system can superimpose a ruler whose scale fits the 3-D coordinates of the object. Thus, physicians can measure the size and position of objects in real time on undistorted images similar to placing rulers on the surface of an organ.

Hand motion tracking based on a constraint of three-dimensional continuity

We propose to set a 3-D search volume for tracking a 3-D palm motion efficiently using two cameras. If we perform template matching for right and left images independently, two points in two images do not always correspond to each other. Then, we cannot always track the correct 3-D position. Instead of finding the corresponding point in each image, we set the search volume in the 3-D space, not in the 2-D image planes, so that only valid 2-D pairs are considered in the proposed search process. The tracking process is as follows. First, we set the search volume. The 3-D coordinates of the search volume are projected on two in each image plane. We perform template matching at the projected pixel in each image. The similarity of the 3-D position is computed from two dissimilarities in the two images. We search for the position that has the maximum similarity in the search volume, and we obtain the correct correspondence result. We incorporate this technique into our tracking system, and we compare the proposed method with a method that tracks a palm motion without epipolar constraint. Our experimental results show that use of the proposed 3-D search volume makes the method accurate and efficient for tracking the 3-D motion.

Muscle-Based Feature Models for Analyzing Facial Expressions

We propose deformable models for tracking the continual motions of facial features, such as the eyebrows and mouth, in facial images. Directions and ranges of deformations of each facial feature are physically constrained by facial muscles. By using the directions and locations of facial muscles, feature models can be constructed with a few parameters and can be deformed only in the proper range and to the proper direction. The model parameters are obtained from the deformations of facial muscles, hence the proposed model can be easily applied to the anatomical analysis of facial expressions.

Interactive image retrieval by natural language

This paper presents a method of building an image data retrieval system that can accept Japanese sentences and handle subjective expressions. A naive user who has little knowledge about objects in the database is likely to use subjective words to explain what he or she wants, for instance, ‘‘show me a cute one,’’ or ‘‘I would like to have a simpler one.’’ Objective interpretation of those expressions is difficult but is indispensable to retrieval systems. In this paper we propose a technique for matching subjective expressions with color features and discuss usability of our natural language interface.

Tomogram reconstruction in desired plane based on serial tomograms

Abstract Computer synthesis of an image from tomograms is described. Several ultrasonic cardiograms are taken at every 5 or 2.5 mm from the anterior chest wall, digitized by a flying-spot-scanner and stored on a magnetic disk. Looking at a border-lined tomogram displayed on a CRT, the operator specifies the cross section that he wants displayed. A minicomputer retrieves from the disk the data corresponding to the desired section and displays the sectional image on the CRT. A feature of the computer synthesis is that the tomogram at any angle can be obtained using 3-dimensional information. The method can therefore display a 2-dimensional echocardiogram at a plane parallel to the chest wall in spite of the proximity of the lungs to the heart.

Recognition of facial expressions using muscle-based feature models

We present a technique for recognizing facial expressions from image sequences. The technique uses muscle-based feature models for tracking facial features. Since the feature models are constructed with a small number of parameters and are deformable in the limited range and directions, each search space for a feature can be limited. The technique estimates muscular contractile degrees for classifying six principal facial expressions. The contractile vectors are obtained from the deformations of facial muscle models. Similarities are defined between those vectors and representative vectors of principal expressions and are used for determining facial expressions.

Image region correspondence by structural similarity

Presents an approach for matching regions in images. Images are segmented into regions and are individually described by adjacency relations using region tables. The matching procedure is composed of two processes, global structural matching and local region matching. The global structural matching process selects a pair of region tables in two images which have the best resemblance between their adjacencies; the local region matching process matches regions which are surrounding the best matched region, and it has the ability to tolerate inconsistency of region segmentation to some extent.<<ETX>>

Boundary value problem of image modification

The concept of the boundary value problem into image processing is introduced and an image modification technique is presented that works under the condition that the transformation function for a set of pixels is given a priori. For example, if adjoining pictures are taken separately under different illumination and then put together into a single picture, processing the whole picture uniformly can result in some artifacts along the seams, across which image features change abruptly. To resolve this problem, the image features of the border pixels should be transformed to be continuous with the neighboring pictures. Thus, the transformation function for such pixels should be set a priori to meet the above condition, and for the remaining pixels it should be adjusted accordingly. The same technique can be applied to a single picture. For example, if a picture is taken under nonuniform illumination, which causes some regions of the picture to be dark and others to be light, the ransformation function for those regions should be given as boundary conditions. Then the function at any pixel is interpolated from the boundary values. An interactive technique is discussed for giving the boundary conditions and for determining the image transformation function.

An Error Analysis For Surface Orientation From Vanishing Points

There are many cases in which perspective information can be used to derive three-dimensional spatial information about objects from their two-dimensional images. There are established algorithms for estimating the direction of lines and the orientation of surfaces based on their projections onto the image plane. Given two parallel lines on a plane, their projections onto the viewing plane intersect at a vanishing point, which provides a constraint on the orientation of the plane. Two such independent constraints define a vanishing line, and thereby determine the orientation of the plane uniquely. In order to effectively recover surface orientations via lines extracted from the image, it is necessary to put bounds on the errors while applying these constraints. Our approach involves representing line directions and surface normal vectors as points on a Gaussian sphere and computing the error bounds as regions on the sphere. Multiple constraints are combined by intersecting the corresponding regions. The starting point for computing the error bounds is an estimate of the accuracy of the lines which are extracted from the image. A mathematical analysis of the imaging geometry is used to propagate these errors to vanishing points, vanishing lines, and surface orientations. In addition, constraints based on a priori knowledge can be introduced to improve the accuracy. Some experimental results are presented to illustrate this.