Yasuyo Kita

Real-Time Registration of 3D Cerebral Vessels to X-ray Angiograms

A quick method to obtain the 3D transformation of a 3D free-form shape model from its 2D projection data is proposed. This method has been developed for the real-time registration of a 3D model of a cerebral vessel tree, obtained from pre-operative data (eg. MR Angiogram), to a X-ray image of the vessel (eg. Digital Subtraction Angiogram) taken during an operation. First, the skeleton of the vessel in a 2D image is automatically extracted in a model-based way using a 2D projection of a 3D model skeleton at the initial state (up to ±20 degree difference in rotation). Corresponding pairs of points on the 3D skeleton and points on the 2D skeleton are determined based on the 2D Euclidean distance between the projection of the model skeleton and the observed skeleton. In the process, an adaptive search region for each model point, which is determined according to the projected shape, effectively removes incorrect correspondences. Based on a good ratio of correct pairs, linearization of a rotation matrix can be used to rapidly calculate the 3D transformation of the model which produces the 2D observed projection. Experiments using real data show the practical usefulness of the method.

Clothes state recognition using 3D observed data

In this paper, we propose a deformable-model-driven method to recognize the state of hanging clothes using three-dimensional (3D) observed data. For the task to pick up a specific part of the clothes, it is indispensable to obtain the 3D position and posture of the part. In order to robustly obtain such information from 3D observed data of the clothes, we take a deformable-model-driven approach[4], that recognizes the clothes state by comparing the observed data with candidate shapes which are predicted in advance. To carry out this approach despite large shape variation of the clothes, we propose a two-staged method. First, small number of representative 3D shapes are calculated through physical simulations of hanging the clothes. Then, after observing clothes, each representative shape is deformed so as to fit the observed 3D data better. The consistency between the adjusted shapes and the observed data is checked to select the correct state. Experimental results using actual observations have shown the good prospect of the proposed method.

A deformable model driven visual method for handling clothes

In this paper, we propose a deformable model-driven method to obtain the 3D information necessary for handling clothes by manipulators from observation with stereo cameras. The task considered in this paper is to hold up a target part of clothes (e.g. one shoulder of a pullover) by the second manipulator, when the clothes are held in the air at any point by the first manipulator. First, the method calculates possible 3D shapes of the hanging clothes by simulating the clothes deformation. The 3D shape whose appearance gives the best fit with the observed images is selected as estimation of the current state. Then, based on the estimated shape, the 3D position and normal direction of the part where the second manipulator should hold are calculated. The results of preliminary experiments using actual two manipulators have shown the good potential of the proposed method.

Correspondence between Different View Breast X Rays Using Curved Epipolar Lines

In this paper, we propose a method for matching a deformable object between time-varying stereo images. This method has been developed primarily to find correspondences between a cranio-caudal (CC) (head to toe) and a medio-lateral oblique (MLO) (shoulder to the opposite hip) X-ray mammogram of the same breast. The two mammograms are taken while the breast is compressed between the cassette and plate of the X-ray machine. This results in a different amount of compression in each direction. The deformations of the breast caused by the different compressions cause corresponding points to appear far from straight “epipolar lines” familiar from binocular stereo vision. The method developed in this paper calculates the curve in a MLO image corresponding to a point in the CC image through simulation of the differential deformation and the projection of a 3D curve corresponding to the point. Experiments on real data show that the method predicts well the corresponding position in the other image. We also illustrate the programs ability to estimate the 3D position of a lesion in the uncompressed breast.

A CAD system for the 3D location of lesions in mammograms

A CAD system for estimating the 3D (three-dimensional) positions of lesions found in two mammographic views is described. The system is an extension of our previous method [Comput. Vis. Image Understand. 83 (2001) 38] which finds corresponding 2D positions in different mammographic views. The method calculates curved epipolar lines by developing a simulation of breast deformation into stereo camera geometry. Using such curved epipolar lines, not only can we determine point correspondences, but can estimate the 3D location of a lesion. In this paper, we first explain the underlying principles and system organisation. The correctness of the 3D positions calculated by the system is examined using a set of breast lesions, which appear both in mammograms and in MRI data. The experimental results demonstrate the clinical promise of the CAD system.

A model-driven method of estimating the state of clothes for manipulating it

Aiming at manipulating clothes, a model-driven method of estimating the state of hanging clothes is proposed. We suppose a system consisting of two manipulators and a camera. The task considered in this paper is to hold a pullover at its two shoulders by two manipulators respectively, as a first step for folding it. The proposed method estimates the state of the clothes held by one manipulator in a model-driven way and indicates the position to be held next by the other manipulator. First, the possible appearances of the pullover when it is held at one point are roughly predicted. Using discriminative features of the predicted appearances, the possible states for the observed appearance are selected. Each appearance of the possible state is partially deformed so as to get close to the observed appearance. The state whose appearance successfully approaches closest to the observed appearance is selected as the final decision. The point to be held next is determined according to the state. The results of preliminary experiments using actual images have shown the good potential of the proposed method.

Change detection using joint intensity histogram

In the present paper, a method for detecting changes between two images of the same scene taken at different times using their joint intensity histogram is proposed. First, the joint histogram, which is a two-dimensional (2D) histogram of combinatorial intensity levels, (I1(x), I2 (x)), is calculated. By checking the characteristics of the ridges of clusters on the joint histogram, clusters that are expected to correspond to background are selected. The combinations of (I1, I2) covered by the clusters are determined as insignificant changes. Pixels having a different combinatorial intensity (I1(x), I2 (x)) from these combinations, are extracted as candidates for significant changes. Based on the gradient correlation between the images for each region consisting of these pixels, only regions with significant changes are distinguished. Experiments using real scenes show the practical usefulness of the method

A deformable model driven method for handling clothes

A model-driven method for handling clothes by two manipulators based on observation with stereo cameras is proposed. The task considered in this paper is to hold up a specific part of clothes (e.g. one shoulder of a pullover) by the second manipulator, when the clothes is held in the air by the first manipulator. First, the method calculates possible 3D shapes of the hanging clothes by simulating the clothes deformation. The 3D shape whose appearance gives the best fit with the observed appearance is selected as estimation of the current state. Then, based on the estimated shape, the 3D position and normal direction of the part where the second manipulator should hold are calculated. The experiments using actual two manipulators have shown the good potential of the proposed method.

Clothes handling based on recognition by strategic observation

In this paper, we propose a method to recognize clothing shape based on strategic observation during handling. When a robot handles largely deformed objects like clothes, it is important for the robot to recognize a constantly varying shape. Large variation in shape and complex self-occlusion, however, make recognition very difficult. To address these difficulties, we have proposed a model-driven strategy using actions for informative observation and have developed some core methods based on this strategy [1][2][3]. In this paper, we show how these core methods can be used for an actual task that involves handling an item of clothing. In addition to proposing a sequence for this task, basic functions for realizing the sequence are also described. Using a robot, the experimental results demonstrated practical utility of the proposed strategy.

DETERMINING CORRESPONDENCE BETWEEN VIEWS

Two-view breast screening using cranio-caudal (CC) and medio-lateral oblique (MLO) mammograms has been shown to detect more cancers and lead to less women being recalled to assessment [11], [12] than one-view screening. However, matching signs between two views of the same breast can be a difficult task due to the changing geometry and, crucially, the effects of breast compression. If it were only the geometry that were changing the matching problem would reduce to being one of wide-angle stereo [1]. In this paper we develop a model-based method for finding a curve in the medio-lateral oblique mammogram which corresponds to the potential positions of a point marked in the cranio-caudal mammogram. A more mathematical version of this paper is in [7]. Related work on this problem [10], [9] does not explicitly consider compression. However, work on analysis of stomach x-rays [6] has shown the possibilities of modelling 3D deformations using a model-based approach.