Takayuki Okatani

Shape Reconstruction from an Endoscope Image by Shape from Shading Technique for a Point Light Source at the Projection Center

This paper presents a method for reconstructing the 3D shape of an object from its endoscope image based on image shading. The primary problem is that the endoscope has a light source near the object surface. Most of the conventional shape from shading methods assumed that the light source was distant from the object surface and simplified the analysis. To deal with the near light source, we use the configuration of the endoscope that the light source of the endoscope is well approximated by an imaginary point source at the projection center. In addition, we introduce a notion of equal distance contours of the object surface; by propagating the contours using the image shading, we reconstruct the object shape. This is an extension of the Kimmel?Bruckstein algorithm of shape from shading to the endoscope images. Experimental results for real medical endoscope images of the stomach wall show the feasibility of this method and also show its promising availability for morphological analyses of tumors on human inner organs.

On the Wiberg Algorithm for Matrix Factorization in the Presence of Missing Components

This paper considers the problem of factorizing a matrix with missing components into a product of two smaller matrices, also known as principal component analysis with missing data (PCAMD). The Wiberg algorithm is a numerical algorithm developed for the problem in the community of applied mathematics. We argue that the algorithm has not been correctly understood in the computer vision community. Although there are many studies in our community, almost every one of which refers to the Wiberg study, as far as we know, there is no literature in which the performance of the Wiberg algorithm is investigated or the detail of the algorithm is presented. In this paper, we present derivation of the algorithm along with a problem in its implementation that needs to be carefully considered, and then examine its performance. The experimental results demonstrate that the Wiberg algorithm shows a considerably good performance, which should contradict the conventional view in our community, namely that minimization-based algorithms tend to fail to converge to a global minimum relatively frequently. The performance of the Wiberg algorithm is such that even starting with random initial values, it converges in most cases to a correct solution, even when the matrix has many missing components and the data are contaminated with very strong noise. Our conclusion is that the Wiberg algorithm can also be used as a standard algorithm for the problems of computer vision.

Efficient algorithm for low-rank matrix factorization with missing components and performance comparison of latest algorithms

This paper examines numerical algorithms for factorization of a low-rank matrix with missing components. We first propose a new method that incorporates a damping factor into the Wiberg method to solve the problem. The new method is characterized by the way it constrains the ambiguity of the matrix factorization, which helps improve both the global convergence ability and the local convergence speed. We then present experimental comparisons with the latest methods used to solve the problem. No comprehensive comparison of the methods that have been proposed recently has yet been reported in literature. In our experiments, we prioritize the assessment of the global convergence performance of each method, that is, how often and how fast the method can reach the global optimum starting from random initial values. Our conclusion is that top performance is achieved by a group of methods based on Newton-family minimization with damping factor that reduce the problem by eliminating either of the two factored matrices. Our method, which belongs to this group, consistently shows a 100% global convergence rate for different types of affine structure from motion data with a very high population of missing components.

Object tracking by the mean-shift of regional color distribution combined with the particle-filter algorithms

This paper presents a method for tracking a person in a video sequence in real time. In this method, the profile of color distribution characterises targets feature. It is invariant for rotation and scale changes. It is also robust to non-rigidity and partial occlusion of the target. We employ the mean-shift algorithm to track the target and to reduce the computational cost. Moreover, we incorporate the particle-filter into it to cope with a temporal occlusion of the target, and largely reduce the computational cost of the original particle-filter. Experiments show the availability of this method.

Autocalibration of a projector-camera system

This paper presents a method for calibrating a projector-camera system that consists of multiple projectors (or multiple poses of a single projector), a camera, and a planar screen. We consider the problem of estimating the homography between the screen and the image plane of the camera or the screen-camera homography, in the case where there is no prior knowledge regarding the screen surface that enables the direct computation of the homography. It is assumed that the pose of each projector is unknown while its internal geometry is known. Subsequently, it is shown that the screen-camera homography can be determined from only the images projected by the projectors and then obtained by the camera, up to a transformation with four degrees of freedom. This transformation corresponds to arbitrariness in choosing a two-dimensional coordinate system on the screen surface and when this coordinate system is chosen in some manner, the screen-camera homography as well as the unknown poses of the projectors can be uniquely determined. A noniterative algorithm is presented, which computes the homography from three or more images. Several experimental results on synthetic as well as real images are shown to demonstrate the effectiveness of the method.

Shape reconstruction from an endoscope image by shape-from-shading technique for a point light source at the projection center

Describes an approach to reconstructing a shape from its shaded image in the case where a point light source is at the projection center. This condition well approximates the imaging system of an endoscope. In this case, the image gray level depends on not only the gradient of the object surface but also the distance from the light source to each point on the surface. To deal with this difficulty, the authors introduce the evolution equation for equal-range contours on the surface. Propagating this contour by solving the equation, one can reconstruct a shape. Experimental results far real medical endoscope images of a human stomach inner wall show feasibility of this method, and present a promising technique for morphological analysis of tumors on human inner organs.

Absolute phase measurements using geometric constraints between multiple cameras and projectors.

The method of phase shift with the projection of multiple cyclic patterns enables 3D measurement that is highly accurate, dense, and fast. However, this measurement is only possible for the wrapped phase value, which has ambiguities in its multiples of cycles. Two particular problems are that conventional methods require additional patterns to be projected to determine the absolute phase and that unwrapping the phase tends to fail where depth varies abruptly. Two methods are proposed: the first is to determine the absolute phase without additional patterns being projected by observing the projected pattern with multiple cameras and applying the geometric constraints between them, and the second is to prevent failure in unwrapping the phase by referring to continuities in the relative phases of multiple projected patterns. The proposed methods were achieved with a 3D scanner that can measure approximately a 180 degrees field of view within 0.5 s, with an accuracy of 0.14 mm in depth.

Detecting Changes in 3D Structure of a Scene from Multi-view Images Captured by a Vehicle-Mounted Camera

This paper proposes a method for detecting temporal changes of the three-dimensional structure of an outdoor scene from its multi-view images captured at two separate times. For the images, we consider those captured by a camera mounted on a vehicle running in a city street. The method estimates scene structures probabilistically, not deterministically, and based on their estimates, it evaluates the probability of structural changes in the scene, where the inputs are the similarity of the local image patches among the multi-view images. The aim of the probabilistic treatment is to maximize the accuracy of change detection, behind which there is our conjecture that although it is difficult to estimate the scene structures deterministically, it should be easier to detect their changes. The proposed method is compared with the methods that use multi-view stereo (MVS) to reconstruct the scene structures of the two time points and then differentiate them to detect changes. The experimental results show that the proposed method outperforms such MVS-based methods.

Video Synchronization Based on Co-occurrence of Appearance Changes in Video Sequences

This paper presents a method for synchronizing multiple cameras from only the images captured by the cameras, assuming that they are not connected to an external clock signal source. It is assumed that The cameras are stationary and take the images of the same scene from various viewpoints, in which there are moving objects such as human in motion. The method uses the appearance changes in an image sequence as a temporal feature and matches two or more sequences by evaluating the correlation among their temporal features. We show through several experiments that the method shows good performance of synchronization in spite of its simplicity. We also present a method for synchronization in sub-frame accuracy and also for adaptively selecting regions suitable for deriving the temporal features. The latter resolves the difficulty with the case where some objects appear in some of the cameras and do not in the rest, which could deteriorate accuracy of the synchronization

A color-based tracking by Kalman particle filter

In this paper, a method for real-time tracking of moving objects is proposed. We applied Kalman particle filter (KPF) to color-based tracking. This KPF is a particle filter including the principle of Kalman filter, and it was adopted to the object contour tracking. We modified this KPF for color-based tracking. This modified KPF can approximate the probabilistic density of the position of the tracked object properly and needs fewer particles for tracking than conventional particle filters. We made experiments to confirm the effectiveness of this method.