Kenichi Kanatani

Group-Theoretical Methods in Image Understanding

This book presents the mathematics relevant to image understanding by computer vision and gives examples of actual applications. Group representation theory, Lie groups and Lie algebras, the theory of invariance, tensor calculus, differential geometry and projective geometry are used for three-dimensional shape and motion analysis from images, making use of techniques such as shape from motion, shape from texture, shape from angle and shape from surface. Although the mathematics itself may be well known to mathematicians, people working in areas related to computer science, image understanding, computer vision and image processing have usually never studied such mathematics, and so may be surprised to learn that abstract mathematical concepts can be of enormous help in building intelligent computer vision systems.

Motion segmentation by subspace separation and model selection

Reformulating the Costeira-Kanade algorithm as a pure mathematical theorem independent of the Tomasi-Kanade factorization, we present a robust segmentation algorithm by incorporating such techniques as dimension correction, model selection using the geometric AIC, and least-median fitting. Doing numerical simulations, we demonstrate that oar algorithm dramatically outperforms existing methods. It does not involve any parameters which need to be adjusted empirically.

Analysis of 3-D rotation fitting

Computational techniques for fitting a 3-D rotation to 3-D data are recapitulated in a refined form as minimization over proper rotations, extending three existing methods-the method of singular value decomposition, the method of polar decomposition, and the method of quaternion representation. Then, we describe the problem of 3-D motion estimation in this new light. Finally, we define the covariance matrix of a rotation and analyze the statistical behavior of errors in 3-D rotation fitting. >

Stereological determination of structural anisotropy

Abstract A general mathematical formulation is given to the problem of determining the structural anisotropy by means of the stereological principle. Three cases are considered—distributed curves in the plane, distributed curves in the space and distributed surfaces in the space. The number of intersections with a probe line or plane is viewed as a transformation, which is termed the “Buffon transform”, between two distribution densities, and a form of its inverse transform is given. Then, the change of anisotropy due to the deformation of the material is formulated, and the strain is shown to be determined from the data of the intersection counting. All equations are written in the form of Cartesian tensor equations invariant to coordinate translations and rotations. A typical example is also given.

Statistical bias of conic fitting and renormalization

Introducing a statistical model of noise in terms of the covariance matrix of the N-vector, we point out that the least-squares conic fitting is statistically biased. We present a new fitting scheme called renormalization for computing an unbiased estimate by automatically adjusting to noise. Relationships to existing methods are discussed, and our method is tested using real and synthetic data. >

Shape from texture: general principle

Abstract The 3D shape of a textured surface is recovered from its projected image on the assumption that the texture is homogeneously distributed. Our method does not require recognition of the “structure”—regularity, periodicity, parallelism, orthogonality, etc.—of the texture distribution. First, the “homogeneity” of a discrete texture is precisely defined in terms of the “theory of distributions”. Next, distortion of the observed texture due to perspective projection is described in terms of the “first fundamental form” expressed as a function of the image coordinates. Based on this result, the 3D recovery equations for determining the surface shape are derived for both planar and curved surfaces. Some numerical schemes for solving these equations are proposed. Ambiguity in the interpretation of curved surfaces is also analyzed. Finally, some numerical examples for synthetic data are presented, and our method is compared with other existing methods.

Geometric Information Criterion for Model Selection

In building a 3-D model of the environment from image and sensor data, one must fit to the data an appropriate class of models, which can be regarded as a parametrized manifold, or geometric model, defined in the data space. In this paper, we present a statistical framework for detecting degeneracies of a geometric model by evaluating its predictive capability in terms of the expected residual and derive the geometric AIC. We show that it allows us to detect singularities in a structure-from-motion analysis without introducing any empirically adjustable thresholds. We illustrate our approach by simulation examples. We also discuss the application potential of this theory for a wide range of computer vision and robotics problems.

Detection of surface orientation and motion from texture by a stereoogical technique.

Abstract A new approach is given to detect the surface orientation and motion from the texture on the surface by making use of a mathematical principle called ‘stereology’. Information about the surface orientation is contained in ‘features’ computed by scanning the image by parallel lines and counting the number of intersections with the curves of the texture. A synthetic example is given to illustrate the technique. This scheme can also detect surface motions relative to the viewer by computing features of its texture at one time and a short time later. The motion is specified by explicit formulae of the computed features.

3D recovery of polyhedra by rectangularity heuristics

The three-dimensional shape of a polyhedron is reconstructed from its perspectively projected image by finding rectangular corners. If a corner is known to be rectangular, its three-dimensional configuration is analytically computed, and hence the surface gradients of the faces incident to it are also determined. In order to decide which corners are rectangular, the rectangularity heuristics are introduced, i.e. corners are assumed to be rectangular as long as no inconsistency arises as a result. An optimization technique is presented in order to cope with the problem of insufficiency of information and inconsistency due to error and noise.<<ETX>>

Detecting the motion of a planar surface by line and surface integrals

Abstract A mathematical formulation is presented for detecting the 3D motion of a planar surface from the motion of its perspective image without knowing correspondence of points. The motion is determined explicitly by numerical computation of certain line or surface integrals on the image. The same principle is also used to know the position and orientation of a planar surface fixed in the space by moving the camera or using several appropriately positioned cameras, and no correspondence of points is involved. Some numerical examples are also given.