Andrew P. Witkin

Snakes: Active Contour Models

A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the capture region surrounding a feature. Snakes provide a unified account of a number of visual problems, including detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.

Large steps in cloth simulation

The bottle-neck in most cloth simulation systems is that time steps must be small to avoid numerical instability. This paper describes a cloth simulation system that can stably take large time steps. The simulation system couples a new technique for enforcing constraints on individual cloth particles with an implicit integration method. The simulator models cloth as a triangular mesh, with internal cloth forces derived using a simple continuum formulation that supports modeling operations such as local anisotropic stretch or compression; a unified treatment of damping forces is included as well. The implicit integration method generates a large, unbanded sparse linear system at each time step which is solved using a modified conjugate gradient method that simultaneously enforces particles’ constraints. The constraints are always maintained exactly, independent of the number of conjugate gradient iterations, which is typically small. The resulting simulation system is significantly faster than previous accounts of cloth simulation systems in the literature.

Constraints on deformable models: recovering 3D shape and nongrid motion

Abstract Inferring the 3D structures of nonrigidly moving objects from images is a difficult yet basic problem in computational vision. Our approach makes use of dynamic, elastically deformable object models that offer the geometric flexibility to satisfy a diversity of real-world visual constraints. We specialize these models to include intrinsic forces inducing a preference for axisymmetry. Image-based constraints are applied as extrinsic forces that mold the symmetry-seeking model into shapes consistent with image data. We describe an extrinsic force that applies constraints derived from profiles of monocularly viewed objects. We generalize this constraint force to incorporate profile information from multiple views and use it to exploit binocular image data. For time-varying images, the force becomes dynamic and the model is able to infer not only depth, but nonrigid motion as well. We demonstrate the recovery of 3D shape and nonrigid motion from natural imagery.

Analyzing oriented patterns

Abstract Oriented patterns, such as those produced by propagation, accretion, or deformation, are common in nature and therefore an important class for visual analysis. Our approach to understanding such patterns is to decompose them into two parts: the flow field, describing the direction of anisotropy; and the residual pattern obtained by describing the image in a coordinate system built from the flow field. We develop a method for the local estimation of anisotropy and a method for combining the estimates to construct a flow coordinate system. Several examples of the use of these methods are presented. These include the use of the flow coordinates to provide preferred directions for edge detection, detection of anomalies, fitting simple models to the straightened pattern, and detecting singularities in the flow field.

Motion warping

We describe a simple technique for editing captured or keyframed animation based on warping of the motion parameter curves. The animator interactively defines a set of keyframe-like constraints which are used to derive a smooth deformation that preserves the fine structure of the original motion. Motion clips are combined by overlapping and blending of the parameter curves. We show that whole families of realistic motions can be derived from a single captured motion sequence using only a few keyframes to specify the motion warp. Our technique makes it feasible to create libraries of reusable “clip motion.”

Recovering surface shape and orientation from texture

Texture provides an important source of information about the three-dimensional structure of visible surfaces, particularly for stationary monocular views. To recover 3d structure, the distorting effects of projection must be distinguished from properties of the texture on which the distortion acts. This requires that assumptions must be made about the texture, yet the unpredictability of natural textures precludes the use of highly restrictive assumptions. The recovery method reported in this paper exploits the minimal assumption that textures do not mimic projective effects. This assumption determines the strategy of attributing as much as possible of the variation observed in the image to projection. Equivalently, the interpretation is chosen for which the texture, prior to projection, is made as uniform as possible. This strategy was implemented using statistical methods, first for the restricted case of planar surfaces and then, by extension, for curved surfaces. The technique was applied successfully to natural images.

Physically based motion transformation

We introduce a novel algorithm for transforming character animation sequences that preserves essential physical properties of the motion. By using the spacetime constraints dynamics formulation our algorithm maintains realism of the original motion sequence without sacrificing full user control of the editing process. In contrast to most physically based animation techniques that synthesize motion from scratch, we take the approach of motion transformationas the underlying paradigm for generating computer animations. In doing so, we combine the expressive richness of an input animation sequence with the controllability of spacetime optimization to create a wide range of realistic character animations. The spacetime dynamics formulation also allows editing of intuitive, high-level motion concepts such as the time and placement of footprints, length and mass of various extremities, number of body joints and gravity. Our algorithm is well suited for the reuse of highly-detailed captured motion animations. In addition, we describe a new methodology for mapping a motion between characters with drastically different numbers of degrees of freedom. We use this method to reduce the complexity of the spacetime optimization problems. Furthermore, our approach provides a paradigm for controlling complex dynamic and kinematic systems with simpler ones.

Using particles to sample and control implicit surfaces

We present a new particle-based approach to sampling and controlling implicit surfaces. A simple constraint locks a set of particles onto a surface while the particles and the surface move. We use the constraint to make surfaces follow particles, and to make particles follow surfaces. We implement control points for direct manipulation by specifying particle motions, then solving for surface motion that maintains the constraint. For sampling and rendering, we run the constraint in the other direction, creating floater particles that roam freely over the surface. Local repulsion is used to make floaters spread evenly across the surface. By varying the radius of repulsion adaptively, and fissioning or killing particles based on the local density, we can achieve good sampling distributions very rapidly, and maintain them even in the face of rapid and extreme deformations and changes in surface topology.We present a new particle-based approach to sampling and controlling implicit surfaces. A simple constraint locks a set of particles onto a surface while the particles and the surface move. We use the constraint to make surfaces follow particles, and to make particles follow surfaces. We implement control points for direct manipulation by specifying particle motions, then solving for surface motion that maintains the constraint. For sampling and rendering, we run the constraint in the order direction, creating floater particles that roam freely over the surface. Local repulsion is used to make floaters spread evenly across the surface. By varying the radius of repulsion adaptively, and fissioning or killing particles based on the local density, we can achieve good sampling distributions very rapidly, and maintain them even in the face of rapid and extreme deformations and changes in surface topology.

Variational surface modeling

We present a new approach to interactive modeling of freeform surfaces. Instead of a fixed mesh of control points, the model presented to the user is that of an infinitely malleable surface, with no fixed controls. The user is free to apply control points and curves which are then available as handles for direct manipulation. The complexity of the surface’s shape may be increased by adding more control points and curves, without apparent limit. Within the constraints imposed by the controls, the shape of the surface is fully determined by one or more simple criteria, such as smoothness. Our method for solving the resulting constrained variational optimization problems rests on a surface representation scheme allowing nonuniform subdivision of B-spline surfaces. Automatic subdivision is used to ensure that constraints are met, and to enforce error bounds. Efficient numerical solutions are obtained by exploiting linearities in the problem formulation and the representation.

On the Role of Structure in Vision

Abstract People are able to perceive structure in images, apart from the perception of tri-dimensionality, and apart from the recognition of familiar objects. We impose organization on data (noticing flow fields, regularity, repetition, etc.) even when we have no idea what it is we are organizing. What is remarkable is the degree to which such naively perceived structure survives more or less intact once a semantic context is established: the naive observer often sees essentially the same things an expert does, the difference between naive and informed perception often amounting to little more than labeling the perceptual primitives. It is almost as if the visual system has some basis for guessing what is important without knowing why. Our objective in this paper is to understand the role of primitive structure in visual perception. What does this level of organization mean? What, if anything, is it good for? If it is useful, how do we get it and, once gotten, use it? We will argue that perceptual organization is a primitive level of inference, the basis for which lies in the relation between structural and causal unity: the appearance of spatiotemporal coherence or regularity is so unlikely to arise by the chance interaction of independent entities that such regular structure, when observed, almost certainly denotes some underlying unified cause or process. This view will be shown to have broad implications for computational theories of vision, providing a unifying framework for many current techniques of early and intermediate vision, and enabling a style of interpretation more in keeping with the qualitative and holistic character of human vision.