Bradley Horowitz

Visually controlled graphics

Interactive graphics systems that are driven by visual input are discussed. The underlying computer vision techniques and a theoretical formulation that addresses issues of accuracy, computational efficiency, and compensation for display latency are presented. Experimental results quantitatively compare the accuracy of the visual technique with traditional sensing. An extension to the basic technique to include structure recovery is discussed. >

Recovery of nonrigid motion and structure

The authors introduce a physically correct model of elastic nonrigid motion. This model is based on the finite element method, but decouples the degrees of freedom by breaking down object motion into rigid and nonrigid vibration or deformation modes. The result is an accurate representation for both rigid and nonrigid motion that has greatly reduced dimensionality, capturing the intuition that nonrigid motion is normally coherent and not chaotic. Because of the small number of parameters involved, this representation is used to obtain accurate overstrained estimates of both rigid and nonrigid global motion. It is also shown that these estimates can be integrated over time by use of an extended Kalman filter, resulting in stable and accurate estimates of both three-dimensional shape and three-dimensional velocity. The formulation is then extended to include constrained nonrigid motion. Examples of tracking single nonrigid objects and multiple constrained objects are presented. >

Recovery of non-rigid motion and structure

The elastic properties of real materials provide constraint on the types of non-rigid motion that can occur, and thus allow overconstrained estimates of 3-D non-rigid motion from optical flow data. It is shown that by modeling and simulating the physics of non-rigid motion it is possible to obtain good estimates of both object shape and velocity. Examples using grey-scale and X-ray imagery are presented, including an example of tracking a complex articulated figure.<<ETX>>

Recursive estimation of structure and motion using relative orientation constraints

A recursive estimation technique for recovering the 3-D motion and pointwise structure of an object is presented. It is based on the use of relative orientation constraints in a local coordinate frame. By carefully formulating the problem to propagate all constraints and to use the minimal number of parameters, an estimator is obtained which is remarkably accurate, stable, and fast-conveying. Numerous experiments using both real and synthetic data demonstrate structure recovery with a typical error of 1.5% and typical motion recovery errors of 1% in translation and 2/spl deg/ in rotation.<<ETX>>

A practical approach to fractal-based image compression

A technique for image compression is based on a very simple type of iterative fractal. A wavelet transform (quadrature mirror filter pyramid) is used to decompose an image into bands containing information from different scales (spatial frequencies) and orientations. The conditional probabilities between these different scale bands are then determined, and used as the basis for a predictive coder. The wavelet transforms various scale and orientation bands have a great deal of redundant, self-similar structure in the form of multi-modal conditional probabilities, so that linear predictors perform poorly. A simple histogram method is used to determine the multi-modal conditional probabilities between scales. The resulting predictive coder is easily integrated into existing subband coding schemes.<<ETX>>

Non-rigid motion and structure from contour

The elastic properties of real materials constrain the types of non-rigid motion that can occur. The authors show that by modeling and simulating the physics of non-rigid motion they can obtain good estimates of both object shape and velocity from contour data. The underlying model is based on the finite element method, and decouples the degrees of freedom by breaking down object motion into rigid and non-rigid vibration modes. Shape estimates are integrated over time by use of an extended Kalman filter, resulting in a stable estimate of both 3D shape and 3D velocity. Examples using X-ray imagery (of the heat) are presented.<<ETX>>

Recursive estimation for CAD model recovery

We describe a system for semiautomatically extracting 3-D object models from raw, uncalibrated video. The system utilizes a recursive estimator to accurately recover camera motion, point-wise structure, and camera focal length. Recovered 3-D points are used to compute a piecewise-smooth surface model for the object. Recovered motion and camera geometry are then used along with the original video to texture map the surfaces. We describe extensions to our previously-reported geometry estimation formulation that incorporate focal length estimation and other improvements, so that accurate estimates of structure and camera motion can be recovered from uncalibrated video cameras. We also discuss the buildup of texture maps from sequences of images, which is important in producing realistic looking models. Examples demonstrate generation of a realistic 3-D texture mapped model from a video sequence, the post-production manipulation of video, and the combination of computer graphics models with video.<<ETX>>

Practical approach to fractal-based image compression

Fractal techniques for image compression have recently attracted a great deal of attention. Unfortunately, little in the way of practical algorithms or techniques have been published. We present a technique for image compression that is based on a very simple type of iterative fractal. In our algorithm a wavelet transform (quadrature mirror filter pyramid) is used to decompose an image into bands containing information from different scales (spatial frequencies) and orientations. The conditional probabilities between these different scale bands are then determined, and used as the basis for a predictive coder. We find that the wavelet transforms various scale and orientation bands have a great deal of redundant, self-similar structure. This redundant structure is, however, in the form of multi-modal conditional probabilities, so that linear predictors perform poorly. Our algorithm uses a simple histogram method to determine the multi-modal conditional probabilities between scales. The resulting predictive coder is easily integrated into existing subband coding schemes. Comparison of this fractal-based scheme with our standard wavelet vector coder on 256 X 256 grey-level imagery shows up to a two-fold gain in coding efficiency with no loss in image quality, and up to a four-fold gain with small loss in image quality. Coding and decoding are implemented by small table lookups, making real-time application feasible.