Hongche Liu

A General Motion Model and Spatio-Temporal Filters forComputing Optical Flow

Traditional optical flow algorithms assume local image translational motion and apply simple image filtering techniques. Recent studies have taken two separate approaches toward improving the accuracy of computed flow: the application of spatio-temporal filtering schemes and the use of advanced motion models such as the affine model. Each has achieved some improvement over traditional algorithms in specialized situations but the computation of accurate optical flow for general motion has been elusive. In this paper, we exploit the interdependency between these two approaches and propose a unified approach. The general motion model we adopt characterizes arbitrary 3-D steady motion. Under perspective projection, we derive an image motion equation that describes the spatio-temporal relation of gray-scale intensity in an image sequence, thus making the utilization of 3-D filtering possible. However, to accommodate this motion model, we need to extend the filter design to derive additional motion constraint equations. Using Hermite polynomials, we design differentiation filters, whose orthogonality and Gaussian derivative properties insure numerical stability; a recursive relation facilitates application of the general nonlinear motion model while separability promotes efficiency. The resulting algorithm produces accurate optical flow and other useful motion parameters. It is evaluated quantitatively using the scheme established by Barron et al. (1994) and qualitatively with real images.

Motion-model-based boundary extraction and a real-time implementation

Motion boundary extraction and optical flow computation are two subproblems of the motion recovery problem that cannot be solved independently of one another. These two problems have been treated separately. A popular recent approach uses an iterative scheme that consists of motion boundary extraction and optical flow computation components and refines each result through iteration. We present a local, noniterative algorithm that simultaneously extracts motion boundaries and computes optical flow. This is achieved by modeling 3-D Hermite polynomial decompositions of image sequences representing the perspective projection of 3-D general motion. Local model parameters are used to determine whether motion should be estimated or motion boundaries should be extracted at the neighborhood. A definite advantage of this noniterative algorithm is its efficiency. It is demonstrated by a real-time implementation and supporting experimental results.

A generalized motion model for estimating optical flow using 3-D Hermite polynomials

Classic optical flow algorithms assume local image translational motion and apply some primitive image smoothing. Recent studies have taken two separate approaches toward improving accuracy: the application of spatio-temporal filtering schemes and the use of generalized motion models such as the affine model. Each has achieved improvement in its specialized situations. We analyze the interdependency between them and propose a unified theory. The generalized motion we adopt models arbitrary 3D steady motion. Under perspective projection, we derive an image motion equation that describes the spatio-temporal relation in an image sequence, thus making 3D spatio-temporal filtering possible. Hence we establish a theory of Hermite polynomial differentiation filters, whose orthogonality and Gaussian derivative properties ensure numerical stability. The use of higher order motion constraint equations to accommodate more complex motion is justified by the algorithms reliable performance, as demonstrated by evaluating our algorithm in the scheme established by Barron, et al. (1994).

Motion-model-based boundary extraction

Motion boundary extraction and optical flow computation are two subproblems of the motion recovery problem that cannot be solved independently of one another. We present a local, non-iterative algorithm that extracts motion boundaries and computes optical flow simultaneously. This is achieved by modeling a 3-D image intensity block with a general motion model that presumes locally coherent motion. Local motion coherence, which is measured by the fitness of the motion model, is the criterion we use to determine whether motion should be estimated. If not, then motion boundaries should be located. The motion boundary extraction algorithm is evaluated quantitatively and qualitatively against other existing algorithms in a scheme originally developed for edge detection.

Spatio-temporal filters for transparent motion segmentation

An image is ideally a projection of the 3-D scene. However the imaging process is always imperfect and constrained by the physical environment, for example, viewing through a window with reflections. This paper is concerned with image sequences acquired in such situations, the so-called transparency. When it occurs, the image sequence contains undesirable transparent motion, for example, of the window reflections. This complicates the already difficult motion estimation problem. We present an algorithm to segment transparent motion based on a spatio-temporal filtering technique-3-D Hermite polynomial differentiation filters. With motion segmentation accomplished, we can then focus on the scene analysis. The implementation of our algorithm is fast and accurate.

Image gradient evolution-a visual cue for collision avoidance

This paper is concerned with the task of visual motion-based navigation. A critical requirement of the task is the ability to estimate 3D depth and motion from visual information. Recent studies have demonstrated that the relevant cues are contained in motion parallax or optical flow from which flow field divergence and hence time-to-contact can be extracted. We present a new concept called image gradient evolution, which utilizes the change of image spatial gradients over time as a threat cue; an approaching object induces 2D expanding motion and causes the image spatial structure to stretch so the image gradients decrease. Based on this idea, other method offers a one-step solution directly from image gradients, instead of optical flow and its derived properties. We use a technique that is local and linear so the implementation can be very fast. The threat map is expectedly noisy but sufficiently informative. As seen in demonstrations on several real images these two aspects, fast implementation and useful qualitative information, provide a viable solution to navigational tasks.

A real-time computer vision platform for mobile robot applications

Abstract A portable platform is described that supports real-time computer vision applications for mobile robots. This platform includes conventional processors, an image processing front-end system, and a controller for a pan/tilt/vergence head. The platform is ruggedized to withstand vibration during off-road driving. The platform has successfully supported experiments in video stabilization and detection of moving objects for outdoor surveillance, gradient-based and correlation-based image flow estimators, and indoor mobility using divergence of flow. These applications have been able to run at rates ranging from 3 to 15 Hz for image sizes from 64 × 64 to 256 × 256.