Allen M. Waxman

A visual navigation system for autonomous land vehicles

A modular system architecture has been developed to support visual navigation by an autonomous land vehicle. The system consists of vision modules performing image processing, three-dimensional shape recovery, and geometric reasoning, as well as modules for planning, navigating, and piloting. The system runs in two distinct modes, bootstrap and feedforward. The bootstrap mode requires analysis of entire images to find and model the objects of interest in the scene (e.g., roads). In the feedforward mode (while the vehicle is moving), attention is focused on small parts of the visual field as determined by prior views of the scene, to continue to track and model the objects of interest. General navigational tasks are decomposed into three categories, all of which contribute to planning a vehicle path. They are called long-, intermediate-, and short-range navigation, reflecting the scale to which they apply. The system has been implemented as a set of concurrent communicating modules and used to drive a camera (carried by a robot arm) over a scale model road network on a terrain board. A large subset of the system has been reimplemented on a VICOM image processor and has driven the DARPA Autonomous Land Vehicle (ALV) at Martin Mariettas test site in Denver, CO.

Binocular Image Flows: Steps Toward Stereo-Motion Fusion

The analyses of visual data by stereo and motion modules have typically been treated as separate parallel processes which both feed a common viewer-centered 2.5-D sketch of the scene. When acting separately, stereo and motion analyses are subject to certain inherent difficulties; stereo must resolve a combinatorial correspondence problem and is further complicated by the presence of occluding boundaries, motion analysis involves the solution of nonlinear equations and yields a 3-D interpretation specified up to an undetermined scale factor. A new module is described here which unifies stereo and motion analysis in a manner in which each helps to overcome the others short-comings. One important result is a correlation between relative image flow (i.e., binocular difference flow) and stereo disparity; it points to the importance of the ratio ¿ ¿, rate of change of disparity ¿ to disparity ¿, and its possible role in establishing stereo correspondence. The importance of such ratios was first pointed out by Richards [19]. Our formulation may reflect the human perception channel probed by Regan and Beverley [18].

Closed form solutions to image flow equations for planar surfaces in motion

Abstract In this paper we consider the recovery of the 3-dimensional motion and orientation of a rigid planar surface from its image flow field. Closed form solutions are derived for the image flow equations formulated by Waxman and Ullman [1]. Also we give two important results relating to the uniqueness of solutions for the image flow equations. The first result concerns resolving the duality of interpretations that are generally associated with the instantaneous image flow of an evolving image sequence. It is shown that the interpretation for orientation and motion of planar surfaces is unique when either two successive image flows of one planar surface patch are given or one image flow of two planar patches moving as a rigid body is given. We have proved this by deriving explicit expressions for the evolving solution of an image flow sequence with time. These expressions can be used to resolve this ambiguity of interpretation in practical problems. The second result is the proof of uniqueness for the velocity of approach which satisfies the image flow equations for planar surfaces derived in [1]. In addition, it is shown that this velocity can be computed as the middle root of a cubic equation. These two results together suggest a new method for solving the image flow problem for planar surfaces in motion. We also describe a scheme to use first-order time derivatives of the image flow field in place of the second-order spatial derivatives to solve for the orientation and motion.

Spreading activation layers, visual saccades, and invariant representations for neural pattern recognition systems

Abstract This paper shows how a simple spreading activation network in the form of two-dimensional (2D) diffusion followed by local maximum detection can quickly perform a large number of early vision tasks, for example, feature extraction, feature clustering, feature-centroid determination, and boundary gap completion, all on multiple scales. The results of the spreading activation process can be used to facilitate 2D object learning and recognition from silhouettes by generating representations from bottom-up fixation cues which are invariant to translation, orientation, and scale. In addition, the proposed network suggests a possible approach to longrange apparent motion correspondence and multiscale object decomposition. The theory is described and implementation examples are presented.

Information fusion for image analysis: geospatial foundations for higher-level fusion

In support of the AFOSR program in Information Fusion, the CNS Technology Laboratory at Boston University is developing and applying neural models of image and signal processing, pattern learning and recognition, associative learning dynamics, and 3D visualization, to the domain of Information Fusion for Image Analysis in a geospatial context. Our research is focused by a challenge problem involving the emergence of a crisis in an urban environment, brought on by a terrorist attack or other man-made or natural disaster. We aim to develop methods aiding preparation and monitoring of the battlespace, deriving context from multiple sources of imagery (high-resolution visible and low-resolution hyperspectral) and signals (GMTI from moving vehicles, and ELINT from emitters). This context will serve as a foundation, in conjunction with existing knowledge nets, for exploring neural methods in higher level information fusion supporting situation assessment and creation of a common operating picture (COP).

Convected activation profiles and the measurement of visual motion

A method is developed for the measurement of short-range visual motion in image sequences, making use of the motion of image features such as edges and points. Each feature generates a Gaussian activation profile in a spatiotemporal neighborhood of specified scale around the feature itself; this profile is then convected with motion of the feature. The authors show that image velocity estimates can be obtained from such dynamic activation profiles using a modification of familiar gradient techniques. The resulting estimators can be formulated in terms of simple ratios of spatiotemporal filters (i.e. receptive fields) convolved with image feature maps. A family of activation profiles of varying scale must be utilized to cover a range of possible image velocities. They suggest a characteristic speed normalization of the estimate obtained from each filter in order to decide which estimate is to be accepted. They formulate the velocity estimators for dynamic edges in 1-D and 2-D image sequences, as well as that for dynamic feature points in 2-D image sequences.<<ETX>>

Progress on the prototype PIPE

This paper summarizes our first experience with the prototype Pipelined Image Processing Engine (PIPE) at the National Bureau of Standards during the Summer of 1986. The PIPE machine is a multistage, pipelined configuration of video-rate hardware with a fairly flexible interconnect structure which can be dynamically reconfigured at TV-field rate (60 fields/sec). It is essentially a special-purpose integer processor, tailored to the needs of early vision computations. As the PIPE is a synchronized MIMD machine, the design and programming of algorithms is best done in the form of Data-Flow Graphs which are mapped onto Space-Time Diagrams of the machine. It took us roughly three days to learn to design, implement and debug a program for the PIPE. We have created a variety of routines for real-time image processing; examples are described here, along with algorithm diagrams, latency and update rates.

The analytic structure of image flows: deformation and segmentation

Abstract Time-varying imagery is often described in terms of image flow fields (i.e., image motion), which correspond to the perceptive projection of feature motions in three dimensions (3D). In the case of multiple moving objects with smooth surfaces, the image flow possesses an analytic structure that reflects these 3D properties. This paper describes the analytic structure of image flow fields in the image space-time domain, and its use for segmentation and 3D motion computation. First we discuss thelocal flow structure as embodied in the concept ofneighborhood deformation. The local image deformation is effectively represented by a set of 12 basis deformations, each of which is responsible for an independent deformation. This local representation provides us with sufficient information for the recovery of 3D object structure and motion, in the case of relative rigid body motions. We next discuss theglobal flow structure embodied in the partitioning of the entire image plane intoanalytic regions separated byboundaries of analyticity, such that each small neighborhood within the analytic region is described in terms of deformation bases. This analysis reveals an effective mechanism for detecting the analytic boundaries of flow fields, thereby segmenting the image into meaningful regions. The notion ofconsistency which is often used in the image segmentation is made explicit by the mathematical notion ofanalyticity derived from the projection relation of 3D object motion. The concept of flow analyticity is then extended to the temporal domain, suggesting a more robust algorithm for recovering image flow from multiple frames. Finally, we argue that the process of flow segmentation can be understood in the framework of grouping process. The general concept ofcoherence orgrouping through local support (such as the second-order flows in our case) is discussed.

A visual navigation system

A modular system architecture has been developed to support visual navigation by an autonomous land vehicle. The system consists of vision modules performing image processing, 3-D shape recovery, and rule-based reasoning, as well as modules for planning, navigating and piloting. The system runs in two distinct modes, bootstrap and feed-forward. The bootstrap mode requires analysis of entire images in order to find and model the objects of interest in the scene (e.g. roads). In the feed-forward mode (while the vehicle is moving), attention is focused on small parts of the visual field as determined by prior views of the scene, in order to continue to track and model the objects of interest. We have decomposed general navigational tasks into three categories, all of which contribute to planning a vehicle path. They are called long, intermediate and short range navigation, reflecting the scale to which they apply. We have implemented the system as a set of concurrent, communicating modules and use it to drive a camera (carried by a robot arm) over a scale model road network on a terrain board.

Computing visual motion in the short and the long: from receptive fields to neural networks

Theoretical approaches to the study of perceptual phenomena of short-range and long-range apparent motion are discussed. Short-range motion is estimated by real-time receptive fields sensitive to velocities of image features. The design of these receptive fields follows from the concept of convected activation profiles, where shape-preserving activity waves are excited by, and ride atop, dynamic features. Long-range motion concerns the correspondence between features in disparate images, and the perceptual impletion of a path between these corresponding features. It is argued that this illusory motion is an artifact of a more general spatiotemporal grouping process. This process is realized in a dynamic nonlinear feedback neural network that is called a neural analog diffusion-enhancement layer (NADEL). Computations suggest that the NADEL can support a variety of long-range motion percepts popularized by the Gestalt psychologists. The authors illustrate (on videotape) both classes of computation in real time on the PIPE parallel computer.<<ETX>>