James R. Bergen

Spatiotemporal energy models for the perception of motion

A motion sequence may be represented as a single pattern in x-y-t space; a velocity of motion corresponds to a three-dimensional orientation in this space. Motion sinformation can be extracted by a system that responds to the oriented spatiotemporal energy. We discuss a class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency. The outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy. These responses are then fed into an opponent stage. Energy models can be built from elements that are consistent with known physiology and psychophysics, and they permit a qualitative understanding of a variety of motion phenomena.

Visual odometry for ground vehicle applications

We present a system that estimates the motion of a stereo head, or a single moving camera, based on video input. The system operates in real time with low delay, and the motion estimates are used for navigational purposes. The front end of the system is a feature tracker. Point features are matched between pairs of frames and linked into image trajectories at video rate. Robust estimates of the camera motion are then produced from the feature tracks using a geometric hypothesize-and-test architecture. This generates motion estimates from visual input alone. No prior knowledge of the scene or the motion is necessary. The visual estimates can also be used in conjunction with information from other sources, such as a global positioning system, inertia sensors, wheel encoders, etc. The pose estimation method has been applied successfully to video from aerial, automotive, and handheld platforms. We focus on results obtained with a stereo head mounted on an autonomous ground vehicle. We give examples of camera trajectories estimated in real time purely from images over previously unseen distances (600 m) and periods of time.

Parallel versus serial processing in rapid pattern discrimination

When stimuli are available for just a brief period (∼100 ms) only restricted spatial information can be processed by the visual system. If the stimuli are presented very briefly, eye movements are not possible. The time during which the after-image of the stimulus is available for inspection is terminated by presentation of a masking pattern. We show here that in these conditions a small pattern is easily detected against a background made up of many others, only if this target pattern differs from the background patterns in certain local features. In this case the detectability of the target is almost independent of the number of background elements, suggesting that a parallel process is operating. Detection of patterns not differing from their backgrounds in such features requires focal attention which is a serial process. The aperture of this attention is scaled to minimize the number of shifts of attention required.

Texture segregation and orientation gradient

Rapid texture segregation is examined using filtered noise textures. The stimuli consist of a foreground region of filtered noise with one dominant texture orientation against a background region with a different dominant orientation. Shape discrimination of the foreground region is measured as a function of the difference in orientation between the two regions (delta theta), the distance over which the dominant orientation rotates from the background to the foreground value (delta chi), and the dominant spatial frequency of the textures (f). Performance declines with smaller delta theta, larger delta chi, and lower f. These effects are partially independent of viewing distance, which implies that it is the relative or object spatial frequency, not retinal spatial frequency, which determines performance in this task. We present a model consisting of channels tuned for orientation and spatial frequency which compute local oriented energy, followed by (texture) edge detection and a cross-correlator which performs the shape discrimination. Monte Carlo simulations of this model are in accord with the degradation in performance with increased delta chi and decreased delta theta.

A three-frame algorithm for estimating two-component image motion

A fundamental assumption made in formulating optical-flow algorithms, that motion at any point in an image can be represented as a single pattern component undergoing a simple translation, fails for a number of situations that commonly occur in real-world images. An alternative formulation of the local motion assumption in which there may be two distinct patterns undergoing coherent (e.g. affine) motion within a given local analysis region is proposed. An algorithm for the analysis of two-component motion in which tracking and nulling mechanisms applied to three consecutive image frames separate and estimate the individual components is given. Precise results are obtained, even for components that differ only slightly in velocity as well as for a faint component in the presence of a dominant, masking component. The algorithm provides precise motion estimates for a set of elementary two-motion configurations and is robust in the presence of noise. >

Directionally selective complex cells and the computation of motion energy in cat visual cortex

We applied a set of 1- and 2-bar tests to directionally selective (DS) complex cells in the cats striate cortex, and compared the responses with those predicted by two computational models. Single-bar responses and 2-bar interactions produce distinctive patterns that are highly diagnostic. The observed responses are quite similar to those predicted by a basic (non-opponent) motion-energy model [Adelson & Bergen (1985) Journal of the Optical Society of America A, 2, 284-299]. However, they are not consistent with an opponent combination of energy models, nor are they consistent with any stage of the classic Reichardt model. In particular, the Reichardt model (as well as opponent combinations of energy models) predicts a separable space-time symmetry in the 2-bar interaction that is not observed in our measurements, while the non-opponent energy model predicts an inseparable, oriented interaction very similar to the measured cortical responses. Comparisons between model and measurements suggest possible mechanisms of spatial receptive-field organization and of nonlinear transformations.

Computing two motions from three frames

A fundamental assumption made in formulating optical-flow algorithms is that motion at any point in any image can be represented as a single pattern undergoing a simple translation: even complex motion will appear as a uniform displacement when viewed through a sufficiently small window. This assumption fails in a number of common situations. The authors propose an alternative formulation in which there may be two distinct patterns undergoing coherent motion within a given local analysis region. They then present an algorithm for the analysis of two-component motion. They also demonstrate that the algorithm provides precise motion estimates for a set of elementary two-motion configurations, and show that it is robust in the presence of noise.<<ETX>>

A probabilistic algorithm for computing Hough transforms

Abstract The Hough transform is a common technique in computer vision and pattern recognition for recognizing patterns of points. We describe an efficient probabilistic algorithm for a Monte-Carlo approximation to the Hough transform. Our algorithm requires substantially less computation and storage than the standard Hough transform when applied to patterns that are easily recognized by humans. The probabilistic steps involve randomly choosing small subsets of points that jointly vote for likely patterns.

Video as an image data source: efficient representations and applications

The two fundamental advantages of video over still imagery are: (i) the ability capture temporal information, and (ii) the ability to acquire a continuously varying set of views of a scene. These advantages are obtained, however, at the cost of vastly increased amount of data. This paper describes an approach to video representation that is based on frame-to-frame alignment, mosaic construction, and 3D parallax recovery. The basic motivation behind our approach is to enable rapid access to the contents, while maintaining the data in a form as close to the source as possible. This representation supports a wide variety of applications that involve transmission, storage, visualization, retrieval, analysis, and manipulation of video sequences.

Transparent-motion analysis

A fundamental assumption made in formulating optical flow algorithms is that motion at any point in an image can be represented as a single pattern component undergoing a simple translation: even complex motion will ‘look like’ uniform displacement when viewed through a sufficiently small window. This assumption fails for a number of situations that commonly occur in real world images. For example, transparent surfaces moving past one another yield multiple motion components at a point.