Peter J. Sobey

MEASUREMENT OF OPTICAL-FLOW BY A GENERALIZED GRADIENT SCHEME

We describe a procedure for recovering the global, two-dimensional velocity of translation of an image by incorporating spatial filtering, and, optionally, temporal filtering, into a scheme that employs a novel and generalized version of the gradient algorithm of motion detection. Motion within a patch is analyzed in parallel by six different spatiotemporal filters derived from two linearly independent spatiotemporal kernels. Advantageous features of this scheme are that (a) the average velocity within the patch is determined in a single step and without recourse to constraints imposed by neighboring calculations or assumptions about the global structure of the pattern; (b) there is no need to impose a smoothness constraint on the optical flow; (c) the need to compute spatial derivatives directly is obviated by our combining the outputs of the kernel filters with the outputs of other filters whose weighting functions are partial derivatives, in space, with respect to the first set; (d) there is no need to compute second derivatives, and thus the scheme is potentially more resistant to noise than certain other schemes; (e) the spatiotemporal kernels can be chosen almost completely arbitrarily and can therefore be tailored to maximize signal reliability; and (f) the measurement of velocity can be made as local or as global as desired by altering the size of the patch that is viewed by the filters. The validity of the scheme is demonstrated on a computer by application to a variety of real, moving images.

Implementation of the Template Model of Vision

Adopting principles learnt from insect vision we have constructed a model of a general-purpose front-end visual system for motion detection that is designed to operate in parallel along each photoreceptor axis with only local connections. The model is also designed to assist electrophysiological analysis of visual processing because it puts the response to a moving scene into sets of template responses similar to the distribution of activity among different neurons. An earlier template model divided the visual image into the fields of adjacent receptors, measured as intensity or receptor modulation at small increments of time. As soon as we used this model with natural scenes, however, we found that we had to look at changes in intensity, not intensity itself. Running the new model also generated new insights into the effects of very fast motion, of blurring the image, and the value of lateral inhibition. We also experimented with ways of measuring the angular velocity of the image moving across the eye. The camera eye is moved at a known speed and the range to objects is calculated from the angular velocity of contrasts moving across the receptor array. The original template model is modified so that contrast is saturated in a new representation of the original image data. This reduces the 8-bit grey-scale image to a log2 3 = 1.6-bit image, which becomes the input to a look-up table of templates. The output consists of groups of responding templates in specific ratios that define the input features, and these ratios lead into types of invariance at a higher level of further logic. At any stage, there can be persistent parallel inputs from all earlier stages. This design would enable groups of templates to be tuned to different expected situations, such as different velocities, different directions and different types of edges.

Automated optical grading of timber

Australia grows large quantities of radiata pine for domestic consumption and a significant proportion of this is graded to an Australian Standard Appearance Grade. This paper describes automating the visual inspection of this timber in order to speed processing and improve quality control of the product. The requirement is to detect and identify the visual features on the surface of the timber after the surface has been dressed. These features include sound and encased knots of various sizes pith bark bluestain holes and wane. The image is captured using a linear array CCD camera as the board moves underneath on a conveyor belt. The first stage is to detect the areas that contain features. The image is divided into smaller local areas and first and second order statistical measures are calculated. These form the input to a neural network that has been trained to classify the local areas into clear and feature areas. The choice of measures is crucial to the ability of the neural network to perform the classification of local areas. The second stage is to determine the type of feature in the feature local areas. Various methods are employed to determine a threshold that segments the feature correctly. The size of the feature can be used to identify it uniquely. The list of features and their positions forms the input to the grading program. The grading rules defmed

Determining range information from self-motion: the template model

Insects use a relatively simple visual system to navigate and avoid obstacles. In particular they use self motion to determine the range to objects by the angular velocities of the contrasts across the retina array. Adopting principles learnt from studying insect behaviour and neurophysiology we have modelled aspects of the motion detection mechanism of an insect visual system into a means of categorising edges and computing their motion and thus determining range. Copying insect motion perception a camera is scanned across a scene and a temporal sequence of line images captured. The 8-bit grey scale image is immediately reduced to a 1og23 1. 6 bit image by saturating the contrast. Behind each pixel one state is formed by increasing intensity one by decreasing intensity and a third is indeterminate. Pairs of receptors at two consecutive times forming a 2 by 2 template in space-time give a finite number of combinations of which it is found that only a small subset provide useful motion information. Combinations of selected templates results in a distribution of template responses that is amenable to analysis by the Hough transform. Running the model on real scenes reveals the value of lateral inhibition as well as insights into the effect of different edge types and the use of parallax. The model suggests a possible new neurophysiological construction that can be copied in hardware to provide a fast means inferring 3-d structure in a scene where the observer is moving with a known velocity. 1.

Robust depth extraction for mobile robots

Stereoscopic techniques for recovering depth in scenes are computationally intensive and difficult to specify sufficiently well to ensure that optimal solutions are obtained in any given situation. Apparent motion cues are a far richer and more easily exploitable source of information on depth, but computing depth from motion in spatio-temporal image sequences has many pit-falls associated with it. We show that many of these can be avoided by simultaneous capture of two or more views of the scene, projected onto a single CCD sensor, using angled mirrors. The resulting fixed-camera ranging device is immune to camera vibration and motion as well as to changes in ambient illumination that occur during image capture. A 1-D, generalized gradient scheme is used to compute the apparent image motion induced by objects in the scene and hence the range to the corresponding objects. Furthermore, the fixed camera configuration enables the shape and size of the viewing filter to be preselected to optimize performance and maximize range resolution.

Hardware system for computing image velocity in real time

A multi-purpose hardware system for processing images at video rates is described. Image sequence hardware for temporal analysis in realtime (ISHTAR) uses 18 TI TMS320c40 (c40) DSPs to process input from a CCD camera or VCR source. The hardware architecture consists of a pipeline of nine processor boards, each with two c40 processors, the whole system being synchronized by the vertical sync of the input device. This enables the calculation of a number of two dimensional convolutions to be achieved at video frame rates with a delay between the input and the output dictated by the length of the pipeline. The system is fully reconfigurable in software and partially reconfigurable in hardware so that many different types of image processing algorithms can be implemented. The specific application of a generalized gradient model to measure image motion is described, outlining the particular program structure dictated by the hardware design. The SUN 4 host has access to each processor and has the ability to change parameters and program control while the system is running. In this way active control feedback loops can be employed, particularly when the motion of the camera is under the host control, forming an active vision system. Simulations using real image sequences are presented.

Insect vision as model for machine vision

The neural architecture, neurophysiology and behavioral abilities of insect vision are described, and compared with that of mammals. Insects have a hardwired neural architecture of highly differentiated neurons, quite different from the cerebral cortex, yet their behavioral abilities are in important respects similar to those of mammals. These observations challenge the view that the key to the power of biological neural computation is distributed processing by a plastic, highly interconnected, network of individually undifferentiated and unreliable neurons that has been a dominant picture of biological computation since Pitts and McCullochs seminal work in the 1940s.