Mike J. Chantler

Robust tracking of multiple objects in sector-scan sonar image sequences using optical flow motion estimation

The fast update rate and good performance of new generation electronic sector scanning sonars is now allowing practicable use of temporal information for signal processing tasks such as object classification and motion estimation. Problems remain, however, as objects change appearance, merge, maneuver, move in and out of the field of view, and split due to poor segmentation. This paper presents an approach to the segmentation, two-dimensional motion estimation, and subsequent tracking of multiple objects in sequences of sector scan sonar images. Applications such as ROV obstacle avoidance, visual servoing, and underwater surveillance are relevant. Initially, static and moving objects are distinguished in the sonar image sequence using frequency-domain filtering. Optical flow calculations are then performed on moving objects with significant size to obtain magnitude and direction motion estimates. Matches of these motion estimates, and the future positions they predict, are then used as a basis for identifying corresponding objects in adjacent scans. To enhance robustness, a tracking tree is constructed storing multiple possible correspondences and cumulative confidence values obtained from successive compatibility measures. Deferred decision making is then employed to enable best estimates of object tracks to be updated as subsequent scans produce new information. The method is shown to work well, with good tracking performance when objects merge, split, and change shape. The optical flow is demonstrated to give position prediction errors of between 10 and 50 cm (1%-5% of scan range), with no violation of smoothness assumptions using sample rates between 4 and 1 frames/s.

Capture and Synthesis of 3D Surface Texture

We present and compare five approaches for capturing, synthesising and relighting real 3D surface textures. Unlike 2D texture synthesis techniques they allow the captured textures to be relit using illumination conditions that differ from those of the original. We adapted a texture quilting method due to Efros and combined this with five different relighting representations, comprising: a set of three photometric images; surface gradient and albedo maps; polynomial texture maps; and two eigen based representations using 3 and 6 base images.We used twelve real textures to perform quantitative tests on the relighting methods in isolation. We developed a qualitative test for the assessment of the complete synthesis systems. Ten observers were asked to rank the images obtained from the five methods using five real textures. Statistical tests were applied to the rankings.The six-base-image eigen method produced the best quantitative relighting results and in particular was better able to cope with specular surfaces. However, in the qualitative tests there were no significant performance differences detected between it and the other two top performers. Our conclusion is therefore that the cheaper gradient and three-base-image eigen methods should be used in preference, especially where the surfaces are Lambertian or near Lambertian.

UNION: underwater intelligent operation and navigation

The main goal of the UNION ESPRIT Basic Research Action is to develop methods for increasing the autonomy and intelligence of underwater remotely operated vehicles (ROVs). The project focuses mainly on the development of coordinated control and sensing strategies for combined manipulator and vehicle systems. Both fundamental theories and methods for the design of these heterogeneous systems are investigated. A complex canonical mission in the field of offshore inspection maintenance and repair tasks was chosen as an integration guideline of all the results.

Selecting tools and techniques for model-based diagnosis

We present a methodology for the selection of candidate generation and prediction techniques for model-based diagnostic systems (MBDS). We start by describing our taxonomy of the solution space based upon the three main functional blocks of a top-level MBDS architecture (the predictor, the candidate generator and the diagnostic strategist). We divide the corresponding problem space into user requirements and system constraints which are further subdivided into task and fault requirements, and plant and domain knowledge constraints respectively. Finally we propose a set of guidelines for selecting tools and techniques in the solution space given descriptions of diagnostic tasks in the problem space.

Classifying Surface Texture while Simultaneously Estimating Illumination Direction

We propose a novel classifier that both classifies surface texture and simultaneously estimates the unknown illumination conditions. A new formal model of the dependency of texture features on lighting direction is developed which shows that their mean vectors are trigonometric functions of the illuminations’ tilt and slant angles. This is used to develop a probabilistic description of feature behaviour which forms the basis of the new classifier. Given a feature set from an image of an unknown texture captured under unknown illumination conditions the algorithm first estimates the most likely illumination direction for each possible texture class. These estimates are used to calculate the class likelihoods and the classification is made accordingly.The ability of the classifier to estimate illuminant direction, and to assign the correct class, was tested on 55 real texture samples in two stages. The classifier was able to accurately estimate both the tilt and the slant angles of the light source for the majority of textures and gave a 98% classification rate.

A comparison of inter-frame feature measures for robust object classification in sector scan sonar image sequences

This paper presents an investigation of the robustness of an inter-frame feature measure classifier for underwater sector scan sonar image sequences. In the initial stages the images are of either divers or remotely operated vehicles (ROVs). The inter-frame feature measures are derived from sequences of sonar scans to characterize the behavior of the objects over time. The classifier has been shown to produce error rates of 0%-2% using real nonnoisy images. The investigation looks at the robustness of the classifier with increased noise conditions and changes in the filtering of the images. It also identifies a set of features that are less susceptible to increased noise conditions and changes in the image filters. These features are the mean variance, and the variance of the rate of change in time of the intra-frame feature measures area, perimeter, compactness, maximum dimension and the first and second invariant moments of the objects. It is shown how the performance of the classifier can be improved. Success rates of up to 100% were obtained for a classifier trained under normal noise conditions, signal-to-noise ratio (SNR) around 9.5 dB, and a noisy test sequence of SNR 7.6 dB.

On optimal light configurations in photometric stereo

This paper develops new theory for the optimal placement of photometric stereo lighting in the presence of camera noise. We show that for three lights, any triplet of orthogonal light directions minimises the uncertainty in scaled normal computation. The assumptions are that the camera noise is additive and normally distributed, and uncertainty is defined as the expectation of squared distance of scale normal to the ground truth. If the camera noise is of zero mean and variance sigma2 the optimal (minimum) uncertainty in the scaled normal is 3sigma2 For case of n > 3 lights, we show that the minimum uncertainty is 9sigma2n, and identify sets of light configurations which reach this theoretical minimum

The Effect of Illuminant Rotation on Texture Filters: Lissajous's Ellipses

Changes in the angle of illumination incident upon a 3D surface texture can significantly change its appearance. These changes can affect the output of texture features to such an extent that they cause complete misclassification. We present new theory and experimental results that show that changes in illumination tilt angle cause texture clusters to describe Lissajouss ellipses in feature space. We focus on texture features that may be modelled as a linear filter followed by an energy estimation process e.g. Laws filters, Gabor filters, ring and wedge filters. This general texture filter model is combined with a linear approximation of Lamberts cosine law to predict that the outputs of these filters are sinusoidal functions of illuminant tilt. Experimentation with 30 real textures verifies this proposal. Furthermore we use these results to show that the clusters of distinct textures describe different elliptical paths in feature space as illuminant tilt varies. These results have significant implications for illuminant tilt invariant texture classification.

The use of fault-recorder data for diagnosing timing and other related faults in electricity transmission networks

This paper describes a model-based diagnostic system for diagnosing faults in electrical transmission systems (Timelys off-line system). This diagnostic system uses data available from digital fault recorders which are collected after a network event (such as a short circuit) has occurred. The data are used to detect incipient faults in network equipment by comparing their operation against that predicted by extended finite state automaton known as augmented reactive models (ARM). Thus Timelys off-line system combines signal processing and model based diagnostic techniques to provide a practical model-based system that aids the analysis of the performance of protective equipment after a network event has occurred. In particular, its use of data derived directly from fault recorder files (such as voltage and impedance magnitudes) means that the system can diagnose much more subtle faults (e.g. timing related faults).

Measuring perceived differences in surface texture due to changes in higher order statistics

We investigate the ability of humans to perceive changes in the appearance of images of surface texture caused by the variation of their higher order statistics. We incrementally randomize their phase spectra while holding their first and second order statistics constant in order to ensure that the change in the appearance is due solely to changes in third and other higher order statistics. Stimuli comprise both natural and synthetically generated naturalistic images, with the latter being used to prevent observers from making pixel-wise comparisons. A difference scaling method is used to derive the perceptual scales for each observer, which show a sigmoidal relationship with the degree of randomization. Observers were maximally sensitive to changes within the 20%-60% randomization range. In order to account for this behavior we propose a biologically plausible model that computes the variance of local measurements of phase congruency.