I. Tena Ruiz

Underwater vehicle obstacle avoidance and path planning using a multi-beam forward looking sonar

This paper describes a new framework for segmentation of sonar images, tracking of underwater objects and motion estimation. This framework is applied to the design of an obstacle avoidance and path planning system for underwater vehicles based on a multi-beam forward looking sonar sensor. The real-time data flow (acoustic images) at the input of the system is first segmented and relevant features are extracted. We also take advantage of the real-time data stream to track the obstacles in following frames to obtain their dynamic characteristics. This allows us to optimize the preprocessing phases in segmenting only the relevant part of the images. Once the static (size and shape) as well as dynamic characteristics (velocity, acceleration,...) of the obstacles have been computed, we create a representation of the vehicles workspace based on these features. This representation uses constructive solid geometry (CSG) to create a convex set of obstacles defining the workspace. The tracking takes also into account obstacles which are no longer in the field of view of the sonar in the path planning phase. A well-proven nonlinear search (sequential quadratic programming) is then employed, where obstacles are expressed as constraints in the search space. This approach is less affected by local minima than classical methods using potential fields. The proposed system is not only capable of obstacle avoidance but also of path planning in complex environments which include fast moving obstacles. Results obtained on real sonar data are shown and discussed. Possible applications to sonar servoing and real-time motion estimation are also discussed.

Concurrent mapping and localization using sidescan sonar

This paper describes and evaluates a concurrent mapping and localization (CML) algorithm suitable for localizing an autonomous underwater vehicle. The proposed CML algorithm uses a sidescan sonar to sense the environment. The returns from the sonar are used to detect landmarks in the vehicles vicinity. These landmarks are used, in conjunction with a vehicle model, by the CML algorithm to concurrently build an absolute map of the environment and to localize the vehicle in absolute coordinates. As the vehicle moves forward, the areas covered by a forward-look sonar overlap, whereas little or no overlap occurs when using sidescan sonar. It has been demonstrated that numerous reobservations by a forward-look sonar of the landmarks can be used to perform CML. Multipass missions, such as sets of parallel and regularly spaced linear tracks, allow a few reobservations of each landmark with sidescan sonar. An evaluation of the CML algorithm using sidescan sonar is made on this type of trajectory. The estimated trajectory provided by the CML algorithm shows significant jerks in the positions and heading brought about by the corrections that occur when a landmark is reobserved. Thus, this trajectory is not useful to mosaic the sea bed. This paper proposes the implementation of an optimal smoother on the CML solution. A forward stochastic map is used in conjunction with a backward Rauch-Tung-Striebel filter to provide the smoothed trajectory. This paper presents simulation and real results and shows that the smoothed CML solution helps to produce a more accurate navigation solution and a smooth navigation trajectory. This paper also shows that the qualitative value of the mosaics produced using CML is far superior to those that do not use it.

Feature Tracking in Video and Sonar Subsea Sequences with Applications

This paper deals with automatic target tracking in video and sonar subsea sequences, an essential capability for automating tasks currently performed by remotely operated vehicles under pilot control. We describe two trackers, one for video sequences, the other for sector scan sonar sequences. No assumptions are made about the images, scene, or motion observed. To illustrate applications, we report results of our systems for 3-D structure reconstruction and panoramic mosaic building from video sequences and describe in some detail our path planning and obstacle avoidance system using sonar sequences.

Texture analysis for seabed classification: co-occurrence matrices vs. self-organizing maps

Considers two well-known pattern recognition techniques using texture analysis. The first is the co-occurrence matrix method which relies on statistics and the second is the Kohonen map which comes from the artificial neural networks domain. Both methods are used as feature extraction methods. The extracted feature vectors are fed to a second Kohonen map used as classifier. The authors report briefly some results of their experimental assessment of the merit of each technique when applied to the problem of classifying the seabed from sequences of real images.

A comparison of inter-frame feature measures for robust object classification in sector scan sonar image sequences

This paper presents an investigation of the robustness of an inter-frame feature measure classifier for underwater sector scan sonar image sequences. In the initial stages the images are of either divers or remotely operated vehicles (ROVs). The inter-frame feature measures are derived from sequences of sonar scans to characterize the behavior of the objects over time. The classifier has been shown to produce error rates of 0%-2% using real nonnoisy images. The investigation looks at the robustness of the classifier with increased noise conditions and changes in the filtering of the images. It also identifies a set of features that are less susceptible to increased noise conditions and changes in the image filters. These features are the mean variance, and the variance of the rate of change in time of the intra-frame feature measures area, perimeter, compactness, maximum dimension and the first and second invariant moments of the objects. It is shown how the performance of the classifier can be improved. Success rates of up to 100% were obtained for a classifier trained under normal noise conditions, signal-to-noise ratio (SNR) around 9.5 dB, and a noisy test sequence of SNR 7.6 dB.

Underwater vehicle path planning using a multi-beam forward looking sonar

Describes an obstacle avoidance and path planning system for underwater vehicles based on a multi-beam forward looking sonar sensor. The real-time data flow (acoustic images) at the input of the system is first processed (segmentation and feature extraction) to create a representation of the workspace of the vehicle. This representation uses constructive solid geometry (CSG) to create a convex set of obstacles defining the workspace. We also take advantage of the real-time data stream to track the obstacles in the subsequent frames to obtain their dynamic characteristics. This will also allow us to optimise the preprocessing phases in segmenting only the relevant part of the images as well as to take into account obstacles which are no longer in the field of view of the sonar in the path planning phase. A well proven nonlinear search (sequential quadratic programming) is then employed, where obstacles are expressed as constraints in the search space. This approach is less affected by local minima than classical methods using potential fields. The proposed system is not only capable of obstacle avoidance but also of path planning in complex environments which include fast moving obstacles. Preliminary results obtained on real data are shown and discussed.

Improved AUV navigation using side-scan sonar

The following paper presents results from a novel solution for improving the navigation of an autonomous underwater vehicle (AUV) using a side-scan sonar. It is derived from a system that has been developed to produce high quality mosaics using a Doppler velocity log (DVL), a triaxial compass and a side-scan sonar. The system has been extended by incorporating a concurrent mapping and localization (CML) algorithm. The CML tool chosen is the stochastic map. This is a proven tool for navigating in unknown environments. It can be used as a substitute for common absolute sensors, such as GPS or acoustic baseline systems, or it can work with them. The system concurrently builds a map of the environment using observations of landmarks extracted from side-scan sonar and uses that map an the dead-reckoning to create and estimate of the AUVs location.

Tracking and classification of multiple objects in multibeam sector scan sonar image sequences

Multi-beam forward looking sector scan sonars are commonly used as obstacle avoidance and relative navigation sensors on unmanned underwater vehicles. Their key characteristic is a fast update rate (e.g. 12 Hz at 10 metres range). This offers opportunity to exploit temporal as well as spatial correlation in automatic processing of the data. We present the approach to object segmentation, tracking and classification, exploiting both inter and intra frame processing. Using optical flow motion estimation, coupled to a tree structure allowing object tracks to be revised, we have demonstrated good tracking performance, with prediction errors of between 10 and 50 cm (1-5% of scan range). Supervised object classification has demonstrated errors of 1 to 2 % using non-noisy images. With realistic sensor noise, classification of up to 100% was achieved with signal-to-noise ratio between 7.6 and 9.5 dB.

The automatic fusion of classified sidescan sonar mosaics using CML-RTS and Markov random fields

This paper presents a framework for registering and fusing classified sidescan sonar data. It builds on recent advances in navigation and registration for improved mosaicing, applying novel fusion algorithms to integrate data from overlapping sidescan survey lines to produce large scale classified mosaics. While typical mine-counter-measures (MCM) and rapid environmental assessment (REA) missions provide various over-lapping views of the same region of seafloor, research on sidescan image analysis has traditionally concentrated on the analysis of individual images. The available information from the other images, relating to the same region of seafloor, is generally not considered. The image registration and mosaicing process allows this complementary data to be fused, producing an improved final classification result. The sidescan imagery is first pre-processed through the application of advanced radiosity correction algorithms. Following radiosity correction, texture segmentation for the data presented in this paper is achieved using features derived from the averaged normalised power spectral density. The individual classification maps are georeferenced and coregistered using a Concurrent Mapping and Localisation Rauch-Tung-Striebel (CML-RTS) procedure. This uses local landmarks within the individual images and the AUVs navigation data to generate a more accurate and smooth navigation trajectory. This trajectory is used to produce the registered classification mosaics. The coregistered classification results are then fused to produce an improved class mosaic for the entire survey region. The fusion model uses a voting scheme to initialize the seafloor map after which a Markov random field (MRF) model is used to produce the final fused classification mosaic. The entire process (classification, registration and fusion) is demonstrated on real sidescan data taken at the Saclant Centre, La Spezia, Italy.