Nuno Gracias

Underwater Video Mosaics as Visual Navigation Maps

This paper presents a set of algorithms for the creation of underwater mosaics and illustrates their use as visual maps for underwater vehicle navigation. First, we describe the automatic creation of video mosaics, which deals with the problem of image motion estimation in a robust and automatic way. The motion estimation is based on a initial matching of corresponding areas over pairs of images, followed by the use of a robust matching technique, which can cope with a high percentage of incorrect matches. Several motion models, established under the projective geometry framework, allow for the creation of high quality mosaics where no assumptions are made about the camera motion. Several tests were run on underwater image sequences, testifying to the good performance of the implemented matching and registration methods. Next, we deal with the issue of determining the 3D position and orientation of a vehicle from new views of a previously created mosaic. The problem of pose estimation is tackled, using the available information on the camera intrinsic parameters. This information ranges from the full knowledge to the case where they are estimated using a self-calibration technique based on the analysis of an image sequence captured under pure rotation. The performance of the 3D positioning algorithms is evaluated using images for which accurate ground truth is available.

Mosaic-based navigation for autonomous underwater vehicles

We propose an approach for vision-based navigation of underwater robots that relies on the use of video mosaics of the sea bottom as environmental representations for navigation. We present a methodology for building high-quality video mosaics of the sea bottom in a fully automatic manner, which ensures global spatial coherency. During navigation, a set of efficient visual routines are used for the fast and accurate localization of the underwater vehicle with respect to the mosaic. These visual routines were developed taking into account the operating requirements of real-time position sensing, error bounding, and computational load. A visual servoing controller, based on the vehicles kinematics, is used to drive the vehicle along a computed trajectory, specified in the mosaic, while maintaining constant altitude. The trajectory toward a goal point is generated online to avoid undefined areas in the mosaic. We have conducted a large set of sea trials, under realistic operating conditions. This paper demonstrates that without resorting to additional sensors, visual information can be used to create environment representations of the sea bottom (mosaics) and support long runs of navigation in a robust manner.

Large-Area Photo-Mosaics Using Global Alignment and Navigation Data

Seafloor imagery is a rich source of data for the study of biological and geological processes. Among several applications, still images of the ocean floor can be used to build image composites referred to as photo-mosaics. Photo-mosaics provide a wide-area visual representation of the benthos, and enable applications as diverse as geological surveys, mapping and detection of temporal changes in the morphology of biodiversity. We present an approach for creating globally aligned photo-mosaics using 3D position estimates provided by navigation sensors available in deep water surveys. Without image registration, such navigation data does not provide enough accuracy to produce useful composite images. Results from a challenging data set of the Lucky Strike vent field at the Mid Atlantic Ridge are reported.

Underwater mosaicing and trajectory reconstruction using global alignment

This paper deals with the problem of constructing high quality mosaics of the sea bed. It focuses on the use of long image sequences with time-distant superpositions, such as the ones arising from loop trajectories or zig-zag scanning patterns. An algorithm is presented for the simultaneous creation of mosaics and the estimation of the camera trajectory. The method comprises three major stages. The first stage consists of the sequential estimation of the image motion, using a reduced image motion model. The set of resulting consecutive homographies is cascaded, in order to infer the approximate topology of the camera movement. The topology information is then used to predict the area where there is image overlap resulting from nonconsecutive images. Secondly, a motion refinement is performed, by iteratively executing the following two main steps. (1) Point correspondences are established between non-adjacent pairs of images that present enough overlap. (2) The topology is refined, by searching for the set of homographies that minimizes the overall sum of distances in the point matches. The final stage of the algorithm consists of estimating the set of homographies and a world plane description that best fit the observation data. As the main concern is attaining high registration accuracy, a general parameterization of the homographies with 6 DOF for the pose is used, which is capable of modelling the effects of wave-induced general rotation and translation. The overall method is fully automatic in the sense it does not require human intervention at any of the stages, apart from the beforehand specification of the most adequate motion model for the first stage. We present results obtained from shallow water image sequences acquired by a ROV. The images present some of the common difficulties of underwater mosaicing, such as non-planar sea-bottom, moving objects and severe illumination changes. This sequence also serves to illustrate the robustness and good performance of the presented algorithm.

A Novel Blending Technique for Underwater Gigamosaicing

The fusion of several images of the same scene into a single and larger composite is known as photomosaicing. Unfortunately, the seams along image boundaries are often noticeable, due to photometrical and geometrical registration inaccuracies. Image blending is the merging step in which those artifacts are minimized. Processing bottlenecks and the lack of medium-specific processing tools have restricted underwater photomosaics to small areas despite the hundreds of thousands of square meters that modern surveys can cover. Large underwater photomosaics are increasingly in demand for the characterization of the seafloor for scientific purposes. Producing these mosaics is difficult due to the challenging nature of the underwater environment and the image acquisition conditions, including extreme depth, scattering and light attenuation phenomena, and difficulties in vehicle navigation and positioning. This paper proposes strategies and solutions to tackle the problems of very large underwater optical surveys (gigamosaics), presenting contributions in the image preprocessing, enhancing, and blending steps, resulting in an improved visual quality in the final photomosaic. A comprehensive review of the existing methods is also presented and discussed. Our approach is validated by a large optical survey of a deep-sea hydrothermal field, leading to a high-quality composite in excess of 5 Gpixel.

Image-Based Coral Reef Classification and Thematic Mapping

This paper presents a novel image classification scheme for benthic coral reef images that can be applied to both single image and composite mosaic datasets. The proposed method can be configured to the characteristics (e.g., the size of the dataset, number of classes, resolution of the samples, color information availability, class types, etc.) of individual datasets. The proposed method uses completed local binary pattern (CLBP), grey level co-occurrence matrix (GLCM), Gabor filter response, and opponent angle and hue channel color histograms as feature descriptors. For classification, either k- nearest neighbor (KNN), neural network (NN), support vector machine (SVM) or probability density weighted mean distance (PDWMD) is used. The combination of features and classifiers that attains the best results is presented together with the guidelines for selection. The accuracy and efficiency of our proposed method are compared with other state-of-the-art techniques using three benthic and three texture datasets. The proposed method achieves the highest overall classification accuracy of any of the tested methods and has moderate execution time. Finally, the proposed classification scheme is applied to a large-scale image mosaic of the Red Sea to create a completely classified thematic map of the reef benthos.

Multipurpose autonomous underwater intervention: A systems integration perspective

Nowadays, autonomous intervention is getting more attention in the underwater robotics community. Few research projects on this matter are currently under development. In this context, and after a first successful experience in the RAUVI Spanish project (2009-2011), the authors are currently involved in the TRIDENT project (2010-2013), funded by the European Commission. To succeed in autonomous intervention, an AUV endowed with a manipulator and with a high degree of autonomy is essential. The complexity of the required robotic system is very high and the system integration process becomes critical. This paper presents the problems being solved in TRIDENT, from a systems integration perspective. As a case study, some results, achieved during the last experiments carried out in the Roses harbor (Girona) in October 2011 will be presented, to demonstrate the capabilities exhibited by the AUV for Intervention under development. The experiments were focused on the problem of autonomously searching and recovering a black-box mock-up that was previously thrown to an unknown position. This paper presents the hardware and software integration aspects that were necessary in order to address such a challenging problem.

A motion compensated filtering approach to remove sunlight flicker in shallow water images

A common problem in video surveys in very shallow waters is the presence of strong light fluctuations, due to sun light refraction. Refracted sunlight casts fast moving patterns, which can significantly degrade the quality of the acquired data. Motivated by the growing need to improve the quality of shallow water imagery, we propose a method to remove sunlight patterns in video sequences. The method exploits the fact that video sequences allow several observations of the same area of the sea floor, over time. It is based on computing the image difference between a given reference frame and the temporal median of a registered set of neighboring images. A key observation is that this difference will have two components with separable spectral content. One is related to the illumination field (lower spatial frequencies) and the other to the registration error (higher frequencies). The illumination field, recovered by lowpass filtering, is used to correct the reference image. In addition to removing the sunflickering patterns, an important advantage of the approach is the ability to preserve the sharpness in corrected image, even in the presence of registration inaccuracies. The effectiveness of the method is illustrated in image sets acquired under strong camera motion containing non-rigid benthic structures. The results testify the good performance and generality of the approach.

Automatic mosaic creation of the ocean floor

We describe the automatic creation of video mosaics of the ocean floor, which deals with the problem of image motion estimation in a robust and automatic way. The motion estimation presented in this work is based on an initial matching of corresponding areas over pairs of images. As the error prone nature of the matching process is a commonly overlooked problem, this paper makes use of robust matching techniques, which can cope with a high percentage of wrong matches. In our approach, several motion models are established under the projective geometry framework, allowing the creation of high quality mosaics where no assumptions are made on the camera motion. This as an improvement over traditional approaches for underwater mosaicing, usually relying on the camera to be facing the sea floor, so that the image plane is approximately parallel to the floor plane. Extensive tests were run on underwater image sequences, testifying the good performance of the implemented matching and registration methods. Even with notorious violations of the underlying assumption of static planar scenes, the algorithm can still find the motion parameters as to create mosaics with bearably noticeable misalignments to the human eye.

Fast topology estimation for image mosaicing using adaptive information thresholding

Over the past decade, several image mosaicing methods have been proposed in robotic mapping and remote sensing applications. Owing to rapid developments in obtaining optical data from areas beyond human reach, there is a high demand from different science fields for creating large-area image mosaics, often using images as the only source of information. One of the most important steps in the mosaicing process is motion estimation between overlapping images to obtain the topology, i.e., the spatial relationships between images. In this paper, we propose a generic framework for feature-based image mosaicing capable of obtaining the topology with a reduced number of matching attempts and of getting the best possible trajectory estimation. Innovative aspects include the use of a fast image similarity criterion combined with a Minimum Spanning Tree (MST) solution, to obtain a tentative topology and information theory principles to decide when to update trajectory estimation. Unlike previous approaches for large-area mosaicing, our framework is able to naturally deal with the cases where time-consecutive images cannot be matched successfully, such as completely unordered sets. This characteristic also makes our approach robust to sensor failure. The performance of the method is illustrated with experimental results obtained from different challenging underwater image sequences.