Jonathan Howland

Advances in Underwater Robot Vehicles for Deep Ocean Exploration: Navigation, Control, and Survey Operations

This paper reports recent advances and open problems in navigation, control, and survey methodologies for underwater robotic vehicles. First, the technical challenges of underwater navigation are discussed. Second, an acoustic navigation system combining bottom-lock doppler sonar and time-of-flight long-baseline (LBL) navigation techniques, recently reported by the Authors, is reviewed. The performance of this system is examined in the context of recent deep-submergence operations with the Jason underwater robot. Third, principal theoretical and practical obstacles to the task of acoustic and optical undersea survey operations are reviewed. These issues are illustrated with acoustic bathymetric and optical photomosaic surveys performed by the authors during two recent deployments in the Mediterranean Sea.

Advances in large-area photomosaicking underwater

The propagation of visible light underwater suffers rapid attenuation and extreme scattering. This, in combination with the limited camera-to-light separation available on most imaging platforms, places severe limitations on our ability to optically image large areas of the sea floor at high resolution. We present a general framework for mosaicking large areas underwater with a specific emphasis on the issues that are unique to the underwater environment. At the individual image level, we examine the role of attenuation, scattering, and camera to light separation and present the tradeoffs involved in optimizing a particular imaging geometry. We also examine the arbitrary image-registration problem in the face of conditions prevalent underwater, namely a moving nonuniform lighting source and the effects of a featureless unstructured terrain. Our analysis is based on photomosaics encompassing several hundred images on archaeological, forensic, and geological expeditions from a diverse set of imaging platforms, including the NR-1 nuclear submarine, the manned submersible Alvin, the Argo towed vehicle, the Jason remotely operated vehicle, and the ABE autonomous underwater vehicle.

UWIT: underwater image toolbox for optical image processing and mosaicking in MATLAB

This paper shows results from our development of an extended MATLAB image processing toolbox, which implements some useful optical image processing and mosaicking algorithms found in the literature. We surveyed and selected algorithms from the field which showed promise in application to the underwater environment. We then extended these algorithms to explicitly deal with the unique constraints of underwater imagery in the building of our toolbox. As such, the algorithms implemented include: contrast limited adaptive histogram specification (CLAHS) to deal with the inherent nonuniform lighting in underwater imagery, Fourier based methods for scale, rotation, and translation recovery which provide robustness against dissimilar image regions, local normalized correlation for image registration to handle the low-feature, unstructured environment of the seafloor, multiresolution pyramidal blending of images to form a composite seamless mosaic without blurring or loss of detail near image borders. In this paper we highlight the mathematical formulation behind each of these algorithms using consistent notation and a unified framework. We depict some of our MATLAB toolbox results with an assortment of underwater imagery.

The Nereus hybrid underwater robotic vehicle for global ocean science operations to 11,000m depth

This paper reports an overview of the new Nereus hybrid underwater vehicle and summarizes the vehicles performance during its first sea trials in November 2007. Nereus is a novel operational underwater vehicle designed to perform scientific survey and sampling to the full depth of the ocean of 11,000 meters - almost twice the depth of any present-day operational vehicle. Nereus operates in two different modes. For broad area survey, the vehicle can operate untethered as an autonomous underwater vehicle (AUV) capable of exploring and mapping the sea floor with sonars and cameras. For close up imaging and sampling, Nereus can be converted at sea to operate as a tethered remotely operated vehicle (ROV). This paper reports the overall vehicle design and design elements including ceramic pressure housings and flotation spheres; manipulator and sampling system; light fiber optic tether; lighting and imaging; power and propulsion; navigation; vehicle dynamics and control; and acoustic communications.

Image registration underwater for fluid flow measurements and mosaicking

The authors present a unified framework of increasing complexity that handles image registration ranging from translation only motion between images, to a full eight parameter projective transformation. Their formulation deals specifically with the constraints that are peculiar to these applications underwater. To examine the role of translation only motion across images they look at video imagery from fluid flow across a flange at a hydrothermal vent site in Guaymas Basin in Mexico. They then show the extensions to this algorithm that are required to handle the case for 2D mosaicing, which involves translations, rotations, scale, and shear in an unstructured three dimensional underwater world. Finally, they show the effectiveness of these techniques on data acquired during photographic surveys of an ancient Roman shipwreck located in the Skerki Bank region in /spl sim/800 m of water in the Mediterranean.

Navigation and control of the Nereus hybrid underwater vehicle for global ocean science to 10,903 m depth: Preliminary results

This paper reports an overview of the navigation and control system design for the new Nereus hybrid underwater robotic vehicle (HROV). Vehicle performance during its first sea trials in November 2007 near Hawaii, and in May and June 2009 in the Challenger Deep of the Mariana Trench is reported. During the latter expedition, the vehicle successfully performed scientific observation and sampling operations at depths exceeding 10,903 m. The Nereus underwater vehicle is designed to perform scientific survey and sampling to the full depth of the ocean — significantly deeper than the depth capability of all other present-day operational vehicles. For comparison, the second deepest underwater vehicle currently operational worldwide can dive to 7,000 m maximum depth. Nereus operates in two different modes. For broad-area survey, the vehicle can operate untethered as an autonomous underwater vehicle (AUV) capable of exploring and mapping the sea floor with sonars and cameras. Nereus can be converted at sea to become a tethered remotely operated vehicle (ROV) to enable close-up imaging and sampling. The ROV configuration incorporates a lightweight fiber-optic tether (for high-bandwidth, real-time video and data telemetry to the surface), an electro-hydraulic manipulator arm, and sampling instruments. The Nereus vehicle is designed to render all parts of the Earths seafloor accessible to oceanographic science.

Evolution of a benthic imaging system from a towed camera to an automated habitat characterization system

After two generations of development, we have an operational and practical digital imaging system that delivers high resolution overlapping still images to a computer system on the bridge of a commercial scallop fishing vessel for immediate viewing, storage, and onboard image processing. This non-invasive imaging system produces 100 nautical mile long optical transects of benthic taxa, communities, and associated substrate each day, and is intended to provide fisheries managers with accurate scallop population density estimates and habitat characterization within surveyed areas of the continental shelf. We call the instrument HabCam for habitat mapping camera system. Joint ship operations with NOAA vessels conducting annual scallop surveys has allowed for nearly direct comparison between estimates of scallop abundance by survey dredge and the HabCam imaging system. For 47 transects conducted jointly during 2007, dredge efficiency ranged from 10 to 80% with a mean of 40% (SD 23.9%) depending on area, substrate, tow direction relative to current, and mean distance between the dredge tow track and the HabCam imaging track. Integration of synoptically collected acoustical (675 kHz sidescan, 175 kHz synthetic aperture side scan and 300 kHz multibeam) and optical imaging has allowed for direct registration and comparison of sampling modalities, ground truthing of acoustical data, and extrapolation of information gained at small scale (1m) but high spatial resolution (1 mm) with optics to large scale (>200 m) acoustical data sets. What was initially developed as a scallop survey tool has become an instrument system capable of providing information on habitat characterization, estimates of megafauna abundance, biodiversity, and species richness. A project called the Northeast Bentho-pelagic Observatory (NEBO) is using HabCam to evaluate these ecological parameters at sentinel study sites along the northeast continental shelf repeatedly over several years with the intent of documenting mechanistically how and why benthic community composition is changing over time. A key element in the development of HabCam as a tool for habitat characterization is the automated processing of images for color correction, segmentation of foreground targets from sediment and classification of targets to taxonomic category, and in many cases, to species. A test set of images has been developed consisting of about 30,000 images from each of six sites along the northeast continental shelf representing areas differentially impacted by physical, biological and chemical forcing. Each of these 180,000 images has been manually processed for species counts and sizes so as to provide a training set for automated approaches to target classification. All images and data are available on a public website (http://habcam.whoi.edu).

A New Control System for the Next Generation of US and UK Deep Submergence Oceanographic ROVS

Abstract This paper reports the development and results of at-sea field testing of a new control system for remotely operated under water vehicles. The control system provides model-based closed loop trajectory tracking control in addition to conventional PID control, dynamic power management, and a high level of power control and monitoring. The control system is presently deployed on three operational US vehicles, and is scheduled for deployment on two additional oceanographic vehicles in the UK and US.

Quantitative stereo imaging from the Autonomous Benthic Explorer (ABE)

The Autonomous Benthic Explorer (ABE) is a vehicle designed to perform long-term autonomous repeatable surveys of the deep ocean. In this paper we examine the basic methodology of obtaining quantitative stereo measurements with ABE. We utilize stereo imagery collected during an autonomous survey along a tectonically active segment of the Juan de Fuca Ridge. The ability to make repeatable measurements is crucial for being able to carry out tasks associated with the detection of spatio-temporal changes. As this is one of the stated goals of ABE we have been developing a stereo camera system to make quantitative measurements from underwater imagery. Pursuant to our goals of long term deployments and the low power levels that such a goal imposes on each vehicle subsystem, our efforts have been focussed on the design of a low-energy stereo camera system. In this paper we describe our system and the methodology for its alignment and calibration in the typical harsh operating conditions associated with working on research platforms at sea. We demonstrate photogrammetric measurements at very fine scales of actual pillow lavas near a diffuse hydrothermal vent. We use a calibrated object to independently ground truth our results. Finally, we discuss improvements to these techniques which we intend to pursue.

Quantitative photomosaicking of underwater imagery

Looks at some of the justification behind photomosaicking for underwater imaging. One of the primary reasons for employing photomosaicking underwater is to obtain a better global perspective in the underwater environment. Due to physical constraints underwater, it is virtually impossible to frame large objects of interest within a single picture frame. Thus there has been considerable interest over the years in taking a series of images and composing them together into a composite photomosaic of the area of interest. The metric for judging such efforts have been largely qualitative equalizing the illumination across images to eliminate differences in intensity and paying careful attention to matching features that cross image boundaries to enhance the sense of continuity across the mosaic. Unfortunately these methods yield large overall distortions as the errors tend to accumulate in a photomosaic as successive images are added. The authors present results that highlight the need for more than a qualitative metric for determining the accuracy of a photomosaic. The authors use navigation data to analyze the rate and nature of error propagation in photomosaicking. By examining the nature of the error the authors show the potential of such a technique for generating large scale photomosaics with bounded errors.