Ariell Friedman

Monitoring of Benthic Reference Sites: Using an Autonomous Underwater Vehicle

We have established an Australia-wide observation program that exhibits recent developments in autonomous underwater vehicle (AUV) systems to deliver precisely navigated time series benthic imagery at selected reference stations on Australias continental shelf. These observations are designed to help characterize changes in benthic assemblage composition and cover derived from precisely registered maps collected at regular intervals. This information will provide researchers with the baseline ecological data necessary to make quantitative inferences about the long-term effects of climate change and human activities on the benthos. Incorporating a suite of observations that capitalize on the unique capabilities of AUVs into Australias integrated marine observation system (IMOS) [1] is providing a critical link between oceanographic and benthic processes. IMOS is a nationally coordinated program designed to establish and maintain the research infrastructure required to support Australias marine science research. It has, and will maintain, a strategic focus on the impact of major boundary currents on continental shelf environments, ecosystems, and biodiversity. The IMOS AUV facility observation program is designed to generate physical and biological observations of benthic variables that cannot be cost effectively obtained by other means.

Multi-scale measures of rugosity, slope and aspect from benthic stereo image reconstructions.

This paper demonstrates how multi-scale measures of rugosity, slope and aspect can be derived from fine-scale bathymetric reconstructions created from geo-referenced stereo imagery. We generate three-dimensional reconstructions over large spatial scales using data collected by Autonomous Underwater Vehicles (AUVs), Remotely Operated Vehicles (ROVs), manned submersibles and diver-held imaging systems. We propose a new method for calculating rugosity in a Delaunay triangulated surface mesh by projecting areas onto the plane of best fit using Principal Component Analysis (PCA). Slope and aspect can be calculated with very little extra effort, and fitting a plane serves to decouple rugosity from slope. We compare the results of the virtual terrain complexity calculations with experimental results using conventional in-situ measurement methods. We show that performing calculations over a digital terrain reconstruction is more flexible, robust and easily repeatable. In addition, the method is non-contact and provides much less environmental impact compared to traditional survey techniques. For diver-based surveys, the time underwater needed to collect rugosity data is significantly reduced and, being a technique based on images, it is possible to use robotic platforms that can operate beyond diver depths. Measurements can be calculated exhaustively at multiple scales for surveys with tens of thousands of images covering thousands of square metres. The technique is demonstrated on data gathered by a diver-rig and an AUV, on small single-transect surveys and on a larger, dense survey that covers over . Stereo images provide 3D structure as well as visual appearance, which could potentially feed into automated classification techniques. Our multi-scale rugosity, slope and aspect measures have already been adopted in a number of marine science studies. This paper presents a detailed description of the method and thoroughly validates it against traditional in-situ measurements.

Rugosity, slope and aspect from bathymetric stereo image reconstructions

This paper demonstrates how multi-scale measures of rugosity, slope and aspect can be derived from fine-scale bathymetric reconstructions created using geo-referenced stereo imagery collected by an Autonomous Underwater Vehicle (AUV). We briefly describe the 3D triangular meshes generated from the stereo images and then present a detailed overview of how rugosity can be derived by considering the area of triangles within a window and their projection onto the plane of best fit. By obtaining the plane of best fit, slope and aspect can be calculated with very little extra effort. The results are validated on a simulated surface and the effects of mesh resolution and window size are explored. The technique is demonstrated on real data gathered by an AUV on surveys that cover several linear kilometres and consist of thousands of images. The ability to distinguish habitat types based on rugosity and slope are demonstrated through K-means cluster analysis. A human labelled data set is then used to train a SVM classifier that exhibits promising habitat classification potential based on rugosity and slope.

Reconstructing pavlopetri: Mapping the world's oldest submerged town using stereo-vision

This paper presents a vision-based underwater mapping system, which is demonstrated in an archaeological survey of the submerged ancient town of Pavlopetri. The snorkeler or diver operated system provides a low cost alternative to the use of an AUV or ROV in shallow waters. The system produces textured three-dimensional models, which contain significantly more information than traditional archaeological survey methods. The photo-realistic maps that are produced allow further archaeological research to be performed, without diving on a site during the restrictive time limitations of permits and field seasons. The hardware and software components of the mapping system and its method of operation are described, and initial results are presented and discussed.

Australian sea-floor survey data, with images and expert annotations.

This Australian benthic data set (BENTHOZ-2015) consists of an expert-annotated set of georeferenced benthic images and associated sensor data, captured by an autonomous underwater vehicle (AUV) around Australia. This type of data is of interest to marine scientists studying benthic habitats and organisms. AUVs collect georeferenced images over an area with consistent illumination and altitude, and make it possible to generate broad scale, photo-realistic 3D maps. Marine scientists then typically spend several minutes on each of thousands of images, labeling substratum type and biota at a subset of points. Labels from four Australian research groups were combined using the CATAMI classification scheme, a hierarchical classification scheme based on taxonomy and morphology for scoring marine imagery. This data set consists of 407,968 expert labeled points from around the Australian coast, with associated images, geolocation and other sensor data. The robotic surveys that collected this data form part of Australias Integrated Marine Observing System (IMOS) ongoing benthic monitoring program. There is reuse potential in marine science, robotics, and computer vision research.

Benthic monitoring with robotic platforms — The experience of Australia

Australias Integrated Marine Observing System (IMOS) has a strategic focus on the impact of major boundary currents on continental shelf environments, ecosystems and biodiversity. To improve our understanding of natural, climate change, and human-induced variability in shelf environments, the IMOS Autonomous Underwater Vehicle (AUV) facility has been charged with generating physical and biological observations of benthic variables that cannot be cost-effectively obtained by other means. Starting in 2010, the IMOS AUV facility began collecting precisely navigated benthic imagery using AUVs at selected reference sites on Australias shelf. This observing program capitalizes on the unique capabilities of AUVs that have allowed repeated visits to the reference sites, providing a critical observational link between oceanographic and benthic processes. This paper provides a brief overview of the relevant capabilities of the AUV facility, the design of the IMOS benthic sampling program, and some preliminary results. We also report on some of the challenges and potential benefits to be realized from a benthic observation system that collects several TB of geo-referenced stereo imagery a year. This includes collaborative semi-automated image analysis, clustering and classification, large scale visualization and data mining, and lighting correction for change detection and characterization. We also mention some of the lessons from operating an AUV-based monitoring program and future work in this area.

Active learning using a Variational Dirichlet Process model for pre-clustering and classification of underwater stereo imagery

This paper demonstrates an implementation of pool-based active learning through uncertainty sampling using a Variational Dirichlet Process (VDP) model. The VDP is used for both pre-clustering and classification, and is extended to incorporate fixed labels from an oracle (human annotator). Three different uncertainty sampling techniques are explored - least confident sampling, margin sampling and entropy based sampling. Clustering with the VDP is done in a completely unsupervised manner, without the need to specify the number of clusters. This appears particularly useful in improving the results when there are only few labelled samples, or if the cost of labelling is high. Results are shown for a toy dataset and the performance is compared to similar implementations using an Expectation Maximisation model (EM) and a Naive Bayes classifier (NB). The VDP active learning framework is tested on a stereo image dataset obtained by an autonomous underwater vehicle that covers several linear kilometres and consists of thousands of stereo image pairs. Our results show that combining an active learning strategy with the VDP significantly reduces the number of labelled images required to achieve a desired level of accuracy.

A simple, fast, and repeatable survey method for underwater visual 3D benthic mapping and monitoring

Abstract Visual 3D reconstruction techniques provide rich ecological and habitat structural information from underwater imagery. However, an unaided swimmer or diver struggles to navigate precisely over larger extents with consistent image overlap needed for visual reconstruction. While underwater robots have demonstrated systematic coverage of areas much larger than the footprint of a single image, access to suitable robotic systems is limited and requires specialized operators. Furthermore, robots are poor at navigating hydrodynamic habitats such as shallow coral reefs. We present a simple approach that constrains the motion of a swimmer using a line unwinding from a fixed central drum. The resulting motion is the involute of a circle, a spiral‐like path with constant spacing between revolutions. We test this survey method at a broad range of habitats and hydrodynamic conditions encircling Lizard Island in the Great Barrier Reef, Australia. The approach generates fast, structured, repeatable, and large‐extent surveys (~110 m2 in 15 min) that can be performed with two people and are superior to the commonly used “mow the lawn” method. The amount of image overlap is a design parameter, allowing for surveys that can then be reliably used in an automated processing pipeline to generate 3D reconstructions, orthographically projected mosaics, and structural complexity indices. The individual images or full mosaics can also be labeled for benthic diversity and cover estimates. The survey method we present can serve as a standard approach to repeatedly collecting underwater imagery for high‐resolution 2D mosaics and 3D reconstructions covering spatial extents much larger than a single image footprint without requiring sophisticated robotic systems or lengthy deployment of visual guides. As such, it opens up cost‐effective novel observations to inform studies relating habitat structure to ecological processes and biodiversity at scales and spatial resolutions not readily available previously.

Habitat structural complexity metrics improve predictions of fish abundance and distribution

Habitat structural complexity influences biotic diversity and abundance, but its influence on marine ecosystems has not been widely addressed. Recent advances in computer vision and robotics allow quantification of structural complexity at higher-resolutions than previously achieved. This provides an important opportunity to determine the ecological role of habitat structural complexity in marine ecosystems. We used high-resolution three-dimensional (3D) maps to test multiple structural complexity metrics, depth and benthic biota as surrogates of fish assemblages across hundreds of meters on subtropical reefs. Non-parametric multivariate statistics were used to determine the relationship between these surrogates and the entire fish assemblage. Fish were divided into functional groups, which were used to further investigate the relationship between surrogates and fish abundance using generalized linear models. Fish community composition and abundance were strongly related to habitat complexity metrics, benthic biota and depth. Surface rugosity and its variance had a significant positive influence on the abundance of piscivores and sediment infauna predators, and a negative effect on the abundance of predators, herbivores, planktivores and cleaners. Final models for fish functional groups explained up to 68% of the variance. The best metrics to explain the variance in fish abundance were benthic biota (25 7.5% of variance explained, mean  SE) and complexity metrics (16 6.6%, mean  SE). Our results show that high-resolution 3D maps and derived metrics can predict a large percentage of variance in fish abundance and potentially serve as useful surrogates of fish abundance across all functional groups in spatially dynamic reefs.

Reflections on a decade of autonomous underwater vehicles operations for marine survey at the Australian Centre for Field Robotics

Abstract This paper describes insights gained from a decade of autonomous marine systems development at the University of Sydney’s Australian Centre for Marine Robotics. Over the course of this time, we have deployed numerous vehicles and imaging platforms in support of applications in engineering science, marine ecology, archaeology and geoscience. We have operated an Australia-wide benthic observing program designed to deliver precisely navigated, repeat imagery of the seafloor. This initiative makes extensive use of Autonomous Underwater Vehicles (AUVs) to collect high-resolution stereo imagery, multibeam sonar and water column measurements on an annual or semi-annual basis at sites around Australia, spanning the full latitudinal range of the continent from tropical reefs in the north to temperate regions in the south. We have also contributed to expeditions to document coral bleaching, cyclone recovery, submerged neolithic settlement sites, ancient shipwrecks, methane seeps and deepwater hydrothermal vents. We briefly consider how automated tools for working with this imagery have facilitated the resulting science outcomes.