Duane R. Edgington

A review of techniques for the identification and measurement of fish in underwater stereo-video image sequences

Underwater stereo-video measurement systems are used widely for counting and measuring fish in aquaculture, fisheries and conservation management. To determine population counts, spatial or temporal frequencies, and age or weight distributions, snout to fork length measurements are captured from the video sequences, most commonly using a point and click process by a human operator. Current research aims to automate the measurement and counting task in order to improve the efficiency of the process and expand the use of stereo-video systems within marine science. A fully automated process will require the detection and identification of candidates for measurement, followed by the snout to fork length measurement, as well as the counting and tracking of fish. This paper presents a review of the techniques used for the detection, identification, measurement, counting and tracking of fish in underwater stereo-video image sequences, including consideration of the changing body shape. The review will analyse the most commonly used approaches, leading to an evaluation of the techniques most likely to be a general solution to the complete process of detection, identification, measurement, counting and tracking.

Detecting, Tracking and Classifying Animals in Underwater Video

For oceanographic research, remotely operated underwater vehicles (ROVs) and underwater observatories routinely record several hours of video material every day. Manual processing of such large amounts of video has become a major bottleneck for scientific research based on this data. We have developed an automated system that detects, tracks, and classifies objects that are of potential interest for human video annotators. By pre-selecting salient targets for track initiation using a selective attention algorithm, we reduce the complexity of multi-target tracking. Then, if an object is tracked for several frames, a visual event is created and passed to a Bayesian classifier utilizing a Gaussian mixture model to determine the object class of the detected event

Standards-Based Plug & Work for Instruments in Ocean Observing Systems

Ocean observing systems may include a wide variety of sensor and instrument types, each with its own capabilities, communication protocols, and data formats. Connecting disparate devices into a network typically requires specialized software drivers that translate command and data between the protocols of the individual instruments, and that of the platform on which they are installed. In addition, such platforms typically require extensive manual configuration to match the driver software and other operational details of each network port to a specific connected instrument. In this paper, we describe an approach to “plug & work” interoperability, using standardized protocols to greatly reduce the amount of instrument-specific software and manual configuration required for connecting instruments to an observatory system. Our approach has two main components. First, we use the sensor interface descriptor (SID) model, based on the Open Geospatial Consortiums (OGC) SensorML standard, to describe each instruments protocol and data format, and to provide a generic driver/parser. Second, a new OGC standard known as the programmable underwater connector with knowledge (PUCK) protocol enables storage and retrieval of the SID file from the instrument itself. We demonstrate and evaluate our approach by applying it to three commonly used marine instruments in the OBSEA (Barcelona, Spain) observatory test bed.

An Automated Visual Event Detection System for Cabled Observatory Video

The permanent presence of underwater cameras on cabled observatories, such as the Victoria Experimental Network Under the Sea (VENUS) and Eye-In-The-Sea (EITS) on Monterey Accelerated Research System (MARS), will generate precious data that can move forward the boundaries of understanding the underwater world. However, sightings of underwater animal activities are rare, resulting in the recording of many hours of video with relatively few events of interest. Towards this end, an automated visual event detection system is in development at the Monterey Bay Aquarium Research Institute (MBARI) to address the problem of analyzing cabled observatory video. This paper describes the overall design of the development of a system to process video data and enable science users to analyze the results.

Instrument interface standards for interoperable ocean sensor networks

The utility and cost-effectiveness of instrument networks are enhanced by instrument interoperability. Todays oceanographic instruments are characterized by very diverse non-standard software protocols and data formats. This diversity of protocols poses serious challenges to integration of large-scale sensor networks. Standard instrument protocols are now being developed to address these challenges. Some of these standards apply at the IP-network level and enable integration of existing “lower level” proprietary instrument protocols and software components. Other approaches are intended to be implemented by the instrument device itself. These native instrument protocol standards offer the possibility of more uniform and simpler system architectures. We compare these various approaches, describe how they can be combined with one another, and describe some prototypes that implement them.

MBARI Technology for Self-Configuring Interoperable Ocean Observatories

The ocean science and engineering communities have identified some key requirements for large-scale ocean observatories at a recent ORION-sponsored workshop, and these requirements are being refined by the ORION project and others. MBARI has developed and deployed hardware and software technologies that address many of these requirements. In particular, we describe how these technologies address several key issues: (1) scalable integration, configuration, and management of large numbers of diverse instruments and data streams, (2) reliable association of instrument data and contextual metadata, and (3) development of observatory infrastructure and components that are interoperable among a variety of observatory architectures, including at-sea systems with relatively limited power and bandwidth availability. We focus on three technologies developed at MBARI. These technologies work together to enable MBARIs self-configuring self-describing MOOS mooring-based observatory. Yet these technologies have been designed to be largely independent of an observatorys physical implementation, and will be deployed for testing on the MARS cable-to-shore observatory test-bed. Moreover each of the technologies provide components that could be selectively used by other observatories. For example, PUCKs could be widely useful and are not dependent in any way on SIAM middleware or SSDS metadata structures. We also describe lessons learned during development and deployment of these technologies, and how policies and human-procedures interact with the new technologies. Finally, we discuss how these technologies are being refined through community efforts such as the emerging Marine Plug and Work Consortium and Marine Metadata Initiative

ODSS: A decision support system for ocean exploration

We have designed, built, tested and fielded a decision support system which provides a platform for situational awareness, planning, observation, archiving and data analysis. While still in development, our inter-disciplinary team of computer scientists, engineers, biologists and oceanographers has made extensive use of our system in at-sea experiments since 2010. The novelty of our work lies in the targeted domain, its evolving functionalities that closely tracks how ocean scientists are seeing the evolution of their own work practice, and its actual use by engineers, scientists and marine operations personnel. We describe the architectural elements and lessons learned over the more than two years use of the system.

An ocean observatory sensor network application

We describe our implementation of a novel deep ocean sensor network, the MBARI Free Ocean CO2 Enrichment (FOCE). FOCE is a system designed for installation in the deep ocean to enable manipulative experiments that explore the impact of deep ocean increase in CO2 and resulting pH change on ocean biogeochemistry and ecology. This system uses control feedback and pH sensors to inject CO2 into a small volume of seawater, thus creating a controlled environment per science requirements. To implement this system, we utilized the MBARI-developed network middleware known as “SIAM”, which provides a standardized interface to instruments on a sensor network. For the FOCE application we integrated Open Source DataTurbine (OSDT) into SIAM. OSDT provides asynchronous communication links between distributed components, and is particularly well-suited to streaming instrument data. Combined with the existing synchronous SIAM framework, these features enabled a straightforward and efficient architecture for our application. We describe how we achieved our goals of software reuse of infrastructure and instrument services, instrument-in-the-loop control, and rapid assembly of a scalable end-to-end sensor network system.

Ocean Observing System Instrument Network Infrastructure

We summarize results of a Workshop on Instrument Software Infrastructure held at MBARI, Moss Landing, California USA from September 13-15, 2004, jointly sponsored the National Science Foundation (NSF) and Ocean Research Interactive Observatory Networks (ORION) program. The Workshop included over fifty participants, including international participants from Germany, Canada, and Japan. This was one of the first technical workshops in the development of a series of ocean observatories under the US Ocean Observatory Initiative (OOI) being managed under the ORION program. The specific focus of this workshop was to define the standard requirements to be met by software infrastructure for sensors, instruments and platforms for observing systems in the ORION program. These requirements include the issues of configuring, interfacing, and managing devices, including sensors and actuators, to a networked based observing system as well as managing the resources necessary to support such devices. The topics include the capability of supporting plug-and-work instrumentation using service oriented network architecture. A major issue addressed is the observatory infrastructure requirements necessary for managing data and metadata coming from sensors and instruments of the observatory in support of an integrated data management system

Towards automating underwater measurement of fish length: a comparison of semi-automatic and manual stereo–video measurements

Underwater stereo-video systems are widely used for counting and measuring fish in aquaculture, fisheries, and conservation management. Length measurements are generated from stereo-video recordings by a software operator using a mouse to locate the head and tail of a fish in synchronized pairs of images. This data can be used to compare spatial and temporal changes in the mean length and biomass or frequency distributions of populations of fishes. Since the early 1990s stereo-video has also been used for measuring the lengths of fish in aquaculture for quota and farm management. However, the costs of the equipment, software, the time, and salary costs involved in post processing imagery manually and the subsequent delays in the availability of length information inhibit the adoption of this technology. We present a semi-automatic method for capturing stereo-video measurements to estimate the lengths of fish. We compare the time taken to make measurements of the same fish measured manually from stereo-video imagery to that measured semi-automatically. Using imagery recorded during transfers of Southern Bluefin Tuna (SBT) from tow cages to grow out cages, we demonstrate that the semi-automatic algorithm developed can obtain fork length measurements with an error of less than 1% of the true length and with at least a sixfold reduction in operator time in comparison to manual measurements. Of the 22 138 SBT recorded we were able to measure 52.6% (11 647) manually and 11.8% (2614) semi-automatically. For seven of the eight cage transfers recorded there were no statistical differences in the mean length, weight, or length frequency between manual and semi-automatic measurements. When the data were pooled across the eight cage transfers, there was no statistical difference in mean length or weight between the stereo-video-based manual and semi-automated measurements. Hence, the presented semi-automatic system can be deployed to significantly reduce the cost involved in ad