Maia Hoeberechts

Automatic fish counting system for noisy deep-sea videos

In this paper, we present a non-invasive method of counting fish in their natural habitat using automated analysis of video data. Our approach uses three modular components to preprocess, detect, and track the fish. The preprocessing reduces noise present in the image while enhancing the fish using several different techniques. The fish detection is based on two background subtraction algorithms which are computed independently and later combined with logical operations. The tracking is then carried out by a heuristic blob tracking algorithm. The paper presents a description of the proposed counting method as well as its experimental validation.

Expert, Crowd, Students or Algorithm: who holds the key to deep‐sea imagery ‘big data’ processing?

1.Recent technological development has increased our capacity to study the deep sea and the marine benthic realm, particularly with the development of multidisciplinary seafloor observatories. Since 2006, Ocean Networks Canada cabled observatories, have acquired nearly 65 TB and over 90,000 hours of video data from seafloor cameras and Remotely Operated Vehicles (ROVs). Manual processing of these data is time-consuming and highly labour-intensive, and cannot be comprehensively undertaken by individual researchers. These videos are a crucial source of information for assessing natural variability and ecosystem responses to increasing human activity in the deep sea. 2.We compared the performance of three groups of humans and one computer vision algorithm in counting individuals of the commercially important sablefish (or black cod) Anoplopoma fimbria, in recorded video from a cabled camera platform at 900 m depth in a submarine canyon in the Northeast Pacific. The first group of human observers were untrained volunteers recruited via a crowdsourcing platform and the second were experienced university students, who performed the task for their ichthyology class. Results were validated against counts obtained from a scientific expert. 3.All groups produced relatively accurate results in comparison to the expert and all succeeded in detecting patterns and periodicities in fish abundance data. Trained volunteers displayed the highest accuracy and the algorithm the lowest. 4.As seafloor observatories increase in number around the world, this study demonstrates the value of a hybrid combination of crowdsourcing and computer vision techniques as a tool to help process large volumes of imagery to support basic research and environmental monitoring. Reciprocally, by engaging large numbers of online participants in deep-sea research, this approach can contribute significantly to ocean literacy and informed citizen input to policy development.

Detection of salient events in large datasets of underwater video

The aim of this work is to perform the automatic detection of events of interest, in this case defined as animal motion, in deep-sea videos and then to use the detected events as the basis for creating video abstracts. Video is collected by seafloor cameras connected to a cabled observatory network which provides power to the lights and sensors and enables two-way communication with the cameras. Continuous power and connectivity on the network permit high volumes of data to be collected. Such video data is of importance for marine biologists who are able to remotely observe species in the deep-sea environment through scheduled recordings of the video data. It is extremely time consuming for researchers looking for particular events of interest to manually search in the video database, and therefore, our study focuses on automatic detection of these events. Our approach is based on the Laptev spatio-temporal interest points detection method [1]. The output of the analysis is a summary video clip that contains all detected salient events with their associated start and end frames. We report experimental results on video abstraction using a database of videos from the NEPTUNE Canada cabled observatory.

Real-time quality control practices on a cabled ocean observatory: A 10 year case study in Saanich Inlet, BC

Timely identification and minimization of data quality issues in long-term datasets across multiple instrument deployments at a single site pose a fundamental challenge to cabled ocean observatory operations. Ocean Network Canada (ONC) has developed extensive real-time Quality Assurance (QA) and Quality Control (QC) methodologies, including both automated and manual techniques and workflow. ONCs QA/QC approach was developed with the key objectives of maintaining consistency both within a single dataset and simultaneously among a collection of datasets at each site. This paper highlights a case study of ONCs longest deployment site in Saanich Inlet, BC, Canada which has been continuously instrumented from 2006 through 2016. We focus on real-time QA/QC practices applied to conductivity-temperature depth (CTD) instruments. The methodologies and results presented in this paper have been generalized to other ONC observatory sites, and can serve as a practical guide to others developing real-time QA/QC systems for long-term cabled observatory deployments of CTD instruments.

Obstacle Detection for Image-Guided Surface Water Navigation

Maritime safety is an issue of concern when operating a small to medium sized sailboat, and the appearance of hazards in the navigational route like floating logs can lead to a severe collision if undetected. As a precautionary measure to prevent such a collision with a log, a 2D vision-based detection system algorithm presented to detect these floating hazards. We take a combined approach involving predictive mapping by linear regression and saliency detection. The proposed method has been evaluated using precision and recall measures. The evaluation results show that the algorithm is robust and computationally non-intensive for future implementation of a real time on-board obstacle detection system for autonomous and computer-assisted sailboat navigation.

The Power of Seeing: Experiences using video as a deep-sea engagement and education tool

This paper describes initiatives underway at Ocean Networks Canada (ONC) in using video data as a tool for public engagement and education: live video streams from cameras on the seafloor, citizen science using video data, audience participation in deep-sea expeditions, and K-12 engagement through the Ocean Sense program. Live and archived video attract the majority of user traffic on ONCs website and can be leveraged to direct the viewers to other content and messaging, enhancing their engagement with the deep-sea environment. Public interest in scientific discovery creates a user base for citizen science initiatives, while educational audiences can be connected to both realtime and asynchronous learning materials. The power of live connections is also harnessed during research expeditions, which can be extended from the ship and the seafloor directly into the classroom.

Data quality control and quality assurance practices for Ocean Networks Canada observatories

Cabled observatory installations permit the acquisition of large volumes of continuous, high-resolution data from in-situ instruments. This type of data acquisition presents new challenges and opportunities in the development of data quality assurance and quality control (QAQC) measures. Ocean Networks Canada (ONC) operates the world-leading NEPTUNE and VENUS cabled ocean observing systems in the NE Pacific, and a small seafloor observatory in the Canadian Arctic. ONC collects high-resolution, real-time data on physical, chemical, biological, and geological aspects of the ocean over long time periods, supporting research on complex earth and ocean processes with innovative methods. High quality research depends on high quality data, which in turn depends on robust data quality control practices. For the data to be useful to potential end users, they must be qualified under accepted international standards with additional metadata pertaining to methods of measurement, instrument calibrations, and subsequent data processing included. Ocean Network Canadas QAQC methodology presented here has been developed with the key objectives of maintaining consistency within a single data set and within a collection of data sets. The QAQC model also ensures that the end user has sufficient information on the quality and errors of the data to assess its suitability for their use. The data QAQC procedures and tools have the capability to associate distinctly different but related types of information with data to provide a systematic and timely examination of the measurements. Efforts have been taken to develop efficient and accurate data QAQC techniques and tools to ensure quality data delivery to the end users in a timely manner. The large volume of data coming from extremely complex, diverse, and unpredictable ocean environments has resulted in many challenges as well as opportunities to develop efficient and informative tools for data QAQC at ONC. This paper describes the current and future steps that ONC is undertaking to ensure that data delivered by the observatories are of high quality, easily accessible, and reliable.

Evolutionary Computational Methods for Optimizing the Classification of Sea Stars in Underwater Images

Using video and imagery for assessing the distribution and abundance of marine organisms is a valuable sampling method in that it is non-invasive and permits large volumes of data to be acquired. Quickly and accurately processing large volumes of imagery is a challenge for human analysts, which motivates the need for automated processing methods. In this paper, we present a method for the automatic classification of sea stars in underwater images. The method uses a very small number of features and is efficient. The classification process is optimized by using evolutionary computational methods. Experimental results show excellent performance of our proposed optimized classification approach.

On the Degree of Nondeterminism of Tree Adjoining Languages and Head Grammar Languages

The degree of nondeterminism is a measure of syntactic complexity which was investigated for parallel and sequential rewriting systems. In this paper, we consider the degree of nondeterminsm for tree adjoining grammars and their languages and head grammars and their languages. We show that a degree of nondeterminism of 2 suffices for both formalisms in order to generate all languages in their respective language families. Furthermore, we show that deterministic tree adjoining grammars (those with degree of nondeterminism equal to 1), can generate non-context-free languages, in contrast to deterministic head grammars which can only generate languages containing a single word.

Sparse Coding for Efficient Bioacoustic Data Mining: Preliminary Application to Analysis of Whale Songs

Bioacoustic monitoring, such as surveys of animal populations and migration, needs efficient data mining methods to extract information from large datasets covering multi-year and multi-location recordings. This paper introduces a method for sparse coding of bioacoustic recordings in order to efficiently compress and automatically extract patterns in data. We demonstrate the proposed method on the analysis of humpback whale songs. Previous work suggests that the structure of these songs can be characterized by successive vocalizations called sound units. Most of these analyses are currently done with expert intervention, but the volume of recordings drive the need for automated methods for sound unit classification. This paper proposes that sparse coding of the song at different time scales supports the distinction of stable song components versus those which evolve year to year. The approach is summarized as: first, an unsupervised method is used to encode the entire bioacoustic dataset into a dictionary, second, sparse coding is used to limit the number of elements in the dictionary, third, salient features are identified using the Lasso algorithm, and finally, an interpretation of the evolving and stable components of the songs is derived, supporting an analysis of year to year variation. It is shown that shorter codes are more stable, occurring with similar frequency across two consecutive years, while the occurrence of longer units varies across years as expected based on the prior manual analysis. 250 ms segments appear to be an appropriate length for encoding stable features of whale songs, possibly corresponding to subunits. We conclude by exploring further possibilities of the application of this method for biopopulation analysis.