Kresimir Williams

Tracking Live Fish From Low-Contrast and Low-Frame-Rate Stereo Videos

Nonextractive fish abundance estimation with the aid of visual analysis has drawn increasing attention. Unstable illumination, ubiquitous noise, and low-frame-rate (LFR) video capturing in the underwater environment, however, make conventional tracking methods unreliable. In this paper, we present a multiple fish-tracking system for low-contrast and LFR stereo videos with the use of a trawl-based underwater camera system. An automatic fish segmentation algorithm overcomes the low-contrast issues by adopting a histogram backprojection approach on double local-thresholded images to ensure an accurate segmentation on the fish shape boundaries. Built upon a reliable feature-based object matching method, a multiple-target tracking algorithm via a modified Viterbi data association is proposed to overcome the poor motion continuity and frequent entrance/exit of fish targets under LFR scenarios. In addition, a computationally efficient block-matching approach performs successful stereo matching that enables an automatic fish-body tail compensation to greatly reduce segmentation error and allows for an accurate fish length measurement. Experimental results show that an effective and reliable tracking performance for multiple live fish with underwater stereo cameras is achieved.

Automatic fish segmentation via double local thresholding for trawl-based underwater camera systems

This paper describes an automatic segmentation algorithm for fish sampled by a trawl-based underwater camera system. To overcome the problem caused by very low brightness contrast between fish and their underwater background with dynamically changing luminance, our proposed algorithm adopts an innovative histogram backprojection procedure on double local-thresholded images to ensure a reliable segmentation on the fish shape boundaries. The thresholded results are further validated by area and variance criteria to remove unwanted objects. Finally, a post-processing step is applied to refine the segmentation. Promising results, as validated by expert-generated ground truth data, were obtained via our proposed algorithm.

A Feature Learning and Object Recognition Framework for Underwater Fish Images

Live fish recognition is one of the most crucial elements of fisheries survey applications where the vast amount of data is rapidly acquired. Different from general scenarios, challenges to underwater image recognition are posted by poor image quality, uncontrolled objects and environment, and difficulty in acquiring representative samples. In addition, most existing feature extraction techniques are hindered from automation due to involving human supervision. Toward this end, we propose an underwater fish recognition framework that consists of a fully unsupervised feature learning technique and an error-resilient classifier. Object parts are initialized based on saliency and relaxation labeling to match object parts correctly. A non-rigid part model is then learned based on fitness, separation, and discrimination criteria. For the classifier, an unsupervised clustering approach generates a binary class hierarchy, where each node is a classifier. To exploit information from ambiguous images, the notion of partial classification is introduced to assign coarse labels by optimizing the benefit of indecision made by the classifier. Experiments show that the proposed framework achieves high accuracy on both public and self-collected underwater fish images with high uncertainty and class imbalance.

Underwater Fish Tracking for Moving Cameras Based on Deformable Multiple Kernels

Fishery surveys that call for the use of single or multiple underwater cameras have been an emerging technology as a nonextractive mean to estimate the abundance of fish stocks. Tracking live fish in an open aquatic environment posts challenges that are different from general pedestrian or vehicle tracking in surveillance applications. In many rough habitats, fish are monitored by cameras installed on moving platforms, where tracking is even more challenging due to inapplicability of background models. In this paper, a novel tracking algorithm based on the deformable multiple kernels is proposed to address these challenges. Inspired by the deformable part model technique, a set of kernels is defined to represent the holistic object and several parts that are arranged in a deformable configuration. Color histogram, texture histogram, and the histogram of oriented gradients (HOGs) are extracted and serve as object features. Kernel motion is efficiently estimated by the mean-shift algorithm on color and texture features to realize tracking. Furthermore, the HOG-feature deformation costs are adopted as soft constraints on kernel positions to maintain the part configuration. Experimental results on practical video set from underwater moving cameras show the reliable performance of the proposed method with much less computational cost comparing with state-of-the-art techniques.

Multiple fish tracking via Viterbi data association for low-frame-rate underwater camera systems

Non-extractive fish abundance estimation with the aid of visual analysis has drawn increasing attention. Low frame rate and variable illumination in the underwater environment, however, makes conventional tracking methods unreliable. In this paper, a robust multiple fish tracking system for low-frame-rate underwater stereo cameras is proposed. With the result of fish segmentation, a computationally efficient block-matching method is applied to perform successful stereo matching. A multiple-feature matching cost function is utilized to give a simple but effective metric for finding the temporal match of each target. Built upon reliable stereo matching, a multiple-target tracking algorithm via the Viterbi data association is developed to overcome the poor motion continuity of targets. Experimental results show that an accurate underwater live fish tracking result with stereo cameras is achieved.

Supervised and Unsupervised Feature Extraction Methods for Underwater Fish Species Recognition

Automated fish species identification in open aquatic habitats based on video analytics is the primary area of research in camera-based fisheries surveys. Finding informative features for these analyses, however, is fundamentally challenging due to poor quality of underwater imagery and strong visual similarity among species. In this paper, we compare two different fish feature extraction methods, namely the supervised and unsupervised approaches, which are then applied to a hierarchical partial classification framework. Several specified anatomical parts of fish are automatically located to generate the supervised feature descriptors. For unsupervised feature extraction, a scale-invariant object part learning algorithm is proposed to discover common shape of body parts and then extract appearance, location and size information of each part. Experiments show that the unsupervised approach achieves better recognition performance on live fish images collected by trawl-based cameras.

Recognizing live fish species by hierarchical partial classification based on the exponential benefit

Live fish recognition in open aquatic habitats suffers from the high uncertainty in many of the data. To alleviate this problem without discarding those data, the system should learn a species hierarchy so that high-level labels can be assigned to ambiguous data. In this paper, a systematic hierarchical partial classification algorithm is therefore proposed for underwater fish species recognition. Partial classification is applied at each level of the species hierarchy so that the coarse-to-fine categorization stops once the decision confidence is low. By defining the exponential benefit function, we formulate the selection of decision threshold as an optimization problem. Also, attributes from important fish anatomical parts are focused to generate discriminative feature descriptors. Experiments show that the proposed method achieves an accuracy up to 94%, with partial decision rate less than 5%, on underwater fish images with high uncertainty and class imbalance.

The potential effects of substrate type, currents, depth and fishing pressure on distribution, abundance, diversity, and height of cold-water corals and sponges in temperate, marine waters

Deep-sea benthic environments can be home to diverse communities of corals and sponges which are important habitat for marine fishes and invertebrates. From 2010 to 2014, underwater camera surveys in the Aleutian Islands were completed with the objective of evaluating potential effects of substrate type, tidal currents, depth, and fishing pressure on distribution, abundance, diversity, and size of structure-forming invertebrate (SFI) communities. The presence of rocky substrates was associated with higher probability of presence, higher density, and taller SFI. Multivariate analyses showed community structure changed over gradients of substrate, tidal currents, and longitude, with sea whips typically occupying deeper depths and mostly unconsolidated substrates, while other corals were largely found in rocky, shallower areas. These patterns were also reflected in co-occurrence analyses indicating sea whips were negatively associated with other SFI taxa. Most SFI occupied areas of swift tidal currents; however, heights of individual SFI decreased with increasing tidal currents. Coral and sponge densities at some sites in this study exceeded densities reported from other global coral and sponge habitats. Identifying the environmental conditions leading to high-density and high-diversity SFI communities is important for management of fisheries and evaluating potential impacts of climate change in benthic marine ecosystems.

Shrinking Encoding with Two-Level Codebook Learning for Fine-Grained Fish Recognition

Bag-of-features (BoF) shows a great power in representing images for image classification. Many codebook learning methods have been developed to find discriminative parts of images for fine-grained recognition. Built upon BoF framework, we propose a novel approach for finegrained fish recognition with two-level codebook learning by shrinking coding coefficients. In the framework, only the maximum-valued coefficient will be maintained in the local spatial region if followed by max pooling strategy. However, the maximum-valued coefficient may result from a local descriptor which is not discriminative among fine-grained classes, resulting in difficulty in classification. In this paper, a two-level codebook is learned to represent the importance between the local descriptor and each codeword in its corresponding k-nearest neighbors. A shrinkage function is also introduced to shrink unrelated coefficients after encoding. Our experimental results show that the proposed method achieves significant performance improvement for fine-grained fish recognition tasks.

Closed-Loop Tracking-by-Detection for ROV-Based Multiple Fish Tracking

Fish abundance estimation with the aid of visual analysis has drawn increasing attention based on the underwater videos from a remotely-operated vehicle (ROV). We build a novel fish tracking and counting system followed by tracking-by-detection framework. Since fish may keep entering or leaving the field of view (FOV), an offline trained deformable part model (DPM) fish detector is adopted to detect live fish from video data. Besides that, a multiple kernel tracking approach is used to associate the same object across consecutive frames for fish counting purpose. However, due to the diversity of fish poses, the deformation of fish body shape and the color similarity between fish and background, the detection performance greatly decreases, resulting in a large error in tracking and counting. To deal with such issue, we propose a closed-loop mechanism between tracking and detection. First, we arrange detection results into tracklets and extract motion features from arranged tracklets. A Bayesian classifier is then applied to remove unreliable detections. Finally, the tracking results are modified based on the reliable detections. This proposed strategy effectively addresses the false detection problem and largely decreases the tracking error. Favorable performance is achieved by our proposed closed-loop between tracking and detection on the real-world ROV videos.