Meng-Che Chuang

Tracking Live Fish From Low-Contrast and Low-Frame-Rate Stereo Videos

Nonextractive fish abundance estimation with the aid of visual analysis has drawn increasing attention. Unstable illumination, ubiquitous noise, and low-frame-rate (LFR) video capturing in the underwater environment, however, make conventional tracking methods unreliable. In this paper, we present a multiple fish-tracking system for low-contrast and LFR stereo videos with the use of a trawl-based underwater camera system. An automatic fish segmentation algorithm overcomes the low-contrast issues by adopting a histogram backprojection approach on double local-thresholded images to ensure an accurate segmentation on the fish shape boundaries. Built upon a reliable feature-based object matching method, a multiple-target tracking algorithm via a modified Viterbi data association is proposed to overcome the poor motion continuity and frequent entrance/exit of fish targets under LFR scenarios. In addition, a computationally efficient block-matching approach performs successful stereo matching that enables an automatic fish-body tail compensation to greatly reduce segmentation error and allows for an accurate fish length measurement. Experimental results show that an effective and reliable tracking performance for multiple live fish with underwater stereo cameras is achieved.

Automatic fish segmentation via double local thresholding for trawl-based underwater camera systems

This paper describes an automatic segmentation algorithm for fish sampled by a trawl-based underwater camera system. To overcome the problem caused by very low brightness contrast between fish and their underwater background with dynamically changing luminance, our proposed algorithm adopts an innovative histogram backprojection procedure on double local-thresholded images to ensure a reliable segmentation on the fish shape boundaries. The thresholded results are further validated by area and variance criteria to remove unwanted objects. Finally, a post-processing step is applied to refine the segmentation. Promising results, as validated by expert-generated ground truth data, were obtained via our proposed algorithm.

A Feature Learning and Object Recognition Framework for Underwater Fish Images

Live fish recognition is one of the most crucial elements of fisheries survey applications where the vast amount of data is rapidly acquired. Different from general scenarios, challenges to underwater image recognition are posted by poor image quality, uncontrolled objects and environment, and difficulty in acquiring representative samples. In addition, most existing feature extraction techniques are hindered from automation due to involving human supervision. Toward this end, we propose an underwater fish recognition framework that consists of a fully unsupervised feature learning technique and an error-resilient classifier. Object parts are initialized based on saliency and relaxation labeling to match object parts correctly. A non-rigid part model is then learned based on fitness, separation, and discrimination criteria. For the classifier, an unsupervised clustering approach generates a binary class hierarchy, where each node is a classifier. To exploit information from ambiguous images, the notion of partial classification is introduced to assign coarse labels by optimizing the benefit of indecision made by the classifier. Experiments show that the proposed framework achieves high accuracy on both public and self-collected underwater fish images with high uncertainty and class imbalance.

Underwater Fish Tracking for Moving Cameras Based on Deformable Multiple Kernels

Fishery surveys that call for the use of single or multiple underwater cameras have been an emerging technology as a nonextractive mean to estimate the abundance of fish stocks. Tracking live fish in an open aquatic environment posts challenges that are different from general pedestrian or vehicle tracking in surveillance applications. In many rough habitats, fish are monitored by cameras installed on moving platforms, where tracking is even more challenging due to inapplicability of background models. In this paper, a novel tracking algorithm based on the deformable multiple kernels is proposed to address these challenges. Inspired by the deformable part model technique, a set of kernels is defined to represent the holistic object and several parts that are arranged in a deformable configuration. Color histogram, texture histogram, and the histogram of oriented gradients (HOGs) are extracted and serve as object features. Kernel motion is efficiently estimated by the mean-shift algorithm on color and texture features to realize tracking. Furthermore, the HOG-feature deformation costs are adopted as soft constraints on kernel positions to maintain the part configuration. Experimental results on practical video set from underwater moving cameras show the reliable performance of the proposed method with much less computational cost comparing with state-of-the-art techniques.

Multiple fish tracking via Viterbi data association for low-frame-rate underwater camera systems

Non-extractive fish abundance estimation with the aid of visual analysis has drawn increasing attention. Low frame rate and variable illumination in the underwater environment, however, makes conventional tracking methods unreliable. In this paper, a robust multiple fish tracking system for low-frame-rate underwater stereo cameras is proposed. With the result of fish segmentation, a computationally efficient block-matching method is applied to perform successful stereo matching. A multiple-feature matching cost function is utilized to give a simple but effective metric for finding the temporal match of each target. Built upon reliable stereo matching, a multiple-target tracking algorithm via the Viterbi data association is developed to overcome the poor motion continuity of targets. Experimental results show that an accurate underwater live fish tracking result with stereo cameras is achieved.

Supervised and Unsupervised Feature Extraction Methods for Underwater Fish Species Recognition

Automated fish species identification in open aquatic habitats based on video analytics is the primary area of research in camera-based fisheries surveys. Finding informative features for these analyses, however, is fundamentally challenging due to poor quality of underwater imagery and strong visual similarity among species. In this paper, we compare two different fish feature extraction methods, namely the supervised and unsupervised approaches, which are then applied to a hierarchical partial classification framework. Several specified anatomical parts of fish are automatically located to generate the supervised feature descriptors. For unsupervised feature extraction, a scale-invariant object part learning algorithm is proposed to discover common shape of body parts and then extract appearance, location and size information of each part. Experiments show that the unsupervised approach achieves better recognition performance on live fish images collected by trawl-based cameras.

Recognizing live fish species by hierarchical partial classification based on the exponential benefit

Live fish recognition in open aquatic habitats suffers from the high uncertainty in many of the data. To alleviate this problem without discarding those data, the system should learn a species hierarchy so that high-level labels can be assigned to ambiguous data. In this paper, a systematic hierarchical partial classification algorithm is therefore proposed for underwater fish species recognition. Partial classification is applied at each level of the species hierarchy so that the coarse-to-fine categorization stops once the decision confidence is low. By defining the exponential benefit function, we formulate the selection of decision threshold as an optimization problem. Also, attributes from important fish anatomical parts are focused to generate discriminative feature descriptors. Experiments show that the proposed method achieves an accuracy up to 94%, with partial decision rate less than 5%, on underwater fish images with high uncertainty and class imbalance.

Aggregated segmentation of fish from conveyor belt videos

Automation of fishery survey through the aid of visual analysis has received increasing attention. In this paper, a novel algorithm for the aggregated segmentation of fish images taken from conveyor belt videos is proposed. The watershed algorithm driven by an automatic marker generation scheme successfully separates clustered fish images without damaging their boundaries. A target selection based on appearance classification then rejects non-fish objects. By applying histogram backprojection and kernel density estimation, an innovative algorithm for combining object masks of one tracked fish from multiple frames into a refined single one is also proposed. Experimental results show that accurate fish segmentation from conveyor belt videos is achieved.