Roberto Di Salvo

Automatic fish classification for underwater species behavior understanding

The aim of this work is to propose an automatic fish classification system that operates in the natural underwater environment to assist marine biologists in understanding subehavior. Fish classification is performed by combining two types of features: 1) Texture features extracted by using statistical moments of the gray-level histogram, spatial Gabor filtering and properties of the co-occurrence matrix and 2) Shape Features extracted by using the Curvature Scale Space transform and the histogram of Fourier descriptors of boundaries. An affine transformation is also applied to the acquired images to represent fish in 3D by multiple views for the feature extraction. The system was tested on a database containing 360 images of ten different species achieving as average correct rate of about 92%. Then, fish trajectories extracted using the proposed fish classification combined with a tracking system, are analyzed in order to understand anomalous behavior. In detail, the tracking layer computer fish trajectories, the classification layer associates trajectories to fish species and then by clustering these trajectories we are able to detect unusual fish behaviors to be further investigated by marine biologists.

An innovative web-based collaborative platform for video annotation

Large scale labeled datasets are of key importance for the development of automatic video analysis tools as they, from one hand, allow multi-class classifiers training and, from the other hand, support the algorithms’ evaluation phase. This is widely recognized by the multimedia and computer vision communities, as witnessed by the growing number of available datasets; however, the research still lacks in annotation tools able to meet user needs, since a lot of human concentration is necessary to generate high quality ground truth data. Nevertheless, it is not feasible to collect large video ground truths, covering as much scenarios and object categories as possible, by exploiting only the effort of isolated research groups. In this paper we present a collaborative web-based platform for video ground truth annotation. It features an easy and intuitive user interface that allows plain video annotation and instant sharing/integration of the generated ground truths, in order to not only alleviate a large part of the effort and time needed, but also to increase the quality of the generated annotations. The tool has been on-line in the last four months and, at the current date, we have collected about 70,000 annotations. A comparative performance evaluation has also shown that our system outperforms existing state of the art methods in terms of annotation time, annotation quality and system’s usability.

Understanding fish behavior during typhoon events in real-life underwater environments

The study of fish populations in their own natural environment is a task that has usually been tackled in invasive ways which inevitably influenced the behavior of the fish under observation. Recent projects involving the installation of permanent underwater cameras (e.g. the Fish4Knowledge (F4K) project, for the observation of Taiwan’s coral reefs) allow to gather huge quantities of video data, without interfering with the observed environment, but at the same time require the development of automatic processing tools, since manual analysis would be impractical for such amounts of videos. Event detection is one of the most interesting aspects from the biologists’ point of view, since it allows the analysis of fish activity during particular events, such as typhoons. In order to achieve this goal, in this paper we present an automatic video analysis approach for fish behavior understanding during typhoon events. The first step of the proposed system, therefore, involves the detection of “typhoon” events and it is based on video texture analysis and on classification by means of Support Vector Machines (SVM). As part of our behavior understanding efforts, trajectory extraction and clustering have been performed to study the differences in behavior when disruptive events happen. The integration of event detection with fish behavior understanding surpasses the idea of simply detecting events by low-level features analysis, as it supports the full semantic comprehension of interesting events.

Generating reliable video annotations by exploiting the crowd

In computer vision and machine learning, the availability of annotated datasets is of crucial importance for both learning and performance evaluation. However, annotating visual datasets is a tedious and error-prone task and computer vision researchers usually dedicate a large amount of their time for collecting and generating annotations, which most of the time cannot be re-used in other scenarios. In this paper, we propose a simple, but effective, interactive video object segmentation method exploiting large noisy data gathered from crowd of users while playing a web game. Experimental results, carried out on two challenging video benchmarks, show how it is possible to generate reliable object segmentations in videos with a small human effort, achieving an accuracy comparable to the one obtained with manually-labeled annotations and also outperforming state-of-the-art video object segmentation approaches.

People Flow Control Using Cellular Automata and Computer Vision Technologies

The paper proposes an automatic fuzzy system to control in real time people flows in crowded scenes. An integration of cellular automata and computer vision technologies is envisaged to control the people flow in different conditions and in real time. An adaptive control paradigm has been adopted to manage real scenarios since they usually differ slightly from the theoretical ones. The proposed fuzzy control is not only simple, but it may incorporate measurements derived from both fast and accurate computer vision techniques and perceptions of people operating on the field.

Electronic Multimedia Retrieval Systems: Architecture, Features and Novel Directions

Internet growth and the development of web technologies, transformed the Web from a purely research network to an essential every day tool. In particular, the strong demand and availability of multimedia resources combined to the intrinsic semantic gap, favoured the evolution of content based multimedia retrieval systems. The request of flexibility that is required from the multimedia retrieval users and the semantic gap have been the spring for the evolution of the new generation of media retrieval system. This paper aims at illustrating the architecture and features of electronic multimedia retrieval by pointing out how these systems are provided with a suitable semantic layer to improve the system performance, where annotation, relevance feedback and semantics driven object recognition increase the precision of the multimedia retrieval. How the clustering of the multimedia materials may be used to increase the recall and to support the reuse of multimedia material for developing new artefacts is also discussed, thus envisaging novel research directions.