Silvio Jamil Ferzoli Guimarães

Video segmentation based on 2D image analysis

The video segmentation problem consists in the identification of the boundary between consecutive shots. The common approach to solve this problem is based on dissimilarity measures between frames. In this work, the video segmentation problem is transformed into a problem of pattern detection, where each video event is transformed into a different pattern on a 2D image, called visual rhythm, obtained by a specific transformation. In our analysis we use topological and morphological tools to detect cuts. Also, we use discrete line analysis and max tree analysis to detect fade transitions and flashes, respectively. We present a comparative analysis of our method for cut detection with respect to some other methods, which shows the better results of our method.

A hierarchical image segmentation algorithm based on an observation scale

Hierarchical image segmentation provides a region-oriented scale-space, i.e., a set of image segmentations at different detail levels in which the segmentations at finer levels are nested with respect to those at coarser levels. Most image segmentation algorithms, such as region merging algorithms, rely on a criterion for merging that does not lead to a hierarchy. In addition, for image segmentation, the tuning of the parameters can be difficult. In this work, we propose a hierarchical graph based image segmentation relying on a criterion popularized by Felzenszwalb and Huttenlocher. Quantitative and qualitative assessments of the method on Berkeley image database shows efficiency, ease of use and robustness of our method.

Bipartite graph matching for video clip localization

In this paper, we present a new efficient method for accurate eye localization in color images. Our algorithm is based on robust feature filtering and explicit geometric clustering. This combination enhances localization speed and robustness by relying on geometric relationships between pixel clusters instead of other properties extracted from the image. Furthermore, its efficiency makes it well suited for implementation in low performance devices, such as cell phones and PDAs. Experiments were conducted with 1532 face images taken from a CCD camera under (real-life) varying illumination, pose and expression conditions. The proposed method presented a localization rate of 94.125% under such circumstances.Video clip localization consists in identifying real positions of a specific video clip in a video stream. To cope with this problem, we propose a new approach considering the maximum cardinality matching of a bipartite graph to measure video clip similarity with a target video stream which has not been prep recessed. We show that our approach locates edited video clips, but it does not deal with insertion and removal of frames/shots, allowing only changes in the temporal order of frames/shots. All experiments performed in this work have achieved 100% of precision for two different video datasets. And according to those experiments, our method can achieve a global recall rate of 90%.

A mid-level video representation based on binary descriptors: A case study for pornography detection

Abstract With the growing amount of inappropriate content on the Internet, such as pornography, arises the need to detect and filter such material. The reason for this is given by the fact that such content is often prohibited in certain environments (e.g., schools and workplaces) or for certain publics (e.g., children). In recent years, many works have been mainly focused on detecting pornographic images and videos based on visual content, particularly on the detection of skin color. Although these approaches provide good results, they generally have the disadvantage of a high false positive rate since not all images with large areas of skin exposure are necessarily pornographic images, such as people wearing swimsuits or images related to sports. Local feature based approaches with Bag-of-Words models (BoW) have been successfully applied to visual recognition tasks in the context of pornography detection. Even though existing methods provide promising results, they use local feature descriptors that require a high computational processing time yielding high-dimensional vectors. In this work, we propose an approach for pornography detection based on local binary feature extraction and BossaNova image representation, a BoW model extension that preserves more richly the visual information. Moreover, we propose two approaches for video description based on the combination of mid-level representations namely BossaNova Video Descriptor (BNVD) and BoW Video Descriptor (BoW-VD). The proposed techniques are promising, achieving an accuracy of 92.40%, thus reducing the classification error by 16% over the current state-of-the-art local features approach on the Pornography dataset.

Representing local binary descriptors with BossaNova for visual recognition

Binary descriptors have recently become very popular in visual recognition tasks. This popularity is largely due to their low complexity and for presenting similar performances when compared to non binary descriptors, like SIFT. In literature, many researchers have applied binary descriptors in conjunction with mid-level representations (e.g., Bag-of-Words). However, despite these works have demonstrated promising results, their main problems are due to use of a simple mid-level representation and the use of binary descriptors in which rotation and scale invariance are missing. In order to address those problems, we propose to evaluate state-of-the-art binary descriptors, namely BRIEF, ORB, BRISK and FREAK, in a recent mid-level representation, namely BossaNova, which enriches the Bag-of-Words model, while preserving the binary descriptor information. Our experiments carried out in the challenging PASCAL VOC 2007 dataset revealed outstanding performances. Also, our approach shows good results in the challenging real-world application of pornography detection.

Graph-based hierarchical video segmentation based on a simple dissimilarity measure

Hierarchical video segmentation provides region-oriented scale-space, i.e., a set of video segmentations at different detail levels in which the segmentations at finer levels are nested with respect to those at coarser levels. In this work, the hierarchical video segmentation is transformed into a graph partitioning problem in which each part corresponds to one supervoxel of the video, and we present a new methodology for hierarchical video segmentation which computes a hierarchy of partitions by a reweighting of the original graph using a simple dissimilarity measure in which a not too coarse segmentation can be easily inferred. We also provide an extensive comparative analysis, considering quantitative assessments showing accuracy, ease of use, and temporal coherence of our methods - p-HOScale, cp-HOScale and 2cp-HOScale. According to the experiments, the hierarchy inferred by our methods produces good quantitative results when applied to video segmentation. Moreover, unlike to other tested methods, space and time cost of our methods are not influenced by the number of supervoxels to be computed.

Summarizing video sequence using a graph-based hierarchical approach

Video summarization is a simplification of video content for compacting the video information. The video summarization problem can be transformed into a clustering problem, in which some frames are selected to saliently represent the video content. In this work, we use a graph-based hierarchical clustering method for computing a video summary. In fact, the proposed approach, called HSUMM, adopts a hierarchical clustering method to generate a weight map from the frame similarity graph in which the clusters (or connected components of the graph) can easily be inferred. Moreover, the use of this strategy allows the application of a similarity measure between clusters during graph partition, instead of considering only the similarity between isolated frames. We also provide a unified framework for video summarization based on minimum spanning tree and weight maps in which HSUMM could be seen as an instance that uses a minimum spanning tree of frames and a weight map based on hierarchical observation scales computed over that tree. Furthermore, a new evaluation measure that assesses the diversity of opinions among users when they produce a summary for the same video, called Covering, is also proposed. During tests, different strategies for the identification of summary size and for the selection of keyframes were analyzed. Experimental results provide quantitative and qualitative comparison between the new approach and other popular algorithms from the literature, showing that the new algorithm is robust. Concerning quality measures, HSUMM outperforms the compared methods regardless of the visual feature used in terms of F-measure.

Evaluation of Morphological Hierarchies for Supervised Segmentation

We propose a quantitative evaluation of morphological hierarchies (quasi-flat zones, constraint connectivity, watersheds, observation scale) in a novel framework based on the marked segmentation problem. We created a set of automatically generated markers for the one object image datasets of Grabcut and Weizmann. In order to evaluate the hierarchies, we applied the same segmentation strategy by combining several parameters and markers. Our results, which shows important differences among the considered hierarchies, give clues to understand the behaviour of each method in order to choose the best one for a given application. The code and the marker datasets are available online.

A NEW DISSIMILARITY MEASURE FOR CUT DETECTION USING BIPARTITE GRAPH MATCHING

Cut detection is part of the video segmentation problem, and consists in identifying the boundary between two consecutive shots. In this case, when two consecutive frames are similar, they are considered to be in the same shot. This work presents an approach to cut detection using a new simple and efficient dissimilarity measure (which is also invariant to rotation and translation) based on the size of a bipartite graph matching. To establish some parameter values, a machine learning approach is used. Experimental results provides a comparison between the new approach and other popular algorithms from the literature, showing that the new algorithm is robust and has a high performance compared to other methods for cut detection.

Counting of Video Clip Repetitions using a Modified BMH Algorithm: Preliminary Results

In this work, we cope with the problem of identifying the number of repetitions of a specific video clip in a target video clip. Generally, the methods that deal with this problem can be subdivided into methods that use: (i) video signatures afterward the step of temporal video segmentation; and (ii) string matching algorithms afterward transformation of the video frame content into a feature values. Here, we propose a modification of the fastest exact string matching algorithm, the Boyer-Moore-Horspool, to count video clip repetitions. We also present some experiments to validate our approach, mainly if we are interested in found identical video clips according to spatial and temporal features