İlkay Atıl

Multimodal concept detection in broadcast media: KavTan

Concept detection stands as an important problem for efficient indexing and retrieval in large video archives. In this work, the KavTan System, which performs high-level semantic classification in one of the largest TV archives of Turkey, is presented. In this system, concept detection is performed using generalized visual and audio concept detection modules that are supported by video text detection, audio keyword spotting and specialized audio-visual semantic detection components. The performance of the presented framework was assessed objectively over a wide range of semantic concepts (5 high-level, 14 visual, 9 audio, 2 supplementary) by using a significant amount of precisely labeled ground truth data. KavTan System achieves successful high-level concept detection performance in unconstrained TV broadcast by efficiently utilizing multimodal information that is systematically extracted from both spatial and temporal extent of multimedia data.

Comparison of decision fusion methods for steganalysis

In this work, we compare the effect of using different decision fusion methods on the detection performance of a steganalysis system. It has been shown that different steganalysis features has different detection rates for different steganography methods. Therefore, steganalysis systems combine different steganalysis features to obtain the best detection rates. We compare four different fusion methods; feature concatenation, weighted fusion, neural networks and Support Vector Machines (SVM). We have observed that it is best to use a non-linear fusion method because SVM and neural networks gives the best detection rates when compared to linear fusion methods.

Scene nudity level detection with deep nets

In this paper, we present an approach that can detect scene nudity level with high precision using different deep net configurations. For this purpose, a recent approach [1] which has intense and very deep convolution layers is used. During net modelling, we strive to obtain most successful net configuration by comparing different Dropout models and image sizes -64 × 64, 128 × 128-. Additionally, leveraging the generalization capability of Support Vector Machine (SVM), improvement on success rate is demonstrated by retraining the features obtained at different output levels of the nets with SVM. At test and training stages, scene is investigated under three nudity levels: regular, semi-nudity and full-nudity. In order to evaluate false alarm rates of the net models, tests are conducted on different datasets which are SUN2012, LEAR Human and a dataset contains only semi-nudity samples besides validation set determined for each class. The results indicate that high precision rates can be achieved with low false alarm rate exploiting deep net models.

Fast video search on recurring segments

In this paper, we present a novel visual content search approach that can query quite fast, have low memory need and achieve successful results particularly for the instances with minor content changes. For this purpose, first, the content of a video is represented with sparsely sampled edge energy variations among video frames. Then, these high dimensional features are converted into simple signatures using an approach presented in [1]. During a query, these signatures are compared with a novel data structure model which needs low memory burden, and thus similar videos and their overlapping time intervals are estimated. 11 commercial videos downloaded from YouTube are utilized for the test. Furthermore, these videos are augmented with different compression parameters. Reference video archive consists of 11 new videos composed of the original query videos in different time orders and additional 150 hours video dataset that contains none of the query videos. The results validate that the proposed method is fully effective in computation speed and memory requirements.

Effects of standard image processing methods on steganalysis

In this study, we analyze the effect of standart image processing methods on highly successive steganalysis methods of the literature. As a result, we observed the changes in accuracy of steganalysis methods after we polluted the image data sets with standard image processing methods such as compression, color enhancement and histogram equalization. In total, we analyzed nine steganalysis features with our 1000 image data set and provided detailed accuracy results. Our deductions from the experiments are listed at the end of our work for the benefit of future works.

Image community detection

In this work, we propose a new method which can detect image communities inside an image set. The proposed method differs from previous works by representing image relations with directed graphs and performing community analysis on these directed graphs. By analyzing resulting image communities, we can observe the robustness of the proposed method against image deformations (e.g. cutting, text overlay, color changes). The proposed method can be applied to text-based image search results to achieve content-based image search. We believe that the proposed method can be successfully used to bridge the gap between available text-based methods and content-based image search.

An image community detection method for hierarchical visualisation

Better ways of representing the results of image search can be found rather than regular lists of thumbnails. For this purpose, we propose a hierarchical visualisation scheme with two stages. We utilise the notion of image community and aim to detect communities within a large set of images by means of a novel deterministic community detection method. After image communities are detected, the representative key images of these communities are presented to the user in an intuitive and expressive layout. The layout is determined according to the detected community structure. As a result, the user is presented a distinctive set of images at the first stage. If similar images are desired, the members of the communities can be explored at the second stage. We experimentally show that the proposed community detection algorithm significantly outperforms generic community detection methods. Furthermore, we believe that the proposed hierarchical visualisation can be preferred by many of the users.

Detecting image communities

In this work, we propose a novel community detection method that is specifically designed for image communities. We define image community as a coherent subgroup of images within a large set of images. In order to detect image communities, we construct an image graph by utilizing visual affinity between each image pair and then prune most of the links. Instead of affinity values, we prefer ranking of neighboring images and get rid of range mismatch of affinity values. The resulting directed graph is processed to detect the image communities by using the proposed deterministic method. The proposed method is compared against state-of-the-art community detection methods that can operate on directed graphs. In the experiments, we use various sets of images for which ground truths are determined manually. The results indicate that our method significantly outperforms the compared state-of-the-art methods. Furthermore, the proposed method appears to have a consistent performance between sets unlike the compared methods. We believe that the proposed community detection method can be successfully utilized in many different applications.

Video poster creation

We propose a novel method for summarization of videos as representative frames and visualization of these frames as a video poster with respect to each frames importance and timing in the video. Effectiveness of the proposed method on summarization and visualization is observed on approximately 20 hours of video data in news, serial tv shows and debate genres and some example results are discussed in detail.

Multimodal concept detection on multimedia data- RTUK SKAAS KavTan system

Concept detection stands as an important problem for many applications like efficient indexing and retrieval in large video archives. In this work, for detection of diverse and distinct concepts a concept detection system (KavTan) that combines a variety of information sources under a single structure is proposed. The proposed system consists of Generalized Audio Concept Detection and Audio Keyword Detection sub-modules that use audio data and Generalized Visual Concept Detection, Video Text Detection, Human Detection, Nudity Detection, Blood Detection, Flag Detection and Skin Detection sub-modules that use visual data. Each concept is detected by using one or more of the mentioned modules. Proposed concept detection system is tested against multiple concepts and system performance is reported. It is observed that for most of the concepts high performance can be achieved with this approach.