Banu Oskay Acar

Content Based Copy Detection with Coarse Audio-Visual Fingerprints

Content Based Copy Detection (CBCD) emerges as a viable choice against active detection methodology of watermarking. The very first reason is that the media already under circulation cannot be marked and secondly, CBCD inherently can endure various severe attacks, which watermarking cannot. Although in general, media content is handled independently as visual and audio in this work both information sources are utilized in a unified framework, in which coarse representation of fundamental features are employed. From the copy detection perspective, number of attacks on audio content is limited with respect to visual case. Therefore audio, if present, is an indispensable part of a robust video copy detection system. In this study, the validity of this statement is presented through various experiments on a large data set.

Multimodal concept detection in broadcast media: KavTan

Concept detection stands as an important problem for efficient indexing and retrieval in large video archives. In this work, the KavTan System, which performs high-level semantic classification in one of the largest TV archives of Turkey, is presented. In this system, concept detection is performed using generalized visual and audio concept detection modules that are supported by video text detection, audio keyword spotting and specialized audio-visual semantic detection components. The performance of the presented framework was assessed objectively over a wide range of semantic concepts (5 high-level, 14 visual, 9 audio, 2 supplementary) by using a significant amount of precisely labeled ground truth data. KavTan System achieves successful high-level concept detection performance in unconstrained TV broadcast by efficiently utilizing multimodal information that is systematically extracted from both spatial and temporal extent of multimedia data.

Noninvasive evaluation of swallowing sound is an effective way of diagnosing feeding maturation in newborn infants

Despite extensive research, there is still controversy regarding the time at which sucking and swallowing functions mature in preterm infants. This study aimed to evaluate maturation using the noninvasive method of swallowing sound.

Content based event retrieval on TV broadcast audio

Auditory data contains important information about the content of multimedia data. This paper presents a method for content based event retrieval on broadcast audio. The aim of this study is to retrieve audio events from huge multimedia databases. 17 classes which are most frequently observed in TV broadcast, and which are considered as an important input to higher level semantic analysis of multimedia data are selected. Audio streams are divided into homogenous segments in order to generate fingerprints that describe both temporal and spectral information of audio events. Both spectral and temporal properties of audio events are analyzed and some fingerprints to represent these properties are presented. Audio events are modeled by Gaussian Mixture Models. For the retrieval, an ordered sequence is provided to the user for each event, sorted by the likelihood values of the fingerprints. The system aims to bring the query events with higher likelihood values first. Mean average precision value is used to evaluate retrieval performance.17 audio classes are tested on 11 hours of TV recordings and 18,5% average precision is achieved.

Generalized visual concept detection

For efficient indexing and retrieval of video archives, concept detection stands as an important problem. In this work, a generalized structure that can be used for detection of diverse and distinct concepts is proposed. In the system, MPEG-7 Descriptors and Scale Invariant Transform (SIFT) are utilized as visual features. Furthermore, visual features are transformed by codebooks which are constructed by k-Means clustering. On the other hand, classification is performed on the distribution of visual features over the codebook. Proposed system is firstly tested against an elementary concept. Afterwards for a set of concepts system performance is reported on the TRECVID 2009 test set. It has been observed that with a sufficiently large training set high performance can be achieved with this method.

An unsupervised audio segmentation method using Bayesian information criterion

Audio segmentation is a well-known problem which can be considered from various angles. In the context of this paper, audio segmentation problem is to extract small “homogeneous” pieces of audio in which the content does not change in terms of the present audio events. The proposed method is compared with the well-known segmentation method; Bayesian Information Criterion (BIC) based Divide-and-Conquer, in terms of average segment duration and computational complexity.

Multimodal concept detection on multimedia data- RTUK SKAAS KavTan system

Concept detection stands as an important problem for many applications like efficient indexing and retrieval in large video archives. In this work, for detection of diverse and distinct concepts a concept detection system (KavTan) that combines a variety of information sources under a single structure is proposed. The proposed system consists of Generalized Audio Concept Detection and Audio Keyword Detection sub-modules that use audio data and Generalized Visual Concept Detection, Video Text Detection, Human Detection, Nudity Detection, Blood Detection, Flag Detection and Skin Detection sub-modules that use visual data. Each concept is detected by using one or more of the mentioned modules. Proposed concept detection system is tested against multiple concepts and system performance is reported. It is observed that for most of the concepts high performance can be achieved with this approach.

Bullet matching using SIFT feature

Firearms leave special marks on the bullet while the bullet travels through the barrel. In this work, visual word codes obtained from interest points were used in bullet matching. Visual codebook was constructed by clustering Scale Invariant Feature Transform (SIFT) features using interest point orientation information as semi-supervised clustering constraint. The ratio of the number of visual words in common to the total number of visual words was used as a similarity metric in the comparison of images. Visual words are weighted by inverse document frequency which is frequently used in text document comparisons. Experiment results show that the proposed method presents promising results in bullet matching.

Content based video copy detection with coarse features

Content Based Copy Detection is an alternative approach to invisible watermarking for tracking duplicate data. Primary stages are creating a database using the features belonging to the original data and searching query data in terms of its features in this database. Features must be robust against targeted attacks and discriminative enough to distinguish different content. In this work, we propose reducing the precision of feature values to attain robustness and increasing the number and dimension of features to attain discriminativity. To this end, we create a feature database using different features, which correspond to different information sources, together. We detect the original sources of the query videos in this database, which is composed of coarse features, by feature comparison. Effectiveness of the proposed method against various attacks is observed through experiments.

Voting system based robust and efficient audiocopy detection

Audio Copy Detection(ACD) problem has several difficulties due to signal distortions and huge amount of audio data to be searched. In this paper we propose a fast audio copy detection system which is very robust against common signal distortions.The proposed method performs a vote based search on 15 bit representation of audio data.