Dalibor Mitrovic

Features for Content-Based Audio Retrieval

Abstract Today, a large number of audio features exists in audio retrieval for different purposes, such as automatic speech recognition, music information retrieval, audio segmentation, and environmental sound retrieval. The goal of this chapter is to review latest research in the context of audio feature extraction and to give an application-independent overview of the most important existing techniques. We survey state-of-the-art features from various domains and propose a novel taxonomy for the organization of audio features. Additionally, we identify the building blocks of audio features and propose a scheme that allows for the description of arbitrary features. We present an extensive literature survey and provide more than 200 references to relevant high-quality publications.

Discrimination and retrieval of animal sounds

Until recently few research has been performed in the area of animal sound retrieval. The authors identify state-of-the-art techniques in general purpose sound recognition by a broad survey of literature. Based on the findings, this paper gives a thorough investigation of audio features and classifiers and their applicability in the domain of animal sounds. We introduce a set of novel audio descriptors and compare their quality to other popular features. The results are encouraging and motivate further research in this domain

A novel trajectory clustering approach for motion segmentation

We propose a novel clustering scheme for spatio-temporal segmentation of sparse motion fields obtained from feature tracking. The approach allows for the segmentation of meaningful motion components in a scene, such as short- and long-term motion of single objects, groups of objects and camera motion. The method has been developed within a project on the analysis of low-quality archive films. We qualitatively and quantitatively evaluate the performance and the robustness of the approach. Results show, that our method successfully segments the motion components even in particularly noisy sequences.

Analysis of Historical Artistic Documentaries

The paper introduces a novel interdisciplinary project addressing the analysis of historical artistic films. The type of employed material has not been subject to automatic analyses, so far. It poses challenges in all areas of content-based analysis and retrieval due to its complex temporal structure and due to substantial degradations. We propose robust features and a method for shot cut detection for this material that outperforms established techniques.

Scene segmentation in artistic archive documentaries

Scene segmentation is a crucial task in the structural analysis of film. State-of-the-art scene segmentation algorithms usually target fiction films (e.g. Hollywood films). Documentaries (especially artistic archive documentaries) follow different montage rules than fiction films and consequently require specialized approaches for scene segmentation. We propose a scene segmentation algorithm targeted at artistic archive documentaries. We evaluate the performance of our technique with archive documentaries and contemporary movies and obtain satisfactory results in both domains.

Finding the Missing Piece: Content-Based Video Comparison

The contribution of this paper consists of a framework for video comparison that allows for the analysis of different movie versions. Furthermore, a second contribution is an evaluation of state-of-the-art, local feature-based approaches for content-based video retrieval in a real world scenario. Eventually, the experimental results show the outstanding performance of a simple, edge-based descriptor within the presented framework.

Cross-Modal Analysis of Audio-Visual Film Montage

A stylistic device frequently employed by filmmakers is the synchronous montage (composition) of audio and visual elements. Synchronous montage helps to increase tension and tempo in a scene and highlights important events in the story. Sequences with synchronous montage usually contain rich semantics which is relevant for understanding a movie. This property is currently not exploited in automated indexing, annotation, and summarization of movies. We propose a cross-modal approach that extracts sequences from a movie with synchronous audio-visual montage. Experiments confirm that the extracted sequences have high semantic relevance. Consequently, they represent a useful basis for different high-level movie abstraction tasks such as automated movie annotation and movie summarization.

Film Analysis of Archived Documentaries

This article outlines issues related to automated film analysis for archived documentaries and explores examples of the applicability of existing content-based retrieval methods.

Retrieval of motion composition in film

This article presents methods for the automatic retrieval of motion and motion compositions in movies. We introduce a user-friendly sketch-based query interface that enables the user to describe desired motion compositions. Based on this abstract description, a tolerant matching scheme extracts shots from a movie with a similar composition. We investigate and evaluate two application scenarios: the retrieval of motion compositions and the retrieval of matching motions (a technique in continuity editing). Experiments show that the developed methods accurately and promptly retrieve relevant shots. The presented methods enable new ways of searching and investigating movies.

Camera take reconstruction

In this paper we focus on a novel issue in the field of video retrieval stemming from film analysis, namely the investigation of film montage patterns. For this purpose it is first necessary to reconstruct the original film sequences, i.e. the camera takes. For the decision whether or not two shots occurring anywhere in a film stem from the same take we use edge histograms and local feature tracking. Evaluation results on experimental film material (where montage patterns are of great importance) show a very good performance of the algorithm proposed.