Christian Breiteneder

Data modeling of time-based media

Many aspects of time-based media—complex data encoding, compression, “quality factors,” timing—appear problematic from a data modeling standpoint. This paper proposes timed streams as the basic abstraction for modeling time-based media. Several media-independent structuring mechanisms are introduced and a data model is presented which, rather than leaving the interpretation of multimedia data to applications, addresses the complex organization and relationships present in multimedia.

Features for Content-Based Audio Retrieval

Abstract Today, a large number of audio features exists in audio retrieval for different purposes, such as automatic speech recognition, music information retrieval, audio segmentation, and environmental sound retrieval. The goal of this chapter is to review latest research in the context of audio feature extraction and to give an application-independent overview of the most important existing techniques. We survey state-of-the-art features from various domains and propose a novel taxonomy for the organization of audio features. Additionally, we identify the building blocks of audio features and propose a scheme that allows for the description of arbitrary features. We present an extensive literature survey and provide more than 200 references to relevant high-quality publications.

Virtual studios: an overview

Virtual studio systems began as experimental prototypes that extended traditional chromakeying. Now commercial products based on graphics supercomputers are commonly used for broadcast production. We discuss this evolution and consider extensions, alternative approaches, and issues facing broadcasters who introduce virtual studio systems.

Discrimination and retrieval of animal sounds

Until recently few research has been performed in the area of animal sound retrieval. The authors identify state-of-the-art techniques in general purpose sound recognition by a broad survey of literature. Based on the findings, this paper gives a thorough investigation of audio features and classifiers and their applicability in the domain of animal sounds. We introduce a set of novel audio descriptors and compare their quality to other popular features. The results are encouraging and motivate further research in this domain

Multimedia modeling

Multimedia modeling aims to study computational models for addressing real-world multimedia problems from various perspectives, including information fusion, perceptual understanding, performance evaluation and social media. The topic becomes increasingly important with the massive amount of data available over the Internet, representing different pieces of information in heterogeneous forms that need to be consolidated before being used for multimedia problems. On the other hand, the advancement in technologies such as mobile and sensing devices drive the needs for revisiting the existing models for not only dealing with audio-visual cues but also incorporating various sensory modalities that have potential in providing cheaper and simpler solutions. The selected papers in this special issue were extended by their authors to a journal version and then went through a rigorous review process that included at least three anonymous referees. The first paper entitled “Multimedia Classification and Event Detection Using Double Fusion”, co-authored by Zhen-zong Lei, Lei Bao, Shoou-I Yu, Wei Liu and Alexander G. Hauptmann from Carnegie Mellon University, investigates the issue of information fusion for generic and complex event detection. Detecting events such as “making a sandwich” requires modeling large sources of information from audio-visual, textual and concept detection, thus the issue of fusing diverse features comes into picture. The paper proposes a double fusion Multimed Tools Appl (2014) 71:331–332 DOI 10.1007/s11042-013-1775-3

VizIR—a framework for visual information retrieval

Abstract In this paper the visual information retrieval project VizIR is presented. The goal of the project is the implementation of an open visual information retrieval (VIR) prototype as basis for further research on major problems of VIR. The motivation behind VizIR is the implementation of an open platform for supporting and facilitating research, teaching, the exchange of research results and research cooperation in the field in general. The availability of this platform could make cooperation and such research (especially for smaller institutions) easier. The intention of this paper is to inform interested researchers about the VizIR project and its design and to invite people to participate in the design and implementation process. We describe the goals of the VizIR project, the intended design of the querying framework, the user interface design and major implementation issues. The querying framework consists of classes for feature extraction, similarity measurement, media handling and database access. User interface design includes a description of visual components and their class structure, the communication between panels and the communication between visual components and query engines. The latter is based on the multimedia retrieval markup language (MRML, Website. http://www.mrml.net (last visited: 2003–03–20)). To be compatible with our querying paradigm, we extend MRML with additional elements. Implementation issues include a sketch on advantages and drawbacks of existing cross-platform media processing frameworks: Java Media Framework, OpenML and DirectX/DirectShow and details on the Java components used for user interface implementation, 3D graphics with Java and Java XML parsing.

Semantic feature layers in content-based image retrieval: implementation of human world features

The major problem of most CBIR approaches is bad quality in terms of recall and precision. As a major reason for this, the semantic gap between high-level concepts and low-level features has been identified. In this paper we describe an approach to reduce the impact of the semantic gap by deriving high level (semantic) from low-level features and using these features to improve the quality of CBIR queries. This concept is implemented for a high-level feature class that describes human world properties and evaluated in 300 queries. Results show that using those high-level features improves the quality of result sets by balancing recall and precision.

Automated social event detection in large photo collections

The detection of a specific social event requires for high semantic understanding in the interpretation of particular event characteristics such as its type and location. In many cases, photos capturing different events at the same (or highly similar) locations can hardly be distinguished by each other. Available metadata can provide assistance where there is no expert knowledge at hand. However, metadata often lack completeness and reliability. In this paper, we explore the feasibility of a fully automated approach for the detection of specific social events. In comparison to related approaches, we do not incorporate query-specific processing and we perform no manual adaptation of the input query. The resulting approach is applicable to arbitrary event types.

Content-based image retrieval of coats of arms

The paper describes a content-based image retrieval system for coats of arms. The characteristics of arms are analyzed and specific features for segmentation, object layout, symmetry, etc. developed and implemented. Search queries are formulated and classified into query models which represent similarity of images by the features they use. These models are rated by recall and precision.

Automated petroglyph image segmentation with interactive classifier fusion

The number of high quality images of rock panels containing petroglyphs grows steadily. Different time-consuming manual methods to determine and document the exact shapes and spatial locations of petroglyphs on a panel have been carried out over decades. The first step for classification and retrieval of petroglyphs is the segmentation of the images. In this paper, we present and evaluate an automated approach to segment petroglyph images.