Harald Kosch

Bitstream syntax description: a tool for multimedia resource adaptation within MPEG-21

Abstract In this paper, a generic method is described to allow the adaptation of different multimedia resources by a single, media resource-agnostic processor. This method is based on an XML description of the media resources bitstream syntax, which can be transformed to reflect the desired adaptation and then be used to generate an adapted version of the bitstream. Based on this concept, two complementary technologies, BSDL 2 and gBS Schema, are presented. The two technologies provide solutions for parsing a bitstream to generate its XML description, for the generic structuring and marking of this description, and the generation of an adapted bitstream using its transformed description. The two technologies can be used as stand-alone tools; however, a joint approach has been developed in order to harmonise the two solutions and exploit their strengths. This paper is focusing on the gBS Schema and the joint BSDL/gBS Schema harmonised approach.

The life cycle of multimedia metadata

During its lifetime, multimedia content undergoes different stages or cycles from production to consumption. Content is created, processed or modified in a postproduction stage, delivered to users, and finally, consumed. Metadata, or descriptive data about the multimedia content, pass through similar stages but with different time lines. Metadata may be produced, modified, and consumed by all actors involved in the content production-consumption chain. At each step of the chain, different kinds of metadata may be produced by highly different methods and of substantially different semantic value. Different metadata let us tie the different multimedia processes in a life cycle together. However, to employ these metadata, they must be appropriately generated. The CODAC Project, led by Harald Kosch, implements different multimedia processes and ties them together in the life cycle. CODAC uses distributed systems to implement multimedia processes. The projects core component is a multimedia database management system (MMDBMS) which stores content and MPEG-7-based metadata. It communicates with a streaming server for data delivery. The database is realized in the multimedia data cartridge (MDC) - which is an extension of the Oracle database management system - to handle multimedia content and MPEG-7 metadata.

VIDEX : an integrated generic video indexing approach

This paper presents an integrated generic technique for low-and high-level video indexing. The proposed approach tries to integrate the advantages of existing low- and high-level video indexing approaches by reducing their shortcomings. Furthermore, the model introduces concepts for a detailed structuring of video streams, and for correlations of low-and high-level video objects. The proposed model is called generic, as it only defines a framework of classes for an implementating a prototype of a distibuted multimedia information system supporting content-based video retrieval.

Metadata driven adaptation in the ADMITS project

Abstract The ADMITS project (Adaptation in Distributed Multimedia IT Systems) is building an experimental distributed multimedia system for investigations into adaptation, which we consider an increasingly important tool for multimedia systems. A number of possible adaptation entities (server, proxy, clients, routers) are being explored, different algorithms for media, component and application-level adaptations are being implemented and evaluated, and experimental data are being derived to gain insight into when, where and how to adapt, and how individual, distributed adaptation steps interoperate and interact with each other. In this paper the “adaptation-chain” of (MPEG-conforming) metadata based adaptation is described: from the creation stage at the server side, through its usage in the network (actually in a proxy), up to the consumption at the client. The metadata are used to steer the adaptation processes. MPEG-conformant metadata, the so-called variation descriptions, are introduced; an example of a complete MPEG-7 document describing temporal scaling of an MPEG-4 video is given. The meta-database designed to store the metadata is briefly discussed. We describe how the metadata can be extracted from MPEG-4 visual elementary streams and initial results from a temporal video scaling experiment are given. We further present how the metadata can be utilized by enhanced cache replacement algorithms in a proxy server in order to realize quality-based caching; experimental results using these algorithms are also given. Finally, an adaptive query and presentation interface to the meta- and media database is outlined.

MPEG-7 and multimedia database systems

The Multimedia Description Standard MPEG-7 is an International Standard since February 2002. It defines a huge set of description classes for multimedia content, for its creation and its communication. This article investigates what MPEG-7 means to Multimedia Database Systems (MMDBSs) and vice versa. We argue that MPEG-7 has to be considered complementary to, rather than competing with, data models employed in MMDBSs. Finally we show by an example scenario how these technologies can reasonably complement one another.

The MPEG-7 Multimedia Database System (MPEG-7 MMDB)

Broadly used Database Management Systems (DBMS) propose multimedia extensions, like Oracles Multimedia (formerly interMedia). However, these extensions lack means for managing the requirements of multimedia data in terms of semantic meaningful querying, advanced indexing, content modeling and multimedia programming libraries. In this context, this paper presents the MPEG-7 Multimedia DataBase System (MPEG-7 MMDB). The innovative parts of our system are our metadata model for multimedia content relying on the XML-based MPEG-7 standard, a new indexing and querying system for MPEG-7, the query optimizer and the supporting internal and external application libraries. The resulting system, extending Oracle 10g, is verified and demonstrated by the use of two real multimedia applications in the field of audio recognition and image retrieval.

Similarity-based operators and query optimization for multimedia database systems

The many successful research results in the domain of computer vision have made similarity based data retrieval techniques a promising approach. As a result, the integration of similarity based retrieval techniques of multimedia data into DBMSs is currently an active research issue. We first illustrate the importance of similarity based operations. Then, we present our image data repository model that supports similarity based operations conveniently under an object-relational database paradigm. Furthermore, we present novel similarity based operators on image tables and study their properties. Finally, based on the properties of the operators identified, we derive algebraic rules that are useful for similarity based query optimization and we introduce a cost model for an implementation of one of the major similarity based operators.

Coding format independent multimedia content adaptation using XML

Due to the heterogeneity of the current terminal and network infrastructures, multimedia content needs to be adapted to specific capabilities of these terminals and network devices. Furthermore, user preferences and user environment characteristics must also be taken into consideration. The problem becomes even more complex by the diversity of multimedia content types and encoding formats. In order to meet this heterogeneity and to be applicable to different coding formats, the adaptation must be performed in a generic and interoperable way. As a response to this problem and in the context of MPEG-21, we present an approach which uses XML to describe the high-level structure of a multimedia resource in a generic way, i.e., how the multimedia content is organized, for instance in layers, frames, or scenes. For this purpose, a schema for XML-based bitstream syntax descriptions (generic Bitstream Syntax Descriptions or gBSDs) has been developed. A gBSD can describe the high-level structure of a multimedia resource in a coding format independent way. Adaptation of the resource is based on elementary transformation instructions formulated with respect to the gBSDs. These instructions have been separated from the gBSDs in order to use the same descriptions for different adaptations, e.g., temporal scaling, SNR scaling, or semantic adaptations. In the MPEG-21 framework, those adaptations can be steered for instance by the network characteristics and the user preferences. As a result, it becomes possible for coding format agnostic adaptation engines to transform media bitstreams and associated descriptions to meet the requirements imposed by the network conditions, device capabilities, and user preferences.

Content-Based Indexing and Retrieval Supported by Mobile Agent Technology

In this paper we present the MultiMedia Database Mobile agent technology (M3) which supports personalized content-retrieval and indexing in a distributed Oracle 8i DB. We implemented an agency on top of the Oracle 8i JServer and realized mobility with the embedded Visbroker Corba ORB. A performance comparison of our mobile agent technology with a client-server solution for a nearest-neighbor search in an image database shows the efficiency of the proposed solution.

MPEG-7 Multimedia Data Cartridge

Broadly used Database Management Systems (DBMS) are not able to tackle the requirements of multimedia in querying, indexing and content modeling. Therefore, extenders for multimedia data types have been proposed. These extensions, however, offer only limited semantic modeling and rely on basic index structures which do not meet the whole nature of multimedia, for instance for a Nearest-Neighbor Search. In this context, the paper presents a methodology for enhancing extensible ORDBMS for multimedia data. In particular, we introduce an MPEG-7 Multimedia Data Cartridge which includes a semantically rich metadata model for multimedia content relying on the MPEG-7 standard. Furthermore, to fulfill the needs for efficient multimedia query processing, we created in this Cartridge a new indexing and query framework for various types of retrieval operations.