Mario Döller

The life cycle of multimedia metadata

During its lifetime, multimedia content undergoes different stages or cycles from production to consumption. Content is created, processed or modified in a postproduction stage, delivered to users, and finally, consumed. Metadata, or descriptive data about the multimedia content, pass through similar stages but with different time lines. Metadata may be produced, modified, and consumed by all actors involved in the content production-consumption chain. At each step of the chain, different kinds of metadata may be produced by highly different methods and of substantially different semantic value. Different metadata let us tie the different multimedia processes in a life cycle together. However, to employ these metadata, they must be appropriately generated. The CODAC Project, led by Harald Kosch, implements different multimedia processes and ties them together in the life cycle. CODAC uses distributed systems to implement multimedia processes. The projects core component is a multimedia database management system (MMDBMS) which stores content and MPEG-7-based metadata. It communicates with a streaming server for data delivery. The database is realized in the multimedia data cartridge (MDC) - which is an extension of the Oracle database management system - to handle multimedia content and MPEG-7 metadata.

The MPEG Query Format: Unifying Access to Multimedia Retrieval Systems

The growth of multimedia is increasing the need for standards for accessing and searching distributed repositories. The moving picture experts group (MPEG) is developing the MPEG query format (MPQF) to standardize this interface as part of MPEG-7. The objective is to make multimedia access and search easier and interoperable across search engines and repositories. This article describes the MPQF and highlights some of the ways it goes beyond todays query languages by providing capabilities for multimedia query-by-example and spatiotemporal queries.

The MPEG-7 Multimedia Database System (MPEG-7 MMDB)

Broadly used Database Management Systems (DBMS) propose multimedia extensions, like Oracles Multimedia (formerly interMedia). However, these extensions lack means for managing the requirements of multimedia data in terms of semantic meaningful querying, advanced indexing, content modeling and multimedia programming libraries. In this context, this paper presents the MPEG-7 Multimedia DataBase System (MPEG-7 MMDB). The innovative parts of our system are our metadata model for multimedia content relying on the XML-based MPEG-7 standard, a new indexing and querying system for MPEG-7, the query optimizer and the supporting internal and external application libraries. The resulting system, extending Oracle 10g, is verified and demonstrated by the use of two real multimedia applications in the field of audio recognition and image retrieval.

Content-Based Indexing and Retrieval Supported by Mobile Agent Technology

In this paper we present the MultiMedia Database Mobile agent technology (M3) which supports personalized content-retrieval and indexing in a distributed Oracle 8i DB. We implemented an agency on top of the Oracle 8i JServer and realized mobility with the embedded Visbroker Corba ORB. A performance comparison of our mobile agent technology with a client-server solution for a nearest-neighbor search in an image database shows the efficiency of the proposed solution.

AIR: Architecture for interoperable retrieval on distributed and heterogeneous multimedia repositories

Nowadays multimedia data is produced and consumed at an ever increasing rate. Similarly to this trend, diverse storage approaches for multimedia data have been introduced. These observations lead to the fact that distributed and heterogeneous multimedia repositories exist whereas an unified and easy access to the stored multimedia data is not given. This paper presents an architecture, named AIR, that offers the aformentioned retrieval possibilites. To ensure interoperability, AIR makes use of recently issued standards, namely the MPEG Query Format (MPQF) (multimedia query language) and the JPSearch transformation rules (metadata interoperability).

Landmark-assisted location and tracking in outdoor mobile network

Modern mobile devices integrating sensors, like accelerometers and cameras, are paving the way to the definition of high-quality and accurate geolocation solutions based on the informations acquired by these sensors, and data collected and managed by GSM/3G networks. In this paper, we present a technique that provides geolocation and mobility prediction of mobile devices, mixing the location information acquired with the GSM/3G infrastructure and the results of a landmark matching achieved thanks to the camera integrated on the mobile devices. Our geolocation approach is based on an advanced Time-Forwarding algorithm and on database correlation technique over Received Signal Strength Indication (RSSI) data, and integrates information produced by a landmark recognition infrastructure, to enhance algorithm performances in those areas with poor signal and low accurate geolocation. Performances of the algorithm are evaluated on real data from a complex urban environment.

Standardized interoperable image retrieval

Digital still images are generated, distributed and stored worldwide at an ever increasing rate. Yet, the large number of available metadata description formats prevents consistent and efficient access to image repositories. Standardized solutions for unifying the access and the retrieval of image repositories are therefore strongly needed. The ISO/IEC SC29 WG1 (more commonly known as JPEG) established the JPSearch project, which aims to standardize interfaces of an abstract image retrieval framework. In this context, the paper introduces our contribution consisting of a reference metadata model, which in combination with the JPEG query format improves interoperable image retrieval for users. Second, a transformation rule declaration language for query reformulation is presented, which contributes to interoperability at the repository side.

World-wide scale geotagged image dataset for automatic image annotation and reverse geotagging

In this paper, a dataset of geotagged photos on a world-wide scale is presented. The dataset contains a sample of more than 14 million geotagged photos crawled from Flickr with the corresponding metadata. To guarantee the spatial representativeness of the dataset, a crawling approach based on the small-world phenomena and the Flickr friendships graph is applied. Furthermore, the noisiness of user-provided tags is reduced through an automatic tag cleaning approach. To enable efficient retrieval, photos in the dataset are indexed based on their location information using quad-tree data structure. The dataset can assists different applications, especially, search-based automatic image annotation and reverse geotagging.

Geo-based automatic image annotation

A huge number of user-tagged images are daily uploaded to the web. Recently, a growing number of those images are also geotagged. These provide new opportunities for solutions to automatically tag images so that efficient image management and retrieval can be achieved. In this paper an automatic image annotation approach is proposed. It is based on a statistical model that combines two different kinds of information: high level information represented by user tags of images captured in the same location as a new unlabeled image (input image); and low level information represented by the visual similarity between the input image and the collection of geographically similar images. To maximize the number of images that are visually similar to the input image, an iterative visual matching approach is proposed and evaluated. The results show that a significant recall improvement can be achieved with an increasing number of iterations. The quality of the recommended tags has also been evaluated and an overall good performance has been observed.

Comprehensive treatment of adaptation in distributed multimedia systems in the ADMITS project

Adaptation is becoming an increasingly important tool for re-source and media management in distributed multimedia systems. Best-effort scheduling and worst-case reservation of resources are two extreme cases, none of them well suited to cope with large-scale, dynamic multimedia systems. The middle course can be met by a system which dynamically adapts its data, resource re-quirements, and processing components to achieve user satisfac-tion. Nevertheless, there is no agreement about the questions, where, when, what and who should adapt. A number of papers have been published in recent years, where adaptation is a central issue, however, in most different interpretations and generally in a somehow limited scope; e.g.,[1, 2, 8, 9, 10, 12]. A distributed multimedia system comprises several types of com-ponents, such as media servers, meta-databases, proxies, routers, clients. Also, a large number of adaptation possibilities exist, from simple frame dropping up to virtual server systems which dynamically allocate new resources on demand. The main ques-tion is, which kind of component can be best used for what kind of adaptation. In the