Davy Van Deursen

Multichannel distribution for universal multimedia access in home media gateways

Today, people collect their personal multimedia content on home media servers. In addition to consuming their content on TV sets, people are using mobile multimedia players, PCs and even mobile phones. Since those terminals have various capabilities, it is necessary to adapt the content to a more device specific version. For audio, video, and graphics scalable codecs exist which realize this goal. For multimedia presentations no such scalable coding is available. This paper introduces a multichannel distribution system in home media gateways, this implies that a multimedia presentation can to be created once, and consumed on every possible terminal. The introduced multichannel distribution system is realized by combining MPEG-21 technology with existing device specific presentation languages. This results in a device agnostic Digital Item which can be transformed into a device specific presentation. The resulted presentation takes advantage of the full potential of a terminal.

Functional descriptions as the bridge between hypermedia APIs and the Semantic Web

The early visions for the Semantic Web, from the famous 2001 Scientific American article by Berners-Lee et al., feature intelligent agents that can autonomously perform tasks like discovering information, scheduling events, finding execution plans for complex operations, and in general, use reasoning techniques to come up with sense-making and traceable decisions. While today more than ten years later the building blocks (1) resource-oriented rest infrastructure, (2) Web APIs, and (3) Linked Data are in place, the envisioned intelligent agents have not landed yet. In this paper, we explain why capturing functionality is the connection between those three building blocks, and introduce the functional API description format RESTdesc that creates this bridge between hypermedia APIs and the Semantic Web. Rather than adding yet another component to the Semantic Web stack, RESTdesc offers instead concise descriptions that reuse existing vocabularies to guide hypermedia-driven agents. Its versatile capabilities are illustrated by a real-life agent use case for Web browsers wherein we demonstrate that RESTdesc functional descriptions are capable of fulfilling the promise of autonomous agents on the Web.

Enabling context-aware multimedia annotation by a novel generic semantic problem-solving platform

Automatic generation of metadata, facilitating the retrieval of multimedia items, potentially saves large amounts of manual work. However, the high specialization degree of feature extraction algorithms makes them unaware of the context they operate in, which contains valuable and often necessary information. In this paper, we show how Semantic Web technologies can provide a context that algorithms can interact with. We propose a generic problem-solving platform that uses Web services and various knowledge sources to find solutions to complex requests. The platform employs a reasoner-based composition algorithm, generating an execution plan that combines several algorithms as services. It then supervises the execution of this plan, intervening in case of errors or unexpected behavior. We illustrate our approach by a use case in which we annotate the names of people depicted in a photograph.

Using bitstream structure descriptions for the exploitation of multi-layered temporal scalability in H.264/AVC’s base specification

In this paper, attention is paid to the automatic generation of XML-based descriptions containing information about the high-level structure of binary multimedia resources. These structural metadata can then be transformed in order to reflect a desired adaptation of a multimedia resource, and can subsequently be used to create a tailored version of the resource in question. Based on this concept, two technologies are presented: MPEG-21 BSDL and a modified version of XFlavor being able to create BSDL compatible output. Their usage is elaborated in more detail with respect to the valid exploitation of multi-layered temporal scalability in H.264/MPEG-4 AVC’s base specification, and in particular with a focus on a combined usage of the sub-sequence coding technique and Supplemental Enhancement Information (SEI) messages. Some performance measurements in terms of file sizes and computational times are presented as well.

Three-dimensional information exchange over the semantic web for the domain of architecture, engineering, and construction

Abstract Three-dimensional (3-D) geometry can be described in many ways, with both a varying syntax and a varying semantics. As a result, several very diverse schemas and file formats can be deployed to describe geometry, depending on the application domain in question. In a multidisciplinary domain such as the domain of architecture, engineering, and construction, this range of specialized schemas makes file format conversions inevitable. The approach adopted by current conversion tools, however, often results in a loss of information, most often due to a “mistranslation” between different syntaxes and/or semantics, leading to errors and limitations in the design conception stage and to inefficiency due to the required remodeling efforts. An approach based on semantic web technology may reduce the loss of information significantly, leading to an improved processing of 3-D information and hence to an improved design practice in the architecture, engineering, and construction domain. This paper documents our investigation of the nature of this 3-D information conversion problem and how it may be encompassed using semantic web technology. In an exploratory double test case, we show how the specific deployment of semantic rule languages and an appropriate inference engine are to be adopted to improve this 3-D information exchange. It shows how semantic web technology allows the coexistence of diverse descriptions of the same 3-D information, interlinked through explicit conversion rules. Although only a simple example is used to document the process, and a more in-depth investigation is needed, the initial results indicate the suggested approach to be a useful alternative approach to obtain an improved 3-D information exchange.

A compressed-domain approach for shot boundary detection on H.264/AVC bit streams

The amount of digital video content has grown extensively during recent years, resulting in a rising need for the development of systems for automatic indexing, summarization, and semantic analysis. A prerequisite for video content analysis is the ability to discover the temporal structure of a video sequence. In this paper, a novel shot boundary detection technique is introduced that operates completely in the compressed domain using the H.264/AVC video standard. As this specification contains a number of new coding tools, the characteristics of a compressed bit stream are different from prior video specifications. Furthermore, the H.264/AVC specification introduces new coding structures such as hierarchical coding patterns, which can have a major influence on video analysis algorithms. First, a shot boundary detection algorithm is proposed which can be used to segment H.264/AVC bit streams based on temporal dependencies and spatial dissimilarities. This algorithm is further enhanced to exploit hierarchical coding patterns. As these sequences are characterized by a pyramidal structure, only a subset of frames needs to be considered during analysis, allowing the reduction of the computational complexity. Besides the increased efficiency, experimental results also show that the proposed shot boundary detection algorithm achieves a high accuracy.

Automatic discovery of high-level provenance using semantic similarity

As interest in provenance grows among the Semantic Web community, it is recognized as a useful tool across many domains. However, existing automatic provenance collection techniques are not universally applicable. Most existing methods either rely on (low-level) observed provenance, or require that the user discloses formal workflows. In this paper, we propose a new approach for automatic discovery of provenance, at multiple levels of granularity. To accomplish this, we detect entity derivations, relying on clustering algorithms, linked data and semantic similarity. The resulting derivations are structured in compliance with the Provenance Data Model (PROV-DM). While the proposed approach is purposely kept general, allowing adaptation in many use cases, we provide an implementation for one of these use cases, namely discovering the sources of news articles. With this implementation, we were able to detect 73% of the original sources of 410 news stories, at 68% precision. Lastly, we discuss possible improvements and future work.

A real-time content adaptation framework for exploiting ROI scalability in H.264/AVC

It is well known that motion detection using single frame differencing, while computationally much simpler than other techniques, is more liable to generate large areas of false foregrounds known as ghosts. In order to overcome this problem the authors propose a method based on signed differencing and connectivity analysis. The proposal is suitable to applications which cannot afford the un-avoidable errors of background modeling or the limitations of 3-frames preprocessing.In many application scenarios, the use of Regions of Interest (ROIs) within video sequences is a useful concept. It is shown in this paper how Flexible Macroblock Ordering (FMO), defined in H.264/AVC as an error resilience tool, can be used for the coding arbitrary-shaped ROIs. In order to exploit the coding of ROIs in an H.264/AVC bitstream, a description-driven content adaptation framework is introduced that is able to extract the ROIs of a given bitstream. The results of a series of tests indicate that the ROI extraction process significantly reduces the bit rate of the bitstreams and increases the decoding speed. In case of a fixed camera and a static background, the impact of this reduction on the visual quality of the video sequence is negligible. Regarding the adaptation framework itself, it is shown that in all cases, the framework operates in real time and that it is suited for streaming scenarios by design.

BFlavor: A harmonized approach to media resource adaptation, inspired by MPEG-21 BSDL and XFlavor

During recent years, several tools have been developed that allow the automatic generation of XML descriptions containing information about the syntax of binary media resources. Such a bitstream syntax description (BSD) can then be transformed to reflect a desired adaptation of a media resource, and can subsequently be used to create a tailored version of this resource. The main contribution of this paper is the introduction of BFlavor, a new tool for exposing the syntax of binary media resources as an XML description. Its development was inspired by two other technologies, i.e. MPEG-21 BSDL and XFlavor. Although created from a different point of view, both languages offer solutions for translating the syntax of a media resource into an XML representation for further processing. BFlavor (BSDL+XFlavor) harmonizes the two technologies by combining their strengths and eliminating their weaknesses. More precisely, the processing efficiency and expressive power of XFlavor on the one hand, and the ability to create high-level BSDs using MPEG-21 BSDL on the other hand, were our key motives for its development. To assess the expressive power and performance of a BFlavor-driven content adaptation chain, several experiments were conducted. These experiments test the automatic generation of BSDs for MPEG-1 Video and H.264/AVC, as well as the exploitation of multi-layered temporal scalability in H.264/AVC. Our results show that BFlavor is an efficient and harmonized description tool for enabling XML-driven adaptation of media resources in a format-agnostic way. BSDL and XFlavor are outperformed by BFlavor in terms of execution times, memory consumption, and file sizes.

Implementing the media fragments URI specification

In this paper, we describe two examples of implementations of the Media Fragments URI specification which is currently being developed by the W3C Media Fragments Working Group. The groups mission is to create standard addressing schemes for media fragments on the Web using Uniform Resource Identifiers (URIs). We describe two scenarios to illustrate the implementations. More specifically, we show how User Agents (UA) will either be able to resolve media fragment URIs without help from the server, or will make use of a media fragments-aware server. Finally, we present some ongoing discussions and issues regarding the implementation of the Media Fragments specification.