Michael Ransburg

Automatic adaptation of streaming multimedia content in a dynamic and distributed environment

The diversity of end-terminal and access network capabilities as well as the dynamic nature of wireless connections pose significant challenges to providers of multimedia streaming services. In this paper, we present a system based on MPEG-21 Digital Item Adaptation (DIA) technologies that automatically adapts scalable multimedia resources, like upcoming MPEG-4 Scalable Video Coding (SVC) streams, in a generic and transparent way to the user and session context. This context includes terminal and network capabilities as well as user characteristics. A server side adaptation engine reacts to context changes by dynamic decision taking and accordingly modified bitstream adaptation. Furthermore, novel concepts are presented that facilitate multimedia adaptation in a distributed fashion along the delivery path.

Design options and comparison of in-network H.264/SVC adaptation

This paper explores design options and evaluates implementations of in-network, RTP/RTSP based adaptation MANEs (Media Aware Network Elements) for H.264/SVC content streaming. The obvious technique to be employed by such an adaptation MANE is to perform SVC specific bitstream extraction or truncation. Another mechanism that can be used is description (metadata) driven, coding format independent adaptation based on generic Bitstream Syntax Descriptions (gBSD), as specified within MPEG-21 Digital Item Adaptation (DIA). Adaptation MANE architectures for both approaches are developed and presented, implemented in end-to-end streaming/adaptation prototype systems, and experimentally evaluated and compared. For the gBSD based solution, open issues like the granularity of bitstream descriptions and of bitstream adaptation, metadata overhead, metadata packetization and transport options, and error resilience in case of metadata losses, are addressed. The experimental results indicate that a simple SVC specific adaptation MANE does clearly outperform the gBSD based adaptation variants. Yet, the conceptual advantages of the description driven approach, like coding format independence and flexibility, may outweigh the performance drawbacks in specific applications.

Towards an Improved SVC-to-AVC Rewriter

The Scalable Video Coding (SVC) extension of the H.264/AVC (AVC) video coding standard features spatial, quality and temporal scalability. Backwards compatibility with legacy decoding devices is maintained through an H.264/AVC compliant base layer, which represents the lowest quality of an SVC bit-stream. However, it is often desirable to also provide the higher quality layers to legacy H.264/AVC devices. This is achieved by a process commonly known as “bit-stream rewriting”, which allows for an efficient SVC to AVC conversion by exploiting the similarities of the two codecs. This paper introduces an improved version of the existing JSVM reference software rewriter (JSVM-rewriter). The improvements include a better run-time performance through parallel processing, as well as applicability in streaming scenarios. A detailed evaluation provides performance measurements for the improved rewriter and compares it to the existing JSVM-rewriter. The evaluation shows that notable performance improvements can be achieved using the presented approach. The paper concludes on how the rewriter could be further improved.

Transport mechanisms for metadata-driven distributed multimedia adaptation

The information revolution of the last decade has resulted in a phenomenal increase in the quantity of multimedia content available to an increasing number of different users with different preferences and who access it through a plethora of devices and over heterogeneous networks. In order to address the amount of different content types, MPEG-21 digital item adaptation (DIA) introduces interoperable description tools which enable coding format independent adaptation. Bandwidth-efficient transport of the content to terminals with different capabilities and through a variety of access networks with various characteristics requires adaptation facilities not only on the server, but also within the network. We present transport mechanisms for MPEG-21-based metadata enabling generic adaptation within the network. Three different transport mechanisms for delivering this metadata in conjunction with the corresponding multimedia content are evaluated and a payload format for the transport of this metadata is presented. We performed measurements which demonstrate the bandwidth benefits of our distributed adaptation approach compared to server-centric adaptation in a multicast scenario. Finally, we applied various encoding formats for the metadata which further reduces the metadata overhead.

Scalable Video Coding Impact on Networks

This paper describes the CELTIC project on ”Scalable Video Coding Impact on Networks” with the focus of designing a streaming system based on the Scalable Video Coding extension of the H.264/AVC standard. The system is designed to cope with streaming scenarios that can be classified in four use cases: session handover, network congestion, receiver heterogeneity and user driven adaptation. A complete overview of the architecture of the system is given. Two demonstration scenarios are described in detail, which point out the advantages of scalable video coding compared to single layer approaches in multimedia transmission and adaptation scenarios. A concluding section summarizes the work and provides an outlook to future work items.

An Integrated Management Supervisor for End-to-End Management of Heterogeneous Contents, Networks, and Terminals enabling Quality of Service

End-to-end support for Quality of Service (QoS) has been broadly discussed in the literature. Many technologies have been proposed, each focusing on specific aspects for providing QoS guarantees to the end user. However, the integrated management of the end-to-end chain preserving QoS in heterogeneous environments is still a significant issue and insufficiently addressed to date. In this paper we propose an integrated management supervisor that takes into account the requirements from all stakeholders along the multimedia content delivery chain and provides an end-to-end management solution enabling QoS to the end user. This architecture and the subsystems that can be distributed along the end-to-end chain are detailed in this paper.

Design and Evaluation of a Metadata-Driven Adaptation Node

MPEG-21 digital item adaptation (DIA) allows for a media codec agnostic multimedia adaptation approach which enables the implementation of generic adaptation engines. However, DIA is optimized for static, server-based adaptation. In this paper we introduce novel mechanisms to extend the DIA approach towards dynamic and distributed scenarios. This facilitates the placement of generic adaptation nodes which perform media codec agnostic and dynamic adaptation anywhere along the content delivery path. To validate our work we implemented such an adaptation node and evaluate its performance.

Efficient Transformation of MPEG-21 Metadata for Codec-agnostic Adaptation in Real-Time Streaming Scenarios

Scalable media contents, such as the new MPEG-4 scalable video codec enable to easily retrieve different qualities of the media content by simply disregarding certain media segments. The MPEG-21-based codec-agnostic adaptation approach supports this concept by introducing anXML-based bitstream syntax description (BSD) which describes the different segments of a media content. Based on this BSD, an adaptation node can intelligently adapt any scalable media (i.e., remove specific media segments) without the need for codec-specific knowledge. The adaptation approach consists of 1) transforming this BSD and 2) adapting the media based on the transformed BSD. In this paper, we focus on the BSD transformation step and evaluate different mechanisms w.r.t. their transformation efficiency given several application scenarios. In particular, we compare the traditional style sheet-based mechanisms with a novel mechanism based on regular expressions. We discuss both mechanisms in terms of their expressiveness, and propose how to actually employ regular expressions for codec-agnostic adaptation. Finally, we quantitatively evaluate these mechanisms in different adaptation scenarios, which vary in the size and number of required BSD units.

Combined Adaptation and Caching of MPEG-4 SVC in Streaming Scenarios

A key objective of the ENTHRONE II Project is the ability to optimise the delivery of multimedia content to a wide group of heterogeneous users. One example of this is in the cooperative deployment of adaptation and caching functionality in the edge network. This hybrid approach makes it possible not only to store content locally, thus minimising the cost incurred through subsequent requests, but also to better serve heterogeneous groups of users by dynamically adapting the content to suit a wide range of terminal devices. In this paper, we describe and evaluate how the cooperative deployment of MPEG-21-based adaptation and caching of MPEG-4 SVC can result in improvements both in the quality of the content received at the user terminal and the resources consumed during the delivery.

Efficient SVC-to-AVC Conversion at a Media Aware Network Element

H.264/SVC, the Scalable Video Coding extension of the H.264/AVC video coding standard, features spatial, quality and temporal scalability. Backwards compatibility with legacy decoding devices is maintained through an H.264/AVC compliant base layer, which represents the lowest quality of an H.264/SVC bit-stream. However, it is often desirable to also provide the higher quality layers to legacy H.264/AVC devices. This is achieved by a process commonly known as “bit-stream rewriting”, which allows for an efficient H.264/SVC to H.264/AVC conversion by exploiting the similarities of the two codecs. This paper describes a demonstrator showing the advantages of including an improved version of the bit-stream rewriting tool from the existing JSVM H.264/SVC reference software in an H.264/SVC-based multimedia delivery system, by integrating it into a Media Aware Network Element.