Sam Lerouge

Multi-criteria optimization for scalable bitstreams

Since a few years, the heterogeneity of devices accessing multimedia is increasing. This introduces new problems for multimedia applications, that will probably be solved by the use of scalable or layered coding schemes. The content negotiation process of automatically selecting the best available version for a specific situation is complicated because video quality is composed of different independent aspects. This paper describes how we can apply the principles of multi-criteria optimization to determine the set of optimal solutions in a content negotiation process. It also introduces a basic formal model for describing scalable bitstreams.

BFlavor: A harmonized approach to media resource adaptation, inspired by MPEG-21 BSDL and XFlavor

During recent years, several tools have been developed that allow the automatic generation of XML descriptions containing information about the syntax of binary media resources. Such a bitstream syntax description (BSD) can then be transformed to reflect a desired adaptation of a media resource, and can subsequently be used to create a tailored version of this resource. The main contribution of this paper is the introduction of BFlavor, a new tool for exposing the syntax of binary media resources as an XML description. Its development was inspired by two other technologies, i.e. MPEG-21 BSDL and XFlavor. Although created from a different point of view, both languages offer solutions for translating the syntax of a media resource into an XML representation for further processing. BFlavor (BSDL+XFlavor) harmonizes the two technologies by combining their strengths and eliminating their weaknesses. More precisely, the processing efficiency and expressive power of XFlavor on the one hand, and the ability to create high-level BSDs using MPEG-21 BSDL on the other hand, were our key motives for its development. To assess the expressive power and performance of a BFlavor-driven content adaptation chain, several experiments were conducted. These experiments test the automatic generation of BSDs for MPEG-1 Video and H.264/AVC, as well as the exploitation of multi-layered temporal scalability in H.264/AVC. Our results show that BFlavor is an efficient and harmonized description tool for enabling XML-driven adaptation of media resources in a format-agnostic way. BSDL and XFlavor are outperformed by BFlavor in terms of execution times, memory consumption, and file sizes.

MPEG-21 bitstream syntax descriptions for scalable video codecs

In order to obtain a useful multichannel publication environment, a content producer has to respect the different terminal and network characteristics of the multimedia devices of its target audience. Embedded scalable video bitstreams, together with a complementary content adaptation framework, give the possibility to respond to heterogeneous usage environments. In this paper, temporally scalable H.264/MPEG-4 AVC encoded bitstreams and bitstreams encoded by relying on the fully-embedded MC-EZBC wavelet-based codec are used. The MPEG-21 Bitstream Syntax Description Language (BSDL) specification is used to generate high-level XML descriptions of the structure of a bitstream. As such, the adaptation of a scalable video stream can be realized in the XML domain, rather than on the bitstream itself. Different transformation technologies are compared to each other as well. Finally, a practical setup of a video streaming use case is discussed by relying on the MPEG-21 BSDL framework.

Lightweight object tracking in compressed video streams demonstrated in region-of-interest coding

Video scalability is a recent video coding technology that allows content providers to offer multiple quality versions from a single encoded video file in order to target different kinds of end-user devices and networks. One form of scalability utilizes the region-of-interest concept, that is, the possibility to mark objects or zones within the video as more important than the surrounding area. The scalable video coder ensures that these regions-of-interest are received by an end-user device before the surrounding area and preferably in higher quality. In this paper, novel algorithms are presented making it possible to automatically track the marked objects in the regions of interest. Our methods detect the overall motion of a designated object by retrieving the motion vectors calculated during the motion estimation step of the video encoder. Using this knowledge, the region-of-interest is translated, thus following the objects within. Furthermore, the proposed algorithms allow adequate resizing of the region-of-interest. By using the available information from the video encoder, object tracking can be done in the compressed domain and is suitable for real-time and streaming applications. A time-complexity analysis is given for the algorithms proving the low complexity thereof and the usability for real-time applications. The proposed object tracking methods are generic and can be applied to any codec that calculates the motion vector field. In this paper, the algorithms are implemented within MPEG-4 fine-granularity scalability codec. Different tests on different video sequences are performed to evaluate the accuracy of the methods. Our novel algorithms achieve a precision up to 96.4.

Enhancing RSS feeds: eliminating overhead through binary encoding

While RSS feeds increase in popularity as a mean to stay up to date with the most recent changes on a Web site, its XML representation is causing bandwidth related problems. These issues relate to the verbosity and the indivisible nature of the XML language. As such, bandwidth is wasted twofold as: (1) an RSS feed has no compact representation due to the plain text XML representation; and (2) previously received information about the RSS feed is discarded every time the RSS viewer retrieves the feed. Compression (binary encoding) of the XML data becomes relevant to eliminate the overhead. In this paper, we demonstrate the usefulness of the MPEG-7 binary format for metadata to address both overhead issues. We validate its usability in a typical RSS scenario by calculating the byte size reduction and by comparing the processing speed of creating and parsing binary encoded RSS feeds to traditional RSS feeds.

A performance evaluation of MPEG-21 BSDL in the context of H.264/AVC

H.264/AVC is a new specification for digital video coding that aims at a deployment in a lot of multimedia applications, such as video conferencing, digital television broadcasting, and internet streaming. This is for instance reflected by the design goals of the standard, which are about the provision of an efficient compression scheme and a network-friendly representation of the compressed data. Those requirements have resulted in a very flexible syntax and architecture that is fundamentally different from previous standards for video compression. In this paper, a detailed discussion will be provided on how to apply an extended version of the MPEG-21 Bitstream Syntax Description Language (MPEG-21 BSDL) to the Annex B syntax of the H.264/AVC specification. This XML based language will facilitate the high-level manipulation of an H.264/AVC bitstream in order to take into account the constraints and requirements of a particular usage environment. Our performance measurements and optimizations show that it is possible to make use of MPEG-21 BSDL in the context of the current H.264/AVC standard with a feasible computational complexity when exploiting temporal scalability.

Dynamic adaptation of multimedia data for mobile applications

The current explosive expansion of mobile communication systems will lead to an increased demand for multimedia applications. However, due to the large variety of mobile terminals (such as mobile phones, laptops .) and, because of this, a wide collection of different terminal possibilities and terminal characteristics, it is difficult to create a mobile multimedia application which can be used on mobile devices of different types. In this paper, we propose a mobile multimedia application that adapts its content to the possibilities of the mobile terminal and to the end-user preferences. Also the application takes changing device characteristics into account. To make this possible, a software framework is set up to enable negotiation between the mobile terminal and the content server. During the initial negotiation, the concept of the Universal Multimedia Access framework is used. Subsequent negotiations take place after changing terminal characteristics or end-user preferences, and this by means of time-dependent metadata. This newly created flexible and extendable framework makes it possible that multimedia applications interact with the content provider in order to deliver an optimal multimedia presentation for any arbitrary mobile terminal at any given time.

Advanced mobile multimedia applications: Using MPEG-21 and Time-Dependent metadata

While the price of mobile devices is dropping quickly, the set of features and capabilities of these devices is advancing very dramatically. Because of this, new mobile multimedia applications are conceivable, also thanks to the availability of high speed mobile networks like UMTS and Wireless LAN. However, creating such applications is still difficult due to the huge diversity of features and capabilities of mobile devices. Software developers also have to take into account the rigorous limitation on processing capabilities, display possibilities, and the limited battery life of these devices. On top of that, the availability of the device resources fluctuates strongly during execution of an application, directly and violently influencing the user experience, whereas equivalent fluctuations on traditional desktop PCs are far less prominent. Using new technology like MPEG-4, -7 and -21 can help application developers to overcome these problems. We have created an MPEG-21-based Video-on-Demand application optimized for mobile devices that is aware of the usage environment (i.e., user preference, device capabilities, device conditions, network status, etc.) of the client and adapts the MPEG-4 videos to it. The application is compliant with the Universal Multimedia Access framework, supports Time-Dependent Metadata, and relies on both MPEG-4 and MPEG-21 technology.

Comparison of XML serializations: cost benefits versus complexity

More and more data are structured, stored, and sent over a network using the Extensible Markup Language (XML) language. There are, however, concerns about the verbosity of XML in such a way that it may restrain further adoption of the language, especially when exchanging XML-based data over heterogeneous networks, and when it is used within constrained (mobile) devices. Therefore, alternative (binary) serialization formats of the XML data become relevant in order to reduce this overhead. However, usingbinary-encoded XML should not introduce interoperability issues with existing applications nor add additional complexity to new applications. On top of that, it should have a clear cost reduction over the current plain-text serialization format. A first technology is developed within the ISO/IEC Moving Picture Experts Group, namely the Binary MPEG Format for XML. It provides good compression efficiency, ability to (partially) update existing XML trees, and facilitates random access into, and manipulation of, the binary-encoded bit stream. Another technique is based on the Abstract Syntax Notation One specification with the Packed Encoding Rules created by the ITU-T. This paper evaluates both techniques as alternative XML serialization formats and introduces a solution for the interoperability concerns. This solution and the alternative serialization formats are validated against two real-life use cases in terms of processing speed and cost reduction. The efficiency of the alternative serialization formats are compared to a classic plain text compression technique, in particular ZIP compression.

Personalizing Quality Aspects in Scalable Video Coding

In video coding, certain limitations imposed by the environment, most typically the bit rate, need to be fulfilled. This is achieved by allowing the encoder to reduce the quality in one or several ways, such as the distortion, the resolution and the frame rate. The upcoming scalable video coding mechanisms allow this reduction to take place not exclusively during the encoding step, but at any time. This allows us to reduce the quality in a more personalized way, taking into consideration the preferences of the end user. This paper presents a framework that enables such user dependent quality reductions. We validated this framework by means of a test involving 19 test persons. The results of this mechanism are good, but up to now not sufficiently reliable to use it in commercial applications. At the same time, we still see some room for improvement