Yves Dhondt

Flexible macroblock ordering in H.264/AVC

Abstract H.264/AVC is a new standard for digital video compression jointly developed by ITU-T’s Video Coding Experts Group (VCEG) and ISO/IEC’s Moving Picture Experts Group (MPEG). Besides the numerous tools for efficient video coding, the H.264/AVC specification defines some new error resilience tools. One of them is flexible macroblock ordering (FMO) which is the main focus of this paper. An in-depth overview is given of the internals of FMO. Experiments are presented that demonstrate the benefits of FMO as an error resilience tool in case of packet loss over IP networks. The flexibility of FMO comes with a certain overhead or cost. A quantitative assessment of this cost is presented for a number of scenarios. FMO can, besides for pure error resilience, also be used for other purposes. This is also addressed in this paper.

A Flexible Macroblock Scheme for Unequal Error Protection

This paper proposes an enhanced error protection scheme using flexible macroblock ordering in H.264/AVC. The algorithm uses a two-phase system. In the first phase, the importance of every macroblock is calculated based on its influence on the current frame and future frames. In the second phase, the macroblocks with the highest impact factor are grouped together in a separate slice group using the flexible macroblock ordering feature of H.264/AVC. By using an unequal error protection scheme, the slice group containing the most important macroblocks can be better protected than the other slice group. The proposed algorithm offers better concealment opportunities than the algorithms which are predefined for flexible macroblock ordering in H.264/AVC.

ViQID: A No-Reference bit stream-based visual quality impairment detector

In order to ensure adequate quality towards the end users at all time, video service providers are getting more interested in monitoring their video streams. Objective video quality metrics provide a means of measuring (audio)visual quality in an automated manner. Unfortunately, most of the current existing metrics cannot be used for real-time monitoring due to their dependencies on the original video sequence. In this paper we present a new objective video quality metric which classifies packet loss as visible or invisible based on information extracted solely from the captured encoded H.264/AVC video bit stream. Our results show that the visibility of packet loss can be predicted with a high accuracy, without the need for deep packet inspection. This enables service providers to monitor quality in real-time.

A real-time content adaptation framework for exploiting ROI scalability in H.264/AVC

It is well known that motion detection using single frame differencing, while computationally much simpler than other techniques, is more liable to generate large areas of false foregrounds known as ghosts. In order to overcome this problem the authors propose a method based on signed differencing and connectivity analysis. The proposal is suitable to applications which cannot afford the un-avoidable errors of background modeling or the limitations of 3-frames preprocessing.In many application scenarios, the use of Regions of Interest (ROIs) within video sequences is a useful concept. It is shown in this paper how Flexible Macroblock Ordering (FMO), defined in H.264/AVC as an error resilience tool, can be used for the coding arbitrary-shaped ROIs. In order to exploit the coding of ROIs in an H.264/AVC bitstream, a description-driven content adaptation framework is introduced that is able to extract the ROIs of a given bitstream. The results of a series of tests indicate that the ROI extraction process significantly reduces the bit rate of the bitstreams and increases the decoding speed. In case of a fixed camera and a static background, the impact of this reduction on the visual quality of the video sequence is negligible. Regarding the adaptation framework itself, it is shown that in all cases, the framework operates in real time and that it is suited for streaming scenarios by design.

A performance evaluation of the data partitioning tool in H.264/AVC

In order to be able to better cope with packet loss, H.264/AVC, besides offering superior coding efficiency, also comes with a number of error resilience tools. The goal of these tools is to enable the decoding of a bitstream containing encoded video, even when parts of it are missing. On top of that, the visual quality of the decoded video should remain as high as possible. In this paper, we will discuss and evaluate one of these tools, in particular the data partitioning tool. Experimental results will show that using data partitioning can significantly improve the quality of a video sequence when packet loss occurs. However, this is only possible if the channel used for transmitting the video allows selective protection of the different data partitions. In the most extreme case, an increase in PSNR of up to 9.77 dB can be achieved. This paper will also show that the overhead caused by using data partitioning is acceptable. In terms of bit rate, the overhead amounts to approximately 13 bytes per slice. In general, this is less than 1% of the total bit rate. On top of that, using constrained intra prediction, which is required to fully exploit data partitioning, causes a decrease in quality of about 0.5 dB for high quality video and between 1 and 2 dB for low quality video.

An Alternative Scattered Pattern for Flexible Macroblock Ordering in H.264/AVC

Spatial error concealment methods in block-based video codecs use information from surrounding macroblocks to repair lost macroblocks. Using the slice group tool of H.264/AVC, it is possible to encode the macroblocks of a frame based on a scattered pattern, hereby allowing an encoder to actively help concealment methods at the decoder side. In this paper, the requirements for such a pattern are presented. Two commonly used patterns are verified against those requirements and an alternative scattered pattern is presented. Experiments show that our alternative pattern outperforms the old scattered patterns by up to 1.5 dB in case two or more slice groups are lost.

Low Complexity Multiple Description Coding for H.264/AVC

Todays wireless networks suffer from high error rates in real-life conditions. Moreover errors tend to occur in large bursts. Multiple Description Coding (MDC) of digital video is a promising technique to overcome these problems in wireless environments by taking advantage of frequency or channel diversity. This papers proposes two low complexity MDC techniques for H.264/AVC at the Network Abstraction Layer level, aiming at conversational (video teleconferencing) and sports Video on Demand (VoD) application scenarios, respectively. For conversational applications, an MDC scheme based on H.264/AVC data partitioning is used, offering low additional delay and good performance for video with static backgrounds. For the sports VoD application, a scheme based on redundant slices is proposed. The latter scheme performs better for high-motion sequences than the former, but also introduces more delay.

Using Placeholder Slices and MPEG-21 BSDL for ROI Extraction in H.264/AVC FMO-Encoded Bitstreams

The concept of Regions of Interest (ROIs) within a video sequence is useful for many application scenarios. This paper concentrates on the exploitation of ROI coding within the H.264/AVC specification by making use of Flexible Macroblock Ordering. It shows how ROIs can be coded in an H.264/AVC compliant bitstream and how the MPEG-21 BSDL framework can be used for the extraction of the ROIs.

An evaluation of flexible macroblock ordering in error-prone environments

With all the hype created around multimedia in the last few years, consumers expect to be able to access multimedia content in a real-time manner, anywhere and anytime. One of the problems with the real-time requirement is that transportation networks, such as the Internet, are still prone to errors. Due to real-time constraints, retransmission of lost data is, more often than not, not an option. Therefore, the study of error resilience and error concealment techniques is of the utmost importance since it can seriously limit the impact of a transmission error. In this paper an evaluation of a part of flexible macroblock ordering, one of the new error resilience techniques in H.264/AVC, is made by analyzing its costs and gains in an error-prone environment. This paper concentrates on the study of flexible macroblock ordering (FMO). More specifically a study of scattered slices, FMO type 1, is made. Our analysis shows that FMO type 1 is a good tool to introduce error robustness into an H.264/AVC bitstream as long as the QP is higher than 30. When the QP of the bitstream is below 30, the cost of FMO type 1 becomes a serious burden.

XML-based Exploitation of Region of Interest Scalability in Scalable Video Coding

The use of Regions Of Interest (ROIs) is a useful concept for many application scenarios, especially for those applications that are deployed in heterogeneous multimedia environments. In this paper, we show how Flexible Macroblock Ordering can be used in the scalable extension of the H.264/AVC specification in order to define the ROIs in the coded bit- stream. Furthermore, we introduce an XML-driven adaptation framework based on the MPEG-21 Bitstream Syntax Description Language in order to implement the ROI extraction process. This framework gives us the opportunity to adapt scalable bitstreams by using an engine that has no knowledge of the underlying coding format. From the performance analysis of our adaptation framework, we can conclude that the ROIs can be extracted in the XML domain and that the ROIs in the adapted bitstream are still intact without quality degradation. Furthermore, the traditional drifting problem caused by the ROI extraction can be neglected. Finally, we show that the adaptation process in the XML domain can be executed in real time.