Stephan Wenger

H.264/AVC over IP

H.264 is the ITU-Ts new, nonbackward compatible video compression Recommendation that significantly outperforms all previous video compression standards. It consists of a video coding layer (VCL) which performs all the classic signal processing tasks and generates bit strings containing coded macroblocks, and a network adaptation layer (NAL) which adapts those bit strings in a network friendly way. The paper describes the use of H.264 coded video over best-effort IP networks, using RTP as the real-time transport protocol. After a description of the environment, the error-resilience tools of H.264 and the draft specification of the RTP payload format are introduced. Next the performance of several possible VCL- and NAL-based error-resilience tools of H.264 are verified in simulations.

Error resilience support in H.263

Version 2 of ITU Recommendation H.263, better known as H.263+, includes a number of new mechanisms to improve coding efficiency and support various types of networks more efficiently. This paper provides an overview of the error resilience optional modes of H.263+ and describes the use of such modes in various multimedia network scenarios.

Overview of HEVC High-Level Syntax and Reference Picture Management

The increasing proportion of video traffic in telecommunication networks puts an emphasis on efficient video compression technology. High Efficiency Video Coding (HEVC) is the forthcoming video coding standard that provides substantial bit rate reductions compared to its predecessors. In the HEVC standardization process, technologies such as picture partitioning, reference picture management, and parameter sets are categorized as “high-level syntax.” The design of the high-level syntax impacts the interface to systems and error resilience, and provides new functionalities. This paper presents an overview of the HEVC high-level syntax, including network abstraction layer unit headers, parameter sets, picture partitioning schemes, reference picture management, and supplemental enhancement information messages.

System Layer Integration of High Efficiency Video Coding

This paper describes the integration of High Efficiency Video Coding (HEVC) into end-to-end multimedia systems, formats, and protocols such as Real-time transport Protocol, the transport stream of the MPEG-2 standard suite, and dynamic adaptive streaming over the Hypertext Transport Protocol. This paper gives a brief overview of the high-level syntax of HEVC and the relation to the Advanced Video Coding standard (H.264/AVC). A section on HEVC error resilience concludes the HEVC overview. Furthermore, this paper describes applications of video transport and delivery such as broadcast, television over the Internet Protocol, Internet streaming, video conversation, and storage as provided by the different system layers.

H.26L/JVT coding network abstraction layer and IP-based transport

The JVT/H.26L video coding scheme conceptually consists of a video coding layer (VCL) responsible mainly for coding efficiency and a network abstraction layer (NAL) that supports video specific transport features for a variety of networks. This paper describes the H.26L/JVT NAL in general, including the parameter set concept and the transport over packet-based and bit-stream oriented networks. Encapsulation of coded video data to RTP/UDP/IP transport is discussed, and applications of H.26L in fixed and wireless IP-based transmission environments are presented.

Optimized transmission of H.26L/JVT coded video over packet-lossy networks

Transmission of hybrid coded video including motion compensation and spatial prediction over error-prone channels results in the well-known problem of spatio-temporal error propagation at the decoder. A widely accepted standard-compliant technique to enhance the quality of the decoded video significantly is the more frequent introduction of intra-coded macroblocks. However, intra-coded information generally requires more bit rate. Therefore, a careful selection of intra-updates in terms of rate and distortion is necessary. A flexible and robust rate-distortion optimization technique is presented to select coding mode and reference frame for each macroblock. The channel statistics are included in the optimization process. We derive a method to obtain an estimate of the decoder pixel distortion at the encoder. The presented techniques are verified within the new H.26L/JVT video coding standard based on common test conditions.

Video rate control for streaming and local recording optimized for mobile devices

In this paper, we propose a real-time, low-complexity video rate control algorithm designed to obey buffer constraints. The algorithm is optimized for streaming and local recording applications in mobile devices. Today, most mobile phones include a digital camera that can be used to capture video. The on-phone processor technology has become powerful enough to encode video in real-time. The resulting file can, for example, be archived in the phones memory, or (progressively or as one block) downloaded, through the 3G mobile network, Bluetooth, or WLAN, to Internet-connected computer systems. From here, all forms of multimedia transmission, such as streaming, file sharing, or multimedia mail become possible. In local recording and streaming applications on a mobile phone, we assume that no memory for storage of uncompressed video is available. Therefore, look-ahead and multi-path rate control is not possible. Furthermore, considering the processing power and, more importantly, battery life constraints in mobile devices, the proposed algorithm needs to be as simple as possible. The described algorithm implements a variable bitrate (VBR) by controlling the quantization scale (QS) on a per picture basis. The QS is calculated based on two other QSs, which correspond to constant rate and constant quality rate controls. The algorithm utilizes the variable bitrate benefits as much as possible so as to minimize the variation of the QS scale, and to provide encoded video with high visual quality. Although it strictly obeys buffering constraints as discussed later, the experimental results show that it allows encoded video at average quality levels significantly higher than reported in earlier works

Picture orientation information in video compression

This paper proposes changing of picture orientation on a picture-by-picture basis, without requiring the encoding of an Instantaneous Decoder Refresh (IDR) picture. It can be implemented by small modifications to H.264/MPEG-4 AVC and the design of the upcoming High Efficiency Video Coding (HEVC) standard, or most other video coding technologies employing inter picture prediction. To enable picture orientation changes, a high layer syntax element is needed to indicate the rotation to be applied to the decoded pictures. Modifications to the decoding process are also required to avoid the use of IDR pictures. For some applications with handheld cameras that may be held at different orientations, the proposed modifications allow shifting the rotation function to the decoding end, reducing overall system complexity. Additionally, selection by the encoder of the coded picture orientation may result in better coding efficiency. In this respect, an algorithm for selecting the coded picture orientation limited to intra coding is also described, and experimental results show coding efficiency improvements up to 3.4% BD-rate gain.