Shankar Regunathan

A generalized hypothetical reference decoder for H.264/AVC

In video coding standards, a compliant bit stream must be decoded by a hypothetical decoder that is conceptually connected to the output of an encoder and consists of a decoder buffer, a decoder, and a display unit. This virtual decoder is known as the hypothetical reference decoder (HRD) in H.263 and the video buffering verifier in MPEG. The encoder must create a bit stream so that the hypothetical decoder buffer does not overflow or underflow. These previous decoder models assume that a given bit stream will be transmitted through a channel of a known bit rate and will be decoded (after a given buffering delay) by a device of some given buffer size. Therefore, these models are quite rigid and do not address the requirements of many of todays important video applications such as broadcasting video live or streaming pre-encoded video on demand over network paths with various peak bit rates to devices with various buffer sizes. In this paper, we present a new HRD for H.264/AVC that is more general and flexible than those defined in prior standards and provides significant additional benefits.

Windows Media Video 9: overview and applications

Abstract Microsoft ® Windows Media 9 Series is a set of technologies that enables rich digital media experiences across many types of networks and devices. These technologies are widely used in the industry for media delivery over the internet and other media, and are also applied to broadcast, high definition DVDs, and digital projection in theaters. At the core of these technologies is a state-of-the-art video codec called Windows Media Video 9 (WMV-9), which provides highly competitive video quality for reasonable computational complexity. WMV-9 is currently under standardization by the Society of Motion Picture and Television Engineers (SMPTE) and the spec is at the CD (Committee Draft) stage. This paper includes a brief introduction to Windows Media technologies and their applications, with a focus on the compression algorithms used in WMV-9. We present analysis, experimental results, and independent studies that demonstrate quality benefits of WMV-9 over a variety of codecs, including optimized implementations of MPEG-2, MPEG-4, and H.264/AVC. We also discuss the complexity advantages of WMV-9 over H.264/AVC.

HD Photo: a new image coding technology for digital photography

This paper introduces the HD Photo coding technology developed by Microsoft Corporation. The storage format for this technology is now under consideration in the ITU-T/ISO/IEC JPEG committee as a candidate for standardization under the name JPEG XR. The technology was developed to address end-to-end digital imaging application requirements, particularly including the needs of digital photography. HD Photo includes features such as good compression capability, high dynamic range support, high image quality capability, lossless coding support, full-format 4:4:4 color sampling, simple thumbnail extraction, embedded bitstream scalability of resolution and fidelity, and degradation-free compressed domain support of key manipulations such as cropping, flipping and rotation. HD Photo has been designed to optimize image quality and compression efficiency while also enabling low-complexity encoding and decoding implementations. To ensure low complexity for implementations, the design features have been incorporated in a way that not only minimizes the computational requirements of the individual components (including consideration of such aspects as memory footprint, cache effects, and parallelization opportunities) but results in a self-consistent design that maximizes the commonality of functional processing components.

Low-complexity hierarchical lapped transform for lossy-to-lossless image coding in JPEG XR / HD Photo

JPEG XR is a draft international standard undergoing standardization within the JPEG committee, based on a Microsoft technology known as HD Photo. One of the key innovations in the draft JPEG XR standard is its integer-reversible hierarchical lapped transform. The transform can provide both bit-exact lossless and lossy compression in the same signal flow path. The transform requires only a small memory footprint while providing the compression benefits of a larger block transform. The hierarchical nature of the transform naturally provides three levels of multi-resolution signal representation. Its small dynamic range expansion, use of only integer arithmetic and its amenability to parallelized implementation lead to reduced computational complexity. This paper provides an overview of the key ideas behind the transform design in JPEG XR, and describes how the transform is constructed from simple building blocks.

A generalized video complexity verifier for flexible decoding

Video standards make use of a video complexity verifier (VCV) to characterize and regulate the decoding complexity of a compliant bitstream. In previous VCV models, the minimum level of computational capacity required to decode a given bitstream is strictly related to the peak decoding complexity of an individual frame in the sequence. This contribution presents a new VCV model, which is more flexible than prior VCV models. The proposed VCV model permits the bitstream to be decoded by devices operating at significantly smaller levels of peak computational capacity. The trade-off is a slight increase in additional delay and memory. Simulation results illustrate these benefits of the new VCV model.

Computationally efficient transforms for video coding

This paper describes the construction of computationally efficient transforms which can significantly reduce the complexity of a video decoder without loss in compression efficiency. In particular, these inverse transforms can be implemented using purely 16-bit arithmetic with similar rate-distortion performance to a 32-bit or floating point transform. To allow for variable block size coding, four 2D transforms are considered: one each for 8&8, 8&4, 4&8 and 4&4 blocks respectively. The design criteria for a pure 16 bit arithmetic implementation of the inverse transform are introduced, and used to derive the only useful set of transforms under these conditions. These transforms have been used in the WMV9/VC-1 codec to achieve significant reduction in computational complexity.

A flexible decoder buffer model for JVT video coding

Video coding standards require that a compliant bit stream be decodable by a hypothetical decoder that is conceptually connected to the output of an encoder and consists of a decoder buffer, a decoder, and a display unit. The encoder must create a bit stream so that the hypothetical decoder buffer does not overflow or underflow. Previous decoder models assume that a given bit stream will be transmitted through a channel of a given constant bit rate and will be decoded (after a given buffering delay) by a device of some given buffer size. Therefore, these models are quite rigid and do not address the requirements of many of todays important video applications such as broadcasting live video or streaming pre-encoded video on demand over network paths with various peak bit rates to devices with various buffer sizes. In this contribution, we present a new hypothetical reference decoder for JVT that is more general and flexible than those defined in prior standards and provides significant additional benefits.

Techniques for enhancing JPEG XR / HD Photo rate-distortion performance for particular fidelity metrics

This paper explores several encoder-side techniques aimed at improving the compression performance of encoding for the draft JPEG XR standard. Though the syntax and decoding process are fixed by the standard, significant variation in encoder design and some variation in decoder design are possible. For a variety of selected quality metrics, the paper discusses techniques for achieving better compression performance according to each metric. As a basic reference encoder and decoder for the discussion and modifications, the publically available Microsoft HD Photo DPK (Device Porting Kit) 1.0, on which the draft JPEG XR standard was based, was used. The quality metrics considered include simple mathematical objective metrics (PSNR and L∞) as well as pseudo-perceptual metrics (single-scale and multi-scale MSSIM).

Coding of high dynamic range images in JPEG XR / HD Photo

High Dynamic Range (HDR) imaging support is one of the major features for the emerging draft JPEG XR standard. JPEG XR is being standardized within the JPEG committee based on Microsoft technology known as HD Photo. JPEG XR / HD Photo is primarily an integer-based coding technology design, accepting integer valued samples at the encoder and producing integer valued samples at the decoder, with internal processing entirely in the integer space. Yet, it can support compression of multiple HDR formats, including 16- and 32-bit float, 16-bit and 32-bit signed and unsigned integer, and RGBE. Further, JPEG XR can enable lossless compression of some HDR formats such as 16-bit signed and unsigned, 16-bit float and RGBE. This paper describes how HDR formats are handled in JPEG XR. It examines in depth how these various HDR formats are converted to and from integer valued samples within the JPEG XR codec, and the internal processing of these HDR formats. This paper describes how JPEG XR provides flexible ways to compress HDR formats within the same codec framework as integer-valued formats, while maintaining from the high compression efficiency and low computational complexity.

Quality and compression : The proposed SMPTE video compression standard VC-1

As PCs and other digital devices play an increased role in manipulating, delivering, and managing digital video, in both home and professional broadcast environments, the distinction between consumer electronics and computers begins to blur. Consumers want to use their PCs to easily enjoy, manage, and share digital media across different devices, from cable set-top boxes to DVD players to laptops and television sets. Home networks are becoming increasingly media-savvy, facilitating movement of digital content to/from the living room and den. Broadcasters and content publishers want to deliver the experience that consumers demand and must compress the content-be it standard-definition or high-definition video-for digital broadcast, disk, or download, with as little signal loss and degradation as possible. Overall, there is an increasing need for higher quality video and audio at lower and lower bit rates for an increasing range of devices and applications. This paper will describe the underlying technology and concepts embodied in VC-1, the proposed SMPTE standard based on the Windows Media Video 9 advanced video codec.