Iain E. Garden Richardson

H.264 and MPEG-4 Video Compression

About the Author.Foreword.Preface.Glossary.1. Introduction.2. Video Formats and Quality.3. Video Coding Concepts.4. The MPEG-4 and H.264 Standards.5. MPEG-4 Visual.6. H.264/MPEG-4 Part 10.7. Design and Performance.8. Applications and Directions.Bibliography.Index.

Low-complexity skip prediction for H.264 through Lagrangian cost estimation

A complexity reduction algorithm for an H.264 encoder is proposed. Computational savings are achieved by identifying, prior to motion estimation, macroblocks (MBs) that are likely to be skipped and hence saving further computational processing of these MBs. This early prediction is made by estimating a Lagrangian rate-distortion cost function which incorporates an adaptive model for the Lagrange multiplier parameter based on local sequence statistics. Simulation results demonstrate that the algorithm can achieve computational savings of 19%-67% (depending on the source sequence) with no significant loss of rate-distortion performance.

Computational Complexity Management of a Real-Time H.264/AVC Encoder

The H.264 video coding standard supports efficient coding of video at the expense of high computational complexity. This work addresses the problem of maintaining acceptable video coding performance in a computation-constrained application scenario. A complexity management approach is proposed for an H.264 encoder running in a processor/power-constrained environment. We hypothesize that, in a power-constrained application such as mobile video telephony, good perceptual quality requires a balance between a high frame rate and acceptable image quality. Therefore, the objective of the complexity management approach is to maintain a smooth video frame rate whilst ensuring that the frame quality is not degraded unacceptably. A frame-level algorithm calculates a target coding time for each frame and drops frames when necessary to maintain acceptable image quality. A per-frame algorithm controls the coding complexity of each frame in order to achieve the target coding time. The performance of the approach is evaluated by carrying out subjective tests and comparing the managed complexity encoder with a reference encoder in a computation-constrained scenario. Subjective results show that the managed complexity encoder consistently achieves superior perceptual video quality ratings compared to the reference encoder.

A new perceptual quality metric for compressed video

This paper presents a new video quality metric for automatically estimating the perceptual quality of compressed video sequences. Distortion measures such as the mean squared error (MSE) and the peak signal to noise ratio (PSNR) have been found to poorly correlate with visual quality at lower bit-rates. The proposed quality metric (MOSp) predicts perceptual quality of compressed video using sequence characteristics and the mean squared error (MSE) between the original and compressed video sequences. The metric has been tested on various video sequences compressed using the H.264 video compression standard at different bit-rates. Results show that the proposed metric has better correlation with subjective quality compared to popular metrics such as PSNR, SSIM and PSNRplus. The new metric is simple to compute and hence suitable for incorporation into real-time applications such as the standard video compression codecs inorder to improve the visual quality of compressed video sequences.

Complexity Control of H.264/AVC Based on Mode-Conditional Cost Probability Distributions

A computational complexity control algorithm is proposed for an H.264 encoder running on a processor/power constrained platform. This new computational complexity control algorithm is based on a macroblock mode prediction algorithm that employs a Bayesian framework for accurate early skip decision. Complexity control is achieved by relaxing the Bayesian maximum-likelihood (ML) criterion in order to match the mode decision threshold to a target complexity level. A feedback algorithm is used to maintain the performance of the algorithm with respect to achieving an average target complexity level, reducing frame by frame complexity variance and optimizing rate-distortion performance. Experimental results show that this algorithm can effectively control the encoding computational complexity while maintaining a good rate-distortion performance at a range of target complexity levels.

A Full Reference Quality Metric for Compressed Video Based on Mean Squared Error and Video Content

Visual quality of compressed video sequences depends on factors including spatial texture content and cognition-based factors such as prior knowledge and task in hand. The MOSp metric is a full reference objective quality metric which predicts perceived quality of sequences with video compression-induced impairments based on the spatial texture content and the mean squared error between original and compressed video sequences. In this paper, we extend the MOSp metric to incorporate cognition-based factors to identify regions in a video scene that attract human attention. The proposed metric has been tested on a variety of multimedia sequences of common intermediate format resolution compressed at a wide range of bitrates using the H.264/AVC coding standard. This metric shows a higher correlation with mean opinion score (MOS) than popular metrics, such as peak signal-to noise ratio, National Telecommunications and Information Administration/Institute for Telecommunication Sciences video quality metric, PSNRplus, and the Yonsei University metric. Results also show that by extending the MOSp metric to incorporate cognition-based factors such as skin information, its correlation with subjective scores (MOS) can be significantly improved in video content where humans are present. This algorithm is particularly useful for real-time quality estimation of multimedia sequences with block-based video compression-induced impairments because all the parameters of the metric can be calculated automatically with a modest amount of processing overhead.

Computational complexity management of motion estimation in video encoders

Summary form only given. The performance of software-only video codecs is often constrained by available processing power. Existing fast motion estimation algorithms are not designed to provide flexible, predictable control of computational complexity. We propose an adaptive algorithm, which maintains the computational complexity of the motion estimation function at various target levels by controlling the motion estimation search pattern.

Adaptive algorithms for variable-complexity video coding

Variable-complexity algorithms provide a means of managing the computational complexity of a software video CODEC. The reduction in computational complexity provided by existing variable-complexity algorithms depends on the video scene characteristics and is difficult to predict. A new approach to variable-complexity encoding is proposed. A variable-complexity DCT algorithm is adaptively updated in order to maintain a near-constant computational complexity. The adaptive update algorithm is shown to be capable of providing a significant, predictable, reduction in computational complexity with only a small loss of video quality. The proposed approach may be particularly useful for software-only video encoding, in applications where processing resources are limited.

A novel perceptual quality metric for video compression

Modern video compression systems make optimum coding decisions based on rate-distortion performance. Typically the distortion is evaluated as a mathematical error measurement, such as mean squared error (MSE), between the original and compressed video sequences. However, simple automatic error measurements such as MSE do not correlate well with perceptual quality, leading to sub-optimal coding decisions. In this paper we present a new subjective video quality metric that predicts subjective quality of compressed video using temporal and spatial masking information and the MSE between the original and compressed video sequences. The results show that this metric can predict perceptual quality with significantly higher correlation compared to popular metrics such as PSNR, VSSIM and PSNRplus. This algorithm is particularly useful for real-time, accurate perceptual video quality estimation in video applications because all the parameters of the metric can be calculated with a minimal amount of processing overhead.

Complexity management for video encoders

Computational complexity is an important performance constraint for software-only video CODECs. The aim of this research is to develop a video coding system with variable, controllable computational complexity. Adaptive algorithms for DCT and motion estimation are proposed separately to reduce complexity of each function and maintain it at target level. An integrated approach to video CODEC complexity management is also addressed. This work will have potential benefit for a wide range of computation-constrained or power-constrained multimedia applications.