Bruno Zatt

An adaptive early skip mode decision scheme for multiview video coding

In this work a novel scheme is proposed for adaptive early SKIP mode decision in the multiview video coding based on mode correlation in the 3D-neighborhood, variance, and ratedistortion properties. Our scheme employs an adaptive thresholding mechanism in order to react to the changing values of Quantization Parameter (QP). Experimental results demonstrate that our scheme provides a consistent time saving over a wide range of QP values. Compared to the exhaustive mode decision, our scheme provides a significant reduction in the encoding complexity (up to 77%) at the cost of a small PSNR loss (0.172 dB in average). Compared to state-of-the-art, our scheme provides an average 2× higher complexity reduction with a relatively higher PSNR value (avg. 0.2 dB).

Multi-level pipelined parallel hardware architecture for high throughput motion and disparity estimation in Multiview Video Coding

This paper presents a novel motion and disparity estimation (ME, DE) scheme in Multiview Video Coding (MVC) that addresses the high throughput challenge jointly at the algorithm and hardware levels. Our scheme is composed of a fast ME/DE algorithm and a multi-level pipelined parallel hardware architecture. The proposed fast ME/DE algorithm exploits the correlation available in the 3D-neighborhood (spatial, temporal, and view). It eliminates the search step for different frames by prioritizing and evaluating the neighborhood predictors. It thereby reduces the coding computations by up to 83% with 0.1 dB quality loss. The proposed hardware architecture further improves the throughput by using parallel ME/DE modules with a shared array of SAD (Sum of Absolute Differences) accelerators and by exploiting the four levels of parallelism inherent to the MVC prediction structure (view, frame, reference frame, and macroblock levels). A multi-level pipeline schedule is introduced to reduce the pipeline stalls. The proposed architecture is implemented for a Xilinx Virtex-6 FPGA and as an ASIC with an IBM 65nm low power technology. It is compared to state-of-the-art at both algorithm and hardware levels. Our scheme achieves a real-time (30fps) ME/DE in 4-view High Definition (HD1080p) encoding with a low power consumption of 81 mW.

Power-efficient error-resiliency for H.264/AVC context-adaptive variable length coding

Technology scaling has led to unreliable computing hardware due to high susceptibility against soft errors. In this paper, we propose an error-resilient architecture for Context-Adaptive Variable Length Coding (CAVLC) in H.264/AVC. Due to its context-adaptive nature and intricate control flow CAVLC is very sensitive to soft errors. An error during the CAVLC process (especially during the context adaptation or in VLC tables) may result in severe mismatch between encoder and decoder. The primary goal in our error-resilient CAVLC architecture is to protect codeword/codelength tables and context adaptation in a reliable yet power efficient manner. For reducing the power over-head, the tables are partitioned in various sub-tables each protected with variable-sized parity. Moreover, for further power reduction, our approach incorporates state-retentive power-gating of different sub-tables at run time depending upon the statistical distribution of syntax elements. Compared to the unprotected case, our scheme provides a video quality improvement of 18dB (averaged over various fault injection cases and video sequences) at the cost of a 35% area overhead and 45% performance overhead due to the error-detection logic. However, partitioned sub-tables increase the potential for power-gating, thus bring a leakage energy saving of 58%. Compared to state-of-the-art table protection, our scheme provides 2x reduced area and performance overhead. For function-al verification and area comparison, the architecture is prototyped on a Xilinx Virtex-5 FPGA, though not limited to it. For the soft errors experiments, evaluation of error-resiliency and power efficiency, we have developed a fault injection and simulation setup.

A Model Predictive Controller for Frame-Level Rate Control in Multiview Video Coding

In this work, we present a novel frame-level Rate Control algorithm for Multiview Video Coding encoder that adopts the Model Predictive Control technique in order to provide low bitrate fluctuation and high video quality. Our Model Predictive Rate Control (MPRC) predicts the bitrate for a frame by employing (i) inter-view inter-GOP (Group of Pictures) phase-based bitrate prediction, and (ii) temporal (intra-GOP) target bitrate linear weighting. Moreover, the MPRC also defines an optimal control action through frame-level QP value selection. Experimental results demonstrate that our MPRC bitrate prediction incurs a Mean Bit Estimation Error (MBEE) of 1.13% compared to 2.46% provided by single view-based Rate Control and 1.61% provided by the state-of-the-art MVC Rate Control. Our solution also provides on average 0.876dB BD-PSNR increase and 28.92% BD-Bitrate reduction while providing smoother quality and bitrate variations when compared to state-of-the-art.

A multi-level dynamic complexity reduction scheme for multiview video coding

In this paper, we propose a novel scheme for dynamically reducing the computational complexity of MVC. Our scheme exploits the coding mode correlation available in the 3D-neighborhood (i.e., spatial, temporal, and view) along with the rate-distortion properties of the neighboring Macroblocks. Our scheme incorporates a multi-level mode decision process based on a mode-ranking mechanism that categorizes more-probable and less-probable coding modes. In order to react to the changing bitrates, our scheme deploys Quantization Parameter based threshold equations which are formulated using an offline statistical analysis. Compared to the exhaustive Rate-Distortion-Optimized Mode Decision (RDO-MD), our scheme achieves a complexity reduction of up to 80% (68% on average) with an average PSNR loss of 0.075 dB. Compared to state-of-the-art fast RDO-MD, our scheme achieves a complexity reduction of up to 34% with an average PSNR gain of 0.007 dB.

Power-aware complexity-scalable multiview video coding for mobile devices

We propose a novel power-aware scheme for complexity-scalable multiview video coding on mobile devices. Our scheme exploits the asymmetric view quality which is based on the binocular suppression theory. Our scheme employs different quality-complexity classes (QCCs) and adapts at run time depending upon the current battery state. It thereby enables a run-time tradeoff between complexity and video quality. The experimental results show that our scheme is superior to state-of-the-art and it provides an up to 87% complexity reduction while keeping the PSNR close to the exhaustive mode decision. We have demonstrated the power-aware adaptivity between different QCCs using a laptop with battery charging and discharging scenarios.

Background and Related Work

This monograph envisions adaptive low-power multimedia systems covering both the application and processor perspectives. Besides low power consumption, a special focus is on the support for adaptivity which is inevitable when considering the rapid evolution of the multimedia/video standards and high unpredictability due to user interactions, input data, and inclusion of adaptive algorithms in advanced standards. In order to support adaptivity dynamically reconfigurable processors are considered in this monograph. This chapter provides basics and terminology used in video coding and an overview of the H.264 video encoder which is one of the latest video coding standards. Afterwards, a general background of the reconfigurable processors and their low-power infrastructure is discussed in Sect. 2.3 followed by the prominent related work in dynamically reconfigurable processors and low-power approaches for reconfigurable computing. Especially, the RISPP processor [Bau09] is presented in detail as it is used for detailed benchmarking of the processor-level contribution of this monograph (i.e., adaptive low-power processor architecture).

Results and Comparison

In this chapter the overall results of this work and the comparison with the latest state-of-the-art approaches are presented. Before moving to the actual comparison, a description of the experimental setup is presented discussing the fairness of comparison in relation to the related works. The benchmark video properties, common test conditions, simulation environment, and synthesis tool chain are also introduced in this chapter. The results for energy-efficient algorithms are discussed in terms of complexity reduction while considering the coding efficiency and video quality in relation to state-of-the-art and optimal solutions. The video quality control algorithm based on rate control is compared to other rate control techniques described in the current literature. Energy-efficient architectures are evaluated against the latest hardware solutions for ME/DE on MVC with emphasis on the overall energy consumption for both memory access and processing datapath. Additionally, throughput and IC footprint area are discussed.

Conclusion and Future Works

The presented monograph focuses on the energy reduction of the Multiview Video Coding (MVC) encoder to enable the realization of real-time high-definition 3D-video encoding running on mobile embedded devices with battery-constrained energy. For that, novel energy-efficient techniques are proposed at both algorithmic and architectural abstraction levels. The joint consideration of algorithms and underlying hardware architecture is the key enabler to provide improved energy efficiency, as demonstrated along this monograph.

Energy-Efficient Algorithms for Multiview Video Coding

The energy consumption in MVC encoding is directly related to the high computational effort and the intense memory access driven by the data processing. Therefore, the energy-efficient algorithms for the Multiview Video Coding proposed in this monograph are based on complexity reduction and complexity control techniques. Moreover, in addition to the energy consumption perspective, meaningful complexity reduction is also required at the performance perspective in order to make MVC real-time encoding feasible for real-world embedded devices.