Yongfang Liang

Power-rate-distortion analysis for wireless video communication under energy constraints

Mobile devices performing video coding and streaming over wireless and pervasive communication networks are limited in energy supply. To prolong the operational lifetime of these devices, an embedded video encoding system should be able to adjust its computational complexity and energy consumption as demanded by the situation and its environment. To analyze, control, and optimize the rate-distortion (R-D) behavior of the wireless video communication system under the energy constraint, we develop a power-rate-distortion (P-R-D) analysis framework, which extends the traditional R-D analysis by including another dimension, the power consumption. Specifically, in this paper, we analyze the encoding mechanism of typical video coding systems, and develop a parametric video encoding architecture which is fully scalable in computational complexity. Using dynamic voltage scaling (DVS), an energy consumption management technology recently developed in CMOS circuits design, the complexity scalability can be translated into the energy consumption scalability of the video encoder. We investigate the R-D behavior of the complexity control parameters and establish an analytic P-R-D model. Both theoretically and experimentally, we show that, using this P-R-D model, the video coding system is able to automatically adjust its complexity control parameters to match the available energy supply of the mobile device while maximizing the picture quality. The P-R-D model provides a theoretical guideline for system design and performance optimization in mobile video communication under energy constraints.

Motion Estimation for Content Adaptive Video Compression

Motion estimation is a multistep process that encompasses techniques such as motion vector prediction, determination of search range and search patterns, and identification of termination criteria. Each of these techniques has several diversions that may suit a particular set of video characteristics. It would be hard to conceive a universal algorithm that can perform well for all kinds of video contents. However, if important characteristics of a video sequence can be identified and utilized for adjusting various steps of motion estimation, one can design an adjustable algorithm that can tune its parameters to suit the video at hand. A multistage motion estimation algorithm that includes a pre-stage for analyzing the motion characteristics of a video sequence online is proposed. This stage predicts the motion vector field (MVF) from the previous coded frame, and clusters macroblocks into background and foreground regions based on the predicted MVF. The information from the pre-stage are passed on to the next stage, which includes a mathematical model for block distortion surface to estimate the distance from the current search point to the global optimal position. This allows the motion estimation stage to adjust its search strategies accordingly. The search is performed on a precise area according to the statistical properties of the motion vector prediction error, separately for foreground and background regions. The proposed algorithm, including the pre-stage, is fast and, therefore, is suitable for online and real-time encoding. Extensive simulation results obtained for several video sequences affirm the effectiveness of the proposed algorithm. The self-tuning property enables the algorithm to perform well for several types of benchmark sequences, yielding better video quality without exceeding the time complexity as compared to the other predictive motion estimation algorithms.

Power and Distortion Optimization for Pervasive Video Coding

This paper addresses the problem of how to efficiently manage the power consumption while preserving high visual quality performance in video coding for pervasive video applications. We define an optimization problem that corresponds to the minimization of both the power consumption and video distortion of a video encoder. A joint rate-complexity-distortion(R-C-D) model is developed to capture and control the general R-C-D behavior of video coding. Based on the analysis of the proposed model, optimization strategies are developed to achieve good video quality while maximizing the battery service life. We demonstrate how the observations and optimization strategies can be utilized in a practical parameterized encoder under various preferences of quality and power. We perform simulations on an instruction set simulator to exhibit the effectiveness of the proposed scheme.

Analysis and design of power constrained video encoder

In wireless communication networks, an important issue that must be addressed is the limited energy supply of a mobile device, especially in wireless video applications. In this paper, we develop a parametric video encoding architecture, which is fully scalable in power consumption, and establish the power-rate-distortion (P-R-D) model of the video encoding system. Both theoretically and experimentally, we show that by using this P-R-D model, the encoding system is able to automatically adjust its complexity control parameters to match the available energy supply of the mobile device while maximizing the picture quality. The P-R-D model provides a theoretical guideline for system design and performance optimization in wireless video communication under power constraint.

MPEG-4 to H.264/AVC transcoding

In this paper, we consider the problem of MPEG-4 visual simple profile to H.264/AVC baseline profile (BP) transcoding. An efficient transcoding algorithm with fast mode selection and motion vector refinement is proposed. The proposed algorithm exploits the side information from the decoding stage to predict the coding modes and MVs of H.264/AVC BP coding. The proposed algorithm consists of two parts: INTRA macroblocks (MBs) transcoding and INTER MBs transcoding. By reusing the side information during the decoding stage, these techniques provide significant complexity reduction, when compared with the fully spatial domain approach.

Motion estimation for content adaptive video compression

A multistage motion estimation scheme is proposed. The scheme extracts video characteristics by first performing an online video analysis separately for foreground and background regions. Motion parameters are extracted and passed to the next stage. The next stage includes a mathematical model for the block distortion surface (BDS) that enables the algorithm to accordingly adjust its search technique. The search is performed on a precise search area adaptive to the statistical property of the motion vector prediction error. Due to its self-tuning property, not only does the proposed scheme adapt to scenes by yielding better visual quality but it also yields a lower computational complexity, compared with the other predictive motion estimation algorithms on standard benchmark sequences

Joint power and distortion control in video coding

For video coding in futuristic ubiquitous environments, how to efficiently manage the power consumption while preserving high video quality is crucial. To address the above challenge, we formulate a multiple objective optimization problem to model the behavior of power-distortion-optimized video coding. The objectives in this problem are incommensurate and in conflict with one another. By assessing the performance trade-offs as well as the collective impact of power and distortion, we propose a joint power-distortion control strategy (JPDC), in which the power and distortion are jointly considered. After the analysis on the approach of solving the problem statically, we utilize a sub-optimal “greedy” approach in the JPDC scheme. Each complexity parameter is adjusted individually. The system starts coding at the highest complexity level, and will automatically migrate to lower/higher level until the performance improvement gets saturated, leading to the optimal operation point. We perform simulations to demonstrate the effectiveness of the proposed scheme. Our results show that the proposed JPDC scheme is aware of the power constraint as well as the video content, and achieves significant power savings with well-perceived video quality. Such a feature is particularly desirable in futuristic video applications.

Fast motion estimation using hierarchical motion intensity structure

The embedded motion compensation model of the new H.264/AVC video coding standard dramatically increases the computational complexity of motion estimation. We propose a fast motion estimation algorithm using a hierarchical motion intensity structure to lower the computational complexity of the motion estimation in H.264/AVC. The proposed algorithm is mainly based on a multi-level motion intensity structure. It determines the motion intensity at three levels and, accordingly, uses different motion estimation techniques to find a more accurate, and faster, motion vector (MV). Experimental results show that the proposed algorithm provides promising performance in terms of the computational speedup and video reconstruction quality.

Adaptive Techniques for Simultaneous Optimization of Visual Quality and Battery Power in Video Encoding Sensors

This paper addresses the issues of how to efficiently manage the power consumption while preserving high quality performance for video coding in futuristic pervasive computing environments. We formulate an optimization problem that corresponds to minimize both the power consumption and video distortion. A joint rate-complexity-distortion (R-C-D) model is developed to describe and control the general R-C-D behavior of video coding. The model facilitates to develop effective schemes based on the analysis of the optimization problem in order to achieve good video quality while maximizing the battery service life. We perform simulations on an instruction set simulator to demonstrate the effectiveness of the proposed schemes.

Lowering the Complexity of Multi-view Encoding through Dynamic Segmentation and Registration of Video Object

Exploitation of redundancies for superior coding efficiency is one of the major research issues in multi-view video coding (MVC). However, this incurs extra computation, counteracting the benefit gained from coding efficiency. This paper exploits inter-view correlations among video objects and background to lower the prediction complexity while achieving high coding efficiency in MVC. We propose an effective object-based scheme that utilizes video object information obtained from the coded base view. This information assists to predict disparity vectors and motion vectors in enhancement views (EVs) by employing object registration and warping. This leads to high compression and low-complexity coding scheme for EVs. The proposed scheme provides higher coding efficiency when compared to the segmentation-free scheme, while achieving lower computational complexity.