Jiancong Luo

A fast adaptive motion estimation algorithm

Motion estimation (ME) is a multistep process that involves not one, but a combination of techniques, such as motion starting point, motion search patterns, and adaptive control to curb the search, avoidance of search stationary regions, etc. The collective efficiency of these techniques is what makes a ME algorithm robust and efficient across the board. This paper proposes a ME algorithm that is an embodiment of several effective ideas for finding the most accurate motion vectors (MVs) with the aim to maximize the encoding speed as well as the visual quality. The proposed algorithm takes advantage of the correlation between MVs in both spatial and temporal domains, controls to curb the search, avoids of search stationary regions, and uses switchable shape search patterns to accelerate motion search. The algorithm yields very similar quality compared to the full search but with several hundred times faster speed. We have evaluated the algorithm through a comprehensive performance study that shows that the proposed algorithm achieves substantial speedup without quality loss for a wide range of video sequences, compared with the ME techniques recommended by the MPEG-4 committee.

Motion Estimation for Content Adaptive Video Compression

Motion estimation is a multistep process that encompasses techniques such as motion vector prediction, determination of search range and search patterns, and identification of termination criteria. Each of these techniques has several diversions that may suit a particular set of video characteristics. It would be hard to conceive a universal algorithm that can perform well for all kinds of video contents. However, if important characteristics of a video sequence can be identified and utilized for adjusting various steps of motion estimation, one can design an adjustable algorithm that can tune its parameters to suit the video at hand. A multistage motion estimation algorithm that includes a pre-stage for analyzing the motion characteristics of a video sequence online is proposed. This stage predicts the motion vector field (MVF) from the previous coded frame, and clusters macroblocks into background and foreground regions based on the predicted MVF. The information from the pre-stage are passed on to the next stage, which includes a mathematical model for block distortion surface to estimate the distance from the current search point to the global optimal position. This allows the motion estimation stage to adjust its search strategies accordingly. The search is performed on a precise area according to the statistical properties of the motion vector prediction error, separately for foreground and background regions. The proposed algorithm, including the pre-stage, is fast and, therefore, is suitable for online and real-time encoding. Extensive simulation results obtained for several video sequences affirm the effectiveness of the proposed algorithm. The self-tuning property enables the algorithm to perform well for several types of benchmark sequences, yielding better video quality without exceeding the time complexity as compared to the other predictive motion estimation algorithms.

Motion estimation for content adaptive video compression

A multistage motion estimation scheme is proposed. The scheme extracts video characteristics by first performing an online video analysis separately for foreground and background regions. Motion parameters are extracted and passed to the next stage. The next stage includes a mathematical model for the block distortion surface (BDS) that enables the algorithm to accordingly adjust its search technique. The search is performed on a precise search area adaptive to the statistical property of the motion vector prediction error. Due to its self-tuning property, not only does the proposed scheme adapt to scenes by yielding better visual quality but it also yields a lower computational complexity, compared with the other predictive motion estimation algorithms on standard benchmark sequences

Joint power and distortion control in video coding

For video coding in futuristic ubiquitous environments, how to efficiently manage the power consumption while preserving high video quality is crucial. To address the above challenge, we formulate a multiple objective optimization problem to model the behavior of power-distortion-optimized video coding. The objectives in this problem are incommensurate and in conflict with one another. By assessing the performance trade-offs as well as the collective impact of power and distortion, we propose a joint power-distortion control strategy (JPDC), in which the power and distortion are jointly considered. After the analysis on the approach of solving the problem statically, we utilize a sub-optimal “greedy” approach in the JPDC scheme. Each complexity parameter is adjusted individually. The system starts coding at the highest complexity level, and will automatically migrate to lower/higher level until the performance improvement gets saturated, leading to the optimal operation point. We perform simulations to demonstrate the effectiveness of the proposed scheme. Our results show that the proposed JPDC scheme is aware of the power constraint as well as the video content, and achieves significant power savings with well-perceived video quality. Such a feature is particularly desirable in futuristic video applications.

A rate control algorithm for wireless video transmission using perceptual tuning

In video transmission over two-way wireless channel automatic repeat request (ARQ) retransmission has been used to protect packets from losing, which also causes the channel throughput varying and brings more challenges to rate control of the video compression. This paper proposes a new rate control algorithm based on regions of interest (ROI), PID buffer controlling and perceptual tuning techniques, etc. It intelligently allocates bit budget among different regions and adapts to time-varying wireless channel so as to achieve better perceptual quality in ROI and depress the frame skipping. Simulation results illustrate that our algorithm effectively enhances the perceptual quality for ROI, significantly reduces the number of frame skipping, and thereby improves the smoothness of the video.

Fast motion estimation using hierarchical motion intensity structure

The embedded motion compensation model of the new H.264/AVC video coding standard dramatically increases the computational complexity of motion estimation. We propose a fast motion estimation algorithm using a hierarchical motion intensity structure to lower the computational complexity of the motion estimation in H.264/AVC. The proposed algorithm is mainly based on a multi-level motion intensity structure. It determines the motion intensity at three levels and, accordingly, uses different motion estimation techniques to find a more accurate, and faster, motion vector (MV). Experimental results show that the proposed algorithm provides promising performance in terms of the computational speedup and video reconstruction quality.

Controlling the Bit Rate of Multi-Object Videos With Noncooperative Game Theory

This paper proposes an object-level rate control algorithm to jointly controlling the bit rates of multiple video objects. Utilizing noncooperative game theory, the proposed rate control algorithm mimics the behaviors of players representing video objects. Each player competes for available bits to optimize its visual quality. The algorithm finds an ¿optimal solution¿ in that it conforms to the mixed strategy Nash equilibrium, which is the probability distribution of the actions carried by the players that maximizes their expected payoffs (the number of bits). The game is played iteratively, and the expected payoff of each play is accumulated. The game terminates when all of the available bits for the specific time instant have been distributed to video object planes (VOPs). The advantage of the proposed scheme is that the bidding objects divide the bits among themselves automatically and fairly, according to their encoding complexity, and with an overall solution that is strategically optimal under the given circumstances. To minimize buffer fluctuation and avoid buffer overflow and underflow, a proportional-integral-derivative (PID) control based buffer policy is utilized.

Synchronous and asynchronous multiple object rate control for MPEG-4 video coding

Video scenes containing multiple objects can potentially achieve higher degree of compression and better visual quality with individual coding for each object. Video objects are not always synchronous, implying each object may have a separate temporal resolution. This paper proposes a rate control algorithm for multiple video object encoding. Using a novel bit allocation strategy, the algorithm achieves accurate target bit rate, provides good visual quality, and decreases buffer overflow/underflow. Experimental results for both synchronous and asynchronous multiple video object encoding demonstrate that, when compared with the existing rate control scheme recommended by the MPEG-4 standard, the proposed algorithm provide better temporal-spatial tradeoff with more accurate rate regulation.

A multistage fast motion estimation scheme for video compression

This paper presents a novel multistage motion estimation (ME) scheme called content adaptive search technique (CAST). The proposed scheme consists of four stages: motion vector field (MVF) prediction, block-based segmentation, motion parameter extraction, and adaptive search strategy. Through pre-processing the MVF of the previous reference frame in the first three stages, CAST extracts the motion parameters for each region. The 4th stage is a combination of various techniques including MV prediction, search area decision and an adaptive fast search algorithm that is adjusted by a mathematical model for the block distortion surface (BDS). Experiment shows that the proposed scheme improves the visual quality, while yielding a faster speed, comparing with the other predictive ME algorithms.

An adaptive cross search algorithm for block matching motion estimation

Motion estimation plays an important role in the motion compensated video coding framework. Due to the high computational complexity of the exhaustive search, many sub-optimal fast search algorithms, aiming to achieve the best trade off between distortion and search speed, are proposed. We observe that the distortion gradient of a search point on the block distortion surface (BDS) monotonously decreases with increasing distance from that point to the global minimum point. Based on this property, we propose a novel adaptive cross search (ACS) algorithm that can distribute the computation powers over the search space efficiently. Simulation shows that ACS achieves competitive reconstruction visual quality as well as reduced computational complexity.