Tsz-Kwan Lee

Adaptive Search Range for HEVC Motion Estimation Based on Depth Information

High Efficiency Video Coding achieves twofold coding efficiency improvement compared with its predecessor H.264/MPEG-4 Advanced Video Coding. However, it suffers from high computational complexity due to its quad-tree structure in motion estimation (ME). This paper exposes the use of depth maps in the multiview video plus depth format for relieving the computational burden. The depth map provides an intimation of the objects’ distance from the projected screen in a 3D scene, which is explored in adaptive search range determination in this paper. The proposed algorithm exploits the high temporal correlation between the depth map and the motion in texture. By utilizing this correlation, a depth/motion relationship map is built for a mapping process. For each block, this forms a tailor-made search range with a motion-aware asymmetric shape to skip unnecessary search points in ME. The obtained search range can be further adjusted by taking the influence of 3D-to-2D projection into consideration. Simulation results reveal that, compared to the full search approach, the proposed algorithm can reduce the complexity by 93% on average, whereas the coding efficiency can be maintained. Besides, the proposed search range determination can work well with other fast search ME algorithms in the literature.

Depth-based adaptive search range algorithm for motion estimation in HEVC

High efficient video coding has been developed for ultra-high resolution and multi-view videos. It doubles the compression ratio compared to H.264/MPEG-4 AVC, but requires to adopt a very high computational quad-tree structure in motion estimation. Researchers mainly focus on fast mode decision on coding units or prediction units, and have reused the fast motion estimation techniques of H.264/MPEG-4 AVC. However, they do not fully utilize the characteristics of 3D video which is composed of texture streams and depth maps. The depth maps give cues to the objects in the same distance from the projected screen in a 3D scene. In addition to the high temporal correlation between frames, depth maps could be used to link up the objects in consecutive frames such that movements of the same object could be predicted. Therefore, the proposed algorithm aims to define an adaptive search range in motion estimation according to the predicted movements by depth intensity mapping in order to skip the unnecessary search points. Simulation results reveal that the proposed algorithm can reduce the complexity of motion estimation while the coding efficiency can be maintained.

Motion estimation in low-delay hierarchical p-frame coding using motion vector composition

Low-delay hierarchical prediction structure is currently adopted in various new video coding standards. The only hurdle of this structure is the need of motion estimation in distant reference frames. To maintain high coding efficiency, a large search range for motion estimation can improve the coding efficiency in distant reference pictures. Computational complexity will thus be increased dramatically. In this paper, a fast motion estimation scheme for a low-delay hierarchical P-frame structure is proposed. The proposed scheme adopts a motion vector composition strategy to expedite the motion estimation process for distant reference frames in the hierarchical P structure. In addition, a motion vector composition algorithm is tailor-made with the proposed hierarchical P coding scheme to further improve the coding efficiency. Simulation results show that the proposed scheme can deliver a remarkable complexity savings and coding efficiency improvement on coding a frame in low temporal layers of the hierarchical P structure.

A new motion vector composition algorithm for fast-forward video playback in H.264

With the rapid growth of streaming digital videos, it is desirable to access video segments of interest by searching through the video contents with a faster speed than a normal playback. Fast-forward playback is the key function that enables quick browsing of videos. It can be realized by a frame-skipping transcoder which transcodes only the frames required for playback at the desired fast speed. Various motion vector (MV) composition algorithms aim at reducing the computational complexity of the transcoder. They only perform fairly in limited skipped frames scenarios. In this paper, a new vector selection algorithm is proposed to compose a new motion vector (MV) from a set of candidate MVs for minimizing prediction errors due to a larger frame-skipping factor. Experimental results show that the proposed algorithm can deliver a remarkable improvement on the rate-distortion performance over other algorithms.

An efficient motion vector composition algorithm for fast-forward playback in a video streaming system

Fast-forward playback enables viewers to scan through the video scene of interest efficiently. One approach to realize fast-forward playback is to employ a frame-skipping transcoder which transcodes only the frames required for playback at the desired fast speed. Various motion vector composition algorithms are used to compose the new motion vectors with reduced complexity. These algorithms do not work well for dropping a large number of frames, which is very common in fast-forward playback. In this paper, a new multiple-candidate vector selection algorithm (MCVS) is proposed to select a composed motion vector from a set of candidate motion vectors, which utilizes relevant areas in the target macroblock to ensure a reliable tracking process for motion vector composition. Experimental results show that the proposed MCVS can provide fast-forward playback through video transcoding with significant gain, in terms of rate-distortion performance, especially when a large speed-up factor is required.

Reliable tracking algorithm for multiple reference frame motion estimation

Multiple reference frame motion estimation (MRF-ME) is one of the most crucial tools in H.264/AVC to improve coding efficiency. However, it disciplines an encoder by giving extra computational complexity. The required computation proportionally expands when the number of reference frames used for motion estimation increases. Aiming to reduce the computational complexity of the encoder, various motion vector (MV) composition algorithms for MRF-ME have been proposed. However, these algorithms only perform well in a limited range of reference frames. The performance deteriorates when motion vector composition is processed from the current frame to a distant reference frame. In this paper, a reliable tracking mechanism for MV composition is proposed by utilizing only the relevant areas in the target macroblock and taking different paths through a novel selection process from a set of candidate motion vectors. The proposed algorithm is especially suited for temporally remote reference frames in MRF-ME. Experimental results show that compared with the existing MV composition algorithms, the proposed one can deliver a remarkable improvement on the rate-distortion performance with similar computational complexity.

Adaptive search range by depth variant decaying weights for HEVC inter texture coding

Emerging high-efficiency video coding (HEVC) outperforms H.264 by a gain of 50% bitrate reduction while maintaining almost the same perceptual quality. However, it induces higher coding complexity due to its adoption of recursive block partitioning mechanism in motion estimation (ME) with a fixed search range. For an objective of reducing the computational burden in HEVC, this paper proposes an adaptive search range algorithm by using depth map information. With the aid of depth intensity variations among neighboring blocks, associated weights to the neighboring blocks are derived. The proposed weighted sum of the motions from the neighboring blocks is formulated to provide a suitable search range for each block. The simulation results demonstrated that proposed adaptive search range is compatible to not only full-search (FS) but also fast Test Zone Search (TZS) in HEVC. The proposed algorithm could reduce significant coding time on average with negligible rate-distortion degradation.

Region-based weighted prediction algorithm for H.264/AVC video coding

This paper proposes a novel region-based weighted prediction (WP) algorithm to encode scenes with complex brightness variations. It facilitates the use of multiple WP parameter sets in a single reference frame by utilizing the framework of multiple reference frame motion estimation (MRF-ME). With this arrangement, different macroblocks in the current frame can use different WP parameter sets even when they are predicted from the same reference frame. To support this, a region partitioning process is designed to divide the current frame into different regions where each one has some degree of uniformity in its brightness variation. Multiple sets of region-based WP parameters can then be estimated accurately. Consequently, the proposed algorithm can improve prediction in scenes with different degrees of brightness variations in different regions of the same picture. Results show that the region-based algorithm can achieve significant coding gains of scenes with complex brightness variations.

Motion vector composition in low-delay hierarchical P-frame coding

Recent video coding standards such as high efficiency video coding (HEVC) and multi-view video coding (MVC) can obtain significant gain from hierarchical prediction structures. Hierarchical B structure utilizes future reference frames, which induces high encoding delay. Comparatively, hierarchical P structure brings low-delay coding in which no future references are involved. The only hurdle of the low-delay hierarchical P structure is the need of motion estimation in distant reference frames. To maintain a high coding efficiency, large search range for motion estimation was commonly adopted in distant reference pictures. Therefore, computational complexity will be increased dramatically. In this paper, motion vector composition is proposed in hierarchical P structure to shorten coding process. Simulation results reveal that the proposed scheme can achieve significant complexity savings.