Chang-Hong Fu

Fast Motion and Disparity Estimation With Adaptive Search Range Adjustment in Stereoscopic Video Coding

Stereoscopic video gives viewers more realistic vision than traditional 2D videos by transmitting two different views simultaneously. It doubles the required bandwidth in comparison with single view videos. Motion and disparity estimation therefore play a key role in reducing the bit rate of stereoscopic videos. However, it brings extremely huge computational complexity to an encoder which obstructs it from practical uses. In the past few years, some fast algorithms were proposed where most of them speed up the coding time based on an accurate estimation in one field (motion/disparity field), and then relieve the computational burden in the other field (disparity/motion field). Nevertheless, the complexities of both motion and disparity estimation cannot be fully reduced. In this paper, an iterative motion and disparity estimation algorithm is proposed. The proposed algorithm can determine motion vectors of the right view and disparity vectors of the current stereo pair in an iterative way. The gain of this iterative search is due to the use of a stereo-motion consistency constraint in which the motion and disparity vectors can be estimated by updating the local optimal vectors iteratively. An adaptive search range adjustment through the confidence measure of the constraint is then designed to further strengthen the reliability of each step for the iterative search. Experimental results show that the speed-up gain of the proposed algorithm is 18.76 ~ 229.19 times compared to the full search algorithm with a negligible quality drop.

Efficient reverse-play algorithms for MPEG video with VCR support

Reverse playback is the most common video cassette recording (VCR) function in digital video players and it involves playing video frames in reverse order. However, the predictive processing techniques employed in MPEG severely complicate the reverse-play operation. For displaying single frame during reverse playback, all frames from the previous I-frame to the requested frame must be sent by the server and decoded by the client machine. It requires much higher bandwidth of the network and complexity of the decoder. In this paper, we propose a compressed-domain approach for an efficient implementation of the MPEG video streaming system to provide reverse playback over a network with the minimal requirements on the network bandwidth and the decoder complexity. In the proposed video streaming server, it classifies macroblocks in the requested frame into two categories-backward macroblocks (BMBs) and forward macroblock (FMBs). Two novel MB-based techniques are used to manipulate the necessary MBs in the compressed domain and the server then sends the processed MBs to the client machine. For BMBs, we propose a sign inversion technique, which is operated in the variable length coding (VLC) domain, to reduce the number of MBs that need to be decoded by the decoder and the number of bits that need to be sent over the network in the reverse-play operation. The server also identifies the previous related MBs of FMBs and those related maroblocks coded without motion compensation are then processed by a technique of direction addition of discrete cosine transform (DCT) coefficients to further reduce the computational complexity of the client decoder. With the sign inversion and direct addition of DCT coefficients, the proposed architecture only manipulates MBs either on the VLC domain or DCT domain to achieve the server with low complexity. Experimental results show that, as compared to the conventional system, the new streaming system reduces the required network bandwidth and the decoder complexity significantly.

Iterative search strategy with selective bi-directional prediction for low complexity multiview video coding

The multiview video coding (MVC) extension of H.264/AVC is the emerging standard for compression of impressive 3D and free-viewpoint video. The coding structure in MVC adopts motion and disparity estimation to exploit temporal and inter-view dependencies in MVC. It results in a considerable increase in encoding complexity. Most of the computational burden comes from uni-directional and bi-directional prediction. In this paper, an iterative search strategy is designed to speed up the uni-directional prediction in MVC. It can work with an adaptive search range adjustment through a confidence measure of a loop constraint to obtain both motion and disparity vectors jointly. Furthermore, a selective bi-directional prediction algorithm is proposed to enhance the coding performance by analyzing the statistical characteristics of bi-directional prediction in MVC. Experimental results demonstrate that, by using the proposed fast search, the temporal and inter-view redundancies of multiview video can be eliminated sufficiently with low complexity.

Fast wedgelet pattern decision for DMM in 3D-HEVC

3D-HEVC is in the process of standardization as the highest profile extension of HEVC for 3D video coding, which supports the advanced 3D video format, namely multi-view videos plus depth (MVD). In order to preserve the sharp edges and avoid the ringing artifacts in the synthesized views, depth modelling modes (DMMs) are adopted into 3D-HEVC to achieve high coding efficiency. However, the computational complexity increases significantly with these modes. In this paper, an efficient algorithm of wedgelet pattern decision is proposed for one type of the DMMs: the explicit wedgelet signaling or equivalently DMM1. The proposed algorithm roughly estimates the position of sharp edges by pre-defined regions. Then the optimal wedgelet pattern is only searched in the most probable region. Simulation results show that our algorithm saves about 55% time of the wedgelet pattern selection with the BD-Rate increased by 0.49% in average.

Viewpoint switching in multiview videos using SP-frames

The distinguishing feature of multiview video lies in the interactivity, which allows users to select their favourite viewpoint. It switches bitstream at a particular view when necessary instead of transmitting all the views. The new SP-frame in H.264 is originally developed for multiple bit-rate streaming with the support of seamless switching. The SP-frame can also be directly employed in the viewpoint switching of multiview videos. Notwithstanding the guarantee of seamless switching using SP-frames, the cost is the bulky size of secondary SP-frames. This induces a significant amount of additional space or bandwidth for storage or transmission, especially for the multiview scenario. For this reason, a new motion estimation and compensation technique operating in the quantized transform (QDCT) domain is designed for coding secondary SP-frame in this paper. Our proposed work aims at keeping the secondary SP-frames as small as possible without affecting the size of primary SP-frames by incorporating QDCT-domain motion estimation and compensation in the secondary SP-frame coding. Simulation results show that the size of secondary SP-frames can be reduced remarkably in viewpoint switching.

Fast iterative motion and disparity estimation algorithm for multiview video coding

This paper proposes an iterative search strategy for joint motion and disparity estimation in order to eliminate the temporal and inter-view redundancies of multiview video. This iterative search strategy can work with an adaptive search range adjustment through a confidence measure of a loop constraint to obtain both motion and disparity vectors simultaneously with low complexity. Experimental results demonstrate that the proposed algorithm outperforms the conventional algorithm which estimates disparity and motion vectors separately.

New Architecture for MPEG Video Streaming System With Backward Playback Support

MPEG digital video is becoming ubiquitous for video storage and communications. It is often desirable to perform various video cassette recording (VCR) functions such as backward playback in MPEG videos. However, the predictive processing techniques employed in MPEG severely complicate the backward-play operation. A straightforward implementation of backward playback is to transmit and decode the whole group-of-picture (GOP), store all the decoded frames in the decoder buffer, and play the decoded frames in reverse order. This approach requires a significant buffer in the decoder, which depends on the GOP size, to store the decoded frames. This approach could not be possible in a severely constrained memory requirement. Another alternative is to decode the GOP up to the current frame to be displayed, and then go back to decode the GOP again up to the next frame to be displayed. This approach does not need the huge buffer, but requires much higher bandwidth of the network and complexity of the decoder. In this paper, we propose a macroblock-based algorithm for an efficient implementation of the MPEG video streaming system to provide backward playback over a network with the minimal requirements on the network bandwidth and the decoder complexity. The proposed algorithm classifies macroblocks in the requested frame into backward macroblocks (BMBs) and forward/backward macroblocks (FBMBs). Two macroblock-based techniques are used to manipulate different types of macroblocks in the compressed domain and the server then sends the processed macroblocks to the client machine. For BMBs, a VLC-domain technique is adopted to reduce the number of macroblocks that need to be decoded by the decoder and the number of bits that need to be sent over the network in the backward-play operation. We then propose a newly mixed VLC/DCT-domain technique to handle FBMBs in order to further reduce the computational complexity of the decoder. With these compressed-domain techniques, the proposed architecture only manipulates macroblocks either in the VLC domain or the quantized DCT domain resulting in low server complexity. Experimental results show that, as compared to the conventional system, the new streaming system reduces the required network bandwidth and the decoder complexity significantly.

Efficient depth intra mode decision by reference pixels classification in 3D-HEVC.

The uniform intra prediction increases the intra prediction modes up to 35 and brings better coding efficiency in HEVC. Besides, depth modelling modes (DMMs) are introduced in depth intra coding of 3D-HEVC to preserve sharp edges and avoid ringing artifacts in a synthesized view. Meanwhile, the encoding time of depth intra coding rapidly increases due to a huge number of intra mode candidates. Based on the spatial correlation of a depth map, we find that not all of the intra modes are necessary to be considered in most cases. Hence, a fast content-dependent depth intra mode decision algorithm is raised in this paper by classifying the spatial distribution of the reference pixels. Simulation results show that the proposed adaptive fast algorithm can save 21%-35% time of the depth coding with the insignificant bit rate increase compared with the state-of-the-art algorithm.

A High Performance Lossless Bayer Image Compression Scheme

Demosaicing and compression are generally performed sequentially in most digital cameras. Recent reports show that the compression-first scheme outperforms the conventional demosaicing-first scheme in terms of image quality and complexity. In this paper, an efficient lossless compression scheme for Bayer images is presented. It exploits a context matching technique to rank the neighboring pixels for predicting a pixel. Besides, an adaptive color difference estimation scheme is also proposed to remove the spectral redundancy. Simulation results show that the proposed algorithm can achieve a better compression performance as compared with the existing lossless CFA image coding methods.

Adaptive fast intra mode decision of depth map coding by Low Complexity RD-Cost in 3D-HEVC

The coding efficiency of 3D-HEVC can be improved significantly by adopting more intra modes in depth map coding, including up to 35 conventional HEVC intra prediction modes and 4 depth modelling modes (DMMs). However, traversing these modes dramatically results in unendurable high complexity. In this paper, we propose an efficient intra mode decision algorithm by taking advantage of the Low Complexity Rate-Distortion Cost information in the rough mode decision. Only one conventional intra mode and several most probable modes are then selected for most cases adaptively based on the threshold derived from the prior probability. Experimental results show that the proposed algorithm provides about 34% saving in terms of the total depth coding time with little drop of the coding performance compared with the state-of-the-art algorithm.