Jong-Ki Han

An efficient scheme for motion estimation using multireference frames in H.264/AVC

The multiple reference frame motion compensation (MRMC) supported by H.264 makes use of the redundancy which is between multiple frames to enhance the coding efficiency over a scheme using the single reference frame motion compensation (SRMC) in which motion vectors are searched over a single reference frame. And, the technique using multiple reference frames can combat the channel errors efficiently. However, searching the motion vectors in multiple frames may require a huge computing time. This paper proposes a novel motion estimation procedure, which has a lower search complexity without sacrificing image quality. To reduce the complexity of motion estimation procedure, we use a temporary motion vector generated with little computation. The temporary motion vector is calculated from the motion vector map composed of motion vectors between successive frames, and used to predict the optimal motion vector for a reference frame. The proposed scheme requires the lower complexity than conventional schemes by using the temporary motion vector and refinement process over a narrow search range around the temporary predictive motion vector. Since the temporary predictive motion vector effectively chases the optimal motion vector for each reference frame, the encoded image quality by proposed scheme is very similar to that of full search algorithm. The proposed motion estimation process consists of three phases: 1)making a vector map between two consecutive frames, where the vector map is constructed by copying motion vectors which have been estimated in first reference frame, 2) composing a temporary motion vector with element vectors which are in the vector map, and 3) finally, the temporary predictive motion vector is refined over a narrow search range. We show experimental results which demonstrate the effectiveness of the proposed method. To compare the proposed motion estimation algorithm with the conventional schemes, we check the CPU times consumed by ME module in H.264 encoder using the proposed scheme. In the results, CPU time consumed by the proposed scheme has been reduced significantly without additional distortion of the encoded video quality.

Efficient Bit Allocation and Rate Control Algorithms for Hierarchical Video Coding

Hierarchical structure is a useful tool for providing the necessary scalability in adapting to the variety of channel environments. For schemes involving hierarchical picture structures, bit allocation, and rate control algorithms are vital components for improving video codec performance. Since conventional bit allocation schemes do not efficiently consider the hierarchical structure characteristics, it is difficult to optimize the video quality at an arbitrary bitrate. Similarly, conventional quantization parameter decision methods are not appropriate for controlling the bitrate generated by a codec using a hierarchical encoding structure. In this paper, we propose an effective bit allocation scheme that assigns the target number of bits to pictures or macroblocks (MBs) and improves the overall quality of images encoded by a hierarchical-based encoder. A rate control scheme is also proposed to ensure that the generated bitrate is equal to the assigned target bitrate. From the simulation results, the proposed schemes outperformed conventional methods from a rate-distortion perspective, by efficiently controlling the bitrate of the MB unit. The algorithms regulated the generated bits to achieve the target bits by using the proposed linear R-Q model.

Efficient motion estimation algorithm for MPEG-4 to H.264 transcoder

In this paper, we propose a fast ME (motion estimation) algorithm for MPEG-4 to H.264 transcoder. The transcoding speed is affected dominantly by the computational complexity of encoder part in transcoder, where ME module of H.264 encoder has high complexity due to considering 7 modes. In order to increase the speed of transcoding between MPEG-4 and H.264, we use 3 PMVs (predicted motion vectors) and the mode information of MBs (macroblocks) provided from the decoder part of transcoder. Since the proposed 3 PMVs are very close to an optimal motion vector, and we consider only some restricted modes according to the MB information transferred from decoder part, the proposed scheme can speed up the transcoding procedure without loss of image quality. We show simulation results that CPU time consumed by the proposed scheme has been reduced significantly while the transcoded video quality remains unchanged.

Rate Control for Consistent Objective Quality in High Efficiency Video Coding

Since video quality fluctuation degrades the visual perception significantly in multimedia communication systems, it is important to maintain a consistent objective quality over the entire video sequence. We propose a rate control algorithm to keep the consistent objective quality in high efficiency video coding (HEVC), which is an upcoming standard video codec. In the proposed algorithm, the probability density function of transformed coefficients is modeled based on a Laplacian function that considers the quadtree coding unit structure, which is one of the characteristics of HEVC. In controlling the video quality, distortion-quantization and rate-quantization models are derived by using the Laplacian function. Based on those models, a quantization parameter is determined to control the quality of the encoded frames where the fluctuation of video quality is minimized and the overflow and underflow of buffer are prevented. From the simulation results, it is shown that the proposed rate control algorithm outperforms the other conventional schemes.

Optimization of requantization codebook for vector quantization

Requantization is a key technology for reducing the bit rate of a previously compressed data. When recompression ratio is high, the requantizer may cause unacceptable quality degradation. To improve the quality of the requantized image, an optimization scheme for the requantization codebook has been proposed. The proposed scheme constructs an optimal requantization codebook in an iterative manner for a given original quantization codebook of transmitter. The construction of codebook is iteratively repeated until they reach a local optimum solution. Our approach can be applied not only to the scalar quantization, but to any method which employs vector quantization-based system. Simulation results show that the optimized system based on the proposed algorithm outperforms the conventional system which is made without consideration of requantization. The proposed algorithm enables a reliable image communication over heterogeneous networks.

Enhanced Adaptive Loop Filter for Motion Compensated Frame

We propose an adaptive loop filter to remove the redundancy between current and motion compensated frames so that the residual signal is minimized, thus coding efficiency increases. The loop filter coefficients and offset are optimized for each frame or a set of blocks to minimize the total energy of the residual signal resulting from motion estimation and compensation. The optimized loop filter with offset is applied for the set of blocks where the filtering process gives coding gain based upon rate-distortion cost. The proposed loop filter is used for the motion compensated frame whereas the conventional adaptive interpolation filter (AIF) is applied to the reference frames to interpolate the subpixel values. Another conventional scheme adaptive loop filter (ALF), is used after deblocking filtering to enhance quality of reconstructed frames, not to minimize energy of residual signal. The proposed loop filter can be used in combination with the AIF and ALF. Experimental results show that proposed algorithm provides the averaged bit reduction of 8% compared to conventional H.264/AVC scheme. When the proposed scheme is combined with AIF and ALF, the coding gain increases even further.

Modified cubic convolution scaler for edge-directed nonuniform data

We derive a modified version of the cubic convolution scaler to enlarge or reduce the size of digital images with arbitrary ratio. To enhance the edge information of the scaled image and to obtain a high-quality scaled image, the proposed scaler is applied along the direction of an edge. Since interpolation along the direction of an edge has to process nonuniformly sampled data, the kernel of the cubic convolution scalar is modified to interpolate the data. The proposed scaling scheme can be used to resize pictures in various formats in a transcoding system that transforms a bit stream compressed at one bit rate into one compressed at another bit rate. In many applications, such as transcoders, the resolution conversion is very important for changing the image size while maintaining high quality of the scaled image. We show experimental results that demonstrate the effectiveness of the proposed interpolation method. The proposed scheme provides clearer edges, without artifacts, in the resized image than do conventional schemes. The algorithm exhibits significant improvement in the minimization of information loss when compared with the conventional interpolation algorithms.

Rate Control Scheme for Consistent Video Quality in Scalable Video Codec

Multimedia data delivered to mobile devices over wireless channels or the Internet are complicated by bandwidth fluctuation and the variety of mobile devices. Scalable video coding has been developed as an extension of H.264/AVC to solve this problem. Since scalable video codec provides various scalabilities to adapt the bitstream for the channel conditions and terminal types, scalable codec is one of the useful codecs for wired or wireless multimedia communication systems, such as IPTV and streaming services. In such scalable multimedia communication systems, video quality fluctuation degrades the visual perception significantly. It is important to efficiently use the target bits in order to maintain a consistent video quality or achieve a small distortion variation throughout the whole video sequence. The scheme proposed in this paper provides a useful function to control video quality in applications supporting scalability, whereas conventional schemes have been proposed to control video quality in the H.264 and MPEG-4 systems. The proposed algorithm decides the quantization parameter of the enhancement layer to maintain a consistent video quality throughout the entire sequence. The video quality of the enhancement layer is controlled based on a closed-form formula which utilizes the residual data and quantization error of the base layer. The simulation results show that the proposed algorithm controls the frame quality of the enhancement layer in a simple operation, where the parameter decision algorithm is applied to each frame.

Joint optimization of the motion estimation module and the up/down scaler in transcoders

A joint design scheme is proposed to optimize the up/down scaler and the motion vector estimation module in the transcoder system. The proposed scheme first optimizes the resolution scaler for a fixed motion vector, and then a new motion vector is estimated for the fixed scaler. These two steps are iteratively repeated until they reach a local optimum solution. In the optimization of the scaler, we derive an adaptive version of a cubic convolution interpolator to enlarge or reduce digital images by arbitrary scaling factors. The adaptation is performed at each macroblock of an image. In order to estimate an optimal motion vector, a temporary motion vector is composed from the given motion vectors. Then the motion vector is refined over a narrow search range. It is well-known that this refinement scheme provides the comparable performance compared to the full search method. Simulation results show that a jointly optimized system based on the proposed algorithms outperforms the conventional systems. We can also see that the algorithms exhibit significant improvement in the minimization of information loss compared with other techniques.

An adaptive quantization using modified QP in H.264

An adaptive quantization scheme for the H.264 encoder is proposed In this work. We utilize the information obtained from the previously encoded image. The reconstructed error of each macroblock (MB) can be estimated from the previous frame. For the estimation of the error of the current MB, a block with the size of MB in the previous frame is chosen. Since the original and reconstructed images of the previous frame are available in the encoder, we can evaluate the tendency of reconstruction error of the current MB in advance. Then, this error is considered as the expected error of the current MB. Comparing the error of the MB with the average of overall MB, if it is larger than the average, a small step size is given for this MB, and vice versa. Experimental results show that the rate control algorithm outperforms the conventional scheme. Furthermore, the proposed algorithm is so simple that is only introduces a little computational complexity. Therefore, it can be used in real time video coding.