Hasan F. Ates

Rate-Distortion and Complexity Optimized Motion Estimation for H.264 Video Coding

H.264 video coding standard supports several inter- prediction coding modes that use macroblock (MB) partitions with variable block sizes. Rate-distortion (R-D) optimal selection of both the motion vectors (MVs) and the coding mode of each MB is essential for an H.264 encoder to achieve superior coding efficiency. Unfortunately, searching for optimal MVs of each possible subblock incurs a heavy computational cost. In this paper, in order to reduce the computational burden of integer-pel motion estimation (ME) without sacrificing from the coding performance, we propose a R-D and complexity joint optimization framework. Within this framework, we develop a simple method that determines for each MB which partitions are likely to be optimal. MV search is carried out for only the selected partitions, thus reducing the complexity of the ME step. The mode selection criteria is based on a measure of spatiotemporal activity within the MB. The procedure minimizes the coding loss at a given level of computational complexity either for the full video sequence or for each single frame. For the latter case, the algorithm provides a tight upper bound on the worst case complexity/execution time of the ME module. Simulation results show that the algorithm speeds up integer-pel ME by a factor of up to 40 with less than 0.2 dB loss in coding efficiency.

SAD reuse in hierarchical motion estimation for the H.264 encoder

The adoption of multiple macroblock partitions with variable block sizes is one of the main reasons behind the superior coding efficiency of H.264 video coding standard. Unfortunately, in the motion estimation phase, repeating sum of absolute difference (SAD) calculations for every possible block size incurs a heavy computational cost for the encoder. In this paper, in order to reduce the encoder complexity, we propose a hierarchical block matching based motion estimation algorithm that uses a common set of SAD computations for motion estimation of different block sizes. Based on the hierarchical prediction and the median motion vector predictor of H.264, the algorithm defines a limited set of candidate vectors; and the optimal motion vectors for all partitions are chosen from this common set. Simulation results show that hierarchical estimation with SAD reuse reduces the total computations by a factor of 17.6 with slight loss in coding efficiency.

A high performance hardware architecture for an SAD reuse based hierarchical motion estimation algorithm for H.264 video coding

In this paper, we present a high performance and low cost hardware architecture for real-time implementation of an SAD reuse based hierarchical motion estimation algorithm for H.264/MPEG4 Part 10 video coding. This hardware is designed to be used as part of a complete H.264 video coding system for portable applications. The proposed architecture is implemented in Verilog HDL. The Verilog RTL code is verified to work at 68 MHz in a Xilinx Virtex II FPGA. The FPGA implementation can process 27 VGA frames (640 /spl times/ 480) or 82 CIF frames (352 /spl times/ 288) per second.

Spherical Coding Algorithm for Wavelet Image Compression

In recent literature, there exist many high-performance wavelet coders that use different spatially adaptive coding techniques in order to exploit the spatial energy compaction property of the wavelet transform. Two crucial issues in adaptive methods are the level of flexibility and the coding efficiency achieved while modeling different image regions and allocating bitrate within the wavelet subbands. In this paper, we introduce the ldquospherical coder,rdquo which provides a new adaptive framework for handling these issues in a simple and effective manner. The coder uses local energy as a direct measure to differentiate between parts of the wavelet subband and to decide how to allocate the available bitrate. As local energy becomes available at finer resolutions, i.e., in smaller size windows, the coder automatically updates its decisions about how to spend the bitrate. We use a hierarchical set of variables to specify and code the local energy up to the highest resolution, i.e., the energy of individual wavelet coefficients. The overall scheme is nonredundant, meaning that the subband information is conveyed using this equivalent set of variables without the need for any side parameters. Despite its simplicity, the algorithm produces PSNR results that are competitive with the state-of-art coders in literature.

3-D Mesh Geometry Compression With Set Partitioning in the Spectral Domain

This paper explains the development of a highly efficient progressive 3-D mesh geometry coder based on the region adaptive transform in the spectral mesh compression method. A hierarchical set partitioning technique, originally proposed for the efficient compression of wavelet transform coefficients in high-performance wavelet-based image coding methods, is proposed for the efficient compression of the coefficients of this transform. Experiments confirm that the proposed coder employing such a region adaptive transform has a high compression performance rarely achieved by other state of the art 3-D mesh geometry compression algorithms. A new, high-performance fixed spectral basis method is also proposed for reducing the computational complexity of the transform. Many-to-one mappings are employed to relate the coded irregular mesh region to a regular mesh whose basis is used. To prevent loss of compression performance due to the low-pass nature of such mappings, transitions are made from transform-based coding to spatial coding on a per region basis at high coding rates. Experimental results show the performance advantage of the newly proposed fixed spectral basis method over the original fixed spectral basis method in the literature that employs one-to-one mappings.

Rate-Distortion and Complexity Joint Optimization for Fast Motion Estimation In H.264 Video Coding

H.264 video coding standard offers several coding modes including inter-prediction modes that use macroblock partitions with variable block sizes, Choosing a rate-distortion optimal mode among these possibilities contributes significantly to the superior coding efficiency of the H.264 encoder. Unfortunately, searching for optimal motion vectors of each possible subblock incurs a heavy computational cost, In this paper, in order to reduce the complexity of integer-pel motion estimation, we propose a rate-distortion and complexity-joint optimization method that selects for each MB a subset of partitions to evaluate during motion estimation. This selection is based on simple, measures of spatio-temporal activity within the MB. The procedure is optimized to minimize mode estimation error at a certain level of computational complexity. Simulation results show that the algorithm speeds up the motion estimation module by a factor of up to 20 with little loss in coding efficiency.

A nonlinear image representation in wavelet domain using complex signals with single quadrant spectrum

High frequency structures, such as edges, texture, contain significant portion of the information available in images. For coding purposes, the image representation has to provide direct access to this information content without introducing any redundancy. In this paper, we introduce such a nonredundant representation based on complex signals in wavelet domain, whose spectrum has only one nonzero quadrant. The corresponding magnitude and phase responses are linked with the most significant edge properties, namely energy, location and orientation. Simulations show that phase responses provide unbiased estimates of the edge location and orientation. We also discuss our coding strategies for this new representation.

Image interpolation using wavelet-based contour estimation

Successful image interpolation requires proper enhancement of high frequency content of image pixels around edges. In this paper, we introduce a simple edge model to estimate high resolution edge profiles from lower resolution values. Pixels around edges are viewed as samples taken from one dimensional (1-D) continuous edge profiles according to 1-D smooth edge contours defining the sampling instants. The image is highpass filtered by wavelets and subpixel edge locations are estimated by minimizing the modeling error in the wavelet domain. Interpolation is carried out by applying the model, wherever applicable, together with a baseline interpolator (here, bilinear) in order to make edges look sharper without introducing artifacts. The results are compared to bilinear interpolation, and significant improvement in terms of SNR, edge sharpness and contour smoothness is observed.

Decoder side true motion estimation for very low bitrate B-frame coding

In H.264 standard, coding of motion vectors constitutes a significant portion of total bitrate especially at low bitrate regimes. This is because differential coding of motion vectors is inefficient when the bit budget is very low. In this paper, we propose a novel estimation and coding algorithm for motion vectors of B-frames at very low bitrates. In this method, the encoder selects the optimal motion vector from a limited set of candidate vectors that are determined at the decoder side using true motion estimation. Since these candidate vector sets are fixed by the decoder for each macroblock, there is no need for explicit coding of motion information, which reduces the bitrate required for coding. Also, true motion vector estimates are used for improved direct mode coding in B-frames. The algorithm provides an average of 0.68 dB PSNR gain for B-frames when compared to the reference H.264 results at the same bitrates. Simulation results also indicate significant improvement in visual quality of the compressed B-frames.

Occlusion Aware Motion Compensation for Video Frame Rate Up-Conversion

Since the emergence of high definition (HD) display technologies, video standards conversion problem has become an important issue in storage, transmission and display of video content. Video frame rate up-conversion (FRUC) is considered as a standard task for today’s HD displays because these displays reach high refresh rates of at least 100/120 Hz and low video frame rates should be pulled up by a factor of 2 or more before display. Motion compensated FRUC techniques are proposed to avoid motion blur and motion judder at high refresh rates, but these techniques suffer from spatial inconsistencies and artifacts especially in occluded regions of the interpolated frames. This paper introduces a new video FRUC method that aims to remove both motion judder and occlusion artifacts and generate smooth object motion for high quality displays. An occlusion adaptive overlapped block motion compensation (OBMC) technique is proposed, which provides spatio-temporally consistent frame interpolation. Covered/uncovered regions are detected by analyzing the discontinuities of the motion vector field. The occlusion regions are interpolated using this covered/uncovered decision and a new error metric that measures spatial consistency. Compared to existing methods, the proposed algorithm achieves FRUC with fewer artifacts and better spatial resolution especially in occluded areas.