Chae Eun Rhee

A survey of fast mode decision algorithms for inter-prediction and their applications to high efficiency video coding

The emerging High Efficiency Video Coding (HEVC) standard attempts to improve the coding efficiency by a factor of two over H.264/AVC using new compression tools with high computational complexity. The increased computational complexity makes the real-time execution with reasonable computing power become one of the critical concerns for the commercialization of HEVC. A large number of prediction modes are the main causes of the increased complexity of HEVC. Thus, a fast decision of a prediction mode needs to be effectively used to reduce the computational complexity. To take advantage of large amounts of previous works and to find a guide for application to HEVC, this paper presents a survey of these efforts for the previous standards, especially for H.264/AVC, and examines the possibility of the previous algorithms to be applicable for HEVC. To this end, previous algorithms are categorized and then the effectiveness of each category for HEVC is evaluated. For this evaluation, a previous algorithm is modified for HEVC when it is not applicable to HEVC directly. Simulation results show that most previous algorithms with slight modification, in general, improve the encoding speed with a relatively small degradation of the compression efficiency. Among them, hierarchical mode decision is especially effective whereas mode pre-decision using motion or spatial homogeneity often results in inaccurate results.

A Real-Time H.264/AVC Encoder With Complexity-Aware Time Allocation

This paper presents a novel processing time control algorithm for a hardware-based H.264/AVC encoder. The encoder employs three complexity scaling methods partial cost evaluation for fractional motion estimation (FME), block size adjustment for FME, and search range adjustment for integer motion estimation (IME). With these methods, 12 complexity levels are defined to support tradeoffs between the processing time and compression efficiency. A speed control algorithm is proposed to select the complexity level that compresses most efficiently among those that meet the target time budget. The time budget is allocated to each macroblock based on the complexity of the macroblock and on the execution time of other macroblocks in the frame. For main profile compression, an additional complexity scaling method called direction filtering is proposed to select the prediction direction of FME by comparing the costs resulting from forward and backward IMEs. With direction filtering in addition to the three complexity scaling methods for baseline compression, 32 complexity levels are defined for main profile compression. Experimental results show that the speed control algorithm guarantees the processing time to meet the given time budget with negligible quality degradation. Various complexity levels for speed control are also used to speed up the encoding time with a slight degradation in quality and a minor reduction of the compression efficiency.

An H.264 High-Profile Intra-Prediction with Adaptive Selection Between the Parallel and Pipelined Executions of Prediction Modes

A high-profile H.264 intra-frame encoder is suitable for low-cost and low-power applications and capable of providing enhanced compression efficiency. The high-profile is targeting the high-resolution videos. Thus, the encoding speed should be faster than or comparable to the baseline-profile. In previous work related to a hardware-based baseline-profile intra-frame encoder, a speed-up is achieved by the early termination of the intra modes and by an increase in the rate of hardware utilization only under one of the serialized and parallel schedules. This paper proposes a novel pipeline schedule for a hardware-based high-profile intra-prediction scheme in which the 8 × 8 prediction is performed in Stage 1 and 4 × 4, 16 ×16 and chroma predictions are executed during Stage 2. The processing time of Stage 2 is efficiently accelerated based on the result of the 8 × 8 prediction in Stage 1. According to the distribution of each mode, the schedule is adaptively selected between parallel and pipeline schedules. To increase the hardware utilization of the 8 × 8 prediction, the order of prediction modes and the inverse vertical transform is adaptively adjusted. In addition, early termination of the prediction modes is employed for a fast 8 × 8 prediction. The proposed 8 × 8 intra-prediction is implemented and verified as an entire intra-frame encoder. Experimental results show that the average number of cycles necessary to process one MB for videos with resolutions of 1920 ×1080 and 3840 × 2160 are only 269 and 253 cycles, respectively. Compared to JM13.2, the bitrate is increased by 1.13% on average with a small PSNR degradation of 0.06 dB. The difference in the rate-distortion performance between the proposed high-profile intra-prediction scheme and JM 13.2 is not significant, whereas the achieved speed-up due to the proposed schemes is considerable compared to the conventional hardware-based intra-prediction encoders.

An SoC integrating an H.264 encoder with an ISP

The SoC presented in this paper integrates an H.264 encoder with an ISP (Image Signal Processor). It is currently implemented in an FPGA and processes an HD-size (1280 × 720) image at the speed of 15 fps with the operating clock frequency of 50 MHz. In the presented demo system, a Bayer input from a CMOS image is given to the FPGA and the output stream is transmitted through an USB transceiver to a PC that decodes and displays the H.264 stream.

A Novel Hardware Architecture of the Lucas–Kanade Optical Flow for Reduced Frame Memory Access

The Lucas-Kanade (LK) algorithm is a cost-efficient gradient-based algorithm for real-time optical flow generation. An excessive external memory access limits the LK algorithm from being broadly used in practical high-frame-rate applications. To overcome this limitation, this paper proposes a novel hardware architecture that stores the input image after the Gaussian filtering operation instead of the original input image itself. The Gaussian-filtered image is downsampled in both the horizontal and vertical directions, thus reducing the external memory access to one quarter of the original data. The downsampling operation does not cause a significant degradation of accuracy because the Gaussian filter is a low-pass filter that reduces the aliasing effect of downsampling. The downsampled pixels are selected in an interleaved manner across multiple frames to reduce the degradation of accuracy. Experimental results show that the proposed algorithm reduces the frame memory access by 61%-75% compared with the previous research.

Cascaded Direction Filtering for Fast Multidirectional Inter-Prediction in H.264/AVC Main and High Profile Compression

Early direct mode decision is a popular technique to improve the execution speed of inter-predictions in a B slice. However, the improvement is limited when this technique is applied to a hardware-based pipelined architecture or to the video sequences where the ratio of macroblocks (MBs) encoded as the direct mode is low. This paper proposes a novel fast inter-prediction algorithm for B slices which increases the encoding speed by early decision of the direction of the inter-prediction using spatial and temporal correlation in a video. A speed-up is achieved by selecting a prediction direction with a lower complexity and discarding a prediction direction with a higher complexity. For the case when early mode decision is not possible, additional speed-up is achieved by partially skipping unidirectional motion estimation (ME) if the estimated cost of the ME is greater than the result of the direct mode prediction that is completed much faster than ME. This proposed sequence of early decisions is referred to as cascaded direction filtering (CDF). The proposed algorithm increases the early mode decision rate, which improves the encoding speed effectively for a hardware-based pipelined architecture as well as videos with a low ratio of MBs encoded as the direct mode. The additional memory size and bandwidth overhead required for the proposed CDF are very small. Experimental results show that the proposed CDF scheme improves the encoding speed by 45% for B slices and 32% for overall sequences. The bitrate is increased by less than 1% with a small peak signal-to-noise ratio degradation of 0.03 dB on average.

Power-aware design with various low-power algorithms for an H.264/AVC encoder

H.264/AVC video compression standard provides high coding efficiency, but requires a considerable amount of complexity and power consumption. This paper presents advanced low-power algorithms for an H.264/AVC encoder and a power-aware design composed of low-power algorithms. Power reduction algorithms with frame memory compression and early skip mode decision are presented, and the search range for motion estimation is reduced for further power reduction. The proposed power-aware design controls the power consumption depending on the remaining energy by controlling the operation condition of the proposed low-power algorithms. In order to estimate the power reduction by the proposed algorithms, the power consumed by external memory as well as the bus between an H.264 encoder and an external DRAM is considered. Simulation results show that up to 49.9% of the power consumed by bus and external memory is reduced and that the power consumption from 0% to 41.56% is achieved with a reasonably small degradation of R-D performance.

Complexity Reduction by Modified Scale-Space Construction in SIFT Generation Optimized for a Mobile GPU

Scale-invariant feature transform (SIFT) is one of the most widely used local features for computer vision in mobile devices. A mobile graphic processing unit (GPU) is often used to run computer-vision applications using SIFT features, but the performance in such a case is not powerful enough to generate SIFT features in real time. This paper proposes an efficient scheme to optimize the SIFT algorithm for a mobile GPU. It analyzes the conventional scale-space construction step in the SIFT generation, finding that reducing the size of the Gaussian filter and the scale-space image leads to a significant speedup with only a slight degradation of the quality of the features. Based on this observation, the SIFT algorithm is modified and implemented for real-time execution. Additional optimization techniques are employed for a further speedup by efficiently utilizing both the CPU and the GPU in a mobile processor. The proposed SIFT generation scheme achieves a processing speed of 28.30 frames/s for an image with a resolution of

An Effective Combination of Power Scaling for H.264/AVC Compression

1280 \times 720

Prediction mode reordering and IDCT direction control for fast intra 8×8 prediction

running on a Galaxy S5 LTE-A device, thereby gaining a speedup by the factors of 114.78 and 4.53 over CPU- and GPU-only implementations, respectively.