Dong-Gon Yoo

Multiframe-based bilateral motion estimation with emphasis on stationary caption processing for frame rate up-conversion

In this paper, we present a new motion compensated frame rate up-conversion algorithm that uses multiframes to enhance the accuracy of motion estimation. We also develop adaptive motion vector smoothing to correct outliers in a motion vector field. In addition, stationary caption processing is carried out for removing block artifacts from stationary captions. Finally, we propose adaptive-weighted bidirectional motion compensated interpolation to reduce object blurring. In experiments using benchmark test sequences, the proposed algorithm improves the average PSNR of interpolated frames by up to 5.34 dB compared to the conventional algorithm.

Direction-Select Motion Estimation for Motion-Compensated Frame Rate Up-Conversion

In this paper, we propose a new motion estimation algorithm to be used for motion-compensated frame rate up-conversion. The proposed algorithm independently carries out motion estimations in both forward and backward directions, and selects a more reliable one between forward and backward motion vectors by evaluating the motion vector reliability from the viewpoint of the interpolated frame. The proposed algorithm smooths and refines both the forward and backward motion vectors before selecting the reliable one. This procedure helps to select the reasonable motion estimation direction. In identifying the motion vector outliers, the proposed algorithm uses a circular range of which center is located at the mean of the eight neighboring motion vectors of the motion vector being processed. Experimental results using 1720 test images show that the proposed motion estimation algorithm improves the average peak signal-to-noise ratio and the average structural similarity of the interpolated frames by up to 5.31 dB and 0.053, respectively, compared to conventional motion estimation algorithms.

Design and implementation of median filter based adaptive motion vector smoothing for motion compensated frame rate up-conversion

In this paper, we propose a new motion vector smoothing based on the median filter for motion compensated frame rate up-conversion. The proposed method adaptively detects and corrects outliers in a motion vector field by considering neighboring motion vectors surrounding the current motion vector. Experiments show that the proposed method improves the average PSNR by up to 0.938 dB for all test sequences using the low hardware resource.

Hardware implementation of motion estimation using a sub-sampled block for frame rate up-conversion

In this paper, we present a new motion estimation hardware architecture using a sub-sampled block, which can be used for frame rate up-conversion. The proposed architecture provides the advantage of reducing computational hardware complexity greatly, compared to the conventional architecture, while maintaining the quality of interpolated images. FPGA implementation shows that the proposed motion estimation hardware architecture reduces the hardware size by 51%, compared to the conventional architecture at the cost of average PSNR degradation of only 0.22 dB for interpolated images.

33.3: A Novel Image Compression Algorithm for Overdriving

An image compression algorithm is proposed to reduce the frame memory requirements in overdriving technologies. The algorithm is expected to have a simple hardware architecture. Experiments show that the algorithm provides an average peak signal-to-noise ratio above 35 dB, while also ensuring a compression ratio above 6:1.

Stereo confidence metrics using the costs of surrounding pixels

In this paper, we propose two new stereo confidence metrics which are used for estimating the reliability of a stereo matching result more accurately. Unlike the conventional metrics which exploit the cost information of the pixel being processed only, the proposed metrics utilize the cost information of the surrounding pixels as well as the pixel being processed. By checking whether the costs of the surrounding pixels at the target disparity are the local minimum or not, or calculating the ratio between costs of surrounding pixels, the proposed metrics can estimate the reliability of a stereo matching result more correctly. Experimental results show that the two proposed metrics improve the area under the curve of the error rate as a function of disparity map density by up to 0.012 and 0.025, compared to the existing methods, respectively.

Perceived distortion aware backlight dimming for low power and high quality LCD devices

This paper presents a perceived distortion control algorithm that considers the viewers perception for the image distortion caused by reduced backlight brightness. The basic idea of the proposed method is to maintain the levels of image distortion for all local image areas below the distortion level that can be perceived by viewers.

Motion vector smoothing for motion-compensated frame rate up-conversion

In this paper, we propose a motion vector smoothing scheme for motion-compensated frame rate up-conversion. The proposed motion vector smoothing consists of three steps which are outlier detection, outlier correction, and motion vector refinement steps. The proposed method enhances the image quality of the interpolated frame by up to 0.88 dB, and the visual result of the proposed method is better than those of the benchmark methods

P.48: Adaptive Sum of the Bilateral Absolute Difference for Motion Estimation Using Temporal Symmetry

Abstract In this paper, we propose an adaptive sum of bilateral absolute difference, which is a matching criterion for motion estimation using temporal symmetry. Experimental results show that the average PSNR of the proposed methods was 1.03 dB higher than that of benchmark methods. Also, the interpolated images reconstructed by the proposed methods are better in visual than the interpolated images reconstructed by the benchmark methods in the images including the moving caption. 1. Introduction Frame rate up-conversion (FRUC) increases the frame rate by inserting intermediate frames between original frames of a moving picture. This FRUC is usually used to counter the motion blur problem which occurs in hold-type displays such as liquid crystal displays [1]. For example, the FRUC converts 50~60 Hz video signals used for conventional television into 100~120 Hz video signals. In terms of the motion of an object [2], FRUC is classified into the following two types. One type, that does not consider the motion of an object, is easy to implement in software and hardware because it uses simple operations such as frame repetition and frame averaging. However, this type causes motion jerkiness and has no effect on reducing the motion blur that occurs in hold-type displays. On the other hand, the latter type, called motion compensated FRUC (MC-FRUC), considers the motion of objects. This type creates more accurate interpolated frames compared to the former one, and hence is very effective on reducing the motion blur. The MC-FRUC is broadly divided into two steps: motion estimation (ME) and motion compensated interpolation (MCI). ME calculates the motion vectors (MVs) by estimating the motion of an object using two or more consecutive frames. MCI inserts the interpolated frames between original frames after constructing new interpolated frames using MVs obtained in the ME step. The performance of the MC-FRUC strongly depends on the performance of the ME algorithm used in the MC-FRUC. The block matching method is one of the most popular ME methods, and many block-based ME methods are implemented by utilizing a block matching method [3]. These methods, however, cannot estimate the MVs from the viewpoint of the interpolated frame, and therefore they generate block artifacts such as holes or overlapped blocks [4]. These artifacts degrade the image quality of the interpolated frames constructed in the MCI step. Recently, the ME methods which estimate MVs from the viewpoint of the interpolated frame have been developed to solve this issue. The ME methods include bilateral ME [4], phase correlated bilateral ME (PCB-ME) [5], and etc. These ME methods estimate the MVs using the temporal symmetry between blocks of the previous and current frames from the viewpoint of the processing block in the interpolated frame. However, these ME methods also have a serious problem. While the ME implemented by using the block matching method finds the MV by calculating the absolute difference between a fixed block of the current frame and a block of the previous frame, the ME methods using temporal symmetry find a MV by calculating the absolute difference between temporally symmetrical blocks of the previous and current frames. It increases the possibility of selecting a false MV because there may be many blocks having the similar absolute difference exist. False MVs degrade the image quality of the interpolated frame in the MCI step. Thus, it is necessary to improve the performance of the existing ME methods using the new matching criterion. The rest of the paper is organized as follows. Section 2 describes the proposed matching criterion in detail, and Section 3 presents the experimental results and assesses the performance of the proposed algorithm through objective and subjective evaluations. Finally, Section 4 concludes the paper.

P‐48: Adaptive Frame Rate Up‐Conversion Considering Low Computational Complexity and Complex Motion

This paper presents a new method for frame rate up-conversion using sub-sampled bilateral motion estimation, motion vector smoothing, simplified bidirectional motion compensated interpolation, and complex motion detection. The proposed method produces improved image quality and requires less computational complexity compared to conventional motion estimation. The average PSNRs of the proposed method for frame rate up-conversion outperform those of the benchmark frame rate up-conversion technique by up to 4.39 dB for the test sequences. Furthermore, the motion estimation of the proposed FRUC method reduces the required computational complexity to 12.5% of that required for full-search block matching motion estimation.