Yi-Nung Liu

A no -reference quality evaluation method for CFA Demosaicking

Color Filter Array Demosaicking is one of the key components in a image signal processing pipeline. In this paper, a no-reference quality evaluation method is introduced for CFA demosaicking, where the concept of double interpolation is applied. In addition, the double interpolation (DI) scheme can be used to locate the interpolation artifacts which make us uncomfortable. Experimental results show that the DI difference map can provide local quality assessments and the DI PSNR can provide a very good estimation of PSNR without ground truth image as a global quality assessment.

Algorithm and Architecture Design of High-Quality Video Upscaling Using Database-Free Texture Synthesis

Because of real-time requirements and low hardware-cost constraints, conventional TV scalers can only employ basic interpolation technique and thus introduces some artifacts that degrade the viewing quality of the output sequences. In this paper, a low-complexity super-resolution (SR) algorithm, which can provide vivid output image with rich details and sharp edges, and its associated hardware architecture, is proposed. There are two main contributions in this paper. The first is the development of the database-free texture synthesis technique. With the fractal property of nature images, it is possible to find proper high-resolution patches in a low-resolution input image itself. Therefore, the texture synthesis can be performed without database to provide proper and rich details. In addition, a faithful reconstruction constraint is used to maintain the temporal consistency. The second contribution is the hardware architecture design of the database-free texture synthesis. Partial-sum reuse technique is developed to reduce 76% of computation in the texture synthesis, and a tile-based processing technique is proposed to dramatically reduce the on-chip memory and off-chip memory bandwidth requirements. Experimental results show that the proposed SR algorithm outperforms other ones with reasonable hardware cost, where 766k in gate count and 22 kB in on-chip SRAM are required to achieve full high-definition processing ability at the working frequency of 240 MHz. The results show that the proposed algorithm and architecture are able to provide high-quality output in real-time while solving the problems of zigzag and blurred effects caused by the conventional scalers.

Motion blur reduction of liquid crystal displays using perception-aware motion compensated frame rate up-conversion

A perception-aware motion-compensated frame interpolation algorithm has been proposed. There is no motion estimation process in this low cost frame rate up-conversion (FRUC) algorithm. With the proposed filter-based MV processing techniques and the motion compensation (MC) decision in a motion compensated interpolation (MCI) stage, the proposed algorithm reduces the memory and bandwidth requirement while maintains the visual quality of the interpolation results. The key factor is that most computations in the traditional interpolation are redundant and could be skipped without quality loss because properties of the human visual system.

MRF-based true motion estimation using H.264 decoding information

Markov Random Field (MRF) has been successfully used to formulate the energy minimization problems in computer vision. However, a multi-label MRF model such as the conventional true motion estimation approach requires a significant amount of computation due to its large search space. Besides, we observe that decoding information obtained from H.264/AVC could be applied to reduce the computational complexity of true motion estimation. In this paper, a new true motion estimation scheme is proposed. We analyze the motion information and macroblock types from H.264/AVC decoder. According to the decoding information, predictors from the obtained motion vectors (MVs) are selected for MRF models. With these predictors, the search space of MRF could be reduced from O(n2) to O(n) compared to conventional full search scheme. Experimental results evaluated on the Middlebury optical flow benchmarks show that our proposed scheme is able to optimize the MV field of H.264/AVC decoder to approximate the true motion field.

Direction-adaptive image upsampling using double interpolation

Double interpolation quality evaluation can be used as a measurement of an interpolation operation. By using this double interpolation framework, a high efficient direction-adaptive upsampling algorithm is proposed without any threshold setting and post-processing. With the proposed upsampling algorithm, the problem of zigzagging artifacts on the edge no longer exists. Moreover, the proposed algorithm has low computation complexity. The experimental results show that the proposed algorithm has high quality image output.

Color filter array demosaicking using joint bilateral filter

Bilateral filter has shown its outstanding performance in image denoising and other multimedia applications. In this paper, a new color interpolation technique named joint bilateral demosaicking is proposed. Considering the image gradient, an edge-sensing initialization step is performed. In addition, joint bilateral filter exploits the correlation between color channels with the information from initialization while preserving edges. Experimental results show the superiority of the proposed technique in suppressing demosaicking artifacts and producing more visually acceptable images.

Algorithm adaptive video deinterlacing using self-validation framework

Deinterlacing is well known as an ill-posed problem of video restoration. In this paper, a self-validation framework is proposed to solve the problem. First, various algorithms with different assumptions will be performed to generate candidate results. Then, a method called double interpolation is applied to test the consistency of each algorithm. Finally, the value of each missing pixel will be chosen among the results of those algorithms according to their performances in the previous test. By doing so, each situation can be handled by the most suitable algorithm and the overall result can be better than any of them. Experimental results demonstrate the validity of the proposed framework in both subjective and objective assessments.

Bandwidth and local memory reduction of video encoders using Bit Plane Partitioning Memory Management

This paper presents a new memory management scheme for the reference frame buffer in a video encoder. The proposed BPPMM (Bit Plane Partitioning Memory Management) scheme changes the format of data, and hence we can access different number of the bit-planes of the pixel data on-demand. This BPPMM technique is especially suitable for motion estimation with bit-truncation. Experiments show that when BPPMM scheme is integrated in an H.264 hardware encoder, more than 46% of local SRAM size and 31% of the external memory bandwidth could be reduced with only a little quality degradation.

P‐36: A Psychophysical Analysis on Perceptual Limitation of Motion Image Artifact Reduction Using 120Hz Displays

In this paper, several psychophysical experiments are conducted to find the perceptual limitation in terms of duration, retinal velocity, and direction of motions. Moreover, the jerkiness artifacts are considered together with motion blur artifacts. The analysis results can provide important information for the computation reduction of frame rate up conversion.

Coding Mode Analysis of MPEG-2 to H.264/AVC Transcoding for Digital TV Applications

MPEG-2 to H.264/AVC transcoding is an important module for video recoding in digital TV applications. For pixel domain transcoding, MPEG-2 bitstream is decoded and then re-encoded by H.264 encoder. Since the behavior of the decoded video is different from the original video, in this paper, the performance of each coding mode is analyzed to select the effective coding tools. It is shown from the analysis that transcoding with motion estimation with only one reference frame and 16times16 block size and deblocking filter can achieve almost the same video quality with only 19% of the computation. The analysis result could be an important reference for the implementation of MPEG-2 to H.264 transcoder.