Kyuha Choi

Example-Based Super-Resolution via Structure Analysis of Patches

In example-based super-resolution, it is difficult to determine appropriate high-frequency (HF) patches from a training database by using only the information of one input image. In this letter, we utilize the sharpness of high-resolution (HR) patch candidates for the reliable determination of HF patches. For each input patch, we first preselect a sufficient number of HF patch candidates and produce HR patches by adding the candidates to the input patch. After removing the outlier patches, we then reselect several HF patches according to the patch characteristic for producing the final HR image. This reselection procedure is optimized for edge patches and non-edge patches, respectively. Experimental results show that the proposed algorithm provides sharper details compared to the existing algorithms.

Frame Rate Up Conversion Based on Variational Image Fusion

This paper presents a new framework for motion compensated frame rate up conversion (FRUC) based on variational image fusion. The proposed algorithm consists of two steps: 1) generation of multiple intermediate interpolated frames and 2) fusion of those intermediate frames. In the first step, we determine four different sets of the motion vector field using four neighboring frames. We then generate intermediate interpolated frames corresponding to the determined four sets of the motion vector field, respectively. Multiple sets of the motion vector field are used to solve the occlusion problem in motion estimation. In the second step, the four intermediate interpolated frames are fused into a single frame via a variational image fusion process. For effective fusion, we determine fusion weights for each intermediate interpolated frame by minimizing the energy, which consists of a weighted- L1-norm based data energy and gradient-driven smoothness energy. Experimental results demonstrate that the proposed algorithm improves the performance of FRUC compared with the existing algorithms.

Improvement on learning-based super-resolution by adopting residual information and patch reliability

Learning-based super-resolution algorithms synthesize high-resolution details by using training data. However, since an input image does not belong to a training image set, there is a limitation in recovering its high-frequency details. In our approach, we build and utilize residual training data to complement missing details. We first estimate a pair of mid-and high-frequency images of each training image by using ordinary training data. We then build residual training data by obtaining the residual mid-and high-frequency images that denote the difference between the estimation and original. Thereby, we can synthesize high-resolution details better by using both ordinary and residual training data sets. In addition, in order to use training data more efficiently, we adaptively select low-resolution patches in an input image. Experimental results demonstrate that the proposed method can synthesize higher-resolution images compared to the existing algorithms.

Resolution Improvement of Infrared Images Using Visible Image Information

This letter presents a new framework for improving the spatial resolution of infrared (IR) images based on the high-resolution visible image information. Edge regions in an IR image, which have a strong correlation with the aligned edges in the visible image, are sharpened by using their high frequency patches, which are locally estimated from the visible image. The estimation is performed on the basis of intensity correlations between two images. In addition, in order to improve the resolution in the uncorrelated edge regions and the texture regions where visible image information is not available, we adopt learning-based and reconstruction-based super resolution algorithms, respectively. Experimental results demonstrate that the proposed algorithm improves the spatial resolution compared with the existing upsampling algorithms.

Sinogram-based super-resolution in PET

Spatial resolution is intrinsically limited in positron emission tomography (PET) systems, mainly due to the crystal width. To increase the spatial resolution for a given crystal width, mechanical movements such as wobble and dichotomic motions are introduced to the PET systems. However, multiple sinograms obtained through such movements provide oversampled data. In this paper, to increase the spatial resolution, we present a novel super-resolution (SR) scheme that employs multiple sinograms. For SR, we first propose a blur kernel estimation scheme through a Monte Carlo simulation. Based on the estimated blur kernel, we adopt a maximum a posteriori expectation maximization method in estimating a high-resolution sinogram from multiple low-resolution sinograms. The proposed algorithm provides noticeable improvement of the spatial resolution in real PET images.

Robust learning-based super-resolution

Learning-based super-resolution algorithms synthesize a high-resolution image based on learning patch pairs of low- and high-resolution images. However, since a low-resolution patch is usually mapped to multiple high-resolution patches, unwanted artifacts or blurring can appear in super-resolved images. In this paper, we propose a novel approach to generate a high quality, high-resolution image without introducing noticeable artifacts. Introducing robust statistics to a learning-based super-resolution, we efficiently reject outliers which cause artifacts. Global and local constraints are also applied to produce a more reliable high-resolution image. Experimental results demonstrate that the proposed algorithm can synthesize higher quality, higher-resolution images compared to the existing algorithms.

Post-Processing for Blocking Artifact Reduction Based on Inter-Block Correlation

Block-based coding introduces an undesirable discontinuity between neighboring blocks in reconstructed images. This image degradation, referred to as blocking artifacts, arises mainly due to the loss of inter-block correlation in the quantization process of discrete cosine transform coefficients. In many multimedia broadcasting applications, such as a television, decoded video sequences suffer from blocking artifacts. In this paper, we present a novel post-processing algorithm based on increment of inter-block correlation aimed at reducing blocking artifacts. We first smooth the three lowest frequency discrete cosine transform (DCT) coefficients between neighboring blocks, in order to reduce blocking artifacts in the flat region, which are most sensitive to the human visual system. We then group each edge block and its matched blocks together and apply group-based filtering to increase the correlation between grouped blocks. This suppresses blocking artifacts in the edge region while preserving details. In addition, the algorithm is extended to reduce flickering artifacts as well as blocking artifacts in video sequences. Experimental results show that the proposed method successfully alleviates blocking artifacts in both images and videos coded with low bit-rates.

Post processing for blocking artifact reduction

Since current coding standards rely on block based processing, reconstructed images include horizontal and vertical grid noise along the block boundaries. This image degradation, called blocking artifacts, is mainly caused due to the quantization process of discrete cosine transform (DCT) coefficients. In this paper, we present a novel post processing framework for blocking artifact reduction, which is based on the correction of a few quantized DCT coefficients. We first smooth inter-block DCT coefficients for the three lowest frequency (LF) ones, in order to reduce blocking artifacts in the flat region which are most sensitive to the human visual system (HVS). We then apply block based 3-D filtering to the edge region to reduce the remaining artifacts. Experimental results show that the proposed method successfully alleviates blocking artifacts in the images coded with low bit-rates.

Texture enhancement for improving single-image super-resolution performance

Although various single-image super-resolution algorithms have been developed to increase image resolution, they still do not provide adequate performance in the texture region due to the lack of fine textures in the processed image. In this paper, we present a novel texture enhancement strategy in order to improve the super-resolution performance in the texture region. For texture enhancement, we extract a low-resolution texture layer from an input image and generate a high-resolution texture layer by applying the proposed texture synthesis algorithm. A texture enhanced high-resolution image is then obtained by properly combining the generated high-resolution texture layer with an image obtained by using an existing single-image super-resolution algorithm. Experimental results show that the proposed texture enhancement strategy provides sharper and more natural looking textures compared with the existing super-resolution algorithms. This paper addresses a key remaining issue, texture SR, which is yet unresolved.Patch-based texture synthesis is performed by using a LR texture layer.A HR texture patch is synthesized based on both AR and reconstruction models.The proposed algorithm provides realistic SR images with sharp and fine textures.

Blind Post-Processing for Ringing and Mosquito Artifact Reduction in Coded Videos

Block-based video-coding standards produce unwanted spatial and temporal artifacts in reconstructed videos. Among them, ringing and mosquito artifacts arise due to the quantization of high-frequency discrete cosine transform coefficients. Most of the existing artifact-reduction algorithms assume that coding information such as a standard quantization table and the corresponding quantization parameter for each block are available. In many multimedia applications, however, external video inputs are usually supplied without coding information. To effectively reduce the ringing and mosquito artifacts in a decoded input video sequence, it is necessary to control the filter strength block by block. In this paper, we present a blind block-based method to estimate the quantization amount and propose a novel post-processing algorithm based on the visibility of artifacts in terms of the human visual system using the estimated quantization amount. Experimental results demonstrate that the proposed algorithm better alleviates ringing and mosquito artifacts in various coded videos compared with the existing algorithms.