Seungchul Ryu

No-Reference Quality Assessment for Stereoscopic Images Based on Binocular Quality Perception

Quality perception of 3-D images is one of the most important parameters for accelerating advances in 3-D imaging fields. Despite active research in recent years for understanding the quality perception of 3-D images, binocular quality perception of asymmetric distortions in stereoscopic images is not thoroughly comprehended. In this paper, we explore the relationship between the perceptual quality of stereoscopic images and visual information, and introduce a model for binocular quality perception. Based on this binocular quality perception model, a no-reference quality metric for stereoscopic images is proposed. The proposed metric is a top-down method modeling the binocular quality perception of the human visual system in the context of blurriness and blockiness. Perceptual blurriness and blockiness scores of left and right images were computed using local blurriness, blockiness, and visual saliency information and then combined into an overall quality index using the binocular quality perception model. Experiments for image and video databases show that the proposed metric provides consistent correlations with subjective quality scores. The results also show that the proposed metric provides higher performance than existing full-reference methods even though the proposed method is a no-reference approach.

Stereoscopic image quality metric based on binocular perception model

Measuring a perceptual quality of an image is one of the important tasks in various applications such as image coding, processing, enhancement, and monitoring system. Although active researches have been made for objective quality assessment of 2D images for some decades, still very few efforts have been concentrated on 3D image quality assessment. In this paper, we propose a new quality metric for stereoscopic images based on the binocular perception model considering asymmetric property of a stereoscopic image pair. Experiments for publicly available databases show that the proposed metric provides consistent correlations with subjective quality scores. The results also show that the proposed metric outperforms state-of-the-arts metrics.

Adaptive competition for motion vector prediction in multi-view video coding

An adaptive competition method is proposed in order to increase the accuracy of a motion vector (MV) prediction in multi-view video coding (MVC). Motion vector predictors for INTER mode and SKIP (DIRECT) mode is optimally selected from a given adaptive set of predictors by a slightly modified rate-distortion criterion. The adaptive set of the predictor candidates is determined, based on the analysis of the predictors according to their prediction structures. The analyzed predictors include spatial, temporal, and inter-view predictors. The proposed scheme leads to the accurate and efficient MV prediction compared to the MVC reference software, JMVC 6.0. As a result, bit-rates are decreased by nearly 5% in average, up to 7.6%, in terms of the Bjontegaard Metric.

DASC: Dense adaptive self-correlation descriptor for multi-modal and multi-spectral correspondence

Establishing dense visual correspondence between multiple images is a fundamental task in many applications of computer vision and computational photography. Classical approaches, which aim to estimate dense stereo and optical flow fields for images adjacent in viewpoint or in time, have been dramatically advanced in recent studies. However, finding reliable visual correspondence in multi-modal or multi-spectral images still remains unsolved. In this paper, we propose a novel dense matching descriptor, called dense adaptive self-correlation (DASC), to effectively address this kind of matching scenarios. Based on the observation that a self-similarity existing within images is less sensitive to modality variations, we define the descriptor with a series of an adaptive self-correlation similarity for patches within a local support window. To further improve the matching quality and runtime efficiency, we propose a randomized receptive field pooling, in which a sampling pattern is optimized with a discriminative learning. Moreover, the computational redundancy that arises when computing densely sampled descriptor over an entire image is dramatically reduced by applying fast edge-aware filtering. Experiments demonstrate the outstanding performance of the DASC descriptor in many cases of multi-modal and multi-spectral correspondence.

Depth perception and motion cue based 3D video quality assessment

In this paper, we propose a depth perception and motion cue based three-dimensional (3D) video quality assessment (VQA). Depth perception provides the real 3D impression during viewing the 3D video (3DV), and motion cue is also important factor to simulate a Human Visual Systems (HVS) for 3DV. We combine the depth perception and motion cue, and generate a weighting map for 3D VQA. For adjusting contribution of every index in traditional VQA which are unsuitable for 3D VQA, we propose the weighting map based Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM) to evaluate the quality of 3DV. In experimental results section, the proposed 3D VQA have been validated using both our subjective test scores as well as traditional VQA. Our proposed method yields high correlation with measured Mean Opinion Score (MOS) and consistent performance in an asymmetric coding condition.

Local self-similarity frequency descriptor for multispectral feature matching

This paper describes a robust feature descriptor called the local self-similarity frequency (LSSF) for the multispectral RGB-NIR feature matching, which uses the frequency response of the local internal layout of self-similarities. A nonlinear relationship between multi-spectral image pairs makes conventional descriptors be sensitive to spectral deformation. To alleviate this problem, the LSSF employs a weighted correlation surface reducing the discrepancy between mul-tispectral images. Furthermore, the LSSF provides a rotation invariance exploiting the frequency response of maximal values on logpolar bins based on the fact that a cyclic shift on the log-polar representation leads only a phase shift in a frequency domain. Experimental results show that LSSF outperforms state-of-the-art descriptors in terms of a recognition rate for multispectral RGB-NIR image pairs.

Analysis of Motion Vector Predictor in Multiview Video Coding System

In this paper, various motion vector predictors are analyzed and their selection method is proposed for 3D video coding. In a ubiquitous multimedia system, video compression has been expected to be an important element since an available bandwidth is very limited. In the proposed method, several motion vector predictors are competed each other with the slightly modified rate-distortion criterion. Spatial, temporal, and inter-view predictors are considered as motion vector predictors to reduce spatial, temporal, and interview redundancies, respectively. The proposed method increases the motion vector coding efficiency by selecting the best motion vector predictor among them. Accordingly, overall bit rates are reduced by 5.2 % in average, up to 6.1 %compared to reference software JMVC 6.0 in terms of the Bjontegaard Metric.

Blind blockiness measure based on marginal distribution of wavelet coefficient and saliency

The objective measurement of blockiness plays an important role in many applications, such as the quality assessment of an image, and the design of image and video coding system. However, most of the existing no-reference blockiness metrics do not consider important influences of grid distortion of an image on the performance of the metric. In this paper, we propose a new blockiness metric, which is robust to grid distortion, based on the marginal distribution of local wavelet coefficients and saliency information. Experiments for several public image databases showed that the proposed metric provides consistent correlations with subjective blockiness scores and outperforms other existing no-reference blockiness metrics.

ABFT: Anisotropic binary feature transform based on structure tensor space

Local feature matching is a fundamental step for many computer vision applications. Recently, binary feature transforms have been popularly proposed to improve the computational efficiency while preserving high matching performance. However, it is sensitive to noise and geometrical distortion such as affine transformation. In this paper, we propose ABFT framework, composed of a noise robust feature detection and affine invariant binary feature description based on a structure tensor space. Experimental results show that ABFT outperforms other state-of-the-art feature transforms in terms of the repeatability, recognition rate, and computational time.

Synthesis quality prediction model based on distortion intolerance

Free-viewpoint video system will provide viewers with freedom to navigate through the scene at different viewpoints. In the system, arbitrary viewpoints of videos are synthesized by the depth image-based rendering with multi-view plus depth videos. Despite the widespread of technologies for free-viewpoint video system, the field of quality assessment for the free-viewpoint video, especially the quality prediction of a synthesized image, has not yet been thoroughly investigated. This paper analyzes how distortions in color and depth images influence on the quality of a synthesized image. Then, an objective quality prediction model for a synthesized image is proposed based on the concept of intolerance of synthesis distortion. Experimental results show that the proposed model provides outstanding performance in predicting the quality of a synthesized image compared to other models.