Gangyi Jiang

Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics

Perceptual quality assessment is a challenging issue in 3D signal processing research. It is important to study 3D signal directly instead of studying simple extension of the 2D metrics directly to the 3D case as in some previous studies. In this paper, we propose a new perceptual full-reference quality assessment metric of stereoscopic images by considering the binocular visual characteristics. The major technical contribution of this paper is that the binocular perception and combination properties are considered in quality assessment. To be more specific, we first perform left-right consistency checks and compare matching error between the corresponding pixels in binocular disparity calculation, and classify the stereoscopic images into non-corresponding, binocular fusion, and binocular suppression regions. Also, local phase and local amplitude maps are extracted from the original and distorted stereoscopic images as features in quality assessment. Then, each region is evaluated independently by considering its binocular perception property, and all evaluation results are integrated into an overall score. Besides, a binocular just noticeable difference model is used to reflect the visual sensitivity for the binocular fusion and suppression regions. Experimental results show that compared with the relevant existing metrics, the proposed metric can achieve higher consistency with subjective assessment of stereoscopic images.Perceptual quality assessment is a challenging issue in 3D signal processing research. It is important to study 3D signal directly instead of studying simple extension of the 2D metrics directly to...

Stereoscopic visual attention model for 3d video

Compared with traditional mono-view video, three-dimensional video (3DV) provides user interactive functionalities and stereoscopic perception, which makes people more interested in pop-out regions or the regions with small depth value. Thus, traditional visual attention model for mono-view video can hardly be directly applied to stereoscopic visual attention (SVA) analysis for 3DV. In this paper, we propose a bottom-up SVA model to simulate human visual system with stereoscopic vision more accurately. The proposed model is based on multiple perceptual stimuli including depth information, luminance, color, orientation and motion contrast. Then, a depth based dynamic fusion is proposed to integrate these features. The experimental results on multi-view video test sequences show that the proposed model maintains high robustness and is able to efficiently simulate SVA of human eyes.

Asymmetric Coding of Multi-View Video Plus Depth Based 3-D Video for View Rendering

The recent years have witnessed three-dimensional (3-D) video technology to become increasingly popular, as it can provide high-quality and immersive experience to end users, where view rendering with depth-image-based rendering (DIBR) technique is employed to generate the virtual views. Distortions in depth map may induce geometry changes in the virtual views, and distortions in texture video may be propagated to the virtual views. Thus, effective compression of both texture videos and depth maps is important for 3-D video system. From the perspective of bit allocation, asymmetric coding of the texture videos and depth maps is an effective way to get the optimal solution of 3-D video compression and view rendering problems. In this paper, a novel asymmetric coding method of multi-view video plus depth (MVD) based 3-D video is proposed on purpose of providing high-quality view rendering. In the proposed method, two models are proposed to characterize view rendering distortion and binocular suppression in 3-D video. Then, an asymmetric coding method of MVD-based 3-D video is proposed by combining two models in encoding framework. Finally, a chrominance reconstruction algorithm is presented to achieve accurate reconstruction. Experimental results show that compared with other methods, the proposed method can obtain higher performance of view rendering under the total bitrate constraint. Moreover, the perceptual visual quality of 3-D video is almost unaffected with the proposed method.

Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties.

Quality assessment of 3D images encounters more challenges than its 2D counterparts. Directly applying 2D image quality metrics is not the solution. In this paper, we propose a new full-reference quality assessment for stereoscopic images by learning binocular receptive field properties to be more in line with human visual perception. To be more specific, in the training phase, we learn a multiscale dictionary from the training database, so that the latent structure of images can be represented as a set of basis vectors. In the quality estimation phase, we compute sparse feature similarity index based on the estimated sparse coefficient vectors by considering their phase difference and amplitude difference, and compute global luminance similarity index by considering luminance changes. The final quality score is obtained by incorporating binocular combination based on sparse energy and sparse complexity. Experimental results on five public 3D image quality assessment databases demonstrate that in comparison with the most related existing methods, the devised algorithm achieves high consistency with subjective assessment.

Research on subjective stereoscopic image quality assessment

Research on stereoscopic image and video processing has been the new trend in the recent years. Measurement of visual quality is of fundamental importance for numerous stereoscopic image and video processing applications. The goal of quality assessment is to automatically assess the quality of images or videos in agreement with human quality judgments and optimize the stereoscopic image or video systems. Unfortunately, human are lack of the knowledge for the perception quality of stereoscopic images. In this paper, we present experimental results of an extensive subjective quality assessment experiment for stereoscopic images in which a total 400 distorted stereoscopic images were evaluated by about twenty human subjects. The stereoscopic images quality data obtained from 8,000 individual human quality judgments is used to build a database that can be exploited for understanding perception of stereoscopic images and provide data for objective assessment metrics designing. The experimental results indicated that the quality perception of the distorted stereoscopic images is content and distortion types dependent.

Joint Bit Allocation and Rate Control for Coding Multi-View Video Plus Depth Based 3D Video

In three-dimensional (3D) video coding, distortion in texture video and depth maps can all affect the quality of the synthesized virtual views. Therefore, under the total bitrate constraint, effective bit allocation between texture and depth information is very important for 3D video coding. In this paper, the major technical contribution is to formulate view synthesis quality for optimal resource allocation in 3D video coding, since such quality is what that matters most to the ultimate user (i.e., the viewer) of the system; to be more specific, a new joint bit allocation and rate control method for multi-view video plus depth (MVD) based 3D video coding is proposed accordingly. We firstly derive a view synthesis distortion model to characterize the effect of coding distortion of texture video and depth maps on the synthesized virtual views. Based on this model, we derive a rate-distortion model to characterize the relationship between the bitrate and the view synthesis distortion, and the optimal bitrate ratio between texture and depth is established adaptively by solving the associated optimization problem. Finally, the rate control algorithm is performed on view level, texture/depth level and frame level. Experimental results show that compared with other methods, the proposed bit allocation method obtains higher performance of view synthesis. Moreover, the proposed rate control method can accurately control the bitrate to satisfy the total bitrate constraint.

Regional Bit Allocation and Rate Distortion Optimization for Multiview Depth Video Coding With View Synthesis Distortion Model

In this paper, we propose a view synthesis distortion model (VSDM) that establishes the relationship between depth distortion and view synthesis distortion for the regions with different characteristics: color texture area corresponding depth (CTAD) region and color smooth area corresponding depth (CSAD), respectively. With this VSDM, we propose regional bit allocation (RBA) and rate distortion optimization (RDO) algorithms for multiview depth video coding (MDVC) by allocating more bits on CTAD for rendering quality and fewer bits on CSAD for compression efficiency. Experimental results show that the proposed VSDM based RBA and RDO can improve the coding efficiency significantly for the test sequences. In addition, for the proposed overall MDVC algorithm that integrates VSDM based RBA and RDO, it achieves 9.99% and 14.51% bit rate reduction on average for the high and low bit rate, respectively. It can improve virtual view image quality 0.22 and 0.24 dB on average at the high and low bit rate, respectively, when compared with the original joint multiview video coding model. The RD performance comparisons using five different metrics also validate the effectiveness of the proposed overall algorithm. In addition, the proposed algorithms can be applied to both INTRA and INTER frames.

Subjective quality analyses of stereoscopic images in 3DTV system

Subjective quality evaluation is the basis of quality evaluation of stereoscopic images. As the lack of a public and diverse testing database currently, in this paper, a symmetric stereoscopic images database is built. And then the subjective quality of stereoscopic images is analyzed from two aspects, one is the effects of JPEG, JPEG2000, H.264. The other is the comparisons between symmetric and asymmetric stereoscopic images from Gaussian blurring, white Gaussian noise, JPEG and JPEG2000, respectively. The results show three compressions are quite different in the subjective quality of symmetric stereoscopic images at different bitrates, and the comparisons between symmetric and asymmetric stereoscopic images investigate the properties of binocular fusion, binocular suppression, and binocular summation.

Blind Image Quality Assessment for Stereoscopic Images Using Binocular Guided Quality Lookup and Visual Codebook

The field of assessing three-dimensional (3-D) visual experience is challenging. In this paper, we propose a new blind image quality assessment for stereoscopic images by using binocular guided quality lookup and visual codebook. To be more specific, in the training stage, we construct phase-tuned quality lookup (PTQL) and phase-tuned visual codebook (PTVC) from the binocular energy responses based on stimuli from different spatial frequencies, orientations, and phase shifts. In the test stage, blind quality pooling can be easily achieved by searching the PTQL and PTVC, and the quality score is obtained by averaging the largest values of all patchs quality. Experimental results on three 3-D image quality assessment databases demonstrate that in comparison with the most related existing methods, the devised algorithm achieves high consistency alignment with subjective assessment and low-complexity pooling.

Stereoscopic video coding with asymmetric luminance and chrominance qualities

Three-dimensional (3D) video technology is becoming increasingly popular, as it can provide stereoscopic perception and immersive experience to end users. Most three-dimensional video compression schemes had been developed for stereoscopic video using traditional two-dimensional video compression techniques. However, stereoscopic perception had not been considered in these methods to further improve the coding efficiency. In this paper, a novel stereoscopic video coding method is proposed with asymmetric luminance and chrominance qualities based on the suppression theory of binocular vision. We base our method on two hypotheses: a quantitative just noticeable distortion (JND) threshold exists between left and right views, and chrominance degradation in right view will not affect the stereoscopic effect. Then, two subjective perception experiments are conducted to verify the above hypotheses. Finally, an asymmetric stereoscopic video coding method is engineered deriving from the hypotheses. Experimental results show that the proposed method saves bitrate significantly by from 15% to 35%, and can achieve superior reconstruction quality at trivial degradation cost.