Mohamed-Chaker Larabi

A perceptual metric for stereoscopic image quality assessment based on the binocular energy

Stereoscopic imaging is becoming very popular and its deployment by means of photography, television, cinema. . .is rapidly increasing. Obviously, the access to this type of images imposes the use of compression and transmission that may generate artifacts of different natures. Consequently, it is important to have appropriate tools to measure the quality of stereoscopic content. Several studies tried to extend well-known metrics, such as the PSNR or SSIM, to 3D. However, the results are not as good as for 2D images and it becomes important to have metrics dealing with 3D perception. In this work, we propose a full reference metric for quality assessment of stereoscopic images based on the binocular fusion process characterizing the 3D human perception. The main idea consists of the development of a model allowing to reproduce the binocular signal generated by simple and complex cells, and to estimate the associated binocular energy. The difference of binocular energy has shown a high correlation with the human judgement for different impairments and is used to build the Binocular Energy Quality Metric (BEQM). Extensive experiments demonstrated the performance of the BEQM with regards to literature.

CYCLOP: A stereo color image quality assessment metric

In this work, a reduced reference (RR) perceptual quality metric for color stereoscopic images is presented. Given a reference stereo pair of images and their “distorted” version, we first compute the disparity map of both the reference and the distorted stereoscopic images. To this end, we define a method for color image disparity estimation based on the structure tensors properties and eigenvalues/eigenvectors analysis. Then, we compute the cyclopean images of both the reference and the distorted pairs. Thereafter, we apply a multispectral wavelet decomposition to the two cyclopean color images in order to describe the different channels in the human visual system (HVS). Then, contrast sensitivity function (CSF) filtering is performed to obtain the same visual sensitivity information within the original and the distorted cyclopean images. Thereafter, based on the properties of the human visual system (HVS), rational sensitivity thresholding is performed to obtain the sensitivity coefficients of the cyclopean images. Finally, RR stereo color image quality assessment (SCIQA) is performed by comparing the sensitivity coefficients of the cyclopean images and studying the coherence between the disparity maps of the reference and the distorted pairs. Experiments performed on color stereoscopic images indicate that the objective scores obtained by the proposed metric agree well with the subjective assessment scores.

Factors Influencing Quality of Experience

In this chapter different factors that may influence Quality of Experience (QoE) in the context of media consumption, networked services, and other electronic communication services and applications, are discussed. QoE can be subject to a range of complex and strongly interrelated factors, falling into three categories: human, system and context influence factors (IFs). With respect to Human IFs, we discuss variant and stable factors that may potentially bear an influence on QoE, either for low-level (bottom-up) or higher-level (top-down) cognitive processing. System IFs are classified into four distinct categories, namely content-, media-, network- and device-related IFs. Finally, the broad category of possible Context IFs is decomposed into factors linked to the physical, temporal, social, economic, task and technical information context. The overview given here illustrates the complexity of QoE and the broad range of aspects that potentially have a major influence on it.

A survey of perceptual image processing methods

Perceptual approaches have been widely used in many areas of visual information processing. This paper presents an overview of perceptual based approaches for image enhancement, segmentation and coding. The paper also provides a brief review of image quality assessment (IQA) methods, which are used to evaluate the performance of visual information processing techniques. The intent of this paper is not to review all the relevant works that have appeared in the literature, but rather to focus on few topics that have been extensively researched and developed over the past few decades. The goal is to present a perspective as broad as possible on this actively evolving domain due to relevant advances in vision research and signal processing. Therefore, for each topic, we identify the main contributions of perceptual approaches and their limitations, and outline how perceptual vision has influenced current state-of-the-art techniques in image enhancement, segmentation, coding and visual information quality assessment.

Perceptual Metrics for Static and Dynamic Triangle Meshes

Almost all mesh processing procedures cause some more or less visible changes in the appearance of objects represented by polygonal meshes. In many cases, such as mesh watermarking, simplification or lossy compression, the objective is to make the change in appearance negligible, or as small as possible, given some other constraints. Measuring the amount of distortion requires taking into account the final purpose of the data. In many applications, the final consumer of the data is a human observer, and therefore the perceptibility of the introduced appearance change by a human observer should be the criterion that is taken into account when designing and configuring the processing algorithms. In this review, we discuss the existing comparison metrics for static and dynamic (animated) triangle meshes. We describe the concepts used in perception-oriented metrics used for 2D image comparison, and we show how these concepts are employed in existing 3D mesh metrics. We describe the character of subjective data used for evaluation of mesh metrics and provide comparison results identifying the advantages and drawbacks of each method. Finally, we also discuss employing the perception-correlated metrics in perception-oriented mesh processing algorithms.

Spatial-temporal Video Quality Metric based on an estimation of QoE

In this work a new Reduced Reference (RR) Video Quality Metric (VQM) is proposed. The method takes advantage of the Human Visual System (HVS) sensitivity to sharp changes in the video. The proposed method has a spatial-temporal approach and because of that it is named as STAQ (Spatial-Temporal Assessment of Quality). In the first step of STAQ we take a temporal approach and find the matching regions in consecutive frames. In the next step, a spatial approach is taken in the way of calculating the quality of the matching regions in the temporal approach. In the last step, the quality of the video is calculated based on the parameters gathered in the spatial and temporal domain and using the motion activity density of the video as a controlling factor. An important improvement lies in taking into account the Quality of Experience (QoE) represented as the motion activity density of the reference video. The results show a great improvement in the case of H.264 and MPEG-2 compressed and IP distorted videos even when compared to state of the art Full Reference (FR) metrics.

Feature-Based Color Correction of Multiview Video for Coding and Rendering Enhancement

Multiview video (MVV) consists of capturing the same scene with multiple cameras from different viewpoints. Therefore, substantial illumination and color inconsistencies can be observed between different views. These color mismatches can significantly reduce compression efficiency and rendering quality. In this paper, we propose a preprocessing method for correcting these color discrepancies in MVV. To consider the occlusion problem, our method is based on an improvement of histogram matching (HM) algorithm using only common regions across views. These regions are defined by an invariant feature detector (scale invariant feature transform), followed by random sample consensus algorithm to increase the matching robustness. In addition, to maintain temporal correlation, HM algorithm is applied on a temporal sliding window, allowing to cope with time-varying acquiring system, camera moving capture, and real-time broadcasting. Moreover, unlike always choosing the center view as the reference by default, we propose an automatic selection algorithm based on both views statistics and quality. The experimental results show that the proposed method increases coding efficiency with gains of up to 1.1 and 2.2 dB for the luminance and chrominance components, respectively. Furthermore, once the correction is performed, the color of real and rendered views is harmonized and looks very consistent as a whole.

A grouplet-based reduced reference image quality assessment

The past decades have witnessed the tremendous growth of digital image processing techniques for visual information representation and communication. Particularly, computational representation of perceived image quality has become a fundamental problem in computer vision and image processing. It is well known that the commonly used Peak Signal-to-Noise Ratio (PSNR), although analysis friendly, falls far short of this need. In this work, we propose a reduced reference (RR) perceptual image quality measure (IQM) based on the grouplet transform. Given a reference image and its “distorted” version, we first compute the grouplet transform in order to extract the information of textures and directions of both images. Then, contrast sensitivity function (CSF) filtering is performed to obtain same visual sensitivity information within both images. Thereafter, based on the properties of the human visual system (HVS), rational sensitivity thresholding is performed to obtain the sensitivity coefficients of both images. Finally, RR image quality assessment (IQA) is performed by comparing the sensitivity coefficients of both images.

Towards a perceptual quality metric for color stereo images

In this paper, we propose a quality metric for color stereo images. The concept of our metric is inspired by the behavior of simple and complex cells located in the primary visual cortex. These cells are responsible for merging left and right retinal images. To replicate the task performed by these cells, we adopted an approach based on spatial-frequency transform with the processing of selective orientations. From that, a model that calculates the binocular energy contained in the left and right retinal images has been proposed. The amplitude variation of the binocular energy defines the quality criterion of the reconstructed depth within the Human Visual System (HVS). Finally, from the experimental results, the used criterion seems to be correlated to human judgment obtained by psychophysical tests.

A no-reference color video quality metric based on a 3D multispectral wavelet transform

In this work, a no reference objective color video quality assessment metric is presented. First, a multi-valued 3D subband wavelet decomposition is defined. This wavelet representation is used to decompose the video sequence in order to describe the different channels in the human visual system (HVS). Thereafter, based on the properties of the HVS, a perceptual mask that integrates spatio-temporal contrast sensitivity function (CSF) and luminance sensitivity is applied to each wavelet band. Then, we define a flow tensor between successive frames. This flow tensor is weighted by the perceptual mask and is used to define a NR color video quality metric. Particularly, based on the eigenvalues and eigenvectors of the flow tensor, we study the inter-frame coherence and the sharpness of edges in the successive frames. Experiments performed on video sequences indicate that the objective scores obtained by the proposed metric agree well with the subjective assessment scores.