Rafik Bensalma

A perceptual metric for stereoscopic image quality assessment based on the binocular energy

Stereoscopic imaging is becoming very popular and its deployment by means of photography, television, cinema. . .is rapidly increasing. Obviously, the access to this type of images imposes the use of compression and transmission that may generate artifacts of different natures. Consequently, it is important to have appropriate tools to measure the quality of stereoscopic content. Several studies tried to extend well-known metrics, such as the PSNR or SSIM, to 3D. However, the results are not as good as for 2D images and it becomes important to have metrics dealing with 3D perception. In this work, we propose a full reference metric for quality assessment of stereoscopic images based on the binocular fusion process characterizing the 3D human perception. The main idea consists of the development of a model allowing to reproduce the binocular signal generated by simple and complex cells, and to estimate the associated binocular energy. The difference of binocular energy has shown a high correlation with the human judgement for different impairments and is used to build the Binocular Energy Quality Metric (BEQM). Extensive experiments demonstrated the performance of the BEQM with regards to literature.

Towards a perceptual quality metric for color stereo images

In this paper, we propose a quality metric for color stereo images. The concept of our metric is inspired by the behavior of simple and complex cells located in the primary visual cortex. These cells are responsible for merging left and right retinal images. To replicate the task performed by these cells, we adopted an approach based on spatial-frequency transform with the processing of selective orientations. From that, a model that calculates the binocular energy contained in the left and right retinal images has been proposed. The amplitude variation of the binocular energy defines the quality criterion of the reconstructed depth within the Human Visual System (HVS). Finally, from the experimental results, the used criterion seems to be correlated to human judgment obtained by psychophysical tests.

Binocular Energy Estimation Based on Properties of the Human Visual System

Abstract3D applications are very popular nowadays. They allow to bring new sensations (e.g., cinema, gaming, etc) and new ways for analyzing data (e.g., video-surveillance, pattern recognition, etc). The large availability of 3D data ia better understanding of human 3D perception in order to improve the quality of 3D visual data, increase the visual comfort and avoid visual fatigue and visual illness. In this paper, we explore the binocular perception through its various indices. The focus is put on the binocular energy and its evolution with regard to image impairments. Two types of cells are explored, that is, simple and complex cells responsible of the sensory fusion in the visual cortex. A model is proposed for these cells in order to estimate the binocular energy from color stereo-pairs. The integration of stereoscopic constraints such as unicity, coherence and occlusion allows to refine the proposed model described previously by taking into account the occluded and the non-occluded information. A deep experimentation demonstrates the efficiency of the described modeling. The estimated binocular energy presents a correlation with the impairment level caused by compression or noise.

Stereo image coding based on binocular energy modeling

Stereoscopic imaging technologies are seen as the next generation of visual presentation, improving the quality of experience of the viewer. It uses two different sequences acquired from two regular cameras or from a regular camera with an additional specific depth camera. This means that the size of data is at least doubled. Thus, the coding process becomes very crucial. In this framework, we propose a stereoscopic coder based on visual properties. The matching of two images is computed by a binocular energy model based on the simple and complex cells functions allowing the fusion of both retinal images in the visual cortex. Mathematical functions were used to reproduce the behavior of these cells particularly complex wavelet transform (CWT) and bandelet transform. Our coder output is a disparity map, a residual image and the reference image. The innovative part of this work lies in a matching technique based on the binocular energy. The results are presented in comparative curves with one of the most known coder in literature.

A stereoscopic quality metric based on binocular perception

This paper presents a new metric for the quality evaluation of stereoscopic images. The proposed metric is inspired from the simple and complex cells response, in which the matching of the left and right retinal images is performed, in the visual cortex. In order to estimate the binocular energy, we used the bandelet transform which is computed on complex wavelet transform. By using the complex wavelet transform, we were able to model the behavior of the simple and complex cells. The proposed model allows the estimation of the degradation of the binocular energy according to the impairments of the right and the left images. Subjective tests have been performed to validate the results obtained by our metric. The results shows a high correlation between the predicted score and the human judgement.

Optimizing the disparity map by the integration of HVS binocular properties for efficient coding of stereoscopic images

Stereoscopic imaging technologies are seen as the next generation of visual presentation, improving the quality of experience of the viewer. It uses two different sequences acquired from two regular cameras or from a regular camera with an additional specific depth camera. This means that the size of data is at least doubled. Thus, the coding process becomes very crucial. In this framework, we propose a stereoscopic coder based on visual properties. The matching of two images is computed by a binocular energy model based on the simple and complex cells functions allowing the fusion of both retinal images in the visual cortex. Mathematical functions were used to reproduce the behavior of these cells particularly complex wavelet transform (CWT) and bandelet transform. Our coder output is a disparity map, a residual image and the reference image. The innovative part of this work lies in a matching technique based on the binocular energy. The results are presented through comparative curves with one of the most known coder in literature.

Using the HVS binocular properties for the construction of a comprehensive stereoscopic matching model

3D perception consists in receiving two dissimilar retinal images on both eyes. These retinal images undergo pretreatments before arriving at the visual cortex to be merged. In this paper we propose an approach which redraws this pretreatment process. A model for computing the binocular energy is proposed. This model is based on the simple cells and the complex cells properties, which are the source of binocular fusion in the visual cortex. To refine the obtained matching, a set of stereoscopic constraints are integrated into the model, to filter the information as it is made at the lateral geniculate body (LGB). This modeling also allows us to detect the regions affected by the binocular rivalry phenomenon. The obtained results can be exploited by the stereoscopic vision applications, such as the 3D reconstruction, the stereoscopic coding, stereoscopic quality assessment and other applications.

A Stereo Image Coding Using Wavelet Based Quadtree and Binocular Rivalry

In this work, a new stereo image coding technique is proposed. The encoder takes into account the case of relatively different images, by reproducing the phenomenon of binocular rivalry. This approach integrates residual image coding with the disparity map. The latter is computed in the wavelet transform domain. The motivation behind using this transform lies in its similarity with some proprieties of the human visual system (HVS); Particularly, the decomposition in perceptual channels. Therefore, the usage of wavelet transform allows better preservation of visual quality. In order to estimate the disparity map, we used quadtree segmentation in each wavelet frequency band. This segmentation has the advantage of minimizing the entropy, and will allow to match the reference (left) image and the target (right) image. Dyadic squares in the subbands generate the residual image coded by using an arithmetic coder. The results, of the proposed method, are evaluated in comparison to state of the art methods, using the PSNR criteria.

A wavelet-based quadtree driven stereo image coding

In this work, a new stereo image coding technique is proposed. The new approach integrates the coding of the residual image with the disparity map. The latter computed in the wavelet transform domain. The motivation behind using this transform is that it imitates some properties of the human visual system (HVS), particularly, the decomposition in the perspective canals. Therefore, using the wavelet transform allows for better perceptual image quality preservation. In order to estimate the disparity map, we used a quadtree segmentation in each wavelet frequency band. This segmentation has the advantage of minimizing the entropy. Dyadic squares in the subbands of target image that they are not matched with other in the reference image constitutes the residuals are coded by using an arithmetic codec. The obtained results are evaluated by using the SSIM and PSNR criteria.

Stereo image coding based on the binocular compensation/suppression

In this work, a wavelet-based coding scheme for stereoscopic images is presented. The proposed scheme effectively integrates the coding of the disparity map in addition to the reference image. Hence, it first calculates the wavelet transform of the left and right images. A process for detecting an optimal scale for the Discrete Wavelet Transform (DWT) is then applied. The sub-bands of the last retained level of the DWT are then segmented into quadtree. The disparity map is obtained by studying the similarity between the dyadic squares belonging to the left image and those of the right image. This is achieved by minimizing an energy function defined on the spatial properties of the dyadic squares. Thus, the binocular compensation/suppression process allows at this stage to take advantage of the gap existing generally between the two views. Finally, the coefficients of the reference image, the residual image and the disparity map are encoded using a quantization step followed by an entropic coding. Of course, the target image is predicted using the aforementioned content. The experimental results of the proposed approach shows a good performance compared to the literature.