Jérôme Fournier

Quality of experience model for 3DTV

Modern stereoscopic 3DTV brings new QoE (quality of experience) to viewers, which not only enhances the 3D sensation due to the added binocular depth, but may also induce new problems such as visual discomfort. Subjective quality assessment is the conventional method to assess the QoE. However, the conventional perceived image quality concept is not enough to reveal the advantages and the drawbacks of stereoscopic images in 3DTV. Higher-level concepts such as visual experience were proposed to represent the overall visual QoE for stereoscopic images. In this paper, both the higher-level concept quality indicator, i.e. visual experience and the basic level concepts quality indicators including image quality, depth quantity, and visual comfort are defined. We aim to explore 3D QoE by constructing the visual experience as a weight sum of image quality, depth quantity and visual comfort. Two experiments in which depth quantity and image quality are varied respectively are designed to validate this model. In the first experiment, the stimuli consist of three natural scenes and for each scene, there are four levels of perceived depth variation in terms of depth of focus: 0, 0.1, 0.2 and 0.3 diopters. In the second experiment, five levels of JPEG 2000 compression ratio, 0, 50, 100, 175 and 250 are used to represent the image quality variation. Subjective quality assessments based on the SAMVIQ method are used in both experiments to evaluate the subjects opinion in basic level quality indicators as well as the higher-level indicator. Statistical analysis of result reveals how the perceived depth and image quality variation affect different perceptual scales as well as the relationship between different quality aspects.

New stereoscopic video shooting rule based on stereoscopic distortion parameters and comfortable viewing zone

Human binocular depth perception, the most important element brought by 3DTV, is proved to be closely connected to not only the content acquisition (camera focal length, camera baseline and etc.) but also the viewing environment (viewing distance, screen size and etc.). Conventional 3D stereography rule in the literature usually consider the general viewing condition and basic human factors to guide the content acquisition, such as assuming human inter-pupil baseline as the maximum disparity. A lot of new elements or problems of stereoscopic viewing was not considered or precisely defined so that advanced shooting rule is needed to guarantee the overall quality of stereoscopic video. In this paper, we proposed a new stereoscopic video shooting rule considering two most important issues in 3DTV: stereoscopic distortion and comfortable viewing zone. Firstly, a mathematic model mapping the camera space to visualization space is established in order to geometrically estimate the stereoscopic depth distortion. Depth and shape distortion factors are defined and used to describe the stereoscopic distortion. Secondly, comfortable viewing zone (or Depth of focus) is considered to reduce the problem of visual discomfort and visual fatigue. The new shooting rule is to optimize the camera parameters (focal length, camera baseline and etc.) in order to control depth and shape distortion and also guarantee that the perceived scene is located in comfortable viewing zone as possible. However, in some scenarios, the above two conditions cannot be fulfill simultaneously, even sometimes contradict with each other so that priority should be decided. In this paper, experimental stereoscopic synthetic content generation with various sets of camera parameters and various sets of scenes representing different depth range are presented. Justification of the proposed new shooting rule is based on 3D concepts (depth rendering, visual comfort and visual experience) subjective video assessment. The results of this study will provide a new method to propose camera parameters based on management of new criteria (shape distortion and depth of focus) in order to produce optimized stereoscopic images and videos.

An investigation of visual selection priority of objects with texture and crossed and uncrossed disparities

The aim of this research is to understand the difference in visual attention to 2D and 3D content depending on texture and amount of depth. Two experiments were conducted using an eye-tracker and a 3DTV display. Collected fixation data were used to build saliency maps and to analyze the differences between 2D and 3D conditions. In the first experiment 51 observers participated in the test. Using scenes that contained objects with crossed disparity, it was discovered that such objects are the most salient, even if observers experience discomfort due to the high level of disparity. The goal of the second experiment is to decide whether depth is a determinative factor for visual attention. During the experiment, 28 observers watched the scenes that contained objects with crossed and uncrossed disparities. We evaluated features influencing the saliency of the objects in stereoscopic conditions by using contents with low-level visual features. With univariate tests of significance (MANOVA), it was detected that texture is more important than depth for selection of objects. Objects with crossed disparity are significantly more important for selection processes when compared to 2D. However, objects with uncrossed disparity have the same influence on visual attention as 2D objects. Analysis of eyemovements indicated that there is no difference in saccade length. Fixation durations were significantly higher in stereoscopic conditions for low-level stimuli than in 2D. We believe that these experiments can help to refine existing models of visual attention for 3D content.

How visual attention is modified by disparities and textures changes

The 3D image/video quality of experience is a multidimensional concept that depends on 2D image quality, depth quantity and visual comfort. The relationship between these parameters is not yet clearly defined. From this perspective, we aim to understand how texture complexity, depth quantity and visual comfort influence the way people observe 3D content in comparison with 2D. Six scenes with different structural parameters were generated using Blender software. For these six scenes, the following parameters were modified: texture complexity and the amount of depth changing the camera baseline and the convergence distance at the shooting side. Our study was conducted using an eye-tracker and a 3DTV display. During the eye-tracking experiment, each observer freely examined images with different depth levels and texture complexities. To avoid memory bias, we ensured that each observer had only seen scene content once. Collected fixation data were used to build saliency maps and to analyze differences between 2D and 3D conditions. Our results show that the introduction of disparity shortened saccade length; however fixation durations remained unaffected. An analysis of the saliency maps did not reveal any differences between 2D and 3D conditions for the viewing duration of 20 s. When the whole period was divided into smaller intervals, we found that for the first 4 s the introduced disparity was conducive to the section of saliency regions. However, this contribution is quite minimal if the correlation between saliency maps is analyzed. Nevertheless, we did not find that discomfort (comfort) had any influence on visual attention. We believe that existing metrics and methods are depth insensitive and do not reveal such differences. Based on the analysis of heat maps and paired t-tests of inter-observer visual congruency values we deduced that the selected areas of interest depend on texture complexities.

How perception of ultra-high definition is modified by viewing distance and screen size

Ultra High Definition (UHD) is a new technology, which main idea is to improve user’s perception of details and sensation of immersion in comparison with High Definition systems (HD). However, it is important to understand the influence of the new UHD technical parameters on user’s perception. Hence, to investigate the influence of the viewing distance, screen size and scene content on perceived video quality and feelings of users, a series of subjective experiments with four different contents (3 documentaries and 1 sport content) shooted by UHD camera were performed. These contents were displayed using three different image resolutions (SD, HD, UHD) and two UHD displays (55-inch and 84-inch). Each subject had to assess content for three different viewing distances (1.5, 3, 4.5 times of the screen height corresponding to optimal viewing distances of respectively UHD, HD, and close to SD optimal distance). Finally, 72 test conditions were evaluated. For each scene, observers reported their opinion on the perceived video quality using a 5-grade subjective scale. Results have shown that viewing distance has a significant influence on perceived quality. Moreover the highest MOS was obtained at optimal viewing for UHD, with a small difference between HD an UHD. At 3H and 4.5H, there is no difference from a statistical point of view. Screen size influences the perception of quality but not in the same way for the three image resolution and three viewing distances.

A new video coder powered with second-generation wavelets

In this paper we propose a new method based on Second Generation Wavelets (SGW), a recently emerged mathematic transform, already successfully applied in 3d coding. This new wavelet transform (SGW) applied to meshes is performed using the powerful lifting scheme, keeping the main features of the first generation wavelets whilst introducing some useful properties such as: analysis on general domains (typically arbitrary, non linear domains of Rn), adapted basis for weighted approximation/interpolation and weighted measure and transform adapted to irregular sampled data. Thus in this paper we propose a video coder based on second generation wavelet theory and mesh geometry. In fact, we focus on video coder error images which are mainly composed of high frequencies and singularities, i.e. typically the kind of data that second generation wavelet can successfully process. The main advantage is that these wavelets can be designed to fit exactly on singularities, shapes, textures, and edges. Hence, we can further reduce redundancy and improve coding efficiency over those peculiar settings.

A new video coder based on second generation wavelets

We consider a new way to encode video sequences. The proposed method is based on second generation wavelets (SOW), a novel mathematic transform, successfully applied in 3D coding. This video coder is created by combining the wavelet theory and mesh geometry. With this and the SOW, we will be able to apply onto the video image, the mathematical transform to locate and to encode the error, results of the motion compensation process. This powerful signal processing theory allows a very well adapted coding of high frequencies. The main advantage is that these wavelets can be designed to fit exactly on singularities, shapes, textures, and edges. Hence, we can reduce redundancy and improve considerably coding efficiency over those peculiar settings. This new coder can be applied on all classical applications, such as very low bit rate transmissions, video streaming, vision conference and video on demand.