Atanas Gotchev

Quality assessment of 3D video in rate allocation experiments

In this contribution, we address the problem of measuring and optimizing the visual quality of encoded 3D video. The 3D video is represented in the format of monoscopic color video augmented by per-pixel depth map and then encoded with H.264 encoder. To optimize the encoding performance we test different bit budgets for the color video and the depth and measure the quality by virtual view quality metrics. Small scale subjective tests supplement the objective measurements. The obtained results show that, for similar overall quality numbers, observers favorably trade off lower depth quality for higher color quality and that depth distortions are perceived but considered less significant than the color distortions.

3D-DCT based perceptual quality assessment of stereo video

In this paper, we present a novel stereoscopic video quality assessment method based on 3D-DCT transform. In our approach, similar blocks from left and right views of stereoscopic video frames are found by block-matching, grouped into 3D stack and then analyzed by 3D-DCT. Comparison between reference and distorted images are made in terms of MSE calculated within the 3D-DCT domain and modified to reflect the contrast sensitive function and luminance masking. We validate our quality assessment method using test videos annotated with results from subjective tests. The results show that the proposed algorithm outperforms current popular metrics over a wide range of distortion levels.

Classification and simulation of stereoscopic artifacts in mobile 3DTV content

We identify, categorize and simulate artifacts which might occur during delivery stereoscopic video to mobile devices. We consider the stages of 3D video delivery dataflow: content creation, conversion to the desired format (multiview or source-plus-depth), coding/decoding, transmission, and visualization on 3D display. Human 3D vision works by assessing various depth cues - accommodation, binocular depth cues, pictorial cues and motion parallax. As a consequence any artifact which modifies these cues impairs the quality of a 3D scene. The perceptibility of each artifact can be estimated through subjective tests. The material for such tests needs to contain various artifacts with different amounts of impairment. We present a system for simulation of these artifacts. The artifacts are organized in groups with similar origins, and each group is simulated by a block in a simulation channel. The channel introduces the following groups of artifacts: sensor limitations, geometric distortions caused by camera optics, spatial and temporal misalignments between video channels, spatial and temporal artifacts caused by coding, transmission losses, and visualization artifacts. For the case of source-plus-depth representation, artifacts caused by format conversion are added as well.

Three-Dimensional Media for Mobile Devices

This paper aims at providing an overview of the core technologies enabling the delivery of 3-D Media to next-generation mobile devices. To succeed in the design of the corresponding system, a profound knowledge about the human visual system and the visual cues that form the perception of depth, combined with understanding of the user requirements for designing user experience for mobile 3-D media, are required. These aspects are addressed first and related with the critical parts of the generic system within a novel user-centered research framework. Next-generation mobile devices are characterized through their portable 3-D displays, as those are considered critical for enabling a genuine 3-D experience on mobiles. Quality of 3-D content is emphasized as the most important factor for the adoption of the new technology. Quality is characterized through the most typical, 3-D-specific visual artifacts on portable 3-D displays and through subjective tests addressing the acceptance and satisfaction of different 3-D video representation, coding, and transmission methods. An emphasis is put on 3-D video broadcast over digital video broadcasting-handheld (DVB-H) in order to illustrate the importance of the joint source-channel optimization of 3-D video for its efficient compression and robust transmission over error-prone channels. The comparative results obtained identify the best coding and transmission approaches and enlighten the interaction between video quality and depth perception along with the influence of the context of media use. Finally, the paper speculates on the role and place of 3-D multimedia mobile devices in the future internet continuum involving the users in cocreation and refining of rich 3-D media content.

Relative importance of depth cues on portable autostereoscopic display

Depth of a three-dimensional scene is perceived using several different visual cues. The process of perceiving depth is delicate and vulnerable to artefacts. On autostereoscopic displays the aim is to enhance the sensation of reality of the experience by the added three-dimensionality without the need for aiding glasses. The factors that contribute to achieving a plausible 3D effect on autostereoscopic displays are numerous: for example stereoscopy combined with a supporting set of monocular cues: perspective lines, shadows, and textures.n The goal of this study is two-fold. First, to gather knowledge on how different characteristics of a 3D scene impact the correct estimation of depth. Second, how the same characteristics impact on perceived quality. These were studied using subjective tests. The studied characteristics were chosen to be five different depth cues - three monocular and two binocular cues. The chosen monocular cues were shadows, texture and focal depth. For binocular cues, two different stereoscopic disparities were used: one optimal for a portable 3D display, and another designed for a HD resolution and later scaled down for a portable device making the disparity decrease. In addition, combination of all aforementioned cues was examined. All of the cues were also studied with three different compression levels (no compression, JPEG with quality factor of 50, and with quality factor of 90).n A quantitative study using a portable autostereoscopic display in controlled laboratory environment was conducted. The results of this study indicate that the stereoscopic depth cues outperform the monocular depth cues in accuracy and speed of estimating depth in a 3D video. Interestingly enough, these results are not consistent with the ratings of perceived quality and acceptance. Furthermore, compressing images seems to have a significant effect on the accuracy and speed of depth estimation, and also on the perceived quality.

Comparative study of autostereoscopic displays for mobile devices

We perform comparative analysis of the visual quality of multiple 3D displays - seven portable ones, and a large 3D television set. We discuss two groups of parameters that influence the perceived quality of mobile 3D displays. The first group is related with the optical parameters of the displays, such as crosstalk or size of sweet spots. The second group includes content related parameters, such as objective and subjective comfort disparity range, suitable for a given display. We identify eight important parameters to be measured, and for each parameter we present the measurement methodology, and give comparative results for each display. Finally, we discuss the possibility of each display to visualize downscaled stereoscopic HD content with sufficient visual quality.

A study on the effect of MPE-FEC for 3D video broadcasting over DVB-H

Wireless networks are often error prone due to factors such as multi-path fading and interferences. In addition, the channel conditions of these networks are often non-stationary, such that the available bandwidth and channel error rates are changing over time with large variations. Therefore, they present a challenge for error-resilient video transmission. In order to maintain satisfactory QoS, a number of technologies have been proposed targeting different layers of the networks. DVB-H uses FEC for error protection and comes with an optional FEC tool (MPE-FEC) at the link-layer. In this paper, we propose to use the a priori knowledge of the transmitted media and apply MPE-FEC intelligently to provide better robustness by so-called Unequal Error Protection (UEP) mechanism. Extensive simulation results are given to show the effect of MPE-FEC on 3D compressed video under different channel conditions.

Mobile 3D television: development of core technological elements and user-centered evaluation methods toward an optimized system

A European consortium of six partners has been developing core technological components of a mobile 3D television system over DVB-H channel. In this overview paper, we present our current results on developing optimal methods for stereo-video content creation, coding and transmission and emphasize their significance for the power-constrained mobile platform, equipped with auto-stereoscopic display. We address the user requirements by applying modern usercentered approaches taking into account different user groups and usage contexts in contrast to the laboratory assessment methods which, though standardized, offer limited applicability to real applications. To this end, we have been aiming at developing a methodological framework for the whole system development process. One of our goals has been to further develop the user-centered approach towards experienced quality of critical system components. In this paper, we classify different research methods and technological solutions analyzing their pros and constraints. Based on this analysis we present the user-centered methodological framework being used throughout the whole development process of the system and aimed at achieving the best performance and quality appealing to the end user.

Comparative analysis of occlusion-filling techniques in depth image-based rendering for 3D videos

Due to the recent success of 3D cinema, 3D video has been gathering considerable interest and delivery of 3D video to home TVs and mobile devices has been actively researched. A broadcast-friendly 3D video delivery format is so-called view plus depth, where single video sequence is augmented by dense depth information. Based on it, desired views can be synthesized at the receiver side. While well susceptible for effective compression, this representation presents problems related with occluded areas which become visible in the synthesized views. Different occlusion filling techniques have been developed varying in quality and complexity. This contribution aims at providing an objective and subjective comparison of three such techniques. These include adaptive pre-filtering of depth maps, and one simplified and one high-performance inpainting techniques. The latter two have been specifically modified for the case of occlusion filling. We have aimed especially at ranking the performance in the case of mobile 3D imaging. Along with objective comparisons, results from subjective tests are presented as well. Both groups of results demonstrate the superiority of the second inpainting technique, which is however also the most computationally expensive one.

A modified non-local mean inpainting technique for occlusion filling in depth-image-based rendering

View plus depth is an attractive compact representation format for 3D video compression and transmission. It combines 2D video with depth map sequence aligned in a per-pixel manner to represent the moving 3D scene in interest. Any different-perspective view can be synthesized out if this representation through Depth-Image Based Rendering (DIBR). However, such rendering is prone to disocclusion errors: regions originally covered by foreground objects become visible in the synthesized view and have to be filled with perceptually-meaningful data. In this work, a technique for reducing the perceived artifacts by inpainting the disoccluded areas is proposed. Based on Criminisis exemplar-based inpainting algorithm, the developed technique recovers the disoccluded areas by using pixels of similar blocks surrounding it. In the original work, a moving window is centered on the boundaries between known and unknown parts (target window). The known pixels are used to select windows which are most similar to the target one. When this process is completed, the unknown region of the target patch is filled with a weighted combination of pixels from the selected windows. In the proposed scheme, the priority map, which defines the rule for selecting the order of pixels to be filled, has been modified to meet the requirement for disocclusion hole filling and a better non-local mean estimate has been suggested accordingly. Furthermore, the search for similar patches has also been extended to previous and following frames of the video under processing, thus improving both computational efficiency and resulting quality.