Patrick Lopez

View synthesis techniques for 3D video

To facilitate new video applications such as three-dimensional video (3DV) and free-viewpoint video (FVV), multiple view plus depth format (MVD), which consists of both video views and the corresponding per-pixel depth images, is being investigated. Virtual views can be generated using depth image based rendering (DIBR), which takes video and the corresponding depth images as input. This paper discusses view synthesis techniques based on DIBR, which includes forward warping, blending and hole filling. Especially, we will emphasize on the techniques brought to the MPEG view synthesis reference software (VSRS). Unlike the case in the field of computer graphics, the ground truth depth images for nature content are very difficult to obtain. The estimated depth images used for view synthesis typically contain different types of noises. Some robust synthesis modes to combat against the depth errors are also presented in this paper. In addition, we briefly discuss how to use synthesis techniques with minor modifications to generate the occlusion layer information for layered depth video (LDV) data, which is another potential format for 3DV applications.

Adaptive 3D Rendering based on Region-of-Interest

3D processing techniques are really promising. However, several hurdles have to be overcome. In this paper, two of them are examined. The first is related to the high disparity management. It is currently not well mastered and its impact is strong for viewing 3D scene on stereoscopic screens. The second concerns the salient regions of the scene. These areas are commonly called Region-Of-Interest (RoI) in the image processing domain. The problem appears when there are more than one region-of-interest in a video scene. Indeed, it is then complicated for the eyes to scan them and especially if the depth difference between them is high. In this contribution, the 3D experience is improved by applying some effects related to RoIs. The shift between the two views is adaptively adjusted in order to have a null disparity on a given area in the scene. In the proposed approach, these areas are the visually interesting areas. A constant disparity on the salient areas improves the viewing experience over the video sequence.

Depth map processing with iterative joint multilateral filtering

Depth maps estimated using stereo matching between frames from different video views typically exhibit false contours and noisy artifacts around object boundaries. In this paper, iterative joint multilateral filtering is proposed to deal with these artifacts. The proposed filter consists of multiple filter kernels. Knowing that the estimated depth maps are erroneous, besides the kernels which measure the proximity of depth samples and the similarity between depth sample values, we further develop kernels which measure similarity between the corresponding video pixel values. To increase reliability, these novel kernels operate on the color (RGB) domain instead of only on the luminance domain. Furthermore, the filter shapes are designed to adapt brightness variations. Finally, to tackle large misalignment between boundaries in depth maps and in the corresponding video frames, iterative approach is utilized. Our results demonstrate that the proposed method can significantly improve the boundaries in depth maps and can reduce false contours. With the processed depth maps, it is observed that the quality of object boundaries in synthesized views can be improved.

Flexible stream switching based on HEVC

This paper presents a flexible stream switching technique for HEVC. The proposed scheme allows to switch at any frame type and to increase the quality of the reconstructed video compared to basic stream switching. Additional information is inserted and encoded using a subset of SHVC coding modes. This technique maintains a low insertion rate of IDR/CRA pictures. Experimental results show that the overall performance is improved significantly.

HDTV video compression for professional studio applications

This paper addresses the issue of video compression within professional studio environments. In particular, the study concerns the degradation of the video quality in the case of successive encoding/decoding operations in the presence of pixel shifting to reflect some processing that may occur during the production workflow.