Payman Aflaki

Subjective study on compressed asymmetric stereoscopic video

Asymmetric stereoscopic video coding takes advantage of the binocular suppression of the human vision by representing one of the views with a lower quality. This paper describes a subjective quality test with asymmetric stereoscopic video. Different options for achieving compressed mixed-quality and mixed-resolution asymmetric stereo video were studied and compared to symmetric stereo video. The bitstreams for different coding arrangements were simulcast-coded according to the Advanced Video Coding (H.264/AVC) standard. The results showed that in most cases, resolution-asymmetric stereo video with the downsampling ratio of 1/2 along both coordinate axes provided similar quality as symmetric and quality-asymmetric full-resolution stereo video. These results were achieved under same bitrate constrain while the processing complexity decreased considerably. Moreover, in all test cases, the symmetric and mixed-quality full-resolution stereoscopic video bitstreams resulted in a similar quality at the same bitrates.

Multiview-Video-Plus-Depth Coding Based on the Advanced Video Coding Standard

This paper presents a multiview-video-plus-depth coding scheme, which is compatible with the advanced video coding (H.264/AVC) standard and its multiview video coding (MVC) extension. This scheme introduces several encoding and in-loop coding tools for depth and texture video coding, such as depth-based texture motion vector prediction, depth-range-based weighted prediction, joint inter-view depth filtering, and gradual view refresh. The presented coding scheme is submitted to the 3D video coding (3DV) call for proposals (CfP) of the Moving Picture Experts Group standardization committee. When measured with commonly used objective metrics against the MVC anchor, the proposed scheme provides an average bitrate reduction of 26% and 35% for the 3DV CfP test scenarios with two and three views, respectively. The observed bitrate reduction is similar according to an analysis of the results obtained for the subjective tests on the 3DV CfP submissions.

Impact of downsampling ratio in mixed-resolution stereoscopic video

In mixed-resolution (MR) stereoscopic video, one of the views has a lower resolution compared to the other one, hence providing means for improved compression. The underlying assumption in MR stereoscopic video is that the human visual system fuses the left and right views in such a way that the perceived image quality is close to that of the higher-resolution view. This paper describes a subjective quality evaluation experiment with uncompressed MR stereoscopic video, in which the aim was to discover the approximate limit of the downsampling ratio where the perceived quality is still close to the quality of the higher view. Different downsampling ratios, namely 1/2, 3/8, and 1/4 along both coordinate axes, were tested. The threshold of the higher-resolution view being dominant in the perceived quality lied in between downsampling ratios 1/2 and 3/8, corresponding to 11.4 and 7.6 pixels per degree of viewing angle, respectively. At downsampling ratios 3/8 and 1/4, the perceived quality linearly correlated with the average luma peak signal-to-noise ratio of the lower-resolution view.

Nonlinear Depth Map Resampling for Depth-Enhanced 3-D Video Coding

Depth-enhanced 3-D video coding includes coding of texture views and associated depth maps. It has been observed that coding of depth map at reduced resolution provides better rate-distortion performance on synthesized views comparing to utilization of full resolution (FR) depth maps in many coding scenarios based on the Advanced Video Coding (H.264/AVC) standard. Conventional techniques for down and upsampling do not take typical characteristics of depth maps, such as distinct edges and smooth regions within depth objects, into account. Hence, more efficient down and upsampling tools, capable of preserving edges better, are needed. In this letter, novel non-linear methods to down and upsample depth maps are presented. Bitrate comparison of synthesized views, including texture and depth map bitstreams, is presented against a conventional linear resampling algorithm. Objective results show an average bitrate reduction of 5.29% and 3.31% for the proposed down and upsampling methods with ratio ½, respectively, comparing to the anchor method. Moreover, a joint utilization of the proposed down and upsampling brings up to 20% and on average 7.35% bitrate reduction.

Multiview depth coding based on combined color/depth segmentation

In this paper, a new coding method for multiview depth video is presented. Considering the smooth structure and sharp edges of depth maps, a segmentation based approach is proposed. This allows further preserving the depth contours thus introducing fewer artifacts in the depth perception of the video. To reduce the cost associated with partition coding, an approximation of the depth partition is built using the decoded color view segmentation. This approximation is refined by sending some complementary information about the relevant differences between color and depth partitions. For coding the depth content of each region, a decomposition into orthogonal basis is used in this paper although similar decompositions may be also employed. Experimental results show that the proposed segmentation based depth coding method outperforms H.264/AVC and H.264/MVC by more than 2dB at similar bitrates.

Coding of mixed-resolution multiview video in 3D video application

The emerging MVC+D standard specifies the coding of Multiview Video plus Depth (MVD) data for enabling advanced 3D video applications. MVC+D specifications define the coding of all views of MVD at equal spatial resolution and apply a conventional MVC technique for coding the multiview texture and the depth independently. This paper presents a modified MVC+D coding scheme, where only the base view is coded at the original resolution whereas dependent views are coded at reduced resolution. To enable inter-view prediction, the base view is downsampled within the MVC coding loop to provide a relevant reference for dependent views. At the decoder side, the proposed scheme consists of a post-processing scheme which upsamples of the decoded views to their original resolution. The proposed scheme is compared against the original MVC+D scheme and an average of 4% delta bitrate reduction (dBR) in the coded views and 14.5% of dBR in the synthesized views are reported.

Flexible depth map spatial resolution in depth-enhanced multiview video coding

Multiview video plus depth (MVD) has proved to be a promising format enabling various 3D applications. One approach to achieve better MVD compression is to adjust the spatial resolution of depth map based on the content and the application. In this research two schemes are considered: first, multiview video coding accompanied by depth maps to improve the texture coding performance. Second, multiview video plus depth (MVD) coding targeting highest quality for synthesized views. Two algorithms to select the best spatial resolution for each scheme are proposed and the results show 10.8% and 16.5% bitrate reduction compared to the anchor case where depth map resolution is fixed.

Evaluation of Depth-Based Super Resolution on Compressed Mixed Resolution 3D Video

The MVC+D standard specifies coding of Multiview Video plus Depth (MVD) data for enabling advanced 3D video applications. MVC+D defines that all views are coded with H.264/MVC encoder at equal spatial resolution. To improve compression efficiency it is possible to use mixed resolution coding in which part of texture views are coded at reduced spatial resolution. In this paper we evaluate the performance of Depth-Based Super Resolution (DBSR) on compressed mixed resolution MVD data. Experimental results show that for sequences with accurate depth data the objective coding performance metric increases. Even though some sequences, with poor depth quality, show slight decrease in coding performance with respect to objective metric, subjective evaluation shows that perceived quality of DBSR method is equal to symmetric resolution case. We also show that depth re-projection consistency check step of the DBSR can be changed to simpler consistency check method. In this way the DBSR computational complexity is reduced by 26 % with 0.2 % dBR average bitrate reduction for coded views and 0.1 % average bitrate increase for synthesized views. We show that proposed scheme outperforms the anchor MVC+D coding scheme by 7.2 % of dBR on average for total coded bitrate and by 10.9 % of dBR on average for synthesized views.

Estimation of subjective quality for mixed-resolution stereoscopic video

In mixed-resolution (MR) stereoscopic video, one view is presented with a lower resolution compared with the other one; therefore, a lower bitrate, a reduced computational complexity, and a decrease in memory access bandwidth can be expected in coding. The human visual system is known to fuse left and right views in such a way that the perceptual visual quality is closer to that of the higher-resolution view. In this paper, a subjective assessment of mixed resolution (MR) stereoscopic videos is presented and the results are analyzed and compared with previous subjective tests presented in the literature. Three downsampling ratios 1/2, 3/8, and 1/4 were used to create lower-resolution views. Hence, the lower-resolution view had different spatial resolutions in terms of pixels per degree (PPD) for each downsampling ratio. It was discovered that the subjective viewing experience tended to follow a logarithmic function of the spatial resolution of the lower-resolution view measured in PPD. A similar behavior was also found from the results of an earlier experiment. Thus, the results suggest that the presented logarithmic function characterizes the expected viewing experience of MR stereoscopic video.

Study of asymmetric quality between coded views in depth-enhanced multiview video coding

Depth-enhanced multiview video formats, such as the multiview video plus depth (MVD) format, enable a natural 3D visual experience which cannot be brought by traditional 2D or stereo video services. In this paper we studied an asymmetric MVD technique for coding of three views that enabled rendering of the same bitstream on stereoscopic displays and multiview autostereoscopic displays. A larger share of bitrate was allocated to a central view, whereas two side views were coded at lower quality. The three decoded views were used by a Depth-Image-Based Rendering algorithm (DIBR) to produce virtual intermediate views. A stereopair at a suitable separation for viewing on a stereoscopic display was selected among the synthesized views. A large-scale subjective assessment of the selected synthesized stereopair was performed. A bitrate reduction of 20% on average and up to 22% was achieved with no penalties on subjective perceived quality. In addition, our analysis shows that a similar bitrate reduction gain with no difference in subjective quality can be achieved in multiview autostereoscopic display scenario.