Nukhet Ozbek

Scalable Multi-View Video Coding for Interactive 3DTV

A standard for scalable video coding (SVC) is currently being worked on by the ISO MPEG group. Work on standardization of multiple-view video coding (MVC) has also recently started under the ISO MPEG. Although there are many approaches published on SVC and MVC, there is no current work reported on scalable multi-view video coding (SMVC). This paper presents new coding structures for scalable stereo and multi-view video coding. The proposed structures are implemented as extensions to the JSVM software and resulting bitrates and PSNR are demonstrated. SMVC can be used for transport of multiview video over IP for interactive 3DTV by dynamic adaptive combination of temporal, spatial, and SNR scalability according to network conditions

Rate Allocation Between Views in Scalable Stereo Video Coding using an Objective Stereo Video Quality Measure

It is well-known that in stereoscopic 3D video systems humans perceive good quality 3D video as long as one of the eyes sees a high quality view. Hence, in stereo video encoding/streaming, best rate allocation between views can be addressed by reduction of the spatial resolution, frame rate, and/or quantization parameter of the second view with respect to the first view. In this paper, we address selection of the rate allocation strategy between views for our recently developed scalable multi-view video codec (SMVC) (N. Ozbek and A. M. Tekalp, 2006) to obtain the best rate-distortion performance. Since 3D video quality perception does not correlate well with the overall PSNR of the two views, we propose a new quantitative measure for stereo video quality as weighted combination of two PSNR values and a jerkiness measure. The weights are determined by means of correlating subjective quality test results and the objective measure scores on a set of test videos. DSCQS test methodology is used for subjective evaluation of stereo videos. Experimental results are presented to demonstrate how the objective and subjective 3D video quality varies for different choices of rate allocation between the views.

Unequal inter-view rate allocation using scalable stereo video coding and an objective stereo video quality measure

In stereoscopic 3D video, it is well-known that humans can perceive high quality 3D video provided that one of the views is in high quality. Hence, in stereo video encoding, the best overall rate vs. perceived-distortion performance may be achieved by reduction of the spatial, temporal, and/or quantization resolution of the second view, while keeping the first view in full resolution. In this paper, we address the best selection of unequal inter-view rate allocation strategy depending on the content of the video for a scalable multi-view video codec (SMVC) (Ozbek et al., 2006). Since the perceived 3D video quality does not correlate well with the average PSNR of the two views, we propose a new quantitative measure using a weighted combination of two PSNR values and a jerkiness measure. We verified that unequal rate allocation between the left and right views results in better perceived stereo video quality.

Adaptive Streaming Of Scalable Stereoscopic Video Over DCCP

We propose a new adaptive streaming model that utilizes DCCP in order to efficiently stream multi-view video over the Internet for 3DTV transport. The model effectively allocates the available channel bandwidth, which is calculated by DCCP, among the views. The video rate is adapted to the DCCP rate for each group of pictures (GoP) by adaptive extraction of layers from a scalable multi-view bitstream. The objective of the streaming model is to maximize perceived quality of the received 3D video while minimizing the number of possible display interrupts. Experimental results successfully demonstrate stereo video streaming over DCCP on wide area network.

Robust Quality Adaptation for Internet Video Streaming

Internet video streaming is a widely popular application however, in many cases, congestion control facilities are not well integrated into such applications. In order to be fair to other users that do not stream video, rate adaptation should be performed to respond to congestion. On the other hand, the effect of rate adaptation on the viewer should be minimized and this extra mechanism should not overload the client and the server. In this paper, we develop a heuristic approach for unicast congestion control. The primary feature of our approach is the two level adaptation algorithm that utilizes packet loss rate as well as receiver buffer data to maintain satisfactory buffer levels at the receiver. This is particularly important if receiver has limited buffer such as in mobile devices. When there is no congestion, to maintain best buffer levels, fine grain adjustments are carried out at the packet level. Depending on the level of congestion and receiver buffer level, rate shaping that involves frame discard and finally rate adaptation by switching to a different pre-encoded video stream are carried out. Additive increase multiplicative decrease policy is maintained to respond to congestion in a TCP- friendly manner. The algorithm is implemented and performance results show that it has adaptation ability that is suitable for both local area and wide area networks.

A New Scalable Multi-View Video Coding Configuration for Robust Selective Streaming of Free-Viewpoint TV

Free viewpoint TV (FTV) is a new media format that allows a user to change his/her viewpoint freely. To this effect, multi-view video must be coded to satisfy two conflicting requirements: (i) achieve high compression efficiency, and (ii) allow view switching with low delay. This paper proposes a new encoding configuration for scalable multi-view video coding, which achieves a compromise between the two requirements. In the new scalable multi-view configuration, the base layer is encoded with inter-view prediction at a minimum acceptable quality, while enhancement layers for each view only depend on their respective base layers (with no interview prediction). Thus, the base layer shall be served to all users, while enhancement layers shall be served selectively to users depending on their channel bandwidth and viewing direction. We compare the compression efficiency of the proposed method with those of non-scalable multi-view coding (MVC) and simulcast (H.264/AVC of each view independently) solutions.

Content-aware bit allocation in scalable multi-view video coding

We propose a new scalable multi-view video coding (SMVC) method with content-aware bit allocation among multiple views. The video is encoded off-line with a predetermined number of temporal and SNR scalability layers. Content-aware bit allocation among the views is performed during bitstream extraction by adaptive selection of the number of temporal and SNR scalability layers for each group of pictures (GOP) according to motion and spatial activity of that GOP. The effect of bit allocation among the multiple views on the overall video quality has been studied on a number of training sequences by means of both quantitative quality measures as well as qualitative visual tests. The number of temporal and SNR scalability layers selected as a function of motion and spatial activity measures for the actual test sequences are “learned” from these bit allocation vs. video quality studies on the training sequences. SMVC with content-aware bit allocation among views can be used for multi-view video transport over the Internet for interactive 3DTV. Experimental results are provided on stereo video sequences.

FAST H.264/AVC video encoding with multiple frame references

We focus on the question of how we can select the best multiple reference pictures for enhanced H.264 video encoding by a fast, computationally efficient method. We propose a simple histogram-similarity based method for selecting the best set of multiple reference pictures. Out-of-order coding of these frames is implemented by means of pyramid encoding. Experimental results show that the proposed approach can provide encoding time saving up to 23% with similar picture quality and bitrate for selected video sequences.

Adaptive Streaming of Scalable Stereoscopic Video Over DCCP

We propose a new adaptive streaming model that utilizes DCCP in order to efficiently stream stereoscopic video over the Internet for 3DTV transport. The model allocates the available channel bandwidth, which is calculated by the DCCP, among the views according to the suppression theory of human vision. The video rate is adapted to the DCCP rate for each group of pictures (GoP) by adaptive extraction of layers from a scalable multi-view bitstream. The objective of the streaming model is to maximize perceived quality of the received 3D video while minimizing the number of possible display interrupts. Experimental results successfully demonstrate stereo video streaming over DCCP on wide area network.

Rate adaptive video streaming under lossy network conditions

In this study, the performance of a rate adaptive video streaming algorithm is examined under controlled packet loss rates and delays. The developed algorithm is a set of heuristics that consider the packet loss rate and receiver buffer level during the adaptation decision. The algorithm is content aware in such a way that it employs quality or temporal scaling, or both, in accordance with the amount of motion in a scene. Extensive periodic and non-periodic packet loss scenarios are implemented to examine the behavior of the algorithm. It has been observed that the algorithm reacts to congestion by reducing its data rate and maintain interrupt-free display even if the continuous packet loss rate approaches 15%. The results of this study confirm the suitability of our algorithm for Internet video streaming where congestion can occur any time unpredictably.