Engin Kurutepe

Client-Driven Selective Streaming of Multiview Video for Interactive 3DTV

We present a novel client-driven multiview video streaming system that allows a user to watch 3D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his/her head position. The users head position is tracked and predicted into the future to select the views that best match the users current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be prefetched in order to account for delays due to network transport and stream switching. The system allocates more bandwidth to the selected views in order to render the current viewing angle. Highly compressed, lower quality versions of some other views are also prefetched for concealment if the current user viewpoint differs from the predicted viewpoint. An objective measure based on the abruptness of the head movements and delays in the system is introduced to determine the number of additional lower quality views to be prefetched. The proposed system makes use of multiview coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compression efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions.

Subjective evaluation of scalable video coding for content distribution

This paper investigates the influence of the combination of the scalability parameters in scalable video coding (SVC) schemes on the subjective visual quality. We aim at providing guidelines for an adaptation strategy of SVC that can select the optimal scalability options for resource-constrained networks. Extensive subjective tests are conducted by using two different scalable video codecs and high definition contents. The results are analyzed with respect to five dimensions, namely, codec, content, spatial resolution, temporal resolution, and frame quality.

End-to-end stereoscopic video streaming with content-adaptive rate and format control

We address efficient compression and real-time streaming of stereoscopic video over the current Internet. We first propose content-adaptive stereo video coding (CA-SC), where additional coding gain, over that can be achieved by exploiting only inter-view correlations, is targeted by down-sampling one of the views spatially or temporally depending on the content, based on the well-known theory that the human visual system can perceive high frequencies in three-dimensional (3D) from the higher quality view. We also developed stereoscopic 3D video streaming server and clients by modifying available open source platforms, where each client can view the video in mono or stereo mode depending on its display capabilities. The performance of the end-to-end stereoscopic streaming system is demonstrated using subjective quality tests.

A standards-based, flexible, end-to-end multi-view video streaming architecture

In this paper we propose a novel framework for the streaming of 3-D representations in the form of Multi- View Videos (MVV). The proposed streaming system is completely standards based, flexible and backwards compatible in order to support monoscopic streaming to legacy clients. We demonstrate compatibility of the proposed system with various possible encoding schemes and operating scenarios. In the current implementation, the MVVs in the server are compressed using a simplified form of MVC with negligible loss of compression efficiency and streamed using Real Time Streaming Protocol (RTSP), Session Description Protocol (SDP) and Real Time Protocol (RTP) to the clients. We describe our extensions to SDP and discuss a preliminary RTP payload format for MVC. The clients in this implementation perform basic error concealment to reduce the effects of packet losses and decode MVC in near-real-time. The modular clients can display decoded 3-D content on a multitude of 3-D display systems.

Feasibility of Multi-View Video Streaming Over P2P Networks

We propose to stream multi-view video over a multi-tree peer- to-peer (P2P) network using the NUEPMuT protocol. Each view of the multi-view video is streamed over an independent P2P streaming tree and each peer only contributes upload capacity in a single tree, in order to limit the adverse effects of ungraceful peer departures. Additionally, we investigate the feasibility of using the proposed P2P networking architecture, NUEPMuT, for the streaming of multi-view video content with the currently available Internet connection bandwidths.

Multi-view video streaming over P2P networks with low start-up delay

We propose to stream multi-view video over a multi-tree peer- to-peer (P2P) network using the NUEPMuT protocol. Each view of the multi-view video is streamed over an independent P2P streaming tree and each peer only contributes upload capacity in a single tree, in order to limit the adverse effects of ungraceful peer departures. Additionally, we introduce a quick join procedure to reduce the start-up delay for the first data packet after a join request. Continuity index and decoded video quality performance for simulcast and MVC encoding in a large topology under different settings are reported, in addition to the improvements achieved by the quick join procedure.

A receiver-driven multicast framework for 3DTV transmission

Contemporary television and video experience is not interactive and users have little or no choice over their viewing angle in the scenes they watch. There is a demand for a real 3-D interactive experience which would allow users to view scenes through virtual cameras chosen by their head and eye locations as in real life. However, among other issues the amount of bandwidth required to transmit very large Image Based Rendering (IBR) representations of the scene to end users is still an unsolved problem. In this paper we propose a novel networking scheme to enable users to automatically stream only the parts of the light field representation, which will be used to render the current viewpoint. The proposed system also incorporates prediction of future views to prefetch streams, which are likely to be needed in the near future as the viewpoint changes over time.

Selective Streaming of Multi-View Video for Head-Tracking 3D Displays

We present a novel client-driven multi-view video streaming system that allows a user watch 3-D video interactively with significantly reduced bandwidth requirements by transmitting a small number of views selected according to his/her head position. The proposed scheme can be used to efficiently stream a dense set of multi-view sequences (light-fields) or wider baseline multi-view sequences together with depth information. The users head position is tracked and predicted into the future to select the views that best match the users current viewing angle dynamically. Prediction of future head positions is needed so that views matching the predicted head positions can be requested from the server ahead of time in order to account for delays due to network transport and stream switching. Highly compressed, lower quality versions of some other views are also requested in order to provide protection against having to display the wrong view when the current user viewpoint differs from the predicted viewpoint. The proposed system makes use of multi-view coding (MVC) and scalable video coding (SVC) concepts together to obtain improved compress ion efficiency while providing flexibility in bandwidth allocation to the selected views. Rate-distortion performance of the proposed system is demonstrated under different experimental conditions.

Interactive multi-view video delivery with view-point tracking and fast stream switching

We present a 3-D Multi-view video delivery system where each user receives only the streams required for rendering their viewpoint. This paper proposes a novel method to alleviate adverse effects of the unavoidable delay between the time a client requests a new stream and the time it becomes available. To this effect, lower bit-rate versions of a set of adjacent views are also streamed to the viewer in addition to the currently required views. This ensures that the viewer has a low quality version of a view ready and decodable when an unpredicted viewpoint change occurs until the high quality stream arrives. Bandwidth implications and PSNR improvements are reported for various low quality streams encoded at different bit-rates. Performance comparisons of the proposed system with respect to transmitting all views using MVC and only two views with no low quality neighbors are presented.

Resampling of multiple camera point cloud data

Even though the sampling grid of a digital camera is uniform and rectangular, the depth information reconstructed from multi-camera samples is neither uniform nor rectangular due to the rotations and translations of the camera coordinate systems with respect to each other. In order to facilitate further processing, e.g. compression of multi camera depth data, resampling of non-uniform multi-camera depth samples to a uniform rectangular grid is advantageous. In this paper, we introduce and compare two resampling methods using a voxel-based and a triangular-mesh-based approach.