Gozde Bozdagi Akar

Towards compound stereo-video quality metric: a specific encoder-based framework

We suggest a compound full-reference stereo-video quality metric composed of two components: a monoscopic quality component and stereoscopic quality component. While the former assesses the trivial monoscopic perceived distortions caused by blur, noise, contrast change etc., the latter assesses the perceived degradation of binocular depth cues only. We use the structural similarity index as a measure for perceptual similarity and design a multiscale algorithm for obtaining a perceptual disparity map and a stereo-similarity map to be used in the suggested metric. We verify the performance of the metric with subjective tests on distorted stereo images and coded stereo-video sequences with a final aim to build a perceptually-aware feedback for a H.264 based stereo-video encoder

Face verification competition on the XM2VTS database

In the year 2000 a competition was organised to collect face verification results on an identical, publicly available data set using a standard evaluation protocol. The database used was the Xm2vts database along with the Lausanne protocol [14]. Four different institutions submitted results on the database which were subsequently published in [13]. Three years later, a second contest using the same dataset and protocol was organised as part of AVBPA 2003. This time round seven seperate institutions submitted results to the competition. This paper presents the results of the competition and shows that verification results on this protocol have increased in performance by a factor of 3.

End-to-end stereoscopic video streaming with content-adaptive rate and format control

We address efficient compression and real-time streaming of stereoscopic video over the current Internet. We first propose content-adaptive stereo video coding (CA-SC), where additional coding gain, over that can be achieved by exploiting only inter-view correlations, is targeted by down-sampling one of the views spatially or temporally depending on the content, based on the well-known theory that the human visual system can perceive high frequencies in three-dimensional (3D) from the higher quality view. We also developed stereoscopic 3D video streaming server and clients by modifying available open source platforms, where each client can view the video in mono or stereo mode depending on its display capabilities. The performance of the end-to-end stereoscopic streaming system is demonstrated using subjective quality tests.

Three-Dimensional Media for Mobile Devices

This paper aims at providing an overview of the core technologies enabling the delivery of 3-D Media to next-generation mobile devices. To succeed in the design of the corresponding system, a profound knowledge about the human visual system and the visual cues that form the perception of depth, combined with understanding of the user requirements for designing user experience for mobile 3-D media, are required. These aspects are addressed first and related with the critical parts of the generic system within a novel user-centered research framework. Next-generation mobile devices are characterized through their portable 3-D displays, as those are considered critical for enabling a genuine 3-D experience on mobiles. Quality of 3-D content is emphasized as the most important factor for the adoption of the new technology. Quality is characterized through the most typical, 3-D-specific visual artifacts on portable 3-D displays and through subjective tests addressing the acceptance and satisfaction of different 3-D video representation, coding, and transmission methods. An emphasis is put on 3-D video broadcast over digital video broadcasting-handheld (DVB-H) in order to illustrate the importance of the joint source-channel optimization of 3-D video for its efficient compression and robust transmission over error-prone channels. The comparative results obtained identify the best coding and transmission approaches and enlighten the interaction between video quality and depth perception along with the influence of the context of media use. Finally, the paper speculates on the role and place of 3-D multimedia mobile devices in the future internet continuum involving the users in cocreation and refining of rich 3-D media content.

Rate-distortion optimization for stereoscopic video streaming with unequal error protection

We consider an error-resilient stereoscopic streaming system that uses an H.264-based multiview video codec and a rateless Raptor code for recovery from packet losses. One aim of the present work is to suggest a heuristic methodology for modeling the end-to-end rate-distortion (RD) characteristic of such a system. Another aim is to show how to make use of such a model to optimally select the parameters of the video codec and the Raptor code to minimize the overall distortion. Specifically, the proposed system models the RD curve of video encoder and performance of channel codec to jointly derive the optimal encoder bit rates and unequal error protection (UEP) rates specific to the layered stereoscopic video streaming. We define analytical RD curve modeling for each layer that includes the interdependency of these layers. A heuristic analytical model of the performance of Raptor codes is also defined. Furthermore, the distortion on the stereoscopic video quality caused by packet losses is estimated. Finally, analytical models and estimated single-packet loss distortions are used to minimize the end-to-end distortion and to obtain optimal encoder bit rates and UEP rates. The simulation results clearly demonstrate the significant quality gain against the nonoptimized schemes.

A Multi-View Video Codec Based on H.264

H.264 is the current state-of-the-art monoscopic video codec providing almost twice the coding efficiency with the same quality comparing the previous codecs. With the increasing interest in 3D TV, multi-view video sequences that are provided by multiple cameras capturing the three dimensional objects and/or scene are more widely used. Compressing multi-view sequences independently with H.264 (simulcast) is not efficient since the redundancy between the closer cameras is not exploited. In order to reduce these redundancies, we propose a multi-view video codec based on H.264 using disparity estimation/compensation as well as motion estimation/compensation. In order to effectively search for disparity/motion without increasing computational complexity, we modified the buffering structure of H.264 and implemented several referencing modes. Our results show that for closely located cameras, our codec outperforms simulcast H.264 coding. For sparsely located cameras, our method can still improve coding gain depending on the video characteristics.

A simple and effective mechanism for stored video streaming with TCP transport and server-side adaptive frame discard

Transmission control protocol (TCP) with its well-established congestion control mechanism is the prevailing transport layer protocol for non-real time data in current Internet Protocol (IP) networks. It would be desirable to transmit any type of multimedia data using TCP in order to take advantage of the extensive operational experience behind TCP in the Internet. However, some features of TCP including retransmissions and variations in throughput and delay, although not catastrophic for non-real time data, may result in inefficiencies for video streaming applications. In this paper, we propose an architecture which consists of an input buffer at the server side, coupled with the congestion control mechanism of TCP at the transport layer, for efficiently streaming stored video in the best-effort Internet. The proposed buffer management scheme selectively discards low priority frames from its head-end, which otherwise would jeopardize the successful playout of high priority frames. Moreover, the proposed discarding policy is adaptive to changes in the bandwidth available to the video stream.

A standards-based, flexible, end-to-end multi-view video streaming architecture

In this paper we propose a novel framework for the streaming of 3-D representations in the form of Multi- View Videos (MVV). The proposed streaming system is completely standards based, flexible and backwards compatible in order to support monoscopic streaming to legacy clients. We demonstrate compatibility of the proposed system with various possible encoding schemes and operating scenarios. In the current implementation, the MVVs in the server are compressed using a simplified form of MVC with negligible loss of compression efficiency and streamed using Real Time Streaming Protocol (RTSP), Session Description Protocol (SDP) and Real Time Protocol (RTP) to the clients. We describe our extensions to SDP and discuss a preliminary RTP payload format for MVC. The clients in this implementation perform basic error concealment to reduce the effects of packet losses and decode MVC in near-real-time. The modular clients can display decoded 3-D content on a multitude of 3-D display systems.

Error Resilient Layered Stereoscopic Video Streaming

In this paper, error resilient stereoscopic video streaming problem is addressed. Two different forward error correction (FEC) codes namely systematic LT and RS codes are utilized to protect the stereoscopic video data against transmission errors. Initially, the stereoscopic video is categorized in 3 layers with different priorities. Then, a packetization scheme is used to increase the efficiency of error protection. A comparative analysis of RS and LT codes are provided via simulations to observe the optimum packetization and UEP strategies.

Schemes for multiple description coding of stereoscopic video

This paper presents and compares two multiple description schemes for coding of stereoscopic video, which are based on H.264. The SS-MDC scheme exploits spatial scaling of one view. In case of one channel failure, SS-MDC can reconstruct the stereoscopic video with one view low-pass filtered. SS-MDC can achieve low redundancy (less than 10%) for video sequences with lower inter-view correlation. MS-MDC method is based on multi-state coding and is beneficial for video sequences with higher inter-view correlation. The encoder can switch between these two methods depending on the characteristics of video.