Jacob Chakareski

Layered coding vs. multiple descriptions for video streaming over multiple paths

In this paper, we examine the performance of specific implementations of multiple description coding and of layered coding for video streaming over error-prone packet switched networks. We compare their performance using different transmission schemes with and without network path diversity. It is shown that given the specific implementations there is a large variation in relative performance between multiple description coding and layered coding depending on the employed transmission scheme. For scenarios where the packet transmission schedules can be optimized in a rate-distortion sense, layered coding provides a better performance. The converse is true for scenarios where the packet schedules are not rate-distortion optimized.

Rate-distortion optimized packet scheduling and routing for media streaming with path diversity

The diversity for media streaming in a rate-distortion optimization framework was considered. A sender-driven transmission scenario was also investigated. Diversity was achieved by using multiple transmission paths over the network. The proposed framework enables the sender to decide at every instant which packets, if any, to transmit and over which transmission paths in order to meet a rate constraint while minimizing the end-to-end distortion. Experimental results demonstrate the benefit of exploiting packet diversity in rate-distortion optimized sender-driven streaming of packetized media.

Rate-distortion hint tracks for adaptive video streaming

We present a technique for low-complexity rate-distortion (R-D) optimized adaptive video streaming based on the concept of rate-distortion hint track (RDHT). RDHTs store the precomputed characteristics of a compressed media source that are crucial for high performance online streaming but difficult to compute in real time. This enables low-complexity adaptation to variations in transport conditions such as available data rate or packet loss. An RDHT-based streaming system has three components: 1) information that summarizes the R-D attributes of the media; 2) an algorithm for using the RDHT to predict the distortion for a feasible packet schedule; and 3) a method for determining the best packet schedule to adapt the streaming to the communication channel. A family of distortion models, denoted distortion chains, are presented which accurately predict the distortion produced by arbitrary packet loss patterns. Two distortion chain models are examined which lead to two RDHT-based techniques. We evaluate the proposed techniques for two canonical problems in streaming media, adaptation to available data rate and to packet loss. Experimental results demonstrate that for the difficult case of nonscalably coded H.264 video, the proposed systems provide significant performance gains over conventional low-complexity streaming systems, and achieve this gain with a comparable level of complexity making them suitable for online R-D optimized streaming.

Application layer error-correction coding for rate-distortion optimized streaming to wireless clients

This paper addresses the problem of streaming packetized media over a lossy packet network to a wireless client, in a rate-distortion optimized way. We introduce an incremental redundancy error-correction scheme that combats the effects of both packet loss and bit errors in an end-to-end fashion, without support from the underlying network or from an intermediate base station. The scheme is employed within an optimization framework that enables the sender to compute which packets it should send, out of all the packets it could send at a given transmission opportunity, in order to meet an average transmission-rate constraint while minimizing the average end-to-end distortion. Experimental results show that our system is robust and maintains quality of service over a wide range of channel conditions. Up to 8 dB performance gains are registered over systems that are not rate-distortion optimized, at bit-error rates as large as 10/sup -2/.

Prioritized Distributed Video Delivery With Randomized Network Coding

We address the problem of prioritized video streaming over lossy overlay networks. We propose to exploit network path diversity via a novel randomized network coding (RNC) approach that provides unequal error protection (UEP) to the packets conveying the video content. We design a distributed receiver-driven streaming solution, where a client requests packets from the different priority classes from its neighbors in the overlay. Based on the received requests, a node in turn forwards combinations of the selected packets to the requesting peers. Choosing a network coding strategy at every node can be cast as an optimization problem that determines the rate allocation between the different packet classes such that the average distortion at the requesting peer is minimized. As the optimization problem has log-concavity properties, it can be solved with low complexity by an iterative algorithm. Our simulation results demonstrate that the proposed scheme respects the relative priorities of the different packet classes and achieves a graceful quality adaptation to network resource constraints. Therefore, our scheme substantially outperforms reference schemes such as baseline network coding techniques as well as solutions that employ rateless codes with built-in UEP properties. The performance evaluation provides additional evidence of the substantial robustness of the proposed scheme in a variety of transmission scenarios.

Adaptive systems for improved media streaming experience

Supporting streaming media applications over current packet network infrastructures represents a challenging task in many regards. For one, the lack of quality of service (QoS) guarantees in existing networks such as the Internet means that time-constrained media packets will face dynamic variations in bandwidth, loss rate, and delay as they traverse the network from the sender to the receiver. The variable rate of media traffic represents yet another difficulty when transmission constraints need to be met. Finally, the heterogeneity of client devices and access bandwidth coupled with custom user preferences exacerbate the problem of smooth and quality-optimized media playback even further. In this article we provide an overview of the various techniques for media and streaming strategy adaptation, which can be employed to deal with the difficulties imposed by such dynamic environments. These techniques depend on the characteristics of the media application, in particular on the network streaming infrastructure and the timing constraints imposed on the media packets delivery. We survey adaptation techniques that act on the encoding of the multimedia information, on the scheduling of the media packets, or that try to combat transmission errors. We also briefly overview some media-friendly networking solutions, which contribute to increased QoS by incorporating some level of intelligence in intermediate network nodes. Finally, we describe a few open challenges in media streaming, emphasizing strategies based on promising cross-layer approaches where adaptation strategies are applied in a coordinated manner, across different layers of the network protocol stack

Adaptive multiview video streaming: challenges and opportunities

Delivering multiview video content over present packet networks poses multiple challenges. First, the best effort nature of the Internet exposes media packets to variable bandwidth, loss, and delay as they traverse the network. Second, the prediction dependencies employed to maximize compression efficiency make the reconstruction process at the client extremely vulnerable to missing data. Third, the heterogeneity of client devices in terms of computing power, display capabilities, and access link capacity necessitates customizing the streaming process per user. My article reviews existing opportunities for addressing these challenges from within each of the three main stages of the content delivery pipeline (i.e., encoding, transmission, and reconstruction). Concretely, I first describe adaptive source coding techniques that construct a compressed representation of the multiview video source that exhibits resilience to network bandwidth variations and client view selection uncertainty. Then I discuss intelligent methods for error protection, caching, and packet scheduling that organize the transmission of multiview data in a bandwidth-effective way. Here, I also review prospective multipath and cloud-assisted techniques for multiview video streaming. Finally, I identify robust client-side content reconstruction schemes and adaptive media playout methods that can minimize the impact of missing data and enhance the users interactive experience. Then I proceed to describe community-driven streaming techniques for delivering interactive multiview content over a population of social peers. The article concludes with an outline of approaches for synergistic exploitation of the techniques I will present theretofore, jointly across the different layers of the network protocol stack at which they individually operate. Here, I also highlight the main deployment challenges for some of these techniques, and how their design should be addressed accordingly, to overcome them.

Server diversity in rate-distortion optimized media streaming

We consider diversity for media streaming in a receiver-driven rate-distortion optimization framework. Diversity is achieved by requesting media packets from multiple servers. A framework is proposed that enables the receiver to decide at every instant which packets, if any, to request for transmission and from which servers in order to meet a rate constraint while minimizing the end-to-end distortion. Experimental results demonstrate the benefit of exploiting server diversity in rate-distortion optimized receiver-driven streaming of packetized media.

Low-complexity rate-distortion optimized video streaming

This paper proposes two techniques for low-complexity rate-distortion (R-D) optimized streaming of packetized video. These techniques enable computing packet transmission schedules which satisfy a constraint on the average transmission rate while at the same time minimizing the average end-to-end distortion. Optimized packet schedules are computed with considerably lower complexity as compared to conventional algorithms for R-D optimized streaming, which makes these techniques suitable for on-line optimized streaming. Simulation experiments examine the performance of the proposed techniques using JVT/H.264 encoded video sequences and previous frame error concealment. The two techniques demonstrate substantial performance gains of 2-8 dB over a conventional streaming system that is not R-D optimized, which corresponds to a significant fraction of the gain achieved by current (high-complexity) R-D optimized schemes. Furthermore, this performance improvement is achieved with a complexity comparable to that of the conventional, non-R-D optimized, system.

Distributed Collaboration for Enhanced Sender-Driven Video Streaming

We propose a sender-driven system for adaptive streaming from multiple servers to a single receiver over separate network paths. The servers employ information in receiver feedbacks to estimate the available bandwidth on the paths and then compute appropriate transmission schedules for streaming media packets to the receiver based on the bandwidth estimates. An optimization framework is proposed that enables the senders to compute their transmission schedules in a distributed way, and yet to dynamically coordinate them over time such that the resulting video quality at the receiver is maximized. To reduce the computational complexity of the optimization framework an alternative technique based on packet classification is proposed. The substantial reduction in online complexity due to the resulting packet partitioning makes the technique suitable for practical implementations of adaptive and efficient distributed streaming systems. Simulations with Internet network traces demonstrate that the proposed solution adapts effectively to bandwidth variations and packet loss. They show that the proposed streaming framework provides superior performance over a conventional distortion-agnostic scheme that performs proportional packet scheduling on the network paths according to their respective bandwidth values.