Tom Z. J. Fu

Challenges, design and analysis of a large-scale p2p-vod system

P2P file downloading and streaming have already become very popular Internet applications. These systems dramatically reduce the server loading, and provide a platform for scalable content distribution, as long as there is interest for the content. P2P-based video-on-demand (P2P-VoD) is a new challenge for the P2P technology. Unlike streaming live content, P2P-VoD has less synchrony in the users sharing video content, therefore it is much more difficult to alleviate the server loading and at the same time maintaining the streaming performance. To compensate, a small storage is contributed by every peer, and new mechanisms for coordinating content replication, content discovery, and peer scheduling are carefully designed. In this paper, we describe and discuss the challenges and the architectural design issues of a large-scale P2P-VoD system based on the experiences of a real system deployed by PPLive. The system is also designed and instrumented with monitoring capability to measure both system and component specific performance metrics (for design improvements) as well as user satisfaction. After analyzing a large amount of collected data, we present a number of results on user behavior, various system performance metrics, including user satisfaction, and discuss what we observe based on the system design. The study of a real life system provides valuable insights for the future development of P2P-VoD technology.

Statistical modeling and analysis of P2P replication to support VoD service

Traditional Video-on-Demand (VoD) systems reply purely on servers to stream video content to clients, which does not scale. In recent years, Peer-to-peer assisted VoD (P2P VoD) has proven to be practical and effective [1]. In P2P VoD, each peer contributes some storage to store videos (or segments of videos) to help the video server. Assuming peers have sufficient bandwidth for the given video playback rate, a fundamental question is what is the relationship between the storage capacity (at each peer), the number of videos, the number of peers and the resultant off-loading of video server bandwidth. In this paper, we use a simple statistical model to derive this relationship. We propose and analyze a generic replication algorithm RLB which balances the service to all movies, for both deterministic and random demand models, and both homogeneous and heterogeneous peers (in upload bandwidth). We use simulation to validate our results, for sensitivity analysis and for comparisons with other popular replication algorithms. This study leads to several fundamental insights for design P2P VoD systems in practice.

DRS: Dynamic Resource Scheduling for Real-Time Analytics over Fast Streams

In a data stream management system (DSMS), users register continuous queries, and receive result updates as data arrive and expire. We focus on applications with real-time constraints, in which the user must receive each result update within a given period after the update occurs. To handle fast data, the DSMS is commonly placed on top of a cloud infrastructure. Because stream properties such as arrival rates can fluctuate unpredictably, cloud resources must be dynamically provisioned and scheduled accordingly to ensure real-time response. It is essential, for the existing systems or future developments, to possess the ability of scheduling resources dynamically according to the current workload, in order to avoid wasting resources, or failing in delivering correct results on time. Motivated by this, we propose DRS, a novel dynamic resource scheduler for cloud-based DSMSs. DRS overcomes three fundamental challenges: (a) how to model the relationship between the provisioned resources and query response time (b) where to best place resources, and (c) how to measure system load with minimal overhead. In particular, DRS includes an accurate performance model based on the theory of Jackson open queueing networks and is capable of handling arbitrary operator topologies, possibly with loops, splits and joins. Extensive experiments with real data confirm that DRS achieves real-time response with close to optimal resource consumption.

A unifying model and analysis of P2P VoD replication and scheduling

We consider a P2P-assisted Video-on-Demand (VoD) system where each peer can store a relatively small number of movies to offload the server when these movies are requested. User requests are stochastic based on some movie popularity distribution. The problem is how to replicate (or place) content at peer storage to minimize the server load. Several variation of this replication problem have been studied recently with somewhat different conclusions. In this paper, we first point out that the main difference between these studies is in how they model the scheduling of peers to serve user requests, and show that these different scheduling assumptions will lead to different “optimal” replication strategies. We then propose a unifying request scheduling model, parameterized by the maximum number of peers that can be used to serve a single request. This scheduling is called Fair Sharing with Bounded Out-Degree (FSBD). Based on this unifying model, we can compare the different replication strategies for different out-degree bounds and see how and why different replication strategies are favored depending on the out-degree. We also propose a new simple, adaptive, and essentially distributed replication algorithm, and show that this algorithm is able to adapt itself to work well for different out-degree in scheduling.

On replication algorithm in P2P VoD

Traditional video-on-demand (VoD) systems rely purely on servers to stream video content to clients, which does not scale. In recent years, peer-to-peer assisted VoD (P2P VoD) has proven to be practical and effective. In P2P VoD, each peer contributes some storage to store videos (or segments of videos) to help the video server. Assuming peers have sufficient bandwidth for the given video playback rate, a fundamental question is what is the relationship between the storage capacity (at each peer), the number of videos, the number of peers, and the resultant off-loading of video server bandwidth. In this paper, we use a simple statistical model to derive this relationship. We propose and analyze a generic replication algorithm Random with Load Balancing (RLB) that balances the service to all movies for both deterministic and random (but stationary) demand models and both homogeneous and heterogeneous peers (in upload bandwidth). We use simulation to validate our results for sensitivity analysis and for comparisons to other popular replication algorithms. This study leads to several fundamental insights for P2P VoD system design in practice.

Generalized preferential attachment considering aging

Preferential Attachment (PA) models the scientific citation process. In the PA model, a new paper attaches itself to the citation network based only on the popularity of the currently existing papers. This invariably leads to a network whose degree distribution satisfies the Power Law. Yet, empirical results show that paper age should also play a role in the citation process. In other words, when references are chosen for a new paper, the age of an existing paper may also affect the choice for citing. In this paper, we derive a generalized PA model that includes the effect of aging, with analytical solution. Such a model can be used to analyze the competing influence of preferential attachment and aging effect quantitatively in citation process and explain differences in various research domains by the extent of aging. It may also serve as a general model of network formation.

The academic social network

By means of their academic publications, authors form a social network. Instead of sharing casual thoughts and photos (as in Facebook), authors select co-authors and reference papers written by other authors. Thanks to various efforts (such as Microsoft Academic Search and DBLP), the data necessary for analyzing the academic social network is becoming more available on the Internet. What type of information and queries would be useful for users to discover, beyond the search queries already available from services such as Google Scholar? In this paper, we explore this question by defining a variety of ranking metrics on different entities—authors, publication venues, and institutions. We go beyond traditional metrics such as paper counts, citations, and h-index. Specifically, we define metrics such as influence, connections, and exposure for authors. An author gains influence by receiving more citations, but also citations from influential authors. An author increases his or her connections by co-authoring with other authors, and especially from other authors with high connections. An author receives exposure by publishing in selective venues where publications have received high citations in the past, and the selectivity of these venues also depends on the influence of the authors who publish there. We discuss the computation aspects of these metrics, and the similarity between different metrics. With additional information of author-institution relationships, we are able to study institution rankings based on the corresponding authors’ rankings for each type of metric as well as different domains. We are prepared to demonstrate these ideas with a web site (http://pubstat.org) built from millions of publications and authors.

An Adaptive Cloud Downloading Service

Video content downloading using the P2P approach is scalable, but does not always give good performance. Recently, subscription-based premium services have emerged, referred to as cloud downloading. In this service, the cloud storage and server caches user-interested content and updates the cache based on user downloading requests. If a requested video is not in the cache, the request is held in a waiting state until the cache is updated. We call this design server mode. An alternative design is to let the cloud server serve all downloading requests as soon as they arrive, behaving as a helper peer. We call this design helper mode. Our model and analysis show that both these designs are useful for certain operating regimes. The helper mode is good at handling a high request rate, while the server mode is good at scaling with video population size. We design an adaptive algorithm (AMS) to select the service mode automatically. Intuitively, AMS switches service mode from server mode to helper mode when too many peers request blocked movies, and vice versa. The ability of AMS to achieve good performance in different operating regimes is validated by simulation .

Designing QoE experiments to evaluate peer-to-peer streaming applications

Quality of Experience (QoE) refers to subjective criteria for evaluating multimedia content. Methods have been devised to study the design of Voice over IP systems and video codecs. In recent years, due to more abundant network bandwidth, it has become quite popular to watch video streamed over the Internet, whether by clientserver method or through a peer-to-peer (P2P) network. In this paper, we describe our experience in conducting QoE studies of P2P streaming using a chunk-level model. Instead of considering fine-grained network service impairments such as bit errors, packet losses or delays, we focus on chunk level delays. We carry out some preliminary QoE experiments on low-bit rate, low-frame rate and low-resolution video (3L-video) sequences. We apply the chunk-level model to help improve the design of the P2P streaming algorithms, and the design of video players that playback network streamed video.

Server-assisted adaptive video replication for P2P VoD

In recent years, Peer-to-Peer assisted Video-on-Demand (P2P VoD) has become an effective and efficient approach to distribute high-quality videos to large number of peers. In a P2P VoD system, each peer contributes storage to store several videos to help offload the server. The replication strategy, which determines the videos to be stored at each peers local storage, plays an important role in system performance. There are two approaches: (a) solve a huge combinatorial optimization problem and (b) use simple cache replacement algorithms, such as Least-Frequently-Requested (LFR) or FIFO. The first approach needs to collect a large number of parameters whose values may be changing, and use some approximation method (such as linearization) to solve the optimization problem, both aspects have accuracy issues. In the second approach, a peer replaces some video in the cache with the currently viewed video, based on local information. While it is simple, we show their performance can be improved by a little centrally collected state information. Specifically, the needed feedback information is the current downloading rate provided by peers for each video. In this paper, we describe a hybrid replication strategy, and give detailed description of how the server collects and maintains the feedback information, and how peers use that information to determine what videos to store and indirectly control their uplink bandwidth contribution. This explains why the hybrid strategy is much simpler and more practical than the combinatory optimization approach. We then use simulation to demonstrate how our scheme out-performs the simple adaptive algorithms. Our simulation results also demonstrate how our scheme is able to quickly respond to peer churn and video popularity churn.