John C. S. Lui

Challenges, design and analysis of a large-scale p2p-vod system

P2P file downloading and streaming have already become very popular Internet applications. These systems dramatically reduce the server loading, and provide a platform for scalable content distribution, as long as there is interest for the content. P2P-based video-on-demand (P2P-VoD) is a new challenge for the P2P technology. Unlike streaming live content, P2P-VoD has less synchrony in the users sharing video content, therefore it is much more difficult to alleviate the server loading and at the same time maintaining the streaming performance. To compensate, a small storage is contributed by every peer, and new mechanisms for coordinating content replication, content discovery, and peer scheduling are carefully designed. In this paper, we describe and discuss the challenges and the architectural design issues of a large-scale P2P-VoD system based on the experiences of a real system deployed by PPLive. The system is also designed and instrumented with monitoring capability to measure both system and component specific performance metrics (for design improvements) as well as user satisfaction. After analyzing a large amount of collected data, we present a number of results on user behavior, various system performance metrics, including user satisfaction, and discuss what we observe based on the system design. The study of a real life system provides valuable insights for the future development of P2P-VoD technology.

Adaptive piggybacking: a novel technique for data sharing in video-on-demand storage servers

Recent technology advances have made multimedia on-demand services, such as home entertainment and home-shopping, important to the consumer market. One of the most challenging aspects of this type of service is providing access either instantaneously or within a small and reasonable latency upon request. We consider improvements in the performance of multimedia storage servers through data sharing between requests for popular objects, assuming that the I/O bandwidth is the critical resource in the system. We discuss a novel approach to data sharing, termed adaptive piggybacking, which can be used to reduce the aggregate I/O demand on the multimedia storage server and thus reduce latency for servicing new requests.

An efficient partitioning algorithm for distributed virtual environment systems

Distributed virtual environment (DVE) systems model and simulate the activities of thousands of entities interacting in a virtual world over a wide area network. Possible applications for DVE systems are multiplayer video games, military and industrial trainings, and collaborative engineering. In general, a DVE system is composed of many servers and each server is responsible to manage multiple clients who want to participate in the virtual world. Each server receives updates from different clients (such as the current position and orientation of each client) and then delivers this information to other clients in the virtual world. The server also needs to perform other tasks, such as object collision detection and synchronization control. A large scale DVE system needs to support many clients and this imposes a heavy requirement on networking resources and computational resources. Therefore, how to meet the growing requirement of bandwidth and computational resources is one of the major challenges in designing a scalable and cost-effective DVE system. In this paper, we propose an efficient partitioning algorithm that addresses the scalability issue of designing a large scale DVE system. The main idea is to dynamically divide the virtual world into different partitions and then efficiently assign these partitions to different servers. This way, each server will process approximately the same amount of workload. Another objective of the partitioning algorithm is to reduce the server-to-server communication overhead. The theoretical foundation of our dynamic partitioning algorithm is based on the linear optimization principle. We also illustrate how one can parallelize the proposed partitioning algorithm so that it can efficiently partition a very large scale DVE system. Lastly, experiments are carried out to illustrate the effectiveness of the proposed partitioning algorithm under various settings of the virtual world.

A Simple Model for Analyzing P2P Streaming Protocols

P2P streaming tries to achieve scalability (like P2P file distribution) and at the same time meet real-time playback requirements. It is a challenging problem still not well understood. In this paper, we describe a simple stochastic model that can be used to compare different data-driven downloading strategies based on two performance metrics: continuity (probability of continuous playback), and startup latency (expected time to start playback). We first study two simple strategies: rarest first and greedy. The former is a well-known strategy for P2P file sharing that gives good scalability, whereas the latter an intuitively reasonable strategy to optimize continuity and startup latency from a single peers viewpoint. Greedy, while achieving low startup latency, fares poorly in continuity by failing to maximize P2P sharing; whereas rarest first is the opposite. This highlights the trade-off between startup latency and continuity, and how system scalability improves continuity. Based on this insight, we propose a mixed strategy that can be used to achieve the best of both worlds. Our algorithm dynamically adapts to the peer population size to ensure scalability; at the same time, it reserves part of a peers effort to the immediate playback requirements to ensure low startup latency.

How Many Packets Can We Encode? - An Analysis of Practical Wireless Network Coding

While the practical coding scheme has been shown to be able to improve throughput of wireless networks, there still lacks fundamental understanding on how the coding scheme works under realistic settings, namely, when it operates on a realistic physical layer and the medium access is controlled by some random access methods. In this paper, we provide a formal analysis on the performance of the practical coding scheme under such realistic settings. The key performance measure is the encoding number, i.e., the number of packets that can be encoded by a coding node in each transmission. We provide an upper bound on the encoding number for the general coding topology, and derive the average encoding number and system throughput for a general class of random access mechanisms. Based on the practical coding scheme, we also derive a tighter upper bound on the throughput gain for a general wireless network. Our results can be particularly useful for coding-related MAC/Routing protocol design and analysis.

Secure Overlay Cloud Storage with Access Control and Assured Deletion

We can now outsource data backups off-site to third-party cloud storage services so as to reduce data management costs. However, we must provide security guarantees for the outsourced data, which is now maintained by third parties. We design and implement FADE, a secure overlay cloud storage system that achieves fine-grained, policy-based access control and file assured deletion. It associates outsourced files with file access policies, and assuredly deletes files to make them unrecoverable to anyone upon revocations of file access policies. To achieve such security goals, FADE is built upon a set of cryptographic key operations that are self-maintained by a quorum of key managers that are independent of third-party clouds. In particular, FADE acts as an overlay system that works seamlessly atop todays cloud storage services. We implement a proof-of-concept prototype of FADE atop Amazon S3, one of todays cloud storage services. We conduct extensive empirical studies, and demonstrate that FADE provides security protection for outsourced data, while introducing only minimal performance and monetary cost overhead. Our work provides insights of how to incorporate value-added security features into todays cloud storage services.

Defending against low-rate TCP attacks: dynamic detection and protection

We consider a distributed approach to detect and to defend against the low-rate TCP attack (A. Kuzmanovic et al., August 2003). The low-rate TCP attack is essentially a periodic short burst which exploits the homogeneity of the minimum retransmission timeout (RTO) of TCP flows and forces all affected TCP flows to back off and enter the retransmission timeout state. This sort of attack is difficult to identify due to a large family of attack patterns. We propose a distributed detection mechanism which uses the dynamic time warping method to robustly and accurately identify the existence of this sort of attack. Once the attack is detected, a fair resource allocation mechanism is used so that (1) the number of affected TCP flows is minimized, and (2) we provide sufficient resource protection for the affected TCP flows. We report experimental results to quantify the robustness and accuracy of the proposed detection mechanism and the efficiency of the defense method.

Reducing I/O demand in video-on-demand storage servers

Recent technological advances have made multimedia on-demand services, such as home entertainment and home-shopping, important to the consumer market. One of the most challenging aspects of this type of service is providing access either instantaneously or within a small and reasonable latency upon request. In this paper, we discuss a novel approach, termed adaptive piggybacking, which can be used to provide on-demand or nearly-on-demand service and at the same time reduce the I/O demand on the multimedia storage server.

Determining the end-to-end throughput capacity in multi-hop networks: methodology and applications

In this paper, we present a methodology to analytically compute the throughput capacity, or the maximum end-to-end throughput of a given source and destination pair in a multi-hop wireless network. The end-to-end throughput capacity is computed by considering the interference due to neighboring nodes, as well as various modes of hidden node interference. Knowing the throughput capacity is important because it facilitates the design of routing policy, admission control for realtime traffic, as well as load control for wireless networks. We model location-dependent neighboring interference and we use a contention graph to represent these interference relationships. Based on the contention graph, we formulate the individual link capacity as a set of fixed point equations. The end-to-end throughput capacity can then be determined once these link capacities are obtained. To illustrate the utility of our proposed methodology, we present two important applications: (a) route optimization to determine the path with the maximum end-to-end throughput capacity and, (b) optimal offered load control for a given path so that the maximum end-to-end capacity can be achieved. Extensive simulations are carried out to verify and validate the proposed analytical methodology.

Defending against distributed denial-of-service attacks with max-min fair server-centric router throttles

We present a network architecture and accompanying algorithms for countering distributed denial-of-service (DDoS) attacks directed at an Internet server. The basic mechanism is for a server under stress to install a router throttle at selected upstream routers. The throttle can be the leaky-bucket rate at which a router can forward packets destined for the server. Hence, before aggressive packets can converge to overwhelm the server, participating routers proactively regulate the contributing packet rates to more moderate levels, thus forestalling an impending attack. In allocating the server capacity among the routers, we propose a notion of level-k max-min fairness. We present a control-theoretic model to evaluate algorithm convergence under a variety of system parameters. In addition, we present packet network simulation results using a realistic global network topology, and various models of good user and attacker distributions and behavior. Using a generator model of Web requests parameterized by empirical data, we also evaluate the impact of throttling in protecting user access to a Web server. First, for aggressive attackers, the throttle mechanism is highly effective in preferentially dropping attacker traffic over good user traffic. In particular, level-k max-min fairness gives better good-user protection than recursive pushback of max-min fair rate limits proposed in the literature. Second, throttling can regulate the experienced server load to below its design limit - in the presence of user dynamics - so that the server can remain operational during a DDoS attack.