Yennun Huang

Software rejuvenation: analysis, module and applications

Software rejuvenation is the concept of gracefully terminating an application and immediately restarting it at a clean internal state. In a client-server type of application where the server is intended to ran perpetually for providing a service to its clients, rejuvenating the server process periodically during the most idle time of the server increases the availability of that service. In a long-running computation-intensive application, rejuvenating the application periodically and restarting it at a previous checkpoint increases the likelihood of successfully completing the application execution. We present a model for analyzing software rejuvenation in such continuously-running applications and express downtime and costs due to downtime during rejuvenation in terms of the parameters in that model. Threshold conditions for rejuvenation to be beneficial are also derived. We implemented a reusable module to perform software rejuvenation. That module can be embedded in any existing application on a UNIX platform with minimal effort. Experiences with software rejuvenation in a billing data collection subsystem of a telecommunications operations system and other continuously-running systems and scientific applications in AT&T are described.<<ETX>>

ONE-IP: techniques for hosting a service on a cluster of machines

Abstract With the explosive growth of the World Wide Web, some popular web sites are getting thousands of hits per second. As a result, clients (browsers) experience slow response times and sometimes may not be able to access some web sites at all. Upgrading the server nodes to more powerful machines may not always be cost-effective. A natural solution is to deploy a set of machines, or a cluster, and have them work together to host a single service. Such a server cluster should preferably publicize only one server name for the entire cluster so that any configuration change inside the cluster does not affect client applications. In this paper, we first discuss existing approaches to distributing clients requests for a single service to different machines in a cluster. We then propose two new techniques, collectively called ONE-IP , based on dispatching packets at the IP level. They have the advantages of fast dispatching, and ease of implementation. Ideas presented here are generic and should be applicable to other services as well.

Checkpointing and its applications

The paper describes our experience with the implementation and applications of the Unix checkpointing library libckp, and identifies two concepts that have proven to be the key to making checkpointing a powerful tool. First, including all persistent states, i.e., user files, as part of the process state that can be checkpointed and recovered provides a truly transparent and consistent rollback. Second, excluding part of the persistent state from the process state allows user programs to process future inputs from a desirable state, which leads to interesting new applications of checkpointing. We use real-life examples to demonstrate the use of libckp for bypassing premature software exits, for fast initialization and for memory rejuvenation.<<ETX>>

Progressive retry for software error recovery in distributed systems

A method of execution retry for bypassing software faults based on checkpointing, rollback, message reordering, and replaying is described. The authors demonstrate how rollback techniques, previously developed for transient hardware failure recovery, can also be used to recover from software errors by exploiting message reordering to bypass software faults. The approach intentionally increases the degree of nondeterminism and the scope of rollback when a previous retry fails. Examples from experience with telecommunications software systems illustrate the benefits of the scheme.

Performance and reliability evaluation of passive replication schemes in application level fault tolerance

Process replication is provided as the central mechanism for application level software fault tolerance in SwiFT and DOORS. These technologies, implemented as reusable software modules, support cold and warm schemes of passive replication. The choice of a scheme for a particular application is based on its availability and performance requirements. In this paper we analyze the performability of a server software which may potentially use these technologies. We derive closed form formulae for availability throughput and probability of loss of a job. Six scenarios of loss are modeled and for each, these expressions are derived. The formulae can be used either of time or online to determine the optimal replication scheme.

NT-SwiFT: software implemented fault tolerance on Windows NT

More and more high available applications are implemented on Windows NT. However, the current version of Windows NT (NT4) does not provide some facilities that are needed to implement these fault tolerant applications. In this paper, we describe a set of components collectively named NT-SwiFT (Software Implemented Fault Tolerance) which facilitates building fault-tolerant and highly available applications on Windows NT. NT-SwiFT provides components for automatic error detection and recovery, checkpointing, event logging and replay, communication error recovery, incremental data replications, IP packets re-routing, etc. SwiFT components were originally designed on UNIX. The UNIX version was first ported to NT to run on UWIN [Korn97]. Gradually a large portion of the software has been re-implemented to take advantage of native NT system services. This paper describes these components and compares the differences in the UNIX and NT implementations. We also describe some applications using these components and discuss how to leverage NT system services and cope with some missing features.

Why optimistic message logging has not been used in telecommunications systems

Much of the literature on message logging and checkpointing in the past decade has been based on a so-called optimistic approach that places more emphasis on failure-free overhead than recovery efficiency. Our experience has shown that most telecommunications systems use a pessimistic approach because the main purpose of using message logging and checkpointing is to achieve fast and localized recovery, and the failure-free overhead of a pessimistic approach can often be made reasonably low by exploiting application-specific information.<<ETX>>

Towards capacity and profit optimization of video-on-demand services in a peer-assisted IPTV platform

This paper studies the conditions under which peer-to-peer (P2P) technology may be beneficial in providing IPTV services over typical network architectures. It has three major contributions. First, we contrast two network models used to study the performance of such a system: a commonly used logical “Internet as a cloud” model and a “physical” model that reflects the characteristics of the underlying network. Specifically, we show that the cloud model overlooks important architectural aspects of the network and may drastically overstate the benefits of P2P technology. Second, we propose an algorithm called Zebra that pre-stripes content across multiple peers during idle hours to speed up P2P content delivery in an IPTV environment with limited upload bandwidth. We also perform simulations to measure Zebra’s effectiveness at reducing load on the content server during peak hours. Third, we provide a cost-benefit analysis of P2P video content delivery, focusing on the profit trade-offs for different pricing/incentive models rather than purely on capacity maximization. In particular, we find that under high volume of video demand, a P2P built-in incentive model performs better than any other model, while the conventional no-P2P model generates more profits when the request rate is low. The flat-reward model generally falls in between the usage-based model and the built-in model in terms of profitability except for low request rates. We also find that built-in and flat-reward models are more profitable than the usage-based model for a wide range of subscriber community sizes.

Xept: a software instrumentation method for exception handling

Modern software systems are often built from existing library components. A common problem is how to fix bugs when source code is not available. Xept is an instrumentation language and tool that can be used to add to object code the ability to detect, mask, recover and propagate exceptions from library functions. This helps to alleviate or avoid a large class of errors resulting from function misuses. Examples are given to show applications of Xept in actual software systems.

Capacity analysis of MediaGrid: a P2P IPTV platform for fiber to the node (FTTN) networks

This paper studies the conditions under which P2P sharing can increase the capacity of IPTV services over FTTN networks. For a typical FTTN network, our study shows a) P2P sharing is not beneficial when the total traffic in a local video office is low; b) P2P sharing increases the load on FTTN switches and routers in local video offices; c) P2P sharing is the most beneficial when the network bottleneck is experienced in the southbound segment of a local video office (equivalently a northbound segment of an FTTN switch); and d) sharing among all FTTN serving communities is not needed when network congestion problems are solved by using some other technologies such as program pre-caching or replication. Based on the analytical results, design for IPTV services which monitors FTTN network conditions and decides when and how to share videos among peers to maximize the service capacity. Simulations and bounds both validate the potential benefits of the MediaGrid IPTV service platform.