Valeria Cardellini

Dynamic load balancing on Web-server systems

Popular Web sites cannot rely on a single powerful server nor on independent mirrored-servers to support the ever-increasing request load. Distributed Web server architectures that transparently schedule client requests offer a way to meet dynamic scalability and availability requirements. The authors review the state of the art in load balancing techniques on distributed Web-server systems, and analyze the efficiencies and limitations of the various approaches.

The state of the art in locally distributed Web-server systems

The overall increase in traffic on the World Wide Web is augmenting user-perceived response times from popular Web sites, especially in conjunction with special events. System platforms that do not replicate information content cannot provide the needed scalability to handle large traffic volumes and to match rapid and dramatic changes in the number of clients. The need to improve the performance of Web-based services has produced a variety of novel content delivery architectures. This article will focus on Web system architectures that consist of multiple server nodes distributed on a local area, with one or more mechanisms to spread client requests among the nodes. After years of continual proposals of new system solutions, routing mechanisms, and policies (the first dated back to 1994 when the NCSA Web site had to face the first million of requests per day), many problems concerning multiple server architectures for Web sites have been solved. Other issues remain to be addressed, especially at the network application layer, but the main techniques and methodologies for building scalable Web content delivery architectures placed in a single location are settled now. This article classifies and describes main mechanisms to split the traffic load among the server nodes, discussing both the alternative architectures and the load sharing policies. To this purpose, it focuses on architectures, internal routing mechanisms, and dispatching request algorithms for designing and implementing scalable Web-server systems under the control of one content provider. It identifies also some of the open research issues associated with the use of distributed systems for highly accessed Web sites.

Flow-Based Service Selection forWeb Service Composition Supporting Multiple QoS Classes

In the service oriented paradigm applications are created as a composition of independently developed Web services. Since the same service may be offered by different providers with different non-functional Quality of Service (QoS) attributes, a selection process is needed to identify the constituent services for a given composite service that best meet the users QoS requirements. In this paper, we consider a broker that offers a composite service with multiple QoS classes to several users each generating a flow of requests over time. We propose a service selection scheme which optimizes the end-to-end aggregated QoS of all incoming flows of requests by means of a simple linear programming problem which scales as the number of users, request volumes and/or services grows. This approach differs from most of the current proposals which may not scale well since: a) requests, even from the same user, are handled independently from one another; and b) the selection process often requires the solution of an NP-hard problem.

MOSES: A Framework for QoS Driven Runtime Adaptation of Service-Oriented Systems

Architecting software systems according to the service-oriented paradigm and designing runtime self-adaptable systems are two relevant research areas in todays software engineering. In this paper, we address issues that lie at the intersection of these two important fields. First, we present a characterization of the problem space of self-adaptation for service-oriented systems, thus providing a frame of reference where our and other approaches can be classified. Then, we present MOSES, a methodology and a software tool implementing it to support QoS-driven adaptation of a service-oriented system. It works in a specific region of the identified problem space, corresponding to the scenario where a service-oriented system architected as a composite service needs to sustain a traffic of requests generated by several users. MOSES integrates within a unified framework different adaptation mechanisms. In this way it achieves greater flexibility in facing various operating environments and the possibly conflicting QoS requirements of several concurrent users. Experimental results obtained with a prototype implementation of MOSES show the effectiveness of the proposed approach.

Dynamic load balancing in geographically distributed heterogeneous Web servers

With ever increasing Web traffic, a distributed multi server Web site can provide scalability and flexibility to cope with growing client demands. Load balancing algorithms to spread the requests across multiple Web servers are crucial to achieve the scalability. Various domain name server (DNS) based schedulers have been proposed in the literature, mainly for multiple homogeneous servers. The presence of heterogeneous Web servers not only increases the complexity of the DNS scheduling problem, but also makes previously proposed algorithms for homogeneous distributed systems not directly applicable. This leads us to propose new policies, cabled adaptive TTL algorithms, that take into account both the uneven distribution of client request rates and heterogeneity of Web servers to adaptively set the time-to-live (TTL) value for each address mapping request. Extensive simulation results show that these strategies are robust and effective in balancing load among geographically distributed heterogeneous Web servers.

Qos-driven runtime adaptation of service oriented architectures

Runtime adaptation is recognized as a viable way for a service-oriented system to meet QoS requirements in its volatile operating environment. In this paper we propose a methodology to drive the adaptation of such a system, that integrates within a unified framework different adaptation mechanisms, to achieve a greater flexibility in facing different operating environments and the possibly conflicting QoS requirements of several concurrent users. To determine the most suitable adaptation action(s), the methodology is based on the formulation and solution of a linear programming problem, which is derived from a behavioral model of the system updated at runtime by a monitoring activity. Numerical experiments show the effectiveness of our approach. Besides the methodology, we also present a prototype tool that implements it.

Geographic load balancing for scalable distributed Web systems

Users of highly popular Web sites may experience long delays when accessing information. Upgrading content site infrastructure from a single node to a locally distributed Web cluster composed by multiple server nodes provides limited relief, because the cluster wide-area connectivity may become the bottleneck. A better solution is to distribute Web clusters over the Internet by placing content nodes in strategic locations. A geographically distributed architecture where the Domain Name System (DNS) servers evaluate network proximity and users are served from the closest cluster reduces network impact on response time. On the other hand, serving closest requests only may cause unbalanced servers and may increase system impact on response time. To achieve a scalable Web system, we propose to integrate DNS proximity scheduling with an HTTP request redirection mechanism that any Web server can activate. We demonstrate through simulation experiments that this further dispatching mechanism augments the percentage of requests with guaranteed response time, thereby enhancing the Quality of Service of geographically distributed Web sites. However, HTTP request redirection should be used selectively because the additional round-trip increases network impact on latency time experienced by users. As a further contribution, this paper proposes and compares various mechanisms to limit reassignments with no negative consequences on load balancing.

Request redirection algorithms for distributed Web systems

Replication of information among multiple servers is necessary to support high request rates to popular Web sites. We consider systems that maintain one interface to users, even it they consist of multiple nodes with visible IP addresses that are distributed among different networks. In these systems, first-level dispatching is achieved through the Domain Name System (DNS) during the address lookup phase. Distributed Web systems can use a request redirection mechanism as second-level dispatching because the DNS routing scheme has limited control on offered load. Redirection is always executed by the servers, but there are many alternatives that are worth investigating. We explore the combination of DNS dispatching with redirection schemes that use centralized or distributed control on the basis of global or local state information. In fully distributed schemes, DNS dispatching is carried out by simple algorithms because load sharing is taken by some redirection mechanisms that each server activates autonomously. On the other hand, in fully centralized schemes, redirection is used as a tool to enforce decisions taken by the same centralized entity that provides the first-level dispatching. We also investigate hybrid strategies. We conclude that distributed algorithms are preferable over their centralized counterpart because they provide stable performance, take content-aware dispatching decisions, limit the percentage of redirected requests, and their implementation is much simpler than that required by centralized schemes.

Redirection algorithms for load sharing in distributed Web-server systems

Replication of information among multiple World Wide Web servers is necessary to support high request rates to popular Web sites. A clustered Web server organization is preferable to multiple independent mirrored servers because it maintains a single interface to the users and has the potential to be more scalable, fault-tolerant and better load-balanced. In this paper, we propose a Web cluster architecture in which the Domain Name System (DNS) server, which dispatches the user requests among the servers through the URL name to the IP address mapping mechanism, is integrated with a redirection request mechanism based on HTTP. This should alleviate the side-effect of caching the IP address mapping at intermediate name servers. We compare many alternative mechanisms, including synchronous vs. asynchronous activation and centralized vs. distributed decisions on redirection. Moreover, we analyze the reassignment of entire domains or individual client requests, different types of status information and different server selection policies for redirecting requests. Our results show that the combination of centralized and distributed dispatching policies allows the Web server cluster to handle high load skews in the WWW environment.

Web switch support for differentiated services

As the Web is becoming a medium widely used as a preferential channel for critical information exchange, business, and e-commerce, it is necessary to enable differentiated service mechanisms not only at the network but also at the Web server level. In this paper, we propose the concept of Quality of Web Services (QoWS), which is inspired by the basic principles of network QoS, while looking at the server components of the Web system. In particular, we analyze how QoWS principles can be realized in a Web site hosted on a Web-server cluster that is, an architecture composed by multiple Web servers locally distributed and a single front-end node, called a Web switch. We propose a new centralized policy, namely DynamicPartitioning, which satisfies through dynamic server partition all basic QoS principles for a Web switch working at application level. We compare it against other proposed classes of policies which implement part or all of basic QoS principles. We demonstrate through a large set of simulation experiments under a realistic workload model that DynamicPartitioning always achieves superior performance for the high service class, at the price of some penalty for low service classes.