Wolfgang Segmuller

CPU demand for web serving: Measurement analysis and dynamic estimation

Managing the resources in a large Web serving system requires knowledge of the resource needs for service requests of various types. In order to investigate the properties of Web traffic and its demand, we collected measurements of throughput and CPU utilization and performed some data analyses. First, we present our findings in relation to the time-varying nature of the traffic, the skewness of traffic intensity among the various types of requests, the correlation among traffic streams, and other system-related phenomena. Then, given such nature of web traffic, we devise and implement an on-line method for the dynamic estimation of CPU demand. Assessing resource needs is commonly performed using techniques such as off-line profiling, application instrumentation, and kernel-based instrumentation. Little attention has been given to the dynamic estimation of dynamic resource needs, relying only on external and high-level measurements such as overall resource utilization and request rates. We consider the problem of dynamically estimating dynamic CPU demands of multiple kinds of requests using CPU utilization and throughput measurements. We formulate the problem as a multivariate linear regression problem and obtain its basic solution. However, as our measurement data analysis indicates, one is faced with issues such as insignificant flows, collinear flows, space and temporal variations, and background noise. In order to deal with such issues, we present several mechanisms such as data aging, flow rejection, flow combining, noise reduction, and smoothing. We implemented these techniques in a Work Profiler component that we delivered as part of a broader system management product. We present experimental results from using this component in scenarios inspired by real-world usage of that product.

Dynamic estimation of CPU demand of web traffic

Managing the resources in a large Web serving system requires knowledge of the resource needs for service request-s of various kinds, and these needs may change over time. Assessing resource needs is commonly performed using techniques such as offline profiling, application instrumentation, and kernel-based instrumentation. Little attention has been given to the dynamic estimation of dynamic resource needs, relying only on external and high-level measurements such as overall resource utilization and request rates. We consider the problem of dynamically estimating dynamic CPU demands of multiple kinds of requests using CPU utilization and throughput measurements. We formulate the problem as a linear regression problem and obtain its basic solution. However, in practice one is faced with issues such as insignificant flows, collinear flows, space and temporal variations, and background noise. In order to deal with such issues, we present several mechanisms such as data aging, flow rejection, flow combining, noise reduction, and smoothing. We implemented these techniques in a Work Profiler component that we delivered as part of a broader system management product. We present experimental results from using this component in scenarios inspired by real-world usage of that product; our technique produces estimates that are roughly within a factor of 2 of the right answer, for the request flows that draw significant CPU power.

Design, implementation, and performance of a load balancer for SIP server clusters

This paper introduces several novel load-balancing algorithms for distributing Session Initiation Protocol (SIP) requests to a cluster of SIP servers. Our load balancer improves both throughput and response time versus a single node while exposing a single interface to external clients. We present the design, implementation, and evaluation of our system using a cluster of Intel x86 machines running Linux. We compare our algorithms to several well-known approaches and present scalability results for up to 10 nodes. Our best algorithm, Transaction Least-Work-Left (TLWL), achieves its performance by integrating several features: knowledge of the SIP protocol, dynamic estimates of back-end server load, distinguishing transactions from calls, recognizing variability in call length, and exploiting differences in processing costs for different SIP transactions. By combining these features, our algorithm provides finer-grained load balancing than standard approaches, resulting in throughput improvements of up to 24% and response-time improvements of up to two orders of magnitude. We present a detailed analysis of occupancy to show how our algorithms significantly reduce response time.

Load Balancing for SIP Server Clusters

This paper introduces several novel load balancing algorithms for distributing Session Initiation Protocol (SIP) requests to a cluster of SIP servers. Our load balancer improves both throughput and response time versus a single node, while exposing a single interface to external clients. We present the design, implementation and evaluation of our system using a cluster of Intel x86 machines running Linux. We compare our algorithms with several well-known approaches and present scalability results for up to 10 nodes. Our best algorithm, Transaction Least-Work-Left (TLWL), achieves its performance by integrating several features: knowledge of the SIP proto- col; dynamic estimates of back-end server load; distinguishing transactions from calls; recognizing variability in call length; and exploiting differences in processing costs for different SIP transactions. By combining these features, our algorithm provides finer-grained load balancing than standard approaches, resulting in throughput improvements of up to 24 percent and response time improvements of up to two orders of magnitude. We present a detailed analysis of occupancy to show how our algorithms significantly reduce response time.

Workload orchestration and optimization for software defined environments

The software defined environment (SDE) provides a powerful programmable interface to a cloud infrastructure through an abstraction of compute, network, and storage resources. A workload refers to the application to be deployed in such an infrastructure. To take advantage of the SDE interface, the workload is described using a declarative workload definition language and is then deployed in the infrastructure through an automated workload orchestration and optimization layer. This paper describes the architecture and algorithms that make up this layer. Given a definition of the workload, including the virtual components of the application and their resource needs, as well as other meta-information relating to factors such as performance, availability, and privacy, the function of the workload orchestration and optimization layer is to map virtual resources to physical resources and realize such a mapping in the infrastructure. This mapping, known as placement, is optimized so that the infrastructure is efficiently utilized, and the workload requirements are satisfied. We present the overall architecture of the workload orchestration and optimization runtime. We focus on the workload placement problem and describe our optimization framework. Then, we consider a real application, IBM Connections, as a use-case to demonstrate the orchestration and optimization functionalities.