Greg Franks

The Future of Software Performance Engineering

Performance is a pervasive quality of software systems; everything affects it, from the software itself to all underlying layers, such as operating system, middleware, hardware, communication networks, etc. Software Performance Engineering encompasses efforts to describe and improve performance, with two distinct approaches: an early-cycle predictive model-based approach, and a late-cycle measurement-based approach. Current progress and future trends within these two approaches are described, with a tendency (and a need) for them to converge, in order to cover the entire development cycle.

Enhanced Modeling and Solution of Layered Queueing Networks

Layered queues are a canonical form of extended queueing network for systems with nested multiple resource possession, in which successive depths of nesting define the layers. The model has been applied to most modern distributed systems, which use different kinds of client-server and master-slave relationships, and scales up well. The layered queueing network (LQN) model is described here in a unified fashion, including its many more extensions to match the semantics of sophisticated practical distributed and parallel systems. These include efficient representation of replicated services, parallel and quorum execution, and dependability analysis under failure and reconfiguration. The full LQN model is defined here and its solver is described. A substantial case study to an air traffic control system shows errors (compared to simulation) of a few percent. The LQN model is compared to other models and solutions, and is shown to cover all their features.

A toolset for performance engineering and software design of client-server systems

Abstract TimeBench/SRVN is a prototype toolset for computer-aided design and performance analysis of software, with an emphasis on distributed client-server systems. The performance behaviour of such systems may defy intuition because it involves factors in the software design (such as the partitioning of the functionality and the frequency with which requests will be made to each server) and in the configuration of the distributed system (including replication of services, the distribution of data, and the speed of network access). The novelty of the tool consists in providing support both for developing design specifications and also for performance analysis. The integrated approach avoids the semantic gap between a designers domain and the performance modeling domain, and assists the designer to explore factors that impact the performance of a design. The performance models are based on the Stochastic Rendezvous Network (SRVN) formalism for client-server systems with synchronous service requests. The distinctive features of SRVNs are nested services (since servers can also act as clients to other servers) and the existence of two or more phases of service (the first executed while the client is blocked, and the others executed in parallel with the client). TimeBench/SRVN is intended as a demonstration of the concept of an integrated designer/performance interface, and as a research environment for fast analytic solvers for the models. Besides a simulation solver, it offers three approximate analytic solvers based on recent research, a Markovian solver, a technique for finding bounds on the throughput without too many assumptions, and a tool for rapidly exploring the space of possible parameter values.

Layered Bottlenecks and Their Mitigation

Bottlenecks are a simple and well-understood phenomenon in service systems and queueing models. However in systems with layered resources bottlenecks are more complicated, because of simultaneous resource possession. Thus, the holding time of a higher-layer resource, such as a process thread, may include a small execution demand, but a large time to use other resources at a lower layer (such as a disk). A single saturation point may in fact saturate many other resources by push-back, making diagnosis of the problem difficult. This paper gives a new corrected definition of a layered bottleneck, and develops a framework for systematic detection of the source of a bottleneck, for applying improvements and for estimating their effectiveness. Many of the techniques are specific to layered bottlenecks

Performance of multi-level client-server systems with parallel service operations

l’arcrllel extcufzon can enhance the performance of dzsfrzhufed claen,f-server systems, but the enhancement ma,y he less than expected. Evaluatzons of such deszgns must rn,clude the complex eflects of overheads, heterogeneous parallel branch,es, contention by the parallel parts for servers in lower levels, and simultaneous resource possession effects. A “compensated complelnentary delay” approximation is described which explozis layered queuean,g approEzmations for layered re.sources tuhach occur in client-server architectures, based on synchronazation dela,y estimates and adjusted levels of conieniion. The new approximataon uses the overlap of parallel branches and a new fast calculation of join drlalys. II gives acceptable errors (averaging about two percent), and has an enormously lower computational cost compared to the competang approach based on decompositzon. The new approximation may moderate a concluszon made by Heidelberger and Trivedi, that decomposition gives greatly superior accuracy to the much cheaper complementary delay approach; these delay approximations are only a little less accurate.

Automatic generation of layered queuing software performance models from commonly available traces

Performance models of software designs can give early warnings of problems such as resource saturation or excessive delays. However models are seldom used because of the considerable effort needed to construct them. Software Architecture and Model Extraction (SAME) is a lightweight model building technique that extracts communication patterns from executable designs or prototypes that use message passing, to develop a Layered Queuing Network model in an automated fashion. It is a formal, traceable model building process. The transformation follows a series of well-defined transformation steps, from input domain, (an executable software design or the implementation of software itself) to output domain, a Layered Queuing Network (LQN) Performance model. The SAME technique is appropriate for a message passing distributed system where tasks interact by point-to-point communication. With SAME, the performance analyst can focus on the principles of software performance analysis rather than model building.

On the effect of traffic model to the performance evaluation of multicast protocols in MANET

While efforts have been made recently to evaluate the performance of multicasting protocols in mobile ad hoc networks (MANET) and to evaluate the capacity of ad hoc networks, very little is understood or known about the traffic characteristics of mobile ad hoc networks and its effect on the performance of the existing multicasting protocols. In this paper, traffic models for multicast protocols in MANET are outlined. A literature survey for multicast protocols in MANET is conducted. Classification for most MANET multicast protocols is presented. A research to evaluate the performance of the existing multicast protocols in MANET using self-similarity traffic model is proposed & justified

Efficient performance models for layered server systems with replicated servers and parallel behaviour

Capacity planning for large computer systems may require very large performance models, which are difficult or slow to solve. Layered queueing models solved by mean value analysis can be scaled to dozens of servers and hundreds of service classes, with large class populations, but this may not be enough. A common feature of planning models for large systems is structural repetition expressed through replicated subsystems, which can provide both scalability and reliability, and this replication can be exploited to scale the solution technique. A model has recently been described for symmetrically replicated layered servers, and their integration into the system, with a mean-value solution approximation. However, parallelism is often combined with replication; high-availability systems use parallel data-update operations on redundant replicas, to enhance reliability, and grid systems use parallel computations for scalability. This work extends the replicated layered server model to systems with parallel execution paths. Different servers may be replicated to different degrees, with different relationships between them. The solution time is insensitive to the number of replicas of each replicated server, so systems with thousands or even millions of servers can be modelled efficiently.

Solving layered queueing networks of large client-server systems with symmetric replication

Large distributed client-server systems often contain subsystems which are either identical to each other, or very nearly so, and this simplifies the system description for planning purposes. These replicated components and subsystems all have the same workload and performance parameters. It is known how to exploit this symmetry to simplify the solution of some kinds of performance models, using state aggregation in Markov Chains. This work considers the same problem for layered queueing models, using mean value analysis. The mean values are found for each group of replicas just once, and then are inserted appropriately into the solution of the system as a whole. An algorithm has been implemented in the Layered Queueing Network Solver (LQNS), including approximations to deal with interactions among the replicas, and is evaluated for accuracy and for efficiency. The resulting solver is insensitive (in time of solution) to the number of replicas in a group, and can efficiently calculate waiting times and throughputs for systems with tens of thousands of nodes and processes.

Performance modeling of systems using fair share scheduling with Layered Queueing Networks

Fair-share scheduling attempts to grant access to a resource based on the amount of ¿share¿ that a task possesses. It is widely used in places such as Internet routing, and recently, in the Linux kernel. Software performance engineering is concerned with creating responsive applications and often uses modeling to predict the behaviour of a system before the system is built. This work extends the Layered Queueing Network (LQN) performance model used to model distributed software systems by including hierarchical fair-share scheduling with both guarantees and caps. To exercise the model, the Completely Fair Scheduler, found in recent Linux kernels, is incorporated into PARASOL, the underlying simulation engine of the LQN simulator, lqsim. This simulator is then used to study the effects of fair-share scheduling on a multi-tier implementation of a building security system. The results here show that fair-share scheduling with guarantees is not sufficient when an application is layered into multiple tiers because of contention at lower layers in the system. Fair-share scheduling with caps must be used instead.