Kenneth R. Mayes

Towards performance control on the Grid

Advances in computational Grid technologies are enabling the development of simulations of complex biological and physical systems. Such simulations can be assembled from separate components—separately deployable computation units of well-defined functionality. Such an assemblage can represent an application composed of interacting simulations or might comprise multiple instances of a simulation executing together, each running with different simulation parameters. However, such assemblages need the ability to cope with heterogeneous and dynamically changing execution environments, particularly where such changes can affect performance. This paper describes the design and implementation of a prototype performance control system (PerCo), which is capable of monitoring the progress of simulations and redeploying them so as to optimize performance. The ability to control performance by redeployment is demonstrated using an assemblage of lattice Boltzmann simulations running with and without control policies. The cost of using PerCo is evaluated and it is shown that PerCo is able to reduce overall execution time.

Performance control of scientific coupled models in Grid environments

In recent years, there has been increasing interest in the development of computer simulations of complex biological systems, and of multi‐physics and multi‐scale physical phenomena. Applications have been developed that involve the coupling together of separate executable models of individual systems, where these models may have been developed in isolation. A lightweight yet general solution is required to problems of linking coupled models, and of handling the incompatibilities between interacting models that arise from their diverse origins and natures. Many such models require high‐performance computers to provide acceptable execution times, and there is increasing interest in utilizing Grid technologies. However, Grid applications need the ability to cope with heterogeneous and dynamically changing execution environments, particularly where run‐time changes can affect application performance. A general coupling framework (GCF) is described that allows the construction of flexible coupled models. This approach results in a component‐based implementation of a coupled model application. A semi‐formal presentation of GCF is given. Components under GCF are separately deployable and coupled by simple data flows, making them appropriate structures for dynamic execution platforms such as the Grid. The design and initial implementation of a performance control system (PERCO) is reviewed. PERCO acts by redeploying components, and is thus appropriate for controlling GCF coupled model applications. Redeployment decisions in PERCO require performance prediction capabilities. A proof‐of‐concept performance prediction algorithm is presented, based on the descriptions of GCF and PERCO. Copyright

Levels of atomic action in the Flagship parallel system

The Flagship system is a graph reduction machine having a distributed physical architecture. Although Flagship sits firmly in the declarative world, explicit state is supported to express the behaviour of the operating system. This state not only has to be isolated from the declarative aspects of the Flagship machine, but also has to be supported with respect to distribution. The mechanisms provided for maintaining consistency of state are discussed with respect to atomic actions at levels in the Flagship system. This approach is used to demonstrate how the software environment was supported by the basic execution mechanism of the machine. The nature, construction and creation of system software components are described and the structure of the system software is discussed with particular reference to optimising access to distributed resources.

Adaptive performance control for distributed scientific coupled models

The PerCo performance control framework is capable of managing the distributed execution of scientific coupled models using migration, for example, in response to changes in an execution environment. PerCo monitors execution times and reacts according to an adaptive performance control strategy whenever serious changes of behaviour occur. A computationally cheap technique is used per model to smooth the series of monitored execution times and to provide a short-term forecast for future execution times on currently assigned resources. Where this short-term forecast fails to be achieved, the system analyses whether migration would improve matters. For models that are candidates for migration, more accurate but computationally expensive techniques are used to form a longer-term prediction of future execution times on various candidate resources. Based on the predicted gain, a migration decision is made taking account of the expected cost of migration. Experimental results for small real scientific coupled models show that the performance control strategy behaves effectively in scenarios in which the ambient load is varied during execution.

Autonomous performance control of distributed applications in a heterogeneous environment

A framework is proposed that dynamically adapts to resource changes in a distributed heterogeneous environment. In this framework, computational tasks are wrapped into autonomous entities which are able to control themselves locally. Global control is provided in a decentralised manner via control units which link with these local entities in hierarchies, monitor them and coordinate their behaviour. With these mechanisms, the framework controls performance of a distributed application in a heterogeneous environment by adjusting load balance and adapting to resource changes. Fault tolerance is provided, being viewed as a special case of performance loss. Mixed strategies are applied, including global and local control policies, and their benefits are illustrated in terms of scalability and efficiency.

Resource Management on a Packet-Based Parallel Graph Reduction Machine

This paper is concerned with resource management on Flagship: a packet-based parallel graph reduction machine specifically designed to support the efficient execution of functional languages. Much of the system software of the machine was implemented above the computational model. The major concerns of the system software were the sharing of resources in a way that is compatible with a declarative approach, and to enable the distributed resources of the machine to be used in an efficient manner whilst maintaining consistency of these resources. The paper concentrates on describing how the kernel was structured into a set of resource managers. The resource managers were in turn structured in order to optimise access to distributed resources.

Collaboration and composition: issues for a second generation process language

Over the past decade a variety of process languages have been defined and applied to software engineering environments. The idea of using a process language to encode a software process as a “process model”, and enacting this using a process-sensitive environment is now well established. Many prototype process-sensitive environments have been developed; but their use in earnest has been limited. We are designing a second generation process language which is a significant departure from current conventional thinking. Firstly a process is viewed as a set of mediated collaborations rather than a set of partially ordered activities. Secondly emphasis is given to how process models are developed, used, and enhanced over a potentially long lifetime. In particular the issue of composing both new and existing model fragments is central to our development approach. This paper outlines these features, and gives the motivations behind them. It also presents a view of process support for software engineering drawing on our decade of experience in exploiting a “first generation” process language, and our experience in designing and exploiting programming languages.