Yudong Sun

Scalable load balancing on distributed web servers using mobile agents

Distributed web servers on the immensely expanding Internet require high scalability and availability to provide efficient services to millions of clients on the web. To provide rapid responses to enormous number of client requests, load balancing is an important technique to evenly distribute the requests to web servers. In this paper, we propose a framework called Mobile Agent based LoaD balancing (MALD) that uses mobile agents technology to implement scalable load balancing on distributed web servers. The web servers can dispatch mobile agents to collect system-wide load information and accomplish load redistribution on all servers. Various load-balancing policies can be incorporated with versatile mobile agents in the framework. Compared with the traditional message-passing-based load-balancing methods, the mobile-agent-based approaches have the merits of high flexibility, low network traffic and high asynchrony. The performance evaluation demonstrates that the MALD framework provides a foundation to develop efficient load-balancing schemes on wide range of web server systems from cluster to the Internet.

A taxonomy of application scheduling tools for high performance cluster computing

Application scheduling plays an important role in high-performance cluster computing. Application scheduling can be classified as job scheduling and task scheduling. This paper presents a survey on the software tools for the graph-based scheduling on cluster systems with the focus on task scheduling. The tasks of a parallel or distributed application can be properly scheduled onto multi-processors in order to optimize the performance of the program (e.g., execution time or resource utilization). In general, scheduling algorithms are designed based on the notion of task graph that represents the relationship of parallel tasks. The scheduling algorithms map the nodes of a graph to the processors in order to minimize overall execution time. Although many scheduling algorithms have been proposed in the literature, surprisingly not many practical tools can be found in practical use. After discussing the fundamental scheduling techniques, we propose a framework and taxonomy for the scheduling tools on clusters. Using this framework, the features of existing scheduling tools are analyzed and compared. We also discuss the important issues in improving the usability of the scheduling tools.

Dynamic configuration management in a graph-oriented distributed programming environment

Dynamic configuration is a desirable property of a distributed system where dynamic modification and extension to the system and the applications are required. It allows the system configuration to be specified and changed while the system is executing. This paper describes a software platform that facilitates a novel approach to the dynamically configurable programming of parallel and distributed applications and systems. This platform is based on a graph-oriented model and it provides support for constructing reconfigurable distributed programs. We describe the design and implementation of a dynamic configuration manager for the graph-oriented distributed programming environment. The requirements and services for dynamic reconfiguration are identified. The architectural design of a dynamic configuration manager is presented, and a parallel virtual machine-based prototypical implementation of the manager, on a local area network of workstations, is described.

Distributed particle simulation method on adaptive collaborative system

Abstract This paper presents a distributed N-body method based on an adaptive collaborative system model. The collaborative system is formed by the distributed objects on a distributed system. The system can be reconfigured during the computation to fully utilize the computing power of the networked hosts. The method is implemented in Java and RMI to support distributed computing in heterogeneous environment. A distributed tree structure is designed for communication-efficient computation of N-body method. The performance test shows satisfactory speedup and portability of the method on both homogeneous and heterogeneous clusters. The collaborative system model can be used in various applications and it is expandable to wide-area environment.

Exploring Microbial Genome Sequences to Identify Protein Families on the Grid

The analysis of microbial genome sequences can identify protein families that provide potential drug targets for new antibiotics. With the rapid accumulation of newly sequenced genomes, this analysis has become a computationally intensive and data-intensive problem. This paper describes the development of a Web-service-enabled, component-based, architecture to support the large-scale comparative analysis of complete microbial genome sequences and the subsequent identification of orthologues and protein families (Microbase). The system is coordinated through the use of Web-service-based notifications and integrates distributed computing resources together with genomic databases to realize all-against-all comparisons for a large volume of genome sequences and to present the data in a computationally amenable format through a Web service interface. We demonstrate the use of the system in searching for orthologues and candidate protein families, which ultimately could lead to the identification of potential therapeutic targets.

Graph Scaling: A Technique for Automating Program Construction and Deployment in ClusterGOP

Program development and resource management are critical issues in large-scaled parallel applications and they raise diffculties for the programmers. Automation tools can benefit the programmer by reducing the time and work required for programming, deploying, and managing parallel applications. In our previous work, we have developed a visual tool, VisualGOP, to help visual construction and automatic mapping of parallel programs to execute on the ClusterGOP platform, which provides a graph-oriented model and the environment for running the parallel applications on clusters. In VisualGOP, the programmer needs to manually build the task interaction graph. This may lead to scalability problem for large applications. In this paper, we propose a graph scaling approach that helps the programmer to develop and deploy a large-scale parallel application minimizing the effort of graph construction, task binding and program deployment. The graph scaling algorithms expand or reduce a task graph to match the specified scale of the program and the hardware architecture, e.g., the problem size, the number of processors and interconnection topology, so as to produce an automatic mapping. An example is used to illustrate the proposed approach and how programmer benefits in the automation tools.

A Distributed Object-Oriented Method for Particle Simulations on Clusters

This paper describes a distributed object-oriented method for solving N-body problem of particle simulations. The method allows dynamic construction of a collaborative system based on the computational requirement of an application and the available resources in the cluster. In the system, a group of objects on distributed hosts cooperate to execute the application. The method is implemented in Java and RMI. The platform-independent features of Java enable the method to support efficient distributed computing in heterogeneous environment. The performance test shows that the method can achieve good speedup and portability. The proposed method can be extended to support other scientific computing applications in distributed environment.

Solving irregularly structured problems based on distributed object model

This paper presents a distributed object model called MOIDE (multi-threading object-oriented infrastructure on distributed environment) for solving irregularly structured problems. The model creates an adaptive computing infrastructure for developing and executing irregular applications on distributed systems. The infrastructure allows dynamic reconfiguration to match the evolution of irregular computation and available system resources. A unified communication mechanism is built to integrate different communication paths on heterogeneous systems to support efficient communication. Autonomous load scheduling approach is proposed for dynamic load balancing. A runtime system is developed to implement MOIDE-based computing. Applications including N-body problem, ray tracing, and conjugate gradient are developed to demonstrate the advantages of the model.

GOP: A Graph-Oriented Programming Model for Parallel and Distributed Systems

The advances of parallel and distributed computing demand high-level programming models that support efficient software development and execution. Graphs can effectively represent the logical structures of distributed systems and applications so as to facilitate the programming of distributed applications and support efficient mapping of programs to hardware architecture. This chapter presents a Graph-Oriented Programming (GOP) model that provides flexible graph constructs and graph-oriented primitives to build a programming paradigm based on graph topology and also provides a formal specification of software architecture for distributed programs. The GOP model creates an abstract programming framework and supports dynamic reconfiguration of distributed computing system to implement adaptive computation and fault-tolerance. Various computing environments have been developed based on GOP for cluster computing, web service, and component-based computation.

A distributed object model for solving irregularly structured problems on cluster

This paper presents a distributed object model MOIDE (Multithreaded Object-oriented Infrastructure on Distributed Environment) for solving irregularly structured problems on cluster. The primary appeal of MOIDE is its flexible system structure that is adaptive to heterogeneous architecture of a cluster. MOIDE integrates the object-oriented and multithreaded methodologies to set up a unified computing environment. Both the shared-data access and remote messaging are incorporated in a two-layer communication mechanism for efficient inter-object communication with the common communication interface. MOIDE supports dynamic load balancing by its autonomous load scheduling technique. A runtime support system implements the MOIDE model as a platform-independent infrastructure for developing and executing irregularly structured applications. N-body, ray tracing, and conjugate gradient applications are implemented to illustrate the advantages of MOIDE model.