Fran Berman

Application-Level Scheduling on Distributed Heterogeneous Networks

Heterogeneous networks are increasingly being used as platforms for resource-intensive distributed parallel applications. A critical contributor to the performance of such applications is the scheduling of constituent application tasks on the network. Since often the distributed resources cannot be brought under the control of a single global scheduler, the application must be scheduled by the user. To obtain the best performance, the user must take into account both application-specific and dynamic system information in developing a schedule which meets his or her performance criteria. In this paper, we define a set of principles underlying application-level scheduling and describe our work-in-progress building AppLeS (application-level scheduling) agents. We illustrate the application-level scheduling approach with a detailed description and results for a distributed 2D Jacobi application on two production heterogeneous platforms.

New grid scheduling and rescheduling methods in the GrADS project

The goal of the Grid Application Development Software (GrADS) Project is to provide programming tools and an execution environment to ease program development for the Grid. This paper presents recent extensions to the GrADS software framework: a new approach to scheduling workflow computations, applied to a 3-D image reconstruction application; a simple stop/migrate/restart approach to rescheduling Grid applications, applied to a QR factorization benchmark; and a process-swapping approach to rescheduling, applied to an N-body simulation. Experiments validating these methods were carried out on both the GrADS MacroGrid (a small but functional Grid) and the MicroGrid (a controlled emulation of the Grid).

Combining workstations and supercomputers to support grid applications: the parallel tomography experience

Computational grids are becoming an increasingly important and powerful platform for the execution of large-scale, resource-intensive applications. However, it remains a challenge for applications to tap into the potential of grid resources in order to achieve performance. In this paper, we illustrate how work queue applications can leverage grids to achieve performance through coallocation. We describe our experiences in developing a scheduling strategy for a production tomography application targeted at grids that contain both workstations and parallel supercomputers. Our strategy uses dynamic information exported by a supercomputers batch scheduler to simultaneously schedule tasks on workstations and immediately-available supercomputer nodes. This strategy is of great practical interest because it combines resources that are available in a typical research laboratory: time-shared workstations and CPU time in remote space-shared supercomputers. We show that this strategy improves the performance of the tomography application compared to traditional scheduling strategies, which target the application to either type of resource alone.

Scheduling in the Grid application development software project

Developing Grid applications is a challenging endeavor that at the moment requires both extensive labor and expertise. The Grid Application Development Software Project (GrADS) provides a system to simplify Grid application development. This system incorporates tools at all stages of the application development and execution cycle. In this chapter we focus on application scheduling, and present the three scheduling approaches developed in GrADS: development of an initial application schedule (launch-time scheduling), modification of the execution platform during execution (rescheduling), and negotiation between multiple applications in the system (metascheduling). These approaches have been developed and evaluated for platforms that consist of distributed networks of shared workstations, and applied to real-world parallel applications.

Generalized planar matching

Abstract In this paper, we prove that maximum planar H-matching (the problem of determining the maximum number of node-disjoint copies of the fixed graph H contained in a variable planar graph G) is NP-complete for any connected planar graph H with three or more nodes. We also show that perfect planar H-matching is NP-complete for any connected outerplanar graph H with three or more nodes. The results generalize and unify several special-case results proved in the literature. The techniques can also be applied to determine the complexity of several problems, including the optimal tile salvage problem from wafer-scale integration and the classic dots and boxes game. Although we prove that the optimal tile salvage problem and others like it are NP-complete, we also describe provably good approximation algorithms that are suitable for practical applications.

The AppLeS Parameter Sweep Template: User-Level Middleware for the Grid

The Computational Grid is a promising platform for the efficient execution of parameter sweep applications over large parameter spaces. To achieve performance on the Grid, such applications must be scheduled so that shared data files are strategically placed to maximize re-use, and so that the application execution can adapt to the deliverable performance potential of target heterogeneous, distributed and shared resources. Parameter sweep applications are an important class of applications and would greatly benefit from the development of Grid middleware that embeds a scheduler for performance and targets Grid resources transparently. In this paper we describe a user-level Grid middleware project, the AppLeS Parameter Sweep Template (APST), that uses application-level scheduling techniques [1] and various Grid technologies to allow the efficient deployment of parameter sweep applications over the Grid. We discuss several possible scheduling algorithms and detail our software design. We then describe our current implementation of APST using systems like Globus [2], NetSolve [3] and the Network Weather Service [4], and present experimental results.

The encyclopedia of life project: grid software and deployment

The ongoing global effort of genome sequencing is making large scale comparative proteomic analysis an intriguing task. The Encyclopedia of Life (EOL; http://eol.sdsc.edu) project aims to provide current functional and structural annotations for all available proteomes, a computational challenge never seen before in biology. Using an integrative genome annotation pipeline (iGAP), we have produced 3D models and functional annotations for more than 100 proteomes thus far. This process is greatly facilitated by grid compute resources, and especially by the development of grid application execution environment. AppLeS (Application-Level Scheduling) Parameter Sweep Template (APST) has been adopted by the EOL project as a mediator to grid middleware. APST has made the annotation process much more efficient, highly automated and scalable. Currently we are building a domain-specific bioinformatics workflow management system (BWMS) on top of APST, which further streamlines grid deployment of life science applications. With these developments in mind, we discuss some common problems and expectations of grid computing for high throughput proteomics.

Chapter 2 – The Scientific Imperative

Publisher Summary This chapter discusses the revolutionary changes in technology and methodology driving scientific and engineering communities to embrace Grid technologies. Today, the scientific community still leads the way as early attempts in Grid computing evolve to the more sophisticated and ubiquitous “virtual organization.” The UK e-Science concept, the NSF vision of a global cyberinfrastructure, and NASAs IPG all recognize that—following a decade of pioneering work in computational science, data technologies, supercomputing, and networking linked with Grid technologies—computational and data management infrastructure has become a global phenomenon that is poised to evolve as a key enabler for science and society. The realization of this vision may require long-term investments of financial resources by governments and of intellectual resources by those who must build and apply the necessary global information infrastructure.

Logistical computing and internetworking: middleware for the use of storage in communication

The Logistical Computing and Internetworking (LoCI) project is a reflection of the way that the next generation internetworking fundamentally changes our definition of high performance wide area computing. A key to achieving this aim is the development of middleware that can provide reliable, flexible, scalable, and cost-effective delivery of data with quality of service (QoS) guarantees to support high performance applications of all types. The LoCI effort attacks this problem with a simple but innovative strategy. At the base of the LoCI project is a richer view of the use of storage in communication and information sharing.

Digital preservation lifecycle management for multi-media collections

Increasingly, intellectual content is “born digital.” In order to make it as easy as possible for content creators to preserve their content for the long-term, preservation processes should be integrated into the content production lifecycle. Our project takes an existing video production workflow and integrates it with a digital preservation life-cycle management process that will enable the digital content to be archived for long-term preservation. The collection, “Conversations with History,” is produced at the University of California, Berkeley, edited by University of California, San Diego–TV (UCSD-TV), and broadcast and Web-cast through UCTV. The proposed system will demonstrate an effective preservation methodology by demonstrating a standard reference model for digital preservation lifecycle management that can be integrated into active production workflows.