Gabriel Mateescu

Quality of Service on the Grid Via Metascheduling with Resource Co-Scheduling and Co-Reservation

Assuring predictable resources (processors, memory, storage) for applications running on the Grid is a critical factor for the success of the Grid for solving real-life problems. We extend the Globus Resource Management Architecture to provide metascheduling, co-scheduling and advance co-reservation of resources, and compare the proposed architecture with existing metascheduling solutions.

GridX1: A Canadian computational grid

The present paper discusses the design and application of GridX1, a computational grid project which uses shared resources at several Canadian research institutions. The infrastructure of GridX1 is built using off-the-shelf Globus Toolkit 2 middleware, a MyProxy credential server, and a resource broker based on Condor-G to manage the distributed computing environment. The broker-based job scheduling and management functionality are exposed as a Globus GRAM job service. Resource brokering is based on the Condor matchmaking mechanism, whereby job and resource attributes are expressed as ClassAds, with the attributes Requirements and Rank being used to define respectively the constraints and preferences that the matched entity must meet. Various strategies for ranking resources are presented, including an Estimated-Waiting-Time (EWT) algorithm, a throttled load balancing strategy, and a novel external ranking strategy based on data location. One of the unique features is a mechanism which transparently presents the GridX1 resources as a single compute element to the LHC Computing Grid (LCG), based at the CERN Laboratory in Geneva. This interface was used during the ATLAS data challenge 2 to federate the Canadian resources into the LCG without the overhead of maintaining separate LCG sites. Further, the BaBar particle physics simulation has been adapted to execute on GridX1 and resulted in a simplified management of the production. The usage of the throttled EWT and load balancing strategies combined with external data ranking was found to be very effective in improving efficiency and reducing the job failure rate.

A method for MPI broadcast in computational grids

For large scale computational grids, where the resources are distributed over areas spanning thousands of miles, achieving efficiency of collective communication operations such as broadcast becomes of paramount importance. We propose a broadcast algorithm constructed in terms of point-to-point communication operations that occur according to a topology determined using a generalization of the single source shortest path algorithm such that the point-to-point operations are ordered according to a heuristic. We show that the proposed approach is competitive with, and in some cases exceeds, the performance of the broadcast operation implemented in MPICH-G2, the most used grid-enabled implementation of MPI.

Time Series Model Mining with Similarity-Based Neuro-Fuzzy Networks and Genetic Algorithms: A Parallel Implementation

This paper presents a parallel implementation of a hybrid data mining technique for multivariate heterogeneous time varying processes based on a combination of neuro-fuzzy techniques and genetic algorithms. The purpose is to discover patterns of dependency in general multivariate time-varying systems, and to construct a suitable representation for the function expressing those dependencies. The patterns of dependency are represented by multivariate, non-linear, autoregressive models. Given a set of time series, the models relate future values of one target series with past values of all such series, including itself. The model space is explored with a genetic algorithm, whereas the functional approximation is constructed with a similarity based neuro-fuzzy heterogeneous network. This approach allows rapid prototyping of interesting interdependencies, especially in poorly known complex multivariate processes. This method contains a high degree of parallelism at different levels of granularity, which can be exploited when designing distributed implementations, such as workcrew computation in a master-slave paradigm. In the present paper, a first implementation at the highest granularity level is presented. The implementation was tested for performance and portability in different homogeneous and heterogeneous Beowulf clusters with satisfactory results. An application example with a known time series problem is presented.

Remote services of spectroscopy instruments using grid computing

Grid computing supports sharing of distributed resources across boundaries and authorization domains, and can help scientists by providing secure, transparent and easy access to computing resources. We have designed and developed grid-based services for accessing, visualizing, and reliably manipulating remote spectrometry instruments. We harness grid-computing technologies to allow scientists to concentrate on performing experiments and data analysis, without having to become experts in the software technologies enabling these activities.

BaBar MC production on the Canadian grid using a web services approach

The present paper highlights the approach used to design and implement a web services based BaBar Monte Carlo (MC) production grid using Globus Toolkit version 4. The grid integrates the resources of two clusters at the University of Victoria, using the ClassAd mechanism provided by the Condor-G metascheduler. Each cluster uses the Portable Batch System (PBS) as its local resource management system (LRMS). Resource brokering is provided by the Condor matchmaking process, whereby the job and resource attributes are expressed as ClassAds. The important features of the grid are automatic registering of resource ClassAds to the central registry, ClassAds extraction from the registry to the metascheduler for matchmaking, and the incorporation of input/output file staging. Web-based monitoring is employed to track the status of grid resources and the jobs for an efficient operation of the grid. The performance of this new grid for BaBar jobs, and the existing Canadian computational grid (GridX1) based on Globus Toolkit version 2 is found to be consistent.

Parallel sorting on heterogeneous platforms

We present a method for load balancing parallel sorting on heterogeneous networks of workstations and clusters. Load balancing is achieved by exploiting information about the available throughput of the processors. First, the problem is partitioned into subproblems such that the times taken by the processors to solve the subproblems are balanced. Determining the partition involves solving a nonlinear system for finding the subproblem sizes. Second, the data are sorted by each process and are merged by choosing a processor topology which minimizes the critical path.

The GridX1 computational Grid: from a set of service-specific protocols to a service-oriented approach

GridXl is a computational grid designed and built to link resources at a number of research institutions across Canada. Building upon the experience of designing, deploying and operating the first generation of GridXl, we have designed a second-generation, Web-services-based, computational grid. The second generation of GridXl leverages the Web services resource framework, implemented by the Globus Toolkit version 4. The value added by GridXl includes metascheduling, file staging, resource registry and resource monitoring.

A DYNAMIC SPACE SHARING METHOD FOR RESOURCE MANAGEMENT

The continuously increasing demand for computation resources is being met by clusters of workstations and servers, as well as distributed or shared memory supercomputers. The usage of this infrastructure must meet a dual objective: optimization of resource utilization, and guarantee of quality of service to users. Batch scheduling methods based on space sharing and static partitioning have limited performance scalability with respect to the number of users.

IMAGE: an approach to building standards-based enterprise grids

We describe a system for aggregating heterogeneous resources from distinct administrative domains into an enterprise-wide compute grid, such that the aggregated resource provides the services of reliable and flexible queuing, scheduling, execution, and monitoring of batch applications. The system provides scheduling across multiple cluster grids, user account mapping across domains, and file staging, thereby enabling the consolidation of organization-wide distributed resources into a virtual resource, while preserving local control of resources. The concept of abstract queue, as the unit of aggregating heterogeneous resources, is introduced and instantiated for distributed resource scheduling. The proposed system is an open source, standards-based alternative to similar commercial systems