Pieter Thysebaert

A view on enabling-consumer oriented grids through optical burst switching

As grid computing continues to gain popularity in the research community, it also attracts more attention from the enterprise and consumer levels. Applications in these domains generate large amounts of jobs, with individual jobs having only modest resource requirements. In this article, a novel architecture to realize a highly scalable and flexible platform for consumer-oriented grids is proposed. The architecture is based on an optical burst switched network, complemented with an advanced control and signaling plane. The architecture, functionality, and interfaces of all the relevant entities are presented and issues, current initiatives, and future directions for the control and management of these grid networks are discussed.

Scalable dimensioning of resilient Lambda Grids

Grids consist of the aggregation of numerous dispersed computational, storage and network resources, able to satisfy even the most demanding computing jobs. Due to the data-intensive nature of Grid jobs, there is an increasing interest in Grids using optical transport networks as this technology allows for the timely delivery of large amounts of data. Such Grids are commonly referred to as Lambda Grids. An important aspect of Grid deployment is the allocation and activation of installed network capacity, needed to transfer data and jobs to and from remote resources. However, the exact nature of a Grids network traffic depends on the way arriving workload is scheduled over the various Grid sites. As Grids possibly feature high numbers of resources, jobs and users, solving the combined Grid network dimensioning and workload scheduling problem requires the use of scalable mathematical methods such as Divisible Load Theory (DLT). Lambda Grids feature additional complexity such as wavelength granularity and continuity or conversion constraints must be enforced. Additionally, Grid resources cannot be expected to be available at all times. Therefore, the extra complexity of resilience against possible resource failures must be taken into account when modelling the combined Grid network dimensioning and workload scheduling problem, enforcing the need for scalable solution methods. In this work, we tackle the Lambda Grid combined dimensioning and workload scheduling problem and incorporate single-resource failure or unavailability scenarios. We use Divisible Load Theory to tackle the scalability problem and compare non-resilient lambda Grid dimensioning to the dimensions needed to survive single-resource failures. We distinguish three failure scenarios relevant to lambda Grid deployment: computational element, network link and optical cross-connect failure. Using regular network topologies, we derive analytical bounds on the dimensioning cost. To validate these bounds, we present comparisons for the resulting Grid dimensions assuming a 2-tier Grid operation as a function of varying wavelength granularity, fiber/wavelength cost models, traffic demand asymmetry and Grid scheduling strategy for a specific set of optical transport networks.

Dimensioning and on-line scheduling in Lambda Grids using divisible load concepts

Abstract Due to the large amounts of data required to be processed by the typical Grid job, it is conceivable that the use of optical transport networks in Grid deployment (hence the term “Lambda Grid”) will increase. The exact topology of the interconnecting network is obtained by solving a dimensioning problem, and the outcome of this strongly depends on both the expected workload characteristics and Grid scheduling policy. Solving this combined scheduling and dimensioning problem using straightforward ILP modelling is cumbersome; however, for steady-state Grid operation, Divisible Load Theory (DLT) can yield scalable formulations of this problem. In this paper, the on-line hierarchical scheduling on a lambda Grid of workload approaching the Grid’s capacity in a two-tier Grid mode of operation is studied. A number of these algorithms are goal-driven, in the sense that target per-resource goals are obtained from the off-line solution to the Divisible Load model. We compare these on-line multiresource scheduling policies for different workloads, Grid interconnection topologies and Grid parameters. We show that these algorithms perform well in the studied scenarios when compared to a fully centralized scheduling algorithm.

Using Divisible Load Theory to Dimension Optical Transport Networks for Grid Excess Load Handling

An important aspect of grid deployment is the allocation and activation of installed network capacity. Due to the data-intensive nature of grid jobs, it is expected that optical transport networks will play an important role in grid deployment. As grids possibly consist of high numbers of re sources and users, solving the network dimensioning problem using straight forward integer linear programs (ILP) does not scale well with increasing number of jobs. There fore, we propose the use of divisible load theory (DLT) when modeling this OCS (with wavelength translation) dimensioning problem. We establish the suitability of this approach as an optical transport network dimensioning tool in an excess load scenario for varying wavelength granularity, fiber/wavelength cost models, network topology, traffic demand asymmetry and Grid scheduling strategy

Scalable dimensioning of optical transport networks for grid excess load handling

Grids consist of the aggregation of numerous dispersed computational and storage resources, able to satisfy even the most demanding computing jobs. An important aspect of Grid deployment is the allocation and activation of installed network capacity, needed to transfer data and jobs to and from remote resources. Due to the data-intensive nature of Grid jobs, it is expected that optical transport networks will play an important role in Grid deployment. As Grids possibly consist of high numbers of resources, and users, solving the network dimensioning problem (i.e. determining the number of wavelength channels per fiber and wavelength granularity required) using straightforward Integer Linear Programs (ILP) does not scale well with increasing number of jobs. Therefore, we propose the use of Divisible Load Theory (DLT) when modeling the OCS (with wavelength translation) dimensioning problem in this context. We compare this approach to both an exact ILP and heuristic (derived from the exact ILP) approach as a function of the job arrival process, network related parameters and the Grid job scheduling strategy on the Grid. Results show the convergence of the DLT-based and the exact ILP approach, which indicates that the DLT-based approach is of practical use in cases where the exact ILP-based problem becomes intractable. We study an excess load scenario and evaluate the network cost for varying wavelength granularity, fiber/wavelength cost models, network topology and traffic demand asymmetry under multiple Grid scheduling strategies. Results indicate the suitability of our DLT-based approach as an Optical Transport Network dimensioning tool to be used by network operators.

Using divisible load theory to dimension optical transport networks for computational grids

Using the notion of divisible load, this paper attempts to shed light on the operational cost of an optical transport network when used to connect different sites of a computational grid.

A distributed resource and network partitioning architecture for service grids

In this paper, we propose the use of a distributed service management architecture for state-of-the-art service-enabled grids. The architecture is capable of performing automated resource and network bandwidth partitioning based on registered grid resource properties and monitored grid service demand. A main characteristic is that it enables the use of different service priority schemes and allows for policy-based differentiation between local and foreign service offerings. Resource and network bandwidth partitioning algorithms are introduced and their performance is evaluated on a sample grid topology using NSGrid, an ns-2 based grid simulator. Our results show that the use of this Service Management Architecture improves resource efficiency, simplifies schedule making decisions, reduces the overall complexity of managing the grid system, and at the same time improves grid service QoS support (with regard to job response times) by automatically making grid resource and network service reservations prior to scheduling.

Flexible Grid service management through resource partitioning

In this paper, a distributed and scalable Grid service management architecture is presented. The proposed architecture is capable of monitoring task submission behaviour and deriving Grid service class characteristics, for use in performing automated computational, storage and network resource-to-service partitioning. This partitioning of Grid resources amongst service classes (each service class is assigned exclusive usage of a distinct subset of the available Grid resources), along with the dynamic deployment of Grid management components dedicated and tuned to the requirements of a particular service class introduces the concept of Virtual Private Grids. We present two distinct algorithmic approaches for the resource partitioning problem, the first based on Divisible Load Theory (DLT) and the second built on Genetic Algorithms (GA). The advantages and drawbacks of each approach are discussed and their performance is evaluated on a sample Grid topology using NSGrid, an ns-2 based Grid simulator. Results show that the use of this Service Management Architecture in combination with the proposed algorithms improves computational and network resource efficiency, simplifies schedule making decisions, reduces the overall complexity of managing the Grid system, and at the same time improves Grid QoS support (with regard to job response times) by automatically assigning Grid resources to the different service classes prior to scheduling.

Application-Specific Hints in Reconfigurable Grid Scheduling Algorithms

In this paper, we investigate the use of application-specific hints when scheduling jobs on a Computational Grid, as these jobs can expose widely differing characteristics regarding CPU and I/O requirements. Specifically, we consider hints that specify the relative importance of network and computational resources w.r.t. their influence on the associated application’s performance. Using our ns-2 based Grid Simulator (NSGrid), we compare schedules that were produced by taking application-specific hints into account to schedules produced by applying the same strategy for all jobs. The results show that better schedules can be obtained when using these scheduling hints intelligently.

Resource partitioning algorithms in a programmable service grid architecture

We propose the use of programmable Grid resource partitioning heuristics in the context of a distributed service Grid management architecture. The architecture is capable of performing automated and exclusive resource-to-service assignations based on Grid resource status/properties and monitored service demand. We present two distinct approaches for the partitioning problem, the first based on Divisible Load Theory and the second built on Genetic Algorithms. Advantages and drawbacks of each approach are discussed and their performance is evaluated using NSGrid. Results show that automated resource-to-service partitioning simplifies scheduling decisions, improves service QoS support and allows efficient computational/network resource usage.