Aleš Křenek

Practical approaches to Grid workload and resource management in the EGEE project

Resource management and scheduling of distributed, data-driven applications in a Grid environment are challenging problems. Although significant results were achieved in the past few years, the development and the proper deployment of generic, reliable, standard components present issues that still need to be completely solved. Interested domains include workload management, resource discovery, resource matchmaking and brokering, accounting, authorization policies, resource access, reliability and dependability. The evolution towards a service-oriented architecture, supported by emerging standards, is another activity that will demand attention. All these issues are being tackled within the EU-funded EGEE project (Enabling Grids for E-science in Europe), whose primary goals are the provision of robust middleware components and the creation of a reliable and dependable Grid infrastructure to support e-Science applications. In this paper we present the plans and the preliminary activities aiming at providing adequate workload and resource management components, suitable to be deployed in a production-quality Grid.

The DataGrid Workload Management System: Challenges and Results

The workload management task of the DataGrid project was mandated to define and implement a suitable architecture for distributed scheduling and resource management in a Grid environment. The result was the design and implementation of a Grid Workload Management System, a super-scheduler with the distinguishing property of being able to take data access requirements into account when scheduling jobs to the available Grid resources. Many novel issues in various fields were faced such as resource management, resource reservation and co-allocation, Grid accounting. In this paper, the architecture and the functionality provided by the DataGrid Workload Management System are presented.

Genome assembly and annotation for red clover (Trifolium pratense; Fabaceae)

PREMISE OF THE STUDY Red clover (Trifolium pratense) is an important forage plant from the legume family with great importance in agronomy and livestock nourishment. Nevertheless, assembling its medium-sized genome presents a challenge, given current hardware and software possibilities. Next-generation sequencing technologies enable us to generate large amounts of sequence data at low cost. In this study, the genome assembly and red clover genome features are presented. METHODS First, assembly software was assessed using data sets from a closely related species to find the best possible combination of assembler plus error correction program to assemble the red clover genome. The newly sequenced genome was characterized by repetitive content, number of protein-coding and nonprotein-coding genes, and gene families and functions. Genome features were also compared with those of other sequenced plant species. KEY RESULTS Abyss with Echo correction was used for de novo assembly of the red clover genome. The presented assembly comprises ∼314.6 Mbp. In contrast to leguminous species with comparable genome sizes, the genome of T. pratense contains a larger repetitive portion and more abundant retrotransposons and DNA transposons. Overall, 47 398 protein-coding genes were annotated from 64 761 predicted genes. Comparative analysis revealed several gene families that are characteristic for T. pratense. Resistance genes, leghemoglobins, and nodule-specific cystein-rich peptides were identified and compared with other sequenced species. CONCLUSIONS The presented red clover genomic data constitute a resource for improvement through molecular breeding and for comparison to other sequenced plant species.

gLite job provenance

The Job Provenance (JP) service is designed to automate keeping track of computations on large scale Grids, giving thus users a tool to correctly archive information about their jobs and to re-submit any job in a reconstructed environment. JP provides a permanent minimal record of job (and its environment) related information, to which free-form user annotations can be added. JP also offers the capability of configuring any number of indexed logical views on the large collections of raw data, allowing efficient processing of even complex user queries selecting on both system data and the annotations. The scalable architecture, capable to handle millions of jobs in a single JP installation, and integrated into the EGEE gLite middleware environment is presented.

Distributed Tracking, Storage, and Re-use of Job State Information on the Grid

The Logging and Bookkeeping service tracks jobs passing through the Grid. It collects important events generated by both the grid middleware components and applications, and processes them at a chosen LB server to provide the job state. The events are transported through secure and reliable channels. Job tracking is fully distributed and does not depend on a single information source, the robustness is achieved through speculative job state computation in case of reordered, delayed or lost events. The state computation is easily adaptable to modified job control flow.

Real Time Monitor of Grid job executions

In this paper we describe the architecture and operation of the Real Time Monitor (RTM), developed by the Grid team in the HEP group at Imperial College London. This is arguably the most popular dissemination tool within the EGEE [1] Grid. Having been used, on many occasions including GridFest and LHC inauguration events held at CERN in October 2008. The RTM gathers information from EGEE sites hosting Logging and Bookkeeping (LB) services. Information is cached locally at a dedicated server at Imperial College London and made available for clients to use in near real time. The system consists of three main components: the RTM server, enquirer and an apache Web Server which is queried by clients. The RTM server queries the LB servers at fixed time intervals, collecting job related information and storing this in a local database. Job related data includes not only job state (i.e. Scheduled, Waiting, Running or Done) along with timing information but also other attributes such as Virtual Organization and Computing Element (CE) queue – if known. The job data stored in the RTM database is read by the enquirer every minute and converted to an XML format which is stored on a Web Server. This decouples the RTM server database from the client removing the bottleneck problem caused by many clients simultaneously accessing the database. This information can be visualized through either a 2D or 3D Java based client with live job data either being overlaid on to a 2 dimensional map of the world or rendered in 3 dimensions over a globe map using OpenGL.

Nonlinear vs. linear biasing in Trp-cage folding simulations

Biased simulations have great potential for the study of slow processes, including protein folding. Atomic motions in molecules are nonlinear, which suggests that simulations with enhanced sampling of collective motions traced by nonlinear dimensionality reduction methods may perform better than linear ones. In this study, we compare an unbiased folding simulation of the Trp-cage miniprotein with metadynamics simulations using both linear (principle component analysis) and nonlinear (Isomap) low dimensional embeddings as collective variables. Folding of the mini-protein was successfully simulated in 200 ns simulation with linear biasing and non-linear motion biasing. The folded state was correctly predicted as the free energy minimum in both simulations. We found that the advantage of linear motion biasing is that it can sample a larger conformational space, whereas the advantage of nonlinear motion biasing lies in slightly better resolution of the resulting free energy surface. In terms of sampling efficiency, both methods are comparable.

A uniform job monitoring service in multiple job universes

We describe an ongoing work of extending the gLite Logging and Bookkeeping (L&B) service to be able to track additional types of jobs, with the vision of being able to uniformly follow jobs on the Grid, even when they pass between different middleware domains. Details are given on the simpler case of PBS jobs, which prove the cababilityof L&B to deal with additional job types,as well as started more complex and challenging work on Condor jobs, where theimpact of eventual success is larger.

Haptic rendering of complex force fields

With a particular focus in scientific applications, we propose a method of haptic rendering of virtual environments which require rather complex calculations to evaluate their behaviour. Therefore it is not possible to perform the calculations within a haptic loop. We specify a class of problems for which the method is applicable, describe the designed data structure, present involved algorithms, and prove their critical properties. The general section is followed by a case study of applying the method in a specific application in computational chemistry. The paper is concluded with first practical results of deployment of the method as well as preliminary quantitative assessment.

Experimental evaluation of job provenance in ATLAS environment

Grid middleware stacks, including gLite, matured into the state of being able to process up to millions of jobs per day. Logging and Bookkeeping, the gLite job-tracking service, keeps pace with this rate; however, it is not designed to provide a long-term archive of information on executed jobs. ATLAS — representative of a large user community — addresses this issue with its own job catalogue (ProdDB). Development of such a customized service, not easily reusable, took considerable effort which is not affordable by smaller communities. On the contrary, Job Provenance (JP), a generic gLite service designed for long-term archiving of information on executed jobs focusing on scalability, extensibility, uniform data view, and configurability, allows more specialized catalogues to be easily built. We present the first results of an experimental JP deployment for the ATLAS production infrastructure where a JP installation was fed with a part of ATLAS jobs, and also stress tested with real production data. The main outcome of this work is a demonstration that JP can complement large-scale application-specific job catalogue services, while serving a similar purpose where there are none available.