Ian Willers
CERN
Network
Latest external collaboration on country level. Dive into details by clicking on the dots.
Publication
Featured researches published by Ian Willers.
Journal of Grid Computing | 2007
Richard McClatchey; Ashiq Anjum; Heinz Stockinger; Arshad Ali; Ian Willers; M. Thomas
In Grids scheduling decisions are often made on the basis of jobs being either data or computation intensive: in data intensive situations jobs may be pushed to the data and in computation intensive situations data may be pulled to the jobs. This kind of scheduling, in which there is no consideration of network characteristics, can lead to performance degradation in a Grid environment and may result in large processing queues and job execution delays due to site overloads. In this paper we describe a Data Intensive and Network Aware (DIANA) meta-scheduling approach, which takes into account data, processing power and network characteristics when making scheduling decisions across multiple sites. Through a practical implementation on a Grid testbed, we demonstrate that queue and execution times of data-intensive jobs can be significantly improved when we introduce our proposed DIANA scheduler. The basic scheduling decisions are dictated by a weighting factor for each potential target location which is a calculated function of network characteristics, processing cycles and data location and size. The job scheduler provides a global ranking of the computing resources and then selects an optimal one on the basis of this overall access and execution cost. The DIANA approach considers the Grid as a combination of active network elements and takes network characteristics as a first class criterion in the scheduling decision matrix along with computations and data. The scheduler can then make informed decisions by taking into account the changing state of the network, locality and size of the data and the pool of available processing cycles.
euromicro workshop on parallel and distributed processing | 2001
Heinz Stockinger; Kurt Stockinger; Erich Schikuta; Ian Willers
Large, Petabyte-scale data stores need detailed design considerations about distributing and replicating particular parts of the data store in a cost-effective way. Technical issues need to be analysed and, based on these constraints, an optimisation problem can be formulated. In this paper we provide a novel cost model for building a world-wide distributed Petabyte data store which will be in place starting from 2005 at CERN and its collaborating, world-wide distributed institutes. We elaborate on a framework for assessing potential system costs and influences which are essential for the design of the data store.
international database engineering and applications symposium | 1998
Koen Holtman; P.D.V. van der Stok; Ian Willers
In the very large object database systems planned for some future particle physics experiments, typical physics analysis jobs will traverse millions of read-only objects, many more objects than fit in the database cache. Thus, a good clustering of objects on disk is highly critical to database performance. We present the implementation and performance measurements of a prototype reclustering mechanism which was developed to optimise I/O performance under the changing access patterns in a high energy physics database. Reclustering is done automatically and on-line. The methods used by our prototype differ greatly from those commonly found in proposed general-purpose reclustering systems. By exploiting some special characteristics of the access patterns of physics analysis jobs, the prototype manages to keep database I/O throughput close to the optimum throughput of raw sequential disk access.
IEEE Transactions on Nuclear Science | 2006
Ashiq Anjum; Richard McClatchey; Arshad Ali; Ian Willers
Results from the research and development of a Data Intensive and Network Aware (DIANA) scheduling engine, to be used primarily for data intensive sciences such as physics analysis, are described. In Grid analyses, tasks can involve thousands of computing, data handling, and network resources. The central problem in the scheduling of these resources is the coordinated management of computation and data at multiple locations and not just data replication or movement. However, this can prove to be a rather costly operation and efficient scheduling can be a challenge if compute and data resources are mapped without considering network costs. We have implemented an adaptive algorithm within the so-called DIANA Scheduler which takes into account data location and size, network performance and computation capability in order to enable efficient global scheduling. DIANA is a performance-aware and economy-guided Meta Scheduler. It iteratively allocates each job to the site that is most likely to produce the best performance as well as optimizing the global queue for any remaining jobs. Therefore, it is equally suitable whether a single job is being submitted or bulk scheduling is being performed. Results indicate that considerable performance improvements can be gained by adopting the DIANA scheduling approach
Electronic Notes in Theoretical Computer Science | 2007
Mohammad Waseem Hassan; Richard McClatchey; Ian Willers
Centrally managed, traditional security systems put limits on collaborative activities among huge number of entities in current open networks (such as Grids). This requires new approaches to handling security in large distributed systems and the need for new research especially in areas concerned with the provision of security through collaboration. This paper presents the design of a large-scale, self-managing Trust Management Framework (TMF) that makes efficient use of apparently invisible evidences that are scattered across potentially global networks. The TMFs design dictates a layered architecture for capturing evidence at the data layer of a network, transforming it into formed reputations in the information layer and utilizing these reputations to determine trustworthiness of an entity in the knowledge layer of the network. In essence, the main focus of the proposed work is to automate the acquisition of scattered evidence and the formulation, evolution and dissemination of reputations in a scalable way in order to make improved security decisions.
international conference on conceptual modeling | 1998
Florida Estrella; Zsolt Kovacs; Jean-Marie Le Goff; Richard McClatchey; Ian Willers
Large scale engineering and scientific projects demand product and workflow management which may require integration and/or distribution over many separate organisations. The integration of such ‘islands of information’, which ultimately forms the basis of so-called ‘virtual enterprises’, is heavily dependent on the flexibility and accessibility of the data model describing the enterprise’s repository. The model must provide interoperability and reusability so that a range of applications can access the enterprise data. Making the repository self-describing ensures that knowledge about the repository structure is available for applications to interrogate and to navigate around for the extraction of application-specific data. Herein a large application is described which uses a meta-object based repository to capture product and workflow data in an engineering data warehouse. It is shown that adopting a meta-object approach to repository design provides support for interoperability and a suitable environment on which to build data mining applications.
international conference on e science | 2006
Ashiq Anjum; Richard McClatchey; Heinz Stockinger; Arshad Ali; Ian Willers; M. Thomas; Muhammad Sagheer; Khawar Hasham; Omer Alvi
The use of meta-schedulers for resource management in large-scale distributed systems often leads to a hierarchy of schedulers. In this paper, we discuss why existing meta-scheduling hierarchies are sometimes not sufficient for Grid systems due to their inability to re-organise jobs already scheduled locally. Such a job re-organisation is required to adapt to evolving loads which are common in heavily used Grid infrastructures. We propose a peer-topeer scheduling model and evaluate it using case studies and mathematical modelling. We detail the DIANA (Data Intensive and Network Aware) scheduling algorithm and its queue management system for coping with the load distribution and for supporting bulk job scheduling. We demonstrate that such a system is beneficial for dynamic, distributed and self-organizing resource management and can assist in optimizing load or job distribution in complex Grid infrastructures.
international conference on parallel processing | 2005
Arshad Ali; Ashiq Anjum; Tahir Azim; J. Bunn; A. Mehmood; Richard McClatchey; Harvey B Newman; W. ur Rehman; Conrad Steenberg; M. Thomas; F. van Lingen; Ian Willers; M.A. Zafar
Selecting optimal resources for submitting jobs on a computational grid or accessing data from a data grid is one of the most important tasks of any grid middleware. Most modern grid software today satisfies this responsibility and gives a best-effort performance to solve this problem. Almost all decisions regarding scheduling and data access are made by the software automatically, giving users little or no control over the entire process. To solve this problem, a more interactive set of services and middleware is desired that provides users more information about grid weather, and gives them more control over the decision making process. This paper presents a set of services that have been developed to provide more interactive resource management capabilities within the grid analysis environment (GAE) being developed collaboratively by Caltech, NUST and several other institutes. These include a steering service, a job monitoring service and an estimator service that have been designed and written using a common grid-enabled Web services framework named Clarens. The paper also presents a performance analysis of the developed services to show that they have indeed resulted in a more interactive and powerful system for user-centric grid-enabled physics analysis.
grid and cooperative computing | 2004
Naveed Ahmad; Arshad Ali; Ashiq Anjum; Tahir Azim; J. Bunn; Ali Hassan; Ahsan Ikram; Frank van Lingen; Richard McClatchey; Harvey B Newman; Conrad Steenberg; M. Thomas; Ian Willers
Handheld devices, while growing rapidly, are inherently constrained and lack the capability of executing resource hungry applications. This paper presents the design and implementation of distributed analysis and load-balancing system for hand-held devices using multi-agents system. This system enables low resource mobile handheld devices to act as potential clients for Grid enabled applications and analysis environments. We propose a system, in which mobile agents will transport, schedule, execute and return results for heavy computational jobs submitted by handheld devices. Moreover, in this way, our system provides high throughput computing environment for hand-held devices.
Archive | 2004
Arshad Ali; Ashiq Anjum; J. Bunn; Richard Cavanaugh; Frank van Lingen; Richard McClatchey; Harvey B Newman; Wahas ur Rehman; Conrad Steenberg; M. Thomas; Ian Willers
The grid is emerging as a great computational resource but its dynamic behavior makes the Grid environment unpredictable. Systems and networks can fail, and the introduction of more users can result in resource starvation. Once a job has been submitted for execution on the grid, monitoring becomes essential for a user to see that the job is completed in an efficient way, and to detect any problems that occur while the job is running. In current environments once a user submits a job he loses direct control over the job and the system behaves like a batch system: the user submits the job and later gets a result back. The only information a user can obtain about a job is whether it is scheduled, running, cancelled or finished. Today users are becoming increasingly interested in such analysis grid environments in which they can check the progress of the job, obtain intermediate results, terminate the job based on the progress of job or intermediate results, steer the job to other nodes to achieve better performance and check the resources consumed by the job. In order to fulfill their requirements of interactivity a mechanism is needed that can provide the user with real time access to information about different attributes of a job. In this paper we present the design of a Job Monitoring Service, a web service that will provide interactive remote job monitoring by allowing users to access different attributes of a job once it has been submitted to the interactive Grid Analysis Environment.