Adil Hasan

Rule-based curation and preservation of data: A data grid approach using iRODS

Research is generating large quantities of digital material, much of it irreplaceable, and there is a pressing need to maintain long-term access to it. Not only is the quantity of data growing in size, it is becoming much more diverse and complex, significantly complicating the issues around its curation. Automation of curation is key if a scalable solution is to be found. We describe an approach to automation in which digital curation policies and strategies are represented as rules, which are implemented in data grids based on the iRODS middleware.

Management and preservation of research data with iRODS

This paper presents first steps towards implementing a data layer to support a semi-automated preservation management system for research data in the arts and humanities. We suggest to use e-Science technology and grid middleware to implement a virtualised storage system for research data. We will outline how iRODS (Rule-Oriented Data management System)can be used within an architecture to implement complex,automated,scalable digital preservation strategies.

Applying Rules as Policies for Large-Scale Data Sharing

Large scientific projects need collaborative data sharing environments. For projects like the Ocean Observations Initiative (OOI), the Temporal Dynamics of Learning Center (TDLC) and Large-scale Synoptic Survey Telescope (LSST) the amount of data collected will be on the order of Petabytes, stored across distributed heterogeneous resources under multiple administrative organizations. Policy-oriented data management is essential in such collaborations. The integrated Rule-Oriented Data System (iRODS) is a peer-to-peer, federated server-client architecture that uses a distributed rule engine for data management to apply policies encoded as rules. The rules are triggered on data management events (ingestion, access, modifications, annotations, format conversion, etc) as well as periodically (to check integrity of the data collections, intelligent data archiving and placement, load balancing, etc). Rules are applied by system administrators (e.g. for resource creation, user management, etc.) and by individual users, groups and data providers to tailor the sharing and access of data for their own needs. In this paper, we will discuss the architecture of the iRODS middleware system and discuss some of the applications of the software.

Managing Provenance in iRODS

Nowadays provenance is an important issue. Provenance data does not only give a history of events, it also provides enough information to allow the opportunity to verify the authenticity of the data, as well as, determine the quality of the data. The data grid management system, iRODS, comes with metadata which can be used as provenance data. Currently, iRODSs metadata is not sufficient for tracking and reconstructing procedures applied to data. In this paper, we describe the provenance needs of iRODS and we survey briefly current provenance and provenance enabled workflow systems. We describe an architecture that can be used to manage provenance in iRODS (and other systems) in a fault-tolerant way.

A method for reliably managing files with RNS in multi Data Grids

Abstract This paper describes a method for reliably managing files distributed in different kinds of Data Grids with RNS (Resource Namespace Service). RNS provides hierarchical namespace management for name-to-resource mapping as a key technology when using Grid resources for different kinds of middleware. We define attribute expressions in XML for the RNS entries and give algorithms to access distributed files stored within different kinds of Data Grids. The volume of digital data and the size of an individual file are increasing due to the introduction of high-resolution images, high-definition audiovisual files, etc. The reliable storage of such large files is becoming problematic with whole file replication as a failure in the integrity of the file is difficult to localise. Our method involves managing large files in Data Grids by splitting them into smaller units in a traceable manner and then managing the smaller units. The RNS catalog service contains EPR (Endpoint Reference) and metadata that describe the original locations as well as the checksum values. The example in this paper shows how our Grid application can retrieve the actual file locations and the checksum values from the RNS service via SAGA (A Simple API for Grid Applications). An application can access the distributed files as though they were files in the local file-system without worrying about the underlying Data Grids. This approach can be used with various Data Grid systems to enhance file reliability.

Digital Library Storage using iRODS Data Grids

Digital repository software provides a powerful and flexible infrastructure for managing and delivering complex digital resources and metadata. However, issues can arise in managing the very large, distributed data files that may constitute these resources. This paper describes an implementation approach that combines the Fedora digital repository software with a storage layer implemented as a data grid, using the iRODS middleware developed by DICE (Data Intensive Cyber Environments) as the successor to SRB. This approach allows us to use Fedoras flexible architecture to manage the structure of resources and to provide application- layer services to users. The grid-based storage layer provides efficient support for managing and processing the underlying distributed data objects, which may be very large (e.g. audio-visual material). The Rule Engine built into iRODS is used to integrate complex workflows at the data level that need not be visible to users, e.g. digital preservation functionality.

Randomised Controlled Clinical Trial Investigating The Impact of Implementation Planning on Behaviour Related to The Diet

There is a perceived gap between dietary advice given by health practitioners and adherence to the advice by patients. We investigated whether a behaviour change technique (implementation-planning) was more effective than standard-of-care diet advice at reducing dietary acid intake using quantitative erosive tooth wear progression as an objective clinical outcome. This study was a randomised controlled, double-blind, single-centre clinical trial in the UK. Participants (n = 60) with high dietary acid intake (≥2 daily), were recruited and randomly assigned (1:1) to receive either implementation-planning or standard-of-care diet advice in a single clinical session. Questionnaires and impressions were taken at baseline and 6 months later. Dental casts were scanned using laser profilometry and superimposed using surface-matching software. Data were analysed per protocol and intention-to-treat using independent t-tests and Mann-Whitney tests. The intervention group reduced their dietary acid intake between meals to 1 intake per day compared to 2 intakes per day for the controls and demonstrated reduced dental hard tissue volume loss (−0.00 mm3 (SD = 0.01)) compared to controls (−0.07 mm3 (SD 0.17)), p = 0.049. This paper supports the use of implementation planning in clinical practice and presents a non-invasive method of intervention assessment in behaviour change. Larger trials are needed to confirm the generalisability of results.

SCALA: A framework for graphical operations for iRODS

Statistical Charts And Log Analyzer (SCALA) is a framework for graphically displaying operational data for a distributed data management system. The framework has been applied to the integrated Rule Oriented Data Management System (iRODS), but can be applied to any data management system. The framework allows operational information such as disk usage, number of users etc to be displayed. In addition SCALA also allows remote debugging through the discovery and display of iRODS error messages from log files. This makes the detection and debugging of errors in a running iRODS system simpler. The system has also been extended to display iRODS rules and their execution. This SCALA system is currently being used in the KEK production iRODS system. This paper describes the SCALA framework and it’s application to the iRODS.

Managing Cloud Images

The cloud offers scientists a lower barrier to use of the system. But, scientists using cloud middleware are faced with the having to create, manage and deploy the virtual images on the cloud infrastructure. In this paper we describe an application that uses the IBM Image Construction and Composition Tool (ICCT) to capture and create images. We describe the integration with the iRODS data management system that provides management allowing the user to keep track of which images are currently used and to switch between instances.

Capturing User-Generated Metadata

A key component of the management of digital data is the collection of metadata. Metadata is defined as data about data. It encompasses the description of the data and is essential in using or reusing the data. Metadata is often collected at the time the data are produced, but little attention is paid to the metadata that is created through the use or attempted use of the data. In this paper we propose an approach to enhance data management through the collection and management of metadata produced during the use or reuse of the data. The paper also describes an example of the approach applied to the iRODS data management system.