Darin Nikolow

Access cost estimation for unified grid storage systems

In typical data grids large amounts of replicated data are stored all over the globe in different storage systems with access latencies ranging from seconds to hours. The task of a replica management system is not only to keep track of the replicas but also to select those replicas that can be accessed by an application program with a minimal response or transfer time. Most wide-area replication research focuses on network-based replica selection. However, our past experience with data grids has shown that often hierarchical storage systems are the main bottleneck rather than network links. This is due to the fact that access latencies of hierarchical storage systems can be of the order of seconds up to hours in case the data resides on a tape that is not mounted yet. We give an overview of our replica management framework called Reptor and a storage system cost estimator that is used. Furthermore, we give details on access estimation of file replicas that reside on hierarchical storage systems. The results show that the access estimates provide a good basis for a replica management system to perform efficient replica selection.

Implementation of replication methods in the grid environment

Data replication methods are often used in Grid environments for improving the access time and data protection level. Dynamic replication allows the system to automatically create and delete replicas in order to follow dynamically changing system parameters and user access patterns keeping the performance high and resource usage in reasonable limits. In this paper we present the replication aspects of the Virtual Storage System developed within one of the grid projects. We also present results from replication tests done in a real grid environment.

MMSRS - Multimedia Storage and Retrieval System for a Distributed Medical Information System

The Multimedia Storage and Retrieval System described in this paper is aimed at storing and retrieving medical multimedia data such as images and videos. There are three significant problems that automatically need to be addressed: providing enough capacity to fit Terabytes of MPEG files, efficient extraction of video fragments from MPEG files and quality of service issues. We are working on two different approaches to build MMSRS. The first one, described in this article, utilizes commercial HSM software for managing the tertiary storage hardware. The second approach that we have started to investigate is an attempt to build a specialised storage management system from scratch.

A TOOLKIT FOR STORAGE QOS PROVISIONING FOR DATA-INTENSIVE APPLICATIONS

This paper describes a programming toolkit developed in the PL-Grid project, named QStorMan, which supports storage QoS provisioning for data-intensive applications in distributed environments. QStorMan exploits knowledge-oriented methods for matching storage resources to non-functional requirements, which are defined for a data-intensive application. In order to support various usage scenarios, QStorMan provides two interfaces, such as programming libraries or a web portal. The interfaces allow to define the requirements either directly in an application source code or by using an intuitive graphical interface. The first way provides finer granularity, e.g., each portion of data processed by an application can define a different set of requirements. The second method is aimed at legacy applications support, which source code can not be modified. The toolkit has been evaluated using synthetic benchmarks and the production infrastructure of PL-Grid, in particular its storage infrastructure, which utilizes the Lustre file system.

Optimization of Data Access for Grid Environment

In the paper the architecture for accessing data in the grid environment, developed for EU CrossGrid Project, is presented. The local optimization problems for efficient data access and estimation of data access time are discussed mainly, while the overall architecture is proposed on the base of collaboration with European Data Grid.

Access Time Estimation for Tertiary Storage Systems

We propose two approaches for estimating the Tertiary Storage System access time: open approach and gray-box approach. In the first case the source code of the storage system is available, so changes in the code are made by adding event reporting functions. In the second approach - the essential system information is accessible via its native tools only. In this paper we describe an implementation of the open approach for access time estimation. The second approach we have shortly described as our future work.

Gray Box Based Data Access Time Estimation for Tertiary Storage in Grid Environment

In this paper experiments on estimation of data access time for data located on tertiary storage systems are presented. Estimation algorithm based on the gray-box approach using event driven simulation method is described. The implementation of such a system for the Legato DiskXtender HSM system is tested and results are shown. Application of the system for grid environment is discussed.

Metadata Organization and Management for Globalization of Data Access with Onedata

The Big Data revolution means that large amounts of data have not only to be stored, but also to be processed to unlock the potential of access to information and knowledge for scientific research. As a result, scientific communities require simple and convenient global access to data which is effective, secure and shareable. In this article we analyze how researchers use their data working in large scientific projects and show how their requirements may be satisfied with our solution called Onedata. Major technical mechanisms of metadata management and organization are described.

Data access time estimation for the CASTOR HSM system

The paper presents a system for estimating the access time for data stored on the CASTOR Hierarchical Storage Management (HSM) system developed at CERN. The estimation is based on the gray-box approach. The system consists of two modules: Monitor and Simulator. Information about the current state of the HSM system is obtained by the Monitor via CASTOR API functions. The second module is based on event driven simulation of the HSM system. Special attention to the queuing algorithm is paid. Implementation details and tests results are presented.

Tertiary Storage System for Index-Based Retrieving of Video Sequences

In this paper we present a research which is aimed at developinga tertiary storage system architecture for efficient index-based retrievingo f video sequences. During the research we built such a system by creating dedicated tertiary storage management software called VTSS. The system is intended to be a part of a multimedia database system. We also present preliminary experimental results from a prototype implementation.