Mohammad Shorfuzzaman

Adaptive popularity-driven replica placement in hierarchical data grids

Data grids support access to widely distributed storage for large numbers of users accessing potentially many large files. Efficient access is hindered by the high latency of the Internet. To improve access time, replication at nearby sites may be used. Replication also provides high availability, decreased bandwidth use, enhanced fault tolerance, and improved scalability. Resource availability, network latency, and user requests in a grid environment may vary with time. Any replica placement strategy must be able to adapt to such dynamic behavior. In this paper, we describe a new dynamic replica placement algorithm, Popularity Based Replica Placement (PBRP), for hierarchical data grids which is guided by file “popularity”. Our goal is to place replicas close to clients to reduce data access time while still using network and storage resources efficiently. The effectiveness of PBRP depends on the selection of a threshold value related to file popularity. We also present Adaptive-PBRP (APBRP) that determines this threshold dynamically based on data request arrival rates. We evaluate both algorithms using simulation. Results for a range of data access patterns show that our algorithms can shorten job execution time significantly and reduce bandwidth consumption compared to other dynamic replication methods.

Popularity-Driven Dynamic Replica Placement in Hierarchical Data Grids

Data grids provide geographically distributed storage for large-scale data-intensive applications. Ensuring efficient access to such large and widely distributed datasets is hindered by high latencies. To speed up data access, data grid systems replicate data in multiple locations so a user can access the data from a nearby site. In addition to reducing data access time, replication also aims to use network and storage resources efficiently. While replication is a well-known technique, the problem of replica placement has not been widely studied for data grid environments. To obtain the best possible gains from replication, strategic placement of the replicas is critical. In a grid environment resource availability, network latency, and userspsila requests can vary. To address these issues a placement strategy is needed that adapts to dynamic behavior. This paper proposes a new dynamic replica placement algorithm for hierarchical data grids based on file ldquopopularityrdquo. Our goal is to place replicas close to the clients to reduce access time while using the network and storage efficiently thereby effectively balancing storage cost and access latency. We evaluate our algorithm using OptorSim which shows that our approach outperforms other techniques in terms of access time and bandwidth used.

Distributed Popularity Based Replica Placement in Data Grid Environments

Data grids support distributed data-intensive applications that need to access massive datasets stored around the world. Ensuring efficient access to such datasets is hindered by the high latencies of wide-area networks. To speed up access, files can be replicated so a user can access a nearby replica. Replication also provides improved availability, decreased bandwidth use, increased fault tolerance, and improved scalability. Since a grid environment is dynamic, resource availability, network latency, and user requests may change. To address these issues a dynamic replica placement strategy that adapts to changing behaviour is needed. In this paper, we introduce a highly distributed replica placement algorithm for hierarchical data grids. Our algorithm exploits data access histories to identify popular files and determines optimal replication locations to improve access performance by minimizing replication overhead (access and update) assuming a given traffic pattern. The problem is formulated using dynamic programming. We evaluate our algorithm using the OptorSim simulator and find that it offers shorter execution time and reduced bandwidth consumption compared to other dynamic replica placement methods.

Distributed Placement of Replicas in Hierarchical Data Grids with User and System QoS Constraints

Data grids support distributed data-intensive applications that need to access massive datasets stored around the world. Ensuring efficient access to such datasets is hindered by the high latencies of wide-area networks. To speed up access, files can be replicated so a user can access a nearby replica. Much of the work on the replica placement problem in data grids has focused on average system performance and ignored quality assurance issues. In the existing work that considers QoS, a simplified replication model is often assumed, therefore, resulting solutions may not be applicable to real systems. In this paper, we introduce a more realistic model for replica placement in hierarchical Data Grids which determines the positions of a minimum number of replicas expected to satisfy certain quality requirements both from user and system perspectives. Our placement algorithm is based on a highly distributed and decentralized technique that exploits the data access history for popular data files and computes replica locations by minimizing overall replication cost (read and update) while maximizing QoS satisfaction for a given traffic pattern. The problem is formulated using dynamic programming. We assess our algorithm using OptorSim. Simulation results demonstrate the effectiveness of our replica placement technique considering various factors such as storage and workload constraints of replica servers, link capacity constraints, user QoS requirements, etc.

Allocating replicas in large-scale data grids using a QoS-aware distributed technique with workload constraints

An important technique to speed access in data grids is replication, which provides nearby replicas. In a data grid environment, resource availability, network latency and user request patterns may change. In this paper, we introduce a new distributed replica placement algorithm for hierarchical data grids that determines the positions of a minimum number of replicas expected to satisfy certain quality requirements. Our placement algorithm computes replica locations by minimising overall replication cost (read and update) while maximising Quality of Service (QoS) satisfaction for a given traffic pattern. Our algorithm also assumes that the workload capacity of each replica server is bounded. The problem is formulated using dynamic programming. We assess our algorithm using OptorSim. A comparison of our algorithm to its QoS-unconstrained counterpart and to two other existing algorithms (Greedy Add and Greedy Remove) shows that our algorithm can shorten job execution time significantly while requiring only moderate network bandwidth.

Adaptive Replica Placement in Hierarchical Data Grids

Data grids support distributed data-intensive applications that need to access massive (multi-terabyte or larger) datasets stored around the world. Ensuring efficient and fast access to such widely distributed datasets is hindered by the high latencies of wide-area networks. To speed up access, data files can be replicated so users can access nearby copies. Replication also provides high data availability, decreased bandwidth consumption, increased fault tolerance, and improved scalability. Since a grid environment is highly dynamic, resource availability, network latency, and users requests may change frequently. To address these issues a dynamic replica placement strategy that adapts to dynamic behavior in data grids is needed. In this paper, we extend our earlier work on popularity-based replica placement proposing a new adaptive algorithm for use in large-scale hierarchical data grids. Our algorithm dynamically adapts the frequency and degree of replication based on data access arrival rate and available storage capacities. We evaluate our algorithm using OptorSim. Our results show that our algorithm can shorten job execution time greatly and reduce bandwidth consumption compared to its non-adaptive counterpart which outperforms other existing replica placement methods.