Myung-Hoon Cha

Design and Implementation of a Consistent Update Method for Multiple File Replicas in a Distributed File System

The need for saving a great many files becomes necessary in distributed file system environment. Particularly, some file systems maintain multiple file replicas for efficient failure recovery. Under these circumstances, however, all the file replicas have to be modified identically to update a file. If there are many clients to update the same file concurrently, the final images of all replicas can be inconsistent. In this paper, we propose a consistent update method so that a file can have the identical image in all replicas after concurrent modifications by multiple clients.

Adaptive metadata rebalance in exascale file system

This paper presents an effective method of metadata rebalance in exascale distributed file systems. Exponential data growth has led to the need for an adaptive and robust distributed file system whose typical architecture is composed of a large cluster of metadata servers and data servers. Though each metadata server can have an equally divided subset from the entire metadata set at first, there will eventually be a global imbalance in the placement of metadata among metadata servers, and this imbalance worsens over time. To ensure that disproportionate metadata placement will not have a negative effect on the intrinsic performance of a metadata server cluster, it is necessary to recover the balanced performance of the cluster periodically. However, this cannot be easily done because rebalancing seriously hampers the normal operation of a file system. This situation continues to get worse with both an ever-present heavy workload on the file system and frequent failures of server components at exascale. As one of the primary reasons for such a degraded performance, file system clients frequently fail to look up metadata from the metadata server cluster during the period of metadata rebalance; thus, metadata operations cannot proceed at their normal speed. We propose a metadata rebalance model that minimizes failures of metadata operations during the metadata rebalance period and validate the proposed model through a cost analysis. The analysis results demonstrate that our model supports the feasibility of online metadata rebalance without the normal operation obstruction and increases the chances of maintaining balance in a huge cluster of metadata servers.

High Performance Metadata Management Engine for Large-Scale Distributed File Systems

This paper presents a metadata management engine with considerable performance and scalability that can be used in a nationwide file storage service. While a metadata server cluster is needed to support a huge number of metadata requests in a large-scale file storage, in reality, there are few such clusters with a product level quality. We propose a metadata management architecture with several new features including the maximum performance potentialities using a single metadata server. Based on this architecture, we implemented a new file system that is now being used for many storage services on Internet.

Design and Implementation of a File Spanning among Multiple OSDs in OASIS File System

According to the development of the object based storage device (OSD) and networking technologies, some OSD based clustering file systems, for example, the ActiveScale storage cluster of the Panasas Corp. and the Lustre file system of the Cluster File System Corp., are emerging in the marketplace. Typically, these systems connect multiple OSDs on the fast network. When a large number of OSDs are connected as a linear style configuration, a client machine stores a file in its local memory first and then flushes it into an OSD. However, the file stored temporarily in the clients memory may not be completely stored in the selected OSD because the size of the file can be bigger than that of the OSDs available space. In this paper, we propose a detailed algorithm and system structure for solving this problem occurring in the cluster file system using OSDs.