Hanxu Hou

General Fractional Repetition Codes for Distributed Storage Systems

In order to provide fault-tolerance and guarantee reliability, data redundancy should be introduced in distributed storage systems. The emerging coding techniques for storage such as fractional repetition codes, provide the required redundancy more efficiently than the conventional replication scheme. In this letter, we extend the construction of fractional repetition codes and present a new coding scheme, termed general fractional repetition codes. The proposed codes can be applied to storage systems in which the storage capacities of nodes may be different. Based on a combinatorial structure known as group divisible design, the new code construction is available for a large set of parameters. The performance of the proposed codes is evaluated by a metric called node repair alternativity, which measures the number of different subsets of nodes that enable the repair of a specific failed node.

BASIC regenerating code: Binary addition and shift for exact repair

Regenerating code is a class of storage codes that achieve the optimal trade-off between storage capacity and repair bandwidth, which are two important performance metrics in data storage systems. However, existing constructions of regenerating codes rely on expensive computational operations such as finite field multiplication. The high coding and repair complexity limit their applications in large-scale practical storage systems. In this paper, we show that it is possible to achieve the full potential of regenerating codes with low computational complexity. In particular, we propose a new class of regenerating codes, called BASIC codes, that can achieve two specific points (i.e., minimum-bandwidth and minimum-storage regenerating points) on the storage and repair bandwidth trade-off curve, using only binary addition and shift operations in the coding and repair processes. Although in this paper we focus on constructing and analyzing BASIC codes for two specific exact-repair settings, our framework can be generalized to develop BASIC codes for more general exact- and functional-repair regenerating codes.

BASIC Codes: Low-Complexity Regenerating Codes for Distributed Storage Systems

In distributed storage systems, regenerating codes can achieve the optimal tradeoff between storage capacity and repair bandwidth. However, a critical drawback of existing regenerating codes, in general, is the high coding and repair complexity, since the coding and repair processes involve expensive multiplication operations in finite field. In this paper, we present a design framework of regenerating codes, which employ binary addition and bitwise cyclic shift as the elemental operations, named BASIC regenerating codes. The proposed BASIC regenerating codes can be regarded as a concatenated code with the outer code being a binary parity-check code, and the inner code being a regenerating code utilizing the binary parity-check code as the alphabet. We show that the proposed functional-repair BASIC regenerating codes can achieve the fundamental tradeoff curve between the storage and repair bandwidth asymptotically of functional-repair regenerating codes with less computational complexity. Furthermore, we demonstrate that the existing exact-repair product-matrix construction of regenerating codes can be modified to exact-repair BASIC product-matrix regenerating codes with much less encoding, repair, and decoding complexity from the theoretical analysis, and with less encoding time, repair time, and decoding time from the implementation results.

Replication-based distributed storage systems with variable repetition degrees

We consider a class of regenerating codes for distributed storage systems that provide exact and uncoded repair at the minimum bandwidth regenerating point. The desirable properties can be achieved by a coding scheme which concatenates an outer MDS code and an inner fractional repetition (FR) code. However, it is desirable to have the more popular packets with higher repetition degrees in practical systems. Motivated by this, we propose a new framework of code design where the repetition degrees of coded packets can be different. By adopting group divisible designs, our framework allows the design of system over a large range of parameters. Moreover, we make use of the systematic feature of MDS codes and wisely partition the storage nodes into several clusters. We show that in normal cases, data reconstruction time can be greatly reduced while contacting nodes in the same cluster.

New MDS array code correcting multiple disk failures

We present a new family of maximal-distance separable (MDS) array codes which can tolerate five disk failures. The encoding is based on bit-wise exclusive OR (XOR) and bit-wise cyclic shifts, and hence is amenable to practical implementation. Efficient repair method for correcting up to two disk failures is also given. The proposed coding scheme provides a larger spectrum of parameters, with comparable encoding and repairing complexities in compare with existing MDS array codes, such as the row-diagonal parity (RDP) code and the EVENODD code.

Construction of exact-BASIC codes for distributed storage systems at the MSR point

Regenerating codes (RGC) are a class of distributed storage codes that can provide efficient repair of failure nodes in distributed storage systems. In general, the reduction of repair bandwidth of RGC is at the expense of a small increase in storage cost and computational cost. The high computational complexity of data coding over a finite field of large size makes it unsuitable for practical distributed storage systems. BASIC codes, which stands for Binary Addition and Shift Implementable Convolutional codes, is introduced in [1] with the aim of reducing computational complexity, while retaining the benefits of RGC. In this paper, we present a construction of exact-repair BASIC codes at the minimum-storage point (MSR). A helper node needs no coding to repair a failure node for the minimum-storage BASIC codes. The results of simulation show minimum-storage BASIC codes outperform Cauchy Reed-Solomon codes in both repairing cost and coding cost.

A General Co/Decoder of Network Coding in HDL

This paper presents a practical and general coder and decoder of network coding (NC) with HDL (Hardware Description Language) logic for wire-speed nodes in multisource multicast networks. The NC coders apply random linear network coding (RLNC) and the decoders recover the original packets by Cramers rule. All these mathematical operations are carried out in the Galois Field (256). The structures and algorithms of NC coder and decoder were designed in detail and implemented in HDL with NetFPGA boards provided by Stanford University. Comparing with traditional stored-and-forward mechanism, network emulations showed that networks with wire-speed NC coder and decoder nodes could achieve the capacity bound of max-flow min-cut theorem in case of bottlenecks, and the end-to-end delay was guaranteed on a small constant.

On the MDS Condition of Blaum–Bruck–Vardy Codes With Large Number Parity Columns

Binary maximum distance separable (MDS) array codes are widely used in storage systems. EVENODD code and row-diagonal parity (RDP) code are two well-known binary MDS array codes with two parity columns. The Blaum-Bruck-Vardy (BBV) code is an extension of the double-erasure-correcting EVENODD code. It is known that the BBV code is always MDS for three parity columns, and sufficient conditions for up to eight parity columns to be MDS are also known. However, the MDS condition for more than eight parity columns is an open problem since then. In this letter, we study the MDS condition of BBV code and give a sufficient MDS condition of the BBV code with more than eight parity columns.

General self-repairing codes for distributed storage systems

In distributed storage systems, a data file is encoded and distributed to storage nodes, such that the data file can be recovered from some subsets of the nodes. Upon the failure of a storage node, we want to repair it efficiently by contacting and downloading some encoded bits from a small number of surviving nodes. Using projective-geometric self-repairing codes (PSRC), proposed by Oggier and Datta, one can repair a failed node by contacting only two nodes. However, in their construction, the number of storage nodes in the storage system is a large number, and thus the storage efficiency is low. In this paper, we investigate how to be more flexible in the number of storage nodes. The proposed code in this paper is called general projective geometric self-repairing codes (GPSRC). GPSRC reduces high redundancy of PSRC, while retains the basic property of PSRC. We present some methods for repairing a failed node, in which the number of contacted surviving nodes is flexible. These repairing methods provide tradeoff between repair-degree and repair-bandwidth.

STORE: Data recovery with approximate minimum network bandwidth and disk I/O in distributed storage systems

Recently, traditional erasure codes such as Reed-Solomon (RS) codes have been increasingly deployed in many distributed storage systems to reduce the large storage overhead incurred by the widely adopted replication scheme. However, these codes require significantly high resources with respect to network bandwidth and disk I/O during recovery of missing or unavailable data. It is referred as the recovery problem. In this paper, we dedicate to integrating exact minimum bandwidth regenerating codes into practical systems to solve the recovery problem. We design an implementation friendly storage code with the recently proposed BASIC framework and ZigZag decodable code for saving recovery bandwidth and disk I/O. We build a system called STORE based on this code and evaluate our prototype atop a HDFS cluster testbed with 21 nodes. As shown in this paper, the recovery bandwidth achieves minimum approximately during recovery of both data block and parity block with STORE. Another attractive result is that the recovery disk I/O also achieves minimum approximately during recovery of data block. Due to the reduction of recovery bandwidth and disk I/O, the degraded read throughput is boosted notably.