Xunrui Yin

MDR Codes: A New Class of RAID-6 Codes with Optimal Rebuilding and Encoding

As storage systems grow in size, device failures happen more frequently than ever before. Given the commodity nature of hard drives employed, a storage system needs to tolerate a certain number of disk failures while maintaining data integrity, and to recover lost data with minimal interference to normal disk I/O operations. RAID-6, which can tolerate up to two disk failures with the minimum redundancy, is becoming widespread. However, traditional RAID-6 codes suffer from high disk I/O overhead during recovery. In this paper, we propose a new family of RAID-6 codes, the Minimum Disk I/O Repairable (MDR) codes, which achieve the optimal disk I/O overhead for single failure recoveries. Moreover, we show that MDR codes can be encoded with the minimum number of bit-wise XOR operations. Simulation results show that MDR codes help to save about half of disk read operations than traditional RAID-6 codes, and thus can reduce the recovery time by up to 40%.

Min-cost multicast networks in Euclidean space

Space information flow is a new field of research recently proposed by Li and Wu [1], [2]. It studies the transmission of information in a geometric space, where information flows can be routed along any trajectories, and can be encoded wherever they meet. The goal is to satisfy given end-to-end unicast/multicast throughput demands, while minimizing a natural bandwidth-distance sum-product (network volume). Space information flow models the design of a blueprint for a minimum-cost network. We study the multicast version of the space information flow problem, in Euclidean spaces. We present a simple example that demonstrates the design of an information network is indeed different from that of a transportation network. We discuss properties of optimal multicast network embedding, prove that network coding does not make a difference in the basic case of 1-to-2 multicast, and prove upper-bounds on the number of relay nodes required in an optimal acyclic multicast network.

On Space Information Flow: Single multicast

Departing from Network Information Flow (NIF) that studies network coding in graphs, Space Information Flow (SIF) is a new paradigm that studies network coding in a geometric space. This work focuses on the problem of min-cost multicast network coding in a 2-dimensional Euclidean space. We prove a number of properties of the optimal SIF solutions, and propose a two-phase heuristic algorithm for computing the optimal SIF. The first phase computes the optimal topology through space partitioning that translates the SIF problem into a NIF problem, which is then solved using linear optimization. The second phase computes the min-cost embedding of the SIF topology found in the first phase, by fine tuning the location of each relay node using properties that an optimal SIF must satisfy.

On vector linear solvability of multicast networks

In the literature of network coding, vector linear network coding (LNC) is a generalization of the conventional scalar LNC, such that the data unit transmitted on every edge is an L-dimensional vector of data symbols over a base field GF(q). A scalar linear code over GF(q) is simply a vector linear code of dimension 1 over GF(q), and a general network has a scalar linear solution over GF(qL) only if it has a vector linear solution of dimension L over GF(q). Though vector LNC is more powerful in enabling a higher coding diversity, this work will present explicit multicast networks, for the first time in the literature, with the special property that they do not have a vector linear solution of dimension L over GF(2) but have scalar linear solutions over GF(q′), for some q′ < 2L. This reveals the fact that although vector LNC can outperform scalar LNC in terms of yielding a solution for a general network, scalar LNC can also outperform vector LNC of dimension larger than 1 in terms of using a smaller alphabet to yield a solution for a multicast network.

Heterogeneity-aware data regeneration in distributed storage systems

Distributed storage systems provide large-scale reliable data storage services by spreading redundancy across a large group of storage nodes. In such big systems, node failures take place on a regular basis. When a node fails or leaves the system, to maintain the same level of redundancy, it is expected to regenerate the redundant data at a replacement node as soon as possible. Previous studies aim to minimize the network traffic in the regeneration process, but in practical networks, where link capacities vary in a wide range, minimizing network traffic does not always mean minimizing regeneration time. Considering the heterogeneous link capacities, Li et al. proposed a tree-structured regeneration scheme, called RCTREE, to bypass the low-capacitated link encountered in direct transmissions. However, we find that RCTREE may rapidly lose data integrity after several regenerations. In this paper, we reconsider the problem of minimizing regeneration time in networks with heterogeneous link capacities. We derive the minimum amount of data to be transmitted through each link to preserve data integrity. We prove that building an optimal regeneration tree is NP-complete and propose a heuristic algorithm for a near-optimal solution. We further introduce a flexible regeneration scheme, which allows providers to generate different amount of coded data. Simulation results show that the flexible tree-structured regeneration scheme can reduce the regeneration time significantly.

Multicast Network Coding and Field Sizes

In an acyclic multicast network, it is well known that a linear network coding solution over GF(q) exists when q is sufficiently large. In particular, for each prime power q no smaller than the number of receivers, a linear solution over GF(q) can be efficiently constructed. In this paper, we reveal that a linear solution over a given finite field does not necessarily imply the existence of a linear solution over all larger finite fields. In particular, we prove by construction that: 1) for every ω ≥ 3, there is a multicast network with source outdegree ω linearly solvable over GF(7) but not over GF(8), and another multicast network linearly solvable over GF(16) but not over GF(17); 2) there is a multicast network linearly solvable over GF(5) but not over such GF(q) that q > 5 is a Mersenne prime plus 1, which can be extremely large; 3) a multicast network linearly solvable over GF(qm1) and over GF(qm2) is not necessarily linearly solvable over GF(qm1+m2); and 4) there exists a class of multicast networks with a set T of receivers such that the minimum field size qmin for a linear solution over GF(qmin) is lower bounded by O(√|T|), but not every larger field than GF(qmin) suffices to yield a linear solution. The insight brought from this paper is that not only the field size but also the order of subgroups in the multiplicative group of a finite field affects the linear solvability of a multicast network.

On benefits of network coding in bidirected networks and hyper-networks

Network coding is a technique that allows information flows to be encoded while routed across a data network. It was shown that network coding helps increase the throughput and reduce the cost of data transmission, especially for one-to-many multicast applications. An important direction in network coding research is to understand and quantify the coding advantage and cost advantage, i.e., the potential benefits of network coding, as compared to routing, in terms of increasing throughput and reducing transmission cost, respectively. Two classic network models were considered in previous studies of coding advantage: directed networks and undirected networks. The study of coding advantage in this work further focuses on two types of parameterized networks, including bidirected networks and hyper-networks, which generalizes the directed and the undirected network models, respectively. With proper parameter setting, more realistic modeling of networks in practice can be achieved. We prove upper-bounds and lower-bounds on the coding advantage for multicast in these models. Some of our bounds are new and unknown before, some improve upon previously proven bounds, and some answer open questions in the literature.

Two New Classes of Two-Parity MDS Array Codes With Optimal Repair

Two-parity maximum distance separable (MDS) array codes achieve the lowest storage overhead for tolerating two erasures. In this letter, we propose two new classes of twoparity MDS array codes over finite field F2, with which each single node failure can be recovered by reading the minimum amount of data from each surviving node.

Hierarchical Virtual Machine Placement in Modular Data Centers

This work studies how to minimize communication cost for placing Virtual Machines (VMs) in a modular data center. We consider a number of cooperative VMs implementing the same job, with known inter-VM communication patterns. The modular data center has a two-layer network structure, where computing pods constitute basic building blocks and are connected by a core network. At the core network layer, we design spectral clustering algorithms to partition VMs into computing pods, minimizing inter-pod communication cost. We then further apply an SDP relaxation approach to decide the VM placement within each computing pod, targeting both load balancing among physical servers and inter-server communication cost minimization. Extensive simulations are conducted to validate the efficacy of the proposed hierarchical VM placement scheme.

A graph minor perspective to network coding: Connecting algebraic coding with network topologies

Network Coding encourages information coding across a communication network. While the necessity, benefit and complexity of network coding are sensitive to the underlying graph structure of a network, existing theory on network coding often treats the network topology as a black box, focusing on algebraic or information theoretic aspects of the problem. This work aims at an in-depth examination of the relation between algebraic coding and network topologies. We mathematically establish a series of results along the direction of: if network coding is necessary/beneficial, or if a particular finite field is required for coding, then the network must have a corresponding hidden structure embedded in its underlying topology, and such embedding is computationally efficient to verify. Specifically, we first formulate a meta-conjecture, the NC-Minor Conjecture, that articulates such a connection between graph theory and network coding, in the language of graph minors. We next prove that the NC-Minor Conjecture is almost equivalent to the Hadwiger Conjecture, which connects graph minors with graph coloring. Such equivalence implies the existence of K4, K5, K6, and KO(q/ log q) minors, for networks requiring F3, F4, F5 and Fq, respectively. We finally prove that network coding can make a difference from routing only if the network contains a K4 minor, and this minor containment result is tight. Practical implications of the above results are discussed.