Correctable Erasure Patterns in Product Topologies
Lukas Holzbaur, Sven Puchinger, Eitan Yaakobi, Antonia Wachter-Zeh
aa r X i v : . [ c s . I T ] F e b Correctable Erasure Patterns in Product Topologies
Lukas Holzbaur ∗ , Sven Puchinger † , Eitan Yaakobi ‡ , and Antonia Wachter-Zeh ∗∗ Technical University of Munich, { lukas.holzbaur, antonia.wachter-zeh } @tum.de † Technical University of Denmark, [email protected] ‡ Technion — Israel Institute of Technology, [email protected]
Abstract —Locality enables storage systems to recover failednodes from small subsets of surviving nodes. The setting wherenodes are partitioned into subsets, each allowing for local recov-ery, is well understood. In this work we consider a generalizationintroduced by Gopalan et al., where, viewing the codewords asarrays, constraints are imposed on the columns and rows inaddition to some global constraints. Specifically, we present ageneric method of adding such global parity-checks and derivenew results on the set of correctable erasure patterns. Finally, werelate the set of correctable erasure patterns in the consideredtopology to those correctable in tensor-product codes.
I. I
NTRODUCTION
Distributed data storage systems consist of an ever growingnumber of nodes/servers. To protect against data loss, thedata is encoded with error/erasure correcting codes, mostcommonly, maximum distance separable (MDS) codes. Whilethese offer the optimal trade-off between probability of dataloss and storage overhead, the recovery of a failed node mustinvolve a large number of nodes. This undesirable overheadlead to the introduction of locally recoverable codes (LRCs)[1], which allow for the recovery of a small number of failednodes by contacting only a limited number of surviving nodes.In coding theoretic terms, LRCs can recover any small numberof erasures from small subsets of surviving positions. For afixed storage overhead, this improvement comes at the cost ofa reduced minimum distance [1], [2], i.e., number of failuresthat is guaranteed to be correctable. However, codes of optimalminimum distance are not necessarily also optimal in termsof mean time to data loss, as larger numbers of failed nodesmight also be correctable. Codes that are able to recoverall such combinations of failed nodes are coined maximallyrecoverable (MR) [3]. Such MR LRCs (also referred to asPMDS codes) have received considerable attention in recentyears. Constructions for different parameters regimes are givenin [4], [5], [6], [7], [8], [9] and the set of correctable erasurepatterns has been characterized [5], [6].While LRCs represent an important notion of locality, somereal world systems require more complex locality constraints.For example, Facebook’s f4 storage system [10] employs MDScodes of length tolerating failed nodes within each datacenter, plus an additional parity check across data centers.A theoretical framework for such restraints, referred to as the topology of the system, was established in [6]. A topologyis a restriction on the support of the parity-check matrix ofa code, i.e., it specifies a subset of positions in this matrixthat must be zero. One case of particular interest is given bygrid-like/product topologies, which, viewed as an array, imposeconstraints on each row and column, plus an additional numberof global constraints. Even in this simplification of the general The work of L. Holzbaur and A. Wachter-Zeh was supported by the GermanResearch Foundation (Deutsche Forschungsgemeinschaft, DFG) under GrantNo. WA3907/1-1. S. Puchinger has received funding from the EuropeanUnion’s Horizon 2020 research and innovation programme under the MarieSkłodowska-Curie grant agreement no. 713683. The work of E. Yaakobi waspartially supported by the ISF grant 1817/18 and by the Technion HiroshiFujiwara cyber security research center and the Israel cyber directorate. setting, it is a highly non-trivial problem to determine theerasure patterns that are correctable, i.e., the patterns that anMR code must be able to correct. The seminal works of [6], [8]initiated studies of the classification of these patterns for somerestricted cases (see Table I) [11] and bounds on the requiredfield size [12], [13]. In particular, [8] established a necessarycondition for an erasure pattern to be correctable, which isthen shown to also be sufficient for the case of one columnconstraint and no global constraints. Further, the sufficiencyof this condition is conjectured to hold for the case of morecolumn constraints.The main results of this work are twofold. First, we establisha generic method of adding global constraints to MR codes,which allows us to establish the set of correctable erasure pat-terns as a function of the patterns which are correctable withoutthe global constraints. Second, we show that the necessarycondition for correctability given in [8] is not sufficient ingeneral. Finally, we connect MR codes for product topologiesto tensor-product (TP) codes [14], which are another class ofcodes with application to storage.II. P
RELIMINARIES
A. Notation
We denote the set of integers [ a, b ] = { i | a ≤ i ≤ b } andwrite [1 , b ] = [ b ] . For an a × b matrix A and a set I ⊂ [ b ] wedenote by A | I the restriction of A to the columns indexed by I . We write b to denote the all-zero vector of length b . Let F q denote the finite field of size q . We denote a code C of length n and dimension dim( C ) = k over F q by [ n, k ] q . If the field size q is not of interest we write F and [ n, k ] . A set I ⊆ [ n ] with |I| = k is said to be an information set of C if dim( C| I ) = k .For sets U ⊆ [ m ] and V ⊆ [ n ] we denote the Cartesian product U × V = { ( U, V ) | U ∈ U , V ∈ V} . For two codes C ⊆ F m and C ⊆ F n we write C ⊗ C = h G ⊗ G i , where G , G denote some generator matrix of C , C , respectively, ⊗ is theKronecker product, and h G i is the row span of the matrix G . Note that the resulting code is independent of the specificchoice of the generator matrices. We have C ⊗ C ⊆ F mn but we frequently use the equivalent interpretation as a subsetof F m × n . B. Codes for Grid-Like Topologies
In this work we are concerned with codes for grid-liketopologies T m × n ( a, b, h ) , i.e., non-rigorously, product codeswith h additional parity constraints that are allowed to dependon all codeword positions. We adapt the notation of [8]. Definition 1 (Code for grid-like topology [8, Definition 2.1]) . Let C col be an [ m, ≥ m − a ] code and C row be an [ n, ≥ n − b ] code. We define a code for the topology T m × n ( a, b, h ) to beany code with parity-check matrix H = (cid:18) H local H global (cid:19) , where H local is a parity-check matrix of the code C col ⊗ C row and H global is an arbitrary h × mn matrix. ABLE IP
ROPERTIES OF DIFFERENT TOPOLOGIES . L
INES
COLLECT THE KNOWN RESULTS . L
INES
SUMMARIZES THE RESULTS ON GLOBAL REDUNDANCYOF S ECTION
III,
WHERE THE ∼ SYMBOL IMPLIES THAT A CODE FOR THE RESPECTIVE TOPOLOGY CAN BE CONSTRUCTED GIVEN A CODE FOR THE SAMETOPOLOGY WITH h = 0 . L INES
GIVE THE RESULTS ON THE CHARACTERIZATION OF CORRECTABLE ERASURE PATTERNS PRESENTED IN S ECTION
IV.Topology Correctable Patterns Fits conj. Reference Construction Reference T m × n (0 , b, h ) regular + any h - [5], [6] ✓ [2], [15] T m × n (1 , , regular ✓ [8] ✓ [8] T m × n (1 , b, regular ✓ [8] - - T m × n (1 , , h ) regular + any h - [6], [8] + Thm. 1 ✓ [6], [8] + Thm. 1 T m × n (1 , b, h ) regular + any h - [8] + Thm. 1 ∼ Thm. 1 + any
C ∈ C MR m × n (1 , b, T m × n ( a, b, h ) E m × n ( a, b, + any h - Thm. 2 ∼ Thm. 1 + any
C ∈ C MR m × n ( a, b, T × (2 , , See Rem. 2 ✗ Lem. 4 - - T ( ≥ a +3) × ( ≥ b +3) ( ≥ , ≥ , unknown ✗ Thm. 4 - -
We denote the set of all such codes by C m × n ( a, b, h ) . Definition 2 (Correctable erasure pattern [8, Definition 2.2]) . Let
E ⊆ [ m ] × [ n ] denote a set of erased positions. We say the erasure pattern E is correctable in the topology T m × n ( a, b, h ) if and only if there exists a code in C m × n ( a, b, h ) that cancorrect this erasure pattern.We denote the set of erasure patterns which are correctablein T m × n ( a, b, h ) by E m × n ( a, b, h ) and by E max m × n ( a, b, h ) thosethat are not a proper subset of any other correctable pattern. Definition 3 (Maximally recoverable code [8, Definition 2.3]) . We say a code
C ∈ C m × n ( a, b, h ) is maximally recoverable (MR) if it corrects every erasure pattern in E m × n ( a, b, h ) .We denote the set of codes that are MR for a topology T m × n ( a, b, h ) by C MR m × n ( a, b, h ) . Observe that, any code that can correct an erasure pattern E can also correct any erasure pattern E ′ ⊂ E . A maximallyrecoverable code is equivalently defined as being able tocorrect any pattern in E max m × n ( a, b, h ) instead of E m × n ( a, b, h ) .By the same argument, the set of correctable erasure patterns E m × n ( a, b, h ) is uniquely defined by E max m × n ( a, b, h ) .We summarize several important properties of codes forgrid-like topologies in terms of our notation. Throughoutthis work we let m, n, a, b, h be non-negative integers whichsatisfy m > a and n > b . Further, to exclude trivialcases where the dimension of the code C is smaller thanthe dimension of the codes C col and C row , we assume that h ≤ ( m − a )( n − b ) − max { m − a, n − b } for the remainderof this work. Proposition 1 (Properties of codes for grid-like topologies [8,Proposition 2.1]) . For any
C ∈ C MR m × n ( a, b, h ) it holds that • the dimensions of C , C col , and C row are dim( C ) = ( m − a )( n − b ) − h dim( C col ) = m − a and dim( C row ) = n − b , • the codes C col and C row are MDS. Definition 4 ([8, Definition 3.1]) . Consider the topology T m × n ( a, b, and an erasure pattern E ⊂ [ m ] × [ n ] . We saythat E is regular if for all U ⊆ [ m ] , |U| = u ≥ a , and V ⊆ [ n ] , |V| = v ≥ b , we have |E ∩ ( U × V ) | ≤ va + ub − ab . Remark 1.
Intuitively, the restriction of the erasure patternto
U × V can be interpreted as the shortening of the re-spective code in the positions outside of this grid. As wewill show in Section IV-B, this shortened code needs to beable to decode the remaining erasures for the pattern to becorrectable. Definition 4 is a necessary (in general insufficient,see Section IV-A) condition for this to be possible, namely, that Note that the original definition does not include the restriction u ≥ a and v ≥ b . However, it is easy to check that this restriction is indeed necessary. the number of erasures remaining in the shortened code doesnot exceed its redundancy. It was shown in [8] that all erasure patterns that are not regular are not correctable in the topology T m × n ( a, b, . Onthe other hand, for some cases (see Table I) it is known that all regular patterns are correctable, which led to the followingconjecture. Conjecture 1 ([8, Conjecture 3.1]) . An erasure pattern E iscorrectable for the topology T m × n ( a, b, if and only if it isregular. In Section IV-A we disprove this conjecture.III. G
LOBAL R EDUNDANCY h > For some topologies with h = 0 the set of correctable era-sure patterns is fully characterized (as the regular patterns) and,for some cases, explicit constructions are known. For h > only the special cases of a = 0 , b, h ≥ and a = b = h = 1 are known (see Table I). In this section, we characterize theset of correctable erasure patterns E max m × n ( a, b, h ) as a functionof E max m × n ( a, b, , for any m, n, a, b , and h . A similar result forthe extension of codes defined by a binary parity-check matrix,i.e., where H local is in F , has been derived in [6, Section V.A]. A. Construction
We generalize the Gabidulin-based code construction [2]for a = 0 . These codes have been shown to be MR for thetopology T m × n (0 , b, h ) in [15]. We show that by the same two-stage encoding procedure we can “add global redundancy” tocodes for any grid-like topology. In other words, we give ageneral construction that, given a code C out ∈ C MR m × n ( a, b, ,returns a code C ∈ C MR m × n ( a, b, h ) . Definition 5 (Gabidulin codes) . Let g = ( g , . . . , g n ) ∈ F nq s besuch that the g i are linearly independent over F q . The [ n, k ] q s Gabidulin code
Gab ( n, k, g ) is defined as Gab ( n, k, g ) = h G i with G = g g . . . g n g q g q . . . g q n ... ... . . . ... g q k − g q k − . . . g q k − n . Observe that Gabidulin codes exist for any n ≤ s . Gabidulincodes are designed for the rank metric, but it is also well-known that they are MDS in the Hamming metric. We recalla well-known property of Gabidulin codes. Lemma 1 ([16, Lemma 3]) . Let G ∈ F k × nq s be a generatormatrix of an [ n, k, d ] Gabidulin code
Gab ( n, k, g ) . Then, forany full-rank matrix A ∈ F n × nq , the code C ′ = h G · A i is an [ n, k, d ] Gabidulin code
Gab ( n, k, g ′ ) with g ′ = g · A . emma 2. Let I out be the set of information sets of C out [ n out , k out ] q and C in be a Gab ( k out , k in , g ) q s Gabidulincode. Then the code h G in · G out i | I is a Gab ( k out , k in , g ′ ) q s Gabidulin code for any
I ∈ I out .Proof: By definition of an information set, the matrix G out | I is of full rank. The statement follows from observingthat h G in · G out i | I = h G in · ( G out | I ) i and applying Lemma 1. Theorem 1.
Let C out ∈ C MR m × n ( a, b, be an [ n out = mn, k out = ( m − a )( n − b )] code and C in = Gab (cid:0) ( m − a )( n − b ) , ( m − a )( n − b ) − h, g (cid:1) q s , with s ≥ ( m − a )( n − b ) . Then the code h G in · G out i correctsall erasure patterns in {E ′ ∪ I | E ′ ∈ E max m × n ( a, b, , I ⊂ ([ m ] × [ n ] \ E ′ ) , |I| = h } . Proof:
Let E be an erasure pattern of the set above. Then,there is an erasure pattern E ′ of the set E max m × n ( a, b, with E = E ′ ∪ I . The complement of E ′ is, by definition, an informationset of the outer code C out . By restricting the overall code tothis information set, we thus obtain a Gabidulin code withparameters [( m − a )( n − b ) , ( m − a )( n − b ) − h ] by Lemma 2.Note that there are exactly h remaining erasures (given by I ) inthe remaining positions. Since any Gabidulin code is MDS, therestricted code can correct exactly h erasures, which concludesthe proof. Example 1.
Let C out [ mn, m ( n − b )] be the code spanned by diag( G , G , . . . , G | {z } m times ) , where G is the generator matrix of an arbitrary [ n, n − b ] q MDS code. Observe that C out ∈ C MR m × n (0 , b, and the set ofits information sets is given by I out = { From each block pick arbitrary n − b positions } . Choose C in to be an [ m ( n − b ) , m ( n − b ) − h ] q m ( n − b ) Gabidulincode to obtain the PMDS code construction of [2]. Observethat I out are exactly the subsets of positions that must give anMDS code in a PMDS code.B. Correctable Erasure Patterns Let us characterize the patterns correctable in T m × n ( a, b, h ) given the set of patterns correctable in T m × n ( a, b, . Theorem 2.
We have E max m × n ( a, b, h ) = {E ′ ∪ I | E ′ ∈ E max m × n ( a, b, , I ⊂ ( m × n \ E ′ ) , |I| = h } , i.e., any E ∈ E max m × n ( a, b, h ) can be obtained by adding h erasures to some E ′ ∈ E max m × n ( a, b, .Proof: “ ⊇ ” follows by the construction of Theorem 1.The other direction is implied by the following argument.Let E ∈ E max m × n ( a, b, h ) . Denote by G the generator matrix of acode C ∈ C m × n ( a, b, h ) that corrects E and let H local , H global be as in Definition 1. Denote by C ′ ∈ C m × n ( a, b, the codeobtained by setting H global = . Then there exists a generatormatrix of C ′ of the form G ′ = (cid:18) GG global (cid:19) , for some G global ∈ F h × mn . Trivially, we have rank( G ′ | [ mn ] \E ) ≥ rank( G | [ mn ] \E ) = dim( C ) , where the last equality holds because E is correctable in C by definition. By basic linear algebra arguments there exists asubset I ⊂ E with |I| = dim( C ) − dim( C ′ ) ≤ h such that rank( G ′ | ([ mn ] \E ) ∪I ) = rank( G ′ ) = dim( C ′ ) . It follows that
E \ I is correctable in T m × n ( a, b, . Thisconcludes the proof. Corollary 1.
Denote N = mn . Then, for any E ′ ∈ E max m × n ( a, b, and any [ N, k ] code C ∈ C MR m × n ( a, b, h ) the code C| m × n \E ′ must be an [ N − |E ′ | , k, h + 1] MDS code.Proof:
It follows trivially from the dimension of C thatthe code C| [ n ] \E ′ can never correct more than h erasures. Theexistence of a code for which this restriction is an MDS codefollows from Corollary 2. Corollary 2.
Let C out [ N, k ] q ∈ C MR m × n ( a, b, . Then thereexists an [ N, k − h ] q k code C ∈ C MR m × n ( a, b, h ) . IV. N
EGATIVE R ESULTS ON C ORRECTABLE E RASURE P ATTERNS
In this section we turn to providing negative results on theset of correctable erasure patterns for given topology. First,we prove that a specific erasure pattern is never correctablein the topology T × (2 , , , thereby disproving Conjecture 1as given in [8]. Then, using this result, we provide genericmethods of constructing uncorrectable, regular erasure patternsfor larger topologies. A. Disproving a Conjecture on the Correctability of RegularErasure Patterns
We begin with some general observations on the relationbetween low-weight codewords.
Lemma 3.
Consider a linear [5 , code C row with generatormatrix G row = p row , p row , p row , p row , p row , p row , and a codeword (cid:0) α α (cid:1) ∈ C row . (1) Then p row , = − ( α p row , + α p row , ) (2) p row , = − ( α p row , + α p row , ) . (3) Proof:
The statement follows from (cid:0) α α (cid:1) = (cid:0) α α (cid:1) · p row , p row , p row , p row , p row , p row , ! ⇒ (cid:0) α α (cid:1) · p row , p row , p row , p row , p row , p row , ! = (cid:0) (cid:1) . Observe that each row of G row is a codeword of C row , inparticular, (cid:0) p row , p row , (cid:1) ∈ C row . (4)We define the same notions in similar notation for a [5 , code C col , i.e., (cid:0) γ γ (cid:1) ∈ C col (5) (cid:0) p col , p col , (cid:1) ∈ C col (6) p col , = − ( γ p col , + γ p col , ) (7) p col , = − ( γ p col , + γ p col , ) . (8)ote that if C row ( C col ) is MDS we have p row , , p row , = 0 ( p col , , p col , = 0 ) and the codewords given in (1) and (4) ((5) and(6)) are unique. This follows from the well-known facts thatall symbols in the parity part of a systematic generator matrixof an MDS code must be non-zero and that every codeword ofweight d of a linear code is unique up to scalar multiplicationwith an element of F ∗ .With these general relations established, we are now readyto prove that there exists a regular erasure pattern that is nevercorrectable in the topology T × (2 , , , by constructing anon-zero codeword that is zero in all non-erased positions. Lemma 4.
The regular pattern E given by E = ⋆ ⋆ ⋆ ⋆⋆ ⋆ ⋆ ⋆ ⋆ ⋆ ⋆ ⋆ ⋆⋆ ⋆ ⋆ ! E = { ( i, j ) | E ( i, j ) = ⋆ } is not correctable in T × (2 , , .Proof: Let
C ∈ C m × n ( a, b, . We prove the statement byconstructing a non-zero codeword of C that is zero in all non-erased positions. This implies that the code C cannot correctthe pattern uniquely since there are at least two codewords thatcoincide on all non-erased positions—the constructed non-zerocodeword and the all-zero codeword.To construct such a codeword, we replace the ⋆ -symbols in E by elements of F (not all zero) in the following manner: ( a ) Choose the element in position (2 , to be γ ∈ F ∗ . Notethat, since the code is linear, we can always normalize onesingle non-zero position to be an arbitrary element of F ∗ and w.l.o.g. we assume position (2 , to be non-zero. ( b ) Choose the second row to be the γ -multiple of (1). ( c ) Choose the second and third column to be the correspond-ing multiples of (5). ( d ) To obtain a multiple of (1) as the third row, set its firstposition to be γ . ( e ) Encode the first row and column with G row and G col ,respectively. ( f ) Replace the entries according to (2),(3),(7), and (8). ( g ) Fill in the fourth and fifth rows with the correspondingmultiples of (4).The individual steps are given by ( a ) ⇒ ⋆ ⋆ ⋆ ⋆γ ⋆ ⋆ ⋆ ⋆ ⋆ ⋆ ⋆ ⋆⋆ ⋆ ⋆ ! ( b ) ⇒ ⋆ ⋆ ⋆ ⋆γ γ α γ α ⋆ ⋆ ⋆ ⋆ ⋆ ⋆⋆ ⋆ ⋆ ! ( c ) ⇒ α α ⋆ ⋆γ γ α γ α ⋆ γ α γ α ⋆ ⋆ ⋆⋆ ⋆ ⋆ ! ( d ) ⇒ α α ⋆ ⋆γ γ α γ α γ γ α γ α ⋆ ⋆ ⋆⋆ ⋆ ⋆ ! ( e ) ⇒ α α α p row , + α p row , α p row , + α p row , γ γ α γ α γ γ α γ α γ p col , + γ p col , ⋆ ⋆γ p col , + γ p col , ⋆ ⋆ ( f ) ⇒ α α − p row , − p row , γ γ α γ α γ γ α γ α − p col , ⋆ ⋆ − p col , ⋆ ⋆ ( g ) ⇒ α α − p row , − p row , γ γ α γ α γ γ α γ α − p col , − p col , p row , − p col , p row , − p col , − p col , p row , − p col , p row , . It is easy to see that the fourth and fifth columns are alsomultiples of (6). Hence, this array contains a codeword of therow code in every row and a codeword of the column code inevery column. It is therefore a valid codeword of any code for T × (2 , , . As the used properties hold for any linear codewe conclude that this pattern is never correctable.We note that if both C row and C col are MDS, the first step,i.e., choosing the element in position (2 , , determines thewhole matrix, as each of the subsequent steps is unique in thiscase. Remark 2.
The proof of Lemma 4 can be carried out forany column/row permutation of E , i.e., all these permutationsare not correctable in T × (2 , , . Further, computer searchshows that these permutations are exactly the regularpatterns that are not correctable in this topology. Theorem 3.
Conjecture 1 (cf. [8, Conjecture 3.1]) is false.Proof:
Follows immediately from Lemma 4.
B. Implications of Uncorrectable Regular Patterns for LargerTopologies
In this section, we show that the incorrectable erasure patternfor the topology T × (2 , , given in Lemma 4 has implica-tions on the set of correctable erasure for almost all topologies.To this end, we employ arguments based on shortening and puncturing . Definition 6.
Let C be an [ N, k ] code and I ⊆ [ N ] be a setof intgers. We define the shortening operator as short I ( C ) = { c | [ N ] \I | c ∈ C , c | I = |I| } and the puncturing operator as punct I ( C ) = { c | [ N ] \I | c ∈ C} . We collect some well-known/basic properties in the follow-ing proposition.
Proposition 2.
Let G and H denote a generator and parity-check matrix of an [ N, k ] code C , respectively. Then, H | [ N ] \I is a parity-check matrix of short I ( C ) and G | [ N ] \I is a generator matrix of punct I ( C ) . an erasure pattern E ⊂ [ N ] is correctable if and only ifit fulfills the equivalent conditions dim(short [ N ] \E ( C )) = 0dim(punct E ( C )) = k . an erasure pattern E ⊂ [ N ] is correctable only if thepattern E \ I is correctable in the code short I ( C ) forany I ⊂ [ N ] . an erasure pattern E is correctable only if the pattern E \ I is correctable in the code punct I ( C ) for any I ⊆ [ E ] .Proof: Properties 1 and 2 are well-known.
Proof of 3):
By property 2) we known that
E\I is correctablein short I ( C ) if and only if short [ N ] \ ( E\I ) (short I ( C )) = 0 .Now observe that short [ N ] \ ( E\I ) (short I ( C )) = short ([ N ] \ ( E\I )) ∪I ( C )= short ([ N ] \E ) ∪I ( C )= short I\ ([ N ] \E ) (short ([ N ] \E ) ( C )) . As shortening does not increase the code dimension we have dim(short [ N ] \ ( E\I ) (short I ( C ))) ≤ dim(short [ N ] \E ( C )) . The statement follows from observing that E is correctable in C if and only if dim(short [ N ] \E ( C )) = 0 . Proof of 4)
By property 2) an erasure pattern is correctable ifand only if dim(punct E ( C )) = dim( C ) . The statement followsfrom observing that punct E\I (punct I ( C )) = punct E ( C ) . Lemma 5.
Let
C ∈ C m × n ( a, b, . Denote by I col and I row the sets of information sets of the respective column and rowode. Then for any U ⊆ [ m ] such that ([ m ] \ U ) ⊆ I for some I ∈ I col and V ⊆ [ n ] such that ([ n ] \ V ) ⊆ I for some I ∈ I row we have short ([ m ] × [ n ]) \ ( U×V ) ( C ) ∈ C U×V ( a, b, . Further, for any
U ⊆ I for some
I ∈ I col and V ⊆ I for some
I ∈ I row we have punct ([ m ] × [ n ]) \ ( U×V ) ( C ) ∈ C U×V ( a − ( m − u ) , b − ( n − v ) , . Proof: As h = 0 , by Definition 1, the generator matrixof C is given by G col ⊗ G row . Let G short J col , G short J row and G punct J col , G punct J row denote the generator matrices of the col-umn/row code shortened/punctured in the positions indexedby J . It follows directly from the definition of the Kroneckerproduct that short ([ m ] × [ n ]) \ ( U×V ) ( C ) = D G short U col ⊗ G short V row E punct ([ m ] × [ n ]) \ ( U×V ) ( C ) = D G punct U col ⊗ G punct V row E . The statement follows from observing that shortening a po-sition of an information set of an [ m, m − b ] code gives an [ m − , m − b − code. Similarly, puncturing a position inthe complement of an information set gives an [ m − , m − b ] code. Lemma 6.
If there exists a regular erasure pattern E thatis not correctable in T m × n ( a, b, then there exists a regularerasure pattern E ′ that is not correctable in T m + δ × n + γ ( a, b, for any δ, γ .Proof: We show that E ′ = E is not correctable . As E ′ does not contain additional erasures compared to E and therestriction of Definition 4 does not depend on m or n , weconclude that the pattern is regular, i.e., the regularity of apattern in T m × n ( a, b, directly implies the regularity of thesame pattern in T m + δ × n + δ ( a, b, for any δ, γ .By Lemma 5 we have short ([ m + δ ] × [ n + γ ]) \ ([ m ] × [ n ]) ( C ′ ) ∈ C m × n ( a, b, for any code C ′ ∈ C m + δ × n + γ ( a, b, . Byassumption, the restricted pattern E ′ ∩ ([ m ] × [ n ]) = E is notcorrectable in T m × n ( a, b, and the statement follows from 3)in Proposition 2. Lemma 7.
If there exists a regular erasure pattern E that is notcorrectable in T m × n ( a, b, then there exists a regular erasurepattern E ′ that is not correctable in T m + δ × n + γ ( a + δ, b + γ, for any δ, γ .Proof: Denote by C ′ col and C ′ row the column and row codeof a code C ′ ∈ C m + δ × n + γ ( a + δ, b + γ, . Without loss ofgenerality assume that the first m − a and n − b positions area (subset of an) information set of the code C ′ col and C ′ row . Let E ′ be the erasure pattern obtained by adding δ rows and γ columns of erasures to E , i.e., E ′ = E ∪ (cid:0) ([ m + δ ] × [ n + γ ]) \ ([ m ] × [ n ]) (cid:1) . We show that this pattern is regular for thetopology T m + δ × n + γ ( a + δ, b + γ, , i.e., fulfills Definition 4for any U ′ ⊆ [ m + δ ] , |U| = u ≥ a + δ and V ′ ⊆ [ n + γ ] , |V ′ | = v ≥ b + γ . As the δ/γ additional rows/columns consist only oferasures it suffices to show that the subsets of rows/columnswith [ m + 1 , m + δ ] ⊂ U and [ n + 1 , n + γ ] ⊂ V fulfill thecondition. Define U = U ′ ∩ [ m ] and V = V ′ ∩ [ n ] and observethat E ′ can be partitioned into two disjoint subsets with |E ′ ∩ ( U × V ) | ≤ ( v − γ ) a + ( u − δ ) b − ab |E ′ ∩ (( U ′ × V ′ ) \ ( U × V )) | = vδ + uγ − δγ , Even if E is maximal for T m × n ( a, b, , it is not maximal for the topology T m + δ × n + γ ( a, b, for any δ, γ . However, it is also easy to construct a non-correctable maximal pattern by having each additional column/row containexactly a or b erasures, respectively. where the first equality holds because |E ′ ∩ ( U × V ) | = E isregular by definition. Then |E ′ ∩ ( U ′ ×V ′ ) | = |E ′ ∩ ( U ×V ) | + |E ′ ∩ (( U ′ ×V ′ ) \ ( U ×V )) |≤ (( v − γ ) a + ( u − δ ) b − ab ) + ( vδ + uγ − δγ )= v ( a + δ ) + u ( b + γ ) − ( a + δ )( b + γ ) and it follows that the pattern is regular for the topology T m + δ × n + γ ( a + δ, b + γ, .By Lemma 5 we have punct ([ m + δ ] × [ n + γ ]) \ ([ m ] × [ n ]) ( C ′ ) ∈ C m × n ( a, b, . By definition, the pattern E = E ′ \ (cid:0) ([ m + δ ] × [ n + γ ]) \ ([ m ] × [ n ]) (cid:1) is not correctable in this topology andthe statement follows by 4) in Proposition 2. Theorem 4.
Let a, b ≥ . For any m ≥ a + 3 and n ≥ b + 3 there exist regular erasure patterns that are not correctable inthe topology T m × n ( a, b, .Proof: Follows from applying Lemma 6 and 7 to thetopology T × (2 , , , for which an uncorrectable regularerasure pattern exists by Lemma 4.V. C ONNECTION BETWEEN P RODUCT AND T ENSOR -P RODUCT C ODES
TP codes [14] are duals of product codes and of interestfor storage applications due to their small storage overheadand good protection against correlated errors. However, thereis no general classification of erasure patterns correctable bya TP code. In this section, we show how the maximal erasurepatterns of codes and their dual codes are connected. Thisimplies that for certain TP codes, which are duals of MR codesof a grid-like topology with h = 0 , we can exactly characterizetheir correctable erasure patterns. Definition 7 (Tensor-Product Code) . Consider two codes C col [ m, a ] and C row [ n, b ] . Then the tensor-product code TP ( C col , C row ) is the [ mn, mn − ( m − a )( n − b )] code definedas TP ( C col , C row ) = h H col ⊗ H row i ⊥ , where H col and H row denote parity-check matrices of the codes C col and C row ,respectively. Lemma 8.
Let C be an [ m, a ] code and C be an [ n, b ] code.Then TP ( C , C ) ⊥ ∈ C m × n ( a, b, .Proof: Denote by H col and H row parity-check ma-trices of the codes C col and C row , respectively. We have TP ( C col , C row ) ⊥ = h H col ⊗ H row i . The statement follows frominterpreting the matrices H col and H row as generator matricesof [ m, m − a ] and [ n, n − b ] codes, respectively. Lemma 9.
Consider any code C [ n, k ] and denote by E theset of erasure patterns that are correctable in this code. Thenan erasure pattern E is correctable in C ⊥ if and only if thereexists a pattern E max with E ⊆ E max and |E max | = n − k suchthat [ n ] \ E max ∈ E .Proof: An erasure pattern is correctable if and only ifit is the subset of the complement of an information set.Equivalently, an erasure pattern is correctable if and only ifthe corresponding positions are linearly independent in the dualcode. This implies the lemma statement.
Corollary 3.
The set of erasure patterns correctable in a code C uniquely determines the set of erasure patterns correctablein the dual code C ⊥ . Theorem 5.
Let C col be an [ m, a ] code and C row be an [ n, b ] code. The set of maximal erasure patterns correctable by theTP code TP ( C col , C row ) is a subset of { ([ m ] × [ n ]) \ E | E ∈ E max m × n ( a, b, } . oreover, if TP ( C col , C row ) ⊥ ∈ C MR m × n ( a, b, then it correctsall these erasure patterns. Applying Theorem 5 to the cases where E m × n ( a, b, isknown (see Table I) establishes the correctable erasure patternsin the corresponding TP codes.R EFERENCES[1] P. Gopalan, C. Huang, H. Simitci, and S. Yekhanin, “On the locality ofcodeword symbols,”
IEEE Transactions on Information theory , vol. 58,no. 11, pp. 6925–6934, 2012.[2] A. S. Rawat, O. O. Koyluoglu, N. Silberstein, and S. Vishwanath,“Optimal locally repairable and secure codes for distributed storagesystems,”
IEEE Transactions on Information Theory , vol. 60, no. 1, pp.212–236, 2013.[3] M. Chen, C. Huang, and J. Li, “On the maximally recoverable propertyfor multi-protection group codes,” in . IEEE, 2007, pp. 486–490.[4] M. Blaum, J. S. Plank, M. Schwartz, and E. Yaakobi, “Constructionof partial MDS and sector-disk codes with two global parity symbols,”
IEEE Transactions on Information Theory , vol. 62, no. 5, pp. 2673–2681,2016.[5] M. Blaum, J. L. Hafner, and S. Hetzler, “Partial-MDS codes andtheir application to RAID type of architectures,”
IEEE Transactions onInformation Theory , vol. 59, no. 7, pp. 4510–4519, 2013.[6] P. Gopalan, C. Huang, B. Jenkins, and S. Yekhanin, “Explicit maximallyrecoverable codes with locality,”
IEEE Transactions on InformationTheory , vol. 60, no. 9, pp. 5245–5256, 2014.[7] G. Hu and S. Yekhanin, “New constructions of SD and MR codesover small finite fields,” in . IEEE, 2016, pp. 1591–1595.[8] P. Gopalan, G. Hu, S. Kopparty, S. Saraf, C. Wang, and S. Yekhanin,“Maximally recoverable codes for grid-like topologies,” in
Proceedingsof the Twenty-Eighth Annual ACM-SIAM Symposium on Discrete Algo-rithms . SIAM, 2017, pp. 2092–2108.[9] S. Balaji and P. V. Kumar, “On partial maximally-recoverable andmaximally-recoverable codes,” in . IEEE, 2015, pp. 1881–1885.[10] S. Muralidhar, W. Lloyd, S. Roy, C. Hill, E. Lin, W. Liu, S. Pan,S. Shankar, V. Sivakumar, L. Tang et al. , “f4: Facebook’s warm { BLOB } storage system,” in { USENIX } Symposium on Operating SystemsDesign and Implementation ( { OSDI } , 2014, pp. 383–398.[11] D. Shivakrishna, V. A. Rameshwar, V. Lalitha, and B. Sasidharan, “Onmaximally recoverable codes for product topologies,” in . IEEE, 2018,pp. 1–6.[12] X. Kong, J. Ma, and G. Ge, “New bounds on the field size formaximally recoverable codes instantiating grid-like topologies,” arXivpreprint arXiv:1901.06915 , 2019.[13] D. Kane, S. Lovett, and S. Rao, “The independence number of theBirkhoff polytope graph, and applications to maximally recoverablecodes,” SIAM Journal on Computing , vol. 48, no. 4, pp. 1425–1435,2019.[14] J. Wolf, “On codes derivable from the tensor product of check matrices,”
IEEE Transactions on Information Theory , vol. 11, no. 2, pp. 281–284,1965.[15] G. Calis and O. O. Koyluoglu, “A general construction for PMDS codes,”
IEEE Communications Letters , vol. 21, no. 3, pp. 452–455, 2016.[16] T. P. Berger, “Isometries for rank distance and permutation group ofGabidulin codes,”