Explicit MBR All-Symbol Locality Codes
Govinda M. Kamath, Natalia Silberstein, N. Prakash, Ankit S. Rawat, V. Lalitha, O. Ozan Koyluoglu, P. Vijay Kumar, Sriram Vishwanath
11 Explicit MBR All-Symbol Locality Codes
Govinda M. Kamath, Natalia Silberstein, N. Prakash, Ankit S. Rawat,V. Lalitha, O. Ozan Koyluoglu, P. Vijay Kumar, and Sriram Vishwanath
Abstract
Node failures are inevitable in distributed storage systems (DSS). To enable efficient repair when faced with suchfailures, two main techniques are known: Regenerating codes, i.e., codes that minimize the total repair bandwidth;and codes with locality, which minimize the number of nodes participating in the repair process. This paper focuseson regenerating codes with locality, using pre-coding based on Gabidulin codes, and presents constructions that utilizeminimum bandwidth regenerating (MBR) local codes. The constructions achieve maximum resilience (i.e., optimalminimum distance) and have maximum capacity (i.e., maximum rate). Finally, the same pre-coding mechanismcan be combined with a subclass of fractional-repetition codes to enable maximum resilience and repair-by-transfersimultaneously.
I. B
ACKGROUND
A. Vector Codes An [ n, K, d min , α ] vector code over a field F q is a code C of block length n , having a symbol alphabet F αq forsome α > , satisfying the additional property that given c , c (cid:48) ∈ C and a, b ∈ F q , a c + b c (cid:48) also belongs to C . As avector space over F q , C has dimension K , termed the scalar dimension (equivalently, the file size) of the code andas a code over the alphabet F αq , the code has minimum distance d min .Associated with the vector code C is an F q -linear scalar code C ( s ) of length N = nα , where C ( s ) is obtainedby expanding each vector symbol within a codeword into α scalar symbols (in some prescribed order). Given agenerator matrix G for the scalar code C ( s ) , the first code symbol in the vector code is naturally associated withthe first α columns of G etc. We will refer to the collection of α columns of G associated with the i th code symbol c i as the i th thick column and to avoid confusion, the columns of G themselves as thin columns. B. Locality in Vector Codes
Let C be an [ n, K, d min , α ] vector code over a field F q , possessing a ( K × nα ) generator matrix G . The i th codesymbol, c i , is said to have ( r, δ ) locality, δ ≥ , if there exists a punctured code C i := C| S i of C (called a localcode ) with support S i ⊆ { , , · · · , n } such that • i ∈ S i , • | S i | ≤ n L := r + δ − , and • d min ( C| S i ) ≥ δ .The code C is said to have ( r, δ ) information locality if there exists l code symbols with ( r, δ ) locality andrespective support sets { S i } li =1 satisfying • Rank ( G | ∪ li =1 S i ) = K .The code C is said to have ( r, δ ) all-symbol locality if all code symbols have ( r, δ ) locality. A code with ( r, δ ) information (respectively, all-symbol) locality is said to have full ( r, δ ) information (respectively, all-symbol) locality,if all local codes have parameters given by | S i | = r + δ − and d min ( C i ) = δ , for i = 1 , · · · , l .The concept of locality for scalar codes, with δ = 2 , was introduced in [1] and extended in [2] and [3] to scalarcodes with arbitrary δ , and vector codes with δ = 2 , respectively. This was further extended to vector codes witharbitrary δ in [4] and [5], where, in addition to constructions of vector codes with locality, authors derive minimumdistance upper bounds and also consider settings in which the local codes have regeneration properties. Govinda M. Kamath, N. Prakash, V. Lalitha and P. Vijay Kumar are with the Department of ECE, Indian Institute of Science,Bangalore, India (email: { govinda, prakashn, lalitha, vijay } @ece.iisc.ernet.in). Natalia Silberstein, Ankit S. Rawat, O. Ozan Koyluoglu,and Sriram Vishwanath are with Department of ECE, University of Texas at Austin, Austin, USA (email: { natalys, ankitsr, ozan,sriram } @austin.utexas.edu). This research is supported in part by the National Science Foundation under Grant 0964507 and in part bythe NetApp Faculty Fellowship program. The work of V. Lalitha is supported by a TCS Research Scholarship. a r X i v : . [ c s . I T ] M a y onsider now a vector code C with full ( r, δ ) locality whose associated local codes C i have parameters [ n L , K L , δ ] .In this paper, we are interested in local codes that have the uniform rank accumulation property, in particular, localMBR codes and local fractional-repetition codes. Definition 1 (Uniform rank accumulation (URA) codes) . Let G be a generator matrix for a code C , and S i be anarbitrary subset of i thick columns of G , for some i = 1 , · · · , n . Then, C is an URA code, if the restriction G | S i of G to S i , has rank ρ i that is independent of the specific subset S i of i indices chosen and given by ρ i = (cid:80) ij =1 a j for some set of non-negative integers { a j } . We will refer to the sequence { a i , ≤ i ≤ n } as the rank accumulation profile of the code C .We now present the minimum distance upper bound given in [4] for the case when local codes are URA codes.Consider the finite length vector ( a , a , · · · , a n L ) , and its extension to a periodic semi-infinite sequence { a i } ∞ i =1 of period n L by defining a i + jn L = a i , ≤ i ≤ n L , j ≥ . Let P ( · ) denote the sequence of partial sums, P ( s ) = s (cid:88) i =1 a i , s ≥ . (1)Then, given integers u ≥ , ≤ u ≤ n L , P ( u n L + u ) = u K L + P ( u ) . Next, let us define the function P ( inv ) by setting P ( inv ) ( ν ) , for ν ≥ , to be the smallest integer s such that P ( s ) ≥ ν . It can be verified that for v ≥ and ≤ v ≤ K L , P ( inv ) ( v K L + v ) = v n L + P ( inv ) ( v ) , where P ( inv ) ( v ) ≤ r as ≤ v ≤ K L .The minimum distance of a code C whose local codes C i are URA codes can be bounded as follows. Theorem I.1 (Theorem 5.1 of [4]) . The minimum distance of C is upper bounded by d min ≤ n − P ( inv ) ( K ) + 1 . (2)The codes achieving the bound in (2) are referred to as codes having optimal locality. For such locality optimalcodes, one can then analyze whether the code allows for efficient data storage in DSS. Towards this end, file sizebound for codes with locality are given in [5] using the min-cut techniques similar to that of [3]. As noted in [4],when URA codes are used as local codes, the file size bound for d min -optimal codes can be represented in theform K ≤ P ( n − d min + 1)= (cid:18)(cid:24) n − d min + 1 n L (cid:25) − (cid:19) K L + P ( l ) , (3)where l ∈ { , · · · , n L } is such that n − d min + 1 = (cid:18)(cid:24) n − d min + 1 n L (cid:25) − (cid:19) n L + l . We note that P ( l ) = P ( r ) , for r ≤ l . C. MBR Codes An (( n, k, d ) , ( α, β ) , K ) minimum-bandwidth regenerating (MBR) code is an [ n, K, d min = n − k + 1 , α ] vectorcode satisfying additional constraints described below. The code is intended to be used in a distributed storagenetwork in which each code symbol is stored within a distinct node. The code is structured in such a way thatthe entire file can be recovered by processing the contents of any k , ≤ k ≤ n nodes. Further, in case of asingle node failure, the replacement node can reconstruct the data stored in the failed node by connecting to any d , k ≤ d ≤ n − , nodes and downloading β = αd symbols from each node. The scalar dimension (or file size)parameter K can be expressed in terms of the other parameters as: K = (cid:18) dk − (cid:18) k (cid:19)(cid:19) β, s proved in [6]. A cut-set bound derived from network coding shows us that the file size cannot be any larger, andthus, MBR codes are example of regenerating codes that are optimal with respect to file size. A regenerating codeis said to be exact if the replacement of a failed node stores the same data as did the failed node, and functionalotherwise. We are concerned here only with exact-repair codes. Constructions of MBR codes for all k ≤ d = α < n and β = 1 are presented in [7]. MBR codes with repair by transfer and d = n − are presented in [8].It can be inferred from the results in [8] that MBR codes are URA codes. In particular, for an (( n, k, d ) , ( α, β ) , K ) MBR code, the rank accumulation profile is given by a j = (cid:26) α − ( j − β, ≤ j ≤ k , k + 1 ≤ j ≤ n. (4) D. MBR-Local Codes
Let C be an [ n, K, d min , α ] vector code with • full ( r, δ ) -information locality with δ ≥ , and • all of whose associated local codes C i , i ∈ L are MBR codes with identical parameters (( n L = r + δ − , r, d ) , ( α, β ) , K L ) .Then, the dimension of each local code is given by K L = n L (cid:88) i =1 a i = αr − (cid:18) r (cid:19) β, (5)where { a i , ≤ i ≤ n L } is the rank accumulation profile of the MBR code C .
1) Minimum distance bound for MBR-Local Codes:
As MBR codes are URA codes, from Theorem I.1, we have d min ≤ n − P ( inv ) ( K ) + 1 , (6)where, for MBR codes we have P ( inv ) ( v K L + v ) = v n L + ν (7)for some v ≥ , ≤ v ≤ K L , and ν is uniquely determined from α ( ν − − (cid:0) ν − (cid:1) β < v ≤ αν − (cid:0) ν (cid:1) β .
2) File size bound for MBR-Local Codes:
From (3), the file size bound for an optimal locality code with MBRlocal codes is given by K ≤ (cid:18)(cid:24) n − d min + 1 n L (cid:25) − (cid:19) K L + αµ − (cid:18) µ (cid:19) β, (8)where µ = min { l , r } with l as defined in Subsection I-B. Note that (8) follows from the rank accumulationprofile of MBR codes, i.e., from (4). E. Linearized Polynomials
A polynomial f ( x ) over the field F q m , is said to be linearized of q -degree t , if f ( x ) = t (cid:88) i =0 u i x q i , u i ∈ F q m , u t (cid:54) = 0 . (9)A linearized polynomial f ( x ) over F q m satisfies the following property [9]: f ( λ θ + λ θ ) = λ f ( θ ) + λ f ( θ ) ∀ θ , θ ∈ F q m , λ , λ ∈ F q . (10)A linearized polynomial f ( x ) over F q m of q -degree t , m > t , is uniquely determined from its evaluation at a setof ( t + 1) points g , · · · , g t +1 ∈ F q m , that are linearly independent over F q . . Gabidulin Maximum Rank Distance Codes Now, we present a construction of maximum rank distance codes, provided by Gabidulin in [10]. This codes canbe viewed as a rank-metric analog of Reed-Solomon codes.The rank of a vector v ∈ F N q m , denoted by rank( v ) is defined as the rank of the m × N matrix V over F q ,obtained by expansion of every entry of v to a column vector in F mq , based on the isomorphism between F q m and F mq . Similarly, for two vectors v , u ∈ F N q m , the rank distance is defined by d R ( v , u ) = rank( V − U ) .An [ N , K , D ] q m rank-metric code C ⊆ F N q m is a linear block code over F q m of length N , dimension K andminimum rank distance D . A rank-metric code that attains the Singleton bound D ≤ N − K + 1 in rank-metric iscalled a maximum rank distance (MRD) code. For m ≥ N , a construction of MRD codes, called Gabidulin codesis given as follows [10].A codeword in an [ N , K , D = N − K + 1] q m Gabidulin code C Gab , m ≥ N , is defined as c = ( f ( θ ) , f ( θ ) , . . . , f ( θ N )) ∈ F N q m , (11)where f ( x ) is a linearized polynomial over F q m of q -degree at most K − with the coefficients given by theinformation message, and where the θ , . . . , θ N ∈ F q m are linearly independent over F q [10].II. C ONSTRUCTION OF C ODES WITH
MBR L
OCALITY
In this section, we will present two constructions of codes with local regeneration. In both cases, the localcodes are MBR codes with identical parameters and both codes are optimal, i.e., they achieve the upper bound ofTheorem I.1 on minimum distance. The first construction is an all-symbol locality construction, while the secondhas information locality.The constructions presented in this paper, adopt the linearized polynomial approach made use of in [11], [12],[5]. In particular, similar to the constructions proposed in [12], [5], the constructions of this paper have a two-stepencoding process with the first step utilizing Gabidulin codes, which in turn, are based on linearized polynomials.The first code construction given below also proves the tightness of the bound on minimum distance of codes withURA derived in [4] (Theorem 5.1) for the case when K L (cid:45) K , where K L is the scalar dimension of the local MBRcode.Consider a code C BASIC that is simply the concatenation of t local MBR codes having identical parameters (( n L , k, d ) , ( α, β ) , K L ) . Thus a typical codeword c ∈ C BASIC looks like c = (cid:0) c mbr1 c mbr2 · · · c mbr t (cid:1) , where each vector c mbr i is a codeword belonging to the MBR code. The generator matrix G BASIC of the code willclearly have a block-diagonal structure. It is straightforward to show that the smallest number ρ , such that any ρ thick columns of G BASIC have rank ≥ K is given by P ( inv ) ( K ) , for any ≤ K ≤ tK L . Construction II.1.
We will describe the construction by showing how encoding of a message vector takes place.The encoding is illustrated in Fig. 1. Given the message vector u ∈ F Kq m , we first encode u to a tK L longGabidulin codeword using tK L linearly independent points (over F q ) { θ , θ , . . . , θ tK L } ⊂ F q m , i.e., by applyingan [ tK L , K, tK L − K + 1] q m Gabidulin code, assuming m ≥ tK L . We then partition tK L symbols of the Gabidulincodeword, ( f ( θ ) , f ( θ ) , . . . , f ( θ tK L )) , into t disjoint sets of K L symbols each. Each of these sets is then fed inas a message vector to a bank of t identical MBR encoders whose outputs constitute (( n L , r, d ) , ( α, β ) , K L ) MBRcodes. If (cid:8) c mbr i | i = 1 , , . . . , t (cid:9) is the resulting set of t codewords, these codewords are then concatenated toobtain the desired codeword c . The code C thus constructed has: • length n = tn L • t local (( n L , r, d ) , ( α, β ) , K L ) MBR codes with disjoint supports • full ( r, δ ) all-symbol locality where δ is defined from n L = r + δ − . Theorem II.2.
Given any set of parameters n, r, δ, K , such that n = tn L and K ≤ tK L , the construction II.1,yields an optimal MBR-local code with full ( r, δ ) all-symbol locality whose minimum distance is given by d min = n − P (inv) ( K ) + 1 . ig. 1. Illustrating the two-step construction of the all-symbol MBR-local code. We first present a useful lemma on codes that are obtained by concatenating a Gabidulin code (over F q m ) witha vector code (over F q ). Lemma II.3.
Let G be the generator matrix of an [ n, J, d min , α ] vector code over the field F q . Let ˜ J be an integersuch that ˜ J ≤ J . Let ρ be the smallest integer, such that the submatrix of G obtained by selecting any ρ thickcolumns of G results in a matrix of rank ≥ ˜ J . Let f ( x ) = ˜ J − (cid:88) i =0 u i x q i , u i ∈ F q m , m > J, be a linearized polynomial of q -degree at most ˜ J − over the extension field F q m , for m ≥ J . Let { θ i } Ji =1 be anycollection of J elements of F q m that are linearly independent over F q . The mapping ( u , u , · · · , u ˜ J − ) → ( f ( θ ) , f ( θ ) , · · · , f ( θ J )) G defines a linear code C over F q m having message vector ( u , u , · · · , u ˜ J − ) . Then C has minimum distance D min given by D min = n − ρ + 1 , i.e., C is an [ n, ˜ J , D min ] code over F q m .Proof: Since f ( · ) is linearized, we can interchange linear operations with the operation of evaluation: ( f ( θ ) f ( θ ) · · · f ( θ J )) G = f (( θ θ · · · θ J ) G ) . We have extended here the definition of f ( · ) to vectors through termwise application. Consider next, the matrixproduct Γ := [ θ θ · · · θ J ] G. In writing this, we have abused notation and identified elements in F q m with their representations as vectors over F q lying in F mq . The m × J matrix [ θ θ · · · θ J ] on the left has the property that all of its columns are linearlyindependent. Hence linear dependence relations amongst columns of Γ are precisely those inherited from the matrix G . It follows that ρ is also the smallest number, such that any ρ thick columns of the product matrix Γ have rank ≥ ˜ J . Since f ( · ) is uniquely determined by its evaluation at a collection of ˜ J linearly independent vectors lying in F mq , it follows that the maximum number of erasures that the code C can recover from is given by n − ρ . Then,we have D min = n − ρ + 1 . roof: (of Thm. II.2) Let G BASIC be the generator matrix of the code that is simply the disjoint union of the t MBR codes. As it was explained previously, the smallest number ρ of thick columns of G BASIC such that any ρ columns of G BASIC have rank ≥ K is given by P ( inv ) ( K ) . It follows therefore from Lemma II.3 (by substituting ˜ J = K , J = tK L and also assuming that G BASIC is over F q ) that the code has minimum distance given by d min = n − P (inv) ( K ) + 1 , hence the code attains the bound of Theorem I.1, and thus, optimal. Remark 1.
We note that whenever K = v K L + v , v ≥ , ≤ v ≤ K L is such that v = να − (cid:0) ν (cid:1) β for some ≤ ν ≤ r , then the code constructed by Construction II.1 has maximum possible scalar dimension given in (8) .This observation holds for the code we will construct using Construction II.4 as well. Construction II.4.
We describe here a method by which we construct a code of length n = tn L + ∆ , with ( r, δ ) information locality for scalar dimension K ≤ tK L . Given the message vector u ∈ F Kq m , we first encode u to a tK L + ∆ α long Gabidulin codeword using a [ tK L + ∆ α, K, tK L + ∆ α − K + 1] q m Gabidulin code, for m ≥ tK L + ∆ α . We then divide the first tK L symbols of the Gabidulin codeword into t disjoint groups of equalsize and encode each of these t groups using an (( n L , r, d ) , ( α, β ) , K L ) MBR code (similar to the second step ofencoding in Construction II.1). This gives us a code of length tn L with MBR all-symbol locality, whose elementsare (cid:8) c mbr i | i = 1 , , . . . , t (cid:9) . We then partition the remaining ∆ α symbols of the Gabidulin codeword into ∆ equalsets and denote the i th set by c tn L + i . The construction outputs ( c mbr1 , . . . , c mbr t , c tn L +1 , . . . , c tn L +∆ ) as a finalcodeword. The resultant vector code C has: • Length n = tn L + ∆ • t local (( n L , r, d ) , ( α, β ) , K L ) MBR codes with disjoint support • full ( r, δ ) information locality Theorem II.5.
Given any set of parameters n, r, δ, K , such that n = tn L + ∆ and K ≤ tK L , Construction II.1results in an optimal MBR-local code with ( r, δ ) information locality whose minimum distance is given by d min = n − P (inv) ( K ) + 1 . Proof:
The proof follows along the same lines as the proof of Theorem II.2.III. F
RACTIONAL -R EPETITION C ODES AS L OCAL C ODES
In this section, we discuss the usage of fractional repetition (FR) codes as local codes in Constructions II.1and II.4. FR codes can be viewed as a generalization of repair-by-transfer MBR codes, where a repair process is uncoded and table-based , i.e., FR codes have a ”repair-by-transfer” property, while only specific sets of nodes of size d participate in a node repair process. For the sake of completeness, we provide an overview of the t -design-basedconstruction for FR codes presented in [13] .Let t, n, w, λ be integers with n > w ≥ t and λ > . A t - ( n, w, λ ) design is a collection B of w -subsets (the blocks ), of an n -set X (the points ), such that every t -subset of X is contained in exactly λ blocks. Let x , . . . , x t ∈ X be a set of t points. We denote by λ s the number of blocks containing x , . . . , x s , ≤ s ≤ t . Then, λ s = λ (cid:0) n − st − s (cid:1)(cid:0) w − st − s (cid:1) ; the number of blocks in the t -design is b = λ = λ (cid:0) nt (cid:1) / (cid:0) wt (cid:1) ; and each point in X is contained in λ blocks where λ = λ (cid:0) n − t − (cid:1) / (cid:0) w − t − (cid:1) [9]. Construction III.1.
Let B , . . . , B b ∈ B be the blocks and x , . . . , x n ∈ X the points of a t - ( n, w, λ ) design. Thenthe n nodes of a FR code C are given by the points of the design, i.e., a node N i contains α (cid:44) λ symbols givenby N i = { j : x i ∈ B j } . Note that the cardinality of an intersection of any s ≤ t nodes are given by the numbers λ s , The construction in [13] sets t = 2 and λ = 1 ; and the corresponding codes are called transposed codes.ig. 2. Fractional Repetition Code based on − (7 , , design.Fig. 3. Fano Plane, a − (7 , , design. and hence the cardinality of a union of any s ≤ t nodes can be easily derived by the inclusion-exclusion formula.Let k, K be two integers such that k ≤ t and | k − (cid:91) i =1 N i | < K ≤ | k (cid:91) i =1 N i | . (12) Then we have an FR code over an alphabet of size b , with the property that there exists a set of d nodes whichcan repair a failed node and from any set of k nodes one can reconstruct the original K symbols.Given a message vector [ m m · · · m K ] , we encode the message symbols first by using an [ b, K, b − K + 1] MDS code to produce b coded symbols ( c , c · · · c b ) and then by employing the FR code based on the t -designto produce n nodes each containing λ symbols. This family of FR codes based on t -designs is also an example of codes with uniform rank accumulation, andthus the bound of Theorem I.1 can be used here as well. Thus, we have the following result. Theorem III.2.
When FR codes based on a t -design obtained by Construction III.1 are used as the local codesin Constructions II.1 and II.4, then the resulting code with locality attains the bound of Theorem I.1 on minimumdistance. An example of an encoding is shown in Fig. 2, where the encoding is done using - (7 , , design, also knownas the Fano plane (see Fig. 3). When we replace a local MBR code with the FR code based on Fano plane inFig. 1, we obtain a code with locality which has the optimal minimum distance.R EFERENCES [1] P. Gopalan, C. Huang, H. Simitci, and S. Yekhanin, “On the locality of codeword symbols,”
IEEE Trans. Inf. Theory , vol. 58, no. 11,pp. 6925–6934, Nov. 2012.[2] N. Prakash, G. M. Kamath, V. Lalitha, and P. V. Kumar, “Optimal linear codes with a local-error-correction property,” in
Proc. IEEEInt. Symp. Inf. Theory (ISIT) , Cambridge, MA, Jul. 2012, pp. 2776–2780.[3] D. S. Papailiopoulos and A. G. Dimakis, “Locally repairable codes,” in
Proc. IEEE Int. Symp. Inf. Theory (ISIT) , Cambridge, MA, Jul.2012, pp. 2771–2775.4] G. M. Kamath, N. Prakash, V. Lalitha, and P. V. Kumar, “Codes with local regeneration,” Nov. 2012. [Online]. Available:arXiv:1211.1932[5] A. S. Rawat, O. O. Koyluoglu, N. Silberstein, and S. Vishwanath, “Optimal locally repairable and secure codes for distributed storagesystems,” Oct. 2012. [Online]. Available: arXiv:1210.6954[6] A. G. Dimakis, P. B. Godfrey, Y. Wu, M. J. Wainwright, and K. Ramchandran, “Network coding for distributed storage systems,”
IEEETrans. Inf. Theory , vol. 56, no. 9, pp. 4539–4551, Sep. 2010.[7] K. V. Rashmi, N. B. Shah, and P. V. Kumar, “Optimal exact-regenerating codes for distributed storage at the MSR and MBR pointsvia a product-matrix construction,”
IEEE Trans. Inf. Theory , vol. 57, no. 8, pp. 5227–5239, Aug. 2011.[8] N. B. Shah, K. V. Rashmi, P. V. Kumar, and K. Ramchandran, “Distributed storage codes with repair-by-transfer and nonachievabilityof interior points on the storage-bandwidth tradeoff,”
IEEE Trans. Inf. Theory , vol. 58, no. 3, pp. 1837–1852, Mar. 2012.[9] F. J. MacWilliams and N. J. A. Sloane,
The Theory of Error-Correcting Codes . Amsterdam: North-Holland, 1983.[10] E. M. Gabidulin, “Theory of codes with maximum rank distance,”
Problems of Information Transmission , vol. 21, pp. 1–12, July 1985.[11] F. Oggier and A. Datta, “Self-repairing homomorphic codes for distributed storage systems,” in
Proc. IEEE Int. Conf. Comput.Communications (INFOCOM) , Shanghai, China, Apr. 2011, pp. 1215–1223.[12] N. Silberstein, A. S. Rawat, and S. Vishwanath, “Error resilience in distributed storage via rank-metric codes,” in
Proc. 50th AnnualAllerton Conf. on Communication, Control, and Computing (Allerton) , Urbana-Champaign, IL, Oct. 2012, pp. 1150 –1157.[13] S. El Rouayheb and K. Ramchandran, “Fractional repetition codes for repair in distributed storage systems,” in