LDPC Codes Which Can Correct Three Errors Under Iterative Decoding
Shashi Kiran Chilappagari, Anantha Raman Krishnan, Bane Vasic
aa r X i v : . [ c s . I T ] J a n LDPC Codes Which Can Correct Three ErrorsUnder Iterative Decoding
Shashi Kiran Chilappagari
Dept. of Electrical and Computer Eng.University of ArizonaTucson, AZ 85721, USAEmail: [email protected]
Anantha Raman Krishnan
Dept. of Electrical and Computer Eng.University of ArizonaTucson, AZ 85721, USAEmail: [email protected]
Bane Vasi´c
Dept. of ECE and Dept. of MathematicsUniversity of ArizonaTucson, AZ 85721, USAEmail: [email protected]
Abstract — In this paper, we provide necessary and sufficientconditions for a column-weight-three LDPC code to correctthree errors when decoded using Gallager A algorithm. Wethen provide a construction technique which results in a codesatisfying the above conditions. We also provide numericalassessment of code performance via simulation results.
I. I
NTRODUCTION
Iterative message passing algorithms for decoding low-density parity-check (LDPC) codes have been the focus ofresearch over the past decade and most of their properties arewell understood [1],[2]. These algorithms operate by passingmessages along the edges of a graphical representation of thecode known as the Tanner graph and are optimal when theunderlying graph is a tree. Message passing decoders performremarkably well which can be attributed to their abilityto correct errors beyond the traditional bounded distancedecoding capability. However, in contrast to bounded dis-tance decoders (BDDs), iterative decoders cannot guaranteecorrection of a fixed number of errors at relatively short codelengths. This is due to the fact that the associated Tannergraphs for short length codes have cycles and the decod-ing becomes suboptimal and there exist a few low-weightpatterns (termed as near codewords [3] or trapping sets [4])uncorrectable by the decoder. It is now well established thatthe trapping sets lead to the phenomenon of error floor.Roughly, error floor is an abrupt change in the frame errorrate (FER) performance of an iterative decoder in the highsignal-to-noise ratio (SNR) region.The error floor problem is well understood for iterativedecoding over binary erasure channel (BEC) [5]. The decoderfails when the received vector contains erasures in locationscorresponding to a stopping set. For the AWGN channel,Richardson in [4] presented a numerical method to estimateerror floors of LDPC codes. He established a relation betweentrapping sets and the FER performance of the code in theerror floor region (the necessary definitions will be givenin the next section). The approach from [4] was furtherrefined by Stepanov et al in [6]. Vontobel and Koetter [7]established a theoretical framework for finite length analysisof message passing iterative decoding based on graph covers.This approach was used by Smarandache et al in [8] toanalyze performance of LDPC codes from projective andfor LDPC convolutional codes [9]. For the binary symmetricchannel (BSC), error floor estimation based on trapping setswas proposed in [10] and we adopt the notation from [10]. In this paper, we make the following two fundamentalcontributions: (a) give necessary and sufficient conditions fora column-weight-three LDPC code to correct three errors, and(b) propose a construction method which results in a codesatisfying the above conditions.We consider hard decision decoding for transmission overBSC. The BSC is a simple yet useful channel model usedextensively in areas where decoding speed is a major factor.Note that the problem of recovering from a fixed number oferasures is solved for the BEC. If the Tanner graph of a codedoes not contain any stopping sets up to size t (the size ofminimum stopping set is t +1 ), then the decoder is guaranteedto recover from any t erasures. An analogous result forthe BSC is still unknown. The problem of guaranteed errorcorrection capability is known to be difficult and in thispaper, we present a first step toward such result. Previously,expansion arguments were used to show that message passingcan correct a fixed fraction of errors [11]. However, thecode length needed to guarantee such correction capabilityis generally very large and to correct three errors, the lengthwould be in the order of a few hundred thousand. Also, thesearguments cannot be used for column-weight-three codes.Column-weight-three codes are of special importance as theirdecoders have very low complexity and are used in a widerange of applications.We also show that the slope of the frame error rate(FER) is dependent on the critical number of the mostrelevant trapping sets and hence the slope can be improvedby avoiding such trapping sets. We provide a technique toconstruct codes which outperform empirically best knowncodes of the same length. Our method can be seen as amodification of the progressive edge growth (PEG) techniqueproposed in [12].The rest of the paper is organized as follows. In Section IIwe establish the notation, describe the Gallager A algorithmand define trapping sets. In Section III we present the maintheorem which gives the necessary and sufficient conditionsto correct three errors. In Section IV we describe a techniqueto construct codes satisfying the conditions of the theoremand provide numerical results. We conclude with a fewremarks in Section VII. D ECODING A LGORITHMS AND T RAPPING S ETS
In this section, we establish the notation and describea hard decision decoding algorithm known as Gallager Algorithm. We then characterize the failures of the GallagerA decoder with the help of fixed points. We also introducethe notions of trapping sets and critical number.
A. Graphical Representations of LDPC Codes
The Tanner graph of an LDPC code, G , is a bipartitegraph with two sets of nodes: variable (bit) nodes and check(constraint) nodes. Every edge e in the bipartite graph isassociated with a variable node v and check node c . Thecheck nodes / variable nodes connected to a variable node /check node are referred to as its neighbors. The degree of anode is the number of its neighbors. In a ( γ, ρ ) regular LDPCcode, each variable node has degree of γ and each check nodehas degree ρ . The girth g is the length of the shortest cyclein G . In this paper, • represents a variable node, (cid:3) representsan even degree check node and (cid:4) represents an odd degreecheck node. B. Hard Decision Decoding Algorithms
Gallager in [13] proposed two simple binary messagepassing algorithms for decoding over the BSC; GallagerA and Gallager B. See [14] for a detailed description ofGallager B algorithm. For column-weight-three codes, whichare the main focus of this paper, these two algorithms arethe same. Every round of message passing (iteration) startswith sending messages from variable nodes (first half of theiteration) and ends by sending messages from check nodesto variable nodes (second half of the iteration). Initially, thevariable nodes send their received values to the neighboringchecks. In the k th iteration ( k = 2 , , . . . ) , a variable node, v sends the following message, −→ m i ( e ) , along edge e to itsneighboring check node c ; if all incoming messages to v other than the message from c are equal to a certain value,it sends that value; else, it sends the received value. A checknode c sends to a variable node v , the modulo two sum of allincoming messages except the message from v . At the endof each iteration, an estimate of each variable node is madebased on the incoming messages and possibly the receivedvalue. The decoder is run until a valid codeword is found orfor a maximum number of iterations is reached, whicheveris earlier. See [15] for a detailed description of the messagespassed in Gallager A algorithm. A Note on the Decision Rule:
Different rules to estimatea variable node after each iteration are possible and it islikely that changing the rule after certain iterations maybe beneficial. However, the analysis of various scenariosis beyond the scope of this paper. For column-weight-threecodes only two rules are possible. • Decision rule A: if all incoming messages to a variablenode from neighboring checks are equal, set the variablenode to that value; else set it to received value • Decision rule B: set the value of a variable node tothe majority of the incoming messages; majority alwaysexists since the column-weight is threeWe adopt Decision rule A throughout this paper.
C. Trapping Sets of Gallager A Algorithm
We now characterize failures of the Gallager A decoderusing fixed points and trapping sets. Much of the following discussion appears in [16],[15],[10],[17] and we include itfor sake of completeness. Consider an LDPC code of length n and let x be the binary vector which is the input to theGallager A decoder. Let S ( x ) be the support of x . Thesupport of x is defined as the set of all positions i where x i = 0 . Definition 1: [16] A decoder failure is said to have oc-curred if the output of the decoder is not equal to thetransmitted codeword.
Definition 2: [16] x is called a fixed point if for every edge e and its associated variable node v −→ m k ( e ) = x ( v ) , ∀ k That is, the message passed from variable nodes to checknodes along the edges are the same in every iteration. Sincethe outgoing messages from variable nodes are same in everyiteration, it follows that the incoming messages from checknodes to variable nodes are also same in every iteration andso is the estimate of a variable after each iteration. In fact,the estimate after each iteration coincides with the receivedvalue. It is clear from above definition that if the input to thedecoder is a fixed point, then the output of the decoder isthe same fixed point. Without loss of generality, we assumethat the all zero codeword is sent over BSC and the inputto the decoder is the error vector. So, a fixed point withsmall weight means that few errors lead to decoder failure. Adetailed discussion about different kinds of decoder failuresis given in [17]
Definition 3: [10] The support of a fixed point is known asa trapping set. A ( V, C ) trapping set T is a set of V variablenodes whose induced subgraph has C odd degree checks.Our definition of a trapping set gives necessary and suffi-cient conditions for a set of variable nodes to form a trappingset. We state the following theorem which is a consequenceof Fact 3 from [4]. Theorem 1: [16] Let T be a set consisting of v variablenodes with induced subgraph I . Let the checks in I bepartitioned into two disjoint subsets; O consisting of checkswith odd degree and E consisting of checks with even degree.Let |O| = c and |E| = s . T is a trapping set if : (a) Everyvariable node in I is connected to at least two checks in E and at most one checks in O and (b) No two checks of O are connected to a variable node outside I . Proof:
See [16].If the variable nodes corresponding to a trapping set are inerror, then a decoder failure occurs. However, not all variablenodes corresponding to trapping set need to be in error for adecoder failure to occur.
Definition 4: [10] The minimal number of variable nodesthat have to be initially in error for the decoder to end up inthe trapping set T will be referred to as critical number m for that trapping set. Definition 5: [16] A set of variable nodes which if in errorlead to a decoding failure is known as a failure set . Remarks
1) To “end up” in a trapping set T means that, after apossible finite number of iterations, the decoder willbe in error, on at least one variable node from T , atevery iteration [4].) The notion of a failure set is more fundamental than atrapping set. However, from the definition, we cannotderive necessary and sufficient conditions for a set ofvariable nodes to form a failure set.3) A trapping set is a failure set. Subsets of trapping setscan be failure sets. More specifically, for a trapping setof size V , there exists at least one subset of size equalto the critical number which is a failure set.4) The critical number of a trapping set is not fixed. Itdepends on the outside connections of checks in E .However, the maximum value of critical number of a ( V, C ) trapping set is V .III. N ECESSARY AND S UFFICIENT C ONDITIONS TO C ORRECT T HREE E RRORS
In this section, we establish the necessary and sufficientconditions for a column-weight-three code to correct threeerrors. We first illustrate three trapping sets and show thatthe critical number of these trapping sets is three therebyproviding necessary condition to correct three errors. We thenprove that avoiding structures isomorphic to these trappingsets in the Tanner graph is sufficient to guarantee correctionof three errors.Fig. 1 shows three subgraphs induced by different numberof variable nodes. Let us assume that in all these inducedgraphs, no two odd degree checks are connected to a variablenode outside the graph. By the conditions of Theorem 1, allthese induced subgraphs are trapping sets. Fig. 1(a) is a (3 , trapping set, Fig. 1(b) is a (5 , trapping set and Fig. 1(c)is a (8 , trapping set. Note that a (3 , is isomorphic to asix cycle. and the (8 , trapping set is a codeword of weighteight. Lemma 1:
The critical number for (3 , trapping set isthree. There exist (5 , and (8 , trapping sets with criticalnumber three. Proof:
For the (3 , trapping set, the result followsfrom definition. We omit the proof for (5 , and (8 , trapping sets due to space considerations. Detailed proofscan be found in the longer version of the paper [15]. Theorem 2:
To correct three errors in a column-weight-three LDPC code by Gallager A algorithm, it is necessary toavoid (3 , trapping sets and (5 , and (8 , trapping setswith critical number three in its Tanner graph. Proof:
Follows from the above discussion.We now state and prove the main theorem.
Theorem 3:
If the Tanner graph of a column-weight-threeLDPC codes has girth eight and no set of variable nodesinduces a subgraph isomorphic to (5 , trapping set or asubgraph isomorphic to (8 , trapping sets, then any threeerrors can be corrected using Gallager A algorithm. Sketch of proof:
In a column-weight-three code three variablenodes can induce only one of the five subgraphs given inFig. 2 and the proof proceeds by examining these subgraphsone at a time. The complete proof involves many argumentsand here we just illustrate the methodology of the proofby considering two possible subgraphs. The proof for theremaining subgraphs appears in the longer version of thepaper [15].
Subgraph 1:
Since the girth of the code is eight, it hasno six cycles and hence the configuration in Fig. 2(a) is notpossible.
Subgraph 5:
The three variable nodes in error induce asubgraph as shown in Fig. 2(e). In first half of first iteration , and send incorrect messages. In the second half of firstiteration, a, b, c, d, e, f, g, h and i send incorrect messages toneighboring variables except to , and . If there is novariable node which receives three incorrect messages, a validcodeword is reached after first iteration. On the contrary,assume there exists a variable node, say , which receivesthree incorrect messages (w.l.o.g. we can assume that isconnected to a, d and g ). Also, there cannot be two suchvariable nodes as that would introduce a six cycle or agraph isomorphic to (5 , trapping set. Also, there can beat most three variable nodes which receive two incorrectmessages, say, , and . Let the other checks connectedto these variables be j, k and l respectively. In the first halfof second iteration, , and send all correct messages, sends all incorrect messages, , , send incorrect messagesto j, k and l respectively. In second half of second iteration, a, d, g send incorrect messages to their neighbors except to . j, k and l send incorrect messages to neighboring variablesexcept to , and . There cannot be a variable node whichis connected to one check from { j, k, l } and to one checkfrom { a, d, g } . Also, there cannot be a variable node whichis connected to all the three checks j, k and l as this wouldintroduce a graph isomorphic to (8 , trapping set. However,there can be at most two variable nodes which receive twoincorrect messages from the checks j, k and l , say and . Let the other checks connected to and be m and p . At the end of second iteration, , and receive oneincorrect message, and receive two incorrect messages. Inthe first half of third iteration, , and send two incorrectmessages each, and send one incorrect message each.In the second half of third iteration, b, c, e, f, h and i sendincorrect messages to their neighbors except to , and . m and p send incorrect messages to their neighbors exceptto and . It can be shown that there cannot exist a variablenode which receives three incorrect messages. At the end ofthird iteration, , and receive all correct messages andno variable node receives all incorrect messages. So, if adecision is made, a valid codeword is reached and decoderis successful. Remark:
It is worth noting that the complete proof is moreinvolved than the proofs which use expansion arguments.However, the result is also more precise and holds for codesof small lengths.IV. N
UMERICAL R ESULTS
In this section, we describe a technique to construct codeswhich can correct three errors. Codes capable of correctinga fixed number of errors show superior performance on theBSC at low values of probability of transition α . This isbecause the slope of the FER curve is related to the minimumcritical number [18]. A code which can correct i errors hasminimum critical number i + 1 and the slope of FER curveis i + 1 . We restate the arguments from [18] to make thisconnection clear. a) (b) (c)Fig. 1. Examples of trapping sets with critical number three (a) a (3 , trapping set (b) a (5 , trapping set and (c) an (8 , trapping set
21 3ec fdba (a) (b) (c) a 21 3cb d e f g h (d) (e)Fig. 2. All the possible subgraphs that can be induced by three variable nodes in a column-weight-three code
Let α be the transition probability of BSC and c k benumber of configurations of received bits for which k channelerrors lead to codeword (frame) error. The frame error rate(FER) is given by: F ER ( α ) = n X k = i c k α k (1 − α ) ( n − k ) where i is the minimal number of channel errors that canlead to a decoding error (size of instantons) and n is lengthof the code.On a semilog scale the FER is given by the expression log( F ER ( α )) = log (cid:0) n X k = i c k α k (1 − α ) n − k (cid:1) = log( c i ) + i log( α ) + log((1 − α ) n − i )+ log (cid:18) c i +1 c i α (1 − α ) − + . . . + c n c i α n − i (1 − α ) − i (cid:19) In the limit α → we note that lim α → h log((1 − α ) n − i ) i = 0 and lim α → h log (cid:16) c i +1 c i α (1 − α ) − . . . + c n c i α n − i (1 − α ) i − n (cid:17)i = 0 So, the behavior of the FER curve for small α is dominatedby log( F ER ( α )) ≈ log( c i ) + i log( α ) The log(
F ER ) vs log( α ) graph is close to a straight linewith slope equal to i, the minimal critical number. If twocodes C and C have minimum critical numbers i and i , such that i > i , then the code C will perform betterthan C for small enough α, independent of the number oftrapping sets. From the discussion in Section III and Section IV, it isclear that for a code to have a FER curve with slope at least , the corresponding Tanner graph should not contain thetrapping sets shown in Fig. 1 as subgraphs. We now describea method to construct such codes. The method can be seenas a modification of the PEG construction technique used byHu et al. [12]. The algorithm is as follows: Data : The set of n variable nodes ( V ) and m checknodes ( C ). The column weight of the code ( γ ) Result : Code with column weight γ for j = 1 to n dofor k = 1 to γ doif k = 1 then Connect the k th edge of variable node j tothe check node with the smallest positivedegree. else Expand the tree rooted at node j to a depthof .Assimilate all check nodes which do notappear in the tree into C j,T , the set ofcandidates for connecting variable node j to. while k th edge is not found do Find the check node c i in C j,T with thelowest degree. If connecting c i tovariable node j does not create a (5 , trapping set, set this as the k th edge. If itdoes, remove c i from C j,T . endendendend Note that checking for a graph isomorphic to (8 , trap- −2 −8 −7 −6 −5 −4 Probability of transition ( α ) F r a m e e rr o r r a t e ( F E R ) PEG OriginalPEG New
Fig. 3. Performance comparison of original PEG and the new PEG code ping set at every step of code construction is computation-ally complex. Since, the PEG construction empirically givesgood codes, it is unlikely that it introduces a weight-eightcodeword. However, once the graph is grown fully, it canbe checked for the presence of weight-eight codewords andthese can be removed by swapping few edges.Using the above algorithm, a column-weight-three codewith variable nodes and check nodes was con-structed. The code has slight irregularity in check degree.There is one check node degree five and one check nodewith degree seven, but the majority of them have degree six.The code has rate 0.5. In the algorithm, we restrict maximumcheck degree to seven. The performance of the code on BSCis compared with the PEG code of same length. The PEGcode is empirically the best known code at that length onAWGN channel [19]. However, it has fourteen (5 , trappingsets. Fig. 3 shows the performance comparison of the twocodes. As can be seen, the new code performs better thanthe original PEG code at small values of α .V. C ONCLUSION
In this paper, we have given conditions for a column-weight-three code to correct three errors. Since, the checkdegree does not play any part in the proof, it follows thatthe result is independent of code rate. A direction for futurework is extending the analysis to more number of errorsand higher column weight codes. Preliminary investigationshows a lot of promise. The complexity of the proof, evenin the case of three errors, suggests that solving the problemfor an arbitrary number of errors will be a challenge. Onthe code construction front, we have shown that avoidingtrapping sets with minimum critical number is the criterion tosuppress error floor. However, the conditions for correctingmore errors could be more complicated thereby increasingthe complexity of code construction. Deriving bounds onlengths and minimum distance of codes which avoid certainstructures also need to be investigated.A
CKNOWLEDGMENT
This work is funded by NSF under Grant CCF-0634969and INSIC-EHDR program. R
EFERENCES[1] T. J. Richardson and R. Urbanke, “The capacity of low-density parity-check codes under message-passing decoding,”
IEEE Trans. Inform.Theory , vol. 47, no. 2, pp. 599–618, Feb. 2001.[2] T. J. Richardson, M. Shokrollahi, and R. Urbanke, “Design of capacity-approaching irregular low-density parity-check codes,”
IEEE Trans.Inform. Theory , vol. 47, no. 2, pp. 638–656, Feb. 2001.[3] D. J. C. MacKay and M. J. Postol, “Weaknesses of Margulis andRamanujan–Margulis low-density parity-check codes,” in
Proceedingsof MFCSIT2002, Galway , 2003, pp. 1426–1435.[5] C. Di, D. Proietti, T. Richardson, E. Telatar, and R. Urbanke, “Fi-nite length analysis of low-density parity-check codes,”
IEEE Trans.Inform. Theory , vol. 48, pp. 1570–1579, June 2002.[6] M. G. Stepanov, V. Chernyak, M. Chertkov, and B. Vasic, “Diagnosisof weaknesses in modern error correction codes: A physics approach,”
Phys. Rev. Lett.
IEEE Trans.Inform. Theory , vol. 53, no. 7, pp. 2376–2393, July 2007.[9] R. Smarandache, A. E. Pusane, P. O. Vontobel, and J. C. D.J., “Pseudo-codewords in LDPC convolutional codes,” in
International Symposiumon Information Theory , July 2006, pp. 1364–1368.[10] S. K. Chilappagari, S. Sankaranarayanan, and B. Vasic, “Error floorsof LDPC codes on the binary symmetric channel,” in
InternationalConference on Communications , vol. 3, June 11-15 2006, pp. 1089–1094.[11] D. Burshtein and G. Miller, “Expander graph arguments for message-passing algorithms,”
IEEE Trans. Inform. Theory , vol. 47, no. 2, pp.782–790, Feb. 2001.[12] X.-Y. Hu, E. Eleftheriou, and D. M. Arnold, “Regular and irregularprogressive edge-growth Tanner graphs,”
IEEE Trans. Inform. Theory ,vol. 51, no. 1, pp. 386–398, Jan. 2005.[13] R. G. Gallager,
Low Density Parity Check Codes . Cambridge, MA:M.I.T. Press, 1963.[14] A. Shokrollahi, “An introduction to low-density parity-check codes,”in
Theoretical aspects of computer science: advanced lectures . NewYork, NY, USA: Springer-Verlag New York, Inc., 2002, pp. 175–197.[15] S. K. Chilappagari and B. Vasic, “LDPC codes which can correct fixednumber of errors under iterative decoding,” in preparation.[16] ——, “Error correction capability of column-weight-three LDPCcodes,” submitted to IEEE Trans. Inform. Theory. [Online]. Available:http://arxiv.org/abs/0710.3427[17] S. Sankaranarayanan, S. K. Chilappagari, R. Radhakrishnan, andB. Vasic, “Failures of the Gallager B decoder: Analysis andapplications,” in
UCSD Center for Information Theory and itsApplications Inaugural Workshop , Feb 6-9 2006. [Online]. Available:htpp//ita.5i.net/papers/160.pdf[18] M. Ivkovic, S. K. Chilappagari, and B. Vasic, “Eliminating trappingsets in low-density parity check codes using Tanner graph lifting,” in