Minimum-Polytope-Based Linear Programming Decoder for LDPC Codes via ADMM Approach
aa r X i v : . [ c s . I T ] J a n Minimum-Polytope-Based Linear ProgrammingDecoder for LDPC Codes via ADMM Approach
Jing Bai, Yongchao Wang,
Member, IEEE , Francis C. M. Lau,
Senior Member, IEEE
Abstract —In this letter, we develop an efficient linear pro-gramming (LP) decoding algorithm for low-density parity-check (LDPC) codes. We first relax the maximum likelihood(ML) decoding problem to a LP problem by using check-nodedecomposition. Then, to solve the resulting LP problem, wepropose an efficient iterative algorithm based on the alternatingdirection method of multipliers (ADMM) technique. In addition,the feasibility analysis of the proposed algorithm is presented.Furthermore, through exploiting the sparsity and orthogonalitystructures of the LP problem, the computational complexityof the proposed decoding algorithm increases linearly with thelength of the LDPC code. Simulation results demonstrate thatour proposed algorithm achieves better performance than othercompeting algorithms in terms of decoding time.
Index Terms —Linear programming decoding, alternating di-rection method of multipliers (ADMM), minimum polytope.
I. I
NTRODUCTION
Linear programming (LP) decoding has been consideredas a promising decoding approach for low-density parity-check (LDPC) codes [1]. Compared with the classical beliefpropagation (BP) decoder, LP decoding can be supported bytheoretical guarantees for decoding performance. However,the original LP decoder suffers from high complexity andthus cannot be efficiently implemented in practice. Therefore,reducing the complexity of LP decoding becomes an inter-esting but challenging topic. There exist two popular researchdirections: formulating new small-scale LP decoding problemsand designing more efficient problem-solving algorithms.Considering the first direction, several studies have fo-cused on transforming the maximum-likelihood (ML) decod-ing problem into LP ones with fewer constraints. For instance,the authors in [2] proposed a small-sized LP decoding modelby adaptively adding necessary constraints. In [3], another newLP formulation, in which the variables and constraints growonly linearly with the check node degree, was introduced.As for the second research direction, increasing researchefforts have been devoted to investigating how the inherentstructures of LP problems be exploited to develop more effi-cient decoding algorithms. In [4], a distributed algorithm basedon the alternating direction method of multipliers (ADMM)technique [9] was first proposed to solve the original LPdecoding problem [1]. However, this ADMM-based algorithminvolves complex check-polytope projection and thus limitsits efficiency. Subsequently, the authors in [5]–[7] indepen-dently optimized the projection algorithm [4] to further reducethe complexity of the LP decoder [4]. In [8], a projection-reduction method was proposed by decreasing the numberof projection operations to simplify the ADMM decodingalgorithm. To the best of our knowledge, developing efficient algo-rithms for other types of LP decoders has not been well in-vestigated in the existing literatures. In this letter, we considerthe minimum-polytope-based LP (MPB-LP) decoder whichis formulated by check-node decomposition, and develop anefficient decoding algorithm based on the ADMM technique.The main contributions of this work are threefold. • We show the sparsity and orthogonality properties ofthe MPB-LP problem. By exploiting these structures, wepropose an ADMM-based decoding algorithm in whicheach updating step can be solved efficiently. • We analyze the feasibility of the proposed ADMM-basedalgorithm and further show that the complexity of ourproposed algorithm increases linearly with the length ofLDPC codes. • Simulation results present that the proposed ADMM-based LP decoder consumes less decoding time thancompeting LP decoders, and displays superior error-rateperformance than BP decoder at high signal-to-noise(SNR) regions with a shorter decoding time.II. M
INIMUM - POLYTOPE - BASED LP DECODING MODEL
A. General MPB-LP decoding model
Consider a binary LDPC code C defined by an m × n parity-check matrix H = [ H ji ] m × n . Let I = { , . . . , n } and J = { , . . . , m } denote the set of variable nodes andcheck nodes of C respectively. Let the degree of a check nodebe the number of its neighboring variable nodes. Denote theminimum polytope P [ ? ] describing the convex hull of theparity-check constraint of a degree-3 check node by P = { ( x , x , x ) : x + x + x ≤ ,x − x − x ≤ , − x + x − x ≤ , − x − x + x ≤ , ≤ x , x , x ≤ } . (1)Now suppose that a codeword x ∈ C is transmitted overa noisy memoryless binary-input output-symmetric channel,and a vector r is received. By decomposing each high-degreecheck node into a number of degree- check nodes with acertain number auxiliary variables, the maximum likelihood(ML) decoding problem can be relaxed to the following MPB-LP form [3] min x γγγ T x , s.t. x ∈ m \ j =1 Φ j , j ∈ J , (2)where γγγ is the vector of log-likelihood ratios (LLR) definedby γ i = log (cid:16) Pr( r i | x i =0)Pr( r i | x i =1) (cid:17) , and Φ j denotes the intersection ofall the minimum polytopes for the j th parity-check constraint. B. Standard MPB-LP decoding model
To make the above MPB-LP problem (2) easier to analyze,we convert it into a standard LP form. We denote d j as thedegree of the j -th check node in C . Let Γ c and Γ a be thenumber of minimum polytopes and the number of auxiliaryvariables, respectively. Then Γ c = P mj =1 ( d j − and Γ a = P mj =1 ( d j − [3]. We further define q = (cid:20) γγγ Γ a (cid:21) and v = (cid:20) xu Γ a (cid:21) where Γ a is the length- Γ a all-zero vector and u Γ a ∈ [0 , Γ a represents the auxiliary variable vector.Since each minimum polytope is characterized by threevariable nodes, we assume that the τ -th minimum polytope( τ = 1 , , . . . , Γ c ) is related to the v τ -th, v τ -th, and v τ -thvariables in v . For the τ -th minimum polytope, we first con-struct a corresponding matrix Q τ ∈ { , } × ( n +Γ a ) , in whicheach row contains only one nonzero element and the indicesof the three nonzero elements are given by (1 , v τ ) , (2 , v τ ) ,and (3 , v τ ) . Based on the first four inequalities in (1), we alsoconstruct a vector t = [2 0 0 0] T and a matrix F = − − − − − − , (3)where ( · ) T represents the transpose operation. Then the τ -thminimum polytope ( τ = 1 , , . . . , Γ c ) can be expressed as FQ τ v (cid:22) t . (4)We further define a matrix A = [ FQ ; ... ; FQ τ ; ... ; FQ Γ c ] ,which is cascaded by each FQ τ , and a vector b = Γ c ⊗ t where ⊗ denotes the Kronecker product and Γ c is the length- Γ c all-one vector. Letting M = 4Γ c and N = n +Γ a , then A ∈ R M × N and b ∈ R M . Subsequently, we reformulate the MPB-LP problem (2) equivalently to the following linear program min v q T v , s.t. Av (cid:22) b , v ∈ [0 , N . (5)The problem (5) certainly can be solved by general-purposeLP solvers, such as the SIMPLEX method or the interior-point method. However, the high computational complexitylimits their practical applications. Therefore, in the following,we will propose an efficient algorithm based on the ADMMtechnique to solve the problem (5).III. P ROPOSED
ADMM-B
ASED D ECODING A LGORITHM
This section discusses the detailed ADMM-based algorithmfor solving the MPB-LP decoding problem (5), and presentsits feasibility and complexity analysis.
A. ADMM algorithm framework
To make the problem (5) fitting into the ADMM template,we have to add an auxiliary variable vector w ∈ R M + so as tochange the inequality into an equality constraint. With w , thelinear program (5) is equivalent to min v q T v , s.t. Av + w = b , v ∈ [0 , N , w ∈ R M + . (6) The augmented Lagrangian function (using the scaled dualvariable) for problem (6) is expressed by L µ ( v , w , λλλ ) = q T v + µ k Av + w − b + λλλ k − µ k λλλ k , (7)where λλλ ∈ R M denotes the scaled dual variable vector, µ > is the penalty parameter, and k·k represents the 2-normoperator. Then a typical ADMM-based algorithm for solving(6) can be described by the following iterations v ( k +1) = argmin v ∈ [0 , N L µ ( v , w ( k ) , λλλ ( k ) ) , (8a) w ( k +1) = argmin w ∈ R M + L µ ( v ( k +1) , w , λλλ ( k ) ) , (8b) λλλ ( k +1) = λλλ ( k ) + ( Av ( k +1) + w ( k +1) − b ) , (8c)where k denotes the iteration number.Observing (8), the majority of its computational complexitydepends on solving (8a) and (8b). However, both of them canbe implemented efficiently since matrix A owns the followingproperties: Fact
The matrix A possesses the following properties. • The elements of A (i.e., A ji ) are either , −
1, or . • A is sparse. • The column vectors of A are orthogonal to one another. Proof
See Appendix A.In the following we will show that each sub-problem in theabove algorithm (8) can be solved efficiently by exploitingthese favourable features of matrix A . B. Solving the sub-problem (8a)Since matrix A is column orthogonal, A T A is a diagonalmatrix. This implies that variables in the problem (8a) are sep-arable. Therefore, solving the problem (8a) can be equivalentto solving the following N subproblems independently min v
12 ( µe i ) v i + (cid:16) q i + µ ˆ a Ti (cid:0) w ( k ) − b + λλλ ( k ) (cid:1)(cid:17) v i , (9a)s.t. v i ∈ [0 , , (9b)where e = diag( A T A ) = [ e , · · · , e n +Γ a ] T and diag( · ) denotes the operator of extracting the diagonal vector of amatrix, and ˆ a i is the i -th column of the matrix A . Then, theprocedures for solving the sub-problem (9) can be summarizedas follows: setting the gradient of the objective (9a) to zero,then projecting the resulting solution to the interval [0 , .Finally, we can obtain the the following solution of theproblem (9) as v ( k +1) i = Π [0 , (cid:18) e i (cid:16) ˆ a Ti (cid:0) b − w ( k ) − λλλ ( k ) (cid:1) − q i µ (cid:17)(cid:19) , (10)where Π [0 , ( · ) denotes the Euclidean projection operator ontothe interval [0 , . C. Solving the sub-problem (8b)Observing (8b), we can find that the variables in w arealso separable in either objective or constraints. Hence, thesub-problem (8b) can be solved by fixing v and λλλ , and thenminimizing L µ ( v , w , λλλ ) under the constraint w ∈ R M + (i.e., w ∈ [0 , + ∞ ] M ). Using a technique similar to that describedin the previous section, w ( k +1) can be updated by w ( k +1) = Π [0 , + ∞ ] M (cid:16) b − Av ( k +1) − λλλ ( k ) (cid:17) , (11)where Π [0 , + ∞ ] M ( · ) denotes the Euclidean projection onto thepositive quadrants [0 , + ∞ ] M . Clearly, the w -update can alsobe written in a component-wise manner, i.e., w ( k +1) j = Π [0 , + ∞ ] (cid:16) b j − a Tj v ( k +1) − λ ( k ) j (cid:17) , (12)where a Tj is the j -th row of matrix A and j ∈ { , , . . . , M } .Finally, we summarize the proposed minimum-polytope-based ADMM-solved (MPB-ADMM) LP decoding method in Algorithm 1 . Here we make three remarks for the proposedalgorithm: (1)1) In
Algorithm 1 , the multiplications with respect to matrix A , such as ˆ a Ti b et al ., are just considered as additionoperations based on the first property of matrix A .2) All the variables in v can be updated in parallel.3) Compared with previous works [4]–[8] that require com-plex check-polytope projections, each variable update(12) in w only involves a simple Euclidean projectiononto the positive quadrant. Hence the w -update signifi-cantly reduces the decoding complexity of the proposedADMM algorithm and can be implemented much moreeasily in practice. D. Feasibility analysis
The convergence analysis of the our proposed algorithmcan be found in [9]. Since the LP problem (5) is convex, ourproposed algorithm is guaranteed to converge to the globalsolution v ∗ . On the other hand, as shown in [3], the MPB-LPdecoder (5) is equivalent to the original LP decoder. Therefore,our proposed decoder shares the same properties of the originalLP decoder such as all-zeros assumption and ML-certificateproperty [1]. As a result, if the output of
Algorithm 1 is anintegral solution denoted by v ∗ , the ML-certificate property ensures that x ∗ corresponds to the ML solution. E. Complexity analysis
Since the matrix A contains only , − , or elements,its related matrix multiplications can all be performed usingaddition operations. Thus, by taking only the multiplicationsrelated to /e i into account, the computational complexity ofupdating v ( k +1) in Algorithm 1 at each iteration is O ( N ) .By exploiting the desirable properties of matrix A , the com-putations of w ( k +1) and λλλ ( k +1) can be completed using onlyaddition operations. Hence their computational complexitiesare both O (1) . Combining all the above analysis, the totalcomputational complexity of our proposed algorithm in eachiteration is O ( N ) , i.e., O ( n +Γ a ) . Moreover, Γ a is comparableto the code length since the parity check matrix H is sparse inthe case of LDPC codes. Thus, the complexity of the proposedADMM-based algorithm in each iteration is linear to the lengthof LDPC codes.In Table I , we compare the complexities of our proposedalgorithm and the check-polytope-based ADMM-solved (CPB-ADMM) LP decoding algorithms in [5] and [7]. We can
Algorithm 1
Proposed MPB-ADMM LP decoding algorithm Calculate the log-likelihood ratio γγγ based on receivedvector r and construct the vector q based on γγγ . Construct the M × N matrix A based on the parity-checkmatrix H and construct the vector b . Set w (0) and λλλ (0) to the length- M all-zero vector, andinitialize the iteration number k = 0 . repeat for i ∈ { , , . . . , N } do Update v ( k +1) i = Π [0 , (cid:18) e i (cid:16) ˆ a Ti (cid:0) b − w ( k ) − λλλ ( k ) (cid:1) − q i µ (cid:17)(cid:19) . end for for j ∈ { , , . . . , M } do Update w ( k +1) j = Π [0 , + ∞ ] (cid:16) b j − a Tj v ( k +1) − λ ( k ) j (cid:17) . Update λ ( k +1) j = λ ( k ) j + ( a Tj v ( k +1) + w ( k +1) j − b j ) . end for until k Av ( k +1) + w ( k +1) − b k ≤ ξ and k w ( k +1) − w ( k ) k ≤ ξ .observe that our proposed MPB-ADMM LP decoder achievesthe lowest computational complexity.IV. S IMULATION R ESULTS
In this section, we present the simulation results of ourproposed MPB-ADMM LP decoder, the CPB-ADMM LPdecoders in [5] and [7], and the classic sum-product BPdecoder. The simulations are performed on a computer withi5-3470 3.2GHz CPU and 16 GB RAM under the MicrosoftVisual C++ 6.0 environment.We simulate two widely-used LDPC codes, namely the (2640 , rate-1/2 (3,6)-regular “Margulis” LDPC code C [10] and the (576 , rate-1/2 irregular LDPC code C inthe 802.16e standard [11]. All the code bits x are sent over anadditive white Gaussian noise (AWGN) channel using binaryphase-shift-keying (BPSK). In our decoding algorithm, thepenalty parameter µ is set to . and . for C and C ,respectively . The maximum number of decoding iterationsis set to and the tolerance ξ is set to − .Fig. 1 and Fig. 2 plot the frame-error-rate (FER) perfor-mances of C and C , respectively, with different decodingalgorithms. In the case of the MPB/CPB-ADMM LP decodingalgorithms, we collect 50 error frames at E b /N = 2 . dB inFig. 1 and E b /N = 4 . dB in Fig. 2 and 200 for all otherSNRs. The results indicate that our proposed MPB-ADMMLP decoder has the same performance as the CPB-ADMMLP decoders in [5] and [7]. Moreover, our proposed decoderoutperforms the sum-product BP decoder at high SNR regionswithout showing any error floor.In Fig. 3 and Fig. 4, we make a comparison in the averagedecoding time of our proposed algorithm and other competingdecoding algorithms at different SNRs for C and C , respec-tively. The decoding times in these two figures are averaged The parameter µ is chosen reasonably by plotting the error-rate perfor-mance and decoding time as a function of µ respectively. Due to spacelimitation, we do not give these detailed figures. The maximum number of iterations 500 and tolerance − is consideredgood enough to achieve the desired tradeoff between error rate and complexity. TABLE IC
OMPLEXITY OF D IFFERENT
LP D
ECODING A LGORITHMS P ER I TERATION . d DENOTES THE LARGEST CHECK - NODE DEGREE . I max IS THE MAXIMUMNUMBER OF ITERATIONS SET IN THE PROJECTION ALGORITHM OF [7].
Decoding Algorithm
Original variables Auxiliary variables Dual variables Overall ComplexityProposed MPB-ADMM LP O ( n + m ( d − O (1) O (1) O ( n + m ( d − CPB-ADMM LP [5] O ( n ) O ( md log d ) O (1) O ( n + md log d ) CPB-ADMM LP [7] O ( n ) O ( mI max ) O (1) O ( n + mI max ) −7 −6 −5 −4 −3 −2 −1 E b /N (dB) F r a m e e rr o r r a t e ( F E R ) MPB−ADMM LP (proposed)CPB−ADMM LP [5]CPB−ADMM LP [7]Sum−product BP
Fig. 1. FER performance of C using different decoders. C denotes the (2640 , regular “Margulis” LDPC code. −7 −6 −5 −4 −3 −2 −1 E b /N (dB) F r a m e e rr o r r a t e ( F E R ) MPB−ADMM LP (proposed)CPB−ADMM LP [5]CPB−ADMM LP [7]Sum−product BP
Fig. 2. FER performance of C using different decoders. C denotes the (576 , irregular LDPC code in the 802.16e standard. over one million frames for each decoder. And note that overrelaxation (OR) and early termination (ET) techniques are bothapplied in ADMM-based LP decoders. From Fig.3 and Fig.4,we observe that our proposed MPB-ADMM LP decoder is themost efficient among all the competing decoding algorithms,which is consistent with the complexity analysis of Table I .To be specific, in Fig. 3 we see that our decoding algorithmroughly saves 31%, 69% and 83% decoding time respectivelyfor C when E b /N = 2 . dB in comparison with the sum- b /N (dB) A v e r a g e d ec od i ng ti m e ( . s ec ) MPB−ADMM LP (proposed)CPB−ADMM LP [5]CPB−ADMM LP [7]Sum−product BP
Fig. 3. Decoding time of C using different decoders. C denotes the (2640 , regular “Margulis” LDPC code. E b /N (dB) A v e r a g e d ec od i ng ti m e ( . s ec ) MPB−ADMM LP (proposed)CPB−ADMM LP [5]CPB−ADMM LP [7]Sum−product BPx 10 −2 Fig. 4. Decoding time of C using different decoders. C denotes the (576 , irregular LDPC code in the 802.16e standard. product BP decoding and the CPB-ADMM LP algorithms[5] and [7]. Besides, Fig. 4 shows that our proposed decoderroughly reduces 32%, 46% and 63% time respectively for C when E b /N = 5 . dB compared with the sum-product BPdecoder and the other two CPB-ADMM LP decoders.V. C ONCLUSION
In this letter, we propose an efficient LP decoding algorithmbased on ADMM technique for LDPC codes. By exploiting the sparsity and orthogonality of the formulated MPB-LP model,we solve this resulting LP problem efficiently by the ADMM-based algorithm. Detailed analysis shows that the complexityof the proposed algorithm at each iteration grows linearlywith the length of LDPC codes. Simulation results confirmthe efficiency of the proposed LP decoder.A
PPENDIX AP ROOF OF F ACT Proof
We consider the first property of the matrix A . Sinceeach row in the matrix Q τ includes one nonzero element “1”,any element in the matrix FQ τ is either 0, −
1, or 1. The sameapplies to the matrix A . Moreover, we can see that there areonly 12 nonzero elements in FQ τ . It means that there areonly c nonzero elements in the matrix A . It is far smallerthan the size, c × N , of the matrix A . Therefore, we obtainthe sparsity of A . Furthermore, we note that any two columnvectors in F are orthogonal to each other. Thus, each twocolumn vectors in FQ τ are also orthogonal. This implies that A is a matrix with orthogonal columns because A is formedby cascading the matrices { FQ τ : τ = 1 , , . . . , Γ c } .R EFERENCES[1] J. Feldman, M. T. Wainwright, and D. R. Karger, “Using linear progam-ming to decoding binary linear codes,”
IEEE Trans. Inf. Theory , vol. 51,no. 1, pp. 954-972, Jan. 2005.[2] M. H. Taagavi and P. H. Siegel, “Adaptive methods for linear program-ming decoding,”
IEEE Trans. Inf. Theory , vol. 54, no. 12, pp. 5396-5410,Dec. 2008.[3] K. Yang, X. Wang, and J. Feldman, “A new linear programmingapproach to decoding linear block codes,”
IEEE Trans. Inf. Theory , vol.54, no. 3, pp. 1061-1072, Mar. 2008.[4] S. Barman, X. Liu, S. C. Draper, and B. Recht, “Decomposition methodfor large scale LP decoding,”
IEEE Trans. Inf. Theory , vol. 59, no. 12,pp. 7870-7886, Dec. 2013.[5] X. Zhang and P. H. Siegel, “Efficient iterative LP decoding of LDPCcodes with alternating direction method of multipliers,” in Proc.
IEEEInt. Symp. Inf. Theory , Istanbul, Turkey, Jul. 2013, pp. 1501-1505.[6] G. Zhang and R. Heusdens, and W. B. Kleijn, “Large scale LP decodingwith low complexity,”
IEEE Commun. Lett ., vol. 17, no. 11, pp. 2152-2155, Jun. 2015.[7] H. Wei and A. H. Banihashemi, “An iterative check polytope projectionalgorithm for ADMM-based LP decoding of LDPC codes,”
IEEECommun. Lett ., vol. 22, no. 1, pp. 29-32, Jan. 2018.[8] H. Wei, X. Jiao, and J. Mu, “Reduced-complexity linear programmingdecoding based on ADMM for LDPC codes,”
IEEE Commun. Lett ., vol.19, no. 6, pp. 909-912, Jun. 2015.[9] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, “Distributedoptimization and statistical learning via the alternating direction methodof multipliers,”
Found. Trends Mach. Learn.