Supplementary Appendix for: Constrained Perturbation Regularization Approach for Signal Estimation Using Random Matrix Theory
Mohamed Suliman, Tarig Ballal, Abla Kammoun, Tareq Y. Al-Naffouri
aa r X i v : . [ c s . I T ] J un Supplementary Appendix for: ConstrainedPerturbation Regularization Approach for SignalEstimation Using Random Matrix Theory
Mohamed Suliman, Tarig Ballal, Abla Kammoun, and Tareq Y. Al-Naffouri
Abstract —In this supplementary appendix we provide proofsand additional extensive simulations that complement the anal-ysis of the main paper (constrained perturbation regularizationapproach for signal estimation using random matrix theory).
I. P
ROPERTIES OF THE
COPRA
FUNCTION S (˜ γ O ) Firstly, we discuss the general properties of the COPRAfunction S ( ˜ γ o ) = Tr (cid:16) Σ (cid:0) Σ + N ˜ γ o I (cid:1) − bb H (cid:17) h δ o ˜ δ o − ˜ γ o δ o − ˜ γ o δ o ˜ δ o i + Tr (cid:16)(cid:0) Σ + N ˜ γ o I (cid:1) − bb H (cid:17) × h ( N δ o ˜ δ o (cid:16) ˜ γ o − ˜ γ o δ o ˜ δ o − δ o ˜ δ o (cid:17) + M ˜ δ o ˜ γ o (cid:16) ˜ γ o − ˜ γ o δ o + δ o ˜ δ o (cid:17) i = 0 , (1)where b , U H y .The COPRA characteristic equation is a function of theproblem parameters, the received signal, and the unknownregularizer ˜ γ o . Our main interest is to find a positive root for(1). Before that, and to simplify the properties analysis of theCOPRA function, we will assume that M/N ≈ . Thus, (1)can be written as S (˜ γ o ) = Tr (cid:16) Σ (cid:0) Σ + N ˜ γ o I (cid:1) − bb H (cid:17) × " ˜ γ o s ˜ γ o + 4˜ γ o − ! − + Tr (cid:16)(cid:0) Σ + N ˜ γ o I (cid:1) − bb H (cid:17) × " N ˜ γ o s ˜ γ o + 4˜ γ o − ! ˜ γ o + 2 s ˜ γ o + 4˜ γ o − ! = 0 . (2)Before proceeding further, we will define the following twofunctions S (˜ γ o ) , Tr (cid:16) Σ (cid:0) Σ + N ˜ γ o I (cid:1) − bb H (cid:17) × " ˜ γ o s ˜ γ o + 4˜ γ o − ! − , (3)and S (˜ γ o ) , Tr (cid:16)(cid:0) Σ + N ˜ γ o I (cid:1) − bb H (cid:17) × " N ˜ γ o s ˜ γ o + 4˜ γ o − ! ˜ γ o + 2 s ˜ γ o + 4˜ γ o − ! . (4) This assumption is to simplify the properties analysis presentation. How-ever, all the properties and the theorems that will be presented and proved inthis section can be easily extended for a general ratio of
M/N upon followingthe same steps.
Now, let us start our properties discussion by examine somemain properties of the COPRA function that are straightfor-ward to proof.
Property I.1.
The function S (˜ γ o ) has M discontinuities at ˜ γ o = − σ i /N, ∀ i = 1 , · · · , M . However, these discontinuitiesare of no interest as far as COPRA is concerned. Property I.2. S (˜ γ o ) is continuous over the interval (0 , + ∞ ) . Property I.3. lim ˜ γ o → + ∞ S (˜ γ o ) = 0 . Property I.4. lim ˜ γ o → + S (˜ γ o ) = − Tr (cid:0) Σ − bb H (cid:1) .Proof: Let Σ = diag (cid:0) σ , σ , · · · , σ M (cid:1) and bb H = diag (cid:0) b , b , · · · , b M (cid:1) . Then, (2) can be written as S (˜ γ o ) = M X i =1 σ i b i ( σ i + N ˜ γ o ) " − − ˜ γ o + ˜ γ o s ˜ γ o + 4˜ γ o + M X i =1 b i ( σ i + N ˜ γ o ) " − N ˜ γ o + 2 N ˜ γ o s ˜ γ o + 4˜ γ o − N ˜ γ o + N ˜ γ o s ˜ γ o + 4˜ γ o , (5)which after some algebraic manipulations yields S (˜ γ o ) = M X i =1 σ i b i ( σ i + N ˜ γ o ) " − − ˜ γ o + p ˜ γ o p ˜ γ o + 4 + M X i =1 b i ( σ i + N ˜ γ o ) " − N ˜ γ o + 2 N p ˜ γ o p ˜ γ o + 4 − N ˜ γ o + N ˜ γ o p ˜ γ o p ˜ γ o + 4 . (6)Now, taking the limit as ˜ γ o → + yields lim ˜ γ o → + S (˜ γ o ) = − M X i =1 b i σ i = − Tr (cid:0) Σ − bb H (cid:1) . (7) Property I.5. lim ˜ γ o → + ∞ S (˜ γ o ) approaches zero from thenegative region. Since bb H is multiplied by a diagonal matrix Σ inside the trace, wecan only consider its diagonal entries. Proof:
Starting from (6), we can write this equation as S ( ˜ γ o ) = 1 N ˜ γ o M X i =1 σ i b i (cid:16) σ i N ˜ γ o + 1 (cid:17) " − − ˜ γ o + p ˜ γ o p ˜ γ o + 4 + 1 N ˜ γ o M X i =1 b i (cid:16) σ i N ˜ γ o + 1 (cid:17) " − N ˜ γ o + 2 N p ˜ γ o p ˜ γ o + 4 − N ˜ γ o + N ˜ γ o p ˜ γ o p ˜ γ o + 4 . (8)Now, evaluating the limit of (8) as ˜ γ o approaches + ∞ yields lim ˜ γ o → + ∞ S ( ˜ γ o ) = lim ˜ γ o → + ∞ N ˜ γ o h − γ o M X i =1 σ i b i − N M X i =1 b i i . (9)It can be clearly seen that the limit in (9) is equal to zero.However, the COPRA function approaches the zero from thenegative direction. Property I.6.
The functions S (˜ γ o ) in (3) and S (˜ γ o ) in (4)are completely monotonic in the interval (0 , + ∞ ) .Proof: According to [1], [2], a function F (˜ γ o ) is com-pletely monotonic if it satisfies ( − n F ( n ) (˜ γ o ) ≥ , < ˜ γ o < ∞ , ∀ n ∈ N , (10)where ( . ) ( n ) is the n ’th derivative of the function.By continuously differentiating S (˜ γ o ) and S (˜ γ o ) we caneasily show that the two functions are satisfying the monotoniccondition in (10). Theorem I.1.
The COPRA function S (˜ γ o ) in (2) has at mosttwo roots in the interval (0 , + ∞ ) . Proof:
The proof of Theorem I.1 will be conducted intwo steps. Firstly, it has been shown in [3], [4] that anycompletely monotonic function can be approximated as a sumof exponential functions. That is, if F (˜ γ o ) is a completelymonotonic function, it can be approximated as F (˜ γ o ) ≈ l X i =1 a i e − k i ˜ γ o , (11)where l is the number of the terms in the sum. It has beenshown that a best uniform approximation for F (˜ γ o ) alwaysexists, and the error in this approximation gets smaller as weincrease the number of the terms l . However, our main concernhere is the relation defined by (11) more than finding the bestnumber of terms or the unknown parameters a i and k i . Toconclude, both functions S (˜ γ o ) in (3) and S (˜ γ o ) in (4) canbe approximated by a sum of exponential functions. We canassume that l is large enough such that the approximation errorin (11) is negligible.Secondly, it is shown in [5] that the sum of exponentialfunctions has at most two intersections with the abscissa.Consequently, and since the relation in (2) can be expressedas a sum of exponential functions, the function S (˜ γ o ) has atmost two roots in the interval (0 , + ∞ ) . Theorem I.2.
There always exists a sufficiently small positivevalue ǫ , such that ǫ → + and ǫ ≪ σ i N , ∀ i ∈ [0 , M ] , wherethe COPRA function S ( ˜ γ o ) in (2) is zero (i.e., ǫ is a positiveroot for (2)). However, we are not interested in this root.Proof: To start with, let ǫ = ˜ γ o , such that ǫ → + , ǫ ≪ σ i N , ∀ i ∈ [0 , M ] . As a result, equation (6) can be written as S ( ǫ ) = M X i =1 σ i b i ( σ i + N ǫ ) " − − ǫ + √ ǫ √ ǫ + M X i =1 b i ( σ i + N ǫ ) " − N ǫ + 2 N √ ǫ √ ǫ + 4 − N ǫ + N ǫ √ ǫ √ ǫ + 4 . (12)Due to the properties of ǫ , (12) can be approximated as S ( ǫ ) ≈ − M X i =1 σ − i b i + M X i =1 σ − i b i h − N ǫ + 4 N √ ǫ + 2 N ǫ √ ǫ i . (13)Now, to simplify (13), let us define C = P Mi =1 σ − i b i and C = P Mi =1 σ − i b i . Substituting these two new variables in(13) then manipulating result in S ( ǫ ) ≈ N C ǫ √ ǫ + 4 N C √ ǫ − N C ǫ − C . (14)Solving S ( ǫ ) = 0 from (14), we obtain one real root and twoimaginary roots. The real root is given by ǫ = (cid:16) Q + p Q + Z (cid:17) / × / N C − × / N C ( − C + N C ) (cid:16) Q + p Q + Z (cid:17) / , (15)where Q = 108 N C C , (16)and Z = 19 . N C ( − C + N C ) . (17)Now, we would like to know if this real root is positive or not.For (15) to be positive, the following condition must hold (cid:16) Q + p Q + Z (cid:17) / > . N C ( − C + N C ) . (18)By using (17), we can write (18) as (cid:16) Q + p Q + Z (cid:17) / > Z, (19)which can be easily proved. This concludes that ǫ is a positivereal root for the COPRA function in (2).Secondly, we would like to know if ǫ can be considered asa value for our regularization parameter ˜ γ o . A direct way toprove that can be noted from the fact that having ǫ ≪ σ i N , ∀ i ∈ [0 , M ] will not provide any source of regularization tothe problem. Hence, the RLS solution converges to the LS.As a remark, we can assume that the approximation in (13)is uniform such that it does not affect the position of the roots. Thus, we can claim that this root is not coming from thenegative region of the x axis. However, we can easily provethat the function in (12) does not have a negative real rootthat is close to zero (at least in the region from 0 to -2). Thus,this root is not coming from the negative region as a result ofapproximating the function (i.e., perturbed root). Theorem I.3.
The COPRA function S (˜ γ o ) has a uniquepositive root in the interval ( ǫ, + ∞ ) .Proof: According to Theorem I.1, the function S (˜ γ o ) can have noroot, one, or two roots. However, we have already proved inTheorem I.2 that there exists a significantly small positive rootfor COPRA function at ˜ γ o,1 = ǫ but we are not interested inthis root. In other words, we would like to see if there existsa second root for S (˜ γ o ) in the interval ( ǫ, + ∞ ) .Property I.4 shows that the COPRA function starts from anegative value, whereas Property I.5 states that the COPRAfunction approaches zero at + ∞ from a negative value. Thismeans that the COPRA function has a negative value before ǫ , then it switches to the positive region after that. SinceProperty I.5 guarantees that the COPRA function approacheszero from a negative direction as ˜ γ o approaches + ∞ , then S (˜ γ o ) has an extremum in the interval ( ǫ, + ∞ ) and thisextremum is actually a maximum point. If the point of theextremum is considered to be ˜ γ o,m , then the function startsdecreasing for ˜ γ o > ˜ γ o,m until it approaches the second zerocrossing at ˜ γ o, . As Theorem I.1 states clearly that we cannothave more than two roots, we conclude that the COPRAfunction in (2) has only one unique positive root over theinterval ( ǫ, + ∞ ) . A. Finding the root of S (˜ γ o ) To find the positive root of the COPRA function S (˜ γ o ) in(2), Newton’s method [6] can be used. The function S (˜ γ o ) is differentiable in the interval ( ǫ, + ∞ ) and the expression ofthe first derivative S ′ (˜ γ o ) can be easily obtained. Newton’smethod can then be applied in a straightforward manner tofind this root. Starting from an initial value γ n =0 o > ǫ that issufficiency small, the following iterations are performed: ˜ γ n +1 o = ˜ γ n o − S (˜ γ o ) S ′ (˜ γ o ) . (20)The iterations stop when | S (˜ γ n +1 o ) | < ρ , where ρ is asufficiently small positive quantity. B. Convergence
The convergence of Newton’s method can be easily proved.As a result from Theorem I.3, the function S (˜ γ o ) has always apositive value in the interval ( ǫ, ˜ γ o, ) . It is also clear that S (˜ γ o ) is a decreasing function in the interval [˜ γ n =0 o , ˜ γ o,2 ] . Thus, start-ing from ˜ γ n =0 o ≫ ǫ , (20) will produce a consecutive increaseestimation for ˜ γ o . Convergence occurs when S (˜ γ n o ) → and ˜ γ n +1 o → ˜ γ n o . C. COPRA summery
The proposed COPRA can be summarized as in Algo-rithm 1.
Algorithm 1
COPRA Summery Define ρ as the iterations stopping criterion. Set ˜ γ n =0 o to be a sufficiently small positive quantity. Find S (cid:0) ˜ γ n =0 o (cid:1) using (2), and compute its derivative S ′ (cid:0) ˜ γ n =0 o (cid:1) . while | S (˜ γ n o ) | > ρ do Solve (20) to get ˜ γ n +1 o . ˜ γ n o = ˜ γ n +1 o . end while γ o = N ˜ γ o . Find ˆ x using ˆ x = (cid:0) H H H + γ o I (cid:1) − H H y .II. E XTENSION : N
UMERICAL RESULTS
In this section we present an additional simulation resultsfor the proposed COPRA.Firstly, the model matrix is generated as H ∈ C × , H ∼CN ( , I ) with i.i.d entries. This H is combined with x ∼N ( , I ) that has an i.i.d. elements. The performance is eval-uated in terms of normalized MSE (NMSE) versus signal-to-noise-ratio (SNR) defined as SNR , k Hx k /N σ z . Theperformance of all the methods is evaluated over differentnoise and matrix realizations at each SNR. From Fig 1, it canbe seen that the proposed COPRA is providing a NMSE thatis very close to the LMMSE estimator. GCV algorithm is alsoproviding a close performance to the proposed COPRA. For avery high SNR, all methods are providing approximately thesame NMSE. SNR [dB] -10 -5 0 5 10 15 20 25 30 N M S E [ d B ] -25-20-15-10-505101520 LSL-curveQuasiGCVCOPRALMMSE Fig. 1: Performance comparison when H ∈ C × , H ∼CN ( , I ) with i.i.d entries and x ∼ N ( , I ) with i.i.d.elements.Secondly, the matrix H is generated as H ∈ R × , H ∼N ( , I ) that has an i.i.d entries. This H is combined with a stochastic Gaussian signal x that has an independent but notidentically distributed (i.n.d.) entries of zero mean and unitvariance. The performance is presented as the NMSE versusSNR (in dB) and is evaluated over different noise and H realizations at each SNR value.From Fig 2, it can be observed that the proposed COPRAoutperforms all the benchmark methods over all the SNRvalues. It is also clear that the LS performance it above 10 dBoverall the SNR range, which makes it completely unreliable. SNR [dB]-10 -5 0 5 10 15 20 25 30 N M S E [ d B ] -15-10-50510 LSL-curveQuasiGCVCOPRALMMSE Fig. 2: Performance comparison when H ∈ R × , H ∼N ( , I ) with i.i.d entries and x is stochastic Gaussian withi.n.d. elements.Finally, a different scenario of the transmitted signal isconsidered where x is taken to be a gray encoded 8-ary QAMsignal with unit power and the model matrix is generatedas H ∈ C × , H ∼ CN ( , I ) . In this example, theperformance is evaluated in terms of the bit error rate (BER),and also the NMSE. Noise is added to Hx according to acertain E b / N o (energy per bit to noise power spectral densityratio) (in dB) to generate y . Performance is presented asthe BER versus E b / N o (in dB), and also the NMSE (in dB)versus E b / N o (in dB). The performance of all the methods isevaluated over different realizations of the noise z and thematrix H at each E b / N o value.Fig. 3a plots the BER versus E b / N o (in dB). It is evident thatthe proposed COPRA outperforms all the benchmark methodsand stay very close to the LMMSE especially in the range from0 to 15 dB E b / N o where it offers exactly the BER providedby the LMMSE estimator.Fig. 3b depicts the comparison between the methods interms of the NMSE. Again, we can observe that the proposedCOPRA outperforms all the methods and performs almostidentically to the LMMSE estimator in the low E b / N o range. E b /N o [dB]0 5 10 15 20 25 30 B E R -3 -2 -1 LSL-curveQuasiGCVCOPRALMMSE (a) BER versus E b /N o [dB]. E b /N o [dB]0 5 10 15 20 25 30 N M S E [ d B ] -20-15-10-505 LSL-curveQuasiGCVCOPRALMMSE (b) NMSE [dB] versus E b /N o [dB]. Fig. 3: Performance comparison when H ∈ C × , H ∼CN ( , I ) is combined with gray encoded 8-ary QAM signalwith unit power. R EFERENCES[1] Willliam Feller,
An introduction to probability theory and its applications ,vol. 2, John Wiley & Sons, 2008.[2] David Vernon Widder,
Laplace Transform (PMS-6) , Princeton UniversityPress, 2015.[3] David W Kammler, “Chebyshev approximation of completely monotonicfunctions by sums of exponentials,”
SIAM Journal on Numerical Analysis ,vol. 13, no. 5, pp. 761–774, 1976.[4] David W Kammler, “Least squares approximation of completely mono-tonic functions by sums of exponentials,”
SIAM Journal on NumericalAnalysis , vol. 16, no. 5, pp. 801–818, 1979.[5] Yuri K Shestopaloff,
Sums of exponential functions and their newfundamental properties, with applications to natural phenomena , AkvyPress, 2008.[6] Christopher J Zarowski,