RSB Decoupling Property of MAP Estimators
aa r X i v : . [ c s . I T ] N ov RSB Decoupling Property of MAP Estimators
Ali Bereyhi ∗ , Ralf Müller ∗ , Hermann Schulz-Baldes †∗ Institute for Digital Communications (IDC), † Department of Mathematics,Friedrich Alexander Universität (FAU), Erlangen, [email protected], [email protected], [email protected]
Abstract —The large-system decoupling property of a MAPestimator is studied when it estimates the i.i.d. vector x fromthe observation y = A x + z with A being chosen from awide range of matrix ensembles, and the noise vector z beingi.i.d. and Gaussian. Using the replica method, we show that themarginal joint distribution of any two corresponding input andoutput symbols converges to a deterministic distribution whichdescribes the input-output distribution of a single user systemfollowed by a MAP estimator. Under the b RSB assumption, thesingle user system is a scalar channel with additive noise wherethe noise term is given by the sum of an independent Gaussianrandom variable and b correlated interference terms. As the b RSB assumption reduces to RS, the interference terms vanishwhich results in the formerly studied RS decoupling principle.
I. I
NTRODUCTION
A linear vector system with Additive White Gaussian Noise(AWGN) is described by y = A x + z (1)where the independent and identically distributed (i.i.d.) sourcevector x n × , taken from support X n , is measured by the ran-dom system matrix A k × n and corrupted by an i.i.d. Gaussiannoise vector z k × . The observation vector y is given to thevector estimator g ( · ) which maps the k -dimensional vector y to an n -dimensional vector ˆ x n × ∈ X n . The entries of ˆ x are in general correlated due to the coupling imposed by A and g ( · ) . Considering the entries x j and ˆ x j , ≤ j ≤ n ,the marginal joint distribution of (ˆ x j , x j ) in the large-systemlimit, i.e. k, n ↑ ∞ , is of interest. To clarify the point, considerthe linear estimation, i.e. g ( y ) = G T y for some G k × n , anddenote A = [ a · · · a n ] and G = [ g · · · g n ] with a i and g i being k × vectors for i ∈ { , . . . , n } . Thus, ˆ x j = (cid:0) g T j a j (cid:1) x j + n X i =1 ,i = j (cid:0) g T j a i (cid:1) x i + g T j z . (2)One considers the right hand side (r.h.s.) of (2) as the linearestimation of a single user system with additive impairmentin which the impairment term is not necessarily Gaussian andthe system is indexed by j . For some families of A and G , itis shown that the index dependency of these systems vanishesand the impairment term converges to a Gaussian noise termwith modified power level when the system dimensions tendto infinity, e.g. [1]. Thus, one can assume the linear vectorestimator in the large-system limit to decouple into a bank ofsingle user linear estimators operating over n parallel scalarsystems with additive Gaussian noise terms. This decoupling This work was supported by the German Research Foundation, DeutscheForschungsgemeinschaft (DFG), under Grant No. MU 3735/2-1. property of the linear estimators is rigorously justified invokingthe central limit theorem and the properties of large randommatrices. For nonlinear forms of g ( · ) , however, the analysisfaces difficulties, since the output entries do not linearlydecouple. Tanaka noted the similarity between the asymptoticanalysis of spin glasses [2] and vector estimators and showedthat the performance of a vector estimator in the large-systemlimit can be represented as the macroscopic parameter of aspin glass [3]. Consequently, a class of generally nonlinearestimators was analyzed using the nonrigorous replica methoddeveloped in statistical mechanics. Inspired by [3], severalworks employed the replica method to study the performanceof nonlinear estimators in the asymptotic regime consideringdifferent classes of estimators, system matrices, and perfor-mance measures, e.g. [4]. Having the decoupling propertyof the linear estimators in mind, it was conjectured that thisproperty holds for nonlinear estimators as well. In [5], Guoand Verdú justified this conjecture for the postulated MinimumMean Square Error (MMSE) estimator g ( y ) = E { x | y , A } (3)where A is considered to be i.i.d., and the expectation is takenover x due to some postulated posterior distribution q x | y , A .For this setup, the authors showed the Replica Symmetry (RS)decoupling principle which says that under the RS assumptionthe marginal joint distribution of ( x j , ˆ x j ) converges to theinput-output joint distribution of a scalar channel with additiveGaussian noise followed by a single user MMSE estimator.The RS decoupling principle was further extended to the casewith a postulated Maximum-A-Posteriori (MAP) estimator in[6] where Rangan et al. studied g ( y ) = arg min v (cid:20) λ k y − A v k + u ( v ) (cid:21) (4)for some “utility function” u ( · ) : R n → R + and non-negativereal “estimation parameter” λ . Except for the cases with ani.i.d. system matrix, the decoupling property of nonlinearestimators for a larger class of matrix ensembles has notyet been addressed precisely. In [7], the authors investigatedthis issue partially by studying the support recovery of sparseGaussian sources. They considered the case of a source vectorwhich is first randomly measured by a squared matrix, andthen, the measurements are sparsely sampled by an i.i.d. binaryvector. Employing a MAP estimator for recovering, the RSdecoupling principle was justified for the case in which themeasuring matrix belongs to a large set of matrix ensembles.Although the class of matrices is broadened in [7], the resultcannot be considered as a complete generalization of [5] and6], since it is restricted to cases with a sparse Gaussiansource and kn − ≤ . Another issue not investigated in theliterature is the marginal joint distribution under the ReplicaSymmetry Breaking (RSB) assumption. In fact, the previousstudies investigated the decoupling principle considering theRS ansatz; however, despite the RS validity in some particularcases, there are still several cases requiring further RSBinvestigations, e.g. [8], [9].In this paper, we address both the issues and broaden thescope of the decoupling principle stated in [6] to both a largerset of matrix ensembles, and the RSB ansätze. More precisely,we justify the decoupling property of the postulated MAPestimator when1) A is chosen from a large family of random matrices,2) kn − takes any non-negative real number, and3) the RS and RSB ansätze are considered.For this setup, we show that under all replica ansätze, the jointdistribution of ( x j , ˆ x j ) in the large-system limit converges tothe input-output distribution of a scalar system in which thesource symbol is corrupted by effective noise and estimated bya single user MAP estimator. We determine the effective noiseterm and estimation parameter under the RSB assumption with b steps of breaking ( b RSB), and show that the noise termunder this assumption is given by the sum of an independentGaussian random variable and b correlated terms. By reducingthe assumption to RS, the correlated terms vanish, and thenoise term becomes Gaussian. Thus, one can consider thedecoupling principle of [6] to be a special case of the moregeneral decoupling principle illustrated here. Notation:
We represent vectors, scalars and matrices withbold lower case, non-bold lower case, and bold upper caseletters, respectively. The set of real numbers is denoted by R ,and A T and A H indicate the transposed and Hermitian of A . I m is the m × m identity matrix, m is the matrix with allentries equal to one, and ⊗ denotes the Kronecker product. Fora random variable x , p x represents either the Probability MassFunction (PMF) or Probability Density Function (PDF), and F x represents the Cumulative Distribution Function (CDF).We denote the expectation over x by E x , and an expectationover all random variables involved in a given expression by E . For sake of compactness, the set of integers { , . . . , n } isdenoted by [1 : n ] , the zero-mean and unit-variance GaussianPDF by π ( · ) , and Z D t := Z π ( t )d t. (5)Whenever needed, we consider the entries of x to be discreterandom variables; the results of this paper, however, are in fullgenerality and directly extend to continuous distributions.II. P ROBLEM F ORMULATION
Let the system in (1) satisfy the following constraints.(a) The number of observations k is a deterministic sequenceof n such that lim n ↑∞ kn = 1r < ∞ . (6) (b) x n × is an i.i.d. random vector with each element beingdistributed due to p x over X in which X ⊆ R .(c) A k × n is randomly generated over A k × n ⊆ R k × n , suchthat J = A T A has the eigendecomposition J = UDU T (7)where U is an orthogonal Haar distributed matrix and D is a diagonal matrix with the empirical eigenvaluedistribution (density of states) converging as n ↑ ∞ toa deterministic distribution F J .(d) z k × is a real i.i.d. zero-mean Gaussian random vectorwith variance λ , i.e., z ∼ N ( , λ I k ) .(e) x , A , and z are independent.In order to estimate the source vector, the postulated MAP esti-mator as defined in (4) is employed. The estimators postulatesa non-negative estimation parameter λ and a non-negative util-ity function u ( · ) which decouples, i.e., u ( x ) = P ni =1 u ( x i ) .Defining the estimated vector ˆ x := g ( y ) , the conditionaldistribution of ˆ x j given x j for some j ∈ [1 : n ] is denoted by p j ( n )ˆ x | x . Thus, the marginal joint distribution of x j and ˆ x j at themass point (ˆ v, v ) is written as p ˆ x j ,x j (ˆ v, v ) = p x ( v )p j ( n )ˆ x | x (ˆ v | v ) . (8)Considering the large-system limit, we define the asymptoticconditional distribution of ˆ x j given x j at (ˆ v, v ) as p j ˆ x | x (ˆ v | v ) := lim n ↑∞ p j ( n )ˆ x | x (ˆ v | v ) . (9)We also suppose the self averaging assumption which says(f) Given A of the form (7) with F J , the limit in (9) existsand is almost surely constant in realizations of A .III. G ENERAL D ECOUPLING P RINCIPLE
The main contribution of this study is to extend the scope ofthe decoupling principle. To illustrate the result, consider thefollowing single user system: the input x is passed through thechannel y = x + z where z ∼ p z | x for the given input x . Theobservation y is then given to a single user MAP estimatorwith the same utility function as for the vector estimatordefined in Section II, i.e. u ( · ) , and an estimation parameterdenoted by λ s . Indicating the conditional distribution of theestimator’s output ˆ x for the given input x by p ˆ x | x , our generaldecoupling principle says that under a set of assumptions(a) the asymptotic conditional distribution p j ˆ x | x is indepen-dent of the index j , and we have p j ˆ x | x = p ˆ x | x .(b) p z | x and λ s are determined in terms of λ , λ and thestatistics of x and A .The set of assumptions which yields the validity of the abovestatements are enforced within the large-system analysis. In thefollowing, we briefly illustrate our approach and determine theparameters of the single user system.IV. D ERIVATION OF G ENERAL D ECOUPLING P RINCIPLE
Before illustrating our derivation approach, let us define the R -transform. For a random variable t , the Stieltjes transformver the upper half complex plane is defined as G t ( s ) = E [ t − s ] − . Denoting the inverse with respect to (w.r.t.) compositionwith G − t ( · ) , the R -transform is R t ( ω ) = G − t ( ω ) − ω − such that lim ω ↓ R t ( ω ) = E t . The definition also extendsto matrix arguments. Assuming a matrix M n × n to have theeigendecomposition M = U diag[ λ , . . . , λ n ] U − , R t ( M ) isthen defined as R t ( M ) = U diag[R t ( λ ) , . . . , R t ( λ n )] U − .The derivation of the general decoupling principle is basedon the moment method. To clarify the approach, consider thenon-negative integers k and ℓ , and define the joint moment M j ( n ) k,ℓ = E ˆ x kj x ℓj , for j ∈ [1 : n ] . After evaluating the limit of M j ( n ) k,ℓ as n ↑ ∞ , we show that for all k and ℓ the asymptoticjoint moment is equivalent to the corresponding joint momentof the single user system. Consequently, using the uniquenessof the mapping from the set of integer moments’ sequencesto the set of measures, under a set of conditions investigatedin the classical moment problem [10], we conclude that bothcouples (ˆ x j , x j ) and (ˆ x, x ) have a same distribution. We startwith evaluating the limit of M j ( n ) k,ℓ . The evaluation is based onthe nonrigorous method of replicas developed in the theoryof spin glasses [2], and accepted as a mathematical tool ininformation theory. To do so, define the “weighted averagejoint moment” over the index set W ⊂ [1 : n ] as M W ( n ) k,ℓ ( ˆ x ; x ) := E | W | X w ∈ W ˆ x kw x ℓw . (10)Setting W = [ j : j + nη ] for some η ∈ (0 , , the asymptoticjoint moment of j th entry can be written as M jk,ℓ := lim n ↑∞ M j ( n ) k,ℓ = lim n ↑∞ lim η ↓ M W ( n ) k,ℓ ( ˆ x ; x ) . (11)Thus, the evaluation of the asymptotic moment reduces totaking the limits in the r.h.s. of (11) which needs the weightedaverage joint moment in (10) to be explicitly calculated for anarbitrary integer n . Alternatively, we can define the function Z ( β, h ) = X v e − β [ λ k y − A v k + u ( v ) ] + hn M W ( n ) k,ℓ ( v ; x ) . (12)with v ∈ X n . Noting that ˆ x = g ( y ) with g ( · ) defined in (4), M W ( n ) k,ℓ ( ˆ x ; x ) = lim β ↑∞ lim h ↓ n ∂∂h E log Z ( β, h ) . (13)The logarithmic expectation in the r.h.s. of (13) is not a trivialtask to do, and therefore, one bypasses the direct evaluationusing the Riesz equality which for any random variable t states E log t = lim m ↓ m − log E t m . Thus, regarding (11) and (13) M jk,ℓ = lim n ↑∞ lim η ↓ lim β ↑∞ lim h ↓ lim m ↓ n ∂∂h log E [ Z ( β, h )] m m (14)for W = [ j : j + nη ] . In (14), we face two major difficulties:1) evaluating the real moments i.e., E [ Z ( β, h )] m , and 2) tak-ing the limits in the order stated. Basic analytical methodsfail to address these challenges properly, and therefore, weinvoke the nonrigorous method of replicas. The replica methodsuggests to evaluate the moment for an arbitrary integer m asan analytic function in m ; then, assume that 1) the “replica continuity” holds which means that the function analyticallycontinues from the set of integers to the real axis (or at least avicinity of zero), and 2) the limits are exchangeable. Followingthe above prescription, we consider the first assumption andfind E [ Z ( β, h )] m which for an integer m reduces to E m Y a =1 X v a e − β [ λ k A ( x − v a )+ z k + u ( v a ) ] + hn M W ( n ) k,ℓ ( v a ; x ) . (15)In order to evaluate (15), one can initially take the expectationover z and A . Due to the lack of space, we leave the details forthe extended version of the manuscript; however, we brieflyexplain the strategy. After taking the expectations, and definingthe m × m “replica correlation matrix” Q such that [ Q ] ab = n − ( x − v a ) T ( x − v b ) , (15) is given in terms of Q as E [ Z ( β, h )] m = E x Z e − n G ( TQ ) e n I ( Q ) d Q (16)with d Q := Q ma,b =1 d[ Q ] ab , T := λ I m − β λ λ m , and theintegral being taken over R m . For a given x , e n I ( Q ) measuresthe probability weight of the set of replicas, { v a } ma =1 , in whichthe correlation matrix is Q ; moreover, G ( · ) is defined as G ( M ) = Z β Tr { M R J ( − ω M ) } d ω + ǫ n (17)where Tr {·} denotes the trace, R J ( · ) is the R -transform w.r.t. F J , and ǫ n tends to zero as n ↑ ∞ . Here, one can employ theLaplace method of integration and replace the r.h.s. of (16)in the large-system limit with the integrand at its saddle pointmultiplied by some bounded coefficient K n ; thus, as n ↑ ∞ E [ Z ( β, h )] m . = K n e − n [ G ( T ˜ Q ) −I ( ˜ Q ) ] (18)at the saddle point ˜ Q . Substituting (18) in (14), we have M jk,ℓ = lim η ↓ lim β ↑∞ lim m ↓ E P v M [1: m ] k,ℓ ( v ; x ) E ( ˜ Q , x , v ; β ) P v E ( ˜ Q , x , v ; β ) (19)where x m × = [ x, . . . , x ] T with x ∼ p x , v m × ∈ X m and E ( ˜ Q , x , v ; β ) = e − β ( x − v ) T T R J ( − β T ˜ Q )( x − v ) − βu ( v ) . (20)Here, one needs to find the saddle point which is not a feasibletask in general. The strategy for pursuing the analysis is torestrict the saddle point, i.e. ˜ Q , to be of a special form, and findthe solution within the restricted set of matrices. This is wherean additional assumption such as the RS or RSB assumptionarises. It is clear that these restrictions do not lead us to thecorrect solution in general, and therefore, one needs to widenthe set of replica correlation matrices to find a more accuratesolution. In the sequel, we consider different structures onthe correlation matrix and find the replica ansatz under thoseassumptions. However, considering (19), it is observed that“regardless of the structure” on ˜ Q , the joint moment M jk,ℓ isindependent of the index j even before taking the limit η ↓ .Therefore, employing the moment method, we conclude Proposition 1
Let the vector system satisfy the constraintsin Section II; moreover, assume the replica continuity to hold, g map [( · ); λ s , u ] x y ˆ x p λ s z Fig. 1: The decoupled scalar system under the RS ansatz. and the limits in (14) to exist and exchange. Then p j ˆ x | x , asdefined in (9) , does not depend on the index j . Proposition 1 states a more general form of the decouplingprinciple studied in previous works. In fact, the only assump-tions which need to be satisfied are the replica continuity andthe exchange of limits; and, no structure of the correlationmatrix is imposed. However, the decoupled scalar system doesdepend on the structure imposed on ˜ Q . To find the decoupledsingle user system, we start with the most primary structurewhich is imposed by considering the RS assumption. RS Assumption:
Here, we restrict the search to the set ofparameterized matrices which are of the form ˜ Q = q m + χβ I m . (21)for some χ, q ∈ R + . Substituting in (19), we have M jk,ℓ = E Z g k x ℓ D z (22)where g := g map [( y ); λ s , u ] with y = x + p λ s z and g map [( y ); λ s , u ] = arg min v (cid:20) λ s ( y − v ) + u ( v ) (cid:21) . (23)Moreover, λ s and λ s are defined as λ s = h R J ( − χλ ) i − ∂∂χ n [ λ χ − λq ] R J ( − χλ ) o (24a) λ s = h R J ( − χλ ) i − λ (24b)where q = E R [g − x ] D z , and χ satisfies p λ s χ = λ s E Z [g − x ] z D z. (25)(23) describes a single user MAP estimator with the postulatedutility function u ( · ) and estimation parameter λ s . Thus, Proposition 2
Let the assumptions in Proposition 1, as wellas the RS assumption hold, and consider the single user systemin Figure 1 with λ s and λ s as defined in (24a) and (24b) . Thenfor j ∈ [1 : n ] , p j ˆ x | x as defined in (9) describes the conditionaldistribution of ˆ x given x in Figure 1 where g map [( · ); λ s , u ] isa single user MAP defined in (23) , and p z | x ( z | x ) = π ( z ) . The results in the literature have always considered the RSansatz, and can be recovered as special cases of Proposition 2.E.g., results of [6] are derived by setting R J ( ω ) = (1 − r ω ) − .The RS ansatz, however, does not provide a valid solution,in general. Parisi in [11] introduced the RSB scheme whichwidens the restricted set of saddle point matrices recursively.To illustrate the RSB scheme, let Q b be a basic structure forthe replica correlation matrix; moreover, assume m to be amultiple of an integer ξ . Then, the correlation matrix can begrouped as a ξ × ξ matrix of blocks with each block being an + + g map [( · ); λ s , u ] x y ˆ x p λ s z p λ s z Fig. 2: The decoupled scalar system under the 1RSB ansatz. mξ × mξ matrix. In this case, a new structure for the correlationmatrix is obtained by setting the diagonal blocks to be Q b andthe off-diagonal blocks to be κ mξ for some κ . In fact, the newset of correlation matrices is constructed by imposing the RSstructure block-wisely using Q b and κ mξ as basic blocks. TheRSB scheme can be recursively iterated: one can set the basicstructure Q b and find the new structure Q b ; then, by taking Q b as the new basic structure, a wider set of correlation matricesis found. Parisi considers Q b to have the RS structure. Using the RSB scheme with one step ofiteration, the structure of the correlation matrix is found as ˜ Q = q m + p I mβµ ⊗ µβ + χβ I m (26)for some χ, p, q, µ ∈ R + . Therefore, the joint moment reads M jk,ℓ = E Z g k x ℓ dF( z )D z (27)where g := g map [( y ); λ s , u ] with y = x + p λ s z + p λ s z ,and dF( z ) = ΛD z with Λ = [ R ˜ΛD z ] − ˜Λ and ˜Λ = e − µ n λ s [ ( y − g) − ( y − x ) ] + u (g) o . (28)Moreover, by denoting ̺ := χ + µp , we have λ s = h R J ( − χλ ) i − ∂∂̺ n [ λ ̺ − λq + λp ] R J ( − ̺λ ) o , (29a) λ s = h R J ( − χλ ) i − h R J ( − χλ ) − R J ( − ̺λ ) i λµ − , (29b) λ s = h R J ( − χλ ) i − λ. (29c)Here, q = E R [g − x ] dF( z )D z , and χ and p satisfy χ + µp = λ s p λ s E Z [g − x ] z dF( z )D z (30a) χ + µq = λ s p λ s E Z [g − x ] z dF( z )D z (30b)for some µ being a solution to the fixed point equation µ λ s (cid:20) µ λ s λ s q − µ λ s λ s p + p (cid:21) − λ Z ̺χ R J ( − ωλ )d ω == I( z ; x, z ) + D KL (p z k π ) (31)where I( · ; · ) and D KL ( ·k· ) indicate the mutual information andthe so-called “Kullback-Leibler” distance respectively, and therandom variables ( x, z , z ) ∼ p x ( x ) π ( z ) [Λ π ( z )] . Thus,one can conclude the following proposition. Proposition 3
Let the assumptions in Proposition 1, as wellas the 1RSB assumption hold, and consider the single usersystem in Figure 2 with λ s , λ s and λ s as defined in (29a) - (29c) .Then, for j ∈ [1 : n ] , p j ˆ x | x in (9) is the conditional distribution + + g map [( · ); λ s , u ] x y ˆ x p λ s z p λ s z p λ s b z b Fig. 3: The decoupled scalar system under the b RSB ansatz. of ˆ x given x in Figure 2 where g map [( · ); λ s , u ] is a single userMAP estimator defined in (23) , p z | x ( z | x ) = π ( z ) , and p z | x,z ( z | x, z ) = Λ π ( z ) (32) with Λ = [ R ˜ΛD z ] − ˜Λ and ˜Λ defined in (28) . Here, the decoupled system differs from the system obtainedunder the RS ansatz within one additive tap which is intuitivelyapproximating the interference caused by the coupling. In fact,the RS ansatz assumes the coupling caused by the systemmatrix and vector estimator to vanish as the system tends toits large limits; however, the 1RSB solution takes the couplinginto account and approximates it with one tap of interference.This approximation may become more accurate, if we let thecorrelation matrix to be chosen from a larger set of matrices. b RSB Assumption:
Iterating the RSB scheme with b steps, ˜ Q = q m + b X ν =1 p ν I mβµν ⊗ µνβ + χβ I m (33)for some χ, q, { p ν , µ ν } bν =1 ∈ R + . Thus, we have M jk,ℓ = E Z g k x ℓ b Y ν =1 dF( z ν )D z (34)where g := g map [( y ); λ s , u ] with y = x + P bν =0 p λ s ν z ν , and dF( z ν ) = Λ ν D z ν . For ν ∈ [1 : b ] , Λ ν is a function of x and { z ζ } νζ =0 . Due to the page limitations, we leave the expressionsof λ s , { λ s ν } bν =0 and { Λ ν } bν =1 for the extended version. Proposition 4
Let the assumptions in Proposition 1, andthe b RSB assumption hold; moreover, consider the single usersystem in Figure 3. Then, for j ∈ [1 : n ] , p j ˆ x | x as defined in (9) describes the conditional distribution of ˆ x given x in Figure 3where g map [( · ); λ s , u ] is a single user MAP estimator definedin (23) , p z | x ( z | x ) = π ( z ) , and p z ν | x, { z ζ } ν − ζ =0 ( z ν | x, { z ζ } ν − ζ =0 ) = Λ ν π ( z ν ) (35) for ν ∈ [1 : b ] . The factor Λ ν depends on x and { z ζ } νζ =0 , andthe coefficients λ s and { λ s ν } bν =0 are coupled due to a set offixed point equations and bounded as b ↑ ∞ . Considering the b RSB ansatz, one concludes that the ansatzextends the decoupled system in Figure 2 by approximatingthe coupling interference with more taps. The approximation,however, stops to improve at some step b ∗ , if Λ ν = 1 for anyinteger ν > b ∗ . The extreme case is when for any ν ∈ [1 : b ] inthe b RSB ansatz Λ ν = 1 . Here, the random variables { z ν } bν =1 in Figure 3 become independent Gaussian, and therefore, thedecoupled system reduces to Figure 1. In fact in this case, the b RSB solution, as well as any ν RSB ansatz with ν ∈ [1 : b ] , reduces to the RS ansatz. Thus, one can consider the decoupledsystem under the RS ansatz to be a special case of the moregeneral decoupled system given in Figure 3.V. C ONCLUSION
Decoupling seems to be a generic property of MAP estima-tors, as Proposition 1 justifies it for any source distribution anda wide range of matrix ensembles. The validity of the resultrelies only on replica continuity; however, the equivalent singleuser system depends on the structure of the replica correlationmatrix. Recent results in statistical mechanics have shown thatfailures in finding the exact solution via the replica methodare mainly caused by the assumed structure, and not replicacontinuity. Inspired by the Sherrington-Kirkpatrick model ofspin glasses, for which the ∞ RSB ansatz has been provedto be correct, one may consider Figure 3 to be the generaldecoupled system as b ↑ ∞ . However, in many cases anaccurate approximation might be provided by a finite numberof RSB steps. An extreme case is the RS ansatz where allthe interference terms in the RSB decoupled system becomeindependent and Gaussian. Thus, one concludes that the pre-vious results in the literature were both special and extremecases of the RSB decoupled system. The RSB decoupledsystem raises several issues which require further investi-gations. For example, nothing is known about the distancebetween the conditional distributions of the interference termsand independent Gaussian distributions in probability space.The distance variation w.r.t. the number of interference tapscan then describe the improvement caused by increasing thenumber of RSB steps. R EFERENCES[1] D. Guo, S. Verdú, and L. K. Rasmussen, “Asymptotic normality of linearmultiuser receiver outputs,”
Information Theory, IEEE Transactions on ,vol. 48, no. 12, pp. 3080–3095, 2002.[2] S. F. Edwards and P. W. Anderson, “Theory of spin glasses,”
Journalof Physics F: Metal Physics , vol. 5, no. 5, p. 965, 1975.[3] T. Tanaka, “A statistical-mechanics approach to large-system analysisof CDMA multiuser detectors,”
Information Theory, IEEE Transactionson , vol. 48, no. 11, pp. 2888–2910, 2002.[4] R. R. Müller and W. H. Gerstacker, “On the capacity loss due to separa-tion of detection and decoding,”
Information Theory, IEEE Transactionson , vol. 50, no. 8, pp. 1769–1778, 2004.[5] D. Guo and S. Verdú, “Randomly spread cdma: Asymptotics viastatistical physics,”
Information Theory, IEEE Transactions on , vol. 51,no. 6, pp. 1983–2010, 2005.[6] S. Rangan, A. K. Fletcher, and V. Goyal, “Asymptotic analysis ofMAP estimation via the replica method and applications to compressedsensing,” in
IEEE Trans. on Inf. Theory , 2012, pp. 1902–1923.[7] A. M. Tulino, G. Caire, S. Verdu, and S. Shamai, “Support recoverywith sparsely sampled free random matrices,”
Information Theory, IEEETransactions on , vol. 59, no. 7, pp. 4243–4271, 2013.[8] Y. Kabashima, T. Wadayama, and T. Tanaka, “A typical reconstructionlimit for compressed sensing based on ℓ p -norm minimization,” Journalof Statistical Mechanics: Theory and Experiment , vol. 2009, no. 09, p.L09003, 2009.[9] B. M. Zaidel, R. Müller, A. L. Moustakas, and R. de Miguel, “Vectorprecoding for gaussian MIMO broadcast channels: Impact of replicasymmetry breaking,”
Information Theory, IEEE Transactions on , vol. 58,no. 3, pp. 1413–1440, 2012.[10] N. I. Akhiezer,
The classical moment problem: and some relatedquestions in analysis . Oliver & Boyd, 1965, vol. 5.[11] G. Parisi, “A sequence of approximated solutions to the SK model forspin glasses,”