Generalized Approximate Message Passing for Cosparse Analysis Compressive Sensing
aa r X i v : . [ c s . I T ] O c t GENERALIZED APPROXIMATE MESSAGE PASSING FOR COSPARSE ANALYSISCOMPRESSIVE SENSING
Mark Borgerding and Philip Schniter
Dept. ECE, The Ohio State UniversityColumbus, OH 43210
Sundeep Rangan
Dept. of ECE, NYU Polytechnic InstituteBrooklyn, NY 11201.
ABSTRACT
In cosparse analysis compressive sensing (CS), one seeks to estimatea non-sparse signal vector from noisy sub-Nyquist linear measure-ments by exploiting the knowledge that a given linear transform ofthe signal is cosparse, i.e., has sufficiently many zeros. We propose anovel approach to cosparse analysis CS based on the generalized ap-proximate message passing (GAMP) algorithm. Unlike other AMP-based approaches to this problem, ours works with a wide range ofanalysis operators and regularizers. In addition, we propose a novel ℓ -like soft-thresholder based on MMSE denoising for a spike-and-slab distribution with an infinite-variance slab. Numerical demon-strations on synthetic and practical datasets demonstrate advantagesover existing AMP-based, greedy, and reweighted- ℓ approaches. Index Terms — Approximate message passing, belief propaga-tion, compressed sensing.
1. INTRODUCTION
We consider the problem of recovering a signal x ∈ R N (e.g., an N -pixel image) from the possibly noisy linear measurements y = Φ x + w ∈ R M , (1)where Φ represents a known linear measurement operator w repre-sents noise, and M ≪ N . We focus on the analysis compressivesensing (CS) problem [1, 2] where, for a given analysis operator Ω , u , Ω x ∈ R D (2)is assumed to be cosparse (i.e., contain sufficiently many zero-valued coefficients). This differs from the synthesis CS problem,where x is assumed to be sparse (i.e., contain sufficiently few non-zero coefficients). Although the two problems become interchange-able when Ω is invertible, we are mainly interested in non-invertible Ω , as in the “overcomplete” case where D > N . We note that,although we assume real-valued quantities throughout, the proposedmethods can be directly extended to the complex-valued case, whichwe demonstrate using numerical experiments.The analysis CS problem is typically formulated as a regularizedloss-minimization problem of the form b x rlm = arg min x k y − Φ x k + h ( Ω x ) , (3)with separable regularizer h ( u ) = P Dd =1 h d ( u d ) . One of the mostfamous instances of h ( u ) is that of total-variation (TV) regulariza-tion [3], where h ( u ) = λ k u k and Ω computes variation across This work was supported by NSF grants CCF-1018368, CCF-1218754,and an allocation of computing time from the Ohio Supercomputer Center. neighboring pixels. In the anisotropic case, this variation is mea-sured by finite difference operators, e.g., Ω = [ D H h , D H v ] H , where D h computes horizontal differences and D v computes vertical dif-ferences. Of course, ℓ regularization can be used with generic Ω ,with the desirable property that it always renders (3) convex. Theresulting problem, sometimes referred to as the generalized LASSO (GrLASSO) [4], is amenable to a wide range of efficient optimiza-tion techniques like Douglas-Rachford splitting [5] and NESTA [6].Despite the elegance of the ℓ norm, several studies have shownimprovements from the use of ℓ -like norms for h ( u ) , especially forhighly overcomplete Ω (i.e., D ≫ N ). For example, the use ofiteratively reweighted ℓ [7] has demonstrated significant improve-ments over ℓ regularization in the context of analysis CS [8, 9].Likewise, greedy approaches to locate the zero-valued elements in u have also demonstrated significant improvements over ℓ . Examplesinclude greedy analysis pursuit (GAP), analysis iterative hard thresh-olding (AIHT), analysis hard thresholding pursuit (AHTP), analysisCoSaMP (ACoSaMP), and analysis subspace pursuit (ASP) [2, 10].In this paper, we propose a Bayesian approach to analysis CSthat leverages recent advances in approximate message passing(AMP) algorithms [11, 12], and in particular the generalized AMP(GAMP) algorithm from [13].While other AMP-based approaches have been recently pro-posed for the special case where Ω is a 1D finite difference operator,i.e., the TV-AMP from [14] and the ssAMP from [15], our approachworks with a generic analysis operator Ω and a much broader rangeof signal priors and likelihoods. Furthermore, our approach fa-cilitates both MAP and (approximate) MMSE estimation of x ina computationally efficient manner. We also note that a differentBayesian approach to cosparse analysis CS, based on multivariateGauss-mixture priors, was recently presented in [16]. The MAP andMMSE estimation methods proposed in [16], which employ greedypursuit and Gibbs sampling, respectively, have computational com-plexities that scale as O ( D N ) and O ( D N ) (assuming M ≤ D ).In contrast, ours scales like O ( DN ) for generic Ω , or O ( N log N ) when Φ and Ω have fast implementations, which is often the casein imaging applications.
2. GENERALIZED AMP FOR ANALYSIS CS2.1. The proposed Bayesian model
Our approach is Bayesian in that it treats the true signal x as a real-ization of a random vector x ∈ R N with prior pdf p x ( x ) and like-lihood function p y | q ( y | Φ x ) , where y are the observed noisy mea-surements and q , Φ x are akin to hidden noiseless measurements.(For clarity, we write random quantities using san-serif fonts and de-terministic ones using serif fonts.) Furthermore, we assume that therior and likelihood have the forms p y | q ( y | Φ x ) ∝ M Y m =1 exp( − l m ([ Φ x ] m )) (4) p x ( x ) ∝ D Y d =1 exp( − h d ([ Ω x ] d )) N Y n =1 exp( − g n ( x n )) (5)with scalar functions l m ( · ) , h d ( · ) , and g n ( · ) . Note that each mea-surement value y m is coded into the corresponding function l m ( · ) .We discuss the design of these functions in the sequel.Given the form of (4) and (5), the MAP estimate b x MAP , arg max x p x | y ( x | y ) can be written (using Bayes rule) as b x MAP = arg min x (cid:8) l ( Φ x ) + h ( Ω x ) + g ( x ) (cid:9) (6)with separable loss function l ( q ) = P Mm =1 l m ( q m ) and separableregularizers g ( x ) = P Nn =1 g n ( x n ) and h ( u ) = P Dd =1 h d ( u d ) . Notethat, with trivial g ( x ) = 0 and quadratic loss l ( q ) = k q − y k , theMAP estimation problem (6) reduces to the regularized loss mini-mization problem (3). But clearly (6) is more general.As for the MMSE estimate b x MMSE , R x p x | y ( x | y ) d x , exactevaluation requires the computation of a high dimensional integral,which is intractable for most problem sizes of interest. In the sequel,we present a computationally efficient approach to MMSE estima-tion that is based on loopy belief propagation and, in particular, theGAMP algorithm from [13]. The GAMP algorithm [13] aims to estimate the signal x from thecorrupted observations y , where x is assumed to be a realizationof random vector x ∈ R N with known prior p x ( x ) and likelihoodfunction p y | z ( y | Ax ) . Here, the prior and likelihood are assumed tobe separable in the sense that p y | z ( y | z ) ∝ I Y i =1 exp( − f i ( z i )) , p x ( x ) ∝ N Y n =1 exp( − g n ( x n )) , (7)where z , A x ∈ R I can be interpreted as hidden transform out-puts. The MAP version of GAMP aims to compute b x MAP =arg max x p x | y ( x | y ) , i.e., solve the optimization problem b x MAP = arg min x I X i =1 f i ([ Ax ] i ) + N X n =1 g n ( x n ) , (8)while the MMSE version of GAMP aims to compute the MMSE es-timate b x MMSE , R x p x | y ( x | y ) d x , in both cases by iterating simple,scalar optimizations. MAP-GAMP can be considered as the exten-sion of the AMP algorithm [11] from the quadratic loss f ( z ) = k y − z k to generic separable losses of the form f ( z ) = P Ii =1 f i ( z i ) .Likewise, MMSE-GAMP can be considered as a similar extensionof the Bayesian-AMP algorithm [12] from additive white Gaussiannoise (AWGN) p y | z ( y | z ) to generic p y | z ( y | z ) of the form in (7).In the large-system limit (i.e., I, N → ∞ with
I/N convergingto a positive constant) under i.i.d sub-Gaussian A , GAMP is char-acterized by a state evolution whose fixed points, when unique, areBayes optimal [17, 18]. For generic A , it has been shown [19] thatMAP-GAMP’s fixed points coincide with the critical points of thecost function (8) and that MMSE-GAMPs fixed points coincide withthose of a certain variational cost that was connected to the Bethe definitions for MMSE-GAMP: F i ( b p, ν p ) , R z exp( − f i ( z )) N ( z ; b p, ν p ) dz R exp( − f i ( z )) N ( z ; b p, ν p ) dz (D1) G n ( b r, ν r ) , R x exp( − g n ( x )) N ( x ; b r, ν r ) dx R exp( − g n ( x )) N ( x ; b r, ν r ) dx (D2) definitions for MAP-GAMP: F i ( b p, ν p ) , arg min z f i ( z ) + ν p | z − b p | (D3) G n ( b r, ν r ) , arg min x g n ( x ) + ν r | x − b r | (D4) inputs: ∀ i, n : F i , G n , b x n (1) , ν xn (1) , a in , T max ≥ , ǫ ≥ , β ∈ (0 , initialize: ∀ i : b s i (0) = 0 , t = 1 for t = 1 , . . . , T max ,if t = 1 , then β = 1 , else β = β (R1) ∀ i : ν pi ( t ) = β P Nn =1 | a in | ν xn ( t ) + (1 − β (cid:1) ν pi ( t − (R2) ∀ i : b p i ( t ) = P Nn =1 a in b x n ( t ) − ν pi ( t ) b s i ( t − (R3) ∀ i : ν zi ( t ) = ν pi ( t ) F ′ i ( b p i ( t ) , ν pi ( t )) (R4) ∀ i : b z i ( t ) = F i ( b p i ( t ) , ν pi ( t )) (R5) ∀ i : ν si ( t ) = β (cid:18) − ν zi ( t ) ν pi ( t ) (cid:19) ν pi ( t ) + (cid:0) − β (cid:1) ν si ( t − (R6) ∀ i : b s i ( t ) = β b z i ( t ) − b p i ( t ) ν pi ( t ) + (cid:0) − β (cid:1)b s i ( t − (R7) ∀ n : e x n ( t ) = β b x n ( t ) + (cid:0) − β (cid:1)e x n ( t − (R8) ∀ n : ν rn ( t ) = β P Ii =1 | a in | ν si ( t ) ! + (cid:0) − β (cid:1) ν rn ( t − (R9) ∀ n : b r n ( t ) = e x n ( t ) + ν rn ( t ) P Ii =1 a ∗ in b s i ( t ) (R10) ∀ n : ν xn ( t +1) = ν rn ( t ) G ′ n ( b r n ( t ) , ν rn ( t )) (R11) ∀ n : b x n ( t +1) = G n ( b r n ( t ) , ν rn ( t )) (R12) if k b x ( t ) − b x ( t +1) k / k b x ( t +1) k < ǫ, then stop (R13) endoutputs: ∀ n : b x n ( t +1) Table 1 . The damped GAMP algorithm. In (R4) and (R11), F ′ i and G ′ n denote the derivatives of F i and G n w.r.t their first arguments.free entropy in [20]. However, with general A (e.g., non-zero-mean A [21] or ill-conditioned A [22]) GAMP may not converge to itsfixed points, i.e., it may diverge. In an attempt to prevent divergencewith generic A , damped [22, 23], adaptively damped [24], and se-quential [20] versions of GAMP have been proposed.A damped version of the GAMP algorithm is summarized inTable 1. There, smaller values of the damping parameter β makeGAMP more robust to difficult A at the expense of convergencespeed, and β = 1 recovers the original GAMP algorithm from [13].Note that the only difference between MAP-GAMP and MMSE-GAMP is the definition of the scalar denoisers in (D1)-(D4). Denois-ers of the type in (D3)-(D4) are often referred to “proximal opera-tors” in the optimization literature. In fact, as noted in [19] and [22],max-sum GAMP is closely related to primal-dual algorithms fromconvex optimization, such as the classical Arrow-Hurwicz and re-cent Chambolle-Pock and primal-dual hybrid gradient algorithms[25–27]. The primary difference between MAP-GAMP and those al-gorithms is that the primal and dual stepsizes (i.e., ν rn ( t ) and /ν pi ( t ) in Table 1) are adapted, rather than fixed or scheduled. If we configure GAMP’s transform A and loss function f ( · ) as A = (cid:20) ΦΩ (cid:21) , f i ( · ) = ( l i ( · ) i ∈ { , . . . , M } h i − M ( · ) i ∈ { M +1 , . . . , M + D } (9)here Φ and Ω are the measurement and analysis operators fromSec. 2.1, and l i ( · ) and h d ( · ) are the loss and regularization functionsfrom Sec. 2.1, then MAP-GAMP’s optimization problem (8) coin-cides with the MAP optimization (6), which (as discussed earlier) isa generalization of the analysis-CS problem (3). Likewise, MMSE-GAMP will return an approximation of the MMSE estimate b x MMSE under the statistical model (4)-(5).In the sequel, we refer to GAMP under (9) (with suitable choicesof f q , g n , and h d ) as “Generalized AMP for Analysis CS,” orGrAMPA. Despite the simplicity of this idea and its importance to,e.g., image recovery, it has (to our knowledge) not been proposedbefore, outside of our preprint [28]. One of the strengths of GrAMPA is the freedom to choose the lossfunction l i ( · ) and the regularizations g n ( · ) and h d ( · ) .The quadratic loss l m ( q ) = | y m − q | , as used in (3), is appro-priate for many applications. GrAMPA, however, also supports non-quadratic losses, as needed for 1-bit compressed sensing [29], phaseretrieval [23], and Poisson-based photon-limited imaging [30].The pixel regularization g n ( · ) could be used to enforce knownpositivity in x n (via g n ( x ) = − ln 1 x ≥ ), real-valuedness in x n de-spite complex-valued measurements (via g n ( x ) = − ln 1 x ∈ R ∀ n ),or zero-valuedness in x n (via g n ( x ) = − ln 1 x =0 ∀ n ). Here, weuse A ∈ { , } to denote the indicator of the event A .As for the analysis regularization h d ( · ) , the use of h d ( u ) = λ | u | with MAP-GrAMPA would allow it to tackle the GrLASSO andanisotropic TV problems defined in Sec. 1. With MMSE-GrAMPA,a first instinct might be to use the Bernoulli-Gaussian (BG) priorcommonly used for synthesis CS, i.e., h d ( u ) = − ln (cid:0) (1 − β ) δ ( u )+ β N ( u ; 0 , σ ) (cid:1) , where δ ( · ) is the Dirac delta pdf and the parameters β and σ control sparsity and variance, respectively. But the need totune two parameters is inconvenient, and bias effects from the use offinite σ can degrade performance, especially when D ≫ N .Thus, for the MMSE case, we propose a sparse non-informativeparameter estimator (SNIPE) that can be understood as the MMSEdenoiser for a “spike-and-slab” prior with an infinite-variance slab.In particular, SNIPE computes the MMSE estimate of random vari-able u d from a N (0 , ν qd ) -corrupted observation b q d under the prior p u d ( u ) = β d p ( u/σ ) /σ + (1 − β d ) δ ( u ) , (10)in the limiting case that σ → ∞ . Here, β d ∈ (0 , is the prior prob-ability that u d = 0 and the “slab” pdf p ( u ) is continuous, finite,and non-zero at u = 0 , but otherwise arbitrary . Note that, for fixed σ and β d , the MMSE estimator can be stated as F d ( b q d ; ν qd ) , E { u d | b q d ; ν qd } = R u p u d ( u ) N ( u ; b q d , ν qd ) du R p u d ( u ) N ( u ; b q d , ν qd ) du = R up ( u/σ ) N ( u ; b q d , ν qd ) du R p ( u/σ ) N ( u ; b q d , ν qd ) du + σ − β d β d N (0; b q d , ν qd ) . (11)Since, with any fixed sparsity β d < , the estimator (11) trivializesto F d ( b q d ; ν qd ) = 0 ∀ b q d as σ → ∞ , we scale the sparsity with σ as β d = σ/ (cid:0) σ + p (0) p πν qd exp( ω ) (cid:1) for a tunable parameter ω ∈ R ,in which case it can be shown that F d ( b q d ; ν qd , ω ) σ →∞ = b q d ω − | b q d | /ν qd ) . (12) PSfrag replacements sampling ratio
M/N m ed i an r e c o v e r y N S NR [ d B ] Fig. 1 . Recovery of . -sparse Bernoulli-Gaussian finite-differencesignals from AWGN-corrupted measurements at SNR = 60 dB.
3. NUMERICAL RESULTS
We now provide numerical results that compare GrAMPA withSNIPE denoising to several existing algorithms for cosparse anal-ysis CS. In all cases, recovery performance was quantified using
NSNR , k x k / k b x − x k . Each algorithm was given perfectknowledge of relevant statistical parameters (e.g., noise variance)or in cases were an algorithmic parameter needed to be tuned (e.g.,GrLASSO λ or SNIPE ω ), the NSNR -maximizing value was used.
We first replicate an experiment from the ssAMP paper [15]. Us-ing the demonstration code for [15], we generated signal realiza-tions x ∈ R N that yield BG 1D-finite-difference sequences Ω x with sparsity rate . . Then we attempted to recover those sig-nals from AWGN-corrupted observations y = Φ x + w ∈ R M , atan SNR , k Φ x k / k w k of dB, generated with i.i.d Gaussianmeasurement matrices Φ .Figure 1 shows median NMSE versus sampling ratio
M/N for ssAMP, TV-AMP, and GrAMPA, over problem realiza-tions. There we see GrAMPA uniformly outperforming ssAMP,which uniformly outperforms TV-AMP. We attribute the perfor-mance differences to choice of regularization: GrAMPA’s SNIPEregularization is closer to ℓ than ssAMP’s BG-based regularization,which is closer to ℓ than TV-AMP’s ℓ regularization. We note theperformance of GrAMPA in Fig. 1 is much better than that reportedin [15] due to the misconfiguration of GrAMPA in [15]. We now compare GrAMPA with SNIPE denoising to Greedy Analy-sis Pursuit (GAP) [2] using an experiment from [2] that constructed Ω T ∈ R N × D as a random, almost-uniform, almost-tight frame and x as an exactly L -cosparse vector. The objective was then to recover x from noiseless measurements y = Φ x using analysis operator Ω and i.i.d Gaussian Φ . For this experiment, we used N = 200 .Figure 2 shows the empirical phase-transition curves (PTCs) forGAP and GrAMPA versus sampling ratio δ = M/N and uncer- .1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.90.10.20.30.40.50.60.70.80.9 GrAMPAGAP
PSfrag replacements ρ = ( N − L ) / M D/N = 1 . D/N = 1 . D/N = 2 . δ = M/N
PSfrag replacements ρ = ( N − L ) /MD/N = 1 . D/N = 1 . D/N = 2 . δ = M/N
Fig. 2 . Phase transition curves for recovery of L -cosparse N -lengthsignals from M noiseless measurements under i.i.d Gaussian Φ andan N × D random, almost-uniform, almost-tight frame Ω T . PSfrag replacements m ed i an r e c o v e r y N S NR [ d B ] sampling ratio M/N
Fig. 3 . Recovery of the × Lena image under a measurementSNR of dB, spread-spectrum Φ , and Db1-8 concatenated Ω .tainty ratio ρ = ( N − L ) /M . For points below the PTC, recov-ery was successful with high probability, while for points above thePTC, recovery was unsuccessful with high probability. Here, we de-fined “success” as NSNR ≥ . Figure 2 shows that the PTC ofGrAMPA is uniformly better than that of GAP. It also shows that, forboth algorithms, the PTC approaches the feasibility boundary (i.e., ρ = 1 ) as M/N → but that, as the analysis operator becomes moreovercomplete (i.e., D/N increases), the PTC progressively weakens.
Next, we repeat an experiment from [8], where the N = 512 × Lena image x was recovered from M noisy complex-valued mea-surements y = Φ x + w at SNR = 40 dB. The measurements wereof the “spread spectrum” form: Φ = M F C , where C was diagonalwith random ± entries, F was an N -FFT, and M ∈ { , } M × N contained rows of I N selected uniformly at random. An overcom-plete dictionary Ψ ∈ R N × N was constructed from a horizontalconcatenation of the first Daubechies orthogonal DWT matrices,yielding the analysis operator Ω = Ψ T . The use of highly overcom-plete concatenated dictionaries is dubbed “sparsity averaging” in [8].Figure 3 shows median NSNR (over Monte-Carlo trials) ver-sus sampling ratio
M/N for GrAMPA with SNIPE denoising; for
PSfrag replacements
RW-TV m ed i an r e c o v e r y N S NR [ d B ] Fig. 4 . Recovery of the × Shepp-Logan phantom from 2D FFT Φ and 2D horizontal, vertical, and diagonal finite-difference Ω .“SARA” from [8], which employs iteratively-reweighted- ℓ [7]; andfor GrLASSO implemented via the “SOPT” Matlab code that ac-companies [8], which employs Douglas-Rachford splitting [5]. Allalgorithms enforced non-negativity in the estimate. Figure 3 showsGrAMPA outperforming the other algorithms in NSNR at all sam-pling ratios
M/N . Averaging over trials where all algorithms gaverecovery
NSNR ≥ dB, the runtimes of GrAMPA, GrLASSO,and SARA were , , and seconds, respectively. Finally, we investigated the recovery of the N = 64 × Shepp-Logan Phantom image from 2D Fourier radial-line measurements y = Φ x + w at SNR = 80 dB, using an analysis operator Ω com-posed of horizontal, vertical, diagonal, and anti-diagonal 2D finitedifferences, as described in the noise-tolerant GAP paper [31].Figure 4 plots median recovery NSNR (over 11 Monte-Carlo tri-als) versus number of radial lines for GrAMPA with SNIPE denois-ing, GAPn [31], the “RW-TV” approach from [8], which employsiteratively-weighted- ℓ [7], and GrLASSO, implemented using theDouglas-Rachford based “SOPT” Matlab code from [8]. The fig-ure shows that GrAMPA achieved the best phase transition and alsothe best NSNR (for all numbers of radial lines above ). Averagingover trials where all algorithms gave recovery NSNR ≥ dB, theruntimes of GrAMPA, GrLASSO, RW-TV, and GAP were . , . , . , and . seconds, respectively.
4. CONCLUSIONS
In this work, we proposed the “Generalized AMP for Analysis CS”(GrAMPA) algorithm, a new AMP-based approach to analysis CSthat can be used with a wide range of loss functions, regulariza-tion terms, and analysis operators. In addition, we proposed the“Sparse Non-informative Parameter Estimator” (SNIPE), an ℓ -likesoft thresholder that corresponds to the MMSE denoiser for a spike-and-slab distribution with an infinite-variance slab. Numerical ex-periments comparing GrAMPA with SNIPE to several other recentlyproposed analysis-CS algorithms show improved recovery perfor-mance and excellent runtime. Online tuning of the SNIPE parameter ω will be considered in future work. . REFERENCES [1] Michael Elad, Peyman Milanfar, and Ron Rubinstein, “Anal-ysis versus synthesis in signal priors,” Inverse Problems , vol.23, pp. 947–968, 2007.[2] S. Nam, M. E. Davies, M. Elad, and R. Gribonval, “Thecosparse analysis model and algorithms,”
Appl. ComputationalHarmonic Anal. , vol. 34, no. 1, pp. 30–56, Jan. 2013.[3] L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variationbased noise removal algorithms,”
Physica D , vol. 60, pp. 259–268, 1992.[4] Ryan J. Tibshirani, “Solution path of the generalized lasso,”
Ann. Statist. , vol. 39, no. 3, pp. 1335–1371, 2011.[5] P. L. Combettes and J.-C. Pesquet, “A Douglas-Rachford split-ting approach to nonsmooth convex variational signal recov-ery,”
IEEE J. Sel. Topics Signal Process. , vol. 1, no. 4, pp.6564–574, Dec. 2007.[6] S. Becker, J. Bobin, and E. J. Cand`es, “NESTA: A fast andaccurate first-order method for sparse recovery,”
SIAM J. Imag.Sci. , vol. 4, no. 1, pp. 1–39, 2011.[7] E. J. Cand`es, M. B. Wakin, and S. Boyd, “Enhancing sparsityby reweighted ℓ minimization,” J. Fourier Anal. App. , vol. 14,no. 5, pp. 877–905, Dec. 2008.[8] Rafael E. Carrillo, Jason D. McEwen, Dimitri Van De Ville,Jean-Philippe Thiran, and Yves Wiaux, “Sparsity averagingfor compressive imaging,”
IEEE Signal Process. Lett. , vol. 20,no. 6, pp. 591–594, 2013.[9] R. Chartrand, E. Y. Sidky, and X. Pan, “Nonconvex compres-sive sensing for X-ray CT: An algorithm comparison,” in
Proc.Asilomar Conf. Signals Syst. Comput. , Pacific Grove, CA, Nov.2013, pp. 665–669.[10] R. Giryes, S. Nam, M. Elad, R. Gribonval, and M. E. Davies,“Greedy-like algorithms for the cosparse analysis model,” arXiv:1207.2456 , July 2012.[11] D. L. Donoho, A. Maleki, and A. Montanari, “Message passingalgorithms for compressed sensing,”
Proc. Nat. Acad. Sci. , vol.106, no. 45, pp. 18914–18919, Nov. 2009.[12] D. L. Donoho, A. Maleki, and A. Montanari, “Message pass-ing algorithms for compressed sensing: I. Motivation and con-struction,” in
Proc. Inform. Theory Workshop , Cairo, Egypt,Jan. 2010, pp. 1–5.[13] S. Rangan, “Generalized approximate message passing for es-timation with random linear mixing,” in
Proc. IEEE Int. Symp.Inform. Thy. , Saint Petersburg, Russia, Aug. 2011, pp. 2168–2172, (full version at arXiv:1010.5141 ).[14] D. L. Donoho, I. M. Johnstone, and A. Montanari, “Accu-rate prediction of phase transitions in compressed sensing viaa connection to minimax denoising,”
IEEE Trans. Inform. The-ory , vol. 59, no. 6, June 2013.[15] Jaewook Kang, Hyoyoung Jung, Heung-No Lee, andKiseon Kim, “Spike-and-slab approximate message-passing for high-dimensional piecewise-constant re-covery,” arXiv:1406.4311 , Aug. 2014, (Matlabcodes at https://sites.google.com/site/jwkang10/ssAMP_algorithm_ver1_Aug2014.zip ).[16] J. S. Turek, I. Yavneh, and M. Elad, “On MAP and MMSEestimators for the co-sparse analysis model,”
Digital SignalProcess. , vol. 28, pp. 57–74, 2014. [17] Adel Javanmard and Andrea Montanari, “State evolution forgeneral approximate message passing algorithms, with appli-cations to spatial coupling,”
Inform. Inference , vol. 2, no. 2,pp. 115–144, 2013.[18] M. Bayati, M. Lelarge, and A. Montanari, “Universality inpolytope phase transitions and iterative algorithms,” in
Proc.IEEE Int. Symp. Inform. Thy. , Boston, MA, June 2012, pp.1643–1647, (full paper at arXiv:1207.7321 ).[19] S. Rangan, P. Schniter, E. Riegler, A. Fletcher, and V. Cevher,“Fixed points of generalized approximate message passingwith arbitrary matrices,” in
Proc. IEEE Int. Symp. Inform.Thy. , Istanbul, Turkey, July 2013, pp. 664–668, (full versionat arXiv:1301.6295 ).[20] F. Krzakala, A. Manoel, E. W. Tramel, and L. Zdeborov´a,“Variational free energies for compressed sensing,” in
Proc.IEEE Int. Symp. Inform. Thy. , Honolulu, HI, July 2014, pp.1499–1503, (see also arXiv:1402.1384 ).[21] F. Caltagirone, F. Krzakala, and L. Zdeborov´a, “On conver-gence of approximate message passing,” in
Proc. IEEE Int.Symp. Inform. Thy. , Honolulu, HI, July 2014, pp. 1812–1816,(see also arXiv:1401.6384 ).[22] S. Rangan, P. Schniter, and A. Fletcher, “On the convergenceof generalized approximate message passing with arbitrary ma-trices,” in
Proc. IEEE Int. Symp. Inform. Thy. , Honolulu, HI,July 2014, pp. 236–240, (full version at arXiv:1402.3210 ).[23] P. Schniter and S. Rangan, “Compressive phase retrieval viageneralized approximate message passing,” in
Proc. AllertonConf. Commun. Control Comput. , Monticello, IL, Oct. 2012,pp. 815–822, (full version at arXiv:1405.5618 ).[24] Jeremy Vila, Philip Schniter, Sundeep Rangan, Florent Krza-kala, and Lenka Zdeborov´a, “Adaptive damping and mean re-moval for the generalized approximate message passing algo-rithm,” in
IEEE ICASSP , 2015, submitted.[25] E. Esser, X. Zhang, and T. F. Chan, “A general framework for aclass of first order primal-dual algorithms for convex optimiza-tion in imaging science,”
SIAM J. Imaging Sci. , vol. 3, no. 4,pp. 1015–1046, 2010.[26] A. Chambolle and T. Pock, “A first-order primal-dual algo-rithm for convex problems with applications to imaging,”
J.Math. Imaging Vis. , vol. 40, pp. 120–145, 2011.[27] B. He and X. Yuan, “Convergence analysis of primal-dual al-gorithms for a saddle-point problem: From contraction per-spective,”
SIAM J. Imaging Sci. , vol. 5, no. 1, pp. 119–149,2012.[28] M. Borgerding and P. Schniter, “Generalized ap-proximate message passing for the cosparse analysismodel,” arXiv:1312.3968 , 2013, (Matlab codes at ).[29] U. S. Kamilov, V. K. Goyal, and S. Rangan, “Message-passing de-quantization with applications to compressed sens-ing,”
IEEE Trans. Signal Process. , vol. 60, no. 12, pp. 6270–6281, Dec. 2012.[30] D. L. Snyder, A. M. Hammond, and R. L. White, “Image re-covery from data acquired with a charge-coupled-device cam-era,”
J. Am. Statist. Assoc. , vol. 10, pp. 1014–1023, 1993.[31] S. Nam, M. E. Davies, M. Elad, and R. Gribonval, “Recoveryof cosparse signals with greedy analysis pursuit in the presenceof noise,” in