Dmitry M. Malioutov

A sparse signal reconstruction perspective for source localization with sensor arrays

We present a source localization method based on a sparse representation of sensor measurements with an overcomplete basis composed of samples from the array manifold. We enforce sparsity by imposing penalties based on the /spl lscr//sub 1/-norm. A number of recent theoretical results on sparsifying properties of /spl lscr//sub 1/ penalties justify this choice. Explicitly enforcing the sparsity of the representation is motivated by a desire to obtain a sharp estimate of the spatial spectrum that exhibits super-resolution. We propose to use the singular value decomposition (SVD) of the data matrix to summarize multiple time or frequency samples. Our formulation leads to an optimization problem, which we solve efficiently in a second-order cone (SOC) programming framework by an interior point implementation. We propose a grid refinement method to mitigate the effects of limiting estimates to a grid of spatial locations and introduce an automatic selection criterion for the regularization parameter involved in our approach. We demonstrate the effectiveness of the method on simulated data by plots of spatial spectra and by comparing the estimator variance to the Crame/spl acute/r-Rao bound (CRB). We observe that our approach has a number of advantages over other source localization techniques, including increased resolution, improved robustness to noise, limitations in data quantity, and correlation of the sources, as well as not requiring an accurate initialization.

Homotopy continuation for sparse signal representation

We explore the application of a homotopy continuation-based method for sparse signal representation in overcomplete dictionaries. Our problem setup is based on the basis pursuit framework, which involves a convex optimization problem consisting of terms enforcing data fidelity and sparsity, balanced by a regularization parameter. Choosing a good regularization parameter in this framework is a challenging task. We describe a homotopy continuation-based algorithm to find and trace efficiently all solutions of basis pursuit as a function of the regularization parameter. In addition to providing an attractive alternative to existing optimization methods for solving the basis pursuit problem, this algorithm can also be used to provide an automatic choice for the regularization parameter, based on prior information about the desired number of non-zero components in the sparse representation. Our numerical examples demonstrate the effectiveness of this algorithm in accurately and efficiently generating entire solution paths for basis pursuit, as well as producing reasonable regularization parameter choices. Furthermore, exploring the resulting solution paths in various operating conditions reveals insights about the nature of basis pursuit solutions.

Optimal sparse representations in general overcomplete bases

We consider the problem of enforcing a sparsity prior in underdetermined linear problems, which is also known as sparse signal representation in overcomplete bases. The problem is combinatorial in nature, and a direct approach is computationally intractable, even for moderate data sizes. A number of approximations have been considered in the literature, including stepwise regression, matching pursuit and its variants, and, recently, basis pursuit (/spl lscr//sub 1/) and also /spl lscr//sub p/-norm relaxations with p<1. Although the exact notion of sparsity (expressed by an /spl lscr//sub 0/-norm) is replaced by /spl lscr//sub 1/ and /spl lscr//sub p/ norms in the latter two, it can be shown that under some conditions these relaxations solve the original problem exactly. The seminal paper of D.L. Donoho and X. Huo (see Stanford Univ. Tech. report: http://www-sccm.stanford.edu/pub/sccm/sccm02-17.pdf) establishes this fact for /spl lscr//sub 1/ (basis pursuit) for a special case where the linear operator is composed of an orthogonal pair. We extend their results to a general underdetermined linear operator. Furthermore, we derive conditions for the equivalence of /spl lscr//sub 0/ and /spl lscr//sub p/ problems, and extend the results to the problem of enforcing sparsity with respect to a transformation (which includes total variation priors as a special case). Finally, we describe an interesting result relating the sign patterns of solutions to the question of /spl lscr//sub 1/-/spl lscr//sub 0/ equivalence.

Compressed sensing with sequential observations

Compressed sensing allows perfect recovery of sparse signals (or signals sparse in some basis) using only a small number of measurements. The results in the literature have focused on the asymptotics of how many samples are required and the probability of making an error for & fixed batch of samples. We investigate an alternative scenario where observations are available in sequence and can be stopped as soon as there is reasonable certainty of correct reconstruction. This approach does not require knowing how sparse is the signal, and allows reconstruction using the smallest number of samples. Central to our sequential approach is the stopping rule. For the random Gaussian ensemble we show that a simple stopping rule gives the absolute minimum number of observations required for exact recovery, with probability one. However, for other ensembles like Bernoulli or Fourier, this is no longer true, and the rule is modified to trade off delay in stopping and probability of error. We also consider near-sparse signals and describe how to estimate the reconstruction error from the sequence of solutions. This enables stopping once the error falls below a desired tolerance. Our sequential approach to compressed sensing involves a sequence of linear programs, and we outline how such a sequence can be solved efficiently.

A variational technique for source localization based on a sparse signal reconstruction perspective

We propose a novel non-parametric technique for source localization with passive sensor arrays. Our approach involves formulation of the problem in a variational framework where regularizing sparsity constraints are incorporated to achieve super-resolution and noise suppression. Compared to various source localization schemes, our approach offers increased resolution, significantly reduced sidelobes, and improved robustness to limitations in data quality and quantity. We demonstrate the effectiveness of the method on simulated data.

Boolean compressed sensing: LP relaxation for group testing

We revisit the well-known problem of boolean group testing which attempts to discover a sparse subset of faulty items in a large set of mostly good items using a small number of pooled (or grouped) tests. This problem originated during the second WorldWar, and has been the subject of active research during the 70s, and 80s. Recently, there has been a resurgence of interest due to the striking parallels between group testing and the now highly popular field of compressed sensing. In fact, boolean group testing is nothing but compressed sensing in a different algebra - with boolean `AND and `OR operations replacing vector space multiplication and addition. In this paper we review existing solutions for non-adaptive (batch) group testing and propose a linear programming relaxation solution, which has a resemblance to the basis pursuit algorithm for sparse recovery in linear models. We compare its performance to alternative methods for group testing.

Low-Rank Variance Approximation in GMRF Models: Single and Multiscale Approaches

We present a versatile framework for tractable computation of approximate variances in large-scale Gaussian Markov random field estimation problems. In addition to its efficiency and simplicity, it also provides accuracy guarantees. Our approach relies on the construction of a certain low-rank aliasing matrix with respect to the Markov graph of the model. We first construct this matrix for single-scale models with short-range correlations and then introduce spliced wavelets and propose a construction for the long-range correlation case, and also for multiscale models. We describe the accuracy guarantees that the approach provides and apply the method to a large interpolation problem from oceanography with sparse, irregular, and noisy measurements, and to a gravity inversion problem.

Low-Rank Variance Estimation in Large-Scale Gmrf Models

We consider the problem of variance estimation in large-scale Gauss-Markov random field (GMRF) models. While approximate mean estimates can be obtained efficiently for sparse GMRFs of very large size, computing the variances is a challenging problem. We propose a simple rank-reduced method which exploits the graph structure and the correlation length in the model to compute approximate variances with linear complexity in the number of nodes. The method has a separation length parameter trading off complexity versus estimation accuracy. For models with bounded correlation length, we efficiently compute provably accurate variance estimates

Hybrid Distributed Video Coding Using SCA Codes

We describe the architecture for our distributed video coding (DVC) system. Some key differences between our work and previous systems include a new method of enabling decoder motion compensation, and the use of serially concatenated accumulate syndrome codes for distributed source coding. To evaluate performance, we compare our system to the H.263+ and H.264/AVC video codecs. Experiments show that our system is comparable to DVC systems from Stanford and Berkeley in the sense that our system performs better than H.263+Intra, but worse than H.263+Inter and H.264/AVC

Superresolution source localization through data-adaptive regularization

We address the task of source localization using a novel non-parametric data-adaptive approach based on regularized linear inverse problems with sparsity constraints. The class of penalty functions that we use for regularization favors sparsity of the reconstructions, thus producing superb resolution of the sources. We present a computationally efficient technique to carry out the numerical optimization of the resulting cost function. In comparison to conventional source localization methods, the proposed approach provides numerous improvements, including increased resolution, reduced sidelobes, and better robustness properties to noise, limited snapshots, and coherence of the sources. The method is developed for the general source localization scenario, encompassing nearfield and farfield, narrowband and broadband, and non-linear array geometry cases. Simulation results manifest the capabilities of the approach.