Yoram Bresler

Exact maximum likelihood parameter estimation of superimposed exponential signals in noise

A unified framework for the exact maximum likelihood estimation of the parameters of superimposed exponential signals in noise, encompassing both the time series and the array problems, is presented. An exact expression for the ML criterion is derived in terms of the linear prediction polynomial of the signal, and an iterative algorithm for the maximization of this criterion is presented. The algorithm is equally applicable in the case of signal coherence in the array problem. Simulation shows the estimator to be capable of providing more accurate frequency estimates than currently existing techniques. The algorithm is similar to those independently derived by Kumaresan et al. In addition to its practical value, the present formulation is used to interpret previous methods such as Pronys, Pisarenkos, and modifications thereof.

MR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning

Compressed sensing (CS) utilizes the sparsity of magnetic resonance (MR) images to enable accurate reconstruction from undersampled k-space data. Recent CS methods have employed analytical sparsifying transforms such as wavelets, curvelets, and finite differences. In this paper, we propose a novel framework for adaptively learning the sparsifying transform (dictionary), and reconstructing the image simultaneously from highly undersampled k-space data. The sparsity in this framework is enforced on overlapping image patches emphasizing local structure. Moreover, the dictionary is adapted to the particular image instance thereby favoring better sparsities and consequently much higher undersampling rates. The proposed alternating reconstruction algorithm learns the sparsifying dictionary, and uses it to remove aliasing and noise in one step, and subsequently restores and fills-in the k-space data in the other step. Numerical experiments are conducted on MR images and on real MR data of several anatomies with a variety of sampling schemes. The results demonstrate dramatic improvements on the order of 4-18 dB in reconstruction error and doubling of the acceptable undersampling factor using the proposed adaptive dictionary as compared to previous CS methods. These improvements persist over a wide range of practical data signal-to-noise ratios, without any parameter tuning.

Spectrum-blind minimum-rate sampling and reconstruction of multiband signals

We propose a universal sampling pattern and corresponding reconstruction algorithms that guarantee well-conditioned reconstruction of all multiband signals with a given spectral occupancy bound without prior knowledge of the spectral support. It is shown that such a universal sampling pattern can asymptotically achieve the Nyquist-Landau (1957) minimal sampling rate. Also, the new design method replaces the nonaliasing or packability criterion for a reconstructive sampling pattern with a more lenient criterion, allowing reconstruction of signals aliased by sampling.

Globally convergent edge-preserving regularized reconstruction: an application to limited-angle tomography

We introduce a generalization of a deterministic relaxation algorithm for edge-preserving regularization in linear inverse problems. This algorithm transforms the original (possibly nonconvex) optimization problem into a sequence of quadratic optimization problems, and has been shown to converge under certain conditions when the original cost functional being minimized is strictly convex. We prove that our more general algorithm is globally convergent (i.e., converges to a local minimum from any initialization) under less restrictive conditions, even when the original cost functional is nonconvex. We apply this algorithm to tomographic reconstruction from limited-angle data by formulating the problem as one of regularized least-squares optimization. The results demonstrate that the constraint of piecewise smoothness, applied through the use of edge-preserving regularization, can provide excellent limited-angle tomographic reconstructions. Two edge-preserving regularizers-one convex, the other nonconvex-are used in numerous simulations to demonstrate the effectiveness of the algorithm under various limited-angle scenarios, and to explore how factors, such as the choice of error norm, angular sampling rate and amount of noise, affect the reconstruction quality and algorithm performance. These simulation results show that for this application, the nonconvex regularizer produces consistently superior results.

ADMiRA: Atomic Decomposition for Minimum Rank Approximation

In this paper, we address compressed sensing of a low-rank matrix posing the inverse problem as an approximation problem with a specified target rank of the solution. A simple search over the target rank then provides the minimum rank solution satisfying a prescribed data approximation bound. We propose an atomic decomposition providing an analogy between parsimonious representations of a sparse vector and a low-rank matrix and extending efficient greedy algorithms from the vector to the matrix case. In particular, we propose an efficient and guaranteed algorithm named atomic decomposition for minimum rank approximation (ADMiRA) that extends Needell and Tropps compressive sampling matching pursuit (CoSaMP) algorithm from the sparse vector to the low-rank matrix case. The performance guarantee is given in terms of the rank-restricted isometry property (R-RIP) and bounds both the number of iterations and the error in the approximate solution for the general case of noisy measurements and approximately low-rank solution. With a sparse measurement operator as in the matrix completion problem, the computation in ADMiRA is linear in the number of measurements. Numerical experiments for the matrix completion problem show that, although the R-RIP is not satisfied in this case, ADMiRA is a competitive algorithm for matrix completion.

O(N/sup 2/log/sub 2/N) filtered backprojection reconstruction algorithm for tomography

We present a new fast reconstruction algorithm for parallel beam tomography. The new algorithm is an accelerated version of the standard filtered backprojection (FBP) reconstruction, and uses a hierarchical decomposition of the backprojection operation to reduce the computational cost from O(N(3)) to O(N(2)log(2 )N). We discuss the choice of the various parameters that affect the behavior of the algorithm, and present numerical studies that demonstrate the cost versus distortion tradeoff. Comparisons with Fourier reconstruction algorithms and a multilevel inversion algorithm by Brandt et al., both of which also have O(N(2)log(2)N) cost, suggest that the proposed hierarchical scheme has a superior cost versus distortion performance. It offers RMS reconstruction errors comparable to the FBP with considerable speedup. For an example with a 512 x 512-pixel image and 1024 views, the speedup achieved with a particular implementation is over 40 fold, with reconstructions visually indistinguishable from the FBP.

Learning Sparsifying Transforms

The sparsity of signals and images in a certain transform domain or dictionary has been exploited in many applications in signal and image processing. Analytical sparsifying transforms such as Wavelets and DCT have been widely used in compression standards. Recently, synthesis sparsifying dictionaries that are directly adapted to the data have become popular especially in applications such as image denoising, inpainting, and medical image reconstruction. While there has been extensive research on learning synthesis dictionaries and some recent work on learning analysis dictionaries, the idea of learning sparsifying transforms has received no attention. In this work, we propose novel problem formulations for learning sparsifying transforms from data. The proposed alternating minimization algorithms give rise to well-conditioned square transforms. We show the superiority of our approach over analytical sparsifying transforms such as the DCT for signal and image representation. We also show promising performance in signal denoising using the learnt sparsifying transforms. The proposed approach is much faster than previous approaches involving learnt synthesis, or analysis dictionaries.

Subspace Methods for Joint Sparse Recovery

We propose robust and efficient algorithms for the joint sparse recovery problem in compressed sensing, which simultaneously recover the supports of jointly sparse signals from their multiple measurement vectors obtained through a common sensing matrix. In a favorable situation, the unknown matrix, which consists of the jointly sparse signals, has linearly independent nonzero rows. In this case, the MUltiple SIgnal Classification (MUSIC) algorithm, originally proposed by Schmidt for the direction of arrival estimation problem in sensor array processing and later proposed and analyzed for joint sparse recovery by Feng and Bresler, provides a guarantee with the minimum number of measurements. We focus instead on the unfavorable but practically significant case of rank defect or ill-conditioning. This situation arises with a limited number of measurement vectors, or with highly correlated signal components. In this case, MUSIC fails and, in practice, none of the existing methods can consistently approach the fundamental limit. We propose subspace-augmented MUSIC (SA-MUSIC), which improves on MUSIC such that the support is reliably recovered under such unfavorable conditions. Combined with a subspace-based greedy algorithm, known as Orthogonal Subspace Matching Pursuit, which is also proposed and analyzed in this paper, SA-MUSIC provides a computationally efficient algorithm with a performance guarantee. The performance guarantees are given in terms of a version of the restricted isometry property. In particular, we also present a non-asymptotic perturbation analysis of the signal subspace estimation step, which has been missing in the previous studies of MUSIC.

Optimal sub-Nyquist nonuniform sampling and reconstruction for multiband signals

We study the problem of optimal sub-Nyquist sampling for perfect reconstruction of multiband signals. The signals are assumed to have a known spectral support /spl Fscr/ that does not tile under translation. Such signals admit perfect reconstruction from periodic nonuniform sampling at rates approaching Landaus (1967) lower bound equal to the measure of /spl Fscr/. For signals with sparse /spl Fscr/, this rate can be much smaller than the Nyquist rate. Unfortunately the reduced sampling rates afforded by this scheme can be accompanied by increased error sensitivity. In a previous study, we derived bounds on the error due to mismodeling and sample additive noise. Adopting these bounds as performance measures, we consider the problems of optimizing the reconstruction sections of the system, choosing the optimal base sampling rate, and designing the nonuniform sampling pattern. We find that optimizing these parameters can improve system performance significantly. Furthermore, uniform sampling is optimal for signals with /spl Fscr/ that tiles under translation. For signals with nontiling /spl Fscr/, which are not amenable to efficient uniform sampling, the results reveal increased error sensitivities with sub-Nyquist sampling. However, these can be controlled by optimal design, demonstrating the potential for practical multifold reductions in sampling rate.

On the number of signals resolvable by a uniform linear array

An algebraic limitation on the maximum number of directions of arrival of plane waves that can be resolved by a uniform linear sensor array is studied. Achievable lower and upper bounds are derived on that number as a function of the number of elements in the array, number of snapshots, and the rank of the source sample-correlation matrix. The signals are assumed narrow-band and of identical and known center frequency. The results are also applicable in the coherent signal case and when directions of arrival are estimated from few snapshots. While in the multiple snapshot case the lower bounds coincide with known asymptotic results, the upper bounds indicate the potential for resolving more signals than by present methods of array processing.