Kiryung Lee

ADMiRA: Atomic Decomposition for Minimum Rank Approximation

In this paper, we address compressed sensing of a low-rank matrix posing the inverse problem as an approximation problem with a specified target rank of the solution. A simple search over the target rank then provides the minimum rank solution satisfying a prescribed data approximation bound. We propose an atomic decomposition providing an analogy between parsimonious representations of a sparse vector and a low-rank matrix and extending efficient greedy algorithms from the vector to the matrix case. In particular, we propose an efficient and guaranteed algorithm named atomic decomposition for minimum rank approximation (ADMiRA) that extends Needell and Tropps compressive sampling matching pursuit (CoSaMP) algorithm from the sparse vector to the low-rank matrix case. The performance guarantee is given in terms of the rank-restricted isometry property (R-RIP) and bounds both the number of iterations and the error in the approximate solution for the general case of noisy measurements and approximately low-rank solution. With a sparse measurement operator as in the matrix completion problem, the computation in ADMiRA is linear in the number of measurements. Numerical experiments for the matrix completion problem show that, although the R-RIP is not satisfied in this case, ADMiRA is a competitive algorithm for matrix completion.

Subspace Methods for Joint Sparse Recovery

We propose robust and efficient algorithms for the joint sparse recovery problem in compressed sensing, which simultaneously recover the supports of jointly sparse signals from their multiple measurement vectors obtained through a common sensing matrix. In a favorable situation, the unknown matrix, which consists of the jointly sparse signals, has linearly independent nonzero rows. In this case, the MUltiple SIgnal Classification (MUSIC) algorithm, originally proposed by Schmidt for the direction of arrival estimation problem in sensor array processing and later proposed and analyzed for joint sparse recovery by Feng and Bresler, provides a guarantee with the minimum number of measurements. We focus instead on the unfavorable but practically significant case of rank defect or ill-conditioning. This situation arises with a limited number of measurement vectors, or with highly correlated signal components. In this case, MUSIC fails and, in practice, none of the existing methods can consistently approach the fundamental limit. We propose subspace-augmented MUSIC (SA-MUSIC), which improves on MUSIC such that the support is reliably recovered under such unfavorable conditions. Combined with a subspace-based greedy algorithm, known as Orthogonal Subspace Matching Pursuit, which is also proposed and analyzed in this paper, SA-MUSIC provides a computationally efficient algorithm with a performance guarantee. The performance guarantees are given in terms of a version of the restricted isometry property. In particular, we also present a non-asymptotic perturbation analysis of the signal subspace estimation step, which has been missing in the previous studies of MUSIC.

An asymmetric watermarking system with many embedding watermarks corresponding to one detection watermark

In this letter, we propose a new asymmetric watermarking system which can accommodate many embedding watermarks but needs only one reference watermark for detection. Such a system is useful in averting attacks that seek to estimate the embedding watermark. In the proposed system, the phase of the reference watermark is shifted randomly (clockwise or counterclockwise) in the discrete Fourier transform domain to make embedding watermarks. They are correlated with one another and have the same correlation with the reference one. We also address how to select the design parameters.

EM estimation of scale factor for quantization-based audio watermarking

The blind watermarking scheme extracts the embedded message without access to the host signal. Recently, efficient blind watermarking schemes, which exploit the knowledge of the host signal at the encoder, are proposed [1,2,3]. Scalar quantizers are employed for practical implementation. Even though the scalar quantizer can provide simple encoding and decoding schemes, if the watermarked signal is scaled, then the quantizer step size at the decoder should be scaled accordingly for a reliable decoding. In this paper, we propose a preprocessed decoding scheme, which uses an estimated scale factor. The received signal density is approximated by a Gaussian mixture model, and the scale factor is then estimated by employing the expectation maximization algorithm [6]. In the proposed scheme, the scale factor is estimated from the received signal itself without any additional pilot signal. Numerical results show that the proposed scheme provides a reliable decoding from the scaled signal.

Regression-based prediction for blocking artifact reduction in JPEG-compressed images

In order to reduce the blocking artifact in the Joint Photographic Experts Group (JPEG)-compressed images, a new noniterative postprocessing algorithm is proposed. The algorithm consists of a two-step operation: low-pass filtering and then predicting. Predicting the original image from the low-pass filtered image is performed by using the predictors, which are constructed based on a broken line regression model. The constructed predictor is a generalized version of the projector onto the quantization constraint set , , or the narrow quantization constraint set . We employed different predictors depending on the frequency components in the discrete cosine transform (DCT) domain since each component has different statistical properties. Further, by using a simple classifier, we adaptively applied the predictors depending on the local variance of the DCT block. This adaptation enables an appropriate blurring depending on the smooth or detail region, and shows improved performance in terms of the average distortion and the perceptual view. For the major-edge DCT blocks, which usually suffer from the ringing artifact, the quality of fit to the regression model is usually not good. By making a modification of the regression model for such DCT blocks, we can also obtain a good perceptual view. The proposed algorithm does not employ any sophisticated edge-oriented classifiers and nonlinear filters. Compared to the previously proposed algorithms, the proposed algorithm provides comparable or better results with less computational complexity.

Blind Recovery of Sparse Signals From Subsampled Convolution

Subsampled blind deconvolution is the recovery of two unknown signals from samples of their convolution. To overcome the ill-posedness of this problem, solutions based on priors tailored to specific practical application have been developed. In particular, sparsity models have provided promising priors. However, in spite of the empirical success of these methods in many applications, existing analyses are rather limited in two main ways: by disparity between the theoretical assumptions on the signal and/or measurement model versus practical setups; or by failure to provide a performance guarantee for parameter values within the optimal regime defined by the information theoretic limits. In particular, it has been shown that a naive sparsity model is not a strong enough prior for identifiability in the blind deconvolution problem. Instead, in addition to sparsity, we adopt a conic constraint, which enforces spectral flatness of the signals. Under this prior together with random dictionary models, we show that the unknown sparse signals can be recovered from samples of their convolution at a rate scaling near optimally with the problem parameters. We also propose an iterative algorithm that is guaranteed to provide robust recovery at the same near optimal sample complexity provided that certain projection steps in the algorithm are successful. In our analysis, we have not verified the success of these projection steps, but these steps are inactive with high probability. Numerical results show the empirical performance of the iterative algorithm agrees with the performance guarantee.

Near-Optimal Compressed Sensing of a Class of Sparse Low-Rank Matrices Via Sparse Power Factorization

Compressed sensing of simultaneously sparse and low-rank matrices enables recovery of sparse signals from a few linear measurements of their bilinear form. One important question is how many measurements are needed for a stable reconstruction in the presence of measurement noise. Unlike conventional compressed sensing for sparse vectors, where convex relaxation via the

Identifiability in Blind Deconvolution With Subspace or Sparsity Constraints

\ell _{1}

Subspace-augmented MUSIC for joint sparse recovery with any rank

-norm achieves near-optimal performance, for compressed sensing of sparse low-rank matrices, it has been shown recently that convex programmings using the nuclear norm and the mixed norm are highly suboptimal even in the noise-free scenario. We propose an alternating minimization algorithm called sparse power factorization (SPF) for compressed sensing of sparse rank-one matrices. For a class of signals whose sparse representation coefficients are fast-decaying, SPF achieves stable recovery of the rank-one matrix formed by their outer product and requires number of measurements within a logarithmic factor of the information-theoretic fundamental limit. For the recovery of general sparse low-rank matrices, we propose subspace-concatenated SPF (SCSPF), which has analogous near-optimal performance guarantees to SPF in the rank-one case. Numerical results show that SPF and SCSPF empirically outperform convex programmings using the best known combinations of mixed norm and nuclear norm.

Efficient and guaranteed rank minimization by atomic decomposition

Blind deconvolution (BD), the resolution of a signal and a filter given their convolution, arises in many applications. Without further constraints, BD is ill-posed. In practice, subspace or sparsity constraints have been imposed to reduce the search space, and have shown some empirical success. However, the existing theoretical analysis on uniqueness in BD is rather limited. In an effort to address the still open question, we derive sufficient conditions under which two vectors can be uniquely identified from their circular convolution, subject to subspace or sparsity constraints. These sufficient conditions provide the first algebraic sample complexities for BD. We first derive a sufficient condition that applies to almost all bases or frames. For BD of vectors in ℂn, with two subspace constraints of dimensions m1 and m2, the required sample complexity is n ≥ m1m2. Then, we impose a sub-band structure on one basis, and derive a sufficient condition that involves a relaxed sample complexity n≥ m1+m2-1, which we show to be optimal. We present the extensions of these results to BD with sparsity constraints or mixed constraints, with the sparsity level replacing the subspace dimension. The cost for the unknown support in this case is an extra factor of 2 in the sample complexity.