Shirin Jalali

Universal compressed sensing

In this paper, the problem of developing universal algorithms for noiseless compressed sensing of stochastic processes is studied. First, Rényis notion of information dimension (ID) is generalized to analog stationary processes. This provides a measure of complexity for such processes and is connected to the number of measurements required for their accurate recovery. Then the so-called Lagrangian minimum entropy pursuit (Lagrangian-MEP) algorithm, originally proposed by Baron et al. as a heuristic universal recovery algorithm, is studied. It is shown that, if the normalized number of randomized measurements is larger than the ID of the source process, for the right set of parameters, asymptotically, the Lagrangian-MEP algorithm recovers any stationary process satisfying some mixing constraints almost losslessly, without having any prior information about the source distribution.

Universal Compressed Sensing for Almost Lossless Recovery

In this paper, the problem of developing universal algorithms for noiseless compressed sensing of stochastic processes is studied. First, Rényi’s notion of information dimension (ID) is generalized to continuous-valued discrete-time stationary processes. This provides a measure of complexity for such processes and is connected to the rate of measurement (sampling rate) required for their accurate recovery. Then, based on Occam’s razor, a minimum entropy pursuit (MEP) optimization approach for universal compressed sensing is proposed. It is proven that, for any stationary process satisfying certain mixing conditions, if the sampling rate is larger than the ID of the source process, MEP optimization can reliably recover the source vector almost losslessly, without having any prior information about its distribution. Then, a Lagrangian-type relaxation of MEP optimization problem, referred to as Lagrangian-MEP, is studied. It is shown that Lagrangian-MEP is identical to an implementable algorithm proposed by Baron and coauthors, and for the right choice of parameters, has the same asymptotic performance as MEP optimization. Finally, it is proven that Lagrangian-MEP is robust to small measurement noise.

Compressed sensing of compressible signals

A novel low-complexity robust-to-noise iterative algorithm named compression-based gradient descent (C-GD) algorithm is proposed. C-GD is a generic compressed sensing recovery algorithm, that at its core, employs compression codes, such as JPEG2000 and MPEG4. Through using compression codes, C-GD strongly generalizes the scope of structures used by compressed sensing recovery algorithms beyond sparsity or low-rankness. The squared error of the proposed method and its associated convergence is characterized and predicts the strong performance of C-GD. Numerical results suggest that C-GD, when combined with state-of-the-art compression codes, either outperforms or performs comparably to modern compressed sensing recovery methods.

Using compression codes in compressed sensing

Data compression and compressed sensing algorithms exploit the structure present in a signal for its efficient representation and measurement, respectively. While most state-of-the-art data compression codes take advantage of complex patterns present in signals of interest, this is not the case in compressed sensing. This paper explores usage of efficient data compression codes in building compressed sensing recovery methods for stochastic processes. It is proved that for an i.i.d. process, compression-based compressed sensing achieves the fundamental limits in terms of the number of measurements. It is also proved that compressed sensing recovery methods built based on a family of universal compression codes yield a family of universal compressed sensing schemes.

Rate-distortion dimension of stochastic processes

The rate-distortion dimension (RDD) of an analog stationary process is studied as a measure of complexity that captures the amount of information contained in the process. It is shown that the RDD of a process, defined as two times the asymptotic ratio of its rate-distortion function R(D) to log 1/D as the distortion D approaches zero, is equal to its information dimension (ID). This generalizes an earlier result by Kawabata and Dembo and provides an operational approach to evaluate the ID of a process, which previously was shown to be closely related to the effective dimension of the underlying process and also to the fundamental limits of compressed sensing. The relation between RDD and ID is illustrated for a piecewise constant process.

Minimum entropy pursuit: Noise analysis

Universal compressed sensing algorithms recover a “structured” signal from its under-sampled linear measurements, without knowing its distribution. The recently developed minimum entropy pursuit (MEP) optimization suggests a framework for developing universal compressed sensing algorithms. In the noiseless setting, among all signals that satisfy the measurement constraints, MEP seeks the “simplest”. In this work, the effect of noise on the performance of the relaxed version of MEP optimization, namely Lagrangian-MEP, is studied. It is proved that the performance the Lagrangian-MEP algorithm is robust to small additive noise.