M. Salman Asif

Dynamic Updating for

The theory of compressive sensing (CS) has shown us that under certain conditions, a sparse signal can be recovered from a small number of linear incoherent measurements. An effective class of reconstruction algorithms involve solving a convex optimization program that balances the l1 norm of the solution against a data fidelity term. Tremendous progress has been made in recent years on algorithms for solving these l1 minimization programs. These algorithms, however, are for the most part static: they focus on finding the solution for a fixed set of measurements. In this paper, we present a suite of dynamic algorithms for solving l1 minimization programs for streaming sets of measurements. We consider cases where the underlying signal changes slightly between measurements, and where new measurements of a fixed signal are sequentially added to the system. We develop algorithms to quickly update the solution of several different types of l1 optimization problems whenever these changes occur, thus avoiding having to solve a new optimization problem from scratch. Our proposed schemes are based on homotopy continuation, which breaks down the solution update in a systematic and efficient way into a small number of linear steps. Each step consists of a low-rank update and a small number of matrix-vector multiplications - very much like recursive least squares. Our investigation also includes dynamic updating schemes for l1 decoding problems, where an arbitrary signal is to be recovered from redundant coded measurements which have been corrupted by sparse errors.

\ell_{1}

Most of the existing sparse-recovery methods assume a static system: the signal is a finite-length vector for which a fixed set of measurements and sparse representation are available and an l1 problem is solved for the reconstruction. However, the same representation and reconstruction framework is not readily applicable in a streaming system: the signal changes over time, and it is measured and reconstructed sequentially over small intervals. This is particularly desired when dividing signals into disjoint blocks and processing each block separately is infeasible or inefficient. In this paper, we discuss two streaming systems and a new homotopy algorithm for quickly solving the associated l1 problems: 1) recovery of smooth, time-varying signals for which, instead of using block transforms, we use lapped orthogonal transforms for sparse representation and 2) recovery of sparse, time-varying signals that follows a linear dynamic model. For both systems, we iteratively process measurements over a sliding interval and solve a weighted l1-norm minimization problem for estimating sparse coefficients. Since we estimate overlapping portions of the signal while adding and removing measurements, instead of solving a new l1 program as the system changes, we use available signal estimates as starting point in a homotopy formulation and update the solution in a few simple steps. We demonstrate with numerical experiments that our proposed streaming recovery framework provides better reconstruction compared to the methods that represent and reconstruct signals as independent, disjoint blocks, and that our proposed homotopy algorithm updates the solution faster than the current state-of-the-art solvers.

Minimization

In this work we address the problem of state estimation in dynamical systems using recent developments in compressive sensing and sparse approximation. We formulate the traditional Kalman filter as a one-step update optimization procedure which leads us to a more unified framework, useful for incorporating sparsity constraints. We introduce three combinations of two sparsity conditions (sparsity in the state and sparsity in the innovations) and write recursive optimization programs to estimate the state for each model. This paper is meant as an overview of different methods for incorporating sparsity into the dynamic model, a presentation of algorithms that unify the support and coefficient estimation, and a demonstration that these suboptimal schemes can actually show some performance improvements (either in estimation error or convergence time) over standard optimal methods that use an impoverished model.

Sparse Recovery of Streaming Signals Using ℓ 1 -Homotopy.

Accelerated magnetic resonance imaging techniques reduce signal acquisition time by undersampling k‐space. A fundamental problem in accelerated magnetic resonance imaging is the recovery of quality images from undersampled k‐space data. Current state‐of‐the‐art recovery algorithms exploit the spatial and temporal structures in underlying images to improve the reconstruction quality. In recent years, compressed sensing theory has helped formulate mathematical principles and conditions that ensure recovery of (structured) sparse signals from undersampled, incoherent measurements. In this article, a new recovery algorithm, motion‐adaptive spatio‐temporal regularization, is presented that uses spatial and temporal structured sparsity of MR images in the compressed sensing framework to recover dynamic MR images from highly undersampled k‐space data. In contrast to existing algorithms, our proposed algorithm models temporal sparsity using motion‐adaptive linear transformations between neighboring images. The efficiency of motion‐adaptive spatio‐temporal regularization is demonstrated with experiments on cardiac magnetic resonance imaging for a range of reduction factors. Results are also compared with k‐t FOCUSS with motion estimation and compensation—another recently proposed recovery algorithm for dynamic magnetic resonance imaging. Magn Reson Med 70:800–812, 2013.

Sparsity penalties in dynamical system estimation

Many signal processing applications revolve around finding a sparse solution to a (often underdetermined) system of linear equations. Recent results in compressive sensing (CS) have shown that when the signal we are trying to acquire is sparse and the measurements are incoherent, the signal can be reconstructed reliably from an incomplete set of measurements. However, the signal recovery is an involved process, usually requiring the solution of an ℓ1 minimization program. In this paper we discuss the problem of estimating a time-varying sparse signal from a series of linear measurements. We propose an efficient way to dynamically update the solution to two types of ℓ1 problems when the underlying signal changes. The proposed dynamic update scheme is based on homotopy continuation, which systematically breaks down the solution update into a small number of linear steps. The computational cost for each step is just a few matrix-vector multiplications.

Motion‐adaptive spatio‐temporal regularization for accelerated dynamic MRI

Compressive sampling (CS) has emerged as significant signal processing framework to acquire and reconstruct sparse signals at rates significantly below the Nyquist rate. However, most of the CS development to-date has focused on finite-length signals and representations. In this paper we discuss a streaming CS framework and greedy reconstruction algorithm, the Streaming Greedy Pursuit (SGP), to reconstruct signals with sparse frequency content. Our proposed sampling framework and the SGP are explicitly intended for streaming applications and signals of unknown length. The measurement framework we propose is designed to be causal and implementable using existing hardware architectures. Furthermore, our reconstruction algorithm provides specific computational guarantees, which makes it appropriate for real-time system implementations. Our experimental results on very long signals demonstrate the good performance of the SGP and validate our approach.

Dynamic updating for sparse time varying signals

Blind deconvolution arises naturally when dealing with finite multipath interference on a signal. In this paper we present a new method to protect the signals from the effects of sparse multipath channels — we modulate/encode the signal using random waveforms before transmission and estimate the channel and signal from the observations, without any prior knowledge of the channel other than that it is sparse. The problem can be articulated as follows. The original message x is encoded with an overdetermined m × n (m > n) matrix A whose entries are randomly chosen; the encoded message is given by Ax. The received signal is the convolution of the encoded message with h, the S-sparse impulse response of the channel. We explore three different schemes to recover the message x and the channel h simultaneously. The first scheme recasts the problem as a block l1 optimization program. The second scheme imposes a rank-1 structure on the estimated signal. The third scheme uses nuclear norm as a proxy for rank, to recover the x and h. The simulation results are presented to demonstrate the efficiency of the random coding and proposed recovery schemes.

Compressive sampling for streaming signals with sparse frequency content

The ability of Compressive Sensing (CS) to recover sparse signals from limited measurements has been recently exploited in computational imaging to acquire high-speed periodic and near-periodic videos using only a low-speed camera with coded exposure and intensive off-line processing. Each low-speed frame integrates a coded sequence of high-speed frames during its exposure time. The high-speed video can be reconstructed from the low-speed coded frames using a sparse recovery algorithm. This paper presents a new streaming CS algorithm specifically tailored to this application. Our streaming approach allows causal on-line acquisition and reconstruction of the video, with a small, controllable, and guaranteed buffer delay and low computational cost. The algorithm adapts to changes in the signal structure and, thus, outperforms the off-line algorithm in realistic signals.

Random channel coding and blind deconvolution

Recovery of sparse signals from noisy observations is a problem that arises in many information processing contexts. LASSO and the Dantzig selector (DS) are two well-known schemes used to recover high-dimensional sparse signals from linear observations. This paper presents some results on the equivalence between LASSO and DS. We discuss a set of conditions under which the solutions of LASSO and DS are same. With these conditions in place, we formulate a shrinkage procedure for which LASSO and DS follow the same solution path. Furthermore, we show that under these shrinkage conditions the solution to LASSO and DS can be attained in at most S homotopy steps, where S is the number of nonzero elements in the final solution. Thus the computational cost for finding complete homotopy path for an M × N system is merely O(SMN).

Streaming Compressive Sensing for high-speed periodic videos

This paper presents an algorithm for an ℓ1-regularized Kalman filter. Given observations of a discrete-time linear dynamical system with sparse errors in the state evolution, we estimate the state sequence by solving an optimization algorithm that balances fidelity to the measurements (measured by the standard ℓ2 norm) against the sparsity of the innovations (measured using the ℓ1 norm). We also derive an efficient algorithm for updating the estimate as the system evolves. This dynamic updating algorithm uses a homotopy scheme that tracks the solution as new measurements are slowly worked into the system and old measurements are slowly removed. The effective cost of adding new measurements is a number of low-rank updates to the solution of a linear system of equations that is roughly proportional to the joint sparsity of all the innovations in the time interval of interest.