Nicholas Kalouptsidis

A fast sequential algorithm for least-squares filtering and prediction

A new computationally efficient algorithm for sequential least-squares (LS) estimation is presented in this paper. This fast a posteriori error sequential technique (FAEST) requires 5p MADPR (multiplications and divisions per recursion) for AR modeling and 7p MADPR for LS FIR filtering, where p is the number of estimated parameters. In contrast the well-known fast Kalman algorithm requires 8p MADPR for AR modeling and 10p MADPR for FIR filtering. The increased computational speed of the introduced algorithm stems from an alternative definition of the so-called Kalman gain vector, which takes better advantage of the relationships between forward and backward linear prediction.

SPARLS: The Sparse RLS Algorithm

We develop a recursive L1-regularized least squares (SPARLS) algorithm for the estimation of a sparse tap-weight vector in the adaptive filtering setting. The SPARLS algorithm exploits noisy observations of the tap-weight vector output stream and produces its estimate using an expectation-maximization type algorithm. We prove the convergence of the SPARLS algorithm to a near-optimal estimate in a stationary environment and present analytical results for the steady state error. Simulation studies in the context of channel estimation, employing multipath wireless channels, show that the SPARLS algorithm has significant improvement over the conventional widely used recursive least squares (RLS) algorithm in terms of mean squared error (MSE). Moreover, these simulation studies suggest that the SPARLS algorithm (with slight modifications) can operate with lower computational requirements than the RLS algorithm, when applied to tap-weight vectors with fixed support.

Fast recursive algorithms for a class of linear equations

In many signal processing applications, one often seeks the solution of a linear system of equations by means of fast algorithms. The special form of the matrix associated with the linear system may permit the development of algorithms requiring 0 (p2) or fewer operations. Hankel and Toeplitz matrices provide well known examples and various fast schemes have been developed in the literature to cover these cases. These techniques have common characteristics so that they may be generalized to cover a wider class of linear systems. The purpose of this paper is to develop fast algorithms that cover this wider set of systems. An important feature of the general scheme introduced here is that it leads to the definition of two broad classes of matrices, called diagonal innovation matrices (DIM) and peripheral innovation matrices (PIM), for which fast schemes can be developed. The class of PIM matrices includes many structures appearing in signal processing applications. Most of them are extensively studied in this paper and Fortran coding is provided. Finally, ARMA modeling is considered and within the general framework already introduced, fast methods for the determination of the autoregressive (AR) portion of the ARMA model are presented.

Nonlinear system identification using Gaussian inputs

The paper is concerned with the identification of nonlinear systems represented by Volterra expansions and driven by stationary, zero mean Gaussian inputs, with arbitrary spectra that are not necessarily white. Procedures for the computation of the Volterra kernels both in the time as well as in the frequency domain are developed based on cross-cumulant information. The derived kernels are optimal in the mean squared error sense for noncausal systems. Order recursive procedures based on minimum mean squared error reduction are derived. More general input output representations that result when the Volterra kernels are expanded in a given orthogonal base are also considered. >

Efficient algorithms for Volterra system identification

In this paper, nonlinear filtering and identification based on finite-support Volterra models are considered. The Volterra kernels are estimated via input-output statistics or directly in terms of input-output data. It is shown that the normal equations for a finite-support Volterra system excited by zero mean Gaussian input have a unique solution if, and only if, the power spectral process of the input signal is nonzero at least at m distinct frequencies, where m is the memory of the system. A multichannel embedding approach is introduced. A set of primary signals defined in terms of the input signal serve to map efficiently the nonlinear process to an equivalent multichannel format. Efficient algorithms for the estimation of the Volterra parameters are derived for batch, as well as for adaptive processing. An efficient order-recursive method is presented for the determination of the Volterra model structure. The proposed methods are illustrated by simulations.

Stability improvement of nonlinear systems by feedback

This paper is concerned with the property of stabilizing a nonlinear system to a specified equilibrium point arbitrarily fast by appropriate smooth feedback. Two closely related forms of this property are explored. One refers to the asymptotic transfer to the critical point with exponential decay, and it is shown that the systems possessing the above property are precisely those whose corresponding linearization is controllable. The second stabilizability form defined here amounts to driving the system back to an arbitrarily small neighborhood of the given point arbitrarily fast by smooth feedback. A global stabilization result is also included, finding application in Hamiltonian systems. Special emphasis is given to the bilinear case.

Adaptive algorithms for sparse system identification

In this paper, identification of sparse linear and nonlinear systems is considered via compressive sensing methods. Efficient algorithms are developed based on Kalman filtering and Expectation-Maximization. The proposed algorithms are applied to linear and nonlinear channels which are represented by sparse Volterra models and incorporate the effect of power amplifiers. Simulation studies confirm significant performance gains in comparison to conventional non-sparse methods.

Output feedback stabilization

The stabilization of control systems using output feedback is addressed. Necessary and sufficient Lyapunov conditions are established, generalizing the Artsteins stabilization theorem. For linear control systems, a necessary and sufficient Lyapunov condition is provided for stabilization by means of a linear output feedback law. >

An Adaptive Greedy Algorithm With Application to Nonlinear Communications

Greedy algorithms form an essential tool for compressed sensing. However, their inherent batch mode discourages their use in time-varying environments due to significant complexity and storage requirements. In this paper two existing powerful greedy schemes developed in the literature are converted into an adaptive algorithm which is applied to estimation of a class of nonlinear communication systems. Performance is assessed via computer simulations on a variety of linear and nonlinear channels; all confirm significant improvements over conventional methods.

Asymptotic Achievability of the CramÉr–Rao Bound for Noisy Compressive Sampling

We consider a model of the form y = Ax + n, where x isin CM is sparse with at most L nonzero coefficients in unknown locations, y isin CN is the observation vector, A isin CN times M is the measurement matrix and n isin CN is the Gaussian noise. We develop a Cramer-Rao bound on the mean squared estimation error of the nonzero elements of x, corresponding to the genie-aided estimator (GAE) which is provided with the locations of the nonzero elements of x. Intuitively, the mean squared estimation error of any estimator without the knowledge of the locations of the nonzero elements of x is no less than that of the GAE. Assuming that L/N is fixed, we establish the existence of an estimator that asymptotically achieves the Cramer-Rao bound without any knowledge of the locations of the nonzero elements of x as N rarr infin , for A a random Gaussian matrix whose elements are drawn i.i.d. according to N (0,1) .