Bogdan Dumitrescu

Positive Trigonometric Polynomials and Signal Processing Applications

Positive and sum-of-squares polynomials have received a special interest in the latest decade, due to their connections with semidefinite programming. Thus, efficient optimization methods can be employed to solve diverse problems involving polynomials. This book gathers the main recent results on positive trigonometric polynomials within a unitary framework; the theoretical results are obtained partly from the general theory of real polynomials, partly from self-sustained developments. The optimization applications cover a field different from that of real polynomials, mainly in signal processing problems: design of 1-D and 2-D FIR or IIR filters, design of orthogonal filterbanks and wavelets, stability of multidimensional discrete-time systems. Positive Trigonometric Polynomials and Signal Processing Applicationshas two parts: theory and applications. The theory of sum-of-squares trigonometric polynomials is presented unitarily based on the concept of Gram matrix (extended to Gram pair or Gram set). The presentation starts by giving the main results for univariate polynomials, which are later extended and generalized for multivariate polynomials. The applications part is organized as a collection of related problems that use systematically the theoretical results. All the problems are brought to a semidefinite programming form, ready to be solved with algorithms freely available, like those from the library SeDuMi.

On the parameterization of positive real sequences and MA parameter estimation

An algorithm for moving average (MA) parameter estimation was proposed by Stoica et al. (see ibid. vol.48, p.1999-2012, 2000). Its key step (covariance fitting) is a semidefinite programming (SDP) problem with two convex constraints: one reflecting the real positiveness of the desired covariance sequence and the other having a second-order cone form. We analyze two parameterizations of a positive real sequence and show that there is a one-to-one correspondence between them. We also show that the dual of the covariance fitting problem has a significantly smaller number of variables and, thus, a much reduced computational complexity. We discuss in detail the formulations that are best suited for the currently available semidefinite quadratic programming packages. Experimental results show that the execution times of the newly proposed algorithms scale well with the MA order, which are therefore convenient for large-order MA signals.

Multistage IIR filter design using convex stability domains defined by positive realness

In this paper, we consider infinite impulse response (IIR) filter design where both magnitude and phase are optimized using a weighted and sampled least-squares criterion. We propose a new convex stability domain defined by positive realness for ensuring the stability of the filter and adapt the Steiglitz-McBride (SM), Gauss-Newton (GN), and classical descent methods to the new stability domain. We show how to describe the stability domain such that the description is suited to semidefinite programming and is implementable exactly; in addition, we prove that this domain contains the domain given by Rouche/spl acute/s theorem. Finally, we give experimental evidence that the best designs are usually obtained with a multistage algorithm, where the three above methods are used in succession, each one being initialized with the result of the previous and where the positive realness stability domain is used instead of that defined by Rouche/spl acute/s theorem.

Trigonometric Polynomials Positive on Frequency Domains and Applications to 2-D FIR Filter Design

We propose a characterization of multivariate trigonometric polynomials that are positive on a given frequency domain. The positive polynomials are parameterized as a linear function of sum-of-squares polynomials and so semidefinite programming (SDP) is applicable. The frequency domain is expressed via the positivity of some trigonometric polynomials. We also give a bounded real lemma (BRL) in which a bounding condition on the magnitude of the frequency response of a multidimensional finite-impulse-response (FIR) filter is expressed as a linear matrix inequality (LMI). This BRL avoids the problem of a lack of spectral factorization in the multidimensional case. All the proposed theoretical contributions can be implemented only as sufficient conditions, due to degree limitations on the sum-of-square polynomials. However, the two-dimensional (2-D) FIR filter designs we study numerically suggest that these limitations have negligible impact on the optimality

Greedy Sparse RLS

Starting from the orthogonal (greedy) least squares method, we build an adaptive algorithm for finding online sparse solutions to linear systems. The algorithm belongs to the exponentially windowed recursive least squares (RLS) family and maintains a partial orthogonal factorization with pivoting of the system matrix. For complexity reasons, the permutations that bring the relevant columns into the first positions are restrained mainly to interchanges between neighbors at each time moment. The storage scheme allows the computation of the exact factorization, implicitly working on indefinitely long vectors. The sparsity level of the solution, i.e., the number of nonzero elements, is estimated using information theoretic criteria, in particular Bayesian information criterion (BIC) and predictive least squares. We present simulations showing that, for identifying sparse time-varying FIR channels, our algorithm is consistently better than previous sparse RLS methods based on the -norm regularization of the RLS criterion. We also use our sparse greedy RLS algorithm for computing linear predictions in a lossless audio coding scheme and obtain better compression than MPEG4 ALS using an RLS-LMS cascade.

Stagewise K-SVD to Design Efficient Dictionaries for Sparse Representations

The problem of training a dictionary for sparse representations from a given dataset is receiving a lot of attention mainly due to its applications in the fields of coding, classification and pattern recognition. One of the open questions is how to choose the number of atoms in the dictionary: if the dictionary is too small then the representation errors are big and if the dictionary is too big then using it becomes computationally expensive. In this letter, we solve the problem of computing efficient dictionaries of reduced size by a new design method, called Stagewise K-SVD, which is an adaptation of the popular K-SVD algorithm. Since K-SVD performs very well in practice, we use K-SVD steps to gradually build dictionaries that fulfill an imposed error constraint. The conceptual simplicity of the method makes it easy to apply, while the numerical experiments highlight its efficiency for different overcomplete dictionaries.

Parameterization of positive-real transfer functions with fixed poles

A parameterization of continuous-time passive systems (positive real transfer functions) with given poles is proposed. This parameterization has a minimum number of parameters and is suited to convex optimization, especially for semidefinite programming. As an example, we show how to find the best approximation of a stable transfer function with a positive real one, with the same poles.

LMI Stability Tests for the Fornasini-Marchesini Model

We present a linear matrix inequality (LMI) for testing the stability of a 2-D system described by the Fornasini-Marchesini first model. The test is based on the properties of sum-of-squares polynomials with matrix coefficients. Although the test implements a sufficient condition, extensive experiments suggest that the gap to necessity is very small. We also derive an LMI describing stability conditions around a given model and useful in robust stability testing.

Optimization of Symmetric Self-Hilbertian Filters for the Dual-Tree Complex Wavelet Transform

In this letter, we expand upon the method of Tay for the design of orthonormal ldquoQ-shiftrdquo filters for the dual-tree complex wavelet transform. The method of Tay searches for good Hilbert-pairs in a one-parameter family of conjugate-quadrature filters that have one vanishing moment less than the Daubechies conjugate-quadrature filters (CQFs). In this letter, we compute feasible sets for one- and two-parameter families of CQFs by employing the trace parameterization of nonnegative trigonometric polynomials and semidefinite programming. This permits the design of CQF pairs that define complex wavelets that are more nearly analytic, yet still have a high number of vanishing moments.

Optimization of two-dimensional IIR filters with nonseparable and separable denominator

We present algorithms for the optimization of two-dimensional (2-D) infinite impulse response (IIR) filters with separable or nonseparable denominator, for least squares or Chebyshev criteria. The algorithms are iterative, and each iteration consists of solving a semidefinite programming problem. For least squares designs, we adapt the Gauss-Newton idea, which outcomes to a convex approximation of the optimization criterion. For Chebyshev designs, we adapt the iterative reweighted least squares (IRLS) algorithm; in each iteration, a least squares Gauss-Newton step is performed, while the weights are changed as in the basic IRLS algorithm. The stability of the 2-D IIR filters is ensured by keeping the denominator inside convex stability domains, which are defined by linear matrix inequalities. For the 2-D (nonseparable) case, this is a new contribution, based on the parameterization of 2-D polynomials that are positive on the unit bicircle. In the experimental section, 2-D IIR filters with separable and nonseparable denominators are designed and compared. We show that each type may be better than the other, depending on the design specification. We also give an example of filter that is clearly better than a recent very good design.