Branko Kovačević

Robust estimation with unknown noise statistics

The equivalence between the Kalman filter and a particular least squares regression problem is established and the regression problem is solved robustly using a statistical approach, named M-estimation. M-robust estimators are derived for adaptive estimation of the unknown a priori state and observation noise statistics simultaneously with the system states. The feasibility of the approach is demonstrated with simulation.

On robust Kalman filtering

The problem of making the Kalman filter robust is considered in the paper. Proceeding from the equivalence between the Kalman filter and the least squares regression problem, a statistical approach...

Robust recursive AR speech analysis

Abstract In this paper a new robust recursive method of estimating the linear prediction parameters of an auto-regressive speech signal model using weighted least squares with variable forgetting factors (VVFs) is described. The proposed robust recursive least-squares (RRLS) method differs from the conventional recursive least-squares (RLS) method by the insertion of a suitably chosen nonlinear transformation of the prediction residuals. The RRLS algorithm takes into account the contaminated Gaussian nature of the excitation for voiced speech, and the effect of nonlinearity is to assign less weight to the small portions of large residuals so that the spiky excitation will not greatly influence the final AR parameter estimates, while giving unity weight to the bulk of small to moderate residuals generated by the nominal Gaussian distribution. In addition, the VFF is adapted to a nonstationary speech signal by a generalized likelihood ratio algorithm, which accounts for the nonstationarity of a speech signal. The proposed method has a good adaptability to the nonstationary parts of a speech signal, and gives low bias and low variance at the stationary signal segments. The feasibility of the robust approach is demonstrated with both synthesized and natural speech.

Analysis of robust stochastic approximation algorithms for process identification

Abstract An analysis of robust recursive algorithms for dynamic system identification is presented. Problems related to the construction of optimal stochastic approximation algorithms in the min-max sense are demonstrated. Starting from the definition of one class of robustified recursive identification algorithms, several procedures are derived through convenient approximations and initial assumptions. A detailed Monte Carlo analysis gives an insight into the practical robustness of these procedures indicating the most reliable ones. Important relationships between parameters describing the algorithms are pointed out.

Robust non-recursive AR speech analysis

Abstract In this paper a robust non-recursive algorithm for estimating the linear prediction (LP) parameters of autoregressive (AR) speech signal model is proposed. Starting from Hubers robust M-estimation procedure, minimizing the sum of appropriately weighted residuals, a two-step robust LP procedure (RBLP) is derived. In the first step the Hubers convex cost function is selected to give more weights to the bulk of smaller residuals, while down-weighting the small portion of large residuals, and the Newton-type algorithm is used to minimize the adopted criterion. The proposed algorithm takes into account the non-Gaussian nature of the excitation for voiced speech, being characterized by heavier tails of the underlying distribution, which generates high-intensity signal realizations named outliers. The obtained estimates are used as a new start in the weighted least-squares procedure, based on a redescending function of the prediction residuals, which has to cut off the outliers. The experiments on both synthesized and natural speech have shown that the proposed two-step RBLP gives more efficient (less variance) and less biased estimates than the conventional LP algorithms, and a one-step RBLP based on a convex cost function.

On robust AML identification algorithms

Abstract Strong consistency results for a class of nonlinear approximate maximum likelihood algorithms for robust system identification are developed, where the system is assumed to be of the ARMAX form. The analysis uses the Martingale results, and strong consistency is shown to hold under a new assumption, representing a generalization of the strictly positive-real condition. Arguments are also given for using Hubers nonlinearity, in order to reduce the influence of outliers in practice.

QQ-plot approach to robust Kalman filtering

Noise distribution arising in certain applications frequently deviates from the assumed gaussian model, often being characterized by heavier tails generating the outliers. Since, in the presence of outliers, the performance of a Kalman filter can be very poor, a statistical approach-named QQ-plot—is suggested to make the Kalman filter more robust. In addition, the first and the second-order moments of noise processes are estimated simultaneously with the system states, using the QQ-plot of the noise samples generated in the ‘robustified’ Kalman filter algorithm. Results of simulation demonstrating the robustness of the proposed state estimators are also included.

Fundamentals of Stochastic Signals, Systems and Estimation Theory: With worked Examples

The main theme of this book deals with fundamental concepts underlying stochastic signal or linear stochastic systems, their modelling and analysis as well as model-based signal processing. Two popular stochastic models, the polynomial (or transfer function) model and the state space model are employed in schemes that lead to the estimation of unknown system parameters or states. The book is written for undergraduate and graduate students as well as practising engineers, specializing the the areas of electrical communications, signal processing and automatic control. Many examples illustrate the concepts of this book and the reader learns how to write software implementations of estimators on computers.

A statistical pattern recognition approach to robust recursive identification of nonstationary AR model of speech production system

We propose a new robust recursive procedure based on the weighted recursive least squares (WRLS) algorithm with variable forgetting factor (VFF) and frame-based quadratic classifier for identification of nonstationary AR model of speech. Two versions of the frame-based quadratic classifier design procedure are elaborated upon. Experimental results are obtained in analyzing speech signal on voiced and mixed excitation frames.

Robust real-time identification of linear systems with correlated noise

The problem of recursive robust identification of linear discrete-time single-input single-output dynamic systems with correlated disturbances is considered. Problems related to the construction of optimal robust stochastic approximation algorithms in the min-max sense are demonstrated. Since the optimal solution cannot be achieved in practice, several robustified stochastic approximation algorithms are derived on the basis of a suitable non-linear transformation of normalized residuals, as well as step-by-step optimization with respect to the weighting matrix of the algorithm. The convergence of the developed algorithms is established theoretically using the ordinary differential equation approach. Monte Carlo simulation results are presented for the quantitative performance evaluation of the proposed algorithms. The results indicate the most suitable algorithms for applications in engineering practice.