Tillmann Falck

Identification of Wiener-Hammerstein Systems using LS-SVMs

Abstract This paper extends the overparametrization technique as used for Hammerstein systems employing nonlinear Least-Squares Support Vector Machines (LS-SVMs) towards the identification of Wiener-Hammerstein systems. We present some practical guidelines as well as empirical results on the performance of the method with respect to various deficiencies of the excitation signal. Finally we apply our method to the SYSID2009 Wiener-Hammerstein benchmark data set.

Approximate Solutions to Ordinary Differential Equations Using Least Squares Support Vector Machines

In this paper, a new approach based on least squares support vector machines (LS-SVMs) is proposed for solving linear and nonlinear ordinary differential equations (ODEs). The approximate solution is presented in closed form by means of LS-SVMs, whose parameters are adjusted to minimize an appropriate error function. For the linear and nonlinear cases, these parameters are obtained by solving a system of linear and nonlinear equations, respectively. The method is well suited to solving mildly stiff, nonstiff, and singular ODEs with initial and boundary conditions. Numerical results demonstrate the efficiency of the proposed method over existing methods.

Nuclear norm regularization for overparametrized Hammerstein systems

In this paper we study the overparametrization scheme for Hammerstein systems [1] in the presence of regularization. The quality of the convex approximation is analysed, that is obtained by relaxing the implicit rank one constraint. To obtain an improved convex relaxation we propose the use of nuclear norms [2], instead of using ridge regression. On several simple examples we illustrate that this yields a solution close to the best possible convex approximation. Furthermore the experiments suggest that ridge regression in combination with a projection step yield a generalization performance close to the one obtained by nuclear norms.

Parameter Estimation for Time Varying Dynamical Systems using Least Squares Support Vector Machines

Abstract This paper develops a new approach based on Least Squares Support Vector Machines (LS-SVMs) for parameter estimation of time invariant as well as time varying dynamical SISO systems. Closed-form approximate models for the state and its derivative are first derived from the observed data by means of LS-SVMs. The time-derivative information is then substituted into the system of ODEs, converting the parameter estimation problem into an algebraic optimization problem. In the case of time invariant systems one can use least-squares to solve the obtained system of algebraic equations. The estimation of time-varying coefficients in SISO models, is obtained by assuming an LS-SVM model for it.

Linear parametric noise models for Least Squares Support Vector Machines

In the identification of nonlinear dynamical models it happens that not only the system dynamics have to be modeled but also the noise has a dynamic character. We show how to adapt Least Squares Support Vector Machines (LS-SVMs) to take advantage of a known or unknown noise model. We furthermore investigate a convex approximation based on over-parametrization to estimate a linear autoregressive noise model jointly with a model for the nonlinear system. Considering a noise model can improve generalization performance. We discuss several properties of the proposed scheme on synthetic data sets and finally demonstrate its applicability on real world data.

Segmentation of time series from nonlinear dynamical systems

Segmentation of time series data is of interest in many applications, as for example in change detection and fault detection. In the area of convex optimization, the sum-of-norms regularization has ...

Polynomial componentwise LS-SVM: Fast variable selection using low rank updates

This paper describes a Least Squares Support Vector Machines (LS-SVM) approach to estimate additive models as a sum of non-linear components. In particular, this work discuses the low rank matrix modifications for componentwise polynomial kernels, which allow the factors of the modified kernel-matrix to be directly updated. The main concept refers to the use of a valid explicit feature map for polynomial kernels in an additive setting. By exploiting the structure of such feature map the model parameters of the classification/regression problem can be easily modified and updated when new variables are added. Therefore, the low rank updates constitute an algorithmic tool to efficiently obtain the model parameters once the system has been altered in some minimal sense. Such strategy allows, for instance, the development of algorithms for sequential variable ranking in high dimensional settings, while non-linearity is provided by the polynomial feature map. Moreover relevant variables can be robustly ranked using the closed form of the leave-one-out (LOO) error estimator, obtained as a by-product of the low rank modifications.