Lennart Ljung

Nonlinear black-box modeling in system identification: a unified overview

A nonlinear black-box structure for a dynamical system is a model structure that is prepared to describe virtually any nonlinear dynamics. There has been considerable recent interest in this area, with structures based on neural networks, radial basis networks, wavelet networks and hinging hyperplanes, as well as wavelet-transform-based methods and models based on fuzzy sets and fuzzy rules. This paper describes all these approaches in a common framework, from a users perspective. It focuses on what are the common features in the different approaches, the choices that have to be made and what considerations are relevant for a successful system-identification application of these techniques. It is pointed out that the nonlinear structures can be seen as a concatenation of a mapping form observed data to a regression vector and a nonlinear mapping from the regressor space to the output space. These mappings are discussed separately. The latter mapping is usually formed as a basis function expansion. The basis functions are typically formed from one simple scalar function, which is modified in terms of scale and location. The expansion from the scalar argument to the regressor space is achieved by a radial- or a ridge-type approach. Basic techniques for estimating the parameters in the structures are criterion minimization, as well as two-step procedures, where first the relevant basis functions are determined, using data, and then a linear least-squares step to determine the coordinates of the function approximation. A particular problem is to deal with the large number of potentially necessary parameters. This is handled by making the number of ‘used’ parameters considerably less than the number of ‘offered’ parameters, by regularization, shrinking, pruning or regressor selection.

Asymptotic behavior of the extended Kalman filter as a parameter estimator for linear systems

The extended Kalman filter is an approximate filter for nonlinear systems, based on first-order linearization. Its use for the joint parameter and state estimation problem for linear systems with unknown parameters is well known and widely spread. Here a convergence analysis of this method is given. It is shown that in general, the estimates may be biased or divergent and the causes for this are displayed. Some common special cases where convergence is guaranteed are also given. The analysis gives insight into the convergence mechanisms and it is shown that with a modification of the algorithm, global convergence results can be obtained for a general case. The scheme can then be interpreted as maximization of the likelihood function for the estimation problem, or as a recursive prediction error algorithm.

Paper: Theory and applications of self-tuning regulators

This paper reviews work on self-tuning regulators. The regulator algorithms, their theory and industrial applications are reviewed. The paper is expository-the major ideas are covered but detailed analysis is given elsewhere.

On global identifiability for arbitrary model parametrizations

It is a fundamental problem of identification to be able—even before the data have been analyzed—to decide if all the free parameters of a model structure can be uniquely recovered from data. This is the issue of global identifiability. In this contribution we show how global identifiability for an arbitrary model structure (basically with analytic non-linearities) can be analyzed using concepts and algorithms from differential algebra. It is shown how the question of global structural identifiability is reduced to the question of whether the given model structure can be rearranged as a linear regression. An explicit algorithm to test this is also given. Furthermore, the question of ‘persistent excitation’ for the input can also be tested explicitly is a similar fashion. The algorithms involved are very well suited for implementation in computer algebra. One such implementation is also described.

Subspace-based multivariable system identification from frequency response data

Two noniterative subspace-based algorithms which identify linear, time-invariant MIMO (multi-input/multioutput) systems from frequency response data are presented. The algorithms are related to the recent time-domain subspace identification techniques. The first algorithm uses equidistantly, in frequency, spaced data and is strongly consistent under weak noise assumptions. The second algorithm uses arbitrary frequency spacing and is strongly consistent under more restrictive noise assumptions, promising results are obtained when the algorithms are applied to real frequency data originating from a large flexible structure.

Identification of processes in closed loop-identifiability and accuracy aspects

It is often necessary in practice to perform identification experiments on systems operating in closed loop. There has been some confusion about the possibilities of successful identification in such cases, evidently due to the fact that certain common methods then fail. A rapidly increasing literature on the problem is briefly surveyed in this paper, and an overview of a particular approach is given. It is shown that prediction error identification methods, applied in a direct fashion will given correct estimates in a number of feedback cases. Furthermore, the accuracy is not necessarily worse in the presence of feedback; in fact optimal inputs may very well require feedback terms. Some practical applications are also described.

Convergence analysis of parametric identification methods

A certain class of methods to select suitable models of dynamical stochastic systems from measured input-output data is considered. The methods are based on a comparison between the measured outputs and the outputs of a candidate model. Depending on the set of models that is used, such methods are known under a variety of names, like output-error methods, equation-error methods, maximum-likelihood methods, etc. General results are proved concerning the models that are selected asymptotically as the number of observed data tends to infinity. For these results it is not assumed that the true system necessarily can be exactly represented within the chosen set of models. In the particular case when the model set contains the system, general consistency results are obtained and commented upon. Rather than to seek an exact description of the system, it is usually more realistic to be content with a suitable approximation of the true system with reasonable complexity properties. Here, the consequences of such a viewpoint are discussed.

On positive real transfer functions and the convergence of some recursive schemes

The convergence with probability one of a recently suggested recursive identification method by Landau is investigated. The positive realness of a certain transfer function is shown to play a crucial role, both for the proof of convergence and for convergence itself. A completely analogous analysis can be performed also for the extended least squares method and for the self-tuning regulator of Astrom and Wittenmark. Explicit conditions for convergence of all these schemes are given. A more general structure is also discussed, as well as relations to other recursive algorithms.

Fast calculation of gain matrices for recursive estimation schemes

A sequence of vectors {x(t)} with dimension N is given, such that x(t+1) is obtained from x(t) by introducing p new elements, deleting p old ones, and shifting the others in some fashion. The sequence of vectors

Identification of piecewise affine systems via mixed-integer programming

is sought. This paper presents a method of calculating these vectors with proportional-to-Np operations and memory locations, in contrast to the conventional way which requires proportional-to-N 2 operations and memory locations. A non-symmetric case is also treated.