Miguel Galrinho

A weighted least-squares method for parameter estimation in structured models

Parameter estimation in structured models is generally considered a difficult problem. For example, the prediction error method (PEM) typically gives a non-convex optimization problem, while it is difficult to incorporate structural information in subspace identification. In this contribution, we revisit the idea of iteratively using the weighted least-squares method to cope with the problem of non-convex optimization. The method is, essentially, a three-step method. First, a high order least-squares estimate is computed. Next, this model is reduced to a structured estimate using the least-squares method. Finally, the structured estimate is re-estimated, using weighted least-squares, with weights obtained from the first structured estimate. This methodology has a long history, and has been applied to a range of signal processing problems. In particular, it forms the basis of iterative quadratic maximum likelihood (IQML) and the Steiglitz-McBride method. Our contributions are as follows. Firstly, for output-error models, we provide statistically optimal weights. We conjecture that the method is asymptotically efficient under mild assumptions and support this claim by simulations. Secondly, we point to a wide range of structured estimation problems where this technique can be applied. Finally, we relate this type of technique to classical prediction error and subspace methods by showing that it can be interpreted as a link between the two, sharing favorable properties with both domains.

On estimating initial conditions in unstructured models

Estimation of structured models is an important problem in system identification. Some methods, as an intermediate step to obtain the model of interest, estimate the impulse response parameters of the system. This approach dates back to the beginning of subspace identification and is still used in recently proposed methods. A limitation of this procedure is that, when obtaining these parameters from a high-order unstructured model, the initial conditions of the system are typically unknown, which imposes a truncation of the measured output data for the estimation. For finite sample sizes, discarding part of the data limits the performance of the method. To deal with this issue, we propose an approach that uses all the available data, and estimates also the initial conditions of the system. Then, as examples, we show how this approach can be applied to two methods in a beneficial manner. Finally, we use a simulation study to exemplify the potential of the approach.

ARX modeling of unstable linear systems

High-order ARX models can be used to approximate a quite general class of linear systems in a parametric model structure, and well-established methods can then be used to retrieve the true plant and noise models from the ARX polynomials. However, this commonly used approach is only valid when the plant is stable or if the unstable poles are shared with the true noise model. In this contribution, we generalize this approach to allow the unstable poles not to be shared, by introducing modifications to correctly retrieve the noise model and noise variance.

Open-loop asymptotically efficient model reduction with the Steiglitz–McBride method

In system identification, it is often difficult to use a physical intuition when choosing a noise model structure. The importance of this choice is that, for the prediction error method (PEM) to provide asymptotically efficient estimates, the model orders must be chosen according to the true system. However, if only the plant estimates are of interest and the experiment is performed in open loop, the noise model can be over-parameterized without affecting the asymptotic properties of the plant. The limitation is that, as PEM suffers in general from non-convexity, estimating an unnecessarily large number of parameters will increase the risk of getting trapped in local minima. Here, we consider the following alternative approach. First, estimate a high-order ARX model with least squares, providing non-parametric estimates of the plant and noise model. Second, reduce the high-order model to obtain a parametric model of the plant only. We review existing methods to do this, pointing out limitations and connections between them. Then, we propose a method that connects favorable properties from the previously reviewed approaches. We show that the proposed method provides asymptotically efficient estimates of the plant with open-loop data. Finally, we perform a simulation study suggesting that the proposed method is competitive with state-of-the-art methods.

A weighted least squares method for estimation of unstable systems

Estimating unstable systems typically requires additional system identification techniques. In this paper, we consider the weighted null-space fitting method, a three step method that is asymptotically efficient for stable systems. This method first estimates a high order ARX model and then reduces it to a structured model with lower variance using weighted least squares. However, with unstable systems, the method cannot be used to simultaneously estimate the stable and unstable poles. To solve this, we observe that the unstable poles can be estimated from the high order ARX model with relative high accuracy, and use this as an estimate for the unstable poles of the model of interest. Then, the remaining parameters in this model can be estimated by weighted least squares. Because the complete set of parameters is not estimated jointly, asymptotic efficiency is lost. Nevertheless, a simulation study shows good performance.