Stephen A. Billings

Non-linear System Identification using Neural Networks

Multi-layered neural networks offer an exciting alternative for modelling complex non-linear systems. This paper investigates the identification of discrete-time nonlinear systems using neural networks with a single hidden layer. New parameter estimation algorithms are derived for the neural network model based on a prediction error formulation and the application to both simulated and real data is included to demonstrate the effectiveness of the neural network approach.

Representation of non-linear systems: the NARMAX model

Abstract Input-output representations of non-linear discrete-time systems are discussed. It is shown that the NARMAX (Non-linear AutoRegressive Moving Average with eXogenous inputs) model is a general and natural representation of non-linear systems and contains, as special cases, several existing non-linear models. The problem of approximating non-linear input-output systems is also addressed and several properties of non-linear models are highlighted using simple examples.

Identification of systems containing linear dynamic and static nonlinear elements

Identification of nonlinear systems which can be represented by combinations of linear dynamic and static nonlinear elements are considered. Previous results by the authors based on correlation analysis are combined to provide a unified treatment for this class of systems. It is shown that systems composed of cascade, feedforward, feedback and multiplicative connections of linear dynamic and zero memory nonlinear elements can be identified in terms of the individual component subsystems from measurements of the system input and output only.

Recursive hybrid algorithm for non-linear system identification using radial basis function networks

Recursive identification of non-linear systems is investigated using radial basis function networks. A novel approach is adopted which employs a hybrid clustering and least squares algorithm. The recursive clustering algorithm adjusts the centres of the radial basis function network while the recursive least squares algorithm estimates the connection weights of the network. Because these two recursive learning rules are both linear, rapid convergence is guaranteed and this hybrid algorithm significantly enhances the real-time or adaptive capability of radial basis function models. The application to simulated real data are included to demonstrate the effectiveness of this hybrid approach.

Practical identification of NARMAX models using radial basis functions

A wide class of discrete-time non-linear systems can be represented by the nonlinear autoregressive moving average (NARMAX) model with exogenous inputs. This paper develops a practical algorithm for identifying NARMAX models based on radial basis functions from noise-corrupted data. The algorithm consists of an iterative orthogonal-forward-regression routine coupled with model validity tests. The orthogonal-forward-regression routine selects parsimonious radial-basisTunc-tion models, while the model validity tests measure the quality of fit. The modelling of a liquid level system and an automotive diesel engine are included to demonstrate the effectiveness of the identification procedure.

Radial basis function network configuration using genetic algorithms

Abstract Most training algorithms for radial basis function (RBF) neural networks start with a predetermined network structure which is chosen either by using a priori knowledge or based on previous experience. The resulting network is often insufficient or unnecessarily complicated and an appropriate network structure can only be obtained by trial and error. Training algorithms which incorporate structure selection mechanisms are usually based on local search methods and often suffer from a high probability of being trapped at a structural local minima. In the present study, genetic algorithms are proposed to automatically configure RBF networks. The network configuration is formed as a subset selection problem. The task is then to find an optimal subset of nc terms from the Nt training data samples. Each network is coded as a variable length string with distinct integers and genetic operators are proposed to evolve a population of individuals. Criteria including single objective and multiobjective functions are proposed to evaluate the fitness of individual networks. Training based on a practical data set is used to demonstrate the performance of the new algorithms.

Non-linear systems identification using radial basis functions

This paper investigates the identification of discrete-time non-linear systems using radial basis functions. A forward regression algorithm based on an orthogonal decomposition of the regression matrix is employed to select a suitable set of radial basis function centers from a large number of possible candidates and this provides, for the first time, fully automatic selection procedure for identifying parsimonious radial basis function models of structure-unknown non-linear systems. The relationship between neural networks and radial basis functions is discussed and the application of the algorithms to real data is included to demonstrate the effectiveness of this approach.

Output frequency characteristics of nonlinear systems

Some interesting properties of output frequencies of Volterra-type nonlinear systems are particularly investigated. These results provide a very novel and useful insight into the super-harmonic and inter-modulation phenomena in output frequency response of nonlinear systems, with consideration of the effects incurred by different nonlinear components in the system. The new properties theoretically demonstrate several fundamental output frequency characteristics and unveil clearly the mechanism of the interaction (or coupling effects) between different harmonic behaviors in system output frequency response incurred by different nonlinear components. These results have significance in the analysis and design of nonlinear systems and nonlinear filters in order to achieve a specific output spectrum in a desired frequency band by taking advantage of nonlinearities. They can provide an important guidance to modeling, identification, control and signal processing by using the Volterra series theory in practice.

Variable neural networks for adaptive control of nonlinear systems

This paper is concerned with the adaptive control of continuous-time nonlinear dynamical systems using neural networks. A novel neural network architecture, referred to as a variable neural network, is proposed and shown to be useful in approximating the unknown nonlinearities of dynamical systems. In the variable neural networks, the number of basis functions can be either increased or decreased with time, according to specified design strategies, so that the network will not overfit or underfit the data set. Based on the Gaussian radial basis function (GRBF) variable neural network, an adaptive control scheme is presented. The location of the centers and the determination of the widths of the GRBFs in the variable neural network are analyzed to make a compromise between orthogonality and smoothness. The weight-adaptive laws developed using the Lyapunov synthesis approach guarantee the stability of the overall control scheme, even in the presence of modeling error(s). The tracking errors converge to the required accuracy through the adaptive control algorithm derived by combining the variable neural network and Lyapunov synthesis techniques. The operation of an adaptive control scheme using the variable neural network is demonstrated using two simulated examples.

Nonlinear Fisher discriminant analysis using a minimum squared error cost function and the orthogonal least squares algorithm

The nonlinear discriminant function obtained using a minimum squared error cost function can be shown to be directly related to the nonlinear Fisher discriminant (NFD). With the squared error cost function, the orthogonal least squares (OLS) algorithm can be used to find a parsimonious description of the nonlinear discriminant function. Two simple classification techniques will be introduced and tested on a number of real and artificial data sets. The results show that the new classification technique can often perform favourably compared with other state of the art classification techniques.