J Hao

Cellular neural networks as a model of associative memories

Concerns the design of cellular neural networks intended to function as associative memories. The authors consider a discrete-time version of cellular neural nets featuring simple linear thresholding neurons and the synchronous state-updating rule. The Hebbian rule is adopted as the memory design rule. Important issues, such as the memory capacity and the size of the attracting basin, are discussed. The validity of the method is illustrated by a simple example.<<ETX>>

Efficient identification of rbf neural-net models for nonlinear discrete-time multivariable dynamical-systems

Abstract In this paper, we propose a recursive identification technique for nonlinear discrete-time multivariable dynamical systems. Extending an early result to multivariable systems [15], the technique approaches a nonlinear system identification problem in two stages: One is to build up recursively a RBF (Radial-Basis-Function) neural net model structure including the size of the neural net and the parameters in the RBF neurons; the other is to design a stable recursive weight updating algorithm to obtain the weights of the net in an efficient way. Heuristics are employed to analyze the effect of RBF net parameters to the error of identification, leading to a simple but effective means to establish these parameters. The weight updating algorithm is developed based on ideas in the theory of adaptive control. Key stability results are proved in the paper along with illustrative examples to show the effectiveness of applying such a technique and other practical considerations.

A RULE-BASED NEURAL CONTROLLER FOR INVERTED PENDULUM SYSTEM

This paper tries to demonstrate how a heuristic neural control approach can be used to solve a complex nonlinear control problem. The control task is to swing up a pendulum mounted on a cart from its stable position (vertically down) to the zero state (up right) and keep it there by applying a sequence of two opposing constant forces of equal magnitude to the mass center of the cart. In addition, the displacement of the cart itself is confined to within a preset limit during the swinging up action and it will eventually be brought to the origin of the track. This is truly a nontrivial nonlinear regulation problem and is considerably difficult compared to the pendulum balancing problem (and its variations) widely adopted as a benchmarking test system for neural controllers. Through the solution of this specific control problem, we try to illustrate a heuristic neural control approach with task decomposition, control rule extraction and neural net rule implementation as its basic elements. Specializing to the pendulum problem, the global control task is decomposed into subtasks namely pendulum positioning and cart positioning. Accordingly, three separate neural subcontrollers are designed to cater to the subtasks and their coordination, i.e., pendulum subcontroller (PSC), cart subcontroller (CSC) and the switching subcontroller (SSC). Each of the subcontrollers is designed based on the rules and guidelines obtained from the experiences of a human operator. The simulation result is included to show the actual performance of the controller.

A new model of neural associative memories

A novel model of discrete neural associative memories is presented. The most important feature of this model is that static mapping instead of the dynamic convergent process is used to retrieve the stored messages. The model features a two-layer structure, with feedforward connections only and using two kinds of neurons. This model uses an extremely simple weight set-up rule and all the resulting weights can only be -1 or +1. Compared to the Hopfield model, the model can guarantee all the given patterns to be stored as fixed points. Each fixed point is surrounded by an attraction ball with the maximum possible radius. The processing speed is much higher because of the use of layered feedforward nets. The model is flexible in the sense that extra patterns can be easily incorporated into the established net.<<ETX>>

A rule-based neural controller for inverted pendulum system

It is demonstrated how a heuristic neural control approach can be used to solve a complex nonlinear control problem. As well as swinging up the pendulum, the controller is required to bring the cart back to the origin of the track. Through the solution of this specific control problem, a heuristic neural control approach with task decomposition, control rule extraction and neural net rule implementation as its basic elements is illustrated. Specializing to the pendulum problem, the global control task is decomposed into sub-tasks, namely, pendulum positioning and cart positioning. Three separate neural sub-controllers are designed to cater to the sub-tasks and their coordination. The simulation result is provided to show the actual performance of the controller.<<ETX>>

Determination of weights for Hopfield associative memory by error back propagation

A systematic approach to designing discrete Hopfield associative memories is described. The underlying idea is to translate such a design problem into solving a set of linear inequalities. Along with the consideration of the consistency, this later problem is further formulated into determining the weights of a feedforward net, which can then be solved by applying the well-known error back-propagation method. The approach is illustrated by design examples.<<ETX>>

A Geometric Approach to the Structural Synthesis of Multilayer Perceptron Neural Networks

Designing a multilayer perceptron for general purpose classification has important practical implications. Since the capacity of multilayer perceptron to realize arbitrary dichotomies (or two-class classifications) is limited, the most important step in a design procedure is the determination of the number of the layers and the amount of nodes in each layer apart from the determination of the weights and the threshold values. Unfortunately, there has been no general principle or guideline available for such a synthesis task, normal design often proceeds on an ad hoc and empirical basis, the methods generally lead to the structure which only deals with a particular classification problem [1] [2].

A NEW MODEL OF NEURAL ASSOCIATIVE MEMORIES

In this paper, we present a new model of discrete neural associative memories and its design rule. The most important feature of this new model is that a static mapping instead of the dynamic convergent process is used to retrieve the stored messages. The new model features a two-layer structure, with feedforward connections only and uses two kinds of neurons which implement different output functions. Another important feature is that this new model employs an extremely simple weight setup rule and all the resulted weights can only assume two different values, -1 and +1, which facilitates the VLSI implementation. Compared to the famous discrete Hopfield model designed with the well-known Hebbian rule or any other rule, the new model can guarantee all the given patterns to be stored as fixed points. Moreover, each fixed point is surrounded by an attraction basin (which is a ball in the Hamming distance sense) with the maximal possible radius. The performances of the new model are compared through some illustrative examples with those of the Hopfield associative memory designed using different methods.

Predictive control of nonlinear systems based on identification by backpropagation networks.

Using the property of universal approximation of multilayer perceptron neural network, a class of discrete nonlinear dynamical systems are modeled by a perceptron with two hidden layers. A backpropagation algorithm is then used to train the model to identify the nonlinear systems to a desired level of accuracy. Based on the identified model, a one-step-ahead predictive control scheme is proposed in which the future control inputs are obtained through some nonlinear optimization process. Making use of the online learning properties of neural networks, the predictive control scheme is further developed into an adaptive one which is robust to the incompleteness of identification. Simulation results show that this neural control scheme works well even for some very complicated nonlinear systems.

Pattern storage and Hopfield neural associative memory with hidden structure

This paper is concerned with the formulation of neural associative memories. Centered around the fundamental issue of the memory storage, we examine the deficiencies associated with the standard Hopfield net. To overcome the problems, we pursue a data-driven design approach by modifying the configuration of the Hopfield net to allow hidden structures. As important results, we show how the well-known sum-of-outer product rule can be utilized to explore the freedom provided by the hidden structures leading to the desired memory performance.