Binghuang Cai

From Zhang Neural Network to Newton Iteration for Matrix Inversion

Different from gradient-based neural networks, a special kind of recurrent neural network (RNN) has recently been proposed by Zhang for online matrix inversion. Such an RNN is designed based on a matrix-valued error function instead of a scalar-valued error function. In addition, it was depicted in an implicit dynamics instead of an explicit dynamics. In this paper, we develop and investigate a discrete-time model of Zhang neural network (termed as such and abbreviated as ZNN for presentation convenience), which is depicted by a system of difference equations. Comparing with Newton iteration for matrix inversion, we find that the discrete-time ZNN model incorporates Newton iteration as its special case. Noticing this relation, we perform numerical comparisons on different situations of using ZNN and Newton iteration for matrix inversion. Different kinds of activation functions and different step-size values are examined for superior convergence and better stability of ZNN. Numerical examples demonstrate the efficacy of both ZNN and Newton iteration for online matrix inversion.

Zhang neural network solving for time-varying full-rank matrix Moore–Penrose inverse

Zhang neural networks (ZNN), a special kind of recurrent neural networks (RNN) with implicit dynamics, have recently been introduced to generalize to the solution of online time-varying problems. In comparison with conventional gradient-based neural networks, such RNN models are elegantly designed by defining matrix-valued indefinite error functions. In this paper, we generalize, investigate and analyze ZNN models for online time-varying full-rank matrix Moore–Penrose inversion. The computer-simulation results and application to inverse kinematic control of redundant robot arms demonstrate the feasibility and effectiveness of ZNN models for online time-varying full-rank matrix Moore–Penrose inversion.

Zhang neural network without using time-derivative information for constant and time-varying matrix inversion

To obtain the inverses of time-varying matrices in real time, a special kind of recurrent neural networks has recently been proposed by Zhang et al. It is proved that such a Zhang neural network (ZNN) could globally exponentially converge to the exact inverse of a given time-varying matrix. To find out the effect of time-derivative term on global convergence as well as for easier hardware-implementation purposes, the ZNN model without exploiting time-derivative information is investigated in this paper for inverting online matrices. Theoretical results of both constant matrix inversion case and time-varying matrix inversion case are presented for comparative and illustrative purposes. In order to substantiate the presented theoretical results, computer-simulation results are shown, which demonstrate the importance of time derivative term of given matrices on the exact convergence of ZNN model to time-varying matrix inverses.

Common Nature of Learning Exemplified by BP and Hopfield Neural Networks for Solving Online a System of Linear Equations

Many computational problems widely encountered in scientific and engineering applications could finally be transformed to the online linear-equations solving. Classic numerical methods for solving linear equations include Gaussian elimination and matrix factorization methods, which are usually of O(n3) operations. Being important parallel-computational models, both BP (back propagation) and Hopfield neural networks could be exploited for solving such linear equations. BP neural network is evidently different from Hopfield neural network in terms of network definition, architecture and learning pattern. However, both of these two neural networks could have a common nature of learning (i.e., governed by the same mathematical iteration formula) during the online solution of linear equations. In addition, computer-simulation results substantiate the theoretical analysis of both BP and Hopfield neural networks for solving online such a set of linear equations.

On the Variable Step-Size of Discrete-Time Zhang Neural Network and Newton Iteration for Constant Matrix Inversion

A special kind of recurrent neural network has recently been proposed by Zhang et al for matrix inversion. Then, for possible hardware and digital-circuit realization, the corresponding discrete-time model of Zhang neural network (ZNN) is proposed for constant matrix inversion, which reduces exactly to Newton iteration when linear activation functions and constat step-size 1 are used. In this paper, a variable step-size choosing method is investigated for such a discrete-time ZNN model, in which different variable step-size rules are derived for different kinds of activation functions. For comparative purposes, the fixed step-size choosing method is presented as well. Numerical examples demonstrate the efficacy of the discrete-time ZNN model, especially when using the variable step-size method.

MATLAB Simulink Modeling of Zhang Neural Network Solving for Time-Varying Pseudoinverse in Comparison with Gradient Neural Network

A special kind of recurrent neural networks (RNN), i.e., Zhang neural networks (ZNN), has recently been proposed for online time-varying problems solving. In this paper, we generalize and investigate the Matlab Simulink modeling and verification of a ZNN model for online time-varying matrix pseudoinverse solving. Based on click-and-drag mouse operations, Simulink could be easily and conveniently used to model and simulate complicated neural systems in comparison with Matlab coding. For comparative purposes, the conventional gradient-based neural network (or termed gradient neural network, GNN) is also developed for the time-varying pseudoinverse solving. Matlab Simulink modeling results substantiate the feasibility and efficacy of ZNN on time-varying pseudoinverse solving.

Common nature of learning between BP and hopfield-type neural networks for convex quadratic minimization with simplified network models

In this paper, two different types of neural networks are investigated and employed for the online solution of strictly-convex quadratic minimization; i.e., a two-layer back-propagation neural network (BPNN) and a discrete-time Hopfield-type neural network (HNN). As simplified models, their error-functions could be defined directly as the quadratic objective function, from which we further derive the weight-updating formula of such a BPNN and the state-transition equation of such an HNN. It is shown creatively that the two derived learning-expressions turn out to be the same (in mathematics), although the presented neural-networks are evidently different from each other a great deal, in terms of network architecture, physical meaning and training patterns. Computer-simulations further substantiate the efficacy of both BPNN and HNN models on convex quadratic minimization and, more importantly, their common nature of learning.

Quadratic-programming based self-motion planning with no target-configuration assigned for planar robot arms

For better manipulability, a criterion in the form of a quadratic function is presented for the self-motion planning (SMP) of redundant manipulators with no target-configuration assigned. Such SMP scheme could automatically select the desirable configuration so that the manipulator could be more flexible and maneuverable. As physical limits generally exist in actual redundant manipulators, both joint limits and joint velocity limits are taken into consideration in the presented SMP scheme for practical purposes. Computer simulations based on two types of multi-link planar robot arms substantiate the efficiency and reliability of the presented scheme. Moreover, theoretical analysis of the presented quadratic performance index is conducted and proved via two different approaches, i.e., gradient-descent and Zhang et als neural-dynamic methods.

MATLAB Simulink Modeling and Simulation of Zhang Neural Network for Online Time-Varying Matrix Inversion

Recently, a special kind of recurrent neural networks (RNN) with implicit dynamics has been proposed by Zhang et al for online time-varying problems solving (such as time-varying matrix inversion). Such a neural-dynamic system is elegantly designed by defining a matrix-valued error function rather than the usual scalar-valued norm-based error function. Its computational error can be made decrease to zero globally and exponentially. For the final purpose of field programmable gate array (FPGA) and application-specific integrated circuit (ASIC) realization, we investigate in this paper the MATLAB Simulink modeling and simulative verification of such a Zhang neural network (ZNN). By using click-and-drag mouse operations, it is easier to model and simulate in comparison with MATLAB coding. Both convergence and robustness properties of such a ZNN model are analyzed, which substantiate the effectiveness of Zhang neural network on inverting the time-varying matrices.