Hai-Jun Rong

A fast pruned-extreme learning machine for classification problem

Extreme learning machine (ELM) represents one of the recent successful approaches in machine learning, particularly for performing pattern classification. One key strength of ELM is the significantly low computational time required for training new classifiers since the weights of the hidden and output nodes are randomly chosen and analytically determined, respectively. In this paper, we address the architectural design of the ELM classifier network, since too few/many hidden nodes employed would lead to underfitting/overfitting issues in pattern classification. In particular, we describe the proposed pruned-ELM (P-ELM) algorithm as a systematic and automated approach for designing ELM classifier network. P-ELM uses statistical methods to measure the relevance of hidden nodes. Beginning from an initial large number of hidden nodes, irrelevant nodes are then pruned by considering their relevance to the class labels. As a result, the architectural design of ELM network classifier can be automated. Empirical study of P-ELM on several commonly used classification benchmark problems and with diverse forms of hidden node functions show that the proposed approach leads to compact network classifiers that generate fast response and robust prediction accuracy on unseen data, comparing with traditional ELM and other popular machine learning approaches.

Online Sequential Fuzzy Extreme Learning Machine for Function Approximation and Classification Problems

In this correspondence, an online sequential fuzzy extreme learning machine (OS-fuzzy-ELM) has been developed for function approximation and classification problems. The equivalence of a Takagi-Sugeno-Kang (TSK) fuzzy inference system (FIS) to a generalized single hidden-layer feedforward network is shown first, which is then used to develop the OS-fuzzy-ELM algorithm. This results in a FIS that can handle any bounded nonconstant piecewise continuous membership function. Furthermore, the learning in OS-fuzzy-ELM can be done with the input data coming in a one-by-one mode or a chunk-by-chunk (a block of data) mode with fixed or varying chunk size. In OS-fuzzy-ELM, all the antecedent parameters of membership functions are randomly assigned first, and then, the corresponding consequent parameters are determined analytically. Performance comparisons of OS-fuzzy-ELM with other existing algorithms are presented using real-world benchmark problems in the areas of nonlinear system identification, regression, and classification. The results show that the proposed OS-fuzzy-ELM produces similar or better accuracies with at least an order-of-magnitude reduction in the training time.

Extreme learning machine for multi-categories classification applications

In the paper, the multi-class pattern classification using extreme learning machine (ELM) is studied. The study is based on either a series of ELM binary classifiers or a single ELM classifier. When using binary ELM classifiers, the multi-class problem is decomposed into two-class problem using the one-against-all (OAA) and one-against-one (OAO) schemes, which are named as ELM-OAA and ELM-OAO respectively for brevity. In a single ELM classifier, the multi-class problem is implemented with an architecture of multi-output nodes which is equal to the number of pattern classes. Their performance is evaluated using some multi-class benchmark problems and simulation results show that ELM-OAA and ELM-OAO requires fewer hidden nodes than the single ELM classifier. In addition ELM-OAO usually has similar or less computation burden than the single ELM classifier when the pattern class labels is not larger than 10.

Stable indirect adaptive neural controller for a class of nonlinear system

Abstract The paper presents an indirect adaptive neural control scheme for a general high-order nonlinear continuous system. In the proposed scheme a neural controller is constructed based on the single-hidden layer feedforward network (SLFN) for approximating the unknown nonlinearities of dynamic systems. A sliding mode controller is also incorporated to compensate for the modelling errors of SLFN. The parameters of the SLFN are modified using the recently proposed neural algorithm named extreme learning machine (ELM), where the parameters of the hidden nodes are assigned randomly. However different from the original ELM algorithm, the output weights are updated based on the Lyapunov synthesis approach to guarantee the stability of the overall control system, even in the presence of modelling errors which are offset using the sliding mode controller. Finally the proposed adaptive neural controller is applied to control the inverted pendulum system with two different reference trajectories. The simulation results demonstrate that good tracking performance is achieved by the proposed control scheme.

Adaptive fuzzy fault-tolerant controller for aircraft autolanding under failures

This paper presents an adaptive fuzzy control strategy for an aircraft automatic landing problem under the failures of stuck control surfaces and severe winds. The strategy incorporates a dynamic fuzzy system called sequential adaptive fuzzy inference system (SAFIS) and it augments an existing conventional controller called baseline trajectory following controller (BTFC). SAFIS is an online learning fuzzy system in which the rules are added or deleted based on the input data. Also, SAFIS incorporates an online scheme for parameter update of the membership functions. BTFC has been designed using classical control methods under normal operating conditions with winds. For this study, the following fault scenarios have been considered: 1) single fault of either aileron or elevator stuck at certain deflections, and 2) double fault cases where one aileron and one elevator at the same or opposite direction are stuck at different deflections. Simulation studies indicate that the BTFC is unable to handle these failures. Recently, Abhay et al. have proposed a neural-based scheme to augment the BTFC and its performance has been shown to be superior. However, even in this neural scheme there are gaps in the fault cases where performance specifications are not met. In this paper, results show that the SAFIS-aided BTFC improves the fault-tolerant capabilities compared with BTFC and also the earlier neural-aided BTFC performance in filling up the gaps observed earlier.

Adaptive fuzzy control of aircraft wing-rock motion

In the paper, two adaptive fuzzy control schemes including indirect and direct frameworks are developed for suppressing the wing-rock motion that is a highly nonlinear aerodynamic phenomenon in which limit cycle roll oscillations are experienced by aircraft at high angles of attack. In the two control topologies, a dynamic fuzzy system called Extended Sequential Adaptive Fuzzy Inference System (ESAFIS) is constructed to represent the dynamics of the wing-rock system. ESAFIS is an online learning fuzzy system in which the rules are added or deleted based on the input data. In the indirect control scheme, the ESAFIS is used to estimate the nonlinear dynamic function and then a stable indirect fuzzy controller is designed based on the estimator. In the direct control scheme, the ESAFIS controller is directly designed to imitate an ideal stable control law without determining the model of the dynamic function. Different from the original ESAFIS, the adaptive tuning algorithms for the consequent parameters are established in the sense of Lyapunov theorem to ensure the stability of the overall control system. A sliding mode controller is also designed to compensate for the modelling errors of ESAFIS by augmenting the indirect/direct fuzzy controller. Finally, comparisons with a neuron control scheme using the RBF network and a fuzzy control scheme with Takagi-Sugeno (TS) system are presented to depict the effectiveness of the proposed control strategies. Simulation results show that the proposed fuzzy controllers achieve better tracking performance with dynamically allocating the rules online.

Adaptive neural control for a class of MIMO nonlinear systems with extreme learning machine

This paper presents two adaptive neural control schemes for a class of uncertain continuous-time multi-input multi-output (MIMO) nonlinear dynamic systems. Within these schemes, the single-hidden layer feedforward networks (SLFNs) are applied to approximate the unknown nonlinear functions of the systems and then the neural controller is built based on the approximated neural models. The parameters of the SLFNs are modified using the recently proposed neural algorithm named extreme learning machine (ELM), where the parameters of the hidden nodes are assigned randomly. Different from the original ELM algorithm, the output weights are updated using the adaptive laws derived based on the Lyapunov stability theorem and Barbalats lemma so that the asymptotical stability of the system can be guaranteed. The robustifying control term is also constructed to compensate for approximation errors of the SLFNs. In order to avoid the requirement of the approximation error bounds, the estimation laws derived based on the Lyapunov stability theorem and Barbalats lemma are employed to estimate the error bounds in the second adaptive control scheme. Finally the proposed control schemes are applied to control a two-link robot manipulator. The simulation results demonstrate the effectiveness of the proposed control schemes for the MIMO nonlinear system.

Direct adaptive neural control of nonlinear systems with extreme learning machine

A direct adaptive neural control scheme for a class of nonlinear systems is presented in the paper. The proposed control scheme incorporates a neural controller and a sliding mode controller. The neural controller is constructed based on the approximation capability of the single-hidden layer feedforward network (SLFN). The sliding mode controller is built to compensate for the modeling error of SLFN and system uncertainties. In the designed neural controller, its hidden node parameters are modified using the recently proposed neural algorithm named extreme learning machine (ELM), where they are assigned random values. However, different from the original ELM algorithm, the output weight is updated based on the Lyapunov synthesis approach to guarantee the stability of the overall control system. The proposed adaptive neural controller is finally applied to control the inverted pendulum system with two different reference trajectories. The simulation results demonstrate good tracking performance of the proposed control scheme.

Aircraft recognition using modular extreme learning machine

In this paper, a novel recognition scheme is proposed for identifying the aircrafts of different types based on multiple modular neural network classifiers. Three moment invariants including Hu moments, Zernike moments and Wavelet moments are extracted from the characteristics exhibited by aircrafts and used as the input variables of each modular neural network respectively. Each modular neural network consists of multiple single-hidden layer feedforward networks which are trained using the extreme learning machine and different clustering data subsets. A clustering and selection method is used to get the classification rate of each modular neural network and then based on their weighted sum the final classification output is obtained. The proposed recognition scheme is finally evaluated by recognizing six different types of aircraft models and the simulation results show the superiority of the proposed method compared with the single ELM classifier and other classification algorithms.

Fuzzy Fault Tolerant Controller for Actuator Failures during Aircraft Autolanding

This paper presents a fuzzy control strategy for aircraft autolanding under the failures of stuck control surfaces and severe winds. The control strategy incorporates a TSK fuzzy neural network implementing TSK fuzzy model and it aids an existing conventional controller called a Baseline Trajectory Following Controller (BTFC). The TSK fuzzy neural network is trained by the Online Sequential Fuzzy Extreme Learning Machine (Fuzzy-ELM) algorithm. In Fuzzy-ELM algorithm the parameters of fuzzy membership functions need not be adjusted during training and one may simply randomly assign values to them. Performance of the proposed fuzzy control scheme is evaluated for a typical aircraft autolanding with a double failure of left elevator and left aileron stuck at different deflections. The results indicate superior performance of the proposed fuzzy fault tolerant controller.