Yan-Lin He

A hierarchical structure of extreme learning machine (HELM) for high-dimensional datasets with noise

Extreme Learning Machine (ELM), a competitive machine learning technique for single-hidden-layer feedforward neural networks (SLFNNs), is simple in theory and fast in implementation. To deal with high-dimensional data with noise, ELM with a hierarchical structure (HELM) is proposed in this paper. The proposed HELM consists of two parts: some groups of subnets and a main net. The subnets are based on some well-trained auto-associative neural networks (AANNs), which can reduce dimension and filter out noise. The main net is based on the traditional ELM. Additionally, from the perspective of data attributes spaces (DASs), the difficulties in designing subnets are avoided by using a method of Data Attributes Extension Classification (DAEC). Experiments on five high-dimensional datasets with noise are carried out to examine the HELM model. Experimental results show that HELM has higher accuracy with fewer neurons in the main net than ELM.

A data-attribute-space-oriented double parallel (DASODP) structure for enhancing extreme learning machine

Extreme learning machine (ELM), a simple single-hidden-layer feed-forward neural network with fast implementation, has been successfully applied in many fields. This paper proposes an ELM with a constructional structure (CS-ELM) for improving the performance of ELM in dealing with regression problems. In the CS-ELM, there are some partial input subnets (PISs). The first step in designing the PISs is to divide the data-attribute-space into several sub-spaces through using an improved extension clustering algorithm (IECA). The input data attributes in the same sub-space can build a PIS and the similar information of the data attributes is stored in the corresponding PIS. Additionally, a double parallel structure is applied in the CS-ELM, in which there is a special channel that directly connects the input layer neurons to the output layer neurons. In this regard, the proposed procedure can be called ELM with a data-attribute-space-oriented double parallel (DASODP) structure (DASODP-ELM). To test the validity of the proposed method, it is applied to 4 regression applications. The experimental results indicate that, compared with ELM, DASODP-ELM with less number of parameters can achieve higher regression precision in the generalization phase.

Positive and negative correlation input attributes oriented subnets based double parallel extreme learning machine (PNIAOS-DPELM) and its application to monitoring chemical processes in steady state

Extreme learning machine (ELM) is an effective learning algorithm for single-hidden-layer feed-forward neural networks (SLFNNs). Due to its easiness in theory and implementation, ELM has been widely used in many fields. In order to further enhance the generalization performance of ELM, a positive and negative correlation input attributes oriented subnets based double parallel extreme learning machine (PCNCIAOS-DPELM) is proposed in this paper. A salient feature in the PNIAOS-DPELM is that there are two special subnets. In one of the two subnets, the input attributes have a positive correlation to the outputs. In another subnet, the input attributes have a negative correlation to the outputs. The two kinds of input attributes can be obtained by separating the input attributes into two categories using the correlation coefficient analysis. Then according to the categories, the two subnets can be established. The two subnets are based on well-trained auto-associative neural networks (AANNs), which can extract the nonlinear information of the input attributes and remove the redundant information. An advantage in PNIAOS-DPELM is that the proper number of the nodes in the hidden layer can be determined. To test the validity of PNIAOS-DPELM, it is applied to monitoring three chemical processes in steady state. Meanwhile, ELM, double parallel ELM (DP-ELM), and ELM with kernel (ELMK) were developed for comparisons. Experimental results demonstrated that PNIAOS-DPELM could achieve better regression precision and have better stable ability than ELM, DP-ELM, and ELMK did during the generalization phase.

A PSO based virtual sample generation method for small sample sets

In the early period of process industries, it is an intractable challenge to build an accurate and robust forecasting model using the collected scared samples. The information derived from small sample sets is unreliable and weak. Thus, the models established based on the small sample sets are inefficient. Virtual sample generation (VSG) is a promising technology which can be used to generate plenty of new virtual samples by the information acquired from small sample sets, aiming at improving the accuracy of forecasting models. To capture the tendency of the raw sample set and reduce information gaps among individuals, an information-expanded function based on triangular membership (TMIE) is developed to asymmetrically expand the domain range in each attribute in this paper. A novel particle swarm optimization based VSG (PSOVSG) approach is proposed to iteratively generate the most feasible virtual samples over the search-space. The effectiveness of PSOVSG is tested against other three methods of VSG over two real cases: multi-layer ceramic capacitors (MLCC) and purified Terephthalic acid (PTA). The simulation results show the proposed PSOVSG achieves better performance than other methods.

A robust hybrid model integrating enhanced inputs based extreme learning machine with PLSR (PLSR-EIELM) and its application to intelligent measurement

In this paper, a robust hybrid model integrating an enhanced inputs based extreme learning machine with the partial least square regression (PLSR-EIELM) was proposed. The proposed PLSR-EIELM model can overcome two main flaws in the extreme learning machine (ELM), i.e. the intractable problem in determining the optimal number of the hidden layer neurons and the over-fitting phenomenon. First, a traditional extreme learning machine (ELM) is selected. Second, a method of randomly assigning is applied to the weights between the input layer and the hidden layer, and then the nonlinear transformation for independent variables can be obtained from the output of the hidden layer neurons. Especially, the original input variables are regarded as enhanced inputs; then the enhanced inputs and the nonlinear transformed variables are tied together as the whole independent variables. In this way, the PLSR can be carried out to identify the PLS components not only from the nonlinear transformed variables but also from the original input variables, which can remove the correlation among the whole independent variables and the expected outputs. Finally, the optimal relationship model of the whole independent variables with the expected outputs can be achieved by using PLSR. Thus, the PLSR-EIELM model is developed. Then the PLSR-EIELM model served as an intelligent measurement tool for the key variables of the Purified Terephthalic Acid (PTA) process and the High Density Polyethylene (HDPE) process. The experimental results show that the predictive accuracy of PLSR-EIELM is stable, which indicate that PLSR-EIELM has good robust character. Moreover, compared with ELM, PLSR, hierarchical ELM (HELM), and PLSR-ELM, PLSR-EIELM can achieve much smaller predicted relative errors in these two applications.

Hybrid robust model based on an improved functional link neural network integrating with partial least square (IFLNN-PLS) and its application to predicting key process variables.

In this paper, a hybrid robust model based on an improved functional link neural network integrating with partial least square (IFLNN-PLS) is proposed. Firstly, an improved functional link neural network with small norm of expanded weights and high input-output correlation (SNEWHIOC-FLNN) was proposed for enhancing the generalization performance of FLNN. Unlike the traditional FLNN, the expanded variables of the original inputs are not directly used as the inputs in the proposed SNEWHIOC-FLNN model. The original inputs are attached to some small norm of expanded weights. As a result, the correlation coefficient between some of the expanded variables and the outputs is enhanced. The larger the correlation coefficient is, the more relevant the expanded variables tend to be. In the end, the expanded variables with larger correlation coefficient are selected as the inputs to improve the performance of the traditional FLNN. In order to test the proposed SNEWHIOC-FLNN model, three UCI (University of California, Irvine) regression datasets named Housing, Concrete Compressive Strength (CCS), and Yacht Hydro Dynamics (YHD) are selected. Then a hybrid model based on the improved FLNN integrating with partial least square (IFLNN-PLS) was built. In IFLNN-PLS model, the connection weights are calculated using the partial least square method but not the error back propagation algorithm. Lastly, IFLNN-PLS was developed as an intelligent measurement model for accurately predicting the key variables in the Purified Terephthalic Acid (PTA) process and the High Density Polyethylene (HDPE) process. Simulation results illustrated that the IFLNN-PLS could significant improve the prediction performance.

Research and application of KICA-AROMF based fault diagnosis

With the development of the modern industrial system, data-driven fault diagnosis methods have attracted more and more attention. Fault diagnosis of complex industrial processes based on one-dimensional adaptive rank-order morphological filter (AROMF) may miss key information because of excessive dimension reduction of process data. To solve this problem, a method combining the kernel independent component analysis (KICA) with one-dimensional AROMF is proposed. Firstly, KICA is used for nonlinear feature extraction, getting the template signal and the test signal of each pattern. Then, a fault diagnosis method via multi-dimensional signals classification method based on AROMF is presented in this paper. The advantage of the proposed method was confirmed by the simulation of the Tennessee Eastman process.

PID control loop performance assessment and diagnosis based on DEA-related MCDA

Control loop performance assessment and diagnosis have been attracting more and more attention in the academia and industry. Both traditional performance assessment method and minimum variance method often require the process model and provide limited information, which is not particularly convenient for practical applications. Therefore, the method based on data envelopment analysis (DEA)-related multiple criteria decision analysis (MCDA) is developed for assessing and diagnosing PID control loop performance, which relies solely upon the collected process data during routine plant operation. The control loop performance is assessed and sorted by utilizing the self-evaluation DEA-related MCDA model. The operation priority of the control loop is ranked and determined by utilizing the cross-evaluation DEA-related MCDA model. The improving direction and quantitative space of control loop performance can be diagnosed by DEA-related MCDA model with slack variables and non-Archimedean infinitesimal ε. The correctness and effectiveness of the proposed method are confirmed and validated by simulation examples.

A bootstrap based virtual sample generation method for improving the accuracy of modeling complex chemical processes using small datasets

Though in the era of big data, it remains a challenge to be tackled that the forecasting model with high accuracy and robustness needs to be built using small size samples. One effective tool of addressing this problem is the virtual sample generation (VSG), which can generate a mass of new virtual samples on the basis of small sample sets. The bootstrap method is adopted to feasibly resample the virtual samples in this paper. The effectiveness of the proposed bootstrap virtual sample generation (BVSG) is evaluated over one real case. The experimental results show that the proposed approach achieves better performance with the aid of virtual samples.