Punyaphol Horata

Robust extreme learning machine

The output weights computing of extreme learning machine (ELM) encounters two problems, the computational and outlier robustness problems. The computational problem occurs when the hidden layer output matrix is a not full column rank matrix or an ill-conditioned matrix because of randomly generated input weights and biases. An existing solution to this problem is Singular Value Decomposition (SVD) method. However, the training speed is still affected by the large complexity of SVD when computing the Moore-Penrose (MP) pseudo inverse. The outlier robustness problem may occur when the training data set contaminated with outliers then the accuracy rate of ELM is extremely affected. This paper proposes the Extended Complete Orthogonal Decomposition (ECOD) method to solve the computational problem in ELM weights computing via ECODLS algorithm. And the paper also proposes the other three algorithms, i.e. the iteratively reweighted least squares (IRWLS-ELM), ELM based on the multivariate least-trimmed squares (MLTS-ELM), and ELM based on the one-step reweighted MLTS (RMLTS-ELM) to solve the outlier robustness problem. However, they also encounter the computational problem. Therefore, the ECOD via ECODLS algorithm is also used successfully in the three proposed algorithms. The experiments of regression problems were conducted on both toy and real-world data sets. The outlier types are one-sided and two-sided outliers. Each experiment was randomly contaminated with outliers, of one type only, with 10%, 20%, 30%, 40%, and 50% of the total training data size. Meta-metrics evaluation was used to measure the outlier robustness of the proposed algorithms compared to the existing algorithms, i.e. the minimax probability machine regression (MPMR) and the ordinary ELM. The experimental results showed that ECOD can effectively replace SVD. The ECOD is robust to the not full column rank or the ill-conditional problem. The speed of the ELM training using ECOD is also faster than the ordinary training algorithm. Moreover, the meta-metrics measure showed that the proposed algorithms are less affected by the increasing number of outliers than the existing algorithms.

Multi-label classification with extreme learning machine

Extreme learning machine (ELM) is a well-known algorithm for single layer feedforward neural networks (SLFNs) and their learning speed is faster than traditional gradient-based neural networks. However, many of the tasks that ELM focuses on are single-label, where an instance of the input set is associated with one label. This paper proposes a new method for training ELM that will be capable of multi-label classification using the Canonical Correlation Analysis (CCA). The new method is named CCA-ELM. There are 4 steps in the training process: the first step is to compute any correlations between the input features and the set of labels using CCA, the second step maps the input space and label space to the new space, the third step uses ELM to classify and the last step is to map to the original input space. The experimental results show that CCA-ELM can improve ELM for classification on multi-label learning and its recognition performances are better than the other comparative algorithms that use the same standard CCA.

Handwritten Character Recognition Using Histograms of Oriented Gradient Features in Deep Learning of Artificial Neural Network

Feature extraction plays an essential role in hand written character recognition because of its effect on the capability of classifiers. This paper presents a framework for investigating and comparing the recognition ability of two classifiers: Deep-Learning Feedforward-Backpropagation Neural Network (DFBNN) and Extreme Learning Machine (ELM). Three data sets: Thai handwritten characters, Bangla handwritten numerals, and Devanagari handwritten numerals were studied. Each data set was divided into two categories: non-extracted and extracted features by Histograms of Oriented Gradients (HOG). The experimental results showed that using HOG to extract features can improve recognition rates of both of DFBNN and ELM. Furthermore, DFBNN provides higher slightly recognition rates than those of ELM.

Evolutionary Circular Extreme Learning Machine

Circular Extreme Learning Machine (C-ELM) is an extension of Extreme Learning Machine. Its power is mapping both linear and circular separation boundaries. However, C-ELM uses the random determination of the input weights and hidden biases, which may lead to local optimal. This paper proposes a hybrid learning algorithms based on the C-ELM and the Differential Evolution (DE) to select appropriate weights and hidden biases. It called Evolutionary circular extreme learning machine (EC-ELM). From experimental results show EC-ELM can slightly improve C-ELM and also reduce the number of nodes network.

Evolutionary Circular-ELM for the Reduced-Reference Assessment of Perceived Image Quality

At present, the quality of the image is very important. The audience needs to get the undistorted image like the original image. Cause of the loss of image quality such as storage, transmission, compression and rendering. The mechanisms rely on systems that can assess the visual quality with human perception are required. Computational Intelligence (CI) paradigms represent a suitable technology to solve this challenging problem. In this paper present, the Evolutionary Extreme Learning Machine (EC-ELM) is derived into Circular-ELM (C-ELM) that is an extended Extreme Learning Machine (ELM) and the Differential Evolution (DE) to select appropriate weights and hidden biases, which can proves performance in addressing the visual quality assessment problem by embedded in the proposed framework. The experimental results, the EC-ELM can map the visual signals into quality score values that close to the real quality score than ELM, Evolutionary Extreme Learning (E-ELM) and the original C-ELM and also stable as well. Its can confirms that the EC-ELM is proved on recognized benchmarks and for four different types of distortions.

Enhancement of online sequential extreme learning machine based on the householder block exact inverse QRD recursive least squares

The online sequential extreme learning machine (OS-ELM) has been used for training without retraining the ELM when a chunk of data is received. However, OS-ELM may be affected by an improper number of hidden nodes settings which reduces the generalization of OS-ELM. This paper addresses this problem in OS-ELM. A new structural tolerance OS-ELM (STOS-ELM), based on the Householder block exact inverse QRD recursive least squares algorithm having numerical robustness is proposed. Experimental results conducted on four regressions and five classification problems showed that STOS-ELM can handle the situation when the network is constructed with an improper number of hidden nodes. Accordingly, the proposed STOS-ELM can be easily applied; the size of the hidden layer of ELM can be roughly approximated. If a chunk of data is received, it can be updated in the existing network without having to worry about the proper number of given hidden nodes. Furthermore, the accuracy of the network trained by STOS-ELM is comparable to that of the batch ELM when the networks have the same configurations. STOS-ELM can also be applied in ensemble version (ESTOS-ELM). We found that the stability of STOS-ELM can be further improved using the ensemble technique. The results show that ESTOS-ELM is also more stable and accurate than both of the original OS-ELM and EOS-ELM, especially in the classification problems.

Applying Regularization Least Squares Canonical Correlation Analysis in Extreme Learning Machine for Multi-label Classification Problems

Multi-label classification is a type of classification where each instance is associated with a set of labels. Many methods such as BP-MLL, rank-SVM, and MLRBF have been proposed for multi-label classification but their learning abilities are too slow. Extreme Learning Machine (ELM) is a well known algorithm for SLFNs that can learn faster than the traditional gradient-base neural networks and it also provides better generalization performance. However, the classification performance of ELM involving multi-label classification may not be good enough despite its advantage in fast training. Therefore, this paper proposes two multi-label classification approaches in ELM. The first approach uses the 1-norm regularized Least-square for Canonical Correlation Analysis (1-norm LSCCA) to obtain the projection vectors, which in turn uses the vectors to provide the new information. Then, ELM is then used to learn this new information in the new space. The second approach applies the ensemble method to the first approach to reduce the random effects of ELM. The experimental results show that the two proposed methods can improve the performance of ELM in multi-label classification and are also faster than the previous multi-label classification methods.

License plate recognition application using extreme learning machines

Recording a car license plate is an important task for police officers or security officers to check the car of interest. However, manually recording these plates comes with problems. It is easy to make a mistake, or it can be lost. The Extreme Learning Machine (ELM) can classify the plates faster and it is a more accurate system. Therefore, this paper proposes a new license plate recognition system using ELM. The proposed system is composed of two parts: the first is a mobile application to take a picture of the car license plate, and the second is the recognition system using ELM. The recognition system entails two parts: the first is to preprocess and extract features using the histogram of oriented gradients (HOG). The second part is to classify each number and each of the Thai alphabet letters that appear on the car license plates. Also, the system will classify provinces of each plate. The results of the experiment show that the testing recognition rate when trained with 200 hidden nodes is 89.05% while the rate of correctly recognized plates is 252 out of 283 plates.

Improved convex incremental extreme learning machine based on ridgelet and PSO algorithm

The most difficult problem with the extreme learning machine is the selection of the hidden nodes size. The proper number of hidden nodes is predefined through a trial and error approach. The convex incremental extreme learning machine (CI-ELM) has been proposed to tackle this problem. CI-ELM is an incremental constructive neural network with universal approximation abilities. However, we have found that some hidden nodes added into a hidden layer, may play a minor role in the network, which results in an increase in network complexity. In order to avoid this shortcoming, we propose here in an improved convex incremental extreme learning machine with optimal ridgelet hidden nodes (ICOR-ELM). The proposed method uses the ridgelet function as the activation function within the hidden layer. In each step of the learning process, the optimal hidden node parameters, which are optimized through particle swarm optimization (PSO), are added to the existing hidden layer. Experimental results prove that the proposed method can achieve greater generalization performance with more compact architecture than other methods, and demonstrates faster convergence than other incremental ELM methods.

Ensemble Extreme Learning Machine for Multi-instance Learning

Multi-instance learning (MIL) is a classification approach for classifying on a collection of instances which each group is represented as a bag. The main task of MIL is to learn from labels and features of instances to produce a model to predict a label of a testing bag. Traditional MIL algorithms were proposed to address the MIL problem, but most of the algorithms take a large time scale for their training process since they have to computing the parameter tuning. To address the learning time problem, the multi-instance learning method based on extreme learning machine (ELM-MIL) was proposed. However, the randomly generated parameters of ELM-MIL may reduce its generalization performance. Therefore, we proposed a new method to improve the generalization performance of the ELM-MIL which the new method is based on the ensemble with majority voting approach named the ensemble extreme learning machine for multi-instance learning (E-ELM-MIL). To evaluate the new method, several benchmark datasets were studied in this paper. From experimental results show that E-ELM-MIL outperforms ELM-MIL and the other state of the art MIL algorithm.