Hybrid extreme learning machine approach for heterogeneous neural networks

Abstract

Abstract In this paper, a hybrid learning approach, which combines the extreme learning machine (ELM) with a genetic algorithm (GA), is proposed. The utilization of this hybrid algorithm enables the creation of heterogeneous single layer neural networks (SLNNs) with better generalization ability than traditional ELM in terms of lower mean square error (MSE) for regression problems or higher accuracy for classification problems. The architecture of this method is not limited to traditional linear neurons, where each input participates equally to the neuron’s activation, but is extended to support higher order neurons which affect the network’s generalization ability. Initially, the proposed heterogeneous hybrid extreme learning machine (He-HyELM) algorithm creates a number of custom created neurons with different structure, which are used for the creation of homogeneous SLNNs. These networks are trained with ELM and an application specific GA evolves them into heterogeneous networks according to a fitness criterion utilizing the uniform crossover operator for the recombination process. After the completion of the evolution process, the network with the best fitness is selected as the most optimal. Experimental results demonstrate that the proposed learning algorithm can get better results than traditional ELM, homogeneous hybrid extreme learning machine (Ho-HyELM) and optimally pruned extreme learning machine (OP-ELM) for homogeneous and heterogeneous SLNNs.