Self-organizing deep belief modular echo state network for time series prediction

Abstract

Abstract A deep belief echo state network is an effective deep learning framework for solving time series prediction problems. The suitable structure of the hidden layers determines the prediction performance of the neural network. However, a neural network structure designed using artificial experience has difficulty meeting application requirements. To address this problem, this paper proposes a self-organizing deep belief modular echo state network (SDBMESN) model for time series prediction with high accuracy. The basic framework of this model includes two parts: a deep belief network for deep feature extraction and a modular echo state network with subreservoirs for time series prediction. To find a suitable neural network structure, a neuron significance based on mutual information is designed to measure the degree of information of the neurons, and then a self-organizing mechanism is designed to realize the dynamic adjustment of the hidden layer neurons and subreservoirs. In addition, the robust loss function is used to improve the robustness of the prediction. The simulation results of nonlinear system modeling, sunspot prediction and algal bloom prediction demonstrate that the SDBMESN has good prediction performance and robustness.