Ye Ren

Ensemble Classification and Regression-Recent Developments, Applications and Future Directions [Review Article]

Ensemble methods use multiple models to get better performance. Ensemble methods have been used in multiple research fields such as computational intelligence, statistics and machine learning. This paper reviews traditional as well as state-of-the-art ensemble methods and thus can serve as an extensive summary for practitioners and beginners. The ensemble methods are categorized into conventional ensemble methods such as bagging, boosting and random forest, decomposition methods, negative correlation learning methods, multi-objective optimization based ensemble methods, fuzzy ensemble methods, multiple kernel learning ensemble methods and deep learning based ensemble methods. Variations, improvements and typical applications are discussed. Finally this paper gives some recommendations for future research directions.

A Comparative Study of Empirical Mode Decomposition-Based Short-Term Wind Speed Forecasting Methods

Wind speed forecasting is challenging due to its intermittent nature. The wind speed time series (TS) has nonlinear and nonstationary characteristics and not normally distributed, which make it difficult to be predicted by statistical or computational intelligent methods. Empirical mode decomposition (EMD) and its improved versions are powerful tools to decompose a complex TS into a collection of simpler ones. The improved versions discussed in this paper include ensemble EMD (EEMD), complementary EEMD (CEEMD), and complete EEMD with adaptive noise (CEEMDAN). The EMD and its improved versions are hybridized with two computational intelligence-based predictors: support vector regression (SVR) and artificial neural network (ANN). The EMD-based hybrid forecasting methods are evaluated with 12 wind speed TS. The performances of the hybrid methods are compared and discussed. It shows that EMD and its improved versions enhance the performance of SVR significantly but marginally on ANN, and among the EMD-based hybrid methods, the proposed CEEMDAN-SVR is the best method. Possible future works are also recommended for wind speed forecasting.

Ensemble deep learning for regression and time series forecasting

In this paper, for the first time, an ensemble of deep learning belief networks (DBN) is proposed for regression and time series forecasting. Another novel contribution is to aggregate the outputs from various DBNs by a support vector regression (SVR) model. We show the advantage of the proposed method on three electricity load demand datasets, one artificial time series dataset and three regression datasets over other benchmark methods.

Empirical Mode Decomposition based ensemble deep learning for load demand time series forecasting

Graphical abstractDisplay Omitted HighlightsAn ensemble deep learning method has been proposed for load demand forecasting.The hybrid method composes of Empirical Mode Decomposition and Deep Belief Network.Empirical Mode Decomposition based methods outperform the single structure models.Deep learning shows more advantages when the forecasting horizon increases. Load demand forecasting is a critical process in the planning of electric utilities. An ensemble method composed of Empirical Mode Decomposition (EMD) algorithm and deep learning approach is presented in this work. For this purpose, the load demand series were first decomposed into several intrinsic mode functions (IMFs). Then a Deep Belief Network (DBN) including two restricted Boltzmann machines (RBMs) was used to model each of the extracted IMFs, so that the tendencies of these IMFs can be accurately predicted. Finally, the prediction results of all IMFs can be combined by either unbiased or weighted summation to obtain an aggregated output for load demand. The electricity load demand data sets from Australian Energy Market Operator (AEMO) are used to test the effectiveness of the proposed EMD-based DBN approach. Simulation results demonstrated attractiveness of the proposed method compared with nine forecasting methods.

A Novel Empirical Mode Decomposition With Support Vector Regression for Wind Speed Forecasting

Wind energy is a clean and an abundant renewable energy source. Accurate wind speed forecasting is essential for power dispatch planning, unit commitment decision, maintenance scheduling, and regulation. However, wind is intermittent and wind speed is difficult to predict. This brief proposes a novel wind speed forecasting method by integrating empirical mode decomposition (EMD) and support vector regression (SVR) methods. The EMD is used to decompose the wind speed time series into several intrinsic mode functions (IMFs) and a residue. Subsequently, a vector combining one historical data from each IMF and the residue is generated to train the SVR. The proposed EMD-SVR model is evaluated with a wind speed data set. The proposed EMD-SVR model outperforms several recently reported methods with respect to accuracy or computational complexity.

Empirical mode decomposition based adaboost-backpropagation neural network method for wind speed forecasting

Wind speed forecasting is a popular research direction in renewable energy and computational intelligence. Ensemble forecasting and hybrid forecasting models are widely used in wind speed forecasting. This paper proposes a novel ensemble forecasting model by combining Empirical mode decomposition (EMD), Adaptive boosting (AdaBoost) and Backpropagation Neural Network (BPNN) together. The proposed model is compared with six benchmark models: persistent, AdaBoost with regression tree, BPNN, AdaBoost-BPNN, EMD-BPNN and EMD-AdaBoost with regression tree. The comparisons undergoes several statistical tests and the tests show that the proposed EMD-AdaBoost- BPNN model outperformed the other models significantly. The forecasting error of the proposed model also shows significant randomness.

A hybrid ARIMA-DENFIS method for wind speed forecasting

This paper proposes a hybrid autoregressive integrated moving average - dynamic evolving neural-fuzzy inference system (ARIMA-DENFIS) model for wind speed forecasting. The theory of ARIMA, DENFIS and the hybrid of the two are discussed. The proposed model is evaluated with NDBC wind speed data and the results show that the proposed hybrid ARIMA-DENFIS model outperforms DENFIS model in most of the cases. It has comparable or better error measures than ARIMA model. In addition, when the forecasting horizon increases, the advantage of the proposed ARIMA-DENFIS model becomes more significant.

Towards generating random forests via extremely randomized trees

The classification error of a specified classifier can be decomposed into bias and variance. Decision tree based classifier has very low bias and extremely high variance. Ensemble methods such as bagging can significantly reduce the variance of such unstable classifiers and thus return an ensemble classifier with promising generalized performance. In this paper, we compare different tree-induction strategies within a uniform ensemble framework. The results on several public datasets show that random partition (cut-point for univariate decision tree or both coefficients and cut-point for multivariate decision tree) without exhaustive search at each node of a decision tree can yield better performance with less computational complexity.

Instance based random forest with rotated feature space

Random Forest is a competitive ensemble method in the field of machine learning with several advantages such as efficiency, robustness, generalization, ease of implementation, etc. This study attempts to increase the diversity among the pairwise individuals in the forest. On the other hand, we propose an instance based method to select several superior trees to perform the voting. The proposed method is evaluated on 28 datasets from the UCI Repository.

Detecting Wind Power Ramp with Random Vector Functional Link (RVFL) Network

Due to the intermittent nature of the wind, the wind speed is fluctuating. Fluctuating wind speed cause even more fluctuation in wind power generation. The sudden changes of the wind power injected into the power grid within a short time frame is known as power ramp, which can be harmful to the grid. This paper presents algorithms to detect the wind power ramps in a certain forecasting horizon. The importance and challenges of wind power ramp detection are addressed. Several different Wind power ramps are defined in this paper. A random vector functional link (RVFL) network is employed to predict the future occurrence of wind power ramp. The forecasting methods are evaluated with a real world wind power data set. The RVFL network has comparable performance as the benchmark methods: random forests (RF) and support vector machine (SVM) but it has better performance than the artificial neural network (ANN). The computation time of training and testing is also in favor of the RVFL network.