Antti Sorjamaa

OP-ELM: Optimally Pruned Extreme Learning Machine

In this brief, the optimally pruned extreme learning machine (OP-ELM) methodology is presented. It is based on the original extreme learning machine (ELM) algorithm with additional steps to make it more robust and generic. The whole methodology is presented in detail and then applied to several regression and classification problems. Results for both computational time and accuracy (mean square error) are compared to the original ELM and to three other widely used methodologies: multilayer perceptron (MLP), support vector machine (SVM), and Gaussian process (GP). As the experiments for both regression and classification illustrate, the proposed OP-ELM methodology performs several orders of magnitude faster than the other algorithms used in this brief, except the original ELM. Despite the simplicity and fast performance, the OP-ELM is still able to maintain an accuracy that is comparable to the performance of the SVM. A toolbox for the OP-ELM is publicly available online.

Methodology for long-term prediction of time series

In this paper, a global methodology for the long-term prediction of time series is proposed. This methodology combines direct prediction strategy and sophisticated input selection criteria: k-nearest neighbors approximation method (k-NN), mutual information (MI) and nonparametric noise estimation (NNE). A global input selection strategy that combines forward selection, backward elimination (or pruning) and forward-backward selection is introduced. This methodology is used to optimize the three input selection criteria (k-NN, MI and NNE). The methodology is successfully applied to a real life benchmark: the Poland Electricity Load dataset.

A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition

Multi-step ahead forecasting is still an open challenge in time series forecasting. Several approaches that deal with this complex problem have been proposed in the literature but an extensive comparison on a large number of tasks is still missing. This paper aims to fill this gap by reviewing existing strategies for multi-step ahead forecasting and comparing them in theoretical and practical terms. To attain such an objective, we performed a large scale comparison of these different strategies using a large experimental benchmark (namely the 111 series from the NN5 forecasting competition). In addition, we considered the effects of deseasonalization, input variable selection, and forecast combination on these strategies and on multi-step ahead forecasting at large. The following three findings appear to be consistently supported by the experimental results: Multiple-Output strategies are the best performing approaches, deseasonalization leads to uniformly improved forecast accuracy, and input selection is more effective when performed in conjunction with deseasonalization.

Multiple-output modeling for multi-step-ahead time series forecasting

Accurate prediction of time series over long future horizons is the new frontier of forecasting. Conventional approaches to long-term time series forecasting rely either on iterated one-step-ahead predictors or direct predictors. In spite of their diversity, iterated and direct techniques for multi-step-ahead forecasting share a common feature, i.e. they model from data a multiple-input single-output mapping. In previous works, the authors presented an original multiple-output approach to multi-step-ahead prediction. The goal is to improve accuracy by preserving in the forecasted sequence the stochastic properties of the training series. This is not guaranteed for instance in direct approaches where predictions for different horizons are performed independently. This paper presents a review of single-output vs. multiple-output approaches for prediction and goes a step forward with respect to the previous authors contributions by (i) extending the multiple-output approach with a query-based criterion and (ii) presenting an assessment of single-output and multiple-output methods on the NN3 competition datasets. In particular, the experimental section shows that multiple-output approaches represent a competitive choice for tackling long-term forecasting tasks.

OP-ELM: Theory, Experiments and a Toolbox

This paper presents the Optimally-Pruned Extreme Learning Machine (OP-ELM) toolbox. This novel, fast and accurate methodology is applied to several regression and classification problems. The results are compared with widely known Multilayer Perceptron (MLP) and Least-Squares Support Vector Machine (LS-SVM) methods. As the experiments (regression and classification) demonstrate, the OP-ELM methodology is considerably faster than the MLP and the LS-SVM, while maintaining the accuracy in the same level. Finally, a toolbox performing the OP-ELM is introduced and instructions are presented.

X-SOM and L-SOM: A double classification approach for missing value imputation

In this paper, a new method for the determination of missing values in temporal databases is presented. It is based on a robust version of a nonlinear classification algorithm called self-organizing maps and it consists of a combination of two classifications in order to take advantage of spatial as well as temporal dependencies of the dataset. This double classification leads to a significant improvement of the estimation of the missing values. An application of the missing value imputation for hedge fund returns is presented.

Long-term prediction of time series by combining direct and MIMO strategies

Reliable and accurate prediction of time series over large future horizons has become the new frontier of the forecasting discipline. Current approaches to long-term time series forecasting rely either on iterated predictors, direct predictors or, more recently, on the Multi-Input Multi-Output (MIMO) predictors. The iterated approach suffers from the accumulation of errors, the Direct strategy makes a conditional independence assumption, which does not necessarily preserve the stochastic properties of the time series, while the MIMO technique is limited by the reduced flexibility of the predictor. The paper compares the Direct and MIMO strategies and discusses their respective limitations to the problem of long-term time series prediction. It also proposes a new methodology that is a sort of intermediate way between the Direct and the MIMO technique. The paper presents the results obtained with the ESTSP 2007 competition dataset.

Long-term prediction of time series using NNE-based projection and OP-ELM

This paper proposes a combination of methodologies based on a recent development -called Extreme Learning Machine (ELM)- decreasing drastically the training time of nonlinear models. Variable selection is beforehand performed on the original dataset, using the Partial Least Squares (PLS) and a projection based on Nonparametric Noise Estimation (NNE), to ensure proper results by the ELM method. Then, after the network is first created using the original ELM, the selection of the most relevant nodes is performed by using a Least Angle Regression (LARS) ranking of the nodes and a Leave-One-Out estimation of the performances, leading to an Optimally-Pruned ELM (OP-ELM). Finally, the prediction accuracy of the global methodology is demonstrated using the ESTSP 2008 Competition and Poland Electricity Load datasets.

Input and structure selection for k -NN approximator

This paper presents k-NN as an approximator for time series prediction problems. The main advantage of this approximator is its simplicity. Despite the simplicity, k-NN can be used to perform input selection for nonlinear models and it also provides accurate approximations. Three model structure selection methods are presented: Leave-one-out, Bootstrap and Bootstrap 632. We will show that both Bootstraps provide a good estimate of the number of neighbors, k, where Leave-one-out fails. Results of the methods are presented with the Electric load from Poland data set.

OPELM and OPKNN in long-term prediction of time series using projected input data

Long-term time series prediction is a difficult task. This is due to accumulation of errors and inherent uncertainties of a long-term prediction, which leads to deteriorated estimates of the future instances. In order to make accurate predictions, this paper presents a methodology that uses input processing before building the model. Input processing is a necessary step due to the curse of dimensionality, where the aim is to reduce the number of input variables or features. In the paper, we consider the combination of the delta test and the genetic algorithm to obtain two aspects of reduction: scaling and projection. After input processing, two fast models are used to make the predictions: optimally pruned extreme learning machine and optimally pruned k-nearest neighbors. Both models have fast training times, which makes them suitable choice for direct strategy for long-term prediction. The methodology is tested on three different data sets: two time series competition data sets and one financial data set.