Mirosław Kordos

Fusion of instance selection methods in regression tasks

Few instance selection (IS) methods exist for regression.Two different families of instance selection methods for regression are compared.One is based in a simple discretization of the output variable, but with good results.Both approaches can be used to adapt to regression IS methods for classification.The fusion of these IS algorithms in an ensemble for regression is also analyzed. Data pre-processing is a very important aspect of data mining. In this paper we discuss instance selection used for prediction algorithms, which is one of the pre-processing approaches. The purpose of instance selection is to improve the data quality by data size reduction and noise elimination. Until recently, instance selection has been applied mainly to classification problems. Very few recent papers address instance selection for regression tasks. This paper proposes fusion of instance selection algorithms for regression tasks to improve the selection performance. As the members of the ensemble two different families of instance selection methods are evaluated: one based on distance threshold and the other one on converting the regression task into a multiple class classification task. Extensive experimental evaluation performed on the two regression versions of the Edited Nearest Neighbor (ENN) and Condensed Nearest Neighbor (CNN) methods showed that the best performance measured by the error value and data size reduction are in most cases obtained for the ensemble methods.

Bagging of Instance Selection Algorithms

The paper presents bagging ensembles of instance selection algorithms. We use bagging to improve instance selection. The improvement comprises data compression and prediction accuracy. The examined instance selection algorithms for classification are ENN, CNN, RNG and GE and for regression are the developed by us Generalized CNN and Generalized ENN algorithms. Results of the comparative experimental study performed using different configurations on several datasets shows that the approachbased on bagging allowed for significant improvement, especially in terms of data compression.

Instance Selection in Logical Rule Extraction for Regression Problems

The paper presents three algorithms of instance selection for regression problems, which extend the capabilities of the CNN, ENN and CA algorithms used for classification tasks. Various combinations of the algorithms are experimentally evaluated as data preprocessing for regression tree induction. The influence of the instance selection algorithms and their parameters on the accuracy and rules produced by regression trees is evaluated and compared to the results obtained with tree pruning.

Instance selection with neural networks for regression problems

The paper presents algorithms for instance selection for regression problems based upon the CNN and ENN solutions known for classification tasks. A comparative experimental study is performed on several datasets using multilayer perceptrons and k-NN algorithms with different parameters and their various combinations as the method the selection is based on. Also various similarity thresholds are tested. The obtained results are evaluated taking into account the size of the resulting data set and the regression accuracy obtained with multilayer perceptron as the predictive model and the final recommendation regarding instance selection for regression tasks is presented.

Training Neural Networks on Noisy Data

This paper discusses approaches to noise-resistant training of MLP neural networks. We present various aspects of the issue and the ways of obtaining that goal by using two groups of approaches and combinations of them. The first group is based on a different processing of each vector depending of the likelihood of the vector being an outlier. The likelihood is determined by instance selection and outlier detection. The second group is based on training MLP neural networks with non-differentiable robust objective functions. We evaluate the performance of particular methods with different level of noise in the data for regression problems.

A hybrid system with regression trees in steel-making process

The paper presents a hybrid regresseion model with the main emphasis put on the regression tree unit. It discusses input and output variable transformation, determining the final decision of hybrid models and node split optimization of regression trees. Because of the ability to generate logical rules, a regression tree maybe the preferred module if it produces comparable results to other modules, therefore the optimization of node split in regression trees is discussed in more detail. A set of split criteria based on different forms of variance reduction is analyzed and guidelines for the choice of the criterion are discussed, including the trade-off between the accuracy of the tree, its size and balance between minimizing the node variance and keeping a symmetric structure of the tree. The presented approach found practical applications in the metallurgical industry.

Improving MLP Neural Network Performance by Noise Reduction

In this paper we examine several methods for improving the performance of MLP neural networks by eliminating the influence of outliers and compare them experimentally on several classification and regression tasks. The examined method include: pre-training outlier elimination, use of different error measures during network training, replacing the weighted input sum with weighted median in the neuron input functions and various combinations of them. We show how these methods influence the network prediction. Based on the experimental results, we also present a novel hybrid approach improving the network performance.

Temperature prediction in electric arc furnace with neural network tree

This paper presents a neural network tree regression system with dynamic optimization of input variable transformations and post-training optimization. The decision tree consists of MLP neural networks, which optimize the split points and at the leaf level predict final outputs. The system is designed for regression problems of big and complex datasets. It was applied to the problem of steel temperature prediction in the electric arc furnace in order to decrease the process duration at one of the steelworks.

A new approach to neural network based stock trading strategy

The paper presents an idea of using an MLP neural network for determining the optimal buy and sell time on a stock exchange. The inputs in the training set consist of past stock prices and a number of technical indicators. The buy and sell moments on the training data that will become the output to the neural network can be determined either automatically or manually by a user on past data. We discuss also the input space transformation and some improvements to the backpropagation algorithms.

Neural Network Regression for LHF Process Optimization

We present a system for regression using MLP neural networks with hyperbolic tangent functions in the input, hidden and output layer. The activation functions in the input and output layer are adjusted during the network training to fit better the distribution of the underlying data, while the network weights are trained to fit desired input-output mapping. A non-gradient variable step size training algorithm is used since it proved effective for that kind of problems. Finally we present a practical implementation, the system found in the optimization of metallurgical processes.