Ginés Rubio

New method for instance or prototype selection using mutual information in time series prediction

The problem of selecting the patterns to be learned by any model is usually not considered by the time of designing the concrete model but as a preprocessing step. Information theory provides a robust theoretical framework for performing input variable selection thanks to the concept of mutual information. Recently the computation of the mutual information for regression tasks has been proposed so this paper presents a new application of the concept of mutual information not to select the variables but to decide which prototypes should belong to the training data set in regression problems. The proposed methodology consists in deciding if a prototype should belong to or not to the training set using as criteria the estimation of the mutual information between the variables. The novelty of the approach is to focus in prototype selection for regression problems instead of classification as the majority of the literature deals only with the last one. Other element that distinguishes this work from others is that it is not proposed as an outlier detector but as an algorithm that determines the best subset of input vectors by the time of building a model to approximate it. As the experiment section shows, this new method is able to identify a high percentage of the real data set when it is applied to highly distorted data sets.

Using near-infrared spectroscopy in the classification of white and iberian pork with neural networks

The visible/near-infrared spectrum consists of overtones and combination bands of the fundamental molecular absorptions found in the visible and near-infrared region. The analysis of the spectrum might be difficult because overlapping vibrational bands may appear nonspecific and poorly resolved. Nevertheless, the information it could be retrieved from the analysis of the spectrum might be very useful for the food industry producers, consumers, and food distributors because the meat could be classified based on the spectrum in several aspects such as the quality, tenderness, and kind of meat. This paper applies Mutual Information theory and several classification models (Radial Basis Function Neural Networks and Support Vector Machines) in order to determine the breed of pork meat (Iberian or White) using only as input the infrared spectrum. First, the more relevant wavelengths from the spectrum will be chosen, then, those wavelengths will be the input data to design the classifiers. As the experiments will show, the proposed techniques, when applied with a correct design methodology are capable of obtaining quality results for this specific problem.

Design of specific-to-problem kernels and use of kernel weighted K-nearest neighbours for time series modelling

Least squares support vector machines (LSSVM) with Gaussian kernel represent the most used of the kernel methods existing in the literature for regression and time series prediction. These models have a good behaviour for these types of problems due to their generalization capabilities and their smooth interpolation, but they are very dependent on the feature selection performed and their computational cost notably increases with the number of training samples. Time series prediction can be tackled as a regression problem by constructing a set of input/output data from the series; this traditional approach and the use of typical recursive or direct strategies present serious drawbacks in long-term prediction. This paper presents an alternative based on the settings of specific-to-problem kernels to be applied to time series prediction focusing on large scale prediction. A simple methodology for kernel creation based on the periodicities in time series data is proposed. An alternative to LSSVM models with lower computational cost, the Kernel Weighted K-Nearest Neighbours (KWKNN) is described for function approximation. A parallel version of KWKNN is also presented to deal with large data sets.

Efficient Optimization of the Parameters of LS-SVM for Regression versus Cross-Validation Error

Least Squares Support Vector Machines (LS-SVM) are the state of the art in kernel methods for regression and function approximation. In the last few years, these models have been successfully applied to time series modelling and prediction. A key issue for the good performance of a LS-SVM model are the values chosen for both the kernel parameters and its hyperparameters in order to avoid overfitting the underlying system to be modelled. In this paper an efficient method for the evaluation of the cross validation error for LS-SVM is revised. The expressions for its partial derivatives are presented in order to improve the procedure for parameter optimization. Some initial guesses to set the values of both kernel parameters and the regularization factor are also presented. We finally conduct some experiments on a time series data example using a number of methods for parameter optimization for LS-SVM models. The results show that the proposed partial derivatives and heuristics can improve the performance with respect to both execution time and the optimized model obtained.

Mutual Information Based Initialization of Forward-Backward Search for Feature Selection in Regression Problems

Pure feature selection, where variables are chosen or not to be in the training data set, still remains as an unsolved problem, especially when the dimensionality is high. Recently, the Forward-Backward Search algorithm using the Delta Test to evaluate a possible solution was presented, showing a good performance. However, due to the locality of the search procedure, the initial starting point of the search becomes crucial in order to obtain good results. This paper presents new heuristics to find a more adequate starting point that could lead to a better solution. The heuristic is based on the sorting of the variables using the Mutual Information criterion, and then performing parallel local searches. These local searches provide an initial starting point for the actual parallel Forward-Backward algorithm.

Method for Prediction of Protein-Protein Interactions in Yeast Using Genomics/Proteomics Information and Feature Selection

Protein-protein interaction (PPI) prediction is one of the main goals in the current Proteomics. This work presents a method for prediction of protein-protein interactions through a classification technique known as Support Vector Machines. The dataset considered is a set of positive and negative examples taken from a high reliability source, from which we extracted a set of genomic features, proposing a similarity measure. Feature selection was performed to obtain the most relevant variables through a modified method derived from other feature selection methods for classification. Using the selected subset of features, we constructed a support vector classifier that obtains values of specificity and sensitivity higher than 90% in prediction of PPIs, and also providing a confidence score in interaction prediction of each pair of proteins.

Kernel methods applied to time series forecasting

Kernel methods are a class of algorithms whose importance has grown from the 90s in the machine learning field. Their most notable example are Support Vector Machines (SVMs), which are the state of the art for classification problems. Nevertheless, they are applicable to functional approximation problems and there are however several of them available: SVM for regression, Gaussian Process Regression and Least Squares SVM (LS-SVM) for instance. This paper applies and studies these algorithms to a number of Time Series Prediction problems and compares them with some more conventional techniques.

Global and Local Modelling in Radial Basis Functions Networks

In the problem of modelling Input/Output data using neuro-fuzzy systems, the performance of the global model is normally the only objective optimized, and this might cause a misleading performance of the local models. This work presents a modified radial basis function network that maintains the optimization properties of the local sub-models whereas the model is globally optimized, thanks to a special partitioning of the input space in the hidden layer performed to carry out those objectives. The advantage of the methodology proposed is that due to those properties, the global and the local models are both directly optimized. A learning methodology adapted to the proposed model is used in the simulations, consisting of a clustering algorithm for the initialization of the centers and a local search technique.

Creation of Specific-to-Problem Kernel Functions for Function Approximation

Although there is a large diversity in the literature related to kernel methods, there are only a few works which do not use kernels based on Radial Basis Functions (RBF) for regression problems. The reason for that is that they present very good generalization capabilities and smooth interpolation. This paper studies an initial framework to create specific-to-problem kernels for application to regression models. The kernels are created without prior knowledge about the data to be approximated by means of a Genetic Programming algorithm. The quality of a kernel is evaluated independently of a particular model, using a modified version of a non parametric noise estimator. For a particular problem, performances of generated kernels are tested against common ones using weighted k-nn in the kernel space. Results show that the presented method produces specific-to-problem kernels that outperform the common ones for this particular case. Parallel programming is utilized to deal with large computational costs.

On incorporating seasonal information on recursive time series predictors

In time series prediction problems in which the current series presents a certain seasonality, the long term and short term prediction capabilities of a learned model can be improved by considering that seasonality as additional information within it. Kernel methods and specifically LS-SVM are receiving increasing attention in the last years thanks to many interesting properties; among them, these type of models can include any additional information by simply adding new variables to the problem, without increasing the computational cost. This work evaluates how including the seasonal information of a series in a designed recursive model might not only upgrade the performance of the predictor, but also allows to diminish the number of input variables needed to perform the modelling, thus being able to increase both the generalization and interpretability capabilities of the model.