Neamat El Gayar

An Empirical Comparison of Machine Learning Models for Time Series Forecasting

In this work we present a large scale comparison study for the major machine learning models for time series forecasting. Specifically, we apply the models on the monthly M3 time series competition data (around a thousand time series). There have been very few, if any, large scale comparison studies for machine learning models for the regression or the time series forecasting problems, so we hope this study would fill this gap. The models considered are multilayer perceptron, Bayesian neural networks, radial basis functions, generalized regression neural networks (also called kernel regression), K-nearest neighbor regression, CART regression trees, support vector regression, and Gaussian processes. The study reveals significant differences between the different methods. The best two methods turned out to be the multilayer perceptron and the Gaussian process regression. In addition to model comparisons, we have tested different preprocessing methods and have shown that they have different impacts on the performance.

A study of the robustness of KNN classifiers trained using soft labels

Supervised learning models most commonly use crisp labels for classifier training. Crisp labels fail to capture the data characteristics when overlapping classes exist. In this work we attempt to compare between learning using soft and hard labels to train K-nearest neighbor classifiers. We propose a new technique to generate soft labels based on fuzzy-clustering of the data and fuzzy relabelling of cluster prototypes. Experiments were conducted on five data sets to compare between classifiers that learn using different types of soft labels and classifiers that learn with crisp labels. Results reveal that learning with soft labels is more robust against label errors opposed to learning with crisp labels. The proposed technique to find soft labels from the data, was also found to lead to a more robust training in most data sets investigated.

Forward and Backward Forecasting Ensembles for the Estimation of Time Series Missing Data

The presence of missing data in time series is big impediment to the successful performance of forecasting models, as it leads to a significant reduction of useful data. In this work we propose a multiple-imputation-type framework for estimating the missing values of a time series. This framework is based on iterative and successive forward and backward forecasting of the missing values, and constructing ensembles of these forecasts. The iterative nature of the algorithm allows progressive improvement of the forecast accuracy. In addition, the different forward and backward dynamics of the time series provide beneficial diversity for the ensemble. The developed framework is general, and can make use of any underlying machine learning or conventional forecasting model. We have tested the proposed approach on large data sets using linear, as well as nonlinear underlying forecasting models, and show its success.

MLP, Gaussian Processes and Negative Correlation Learning for Time Series Prediction

Time series forecasting is a challenging problem, that has a wide variety of application domains such as in engineering, environment, finance and others. When confronted with a time series forecasting application, typically a number of different forecasting models are tested and the best one is considered. Alternatively, instead of choosing the single best method, a wiser action could be to choose a group of the best models and then to combine their forecasts. In this study we propose a combined model consisting of Multi-layer perceptron (MLP), Gaussian Processes Regression (GPR) and a Negative Correlation Learning (NCL) model. The MLP and the GPR were the top performers in a previous large scale comparative study. On the other hand, NCL suggests an alternative way for building accurate and diverse ensembles. No studies have reported on the performance of the NCL in time series prediction. In this work we test the efficiency of NCL in predicting time series data. Results on two real data sets show that the NCL is a good candidate model for forecasting time series. In addition, the study also shows that the combined MLP/GPR/NCL model outperforms all models under consideration.

Exploiting neural networks to enhance trend forecasting for hotels reservations

Hotel revenue management is perceived as a managerial tool for room revenue maximization. A typical revenue management system contains two main components: Forecasting and Optimization. A forecasting component that gives accurate forecasts is a cornerstone in any revenue management system. It simply draws a good picture for the future demand. The output of the forecast component is then used for optimization and allocation in such a way that maximizes revenue. This shows how it is important to have a reliable and precise forecasting system. Neural Networks have been successful in forecasting in many fields. In this paper, we propose the use of NN to enhance the accuracy of a Simulation based Forecasting system, that was developed in an earlier work. In particular a neural network is used for modeling the trend component in the simulation based forecasting model. In the original model, Holt’s technique was used to forecast the trend. In our experiments using real hotel data we demonstrate that the proposed neural network approach outperforms the Holt’s technique. The proposed enhancement also resulted in better arrivals and occupancy forecasting when incorporated in the simulation based forecasting system.

NOVEL ENSEMBLE TECHNIQUES FOR REGRESSION WITH MISSING DATA

In this paper, we consider the problem of missing data, and develop an ensemble-network model for handling the missing data. The proposed method is based on utilizing the inherent uncertainty of the missing records in generating diverse training sets for the ensembles networks. Specifically we generate the missing values using their probability distribution function. We repeat this procedure many times thereby creating a number of complete data sets. A network is trained for each of these data sets, thereby obtaining an ensemble of networks. Several variants are proposed, and we show analytically that one of these variants is superior to the conventional mean-substitution approach for the limit of large training set. Simulation results confirm the general superiority of the proposed methods compared to the conventional approaches.

Novel Machine Learning Techniques for Micro-Array Data Classification

Machine learning, data mining and pattern recognition have been quite often used in various contexts of medical and bioinformatics applications. Currently computational methods and tools available for that purpose are quite abundant. The main aim of this chapter is to outline to the practitioners the basic concepts of the fields focusing on essential machine learning tools and highlighting their best practices to be successfully used in the medical domain. We present a case study for DNA microarray classification using ensemble methods and feature subset selection techniques. The background section will begin by introducing the reader to the fields of pattern recognition, machine learning and data mining. It will then focus on some of the most important concepts related to machine learning. In particular in section 2 we review the most popular machine learning models for classification used in the context of the medical domain. We then describe one of the most powerful and widely used classifiers for high dimensional feature spaces; the support vector machines (SVM). We cover the area of classifier evaluation and comparison to provide practitioners with essential understanding of how to test, validate and select the appropriate models for their applications. Finally, we summarize the main advances in the field of ensemble learning, feature subset selection and feature subset ensembles. Section 3 presents a review of using machine learning in various fields of bioinformatics. In section 4, a recent case study on DNA microarray data that uses an ensemble of SVMs coupled with feature subset selection methods is presented. We show how the proposed model can alleviate the curse of dimensionality associated with expression-based classification of DNA data in order to achieve stable and reliable results. Section 5 describes the data used and the experiments conducted, while section 6 presents results and a comparative analysis for the proposed models. Finally in section 7 we summarize the main contributions of this chapter and review the main guidelines to effectively use machine learning tools. We end this section by highlighting a set of challenges that need to be addressed and propose some future research directions in the field.

Fuzzy Gaussian Process Classification Model

Soft labels allow a pattern to belong to multiple classes with different degrees. In many real world applications the association of a pattern to multiple classes is more realistic; to describe overlap and uncertainties in class belongingness. The objective of this work is to develop a fuzzy Gaussian process model for classification of soft labeled data. Gaussian process models have gained popularity in the recent years in classification and regression problems and are example of a flexible, probabilistic, non-parametric model with uncertainty predictions. Here we derive a fuzzy Gaussian model for a two class problem and then explain how this can be extended to multiple classes. The derived model is tested on different fuzzified datasets to show that it can adopt to various classification problems. Results reveal that our model outperforms the fuzzy K-Nearest Neighbor (FKNN), applied on the fuzzified dataset, as well as the Gaussian process and the K-Nearest Neighbor models used with crisp labels.

Off the Mainstream: Advances in Neural Networks and Machine Learning for Pattern Recognition

This Special Issue (SI) originates from an event we organized inUlm,Germany, in September 2016, namely the seventh IAPR TC3 Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR2016) [18], sponsored by the International Association for Pattern Recognition (IAPR) and managed by its Technical Committee 3 on Neural Networks and Computational Intelligence1 (TC3). In the era of deep learning, ANNPR2016 aimed to create a common ground for researchers active specifically in applications to pattern recognition tasks of neural networks and other machine learning approaches. After the success of the workshop, and in the light of the exquisite scientific contribution of several presentations given therein, it came to our minds the idea of proposing a special issue of a relevant journal, to be based on extended versions of selected papers from ANNPR2016. The Editorial Board of Neural Processing Letters accepted the proposal, encouraging us to proceed with the initiative. At that point, we decided to extend the scope of the SI to an even broader audience by means of an open call for papers on the topic of “off-the-mainstream” approaches to

A Time Series Classification Approach for Motion Analysis Using Ensembles in Ubiquitous Healthcare Systems

Human motion analysis is a vital research area for healthcare systems. The increasing need for automated activity analysis inspired the design of low cost wireless sensors that can capture information under free living conditions. Body and Visual Sensor Networks can easily record human behavior within a home environment. In this paper we propose a multiple classifier system that uses time series data for human motion analysis. The proposed approach adaptively integrates feature extraction and distance based techniques for classifying impaired and normal walking gaits. Information from body sensors and multiple vision nodes are used to extract local and global features. Our proposed method is tested against various classifiers trained using different feature spaces. The results for the different training schemes are presented. We demonstrate that the proposed model outperforms the other presented classification methods.