Maher Maalouf

Robust weighted kernel logistic regression in imbalanced and rare events data

Recent developments in computing and technology, along with the availability of large amounts of raw data, have contributed to the creation of many effective techniques and algorithms in the fields of pattern recognition and machine learning. The main objectives for developing these algorithms include identifying patterns within the available data or making predictions, or both. Great success has been achieved with many classification techniques in real-life applications. With regard to binary data classification in particular, analysis of data containing rare events or disproportionate class distributions poses a great challenge to industry and to the machine learning community. This study examines rare events (REs) with binary dependent variables containing many more non-events (zeros) than events (ones). These variables are difficult to predict and to explain as has been evidenced in the literature. This research combines rare events corrections to Logistic Regression (LR) with truncated Newton methods and applies these techniques to Kernel Logistic Regression (KLR). The resulting model, Rare Event Weighted Kernel Logistic Regression (RE-WKLR), is a combination of weighting, regularization, approximate numerical methods, kernelization, bias correction, and efficient implementation, all of which are critical to enabling RE-WKLR to be an effective and powerful method for predicting rare events. Comparing RE-WKLR to SVM and TR-KLR, using non-linearly separable, small and large binary rare event datasets, we find that RE-WKLR is as fast as TR-KLR and much faster than SVM. In addition, according to the statistical significance test, RE-WKLR is more accurate than both SVM and TR-KLR.

Kernel Logistic Regression Using Truncated Newton Method

Kernel logistic regression (KLR) is a powerful nonlinear classifier. The combination of KLR and the truncated-regularized iteratively re-weighted least-squares (TR-IRLS) algorithm, has led to a powerful classification method using small-to-medium size data sets. This method (algorithm), is called truncated-regularized kernel logistic regression (TR-KLR). Compared to support vector machines (SVM) and TR-IRLS on twelve benchmark publicly available data sets, the proposed TR-KLR algorithm is as accurate as, and much faster than, SVM and more accurate than TR-IRLS. The TR-KLR algorithm also has the advantage of providing direct prediction probabilities.

Logistic regression in data analysis: an overview

Logistic regression (LR) continues to be one of the most widely used methods in data mining in general and binary data classification in particular. This paper is focused on providing an overview of the most important aspects of LR when used in data analysis, specifically from an algorithmic and machine learning perspective and how LR can be applied to imbalanced and rare events data.

Weighted logistic regression for large-scale imbalanced and rare events data

Latest developments in computing and technology, along with the availability of large amounts of raw data, have led to the development of many computational techniques and algorithms. Concerning binary data classification in particular, analysis of data containing rare events or disproportionate class distributions poses a great challenge to industry and to the machine learning community. Logistic Regression (LR) is a powerful classifier. The combination of LR and the truncated-regularized iteratively re-weighted least squares (TR-IRLS) algorithm, has provided a powerful classification method for large data sets. This study examines imbalanced data with binary response variables containing many more non-events (zeros) than events (ones). It has been established in the literature that these variables are difficult to predict and explain. This research combines rare events corrections to LR with truncated Newton methods. The proposed method, Rare Event Weighted Logistic Regression (RE-WLR), is capable of processing large imbalanced data sets at relatively the same processing speed as the TR-IRLS, however, with higher accuracy.

Kernel ridge regression using truncated newton method

Kernel Ridge Regression (KRR) is a powerful nonlinear regression method. The combination of KRR and the truncated-regularized Newton method, which is based on the conjugate gradient (CG) method, leads to a powerful regression method. The proposed method (algorithm), is called Truncated-Regularized Kernel Ridge Regression (TR-KRR). Compared to the closed-form solution of KRR, Support Vector Machines (SVM) and Least-Squares Support Vector Machines (LS-SVM) algorithms on six data sets, the proposed TR-KRR algorithm is as accurate as, and much faster than all of the other algorithms.

A New Fuzzy Logic Approach to Capacitated Dynamic Dial-a-Ride Problem

Abstract Almost all Dial-a-Ride problems (DARP) described in the literature pertain to the design of optimal routes and schedules for n customers who specify pick-up and drop-off times. In this article we assume that the customer is mainly concerned with the drop-off time because it is the most important to the customer. Based on the drop-off time specified by the customer and the customers location, a pick-up time is calculated and given to the customer by the dispatching office. We base our formulation on a dynamic fuzzy logic approach in which a new request is assigned to a vehicle. The fuzzy logic algorithm chooses the vehicle to transport the customer by seeking to satisfy two objectives. The first reflects the customers preference and minimizes the time a customer spends in the vehicle, and the second reflects the companys preference and minimizes the distance a vehicle needs to travel to transport the customer. The proposed heuristic algorithm is relatively simple and computationally efficient in comparison with most deterministic algorithms for solving both small and large sized problems.

Rare events and imbalanced datasets: an overview

Accurate prediction is important in data mining and data classification. Rare events data, imbalanced or skewed datasets are very important in data mining and classification. However, These types of data are difficult to predict and to explain as has been demonstrated in the literature. The problems arise from various sources. This paper surveys the latest research on such data in the hope of adding further contribution to this important field of data mining.

Logistic regression in large rare events and imbalanced data: A performance comparison of prior correction and weighting methods

The purpose of this study is to use the truncated Newton method in prior correction logistic regression (LR). A regularization term is added to prior correction LR to improve its performance, which results in the truncated‐regularized prior correction algorithm. The performance of this algorithm is compared with that of weighted LR and the regular LR methods for large imbalanced binary class data sets. The results, based on the KDD99 intrusion detection data set, and 6 other data sets at both the prior correction and the weighted LRs have the same computational efficiency when the truncated Newton method is used in both of them. A higher discriminative performance, however, resulted from weighting, which exceeded both the prior correction and the regular LR on nearly all the data sets. From this study, we conclude that weighting outperforms both the regular and prior correction LR models in most data sets and it is the method of choice when LR is used to evaluate imbalanced and rare event data.

Prediction of Resilient Modulus from Post-compaction Moisture Content and Physical Properties Using Support Vector Regression

The present study assesses the use of support vector machine regression to predict the variation of resilient modulus with post-compaction moisture content of soils commonly encountered in Oklahoma, Pennsylvania and Wisconsin. Results show the prediction model using the support vector regression (SVR) approach is a function of degree of saturation, moisture content and plasticity index. The developed model is compared to current models in the literature. Results indicate the proposed SVR model gives more accurate values than current regression models. This model will better predict changes in the bearing capacity of pavements due to seasonal variations of moisture content.

Improved Modeling of Solar Flash Desalination Using Support Vector Regression

AbstractAccurate prediction of heat-transfer rates in condensers is a challenging task because of phase-change dynamics. This is further complicated if noncondensable gases are present since they t...