Sungwan Bang

Simultaneous estimation and factor selection in quantile regression via adaptive sup-norm regularization

Some regularization methods, including the group lasso and the adaptive group lasso, have been developed for the automatic selection of grouped variables (factors) in conditional mean regression. In many practical situations, such a problem arises naturally when a set of dummy variables is used to represent a categorical factor and/or when a set of basis functions of a continuous variable is included in the predictor set. Complementary to these earlier works, the simultaneous and automatic factor selection is examined in quantile regression. To incorporate the factor information into regularized model fitting, the adaptive sup-norm regularized quantile regression is proposed, which penalizes the empirical check loss function by the sum of factor-wise adaptive sup-norm penalties. It is shown that the proposed method possesses the oracle property. A simulation study demonstrates that the proposed method is a more appropriate tool for factor selection than the adaptive lasso regularized quantile regression.

Weighted Support Vector Machine Using k-Means Clustering

The support vector machine (SVM) has been successfully applied to various classification areas with great flexibility and a high level of classification accuracy. However, the SVM is not suitable for the classification of large or imbalanced datasets because of significant computational problems and a classification bias toward the dominant class. The SVM combined with the k-means clustering (KM-SVM) is a fast algorithm developed to accelerate both the training and the prediction of SVM classifiers by using the cluster centers obtained from the k-means clustering. In the KM-SVM algorithm, however, the penalty of misclassification is treated equally for each cluster center even though the contributions of different cluster centers to the classification can be different. In order to improve classification accuracy, we propose the WKM–SVM algorithm which imposes different penalties for the misclassification of cluster centers by using the number of data points within each cluster as a weight. As an extension of the WKM–SVM, the recovery process based on WKM–SVM is suggested to incorporate the information near the optimal boundary. Furthermore, the proposed WKM–SVM can be successfully applied to imbalanced datasets with an appropriate weighting strategy. Experiments show the effectiveness of our proposed methods.

Hierarchically penalized support vector machine with grouped variables

When input features are naturally grouped or generated by factors in a linear classification problem, it is more meaningful to identify important groups or factors rather than individual features. The F∞-norm support vector machine (SVM) and the group lasso penalized SVM have been developed to perform simultaneous classification and factor selection. However, these group-wise penalized SVM methods may suffer from estimation inefficiency and model selection inconsistency because they cannot perform feature selection within an identified group. To overcome this limitation, we propose the hierarchically penalized SVM (H-SVM) that not only effectively identifies important groups but also removes irrelevant features within an identified group. Numerical results are presented to demonstrate the competitive performance of the proposed H-SVM over existing SVM methods.

Adaptive sup-norm regularized simultaneous multiple quantiles regression

When modelling multiple conditional quantiles of univariate and/or multivariate responses, it is of great importance to share strength among them. The simultaneous multiple quantiles regression (SMQR) technique is a novel regularization method that explores the similarity among multiple conditional quantiles and performs simultaneous model selection. However, the SMQR suffers from estimation inefficiency and model selection inconsistency because it applies the same amount of shrinkage to each predictor variable without assessing its relative importance. To overcome such a limitation, we propose the adaptive sup-norm regularized SMQR (ASMQR) method, which allows different amounts of shrinkage to be imposed on different variables according to their relative importance. We show that the proposed ASMQR method, a generalized form of the adaptive lasso regularized quantile regression (ALQR) method, possesses the oracle property and that it is a better tool for selecting a common subset of significant variables than the ALQR and SMQR methods through a simulation study.

Adaptive lasso penalised censored composite quantile regression

To account for censoring in estimating the accelerated failure time AFT model with right censored data, the weighted least squares regression WLSR has been developed by using the inverse-censoring-probability weights. However, it is well known that the traditional ordinary least squares may fail to produce a reliable estimator for data subject to heavy-tailed errors or outliers. For robust estimation in the AFT model, we propose the weighted composite quantile regression WCQR method, in which the sum of weighted multiple quantile objective functions based on the inverse-censoring-probability weights is used as a loss function. As a novel regularisation method for right censored data, we further propose the adaptive lasso penalised WCQR AWCQR method in order to perform simultaneous estimation and variable selection. The large sample properties of the WCQR and AWCQR estimators are established under some regularity conditions. The proposed methods are evaluated through simulation studies and real data applications.

On the Use of Sequential Adaptive Nearest Neighbors for Missing Value Imputation

In this paper, we propose a Sequential Adaptive Nearest Neighbor(SANN) imputation method that combines the Adaptive Nearest Neighbor(ANN) method and the Sequential k-Nearest Neighbor(SKNN) method. When choosing the nearest neighbors of missing observations, the proposed SANN method takes the local feature of the missing observations into account as well as reutilizes the imputed observations in a sequential manner. By using a Monte Carlo study and a real data example, we demonstrate the characteristics of the SANN method and its potential performance.

Simultaneous estimation for non-crossing multiple quantile regression with right censored data

In this paper, we consider the estimation problem of multiple conditional quantile functions with right censored survival data. To account for censoring in estimating a quantile function, weighted quantile regression (WQR) has been developed by using inverse-censoring-probability weights. However, the estimated quantile functions from the WQR often cross each other and consequently violate the basic properties of quantiles. To avoid quantile crossing, we propose non-crossing weighted multiple quantile regression (NWQR), which estimates multiple conditional quantile functions simultaneously. We further propose the adaptive sup-norm regularized NWQR (ANWQR) to perform simultaneous estimation and variable selection. The large sample properties of the NWQR and ANWQR estimators are established under certain regularity conditions. The proposed methods are evaluated through simulation studies and analysis of a real data set.

Non-crossing weighted kernel quantile regression with right censored data.

Regarding survival data analysis in regression modeling, multiple conditional quantiles are useful summary statistics to assess covariate effects on survival times. In this study, we consider an estimation problem of multiple nonlinear quantile functions with right censored survival data. To account for censoring in estimating a nonlinear quantile function, weighted kernel quantile regression (WKQR) has been developed by using the kernel trick and inverse-censoring-probability weights. However, the individually estimated quantile functions based on the WKQR often cross each other and consequently violate the basic properties of quantiles. To avoid this problem of quantile crossing, we propose the non-crossing weighted kernel quantile regression (NWKQR), which estimates multiple nonlinear conditional quantile functions simultaneously by enforcing the non-crossing constraints on kernel coefficients. The numerical results are presented to demonstrate the competitive performance of the proposed NWKQR over the WKQR.

A comparison study of multiple linear quantile regression using non-crossing constraints

Multiple quantile regression that simultaneously estimate several conditional quantiles of response given covariates can provide a comprehensive information about the relationship between the response and covariates. Some quantile estimates can cross if conditional quantiles are separately estimated; however, this violates the definition of the quantile. To tackle this issue, multiple quantile regression with non-crossing constraints have been developed. In this paper, we carry out a comparison study on several popular methods for non-crossing multiple linear quantile regression to provide practical guidance on its application.

Hierarchically penalized quantile regression

In many regression problems, predictors are naturally grouped. For example, when a set of dummy variables is used to represent categorical variables, or a set of basis functions of continuous variables is included in the predictor set, it is important to carry out a feature selection both at the group level and at individual variable levels within the group simultaneously. To incorporate the group and variables within-group information into a regularized model fitting, several regularization methods have been developed, including the Cox regression and the conditional mean regression. Complementary to earlier works, the simultaneous group and within-group variables selection method is examined in quantile regression. We propose a hierarchically penalized quantile regression, and show that the hierarchical penalty possesses the oracle property in quantile regression, as well as in the Cox regression. The proposed method is evaluated through simulation studies and a real data application.