Jicai Liu

A robust and efficient estimation method for single index models

Single index models are natural extensions of linear models and overcome the so-called curse of dimensionality. They have applications to many fields, such as medicine, economics and finance. However, most existing methods based on least squares or likelihood are sensitive when there are outliers or the error distribution is heavy tailed. Although an M-type regression is often considered as a good alternative to those methods, it may lose efficiency for normal errors. In this paper, we propose a new robust and efficient estimation procedure based on local modal regression for single index models. The asymptotic normality of proposed estimators for both the parametric and nonparametric parts is established. We show that the proposed estimators are as asymptotically efficient as the least-square-based estimators when there are no outliers and the error distribution is normal. A modified EM algorithm is presented for efficient implementation. The simulations and real data analysis are conducted to illustrate the finite sample performance of the proposed method.

Variable selection for varying dispersion beta regression model

The beta regression models are commonly used by practitioners to model variables that assume values in the standard unit interval (0, 1). In this paper, we consider the issue of variable selection for beta regression models with varying dispersion (VBRM), in which both the mean and the dispersion depend upon predictor variables. Based on a penalized likelihood method, the consistency and the oracle property of the penalized estimators are established. Following the coordinate descent algorithm idea of generalized linear models, we develop new variable selection procedure for the VBRM, which can efficiently simultaneously estimate and select important variables in both mean model and dispersion model. Simulation studies and body fat data analysis are presented to illustrate the proposed methods.

Quantile regression and variable selection for the single-index model

In this paper, we propose a new full iteration estimation method for quantile regression (QR) of the single-index model (SIM). The asymptotic properties of the proposed estimator are derived. Furthermore, we propose a variable selection procedure for the QR of SIM by combining the estimation method with the adaptive LASSO penalized method to get sparse estimation of the index parameter. The oracle properties of the variable selection method are established. Simulations with various non-normal errors are conducted to demonstrate the finite sample performance of the estimation method and the variable selection procedure. Furthermore, we illustrate the proposed method by analyzing a real data set.

Sparse group variable selection based on quantile hierarchical Lasso

The group Lasso is a penalized regression method, used in regression problems where the covariates are partitioned into groups to promote sparsity at the group level [27]. Quantile group Lasso, a natural extension of quantile Lasso [25], is a good alternative when the data has group information and has many outliers and/or heavy tails. How to discover important features that are correlated with interest of outcomes and immune to outliers has been paid much attention. In many applications, however, we may also want to keep the flexibility of selecting variables within a group. In this paper, we develop a sparse group variable selection based on quantile methods which select important covariates at both the group level and within the group level, which penalizes the empirical check loss function by the sum of square root group-wise L1-norm penalties. The oracle properties are established where the number of parameters diverges. We also apply our new method to varying coefficient model with categorial effect modifiers. Simulations and real data example show that the newly proposed method has robust and superior performance.

Semi Varying Coefficient Zero-Inflated Generalized Poisson Regression Model

In this paper, the semi varying coefficient zero-inflated generalized Poisson model is discussed based on penalized log-likelihood. All the coefficient functions are fitted by penalized spline (P-spline), and Expectation-maximization algorithm is used to drive these estimators. The estimation approach is rapid and computationally stable. Under some mild conditions, the consistency and the asymptotic normality of these resulting estimators are given. The score test statistics about dispersion parameter is discussed based on the P-spline estimation. Both simulated and real data example are used to illustrate our proposed methods.

Regularization and model selection for quantile varying coefficient model with categorical effect modifiers

A varying coefficient model with categorical effect modifiers is an effective modeling strategy when the data set includes categorical variables. With categorial predictors the number of parameters can become very large. This paper focuses on the model selection problem for varying coefficient model with categorical effect modifiers under the framework of quantile regression. After distinguishing between nominal and ordinal effect modifiers, a unified (adaptive-) Lasso-type regularization technique is proposed that allows for selection of covariates and fusion of categories of categorical effect modifiers, which can identify whether the coefficient functions are really varying with the level of a potentially effect modifying factor and provide a sparse model at different quantile levels. Moreover, the large sample properties are derived under appropriate conditions including a fixed bound on the number of parameters. The proposed methods are illustrated and investigated by extensive simulation studies and two real data evaluations.

Robust variable selection for the varying coefficient model based on composite L 1–L 2 regression

The varying coefficient model (VCM) is an important generalization of the linear regression model and many existing estimation procedures for VCM were built on L 2 loss, which is popular for its mathematical beauty but is not robust to non-normal errors and outliers. In this paper, we address the problem of both robustness and efficiency of estimation and variable selection procedure based on the convex combined loss of L 1 and L 2 instead of only quadratic loss for VCM. By using local linear modeling method, the asymptotic normality of estimation is driven and a useful selection method is proposed for the weight of composite L 1 and L 2. Then the variable selection procedure is given by combining local kernel smoothing with adaptive group LASSO. With appropriate selection of tuning parameters by Bayesian information criterion (BIC) the theoretical properties of the new procedure, including consistency in variable selection and the oracle property in estimation, are established. The finite sample performance of the new method is investigated through simulation studies and the analysis of body fat data. Numerical studies show that the new method is better than or at least as well as the least square-based method in terms of both robustness and efficiency for variable selection.

A new local estimation method for single index models for longitudinal data

Single index models are natural extensions of linear models and overcome the so-called curse of dimensionality. They are very useful for longitudinal data analysis. In this paper, we develop a new efficient estimation procedure for single index models with longitudinal data, based on Cholesky decomposition and local linear smoothing method. Asymptotic normality for the proposed estimators of both the parametric and nonparametric parts will be established. Monte Carlo simulation studies show excellent finite sample performance. Furthermore, we illustrate our methods with a real data example.

Robust estimation for partially linear single-index model

ABSTRACT In this article, we investigate a new estimation approach for the partially linear single-index model based on modal regression method, where the non parametric function is estimated by penalized spline method. Moreover, we develop an expection maximum (EM)-type algorithm and establish the large sample properties of the proposed estimation method. A distinguishing characteristic of the newly proposed estimation is robust against outliers through introducing an additional tuning parameter which can be automatically selected using the observed data. Simulation studies and real data example are used to evaluate the finite-sample performance, and the results show that the newly proposed method works very well.

Nonparametric independence screening for ultra-high-dimensional longitudinal data under additive models

ABSTRACT Ultra-high-dimensional data are frequently seen in many contemporary statistical studies, which pose challenges both theoretically and methodologically. To address this issue under longitudinal data setting, we propose a marginal nonparametric screening method to hunt for the relevant covariates in additive models. A new data-driven thresholding and an iterative procedure are developed. Especially, a sample splitting method is proposed to further reduce the false selection rates. Although the repeated measurements within each subjects are correlated, the sure screening property is theoretically established. To the best of our knowledge, screening for longitudinal data rarely appeared in the literatures, and our method can be regarded as a nontrivial extension of nonparametric independence screening method. An extensive simulation study is conducted to illustrate the finite sample performance of the proposed method and procedure. Finally, the proposed method is applied to a yeast cycle gene expression data set to identify cell cycle-regulated genes and transcription factors.