Heng Lian

Bayesian quantile regression for single-index models

Using an asymmetric Laplace distribution, which provides a mechanism for Bayesian inference of quantile regression models, we develop a fully Bayesian approach to fitting single-index models in conditional quantile regression. In this work, we use a Gaussian process prior for the unknown nonparametric link function and a Laplace distribution on the index vector, with the latter motivated by the recent popularity of the Bayesian lasso idea. We design a Markov chain Monte Carlo algorithm for posterior inference. Careful consideration of the singularity of the kernel matrix, and tractability of some of the full conditional distributions leads to a partially collapsed approach where the nonparametric link function is integrated out in some of the sampling steps. Our simulations demonstrate the superior performance of the Bayesian method versus the frequentist approach. The method is further illustrated by an application to the hurricane data.

Gaussian process single-index models as emulators for computer experiments

A single-index model (SIM) provides for parsimonious multidimensional nonlinear regression by combining parametric (linear) projection with univariate nonparametric (nonlinear) regression models. We show that a particular Gaussian process (GP) formulation is simple to work with and ideal as an emulator for some types of computer experiment as it can outperform the canonical separable GP regression model commonly used in this setting. Our contribution focuses on drastically simplifying, reinterpreting, and then generalizing a recently proposed fully Bayesian GP-SIM combination. Favorable performance is illustrated on synthetic data and a real-data computer experiment. Two R packages, both released on CRAN, have been augmented to facilitate inference under our proposed model(s).

Bias-corrected GEE estimation and smooth-threshold GEE variable selection for single-index models with clustered data

In this paper, we present an estimation approach based on generalized estimating equations and a variable selection procedure for single-index models when the observed data are clustered. Unlike the case of independent observations, bias-correction is necessary when general working correlation matrices are used in the estimating equations. Our variable selection procedure based on smooth-threshold estimating equations (Ueki (2009) [23]) can automatically eliminate irrelevant parameters by setting them as zeros and is computationally simpler than alternative approaches based on shrinkage penalty. The resulting estimator consistently identifies the significant variables in the index, even when the working correlation matrix is misspecified. The asymptotic property of the estimator is the same whether or not the nonzero parameters are known (in both cases we use the same estimating equations), thus achieving the oracle property in the sense of Fan and Li (2001) [10]. The finite sample properties of the estimator are illustrated by some simulation examples, as well as a real data application.

Semiparametric Estimation of Additive Quantile Regression Models by Two-Fold Penalty

In this article, we propose a model selection and semiparametric estimation method for additive models in the context of quantile regression problems. In particular, we are interested in finding nonzero components as well as linear components in the conditional quantile function. Our approach is based on spline approximation for the components aided by two Smoothly Clipped Absolute Deviation (SCAD) penalty terms. The advantage of our approach is that one can automatically choose between general additive models, partially linear additive models, and linear models in a single estimation step. The most important contribution is that this is achieved without the need for specifying which covariates enter the linear part, solving one serious practical issue for models with partially linear additive structure. Simulation studies as well as a real dataset are used to illustrate our method.

Quadratic inference functions for partially linear single-index models with longitudinal data

In this paper, we consider the partially linear single-index models with longitudinal data. We propose the bias-corrected quadratic inference function (QIF) method to estimate the parameters in the model by accounting for the within-subject correlation. Asymptotic properties for the proposed estimation methods are demonstrated. A generalized likelihood ratio test is established to test the linearity of the nonparametric part. Under the null hypotheses, the test statistic follows asymptotically a χ2 distribution. We also evaluate the finite sample performance of the proposed methods via Monte Carlo simulation studies and a real data analysis.

Series expansion for functional sufficient dimension reduction

Functional data are infinite-dimensional statistical objects which pose significant challenges to both theorists and practitioners. Both parametric and nonparametric regressions have received attention in the functional data analysis literature. However, the former imposes stringent constraints while the latter suffers from logarithmic convergence rates. In this article, we consider two popular sufficient dimension reduction methods in the context of functional data analysis, which, if desired, can be combined with low-dimensional nonparametric regression in a later step. In computation, predictor processes and index vectors are approximated in finite dimensional spaces using the series expansion approach. In theory, the basis used can be either fixed or estimated, which include both functional principal components and B-spline basis. Thus our study is more general than previous ones. Numerical results from simulations and a real data analysis are presented to illustrate the methods.

Automatic variable selection for longitudinal generalized linear models

We consider the problem of variable selection for the generalized linear models (GLMs) with longitudinal data. An automatic variable selection procedure is developed using smooth-threshold generalized estimating equations (SGEE). The proposed procedure automatically eliminates inactive predictors by setting the corresponding parameters to be zero, and simultaneously estimates the nonzero regression coefficients by solving the SGEE. The proposed method shares some of the desired features of existing variable selection methods: the resulting estimator enjoys the oracle property; the proposed procedure avoids the convex optimization problem and is flexible and easy to implement. Moreover, we propose a penalized weighted deviance criterion for a data-driven choice of the tuning parameters. Simulation studies are carried out to assess the performance of SGEE, and a real dataset is analyzed for further illustration.

Identification of Partially Linear Structure in Additive Models with an Application to Gene Expression Prediction from Sequences

The additive model is a semiparametric class of models that has become extremely popular because it is more flexible than the linear model and can be fitted to high-dimensional data when fully nonparametric models become infeasible. We consider the problem of simultaneous variable selection and parametric component identification using spline approximation aided by two smoothly clipped absolute deviation (SCAD) penalties. The advantage of our approach is that one can automatically choose between additive models, partially linear additive models and linear models, in a single estimation step. Simulation studies are used to illustrate our method, and we also present its applications to motif regression.

Bayesian quantile regression for partially linear additive models

In this article, we develop a semiparametric Bayesian estimation and model selection approach for partially linear additive models in conditional quantile regression. The asymmetric Laplace distribution provides a mechanism for Bayesian inferences of quantile regression models based on the check loss. The advantage of this new method is that nonlinear, linear and zero function components can be separated automatically and simultaneously during model fitting without the need of pre-specification or parameter tuning. This is achieved by spike-and-slab priors using two sets of indicator variables. For posterior inferences, we design an effective partially collapsed Gibbs sampler. Simulation studies are used to illustrate our algorithm. The proposed approach is further illustrated by applications to two real data sets.

Variable selection and estimation for partially linear single-index models with longitudinal data

In this paper, we consider the partially linear single-index models with longitudinal data. To deal with the variable selection problem in this context, we propose a penalized procedure combined with two bias correction methods, resulting in the bias-corrected generalized estimating equation and the bias-corrected quadratic inference function, which can take into account the correlations. Asymptotic properties of these methods are demonstrated. We also evaluate the finite sample performance of the proposed methods via Monte Carlo simulation studies and a real data analysis.