Syed Ejaz Ahmed

Submodel Selection and Post-Estimation of the Linear Mixed Models

Linear mixed models (LMM) are popular in a host of business and engineering applications. In this paper, we consider estimation of the regression parameter vector of the LMM when some of the predictors are suspected to be insignificant for prediction purpose. In many practical situations, the investigators may have some information about the important predictors in a given model. Such information, known as uncertain prior information (UPI), could originate from subjective judgement of the investigator based on acquaintance with the experimental or observational data. Further, it is possible to obtain such information based on a variable selection technique, which we refer to as auxiliary information (AE). In any event, whether the information is subjective or data-driven, the resulting submodels are subject to model selection bias. Consequentially, the estimators based on a selected submodel will be biased if the submodel is misspecified. On the other hand, the estimates based on a full model (including all the predictors) may have large variation and/or subject to interpretability issues. To deal with these issues, in the context of two competing models (full model and submodel), we suggest linear shrinkage and shrinkage pretest estimation strategies which combine full model and submodel estimators in an effective way as a trade-off between bias and variance. We examine the performance of the suggested estimators relative to the full model estimator by theoretically using the mean square criterion. We also conduct a Monte Carlo simulation study to assess the performance of listed estimation strategies numerically. Our proposed shrinkage and pretest estimators perform better than the benchmark estimator in a meaningful way. The proposed method is applied to the analysis of a real data set.

Robust gene–environment interaction analysis using penalized trimmed regression

ABSTRACT In biomedical and epidemiological studies, gene–environment (G–E) interactions have been shown to importantly contribute to the etiology and progression of many complex diseases. Most existing approaches for identifying G–E interactions are limited by the lack of robustness against outliers/contaminations in response and predictor spaces. In this study, we develop a novel robust G–E identification approach using the trimmed regression technique under joint modelling. A robust data-driven criterion and stability selection are adopted to determine the trimmed subset which is free from both vertical outliers and leverage points. An effective penalization approach is developed to identify important G–E interactions, respecting the ‘main effects, interactions’ hierarchical structure. Extensive simulations demonstrate the better performance of the proposed approach compared to multiple alternatives. Interesting findings with superior prediction accuracy and stability are observed in the analysis of The Cancer Genome Atlas data on cutaneous melanoma and breast invasive carcinoma.

A Comparison of Pretest, Stein-Type and Penalty Estimators in Logistic Regression Model

Various estimators are proposed based on the preliminary test and Stein-type strategies to estimate the parameters in a logistic regression model when it is priori suspected that some parameters may be restricted to a subspace. Two different penalty estimators as LASSO and ridge regression are also considered. A Monte Carlo simulation experiment was conducted for different combinations, and the performance of each estimator was evaluated in terms of simulated relative efficiency. The positive-part Stein-type shrinkage estimator is recommended for use since its performance is robust regardless of the reliability of the subspace information. The proposed estimators are applied to a real dataset to appraise their performance.

Accounting for Clustering and Non-ignorable Missingness in Binomial Responses: Application to the Canadian National Survey on Child Safety Seat

Motivated by a Canadian national survey on child safety seat use, we propose a procedure for incorporating non-ignorable missingness in a binary response using logistic regression model with random effects. The proposed method applies an expectation maximization (EM) algorithm to the Penalized quasi-likelihood of the artificially completed data. We provide closed form formulae for estimating the covariance of the regression coefficients as well as the odds ratios of interest and probabilities of missingness. The proposed algorithm can be easily implemented by using familiar statistical packages that fit generalized linear mixed models. The method is illustrated by applying it to data from Canadian national survey on child safety seat use.

High Dimensional Data Analysis: Integrating Submodels

We consider an efficient prediction in sparse high dimensional data. In high dimensional data settings where d ≫ n, many penalized regularization strategies are suggested for simultaneous variable selection and estimation. However, different strategies yield a different submodel with d i < n, where d i represents the number of predictors included in ith submodel. Some procedures may select a submodel with a larger number of predictors than others. Due to the trade-off between model complexity and model prediction accuracy, the statistical inference of model selection becomes extremely important and challenging in high dimensional data analysis. For this reason we suggest shrinkage and pretest strategies to improve the prediction performance of two selected submodels. Such a pretest and shrinkage strategy is constructed by shrinking an overfitted model estimator in the direction of an underfitted model estimator. The numerical studies indicate that our post-selection pretest and shrinkage strategy improved the prediction performance of selected submodels.

Robust Estimation of Capability Index

The process capability indices (PCI) have been popular in the manufacturing environment to quantify the capability of an industrial process. In this paper, a robust estimator of the \(C_p\) index, based on the Gini mean difference statistic, is proposed. The performance of the proposed estimator and its associated confidence intervals are compared to those associated with the classical estimator based on the sample standard deviation. The use of the new method is illustrated by application to data set about membrane thickness of STN color pixels.

Generating Distributions Through Convolution of Characteristic Functions

Financial data such as asset returns, exchange rates, or option prices cannot be modeled effectively by classical distributions such as the Gaussian. These types of data have probability density functions that are thick-tailed and negatively skewed. To account for these features, we propose a new method of generating classes of distribution functions through convolution of smooth and non-smooth characteristic functions where the smoothing parameter is used to control the thickness of the density tails. To illustrate the advantages of using such class of distributions, we consider special cases in which the smooth characteristic functions are of those of the uniform, the normal and the compact supported cosine distributions and the non-smooth is the characteristic function of the Cauchy distribution. As a comparison criterion between distributions, we use the Stiltjes-Hamburger conditions for moments’ existence and show how the proposed distributions outperform the Student and Pearson IV distributions, which are commonly used by financial engineers to model stock returns.

Improved robust ridge M-estimation

ABSTRACT It is developed that non-sample prior information about regression vector-parameter, usually in the form of constraints, improves the risk performance of the ordinary least squares estimator (OLSE) when it is shrunken. However, in practice, it may happen that both multicollinearity and outliers exist simultaneously in the data. In such a situation, the use of robust ridge estimator is suggested to overcome the undesirable effects of the OLSE. In this article, some prior information in the form of constraints is employed to improve the performance of this estimator in the multiple regression model. In this regard, shrinkage ridge robust estimators are defined. Advantages of the proposed estimators over the usual robust ridge estimator are also investigated using Monte-Carlo simulation as well as a real data example.

Efficient estimation for the conditional autoregressive model

Recently, spatial regression models have been attracting a great deal of attention in areas ranging from effect of traffic congestion on accident rates to the analysis of trends in gastric cancer mortality. In this paper, we propose efficient estimators for the regression coefficients of the spatial conditional autoregressive model, when uncertain auxiliary information is available about these coefficients. We provide efficiency comparisons of the proposed estimators based on asymptotic risk analysis and Monte Carlo simulations. We apply the proposed methods to real data on Boston housing prices and illustrate how a bootstrapping approach can be employed to compute prediction errors of the estimators.