Efstathia Bura

Extending Sliced Inverse Regression: the Weighted Chi-Squared Test

Sliced inverse regression (SIR) and an associated chi-squared test for dimension have been introduced as a method for reducing the dimension of regression problems whose predictor variables are normal. In this article the assumptions on the predictor distribution, under which the chi-squared test was proved to apply, are relaxed, and the result is extended. A general weighted chi-squared test that does not require normal regressors for the dimension of a regression is given. Simulations show that the weighted chi-squared test is more reliable than the chi-squared test when the regressor distribution digresses from normality significantly, and that it compares well with the chi-squared test when the regressors are normal.

Estimating the structural dimension of regressions via parametric inverse regression

A new estimation method for the dimension of a regression at the outset of an analysis is proposed. A linear subspace spanned by projections of the regressor vector X, which contains part or all of the modelling information for the regression of a vector Y on X, and its dimension are estimated via the means of parametric inverse regression. Smooth parametric curves are fitted to the p inverse regressions via a multivariate linear model. No restrictions are placed on the distribution of the regressors. The estimate of the dimension of the regression is based on optimal estimation procedures. A simulation study shows the method to be more powerful than sliced inverse regression in some situations.

Graphical methods for class prediction using dimension reduction techniques on DNA microarray data

MOTIVATION We introduce simple graphical classification and prediction tools for tumor status using gene-expression profiles. They are based on two dimension estimation techniques sliced average variance estimation (SAVE) and sliced inverse regression (SIR). Both SAVE and SIR are used to infer on the dimension of the classification problem and obtain linear combinations of genes that contain sufficient information to predict class membership, such as tumor type. Plots of the estimated directions as well as numerical thresholds estimated from the plots are used to predict tumor classes in cDNA microarrays and the performance of the class predictors is assessed by cross-validation. A microarray simulation study is carried out to compare the power and predictive accuracy of the two methods. RESULTS The methods are applied to cDNA microarray data on BRCA1 and BRCA2 mutation carriers as well as sporadic tumors from Hedenfalk et al. (2001). All samples are correctly classified.

The binary regression quantile plot : Assessing the importance of predictors in binary regression visually

We present a graphical measure of assessing the explanatory power of regression models with a binary response. The binary regression quantile plot and an area defined by it are used for the visual comparison and ordering of nested binary response regression models. The plot shows how well various models explain the data. Two data sets are analyzed and the area representing the fit of a model is shown to agree with the usual likelihood ratio test.

Dimension estimation in sufficient dimension reduction: A unifying approach

Sufficient Dimension Reduction (SDR) in regression comprises the estimation of the dimension of the smallest (central) dimension reduction subspace and its basis elements. For SDR methods based on a kernel matrix, such as SIR and SAVE, the dimension estimation is equivalent to the estimation of the rank of a random matrix which is the sample based estimate of the kernel. A test for the rank of a random matrix amounts to testing how many of its eigen or singular values are equal to zero. We propose two tests based on the smallest eigen or singular values of the estimated matrix: an asymptotic weighted chi-square test and a Wald-type asymptotic chi-square test. We also provide an asymptotic chi-square test for assessing whether elements of the left singular vectors of the random matrix are zero. These methods together constitute a unified approach for all SDR methods based on a kernel matrix that covers estimation of the central subspace and its dimension, as well as assessment of variable contribution to the lower-dimensional predictor projections with variable selection, a special case. A small power simulation study shows that the proposed and existing tests, specific to each SDR method, perform similarly with respect to power and achievement of the nominal level. Also, the importance of the choice of the number of slices as a tuning parameter is further exhibited.

Sufficient dimension reduction for longitudinally measured predictors

We propose a method to combine several predictors (markers) that are measured repeatedly over time into a composite marker score without assuming a model and only requiring a mild condition on the predictor distribution. Assuming that the first and second moments of the predictors can be decomposed into a time and a marker component via a Kronecker product structure that accommodates the longitudinal nature of the predictors, we develop first-moment sufficient dimension reduction techniques to replace the original markers with linear transformations that contain sufficient information for the regression of the predictors on the outcome. These linear combinations can then be combined into a score that has better predictive performance than a score built under a general model that ignores the longitudinal structure of the data. Our methods can be applied to either continuous or categorical outcome measures. In simulations, we focus on binary outcomes and show that our method outperforms existing alternatives by using the AUC, the area under the receiver-operator characteristics (ROC) curve, as a summary measure of the discriminatory ability of a single continuous diagnostic marker for binary disease outcomes.

Sufficient Reductions in Regressions With Exponential Family Inverse Predictors

ABSTRACT We develop methodology for identifying and estimating sufficient reductions in regressions with predictors that, given the response, follow a multivariate exponential family distribution. This setup includes regressions where predictors are all continuous, all categorical, or mixtures of categorical and continuous. We derive the minimal sufficient reduction of the predictors and its maximum likelihood estimator by modeling the conditional distribution of the predictors given the response. Whereas nearly all extant estimators of sufficient reductions are linear and only partly capture the sufficient reduction, our method is not limited to linear reductions. It also provides the exact form of the sufficient reduction, which is exhaustive, its maximum likelihood (ML) estimates via an iterated reweighted least-square (IRLS) estimation algorithm, and asymptotic tests for the dimension of the regression. Supplementary materials for this article are available online.

Sufficient Reductions in Regressions With Elliptically Contoured Inverse Predictors

There are two general approaches based on inverse regression for estimating the linear sufficient reductions for the regression of Y on X: the moment-based approach such as SIR, PIR, SAVE, and DR, and the likelihood-based approach such as principal fitted components (PFC) and likelihood acquired directions (LAD) when the inverse predictors, X|Y, are normal. By construction, these methods extract information from the first two conditional moments of X|Y; they can only estimate linear reductions and thus form the linear sufficient dimension reduction (SDR) methodology. When var(X|Y) is constant, E(X|Y) contains the reduction and it can be estimated using PFC. When var(X|Y) is nonconstant, PFC misses the information in the variance and second moment based methods (SAVE, DR, LAD) are used instead, resulting in efficiency loss in the estimation of the mean-based reduction. In this article we prove that (a) if X|Y is elliptically contoured with parameters and density gY, there is no linear nontrivial sufficient reduction except if gY is the normal density with constant variance; (b) for nonnormal elliptically contoured data, all existing linear SDR methods only estimate part of the reduction; (c) a sufficient reduction of X for the regression of Y on X comprises of a linear and a nonlinear component.

Assessing corrections to the weighted chi-squared test for dimension

Abstract The weighted chi-squared test for dimension is an extension to the chi-squared test associated with sliced inverse regression as it lifts restrictive distributional assumptions on the predictors. Its usage requires the estimation of the mixture weights which may affect accuracy, and the computationally intensive calculation of percentiles of a mixture chi-squared distribution. The scaled and adjusted chi-squared corrections to the mixture chi-squared distribution have been proposed as alternatives to the weighted chi-squared test. A simulation study assessing power performance of the four tests indicates that the computationally simple adjusted chi-squared test could be used in place of the weighted chi-squared test, whereas the scaled chi-squared test performs much worse.

Forecasting with Sufficient Dimension Reductions

Factor models have been successfully employed in summarizing large datasets with few underlying latent factors and in building time series forecasting models for economic variables. When the objective is to forecast a target variable y with a large set of predictors x, the construction of the summary of the xs should be driven by how informative on y it is. Most existing methods first reduce the predictors and then forecast y in independent phases of the modeling process. In this paper we present an alternative and potentially more attractive alternative: summarizing x as it relates to y, so that all the information in the conditional distribution of y|x is preserved. These y-targeted reductions of the predictors are obtained using Sufficient Dimension Reduction techniques. We show in simulations and real data analysis that forecasting models based on sufficient reductions have the potential of significantly improved performance.