Hemant Ishwaran

Least angle regression

DISCUSSION OF “LEAST ANGLE REGRESSION” BY EFRONET AL.By Jean-Michel Loubes and Pascal MassartUniversit´e Paris-SudThe issue of model selection has drawn the attention of both applied andtheoretical statisticians for a long time. Indeed, there has been an enor-mous range of contribution in model selection proposals, including work byAkaike (1973), Mallows (1973), Foster and George (1994), Birg´e and Mas-sart (2001a) and Abramovich, Benjamini, Donoho and Johnstone (2000).Over the last decade, modern computer-driven methods have been devel-oped such as All Subsets, Forward Selection, Forward Stagewise or Lasso.Such methods are useful in the setting of the standard linear model, wherewe observe noisy data and wish to predict the response variable using onlya few covariates, since they provide automatically linear models that fit thedata. The procedure described in this paper is, on the one hand, numeri-cally very efficient and, on the other hand, very general, since, with slightmodifications, it enables us to recover the estimates given by the Lasso andStagewise.1. Estimation procedure. The “LARS” method is based on a recursiveprocedure selecting, at each step, the covariates having largest absolute cor-relation with the response y. In the case of an orthogonal design, the esti-mates can then be viewed as an lDISCUSSION OF “LEAST ANGLE REGRESSION” BY EFRONET AL.By Berwin A. TurlachUniversity of Western AustraliaI would like to begin by congratulating the authors (referred to belowas EHJT) for their interesting paper in which they propose a new variableselection method (LARS) for building linear models and show how their newmethod relates to other methods that have been proposed recently. I foundthe paper to be very stimulating and found the additional insight that itprovides about the Lasso technique to be of particular interest.My comments center around the question of how we can select linearmodels that conform with the marginality principle [Nelder (1977, 1994)and McCullagh and Nelder (1989)]; that is, the response surface is invariantunder scaling and translation of the explanatory variables in the model.Recently one of my interests was to explore whether the Lasso techniqueor the nonnegative garrote [Breiman (1995)] could be modified such that itincorporates the marginality principle. However, it does not seem to be atrivial matter to change the criteria that these techniques minimize in such away that the marginality principle is incorporated in a satisfactory manner.On the other hand, it seems to be straightforward to modify the LARStechnique to incorporate this principle. In their paper, EHJT address thisissue somewhat in passing when they suggest toward the end of Section 3that one first fit main effects only and interactions in a second step to controlthe order in which variables are allowed to enter the model. However, sucha two-step procedure may have a somewhat less than optimal behavior asthe following, admittedly artificial, example shows.Assume we have a vector of explanatory variables X =(XThe purpose of model selection algorithms such as All Subsets, Forward Selection and Backward Elimination is to choose a linear model on the basis of the same set of data to which the model will be applied. Typically we have available a large collection of possible covariates from which we hope to select a parsimonious set for the efficient prediction of a response variable. Least Angle Regression (LARS), a new model selection algorithm, is a useful and less greedy version of traditional forward selection methods. Three main properties are derived: (1) A simple modification of the LARS algorithm implements the Lasso, an attractive version of ordinary least squares that constrains the sum of the absolute regression coefficients; the LARS modification calculates all possible Lasso estimates for a given problem, using an order of magnitude less computer time than previous methods. (2) A different LARS modification efficiently implements Forward Stagewise linear regression, another promising new model selection method; this connection explains the similar numerical results previously observed for the Lasso and Stagewise, and helps us understand the properties of both methods, which are seen as constrained versions of the simpler LARS algorithm. (3) A simple approximation for the degrees of freedom of a LARS estimate is available, from which we derive a Cp estimate of prediction error; this allows a principled choice among the range of possible LARS estimates. LARS and its variants are computationally efficient: the paper describes a publicly available algorithm that requires only the same order of magnitude of computational effort as ordinary least squares applied to the full set of covariates.

Gibbs Sampling Methods for Stick-Breaking Priors

A rich and flexible class of random probability measures, which we call stick-breaking priors, can be constructed using a sequence of independent beta random variables. Examples of random measures that have this characterization include the Dirichlet process, its two-parameter extension, the two-parameter Poisson–Dirichlet process, finite dimensional Dirichlet priors, and beta two-parameter processes. The rich nature of stick-breaking priors offers Bayesians a useful class of priors for nonparametric problems, while the similar construction used in each prior can be exploited to develop a general computational procedure for fitting them. In this article we present two general types of Gibbs samplers that can be used to fit posteriors of Bayesian hierarchical models based on stick-breaking priors. The first type of Gibbs sampler, referred to as a Pólya urn Gibbs sampler, is a generalized version of a widely used Gibbs sampling method currently employed for Dirichlet process computing. This method applies to stick-breaking priors with a known Pólya urn characterization, that is, priors with an explicit and simple prediction rule. Our second method, the blocked Gibbs sampler, is based on an entirely different approach that works by directly sampling values from the posterior of the random measure. The blocked Gibbs sampler can be viewed as a more general approach because it works without requiring an explicit prediction rule. We find that the blocked Gibbs avoids some of the limitations seen with the Pólya urn approach and should be simpler for nonexperts to use.

Radiation and dual checkpoint blockade activate non-redundant immune mechanisms in cancer

Immune checkpoint inhibitors result in impressive clinical responses, but optimal results will require combination with each other and other therapies. This raises fundamental questions about mechanisms of non-redundancy and resistance. Here we report major tumour regressions in a subset of patients with metastatic melanoma treated with an anti-CTLA4 antibody (anti-CTLA4) and radiation, and reproduced this effect in mouse models. Although combined treatment improved responses in irradiated and unirradiated tumours, resistance was common. Unbiased analyses of mice revealed that resistance was due to upregulation of PD-L1 on melanoma cells and associated with T-cell exhaustion. Accordingly, optimal response in melanoma and other cancer types requires radiation, anti-CTLA4 and anti-PD-L1/PD-1. Anti-CTLA4 predominantly inhibits T-regulatory cells (Treg cells), thereby increasing the CD8 T-cell to Treg (CD8/Treg) ratio. Radiation enhances the diversity of the T-cell receptor (TCR) repertoire of intratumoral T cells. Together, anti-CTLA4 promotes expansion of T cells, while radiation shapes the TCR repertoire of the expanded peripheral clones. Addition of PD-L1 blockade reverses T-cell exhaustion to mitigate depression in the CD8/Treg ratio and further encourages oligoclonal T-cell expansion. Similarly to results from mice, patients on our clinical trial with melanoma showing high PD-L1 did not respond to radiation plus anti-CTLA4, demonstrated persistent T-cell exhaustion, and rapidly progressed. Thus, PD-L1 on melanoma cells allows tumours to escape anti-CTLA4-based therapy, and the combination of radiation, anti-CTLA4 and anti-PD-L1 promotes response and immunity through distinct mechanisms.

Random survival forests

We introduce random survival forests, a random forests method for the analysis of right-censored survival data. New survival splitting rules for growing survival trees are introduced, as is a new missing data algorithm for imputing missing data. A conservation-of-events principle for survival forests is introduced and used to define ensemble mortality, a simple interpretable measure of mortality that can be used as a predicted outcome. Several illustrative examples are given, including a case study of the prognostic implications of body mass for individuals with coronary artery disease. Computations for all examples were implemented using the freely available R-software package, randomSurvivalForest. 1. Introduction. In this article we introduce random survival forests ,a n ensemble tree method for analysis of right-censored survival data. As is well known, constructing ensembles from base learners, such as trees, can substantially improve prediction performance. Recently it has been shown by Breiman (2001 )t hat ensemble learning can be improved further by injecting randomization into the base learning process, an approach called random forests. Random survival forests (RSF) methodology extends Breiman’s random forests (RF) method. In RF, randomization is introduced in two forms. First, a randomly drawn bootstrap sample of the data is used to grow a tree. Second, at each node of the tree, a randomly selected subset of variables (covariates) is chosen as candidate variables for splitting. Averaging over trees, in combination with the randomization used in growing a tree, enables RF to approximate rich classes of functions while maintaining low generalization error. Considerable empirical evidence has shown RF to be highly accurate, comparable to state-of-the-art methods such as bagging [Breiman (1996)], boosting [Schapire et al. (1998)], and support vector machines [Cortes and Vapnik (1995)]. Until now, applications of RF have focused primarily on classification and regression problems. Even the popular R-software package randomForest [Liaw and Wiener (2002, 2007)] considers only regression and multiclass data settings, not survival analysis. Extending random forests to right-censored survival data is

Spike and slab variable selection: Frequentist and Bayesian strategies

Variable selection in the linear regression model takes many apparent faces from both frequentist and Bayesian standpoints. In this paper we introduce a variable selection method referred to as a rescaled spike and slab model. We study the importance of prior hierarchical specifications and draw connections to frequentist generalized ridge regression estimation. Specifically, we study the usefulness of continuous bimodal priors to model hypervariance parameters, and the effect scaling has on the posterior mean through its relationship to penalization. Several model selection strategies, some frequentist and some Bayesian in nature, are developed and studied theoretically. We demonstrate the importance of selective shrinkage for effective variable selection in terms of risk misclassification, and show this is achieved using the posterior from a rescaled spike and slab model. We also show how to verify a procedures ability to reduce model uncertainty in finite samples using a specialized forward selection strategy. Using this tool, we illustrate the effectiveness of rescaled spike and slab models in reducing model uncertainty.

Cancer of the esophagus and esophagogastric junction: data-driven staging for the seventh edition of the American Joint Committee on Cancer/International Union Against Cancer Cancer Staging Manuals.

Previous American Joint Committee on Cancer/International Union Against Cancer (AJCC/UICC) stage groupings for esophageal cancer have not been data driven or harmonized with stomach cancer. At the request of the AJCC, worldwide data from 3 continents were assembled to develop data‐driven, harmonized esophageal staging for the seventh edition of the AJCC/UICC cancer staging manuals.

Lung metastasis genes couple breast tumor size and metastatic spread

The association between large tumor size and metastatic risk in a majority of clinical cancers has led to questions as to whether these observations are causally related or whether one is simply a marker for the other. This is partly due to an uncertainty about how metastasis-promoting gene expression changes can arise in primary tumors. We investigated this question through the analysis of a previously defined “lung metastasis gene-expression signature” (LMS) that mediates experimental breast cancer metastasis selectively to the lung and is expressed by primary human breast cancer with a high risk for developing lung metastasis. Experimentally, we demonstrate that the LMS promotes primary tumor growth that enriches for LMS+ cells, and it allows for intravasation after reaching a critical tumor size. Clinically, this corresponds to LMS+ tumors being larger at diagnosis compared with LMS− tumors and to a marked rise in the incidence of metastasis after LMS+ tumors reach 2 cm. Patients with LMS-expressing primary tumors selectively fail in the lung compared with the bone or other visceral sites and have a worse overall survival. The mechanistic linkage between metastasis gene expression, accelerated tumor growth, and likelihood of metastatic recurrence provided by the LMS may help to explain observations of prognostic gene signatures in primary cancer and how tumor growth can both lead to metastasis and be a marker for cells destined to metastasize.

Exosome Transfer from Stromal to Breast Cancer Cells Regulates Therapy Resistance Pathways

Stromal communication with cancer cells can influence treatment response. We show that stromal and breast cancer (BrCa) cells utilize paracrine and juxtacrine signaling to drive chemotherapy and radiation resistance. Upon heterotypic interaction, exosomes are transferred from stromal to BrCa cells. RNA within exosomes, which are largely noncoding transcripts and transposable elements, stimulates the pattern recognition receptor RIG-I to activate STAT1-dependent antiviral signaling. In parallel, stromal cells also activate NOTCH3 on BrCa cells. The paracrine antiviral and juxtacrine NOTCH3 pathways converge as STAT1 facilitates transcriptional responses to NOTCH3 and expands therapy-resistant tumor-initiating cells. Primary human and/or mouse BrCa analysis support the role of antiviral/NOTCH3 pathways in NOTCH signaling and stroma-mediated resistance, which is abrogated by combination therapy with gamma secretase inhibitors. Thus, stromal cells orchestrate an intricate crosstalk with BrCa cells by utilizing exosomes to instigate antiviral signaling. This expands BrCa subpopulations adept at resisting therapy and reinitiating tumor growth.

Optimum lymphadenectomy for esophageal cancer.

Objective:Using Worldwide Esophageal Cancer Collaboration data, we sought to (1) characterize the relationship between survival and extent of lymphadenectomy, and (2) from this, define optimum lymphadenectomy. Summary Background Data:What constitutes optimum lymphadenectomy to maximize survival is controversial because of variable goals, analytic methodology, and generalizability of the underpinning data. Methods:A total of 4627 patients who had esophagectomy alone for esophageal cancer were identified from the Worldwide Esophageal Cancer Collaboration database. Patient-specific risk-adjusted survival was estimated using random survival forests. Risk-adjusted 5-year survival was averaged for each number of lymph nodes resected and its relation to cancer characteristics explored. Optimum number of nodes that should be resected to maximize 5-year survival was determined by random forest multivariable regression. Results:For pN0M0 moderately and poorly differentiated cancers, and all node-positive (pN+) cancers, 5-year survival improved with increasing extent of lymphadenectomy. In pN0M0 cancers, no optimum lymphadenectomy was defined for pTis; optimum lymphadenectomy was 10 to 12 nodes for pT1, 15 to 22 for pT2, and 31 to 42 for pT3/T4, depending on histopathologic cell type. In pN+M0 cancers and 1 to 6 nodes positive, optimum lymphadenectomy was 10 for pT1, 15 for pT2, and 29 to 50 for pT3/T4. Conclusions:Greater extent of lymphadenectomy was associated with increased survival for all patients with esophageal cancer except at the extremes (TisN0M0 and ≥7 regional lymph nodes positive for cancer) and well-differentiated pN0M0 cancer. Maximum 5-year survival is modulated by T classification: resecting 10 nodes for pT1, 20 for pT2, and ≥30 for pT3/T4 is recommended.

An interferon-related gene signature for DNA damage resistance is a predictive marker for chemotherapy and radiation for breast cancer

Individualization of cancer management requires prognostic markers and therapy-predictive markers. Prognostic markers assess risk of disease progression independent of therapy, whereas therapy-predictive markers identify patients whose disease is sensitive or resistant to treatment. We show that an experimentally derived IFN-related DNA damage resistance signature (IRDS) is associated with resistance to chemotherapy and/or radiation across different cancer cell lines. The IRDS genes STAT1, ISG15, and IFIT1 all mediate experimental resistance. Clinical analyses reveal that IRDS(+) and IRDS(−) states exist among common human cancers. In breast cancer, a seven–gene-pair classifier predicts for efficacy of adjuvant chemotherapy and for local-regional control after radiation. By providing information on treatment sensitivity or resistance, the IRDS improves outcome prediction when combined with standard markers, risk groups, or other genomic classifiers.