Kevin H. Eng

Erratum: area under the precision-recall curve: point estimates and confidence intervals

The area under the precision-recall curve (AUCPR) is a single number summary of the information in the precision-recall (PR) curve. Similar to the receiver operating characteristic curve, the PR curve has its own unique properties that make estimating its enclosed area challenging. Besides a point estimate of the area, an interval estimate is often required to express magnitude and uncertainty. In this paper we perform a computational analysis of common AUCPR estimators and their confidence intervals. We find both satisfactory estimates and invalid procedures and we recommend two simple intervals that are robust to a variety of assumptions.

Whole-genome sequencing identifies genomic heterogeneity at a nucleotide and chromosomal level in bladder cancer

Significance Genetic alterations are frequently observed in bladder cancer. In this study, we demonstrate that bladder tumors can be classified into two different types based on the spectrum of genetic diversity they confer. In one class of tumors, we observed tumor protein p53 mutations and a large number of single-nucleotide and structural variants. Another characteristic of this group was chromosome shattering, known as chromothripsis, and mutational heterogeneity. The other two bladder tumors did not show these profound genetic aberrations, but we found a novel translocation and amplification of the gene glutamate receptor ionotropic N-methyl D-aspertate, a potentially druggable target. Advancements in bladder cancer treatment have been slow. Understanding the genetic landscape of bladder cancer may therefore help to identify new therapeutic targets and bolster management of this disease. Using complete genome analysis, we sequenced five bladder tumors accrued from patients with muscle-invasive transitional cell carcinoma of the urinary bladder (TCC-UB) and identified a spectrum of genomic aberrations. In three tumors, complex genotype changes were noted. All three had tumor protein p53 mutations and a relatively large number of single-nucleotide variants (SNVs; average of 11.2 per megabase), structural variants (SVs; average of 46), or both. This group was best characterized by chromothripsis and the presence of subclonal populations of neoplastic cells or intratumoral mutational heterogeneity. Here, we provide evidence that the process of chromothripsis in TCC-UB is mediated by nonhomologous end-joining using kilobase, rather than megabase, fragments of DNA, which we refer to as “stitchers,” to repair this process. We postulate that a potential unifying theme among tumors with the more complex genotype group is a defective replication–licensing complex. A second group (two bladder tumors) had no chromothripsis, and a simpler genotype, WT tumor protein p53, had relatively few SNVs (average of 5.9 per megabase) and only a single SV. There was no evidence of a subclonal population of neoplastic cells. In this group, we used a preclinical model of bladder carcinoma cell lines to study a unique SV (translocation and amplification) of the gene glutamate receptor ionotropic N-methyl D-aspertate as a potential new therapeutic target in bladder cancer.

Expression and immune responses to MAGE antigens predict survival in epithelial ovarian cancer.

The MAGE cancer-testis antigens (CTA) are attractive candidates for immunotherapy. The aim of this study was to determine the frequency of expression, humoral immunity and prognostic significance of MAGE CTA in human epithelial ovarian cancer (EOC). mRNA or protein expression frequencies were determined for MAGE-A1, -A3, -A4, -A10 and -C1 (CT7) in tissue samples obtained from 400 patients with EOC. The presence of autologous antibodies against the MAGE antigens was determined from 285 serum samples. The relationships between MAGE expression, humoral immunity to MAGE antigens, and clinico-pathologic characteristics were studied. The individual frequencies of expression were as follows: A1: 15% (42/281), A3: 36% (131/390), A4: 47% (186/399), A10: 52% (204/395), C1: 16% (42/267). Strong concordant expression was noted with MAGE-A1:–A4, MAGE-A1:–C1 and MAGE-A4:–A10 (p<0.0005). Expression of MAGE-A1 or -A10 antigens resulted in poor progression free survival (PFS) (OR 1.44, CI 1.01–2.04, p = 0.044 and OR 1.3, CI 1.03–1.64, p = 0.03, respectively); whereas, MAGE-C1 expression was associated with improved PFS (OR 0.62, CI 0.42–0.92, p = 0.016). The improved PFS observed for MAGE-C1 expression, was diminished by co-expression of MAGE-A1 or -A10. Spontaneous humoral immunity to the MAGE antigens was present in 9% (27/285) of patients, and this predicted poor overall survival (log-rank test p = 0.0137). These findings indicate that MAGE-A1, MAGE-A4, MAGE-A3, and MAGE-A10 are priority attractive targets for polyvalent immunotherapy in ovarian cancer patients.

Cytokine profiling of ascites at primary surgery identifies an interaction of tumor necrosis factor-α and interleukin-6 in predicting reduced progression-free survival in epithelial ovarian cancer

OBJECTIVES Epithelial ovarian cancer (EOC) typically presents with advanced disease. Even with optimal debulking and response to adjuvant chemotherapy, the majority of patients will have disease relapse. We evaluated cytokine and chemokine profiles in ascites at primary surgery as biomarkers for progression-free survival (PFS) and overall survival (OS) in patients with advanced EOC. METHODS Retrospective analysis of patients (n =70) who underwent surgery at Roswell Park Cancer Institute between 2002 and 2012, followed by platinum-based chemotherapy. RESULTS The mean age at diagnosis was 61.8 years, 85.3% had serous EOC, and 95.7% had stage IIIB, IIIC, or IV disease. Univariate analysis showed that ascites levels of tumor necrosis factor (TNF)-α were associated with reduced PFS after primary surgery. Although the ascites concentration of interleukin (IL)-6 was not by itself predictive of PFS, we found that stratifying patients by high TNF-α and high IL-6 levels identified a sub-group of patients at high risk for rapid disease relapse. This effect was largely independent of clinical prognostic variables. CONCLUSIONS The combination of high TNF-α and high IL-6 ascites levels at primary surgery predicts worse PFS in patients with advanced EOC. These results suggest an interaction between ascites TNF-α and IL-6 in driving tumor progression and resistance to chemotherapy in advanced EOC, and raise the potential for pre-treatment ascites levels of these cytokines as prognostic biomarkers. This study involved a small sample of patients and was an exploratory analysis; therefore, findings require validation in a larger independent cohort.

Evaluation of the National Surgical Quality Improvement Program Universal Surgical Risk Calculator for a gynecologic oncology service

Objectives The National Surgical Quality Improvement Program is aimed at preventing perioperative complications. An online calculator was recently published, but the primary studies used limited gynecologic surgery data. The purpose of this study was to evaluate the performance of the National Surgical Quality Improvement Program Universal Surgical Risk Calculator (URC) on the patients of a gynecologic oncology service. Study Design We reviewed 628 consecutive surgeries performed by our gynecologic oncology service between July 2012 and June 2013. Demographic data including diagnosis and cancer stage, if applicable, were collected. Charts were reviewed to determine complication rates. Specific complications were as follows: death, pneumonia, cardiac complications, surgical site infection (SSI) or urinary tract infection, renal failure, or venous thromboembolic event. Data were compared with modeled outcomes using Brier scores and receiver operating characteristic curves. Significance was declared based on P < 0.05. Results The model accurately predicated death and venous thromboembolic event, with Brier scores of 0.004 and 0.003, respectively. Predicted risk was 50% greater than experienced for urinary tract infection; the experienced SSI and pneumonia rates were 43% and 36% greater than predicted. For any complication, the Brier score 0.023 indicates poor performance of the model. Conclusions In this study of gynecologic surgeries, we could not verify the predictive value of the URC for cardiac complications, SSI, and pneumonia. One disadvantage of applying a URC to multiple subspecialties is that with some categories, complications are not accurately estimated. Our data demonstrate that some predicted risks reported by the calculator need to be interpreted with reservation.

Transient Genotype-by-Environment Interactions Following Environmental Shock Provide a Source of Expression Variation for Essential Genes

Understanding complex genotype-by-environment interactions (GEIs) is crucial for understanding phenotypic variation. An important factor often overlooked in GEI studies is time. We measured the contribution of GEIs to expression variation in four nonlaboratory Saccharomyces cerevisiae strains responding dynamically to a 25°–37° heat shock. GEI was a major force explaining expression variation, affecting 55% of the genes analyzed. Importantly, almost half of these expression patterns showed GEI influence only during the transition between environments, but not in acclimated cells. This class reveals a genotype-by-environment-by-time interaction that affected expression of a large fraction of yeast genes. Strikingly, although transcripts subject to persistent GEI effects were enriched for nonessential genes with upstream TATA elements, those displaying transient GEIs were enriched for essential genes regardless of TATA regulation. Genes subject to persistent GEI influences showed relaxed constraint on acclimated gene expression compared to the average yeast gene, whereas genes restricted to transient GEIs did not. We propose that transient GEI during the transition between environments provides a previously unappreciated source of expression variation, particularly for essential genes.

On representing the prognostic value of continuous gene expression biomarkers with the restricted mean survival curve.

Motivation Researchers developing biomarkers for cancer prognosis from quantitative gene expression data are often faced with an odd methodological discrepancy: while Coxs proportional hazards model, the appropriate and popular technique, produces a continuous and relative risk score, it is hard to cast the estimate in clear clinical terms like median months of survival and percent of patients affected. To produce a familiar Kaplan-Meier plot, researchers commonly make the decision to dichotomize a continuous (often unimodal and symmetric) score. It is well known in the statistical literature that this procedure induces significant bias. Results We illustrate the liabilities of common techniques for categorizing a risk score and discuss alternative approaches. We promote the use of the restricted mean survival (RMS) and the corresponding RMS curve that may be thought of as an analog to the best fit line from simple linear regression. Conclusions Continuous biomarker workflows should be modified to include the more rigorous statistical techniques and descriptive plots described in this article. All statistics discussed can be computed via standard functions in the Survival package of the R statistical programming language. Example R language code for the RMS curve is presented in the appendix.

miR-17 deregulates a core RUNX1-miRNA mechanism of CBF acute myeloid leukemia

BackgroundCore Binding Factor acute myeloid leukemia (CBF-AML) with t(8;21) RUNX1-MTG8 or inv(16) CBFB-MYH11 fusion proteins often show upregulation of wild type or mutated KIT receptor. However, also non-CBF-AML frequently displays upregulated KIT expression. In the first part of this study we show that KIT expression can be also upregulated by miR-17, a regulator of RUNX1, the gene encoding a CBF subunit. Interestingly, both CBF leukemia fusion proteins and miR-17, which targets RUNX1-3′UTR, negatively affect a common core RUNX1-miRNA mechanism that forces myeloid cells into an undifferentiated, KIT-induced, proliferating state. In the second part of this study we took advantage of the conservation of the core RUNX1-miRNA mechanism in mouse and human, to mechanistically demonstrate in a mouse myeloid cell model that increased KIT-induced proliferation is per se a mechanism sufficient to delay myeloid differentiation.MethodsHuman (U937) or mouse (32D) myeloid clonal lines were used, respectively, to test: 1) the effect of RUNX1-MTG8 and CBFB-MYH11 fusion proteins, or upregulation of miR-17, on KIT-induced proliferation and myeloid differentiation, and 2) the effect of upregulation of KIT-induced proliferation per se on myeloid cell differentiation.ResultsIn the first part of this study we found that stable miR-17 upregulation affects, like the CBF-AML fusion proteins (RUNX1-MTG8 or CBFB-MYH11), a core RUNX1-miRNA mechanism leading to KIT-induced proliferation of differentiation-arrested U937 myeloid cells. In the second part of the study we harnessed the conservation of this core mechanism in human and mouse to demonstrate that the extent of KIT upregulation in 32D mouse myeloid cells with wild type RUNX1 can per se delay G-CSF-induced differentiation. The integrated information gathered from the two myeloid cell models shows that RUNX1 regulates myeloid differentiation not only by direct transcriptional regulation of coding and non-coding myeloid differentiation functions (e.g. miR-223), but also by modulating KIT-induced proliferation via non-coding miRNAs (e.g. miR-221).ConclusionsThe novelty of this study is dual. On the one hand, miRNAs (e.g. miR-17) can mimic the effects of CBF-AML fusion proteins by affecting a core RUNX1-miRNA mechanism of KIT-induced proliferation of undifferentiated myeloid cells. On the other hand, the extent of KIT-induced proliferation itself can modulate myeloid differentiation of cells with wild type RUNX1 function.

Pathway index models for construction of patient-specific risk profiles.

Statistical methods for variable selection, prediction, and classification have proven extremely useful in moving personalized genomics medicine forward, in particular, leading to a number of genomic-based assays now in clinical use for predicting cancer recurrence. Although invaluable in individual cases, the information provided by these assays is limited. Most often, a patient is classified into one of very few groups (e.g., recur or not), limiting the potential for truly personalized treatment. Furthermore, although these assays provide information on which individuals are at most risk (e.g., those for which recurrence is predicted), they provide no information on the aberrant biological pathways that give rise to the increased risk. We have developed an approach to address these limitations. The approach models a time-to-event outcome as a function of known biological pathways, identifies important genomic aberrations, and provides pathway-based patient-specific assessments of risk. As we demonstrate in a study of ovarian cancer from The Cancer Genome Atlas project, the patient-specific risk profiles are powerful and efficient characterizations useful in addressing a number of questions related to identifying informative patient subtypes and predicting survival.

Robust detection of immune transcripts in FFPE samples using targeted RNA sequencing

Current criteria for identifying cancer patients suitable for immunotherapy with immune checkpoint blockers (ICBs) are subjective and prone to misinterpretation, as they mainly rely on the visual assessment of CD274 (best known as PD-L1) expression levels by immunohistochemistry (IHC). To address this issue, we developed a RNA sequencing (RNAseq)-based approach that specifically measures the abundance of immune transcripts in formalin-fixed paraffin embedded (FFPE) specimens. Besides exhibiting superior sensitivity as compared to whole transcriptome RNAseq, our assay requires little starting material, implying that it is compatible with RNA degradation normally caused by formalin. Here, we demonstrate that a targeted RNAseq panel reliably profiles mRNA expression levels in FFPE samples from a cohort of ovarian carcinoma patients. The expression profile of immune transcripts as measured by targeted RNAseq in FFPE versus freshly frozen (FF) samples from the same tumor was highly concordant, in spite of the RNA quality issues associated with formalin fixation. Moreover, the results of targeted RNAseq on FFPE specimens exhibited a robust correlation with mRNA expression levels as measured on the same samples by quantitative RT-PCR, as well as with protein abundance as determined by IHC. These findings demonstrate that RNAseq profiling on archival FFPE tissues can be used reliably in studies assessing the efficacy of cancer immunotherapy.