Lori E. Dodd

Partial AUC Estimation and Regression

Accurate diagnosis of disease is a critical part of health care. New diagnostic and screening tests must be evaluated based on their abilities to discriminate diseased from nondiseased states. The partial area under the receiver operating characteristic (ROC) curve is a measure of diagnostic test accuracy. We present an interpretation of the partial area under the curve (AUC), which gives rise to a nonparametric estimator. This estimator is more robust than existing estimators, which make parametric assumptions. We show that the robustness is gained with only a moderate loss in efficiency. We describe a regression modeling framework for making inference about covariate effects on the partial AUC. Such models can refine knowledge about test accuracy. Model parameters can be estimated using binary regression methods. We use the regression framework to compare two prostate-specific antigen biomarkers and to evaluate the dependence of biomarker accuracy on the time prior to clinical diagnosis of prostate cancer.

Blinded Independent Central Review of Progression-Free Survival in Phase III Clinical Trials: Important Design Element or Unnecessary Expense?

Progression-free survival is an important end point in advanced disease settings. Blinded independent central review (BICR) of progression in randomized clinical trials has been advocated to control bias that might result from errors in progression assessments. However, although BICR lessens some potential biases, it does not remove all biases from evaluations of treatment effectiveness. In fact, as typically conducted, BICRs may introduce bias because of informative censoring, which results from having to censor unconfirmed locally determined progressions. In this article, we discuss the rationale for BICR and different ways of implementing independent review. We discuss the limitations of these approaches and review published trials that report implementing BICR. We demonstrate the existence of informative censoring using data from a randomized phase II trial. We conclude that double-blinded trials with consistent application of measurement criteria are the best means of ensuring unbiased trial results. When such designs are not practical, BICR is not recommended as a general strategy for reducing bias. However, BICR may be useful as an auditing tool to assess the reliability of marginally positive results.

Frequency, severity, and prediction of tuberculous meningitis immune reconstitution inflammatory syndrome.

Tuberculous meningitis (TBM) immune reconstitution inflammatory syndrome is a severe complication of antiretroviral therapy in human immunodeficiency virus–associated TBM. We found that high cerebrospinal fluid neutrophil counts and Mycobacterium tuberculosis culture positivity at TBM presentation characterize, and cytokine concentrations predict, this syndrome.

Profiling human serum antibodies with a carbohydrate antigen microarray.

Carbohydrate antigen arrays (glycan arrays) have been recently developed for the high-throughput analysis of carbohydrate macromolecule interactions. When profiling serum, information about experimental variability, interindividual biological variability, and intraindividual temporal variability is critical. In this report, we describe the characterization of a carbohydrate antigen array and assay for profiling human serum. Through optimization of assay conditions and development of a normalization strategy, we obtain highly reproducible results with a within-experiment coefficient of variation (CV) of 10.8% and an overall CV (across multiple batches of slides and days) of 28.5%. We also report antibody profiles for 48 human subjects and evaluate for the first time the effects of age, race, sex, geographic location, and blood type on antibody profiles for a large set of carbohydrate antigens. We found significant dependence on age and blood type of antibody levels for a variety of carbohydrates. Finally, we conducted a longitudinal study with a separate group of 7 serum donors to evaluate the variation in anti-carbohydrate antibody levels within an individual over a period ranging from 3 to 13 weeks and found that, for nearly all antigens on our array, antibody levels are generally stable over this period. The results presented here provide the most comprehensive evaluation of experimental and biological variation reported to date for a glycan array and have significant implications for studies involving human serum profiling.

Evaluation of tuberculosis diagnostics in children: 2. Methodological issues for conducting and reporting research evaluations of tuberculosis diagnostics for intrathoracic tuberculosis in children. Consensus from an expert panel.

Confirming the diagnosis of childhood tuberculosis is a major challenge. However, research on childhood tuberculosis as it relates to better diagnostics is often neglected because of technical difficulties, such as the slow growth in culture, the difficulty of obtaining specimens, and the diverse and relatively nonspecific clinical presentation of tuberculosis in this age group. Researchers often use individually designed criteria for enrollment, diagnostic classifications, and reference standards, thereby hindering the interpretation and comparability of their findings. The development of standardized research approaches and definitions is therefore needed to strengthen the evaluation of new diagnostics for detection and confirmation of tuberculosis in children. In this article we present consensus statements on methodological issues for conducting research of Tuberculosis diagnostics among children, with a focus on intrathoracic tuberculosis. The statements are complementary to a clinical research case definition presented in an accompanying publication and suggest a phased approach to diagnostics evaluation; entry criteria for enrollment; methods for classification of disease certainty, including the rational use of culture within the case definition; age categories and comorbidities for reporting results; and the need to use standard operating procedures. Special consideration is given to the performance of microbiological culture in children and we also recommend for alternative methodological approaches to report findings in a standardized manner to overcome these limitations are made. This consensus statement is an important step toward ensuring greater rigor and comparability of pediatric tuberculosis diagnostic research, with the aim of realizing the full potential of better tests for children.

PET/CT imaging correlates with treatment outcome in patients with multidrug-resistant tuberculosis

PET/CT imaging in humans with TB correlates with drug response and final treatment outcomes. Visualizing Drug Responses in TB A pair of papers by Chen et al. and Coleman et al. investigate how changes in quantitative positron emission tomography/computed tomography (PET/CT) scans in both nonhuman primates and humans can be used as early surrogate markers of treatment efficacy in tuberculosis. The Coleman et al. study shows that treatment of Mtb-infected macaques with linezolid and the second-generation oxazolidinone AZD5847 resulted in a reduced bacterial load at necropsy and reduced FDG PET avidity and CT-quantified lung pathology. Similar PET/CT changes were seen in human patients infected with extensively drug-resistant Mtb and treated with linezolid. The companion study by Chen et al. corroborated this effect in a prospective analysis of patients with multidrug-resistant tuberculosis and demonstrated that early PET/CT changes predicted final treatment outcomes. Definitive clinical trials of new chemotherapies for treating tuberculosis (TB) require following subjects until at least 6 months after treatment discontinuation to assess for durable cure, making these trials expensive and lengthy. Surrogate endpoints relating to treatment failure and relapse are currently limited to sputum microbiology, which has limited sensitivity and specificity. We prospectively assessed radiographic changes using 2-deoxy-2-[18F]-fluoro-d-glucose (FDG) positron emission tomography/computed tomography (PET/CT) at 2 and 6 months (CT only) in a cohort of subjects with multidrug-resistant TB, who were treated with second-line TB therapy for 2 years and then followed for an additional 6 months. CT scans were read semiquantitatively by radiologists and were computationally evaluated using custom software to provide volumetric assessment of TB-associated abnormalities. CT scans at 6 months (but not 2 months) assessed by radiologist readers were predictive of outcomes, and changes in computed abnormal volumes were predictive of drug response at both time points. Quantitative changes in FDG uptake 2 months after starting treatment were associated with long-term outcomes. In this cohort, some radiologic markers were more sensitive than conventional sputum microbiology in distinguishing successful from unsuccessful treatment. These results support the potential of imaging scans as possible surrogate endpoints in clinical trials of new TB drug regimens. Larger cohorts confirming these results are needed.

PET/CT imaging reveals a therapeutic response to oxazolidinones in macaques and humans with tuberculosis

PET/CT imaging in macaques and humans with TB shows a beneficial therapeutic response to linezolid and a new oxazolidinone antibiotic, AZD5847. Visualizing Drug Responses in TB A pair of papers by Chen et al. and Coleman et al. investigate how changes in quantitative PET/CT scans in both nonhuman primates and humans can be used as early surrogate markers of treatment efficacy in tuberculosis. The Coleman et al. study shows that treatment of Mtb-infected macaques with linezolid and the second-generation oxazolidinone AZD5847 resulted in a reduced bacterial load at necropsy and reduced FDG PET avidity and CT-quantified lung pathology. Similar PET/CT changes were seen in human patients infected with extensively drug-resistant Mtb and treated with linezolid. The companion study by Chen et al. corroborated this effect in a prospective analysis of patients with multidrug-resistant tuberculosis and demonstrated that early PET/CT changes predicted final treatment outcomes. Oxazolidinone antibiotics such as linezolid have shown significant therapeutic effects in patients with extensively drug-resistant (XDR) tuberculosis (TB) despite modest effects in rodents and no demonstrable early bactericidal activity in human phase 2 trials. We show that monotherapy with either linezolid or AZD5847, a second-generation oxazolidinone, reduced bacterial load at necropsy in Mycobacterium tuberculosis–infected cynomolgus macaques with active TB. This effect coincided with a decline in 2-deoxy-2-[18F]-fluoro-d-glucose positron emission tomography (FDG PET) imaging avidity in the lungs of these animals and with reductions in pulmonary pathology measured by serial computed tomography (CT) scans over 2 months of monotherapy. In a parallel phase 2 clinical study of linezolid in patients infected with XDR-TB, we also collected PET/CT imaging data from subjects receiving linezolid that had been added to their failing treatment regimens. Quantitative comparisons of PET/CT imaging changes in these human subjects were similar in magnitude to those observed in macaques, demonstrating that the therapeutic effect of these oxazolidinones can be reproduced in this model of experimental chemotherapy. PET/CT imaging may be useful as an early quantitative measure of drug efficacy against TB in human patients.

Semiparametric Regression for the Area Under the Receiver Operating Characteristic Curve

Medical advances continue to provide new and potentially better means for detecting disease. Such is true in cancer, for example, where biomarkers are sought for early detection and where improvements in imaging methods may pick up the initial functional and molecular changes associated with cancer development. In other binary classification tasks, computational algorithms such as neural networks, support vector machines, and evolutionary algorithms have been applied to areas as diverse as credit scoring, object recognition, and peptide-binding prediction. Before a classifier becomes an accepted technology, it must undergo rigorous evaluation to determine its ability to discriminate between states. Characterization of factors influencing classifier performance is an important step in this process. Analysis of covariates may reveal subpopulations in which classifier performance is greatest or identify features of the classifier that improve accuracy. We develop regression methods for the nonparametric area under the receiver operating characteristic curve, a well-accepted summary measure of classifier accuracy. The estimating function generalizes standard approaches and, interestingly, is related to the two-sample Mann–Whitney U statistic. Implementation is straightforward, because it is an adaptation of binary regression methods. Asymptotic theory is nonstandard, because the regressor variables are cross-correlated. Nevertheless, simulation studies show that the method produces estimates with small bias and reasonable coverage probability. Application of the method to evaluate the covariate effects on a new device for diagnosing hearing impairment reveals that the device performs better in more severely impaired subjects and that certain test parameters, which are adjustable by the device operator, are key to test performance.

Genes Involved in DNA Repair and Nitrosamine Metabolism and Those Located on Chromosome 14q32 Are Dysregulated in Nasopharyngeal Carcinoma

Polymorphisms in nitrosamine metabolism, DNA repair, and immune response genes have been associated with nasopharyngeal carcinoma (NPC). Studies have suggested chromosomal regions involved in NPC. To shed light on NPC etiology, we evaluated host gene expression patterns in 31 NPC and 10 normal nasopharyngeal tissue specimens using the Affymetrix Human Genome U133 Plus 2.0 Array. We focused on genes in five a priori biological pathways and chromosomal locations. Rates of differential expression within these prespecified lists and overall were tested using a bootstrap method. Differential expression was observed for 7.6% of probe sets overall. Elevations in rate of differential expression were observed within the DNA repair (13.7%; P = 0.01) and nitrosamine metabolism (17.5%; P = 0.04) pathways. Differentially expressed probe sets within the DNA repair pathway were consistently overexpressed (93%), with strong effects observed for PRKDC, PCNA, and CHEK1. Differentially expressed probe sets within the nitrosamine metabolism pathway were consistently underexpressed (100%), with strong effects observed for NQ01, CYP2B6, and CYP2E1. No significant evidence of increases in rate of differential expression was seen within the immune/inflammatory pathway. A significant elevation in rate of differential expression was noted for chromosome 4p15.1-4q12 (13.0%; P = 0.04); both overexpression and underexpression were evident (38% and 62%, respectively). An elevation in the rate of differential expression on chromosome 14q32 was observed (11.3%; P = 0.06) with a consistent pattern of gene underexpression (100%; P < 0.0001). These effects were similar when excluding late-stage tumors. Our results suggest that nitrosamine activation and DNA repair are important in NPC. The consistent down-regulation of expression on chromosome 14q32 suggests loss of heterozygosity in this region. (Cancer Epidemiol Biomarkers Prev 2006;15(11):2216–25)

Efficacy and Safety of Metronidazole for Pulmonary Multidrug-Resistant Tuberculosis

ABSTRACT Pulmonary lesions from active tuberculosis patients are thought to contain persistent, nonreplicating bacilli that arise from hypoxic stress. Metronidazole, approved for anaerobic infections, has antituberculosis activity against anoxic bacilli in vitro and in some animal models and may target persistent, nonreplicating bacilli. In this double-blind, placebo-controlled trial, pulmonary multidrug-resistant tuberculosis subjects were randomly assigned to receive metronidazole (500 mg thrice daily) or placebo for 8 weeks in addition to an individualized background regimen. Outcomes were measured radiologically (change on high-resolution computed tomography [HRCT]), microbiologically (time to sputum smear and culture conversion), and clinically (status 6 months after stopping therapy). Enrollment was stopped early due to excessive peripheral neuropathies in the metronidazole arm. Among 35 randomized subjects, 31 (15 metronidazole, 16 placebo) were included in the modified intent-to-treat analysis. There were no significant differences by arm in improvement of HRCT lesions from baseline to 2 or 6 months. More subjects in the metronidazole arm converted their sputum smear (P = 0.04) and liquid culture (P = 0.04) to negative at 1 month, but these differences were lost by 2 months. Overall, 81% showed clinical success 6 months after stopping therapy, with no differences by arm. However, 8/16 (50%) of subjects in the metronidazole group and 2/17 (12%) of those in the placebo group developed peripheral neuropathy. Subjects who received metronidazole were 4.3-fold (95% confidence interval [CI], 1.1 to 17.1) more likely to develop peripheral neuropathies than subjects who received placebo. Metronidazole may have increased early sputum smear and culture conversion but was too neurotoxic to use over the longer term. Newer nitroimidazoles with both aerobic and anaerobic activity, now in clinical trials, may increase the sterilizing potency of future treatment regimens.