Kenneth J. Rothman

Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations

Misinterpretation and abuse of statistical tests, confidence intervals, and statistical power have been decried for decades, yet remain rampant. A key problem is that there are no interpretations of these concepts that are at once simple, intuitive, correct, and foolproof. Instead, correct use and interpretation of these statistics requires an attention to detail which seems to tax the patience of working scientists. This high cognitive demand has led to an epidemic of shortcut definitions and interpretations that are simply wrong, sometimes disastrously so—and yet these misinterpretations dominate much of the scientific literature. In light of this problem, we provide definitions and a discussion of basic statistics that are more general and critical than typically found in traditional introductory expositions. Our goal is to provide a resource for instructors, researchers, and consumers of statistics whose knowledge of statistical theory and technique may be limited but who wish to avoid and spot misinterpretations. We emphasize how violation of often unstated analysis protocols (such as selecting analyses for presentation based on the P values they produce) can lead to small P values even if the declared test hypothesis is correct, and can lead to large P values even if that hypothesis is incorrect. We then provide an explanatory list of 25 misinterpretations of P values, confidence intervals, and power. We conclude with guidelines for improving statistical interpretation and reporting.

Increasing levels of restriction in pharmacoepidemiologic database studies of elderly and comparison with randomized trial results

Background:The goal of restricting study populations is to make patients more homogeneous regarding potential confounding factors and treatment effects and thereby achieve less biased effect estimates. Objectives:This article describes increasing levels of restrictions for use in pharmacoepidemiology and examines to what extent they change rate ratio estimates and reduce bias in a study of statin treatment and 1-year mortality. Methods:The study cohort was drawn from a population of seniors age 65 years and older enrolled in both Medicare and the Pennsylvania Pharmaceutical Assistance Contract for the Elderly (PACE) between 1995 and 2002. We identified all users of statins during the study period and assessed the time until death within 1 year. The following progressive restrictions were applied: (1) study incident drug users only, (2) choose a comparison group most similar to the intervention group, (3) exclude patients with contraindications, (4) exclude patients with low adherence, and (5) restrict to specific high-risk/low-risk subgroups represented in randomized trails (RCTs). Results:The basic cohort comprised 122,406 statin users, who were on average 78 years old and predominantly white (93%) and showed an unadjusted rate ratio of 0.32 for statin users. When all 5 restrictions were applied (N = 11,673), the unadjusted rate ratio had increased to 0.72. Multivariable Cox regression adjusted rate ratios increased from 0.62 [95% confidence interval (CI), 0.58–0.66] to 0.79 (95% CI, 0.60–1.03). However, after the first 3 restrictions the effect size changed little. The final estimate is similar to that obtained as a pooled estimate of 3 pravastatin RCTs in patients age 65 years and older. We argue that restrictions 1 through 4 compromised generalizability little. Conclusions:In our example of a large database study, restricting to incident drug users, similar comparison groups, patients without contraindication, and to adherent patients was a practical strategy, which limited the effect of confounding, as these approaches yield results closer to those seen in RCTs.

One-to-many propensity score matching in cohort studies.

Among the large number of cohort studies that employ propensity score matching, most match patients 1:1. Increasing the matching ratio is thought to improve precision but may come with a trade‐off with respect to bias.

Antithrombotic Treatment Patterns in Patients with Newly Diagnosed Nonvalvular Atrial Fibrillation: The GLORIA-AF Registry, Phase II

BACKGROUNDnThe Global Registry on Long-Term Oral Antithrombotic Treatment in Patients with Atrial Fibrillation (GLORIA-AF) was designed to provide prospectively collected information on patients with newly diagnosed nonvalvular atrial fibrillation at risk of stroke, with the aim of addressing treatment patterns and questions of effectiveness and safety.nnnMETHODS AND RESULTSnIn this predefined analysis from GLORIA-AF, the baseline characteristics and initial antithrombotic management of the first 10,000 patients in Phase II of this large Registry Program are presented. Overall, 32.3% of patients received vitamin K antagonists (VKAs) and 47.7% received non-VKA oral anticoagulants (NOACs), while 12.3% received antiplatelet treatment and 7.6% did not receive any antithrombotic treatment. Among patients with CHA2DS2-VASc score ≥2, 6.7% received no antithrombotic treatment and 10.0% received aspirin. In Europe, treatment with dabigatran was as common as treatment with VKAs (38.8% and 37.8%, respectively). More than half of the patients were treated with NOACs (52.4%), while antiplatelet treatment was given to 5.7%, and 4.1% did not receive any antithrombotic treatment. In North America, treatment with dabigatran (25.0%) was as common as with VKAs (26.1%), but overall NOAC use was more common (52.1%) than with VKAs (26.1%); however, 14.1% received antiplatelet treatment, while 7.6% received no antithrombotic treatment. In Asia, treatment with VKAs (31.9%) was more prevalent than NOACs (25.5%), but antiplatelet treatment was given to 25.8%, and 16.9% did not receive any antithrombotic treatment. In Asia, only 60.7% of patients with high stroke risk received oral anticoagulants (OACs). Paroxysmal atrial fibrillation and minimally symptomatic (or asymptomatic) patients were often undertreated with OACs.nnnCONCLUSIONnIn this analysis, OAC use was high in Europe and North America, with overall NOAC use higher than VKA use. A considerable percentage of high-risk patients in North America still received antiplatelet treatment or were untreated, while Asian patients had a high proportion of aspirin use and nontreatment.

Subgroup analyses to determine cardiovascular risk associated with nonsteroidal antiinflammatory drugs and coxibs in specific patient groups

OBJECTIVEnTo explore the extent to which clinical characteristics influence the association between cyclooxygenase 2 inhibitors (coxibs) and/or nonselective nonsteroidal antiinflammatory drugs (NSAIDs) and increased cardiovascular disease (CVD) risk in specific patient subgroups. There is substantial concern regarding the potential cardiovascular adverse effects of selective coxibs and nonselective NSAIDs, but many patients with arthritis experience important clinical benefits from these agents.nnnMETHODSnThe study population consisted of Medicare beneficiaries also eligible for a drug benefits program for older adults during the years 1999-2004. We calculated the relative risk (RR) for CVD events (myocardial infarction [MI], stroke, congestive heart failure, and cardiovascular death) among users of coxibs or nonselective NSAIDs in the prior 6 months compared with nonusers. We assessed biologic interaction between these medication exposures and important patient characteristics.nnnRESULTSnIn the primary cohort, we identified 76,082 new users of coxibs, 53,014 new users of nonselective NSAIDs, and 46,558 nonusers. Compared with nonusers, the adjusted RR of CVD events for new users of each agent increased for rofecoxib (RR 1.22, 95% confidence interval [95% CI] 1.14, 1.30) and decreased for naproxen (RR 0.79, 95% CI 0.67, 0.93). Several patient characteristics were found to increase the risk of CVD events among users of some agents in both the primary and secondary cohorts, including age >/=80 years, hypertension, prior MI, prior CVD, rheumatoid arthritis, chronic renal disease, and chronic obstructive pulmonary disease. Rofecoxib and ibuprofen appeared to confer an increased risk in multiple patient subgroups.nnnCONCLUSIONnMany nonselective NSAIDs and coxibs are not associated with an increased risk of CVD events. However, several patient characteristics identify important subgroups that may be at an increased risk when using specific agents.

The implications of propensity score variable selection strategies in pharmacoepidemiology – an empirical illustration

To examine the effect of variable selection strategies on the performance of propensity score (PS) methods in a study of statin initiation, mortality, and hip fracture assuming a true mortality reduction of <u200915% and no effect on hip fracture.

Obesity and mortality risk among younger dialysis patients

BACKGROUND AND OBJECTIVESnMany studies show that obesity in dialysis patients is not strongly associated with mortality but not whether this modest association is constant over age. This study investigated the extent to which the relation of body mass index (BMI) and mortality differs between younger and older dialysis patients.nnnDESIGN, SETTING, PARTICIPANTS, & MEASUREMENTSnAdult dialysis patients were prospectively followed from their first dialysis treatment for 7 years or until death or transplantation. Patients were stratified by age (<65 or ≥65 years) and baseline BMI (<20, 20-24 [reference], 25-29, and ≥30 kg/m(2)).nnnRESULTSnThe study sample included 984 patients younger than 65 years and 765 patients 65 years or older; cumulative survival proportions at end of follow-up were 50% and 16%. Age-standardized mortality rate was 1.7 times higher in obese younger patients than those with normal BMI, corresponding to an excess rate of 5.2 deaths/100 patient-years. Mortality rates were almost equal between obese older patients and those with normal BMI. Excess rates of younger and older patients with low compared with normal BMI were 8.7 and 1.1 deaths/100 patient-years. After adjustment for age, sex, smoking, comorbidity, and treatment modality, hazard ratios by increasing BMI were 2.00, 1, 0.95, and 1.57 for younger patients and 1.07, 1, 0.88, and 0.91 for older patients, implying that obesity is a 1.7-fold (95% confidence interval, 1.1- to 2.9-fold) stronger risk factor in younger than older patients.nnnCONCLUSIONSnIn contrast to older dialysis patients, younger patients with low or very high BMI had a substantially elevated risk for death.

Evaluation of cancer risk related to atopic dermatitis and use of topical calcineurin inhibitors.

Cases of lymphoma or cutaneous cancer have been observed following use of topical calcineurin inhibitors (TCIs), but it is unclear whether TCI use increases cancer risk. We used published literature to assess the extent to which atopic dermatitis (AD) or TCI use is associated with lymphoma, melanoma, basal cell carcinoma and squamous cell carcinoma. We searched the literature and summarized the results of all studies that provided data on the absolute or relative frequency of any malignancy among patients with AD or eczema or among patients using TCIs. The relative risk for all lymphoma in broad populations of AD or eczema ranged from 0·7 to 1·8. Available data on lymphoma following TCI use were inconsistent and insufficient to draw a conclusion about the causal role of TCIs. We found no evidence indicating that melanoma or nonmelanoma skin cancer is associated with TCI use. A bias analysis showed that cutaneous T‐cell lymphomas initially misdiagnosed and treated as AD would lead to overestimation of the association between TCI use and lymphoma. However, there are only sparse data on specific malignancies among TCI‐treated patients. The short duration of typical TCI exposure hinders conclusions about longer exposure. There is insufficient evidence in the epidemiological literature to infer whether TCIs do or do not cause malignancy.

OBSERVE-5: Observational postmarketing safety surveillance registry of etanercept for the treatment of psoriasis final 5-year results

BACKGROUNDnOBSERVE-5 was a 5-year Food and Drug Administration-mandated surveillance registry of patients with psoriasis.nnnOBJECTIVEnWe sought to assess long-term etanercept safety and effectiveness.nnnMETHODSnPatients with moderate to severe psoriasis enrolled; a single baseline dose of etanercept was required. Key outcome measures included serious adverse events, serious infectious events, events of medical interest, psoriasis-affected body surface area, physician global assessment score, and Dermatology Life Quality Index score. Safety outcomes were assessed relative to data from the MarketScan database.nnnRESULTSnFor 2510 patients, 5-year cumulative incidence was 22.2% (95% confidence interval [CI] 20.3%-24.2%) for serious adverse events; 6.5% (95% CI 5.4%-7.7%) for serious infectious events; 3.2% (95% CI 2.3%-4.1%) for malignancies excluding nonmelanoma skin cancer; 3.6% (95% CI 2.7%-4.5%) for nonmelanoma skin cancer; 2.8% (95% CI 2.0%-3.6%) for coronary artery disease; 0.7% (95% CI 0.3%-1.2%) for psoriasis worsening; 0.2% (95% CI 0.0%-0.4%) for central nervous system demyelinating disorder; 0.1% (95% CI 0.0%-0.3%) for lymphoma and for tuberculosis; and 0.1% (95% CI 0.0%-0.2%) for opportunistic infection and for lupus; 55 fatal events were reported. Rates of malignancies, lymphomas, nonmelanoma skin cancer, and hospitalization-associated infections were not higher than expected relative to administrative claims data. The percentage of patients rated as clear/almost clear was 12% at baseline, which increased to 51% at month 6 and remained relatively stable throughout 5 years.nnnLIMITATIONSnNo internal comparator group was included; rare events may not have been detected.nnnCONCLUSIONnNo new safety signals were observed with long-term, real-world etanercept use.

Incidence of osteonecrosis of the jaw among users of bisphosphonates with selected cancers or osteoporosis

To quantify the incidence of osteonecrosis of the jaw (ONJ) by bisphosphonate exposure among two cohorts of patients.