Nicholas J. Horton

Much ado about nothing: A comparison of missing data methods and software to fit incomplete data regression models.

Missing data are a recurring problem that can cause bias or lead to inefficient analyses. Statistical methods to address missingness have been actively pursued in recent years, including imputation, likelihood, and weighting approaches. Each approach is more complicated when there are many patterns of missing values, or when both categorical and continuous random variables are involved. Implementations of routines to incorporate observations with incomplete variables in regression models are now widely available. We review these routines in the context of a motivating example from a large health services research dataset. While there are still limitations to the current implementations, and additional efforts are required of the analyst, it is feasible to incorporate partially observed values, and these methods should be used in practice.

Multiple Imputation in Practice: Comparison of Software Packages for Regression Models With Missing Variables

Missing data frequently complicates data analysis for scientific investigations. The development of statistical methods to address missing data has been an active area of research in recent decades. Multiple imputation, originally proposed by Rubin in a public use dataset setting, is a general purpose method for analyzing datasets with missing data that is broadly applicable to a variety of missing data settings. We review multiple imputation as an analytic strategy for missing data. We describe and evaluate a number of software packages that implement this procedure, and contrast the interface, features, and results. We compare the packages, and detail shortcomings and useful features. The comparisons are illustrated using examples from an artificial dataset and a study of child psychopathology. We suggest additional features as well as discuss limitations and cautions to consider when using multiple imputation as an analytic strategy for incomplete data settings.

Strategy for intention to treat analysis in randomised trials with missing outcome data.

Loss to follow-up is often hard to avoid in randomised trials. This article suggests a framework for intention to treat analysis that depends on making plausible assumptions about the missing data and including all participants in sensitivity analyses

Review of Software to Fit Generalized Estimating Equation Regression Models

Abstract Researchers are often interested in analyzing data that arise from a longitudinal or clustered design. Although there are a variety of standard likelihood-based approaches to analysis when the outcome variables are approximately multivariate normal, models for discrete-type outcomes generally require a different approach. Liang and Zeger formalized an approach to this problem using generalized estimating equations (GEEs) to extend generalized linear models (GLMs) to a regression setting with correlated observations within subjects. In this article, we briefly review GLM, the GEE methodology, introduce some examples, and compare the GEE implementations of several general purpose statistical packages (SAS, Stata, SUDAAN, and S-Plus). We focus on the user interface, accuracy, and completeness of implementations of this methodology.

A Potential for Bias When Rounding in Multiple Imputation

With the advent of general purpose packages that support multiple imputation for analyzing datasets with missing data (e.g., Solas, SAS PROC MI, and S-Plus 6.0), we expect much greater use of multiple imputation in the future. For simplicity, some imputation packages assume the joint distribution of the variables in the multiple imputation model is multivariate normal, and impute the missing data from the conditional normal distribution for the missing data given the observed data. If the possibly missing data are not multivariate normal (say, binary), imputing a normal random variable can yield implausible values. To circumvent this problem, a number of methods have been developed, including rounding the imputed normal to the closest observed value in the dataset. We show that this rounding can cause biased estimates of parameters, whereas if the imputed value is not rounded, no bias would occur. This article shows that rounding should not be used indiscriminately, and thus some caution should be exercised when rounding imputed values, particularly for dichotomous variables.

Assessing missed opportunities for HIV testing in medical settings

BACKGROUND: Many HIV-infected persons learn about their diagnosis years after initial infection. The extent to which missed opportunities for HIV testing occur in medical evaluations prior to one’s HIV diagnosis is not known. DESIGN: We performed a 10-year retrospective chart review of patients seen at an HIV intake clinic between January 1994 and June 2001 who 1) tested positive for HIV during the 12 months prior to their presentation at the intake clinic and 2) had at least one encounter recorded in the medical record prior to their HIV-positive status. Data collection included demographics, clinical presentation, and whether HIV testing was recommended to the patient or addressed in any way in the clinical note. Prespecified triggers for physicians to recommend HIV testing, such as specific patient characteristics, symptoms, and physical findings, were recorded for each visit. Multivariable logistic regression was used to identify factors associated with missed opportunities for discussion of HIV testing. Generalized estimating equations were used to account for multiple visits per subject. RESULTS: Among the 221 patients meeting eligibility criteria, all had triggers for HIV testing found in an encounter note. Triggers were found in 50% (1,702/3,424) of these 221 patients’ medical visits. The median number of visits per patient prior to HIV diagnosis to this single institution was 5; 40% of these visits were to either the emergency department or urgent care clinic. HIV was addressed in 27% of visits in which triggers were identified. The multivariable regression model indicated that patients were more likely to have testing addressed in urgent care clinic (39%), sexually transmitted disease clinic (78%), primary care clinics (32%), and during hospitalization (47%), compared to the emergency department (11%), obstetrics/gynecology (9%), and other specialty clinics (10%) (P<.0001). More recent clinical visits (1997–2001) were more likely to have HIV addressed than earlier visits (P<.0001). Women were offered testing less often than men (P=.07). CONCLUSIONS: Missed opportunities for addressing HIV testing remain unacceptably high when patients seek medical care in the period before their HIV diagnosis. Despite improvement in recent years, variation by site of care remained important. In particular, the emergency department merits consideration for increased resource commitment to facilitate HIV testing. In order to detect HIV infection prior to advanced immunosuppression, clinicians must become more aware of clinical triggers that suggest a patient’s increased risk for this infection and lower the threshold at which HIV testing is recommended.

Randomized controlled trial of proactive web-based alcohol screening and brief intervention for university students

BACKGROUND University students drink more heavily than their nonstudent peers and are often unaware that their drinking is risky and exceeds normative levels. We tested the efficacy of a proactive Web-based alcohol screening and brief intervention program. METHODS A randomized controlled trial was conducted at an Australian university in 2007. Invitations were sent to 13 000 undergraduates (age range, 17-24 years) to complete a Web-based Alcohol Use Disorders Identification Test. Of 7237 students who responded, 2435 scored in the hazardous/harmful range (> or =8) and were randomized, and 2050 (84%) completed at least 1 follow-up assessment. Intervention was 10 minutes of Web-based motivational assessment and personalized feedback. Controls received only screening. Follow-up assessments were conducted at 1 and 6 months with observers and participants blinded to allocation. Outcome measures were drinking frequency, typical occasion quantity, overall volume, number of personal problems, an academic problems score, prevalence of binge drinking, and prevalence of heavy drinking. RESULTS Mean (SD) baseline Alcohol Use Disorders Identification Test scores for control and intervention groups were 14.3 (5.1) and 14.2 (5.1), respectively. After 1 month, participants receiving intervention drank less often (rate ratio [RR], 0.89; 95% confidence interval [CI], 0.83-0.94), smaller quantities per occasion (RR, 0.93; 95% CI, 0.88-0.98), and less alcohol overall (RR, 0.83; 95% CI, 0.78-0.90) than did controls. Differences in alcohol-related harms were nonsignificant. At 6 months, intervention effects persisted for drinking frequency (RR, 0.91; 95% CI, 0.85-0.97) and overall volume (RR, 0.89; 95% CI, 0.82-0.96) but not for other variables. CONCLUSION Proactive Web-based screening and intervention reduces drinking in undergraduates, and such a program could be implemented widely.

Longitudinal Associations Between Binge Eating and Overeating and Adverse Outcomes Among Adolescents and Young Adults Does Loss of Control Matter

OBJECTIVE To investigate the association between overeating (without loss of control) and binge eating (overeating with loss of control) and adverse outcomes. DESIGN Prospective cohort study. SETTING Adolescents and young adults living throughout the United States. PARTICIPANTS Sixteen thousand eight hundred eighty-two males and females participating in the Growing Up Today Study who were 9 to 15 years old at enrollment in 1996. MAIN EXPOSURE Overeating and binge eating assessed via questionnaire every 12 to 24 months between 1996 and 2005. MAIN OUTCOME MEASURES Risk of becoming overweight or obese, starting to binge drink frequently, starting to use marijuana, starting to use other drugs, and developing high levels of depressive symptoms. Generalized estimating equations were used to estimate associations. All models controlled for age and sex; additional covariates varied by outcome. RESULTS Among this large cohort of adolescents and young adults, binge eating was more common among females than males. In fully adjusted models, binge eating, but not overeating, was associated with incident overweight/obesity (odds ratio, 1.73; 95% CI, 1.11-2.69) and the onset of high depressive symptoms (odds ratio, 2.19; 95% CI, 1.40-3.45). Neither overeating nor binge eating was associated with starting to binge drink frequently, while both overeating and binge eating predicted starting to use marijuana and other drugs. CONCLUSIONS Although any overeating, with or without loss of control, predicted the onset of marijuana and other drug use, we found that binge eating is uniquely predictive of incident overweight/obesity and the onset of high depressive symptoms. These findings suggest that loss of control is an important indicator of severity of overeating episodes.

Including all individuals is not enough: Lessons for intention-to-treat analysis

Background Intention-to-treat (ITT) analysis requires all randomised individuals to be included in the analysis in the groups to which they were randomised. However, there is confusion about how ITT analysis should be performed in the presence of missing outcome data. Purposes To explain, justify, and illustrate an ITT analysis strategy for randomised trials with incomplete outcome data. Methods We consider several methods of analysis and compare their underlying assumptions, plausibility, and numbers of individuals included. We illustrate the ITT analysis strategy using data from the UK700 trial in the management of severe mental illness. Results Depending on the assumptions made about the missing data, some methods of analysis that include all randomised individuals may be less valid than methods that do not include all randomised individuals. Furthermore, some methods of analysis that include all randomised individuals are essentially equivalent to methods that do not include all randomised individuals. Limitations This work assumes that the aim of analysis is to obtain an accurate estimate of the difference in outcome between randomised groups and not to obtain a conservative estimate with bias against the experimental intervention. Conclusions Clinical trials should employ an ITT analysis strategy, comprising a design that attempts to follow up all randomised individuals, a main analysis that is valid under a stated plausible assumption about the missing data, and sensitivity analyses that include all randomised individuals in order to explore the impact of departures from the assumption underlying the main analysis. Following this strategy recognises the extra uncertainty arising from missing outcomes and increases the incentive for researchers to minimise the extent of missing data.

Brief intervention for medical inpatients with unhealthy alcohol use: a randomized, controlled trial

Context Brief interventions reduce alcohol use in outpatients who drink unhealthy amounts but are not alcohol-dependent. Their effect in medical inpatients is unknown. Contribution The authors screened all adult medical inpatients at an urban teaching hospital and randomly assigned 341 risky drinkers to a 30-minute motivational counseling intervention followed gy treatment planning or to usual care. By 3 months, the same proportion of patients from both groups had received alcohol assistance, and both groups had reduced their drinking to the same degree. Cautions Three quarters of the participants met Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, criteria for alcohol dependence. Implications In this well-done study, brief intervention did not affect alcohol-related outcomes in persons who drank unhealthy amounts. The Editors Professional organizations recommend that clinicians screen their patients for unhealthy alcohol use (that is, the spectrum from drinking risky amounts to dependence) and conduct a brief intervention when indicated (1, 2). Despite this recommendation and the existence of brief, valid screening tools (35), patients with unhealthy alcohol use often are not identified and do not receive timely care. Although widely recommended, brief intervention has proven efficacy in decreasing alcohol consumption and related consequences only in unhealthy drinkers without alcohol dependence and in outpatient settings (6). Its efficacy among other populations (for example, persons with alcohol dependence) and in inpatient settings remains unclear (7). Evidence suggests, however, that medical inpatientsa group with a high prevalence of alcohol-related problemsmay benefit from brief intervention. Some studies have demonstrated the efficacy of brief intervention in settings similar to medical services in which alcohol-related problems are common and their related consequences are severe (8, 9). Further, brief interventions are well suited to medical services. Patients who otherwise might not seek care are accessible and have time for an intervention. Persons admitted because of an alcohol-related problem may recognize the link between drinking and hospitalization, thus providing a teachable moment (10). Also, busy staff might implement a brief intervention because of its brevity and flexibility. The unmet need for alcohol screening and intervention and opportunities for implementation underscore the importance of determining the efficacy of brief intervention in medical inpatients with unhealthy alcohol use. In addition, evaluating its effectiveness and practicality in real-world settings is critical to help clinicians make informed decisions when treating their patients (11). Therefore, we conducted a randomized, controlled trial to examine whether screening followed by brief intervention would improve alcohol-related outcomes in typical medical inpatients (that is, a racially diverse group with a range of unhealthy alcohol use, comorbid conditions, and readiness to change). We hypothesized that screening and brief intervention would lead to the following: receipt of alcohol assistance (for example, specialty treatment) among persons with alcohol dependence and, among all persons decreased alcohol consumption, alcohol-related problems, and health care utilization and improved readiness to change and health-related quality of life. Methods Patients As previously described, we recruited patients from the inpatient medical service of a large, urban teaching hospital (12). Trained research associates approached all patients who were age 18 years or older and whose physicians did not decline patient contact. Patients fluent in English or Spanish who gave verbal consent were asked to complete a screening interview to determine eligibility: currently (past month) drinking risky amounts (defined for eligibility as >14 standard drinks/wk or 5 drinks/occasion for men and >11 drinks/wk or 4 drinks/occasion for women and persons 66 years); 2 contacts to assist with follow-up; no plans to move from the area in the next year; and a Mini-Mental State Examination score of 21 or greater (13, 14). Research associates assessed demographic characteristics and administered the Alcohol Use Disorders Identification Test (AUDIT) (15) by interview. To better characterize current alcohol use, they assessed the average numbers of drinking-days per week and drinks consumed on a typical day, and the maximum number of drinks consumed per occasion (16, 17). For the first 7 months of the study, research associates asked these additional questions only to patients with an AUDIT score of 8 or greater (a recommended cutoff for screening) (18). For the remaining 22 months, research associates asked the additional questions to anyone who drank in the past 12 months to maximize identification of drinkers of risky amounts. Lastly, the research associates asked all patients who were drinking risky amounts to describe their readiness to change by using a visual analog scale ranging from 0 to 10 (19). Enrolled patients provided written informed consent and were compensated for each completed interview. The institutional review board at Boston University Medical Center approved this study. We secured additional privacy protection with a certificate of confidentiality from the National Institute on Alcohol Abuse and Alcoholism. Assessment at Enrollment Research associates interviewed patients before randomization to assess the characteristics shown in Table 1. One author reviewed the medical records to determine medical diagnoses (29). Diagnoses of alcohol use disorders were based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (30), and were determined with the Composite International Diagnostic Interview (CIDI) Alcohol Module (31, 32). Table 1. Characteristics at Enrollment of All Study Patients and of the Subgroup with Alcohol Dependence* Randomization and Intervention An off-site data management group generated assignments to control and intervention groups by using a permuted block (size 8) randomization procedure stratified by AUDIT score (<12 vs. 12) and provided us the assignments in sealed opaque envelopes. We used the AUDIT score to stratify because we could not score the CIDI before randomization. After each baseline assessment, research associates opened an envelope and informed the patient of his or her assignment. Patients in the control group received usual care (that is, they were told the screening results and that they could discuss their drinking with their physicians). Specialists were available by referral. Systematic alcohol screening and brief intervention were not routine at this hospital. We assigned patients in the intervention group to a 30-minute session of brief motivational counseling (19, 33) conducted by counseling and clinical psychology doctoral students whom we trained and supervised. Sessions were audiotaped and included feedback, an open discussion, and construction of a change plan (Appendix). Outcomes and Measurements The first primary outcome was self-reported receipt of alcohol assistance in the past 3 months by patients with CIDI-determined alcohol dependence. This outcome was measured at the 3-month follow-up visit with a standardized interview based on the Treatment Services Review (34) and Form 90 (35). Assistance included residential treatment, outpatient treatment (for example, specialty counseling or therapy), medications, employee assistance programs, or mutual-help groups (for example, Alcoholics Anonymous). The other primary outcome was the change in the number of mean drinks per day in the past 30 days from enrollment to 12 months among all patients. We determined consumption with the Timeline Follow-back method (36). Five secondary consumption outcomes (past 30 days) included changes from enrollment to 12 months in the numbers of heavy drinking episodes (5 drinks/occasion for men and 4 drinks/occasion for women and for persons 66 y) and days abstinent; and the proportions of patients drinking risky amounts (>14 drinks/wk or 5 drinks/occasion for men and >7 drinks/wk or 4 drinks/occasion for women and persons 66 y) (37), having 1 or more heavy drinking episodes, and abstaining for all 30 days. Other secondary outcomes included the changes at 12 months in readiness to change (Taking Steps scale on the Stages of Change Readiness and Treatment Eagerness Scale) (38), alcohol problems (total score on the Short Inventory of Problems) (39), physical and mental health-related quality of life (Physical and Mental Component Summary scale scores on the Short-Form Health Survey) (40), and emergency department visits and days of medical hospitalization (both determined by a standardized interview based on the Treatment Services Review and Form 90) (34, 35). Follow-up Procedures Research associates conducted follow-up visits, which included reassessment of most domains covered at enrollment, usually in person and at 3 and 12 months (10% and 13%, respectively, by telephone; similar by randomized group). They performed alcohol breath tests at in-person follow-up visits (41). Although they were involved in the randomization assignment, research associates were not involved in the intervention. Further, 64% of patients at 3-month follow-up and 85% of patients at 12-month follow-up were interviewed by a different research associate than at baseline. Statistical Analysis We analyzed all patients in the groups to which they were randomly assigned. Reported P values are 2-tailed and are considered statistically significant if they were less than 0.05. We analyzed data with SAS/STAT software, versions 8.2 and 9.1.3 (SAS Institute, Inc., Cary, North Carolina). To describe the study sample and to compare groups, we used the chi-square test, Fisher exact test, 2-sample t test, and Wilcoxon rank-sum test, as appropriate. For the primary analyses, we used logistic and linear