Neil Klar

Pitfalls of and Controversies in Cluster Randomization Trials

It is now well known that standard statistical procedures become invalidated when applied to cluster randomized trials in which the unit of inference is the individual. A resulting consequence is that researchers conducting such trials are faced with a multitude of design choices, including selection of the primary unit of inference, the degree to which clusters should be matched or stratified by prognostic factors at baseline, and decisions related to cluster subsampling. Moreover, application of ethical principles developed for individually randomized trials may also require modification. We discuss several topics related to these issues, with emphasis on the choices that must be made in the planning stages of a trial and on some potential pitfalls to be avoided.

Statistical considerations in the design and analysis of community intervention trials

Community intervention trials are often characterized by the allocation of intact social units to different intervention groups. The assessment of adequate sample size for such trials must take into account the statistical dependencies among responses observed within an allocated unit. However, the small numbers of units typically involved in such trials imply that many methods of analysis that have been proposed for analyzing correlated data, particularly in the case of a dichotomous outcome variable, are not applicable to such designs. In this article we investigate this issue and determine the minimum number of units required per group, for the case of both a dichotomous and a continuous outcome variable, needed to provide adequate statistical power for detecting various levels of treatment effect. The use of significance testing as a method of detecting intracluster correlation is also investigated, and, in general, discouraged.

Quantifying the magnitude of risk for balance impairment on falls in community-dwelling older adults: a systematic review and meta-analysis

OBJECTIVES To evaluate and summarize the evidence linking balance impairment as a risk factor for falls in community-dwelling older adults. STUDY DESIGN AND SETTING Systematic review and meta-analysis. English language articles in MEDLINE, EMBASE, CINAHL (1988-2009), under keywords of accidental falls, aged, risk factors, and hip, radius, ulna, and humerus fractures; and bibliographies of retrieved articles. Community-dwelling older adults in a prospective study, at least 1-year duration, age more than 60 years, and samples not specific to a single disease-defined population were included. Sample size, inclusion/exclusion criteria, demographics, clinical balance measurement scale, type of fall outcome, method of fall ascertainment, length of follow-up, and odds ratio (OR) or risk ratio (RR) were extracted. Studies must have reported adjustment for confounders. Random effects meta-analysis to generate summary risk estimate was used. A priori evaluation of sources of heterogeneity was performed. RESULTS Twenty-three studies met the selection criteria. A single summary measure could not be calculated because of the nonequivalence of the OR and RR, producing an overall fall risk of RR of 1.42 (1.08, 1.85) and OR of 1.98 (1.60, 2.46). CONCLUSIONS Balance impairment imparts a moderate increase on fall risk in community-dwelling older adults. The type of fall outcome, the length of follow-up, and the balance measurement tool impact the magnitude of the association. Specific balance measurement scales were identified with associations for an increased fall risk, but further research is required to refine recommendations for their use in clinical practice.

Balance Impairment as a Risk Factor for Falls in Community-Dwelling Older Adults Who Are High Functioning: A Prospective Study

Background Screening should have simple and easy-to-administer methods that identify impairments associated with future fall risk, but there is a lack of literature supporting validation for their use. Objective The aim of this study was to evaluate the independent contribution of balance assessment on future fall risk, using 5 methods to quantify balance impairment, for the outcomes “any fall” and “any injurious fall” in community-dwelling older adults who are higher functioning. Design This was a prospective cohort study. Methods A sample of 210 community-dwelling older adults (70% male, 30% female; mean age=79.9 years, SD=4.7) received a comprehensive geriatric assessment at baseline, which included the Berg Balance Scale to measure balance. Information on daily falls was collected for 12 months by each participants monthly submission of a falls log calendar. Results Seventy-eight people (43%) fell, of whom 54 (30%) sustained an injurious fall and 32 (18%) had recurrent falls (≥2 falls). Different balance measurement methods identified different numbers of people as impaired. Adjusted relative risk (RR) estimates for an increased risk of any fall were 1.58 (95% confidence interval [CI]=1.06, 2.35) for self-report of balance problems, 1.58 (95% CI=1.03, 2.41) for one-leg stance, and 1.46 (95% CI=1.02, 2.09) for limits of stability. An adjusted RR estimate for an increased risk of an injurious fall of 1.95 (95% CI=1.15, 3.31) was found for self-report of balance problems. Limitations The study was a secondary analysis of data. Conclusions Not all methods of evaluating balance impairment are associated with falls. The number of people identified as having balance impairment varies with the measurement tool; therefore, the measurement tools are not interchangeable or equivalent in defining an at-risk population. The thresholds established in this study indicate individuals who should receive further comprehensive fall assessment and treatment to prevent falls.

The merits of matching in community intervention trials: a cautionary tale

Concern about potential imbalance on risk factors in community intervention trials often prompts researchers to adopt a pair-matched design in which similar clusters of individuals are paired and one member of each matched pair is then randomly assigned to the intervention group. It is known that if there are few clusters in trial, it becomes increasingly difficult to obtain close matches on all potential risk factors. One may thus offset any gain in precision with loss in degrees of freedom due to matching. We shown in this paper that there are also several analytic limitations with pair-matched designs. These include: the restriction of prediction models to cluster-level baseline risk factors (for example, cluster size), the inability to test for homogeneity of odds ratios, and difficulties in estimating the intracluster correlation coefficient. These limitations lead us to present arguments that favour stratified designs in which there are more than two clusters in each stratum.

Cluster randomization trials in epidemiology: theory and application

Abstract It is becoming increasingly common for epidemiologists to consider randomizing intact clusters (e.g. families, schools, communities) rather than individuals in experimental trials. Reasons are diverse, but include administrative convenience, a desire to reduce the effect of treatment contamination and the need to avoid ethical issues which might otherwise arise. Dependencies among cluster members typical of such designs must be considered when determining sample size and analyzing the resulting data. Well-known methods such as generalized least squares can be used to analyze continuous outcome data, while methods for the analysis of binary outcome data and correlated failure time data are in the development stage. The purpose of this paper is to review methods used in the design and analysis of cluster randomization trials applied in health sciences research.

Confidence interval construction for effect measures arising from cluster randomization trials

Methods of confidence interval construction are provided for summary measures of treatment effect arising from designs randomizing clusters to one of two treatment groups. Three basic designs are considered for the case of continuous and dichotomous variables: completely randomized, pair-matched and stratified.

The statistical analysis of kappa statistics in multiple samples

Methods are presented for assessing and comparing the results of k > or = 2 independent samples of measured agreement or concordance, where in each sample a given member of a pair of observations is classified according to the presence or absence of a binary trait. Examples include the assessment of interobserver agreement across different groups of patients in a clinical study, investigations of sibling concordance across different genetic groups, and meta-analyses of observer agreement across different studies. The methodology described is based on application of goodness-of-fit theory to testing hypotheses concerning kappa statistics. Partitioning methods allow a variety of hypotheses to be tested, including an assessment of the degree of agreement within each sample, a testing procedure based on the pooled data, and a test of heterogeneity that may be used to assess the validity of pooling across samples. Three examples are given.

Imputation Strategies for Missing Continuous Outcomes in Cluster Randomized Trials

In cluster randomized trials, intact social units such as schools, worksites or medical practices - rather than individuals themselves - are randomly allocated to intervention and control conditions, while the outcomes of interest are then observed on individuals within each cluster. Such trials are becoming increasingly common in the fields of health promotion and health services research. Attrition is a common occurrence in randomized trials, and a standard approach for dealing with the resulting missing values is imputation. We consider imputation strategies for missing continuous outcomes, focusing on trials with a completely randomized design in which fixed cohorts from each cluster are enrolled prior to random assignment. We compare five different imputation strategies with respect to Type I and Type II error rates of the adjusted two-sample t -test for the intervention effect. Cluster mean imputation is compared with multiple imputation, using either within-cluster data or data pooled across clusters in each intervention group. In the case of pooling across clusters, we distinguish between standard multiple imputation procedures which do not account for intracluster correlation and a specialized procedure which does account for intracluster correlation but is not yet available in standard statistical software packages. A simulation study is used to evaluate the influence of cluster size, number of clusters, degree of intracluster correlation, and variability among cluster follow-up rates. We show that cluster mean imputation yields valid inferences and given its simplicity, may be an attractive option in some large community intervention trials which are subject to individual-level attrition only; however, it may yield less powerful inferences than alternative procedures which pool across clusters especially when the cluster sizes are small and cluster follow-up rates are highly variable. When pooling across clusters, the imputation procedure should generally take intracluster correlation into account to obtain valid inferences; however, as long as the intracluster correlation coefficient is small, we show that standard multiple imputation procedures may yield acceptable type I error rates; moreover, these procedures may yield more powerful inferences than a specialized procedure, especially when the number of available clusters is small. Within-cluster multiple imputation is shown to be the least powerful among the procedures considered.

TESTING THE HOMOGENEITY OF KAPPA STATISTICS

Procedures are compared for testing the homogeneity of k > or = 2 independent kappa statistics in the case of two raters and a dichotomous outcome. One of the procedures is based on the estimated large sample variance derived under a model frequently adopted for inferences concerning interobserver agreement. The other is based on a goodness-of-fit approach to this model. The results of a Monte Carlo simulation show that the two approaches have similar properties if the number of subjects in each sample is large (> 100), and the prevalence of the underlying trait of interest is not extreme, while the goodness-of-fit approach is recommended for comparisons involving smaller numbers of subjects or in which the prevalence of the underlying trait is small (< 0.3).