Michael Borenstein

Publication bias in meta-analysis : prevention, assessment and adjustments

Preface. Acknowledgements. Notes on Contributors. Chapter 1: Publication Bias in Meta-Analysis (Hannah R. Rothstein, Alexander J. Sutton and Michael Borenstein). Part A: Publication bias in context. Chapter 2: Publication Bias: Recognizing the Problem, Understanding Its Origins and Scope, and Preventing Harm (Kay Dickersin). Chapter 3: Preventing Publication Bias: Registries and Prospective Meta-Analysis (Jesse A. Berlin and Davina Ghersi). Chapter 4: Grey Literature and Systematic Reviews (Sally Hopewell, Mike Clarke and Sue Mallett). Part B: Statistical methods for assessing publication bias. Chapter 5: The Funnel Plot (Jonathan A.C. Sterne, Betsy Jane Becker and Matthias Egger). Chapter 6: Regression Methods to Detect Publication and Other Bias in Meta-Analysis (Jonathan A.C. Sterne and Matthias Egger). Chapter 7: Failsafe N or File-Drawer Number (Betsy Jane Becker). Chapter 8: The Trim and Fill Method (Sue Duval). Chapter 9: Selection Method Approaches (Larry V. Hedges and Jack Vevea). Chapter 10: Evidence Concerning the Consequences of Publication and Related Biases (Alexander J. Sutton). Chapter 11: Software for Publication Bias (Michael Borenstein). Part C: Advanced and emerging approaches. Chapter 12: Bias in Meta-Analysis Induced by Incompletely Reported Studies (Alexander J. Sutton and Therese D. Pigott). Chapter 13: Assessing the Evolution of Effect Sizes over Time (Thomas A. Trikalinos and John P.A. Ioannidis). Chapter 14: Do Systematic Reviews Based on Individual Patient Data Offer a Means of Circumventing Biases Associated with Trial Publications? (Lesley Stewart, Jayne Tierney and Sarah Burdett). Chapter 15: Differentiating Biases from Genuine Heterogeneity: Distinguishing Artifactual from Substantive Effects (John P.A. Ioannidis). Chapter 16: Beyond Conventional Publication Bias: Other Determinants of Data Suppression (Scott D. Halpern and Jesse A. Berlin). Appendices. Appendix A: Data Sets. Appendix B: Annotated Bibliography (Hannah R. Rothstein and Ashley Busing). Glossary. Index.

A basic introduction to fixed‐effect and random‐effects models for meta‐analysis

There are two popular statistical models for meta-analysis, the fixed-effect model and the random-effects model. The fact that these two models employ similar sets of formulas to compute statistics, and sometimes yield similar estimates for the various parameters, may lead people to believe that the models are interchangeable. In fact, though, the models represent fundamentally different assumptions about the data. The selection of the appropriate model is important to ensure that the various statistics are estimated correctly. Additionally, and more fundamentally, the model serves to place the analysis in context. It provides a framework for the goals of the analysis as well as for the interpretation of the statistics. In this paper we explain the key assumptions of each model, and then outline the differences between the models. We conclude with a discussion of factors to consider when choosing between the two models. Copyright

Ingestion of yogurt containing Lactobacillus acidophilus as prophylaxis for candidal vaginitis.

OBJECTIVE To assess whether daily ingestion of yogurt containing Lactobacillus acidophilus prevents vulvovaginal candidal infections. DESIGN Crossover trial for at least 1 year during which patients were examined for candidal infections and colonizations while receiving either a yogurt-free or a yogurt-containing diet. Patients served as their own controls. SETTING Ambulatory infectious disease center in a teaching hospital providing tertiary care. PATIENTS Thirty-three women with recurrent candidal vaginitis were eligible after recruitment from community practices and clinics and through advertising. Twelve patients were eliminated for protocol violations. Of the remaining 21 patients, 8 who were assigned to the yogurt arm initially refused to enter the control phase 6 months later. Thus, 13 patients completed the protocol. INTERVENTIONS Women ate yogurt for 6 months of the study period. MEASUREMENTS Colonization of lactobacilli and candida in the vagina and rectum; candidal infections of the vagina. MAIN RESULTS Thirty-three eligible patients were studied. A threefold decrease in infections was seen when patients consumed yogurt containing Lactobacillus acidophilus. The mean (+/- SD) number of infections per 6 months was 2.54 +/- 1.66 in the control arm and 0.38 +/- 0.51 per 6 months in the yogurt arm (P = 0.001). Candidal colonization decreased from a mean of 3.23 +/- 2.17 per 6 months in the control arm to 0.84 +/- 0.90 per 6 months in the yogurt arm (P = 0.001). CONCLUSION Daily ingestion of 8 ounces of yogurt containing Lactobacillus acidophilus decreased both candidal colonization and infection.

Long-Acting Injectable vs Oral Antipsychotics for Relapse Prevention in Schizophrenia: A Meta-Analysis of Randomized Trials

BACKGROUND While long-acting injectable antipsychotics (LAIs) are hoped to reduce high relapse rates in schizophrenia, recent randomized controlled trials (RCTs) challenged the benefits of LAIs over oral antipsychotics (OAPs). METHODS Systematic review/meta-analysis of RCTs that lasted ≥ 6 months comparing LAIs and OAPs. Primary outcome was study-defined relapse at the longest time point; secondary outcomes included relapse at 3, 6, 12, 18, and 24 months, all-cause discontinuation, discontinuation due to adverse events, drug inefficacy (ie, relapse + discontinuation due to inefficacy), hospitalization, and nonadherence. RESULTS Across 21 RCTs (n = 5176), LAIs were similar to OAPs for relapse prevention at the longest time point (studies = 21, n = 4950, relative risk [RR] = 0.93, 95% confidence interval [CI]: 0.80-1.08, P = .35). The finding was confirmed restricting the analysis to outpatient studies lasting ≥ 1 year (studies = 12, RR = 0.93, 95% CI:0.71-1.07, P = .31). However, studies using first-generation antipsychotic (FGA)-LAIs (studies = 10, RR = 0.82, 95% CI:0.69-0.97, P = .02) and those published ≤ 1991 (consisting exclusively of all 8 fluphenazine-LAI studies; RR = 0.79, 95% CI: 0.65-0.96, P = 0.02) were superior to OAPs regarding the primary outcome. Pooled LAIs also did not separate from OAPs regarding any secondary outcomes. Again, studies using FGA-LAIs and those published ≤ 1991 were associated with LAI superiority over OAPs, eg, hospitalization and drug inefficacy. CONCLUSIONS In RCTs, which are less representative of real-world patients than naturalistic studies, pooled LAIs did not reduce relapse compared with OAPs in schizophrenia patients. The exceptions were FGA-LAIs, mostly consisting of fluphenazine-LAI studies, which were all conducted through 1991. Because this finding is vulnerable to a cohort bias, studies comparing FGA-LAI vs second-generation antipsychotics-LAI and LAI vs OAP RCTs in real-world patients are needed.

Long-acting injectable versus oral antipsychotics in schizophrenia: a systematic review and meta-analysis of mirror-image studies.

OBJECTIVE Recent, large, randomized controlled trials (RCTs) showed no benefit of long-acting injectable (LAI) antipsychotics over oral antipsychotics in preventing relapse in schizophrenia, nor did a recent meta-analysis incorporating these studies. However, RCTs might enroll a disproportionate number of patients with better treatment adherence and lower illness severity. Mirror-image studies, which compare periods of oral antipsychotic versus LAI treatment in the same patients, might therefore better reflect the real-world impact of LAIs. DATA SOURCES A systematic literature search without language restriction was conducted using MEDLINE/PubMed, Cochrane Library, Web of Science, PsycINFO, and CINAHL until May 31, 2012. Search terms included synonyms of (1) antipsychotic(s) AND (2) schizophrenia and related disorders AND (3) depot, (long-acting) injection(s), microsphere, decanoate, palmitate, enanthate. STUDY SELECTION Of 5,483 identified citations, 607 articles were fully inspected, and 582 were ineligible. Finally, 25 mirror-image studies from 28 countries that followed 5,940 patients with schizophrenia for ≥ 12 months (≥ 6 months each on oral antipsychotic and LAI treatment) met the inclusion criteria and were analyzed. DATA EXTRACTION Coprimary outcomes were hospitalization risk and number of hospitalizations. Secondary outcomes included hospitalization days and length of stay. DATA SYNTHESIS LAIs showed strong superiority over oral antipsychotics in preventing hospitalization (16 studies, N = 4,066; risk ratio = 0.43; 95% CI, 0.35-0.53; P < .001) and in decreasing the number of hospitalizations (15 studies, 6,342 person-years; rate ratio = 0.38; 95% CI, 0.28-0.51; P < .001). This strong advantage was also observed for secondary outcomes and in multiple clinically relevant subpopulations and treatment groups. CONCLUSIONS Results from mirror-image studies in patients eligible for clinical use of LAIs showed strong superiority of LAIs compared to oral antipsychotics in preventing hospitalization. The results were in contrast to the recent meta-analysis of RCTs, which showed no superiority of LAIs. Given the possible biases in mirror-image studies, such as expectation bias, natural illness course, and time effect, a cautious interpretation is required. Nevertheless, the population in mirror-image studies better reflects the population receiving LAIs in clinical practice.

Basics of meta‐analysis: I2 is not an absolute measure of heterogeneity

When we speak about heterogeneity in a meta-analysis, our intent is usually to understand the substantive implications of the heterogeneity. If an intervention yields a mean effect size of 50 points, we want to know if the effect size in different populations varies from 40 to 60, or from 10 to 90, because this speaks to the potential utility of the intervention. While there is a common belief that the I2 statistic provides this information, it actually does not. In this example, if we are told that I2 is 50%, we have no way of knowing if the effects range from 40 to 60, or from 10 to 90, or across some other range. Rather, if we want to communicate the predicted range of effects, then we should simply report this range. This gives readers the information they think is being captured by I2 and does so in a way that is concise and unambiguous. Copyright

Meta-analysis and subgroups.

Subgroup analysis is the process of comparing a treatment effect for two or more variants of an intervention—to ask, for example, if an intervention’s impact is affected by the setting (school versus community), by the delivery agent (outside facilitator versus regular classroom teacher), by the quality of delivery, or if the long-term effect differs from the short-term effect. While large-scale studies often employ subgroup analyses, these analyses cannot generally be performed for small-scale studies, since these typically include a homogeneous population and only one variant of the intervention. This limitation can be bypassed by using meta-analysis. Meta-analysis allows the researcher to compare the treatment effect in different subgroups, even if these subgroups appear in separate studies. We discuss several statistical issues related to this procedure, including the selection of a statistical model and statistical power for the comparison. To illustrate these points, we use the example of a meta-analysis of obesity prevention.

Computerized volume measurement of brain structure.

Morphometric analysis of brain structures recently has become a main focus of interest in studies of some neuropsychiatric diseases. Limitations in imaging and mensuration methodology that is available currently for quantitative measurement of anatomic structures have prompted the development of a computerized system to study brain morphometry. A menudriven semi-automated computer system has been developed to assess in vivo brain morphometry using three-dimensional (3-D) magnetic resonance (MR), gradient echo, contiguous images of the whole brain. Accuracy of the system was tested with phantoms creating white on black contrast to simulate the brain tissue surrounded by subarachnoid cerebrospinal fluid (CSF), and a second set of phantoms creating black on white contrast to simulate the ventricular system in the brain tissue. The first set of phantoms was composed of three water-filled balloons (spherical, elliptical, and multiform) and a fresh postmortem brain. The second set of phantoms consisted of three rods of different diameters from a simple geometric plexiglass rod phantom and a life size cast of a human ventricular phantom. System accuracy was generally within 2.0% of the true volumes. System reliability was evaluated in three patient populations; 12 patients with Alzheimers disease, nine with schizophrenia and nine healthy controls age-matched to the patients with Alzheimers disease. Two independent observers measured the ventricular systems of these patients. Reliability of the system was addressed by the correlation between the two sets of measurements. For the sample as a whole, and each of the subgroups, the correlation between the two observers was 0.99. This system compares favorably with other morphometric methods reported.

Tricyclic antidepressant and metabolite levels in chronic renal failure.

Serial blood samples were drawn from 12 patients undergoing hemodialysis who were receiving tricyclic antidepressants (TCAs). Samples were drawn before, during, and after a dialysis session (two to 17 sessions per subject). Samples were analyzed by HPLC before and after hydrolysis with β‐glucuronidase/sulfatase to determine the conjugated and nonconjugated metabolites. Analysis of these data in comparison with those of controls with depression and normal renal function showed that: (1) at steady state, tertiary and secondary amine TCA levels did not differ; (2) levels of the hydroxylated metabolites had greater variability and were somewhat higher at steady state; (3) levels of the conjugated hydroxylated compounds were markedly elevated, reaching 500% to 1500% normal; (4) the time to reach a steady‐state level appeared to be slightly increased; and (5) elimination t½s of unconjugated and conjugated drug forms were longer in our patients with normal renal function than those reported in the literature. Levels of the tertiary, secondary, and hydroxylated metabolites were not changed by dialysis, whereas there were substantial decrements in glucuronidated metabolite levels. These findings demonstrate increased concentrations of conjugated drug forms and suggest an abnormal distribution or delayed elimination of unconjugated and conjugated metabolites. These observations may shed some light on the apparent hypersensitivity of these patients to TCA side effects, particularly because glucuronides may exert peripheral pharmacologic effects.

Hypothesis Testing and Effect Size Estimation in Clinical Trials

LEARNING OBJECTIVES This paper provides the reader with an overview of several key elements in study planning and analysis. In particular, it highlights the differences between significance tests (statistical significance) and effect size estimation (clinical significance). DATA SOURCES This paper focuses on methodologic issues, and provides an overview of trends in research. PAPER SELECTION: References were selected to provide a cross-section of the approaches currently being used. The paper also discusses a number of logical fallacies that have been cited as examples in earlier papers on research design. CONCLUSIONS Significance tests are intended solely to address the viability of the null hypothesis that a treatment has no effect, and not to estimate the magnitude of the treatment effect. Researchers are advised to move away from significance tests and to present instead an estimate of effect size bounded by confidence intervals. This approach incorporates all the information normally included in a test of significance but in a format that highlights the element of interest (clinical significance rather than statistical significance). This approach should also have an impact on study planning--a study should have enough power to reject the null hypothesis and also to yield a precise estimate of the treatment effect.