Theis Lange

Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods

BackgroundThresholds for statistical significance when assessing meta-analysis results are being insufficiently demonstrated by traditional 95% confidence intervals and P-values. Assessment of intervention effects in systematic reviews with meta-analysis deserves greater rigour.MethodsMethodologies for assessing statistical and clinical significance of intervention effects in systematic reviews were considered. Balancing simplicity and comprehensiveness, an operational procedure was developed, based mainly on The Cochrane Collaboration methodology and the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) guidelines.ResultsWe propose an eight-step procedure for better validation of meta-analytic results in systematic reviews (1) Obtain the 95% confidence intervals and the P-values from both fixed-effect and random-effects meta-analyses and report the most conservative results as the main results. (2) Explore the reasons behind substantial statistical heterogeneity using subgroup and sensitivity analyses (see step 6). (3) To take account of problems with multiplicity adjust the thresholds for significance according to the number of primary outcomes. (4) Calculate required information sizes (≈ the a priori required number of participants for a meta-analysis to be conclusive) for all outcomes and analyse each outcome with trial sequential analysis. Report whether the trial sequential monitoring boundaries for benefit, harm, or futility are crossed. (5) Calculate Bayes factors for all primary outcomes. (6) Use subgroup analyses and sensitivity analyses to assess the potential impact of bias on the review results. (7) Assess the risk of publication bias. (8) Assess the clinical significance of the statistically significant review results.ConclusionsIf followed, the proposed eight-step procedure will increase the validity of assessments of intervention effects in systematic reviews of randomised clinical trials.

Direct and indirect effects in a survival context.

A cornerstone of epidemiologic research is to understand the causal pathways from an exposure to an outcome. Mediation analysis based on counterfactuals is an important tool when addressing such questions. However, none of the existing techniques for formal mediation analysis can be applied to survival data. This is a severe shortcoming, as many epidemiologic questions can be addressed only with censored survival data. A solution has been to use a number of Cox models (with and without the potential mediator), but this approach does not allow a causal interpretation and is not mathematically consistent. In this paper, we propose a simple measure of mediation in a survival setting. The measure is based on counterfactuals, and measures the natural direct and indirect effects. The method allows a causal interpretation of the mediated effect (in terms of additional cases per unit of time) and is mathematically consistent. The technique is illustrated by analyzing socioeconomic status, work environment, and long-term sickness absence. A detailed implementation guide is included in an online eAppendix (http://links.lww.com/EDE/A476).

A Simple Unified Approach for Estimating Natural Direct and Indirect Effects

An important problem within both epidemiology and many social sciences is to break down the effect of a given treatment into different causal pathways and to quantify the importance of each pathway. Formal mediation analysis based on counterfactuals is a key tool when addressing this problem. During the last decade, the theoretical framework for mediation analysis has been greatly extended to enable the use of arbitrary statistical models for outcome and mediator. However, the researcher attempting to use these techniques in practice will often find implementation a daunting task, as it tends to require special statistical programming. In this paper, the authors introduce a simple procedure based on marginal structural models that directly parameterize the natural direct and indirect effects of interest. It tends to produce more parsimonious results than current techniques, greatly simplifies testing for the presence of a direct or an indirect effect, and has the advantage that it can be conducted in standard software. However, its simplicity comes at the price of relying on correct specification of models for the distribution of mediator (and exposure) and accepting some loss of precision compared with more complex methods. Web Appendixes 1 and 2, which are posted on the Journals Web site (http://aje.oupjournals.org/), contain implementation examples in SAS software (SAS Institute, Inc., Cary, North Carolina) and R language (R Foundation for Statistical Computing, Vienna, Austria).

The thresholds for statistical and clinical significance – a five-step procedure for evaluation of intervention effects in randomised clinical trials

BackgroundThresholds for statistical significance are insufficiently demonstrated by 95% confidence intervals or P-values when assessing results from randomised clinical trials. First, a P-value only shows the probability of getting a result assuming that the null hypothesis is true and does not reflect the probability of getting a result assuming an alternative hypothesis to the null hypothesis is true. Second, a confidence interval or a P-value showing significance may be caused by multiplicity. Third, statistical significance does not necessarily result in clinical significance. Therefore, assessment of intervention effects in randomised clinical trials deserves more rigour in order to become more valid.MethodsSeveral methodologies for assessing the statistical and clinical significance of intervention effects in randomised clinical trials were considered. Balancing simplicity and comprehensiveness, a simple five-step procedure was developed.ResultsFor a more valid assessment of results from a randomised clinical trial we propose the following five-steps: (1) report the confidence intervals and the exact P-values; (2) report Bayes factor for the primary outcome, being the ratio of the probability that a given trial result is compatible with a ‘null’ effect (corresponding to the P-value) divided by the probability that the trial result is compatible with the intervention effect hypothesised in the sample size calculation; (3) adjust the confidence intervals and the statistical significance threshold if the trial is stopped early or if interim analyses have been conducted; (4) adjust the confidence intervals and the P-values for multiplicity due to number of outcome comparisons; and (5) assess clinical significance of the trial results.ConclusionsIf the proposed five-step procedure is followed, this may increase the validity of assessments of intervention effects in randomised clinical trials.

The joint effect of sleep duration and disturbed sleep on cause-specific mortality: Results from the Whitehall II cohort study

Background Both sleep duration and sleep quality are related to future health, but their combined effects on mortality are unsettled. We aimed to examine the individual and joint effects of sleep duration and sleep disturbances on cause-specific mortality in a large prospective cohort study. Methods We included 9,098 men and women free of pre-existing disease from the Whitehall II study, UK. Sleep measures were self-reported at baseline (1985–1988). Participants were followed until 2010 in a nationwide death register for total and cause-specific (cardiovascular disease, cancer and other) mortality. Results There were 804 deaths over a mean 22 year follow-up period. In men, short sleep (≤6 hrs/night) and disturbed sleep were not independently associated with CVD mortality, but there was an indication of higher risk among men who experienced both (HR = 1.57; 95% CI: 0.96–2.58). In women, short sleep and disturbed sleep were independently associated with CVD mortality, and women with both short and disturbed sleep experienced a much higher risk of CVD mortality (3.19; 1.52–6.72) compared to those who slept 7–8 hours with no sleep disturbances; equivalent to approximately 90 additional deaths per 100,000 person years. Sleep was not associated with death due to cancer or other causes. Conclusion Both short sleep and disturbed sleep are independent risk factors for CVD mortality in women and future studies on sleep may benefit from assessing disturbed sleep in addition to sleep duration in order to capture health-relevant features of inadequate sleep.

Assessing Natural Direct and Indirect Effects Through Multiple Pathways

Within the fields of epidemiology, interventions research and social sciences researchers are often faced with the challenge of decomposing the effect of an exposure into different causal pathways working through defined mediator variables. The goal of such analyses is often to understand the mechanisms of the system or to suggest possible interventions. The case of a single mediator, thus implying only 2 causal pathways (direct and indirect) from exposure to outcome, has been extensively studied. By using the framework of counterfactual variables, researchers have established theoretical properties and developed powerful tools. However, in practical problems, it is not uncommon to have several distinct causal pathways from exposure to outcome operating through different mediators. In this article, we suggest a widely applicable approach to quantifying and ranking different causal pathways. The approach is an extension of the natural effect models proposed by Lange et al. (Am J Epidemiol. 2012;176(3):190-195). By allowing the analysis of distinct multiple pathways, the suggested approach adds to the capabilities of modern mediation techniques. Furthermore, the approach can be implemented using standard software, and we have included with this article implementation examples using R (R Foundation for Statistical Computing, Vienna, Austria) and Stata software (StataCorp LP, College Station, Texas).

Education and cause-specific mortality: the mediating role of differential exposure and vulnerability to behavioral risk factors.

Background: Differential exposures to behavioral risk factors have been shown to play an important mediating role on the education–mortality relation. However, little is known about the extent to which educational attainment interacts with health behavior, possibly through differential vulnerability. Methods: In a cohort study of 76,294 participants 30 to 70 years of age, we estimated educational differences in cause-specific mortality from 1980 through 2009 and the mediating role of behavioral risk factors (smoking, alcohol intake, physical activity, and body mass index). With the use of marginal structural models and three-way effect decomposition, we simultaneously regarded the behavioral risk factors as intermediates and clarified the role of their interaction with educational exposure. Results: Rate differences in mortality comparing participants with low to high education were 1,277 (95% confidence interval = 1,062 to 1,492) per 100,000 person-years for men and 746 (598 to 894) per 100,000 person-years for women. Smoking was the strongest mediator for cardiovascular disease, cancer, and respiratory disease mortality when conditioning on sex, age, and cohort. The proportion mediated through smoking was most pronounced in cancer mortality as a combination of the pure indirect effect, owing to differential exposure (men, 42% [25% to 75%]; women, 36% [17% to 74%]) and the mediated interactive effect, owing to differential vulnerability (men, 18% [2% to 35%], women, 26% [8% to 50%]). The mediating effects through body mass index, alcohol intake, or physical activity were partial and varied for the causes of deaths. Conclusion: Differential exposure and vulnerability should be addressed simultaneously, as these mechanisms are not mutually exclusive and may operate at the same time.

Additive interaction in survival analysis: use of the additive hazards model.

It is a widely held belief in public health and clinical decision-making that interventions or preventive strategies should be aimed at patients or population subgroups where most cases could potentially be prevented. To identify such subgroups, deviation from additivity of absolute effects is the relevant measure of interest. Multiplicative survival models, such as the Cox proportional hazards model, are often used to estimate the association between exposure and risk of disease in prospective studies. In Cox models, deviations from additivity have usually been assessed by surrogate measures of additive interaction derived from multiplicative models—an approach that is both counter-intuitive and sometimes invalid. This paper presents a straightforward and intuitive way of assessing deviation from additivity of effects in survival analysis by use of the additive hazards model. The model directly estimates the absolute size of the deviation from additivity and provides confidence intervals. In addition, the model can accommodate both continuous and categorical exposures and models both exposures and potential confounders on the same underlying scale. To illustrate the approach, we present an empirical example of interaction between education and smoking on risk of lung cancer. We argue that deviations from additivity of effects are important for public health interventions and clinical decision-making, and such estimations should be encouraged in prospective studies on health. A detailed implementation guide of the additive hazards model is provided in the appendix.

World Health Organization‐defined classification of myeloproliferative neoplasms: Morphological reproducibility and clinical correlations—The Danish experience

We examined inter‐ and intraobserver reproducibility and concordance between histological diagnosis and independently collected clinical findings in a large series of patients with the major subtypes of myeloproliferative neoplasms (MPNs) and controls. Seven hematopathologists reviewed 272 bone marrow biopsies including 43 controls. Diagnoses were determined according to the 2008 criteria of the World Health Organization (WHO). The participants were blinded to all clinical data except patient age. After initial evaluation all hematopathologists participated in a 3‐day meeting with a leading clinician chaired by an expert hematopathologists. In cases with lack of consensus on fiber grading (n = 57), a new evaluation was performed. In cases with discordance on morphological diagnosis (n = 129), an additional nonblinded evaluation taking clinical data into consideration was carried out. For remaining cases with a lack of concordance between morphological diagnosis and clinical diagnosis (n = 33), a similar nonblinded evaluation was performed. Consensus on final histological diagnosis and concordance with clinical diagnosis were determined. Blinded histological evaluation resulted in a 53% consensus rate. After re‐evaluation of fiber content, consensus was reached in 60% of cases. Adding clinical data increased the histological consensus to 83%. For cases with a histological consensus, we found a concordance of 71% with the clinicians diagnoses. This is the first study to present a larger cohort of MPN patients mimicking the diagnostic challenges that hematopathologists face in their daily practice. The results support the postulates of the WHO that both morphological and clinical findings are essential for a valid diagnosis Am. J. Hematol. 88:1012–1016, 2013.

Imputation Strategies for the Estimation of Natural Direct and Indirect Effects

Abstract Mediation analysis is widely adopted to infer causal mechanism by disentangling indirect or mediated effects of an exposure on an outcome through given intermediaries, from the remaining direct effect. Traditional approaches build on standard regression models for the outcome and mediator, but easily result in difficult-to-interpret or difficult-to-report results when some of these models involve non-linearities. In this article, we overcome this via a general class of so-called natural effect models, which directly parameterize the (natural) direct and indirect effects of interest. We propose flexible estimation strategies for the direct and indirect effect parameters indexing these models, that are easy to perform with standard statistical software: one based on regression mean imputation and one based on doubly robust imputation. We give a theoretical discussion of the properties of these estimation strategies. We moreover assess their finite-sample performance through a simulation study, and through the analysis of the WHO-LARES study on the association between residence in a damp and moldy dwelling and the risk of depression.