Rei Monden

Toward evidence-based medical statistics

The Food and Drug Administration (FDA) uses a p < 0.05 null‐hypothesis significance testing framework to evaluate “substantial evidence” for drug efficacy. This framework only allows dichotomous conclusions and does not quantify the strength of evidence supporting efficacy. The efficacy of FDA‐approved antidepressants for the treatment of anxiety disorders was re‐evaluated in a Bayesian framework that quantifies the strength of the evidence. Data from 58 double‐blind placebo‐controlled trials were retrieved from the FDA for the second‐generation antidepressants for the treatment of anxiety disorders. Bayes factors (BFs) were calculated for all treatment arms compared to placebo and were compared with the corresponding p‐values and the FDA conclusion categories. BFs ranged from 0.07 to 131,400, indicating a range of no support of evidence to strong evidence for the efficacy. Results also indicate a varying strength of evidence between the trials with p < 0.05. In sum, there were large differences in BFs across trials. Among trials providing “substantial evidence” according to the FDA, only 27 out of 59 dose groups obtained strong support for efficacy according to the typically used cutoff of BF ≥ 20. The Bayesian framework can provide valuable information on the strength of the evidence for drug efficacy. Copyright

Toward evidence-based medical statistics: a Bayesian analysis of double-blind placebo-controlled antidepressant trials in the treatment of anxiety disorders

The Food and Drug Administration (FDA) uses a p < 0.05 null‐hypothesis significance testing framework to evaluate “substantial evidence” for drug efficacy. This framework only allows dichotomous conclusions and does not quantify the strength of evidence supporting efficacy. The efficacy of FDA‐approved antidepressants for the treatment of anxiety disorders was re‐evaluated in a Bayesian framework that quantifies the strength of the evidence. Data from 58 double‐blind placebo‐controlled trials were retrieved from the FDA for the second‐generation antidepressants for the treatment of anxiety disorders. Bayes factors (BFs) were calculated for all treatment arms compared to placebo and were compared with the corresponding p‐values and the FDA conclusion categories. BFs ranged from 0.07 to 131,400, indicating a range of no support of evidence to strong evidence for the efficacy. Results also indicate a varying strength of evidence between the trials with p < 0.05. In sum, there were large differences in BFs across trials. Among trials providing “substantial evidence” according to the FDA, only 27 out of 59 dose groups obtained strong support for efficacy according to the typically used cutoff of BF ≥ 20. The Bayesian framework can provide valuable information on the strength of the evidence for drug efficacy. Copyright

Simultaneous Decomposition of Depression Heterogeneity on the Person-, Symptom- and Time-Level : The Use of Three-Mode Principal Component Analysis

Although heterogeneity of depression hinders research and clinical practice, attempts to reduce it with latent variable models have yielded inconsistent results, probably because these techniques cannot account for all interacting sources of heterogeneity at the same time. Therefore, to simultaneously decompose depression heterogeneity on the person-, symptom and time-level, three-mode Principal Component Analysis (3MPCA) was applied to data of 219 Major Depression patients, who provided Beck Depression Inventory assessments every three months for two years. The resulting person-level components were correlated with external baseline clinical and demographic variables. The 3MPCA extracted two symptom-level components (‘cognitive’, ‘somatic-affective’), two time-level components (‘improving’, ‘persisting’) and three person-level components, characterized by different interaction-patterns between the symptom- and time-components (‘severe non-persisting’, ‘somatic depression’ and ‘cognitive depression’). This model explained 28% of the total variance and 65% when also incorporating the general trend in the data). Correlations with external variables illustrated the content differentiation between the person-components. Severe non-persisting depression was positively correlated with psychopathology (r=0.60) and negatively with quality of life (r=-0.50). Somatic depression was negatively correlated with physical functioning (r=-0.45). Cognitive depression was positively correlated with neuroticism (r=0.38) and negatively with self-esteem (r=-0.47). In conclusion, 3MPCA decomposes depression into homogeneous entities, while accounting for the interactions between different sources of heterogeneity, which shows the utility of the technique to investigate the underlying structure of complex psychopathology data and could help future development of better empirical depression subtypes.

Bayesian reanalysis of null results reported in medicine: Strong yet variable evidence for the absence of treatment effects

Efficient medical progress requires that we know when a treatment effect is absent. We considered all 207 Original Articles published in the 2015 volume of the New England Journal of Medicine and found that 45 (21.7%) reported a null result for at least one of the primary outcome measures. Unfortunately, standard statistical analyses are unable to quantify the degree to which these null results actually support the null hypothesis. Such quantification is possible, however, by conducting a Bayesian hypothesis test. Here we reanalyzed a subset of 43 null results from 36 articles using a default Bayesian test for contingency tables. This Bayesian reanalysis revealed that, on average, the reported null results provided strong evidence for the absence of an effect. However, the degree of this evidence is variable and cannot be reliably predicted from the p-value. For null results, sample size is a better (albeit imperfect) predictor for the strength of evidence in favor of the null hypothesis. Together, our findings suggest that (a) the reported null results generally correspond to strong evidence in favor of the null hypothesis; (b) a Bayesian hypothesis test can provide additional information to assist the interpretation of null results.

The comparative evidence basis for the efficacy of second-generation antidepressants in the treatment of depression in the US: A Bayesian Meta-analysis of Food and Drug Administration Reviews

BACKGROUND Studies have shown similar efficacy of different antidepressants in the treatment of depression. METHOD Data of phase-2 and -3 clinical-trials for 16 antidepressants (levomilnacipran, desvenlafaxine, duloxetine, venlafaxine, paroxetine, escitalopram, vortioxetine, mirtazapine, venlafaxine XR, sertraline, fluoxetine, citalopram, paroxetine CR, nefazodone, bupropion, vilazodone), approved by the FDA for the treatment of depression between 1987 and 2016, were extracted from the FDA reviews that were used to evaluate efficacy prior to marketing approval, which are less liable to reporting biases. Meta-analytic Bayes factors, which quantify the strength of evidence for efficacy, were calculated. In addition, posterior pooled effect-sizes were calculated and compared with classical estimations. RESULTS The resulted Bayes factors showed that the evidence load for efficacy varied strongly across antidepressants. However, all tested drugs except for bupropion and vilazodone showed strong evidence for their efficacy. The posterior effect-size distributions showed variation across antidepressants, with the highest pooled estimated effect size for venlafaxine followed by paroxetine, and the lowest for bupropion and vilazodone. LIMITATIONS Not all published trials were included in the study. CONCLUSIONS The results illustrate the importance of considering both the effect size and the evidence-load when judging the efficacy of a treatment. In doing so, the currently employed Bayesian approach provided clear insights on top of those gained with traditional approaches.