Low statistical power in a study predicting seizure outcome

Abstract

To the Editors We read with great interest the recent article by Kang et al titled Magnetic resonanceguided laser interstitial thermal therapy: Correlations with seizure outcome. They report the important finding that the presence of bilateral interictal epileptiform discharges (IEDs) was associated with a lower odds of seizure freedom (odds ratio 0.05, 95% confidence interval 0.01– 0.46) following magnetic resonanceguided laser interstitial thermal therapy (MRgLiTT) in patients with unilateral mesial temporal sclerosis (MTS). An odds ratio of 0.05 for bilateral IEDs (or 20 for unilateral IEDs) is a large effect size. However, a trial of 56 patients is underpowered to examine the differences in surgical outcome, based on previously reported effect sizes. We are concerned that the true effect size may be overestimated,1 which could impact counseling of patients. We reviewed the prior studies cited by Kang and colleagues to determine, conservatively, how many subjects would be needed to have a reasonable power to detect their reported effect size. The authors cite four study reports that examined the association of unilateral IEDs and seizure freedom after anterior temporal lobectomy (ATL).2– 5 The odds ratio for seizure freedom with unilateral IEDs varied from 2.1 to 3.6. Pooling data across the four studies gives an estimate of 80% (292/367) seizurefree after ATL with unilateral IEDs, vs 61% (93/152) seizurefree after ATL with bilateral IEDs (Supplementary Material). This corresponds to an odds ratio of 2.5 for unilateral IEDs, or 0.4 for bilateral IEDs. Based on those published results after ATL, achieving 80% power to detect a difference in seizure freedom of 80% vs 61% would require at least 174 patients, using a normal approximation and assuming equal group sizes.6 We estimate that the present study of Kang and colleagues has around 20% power to detect a difference in seizure freedom of 80% vs 61%, based on simulation (100,000 repetitions, Fisher s exact test). Underpowered studies necessarily overestimate the effect size of significant results, since the true effect size will not be significant.1 The issue of low statistical power is not unique to this article but rather is a common problem in epilepsy where studies often have a small number of subjects. We agree with the authors that bilateral IEDs, based on available evidence, predicts a lower chance of seizure freedom. The seizurefree rate for patients with unilateral IEDs was 72% (33/46). The prehoc odds ratio of 0.4 for bilateral IEDs, albeit derived from ATL studies, decreases the chance of seizure freedom from 72% to 50%. However, the reported odds ratio of 0.05 decreases the chance of seizure freedom from 72% to 11%. Those numbers have a very different significance for counseling patients considering surgery. Depending on other data, the presence of bilateral typical medial temporal IEDs in the setting of unilateral MTS may indicate the need for intracranial studies, but bilateral IEDs may not be a contradiction to MRgLiTT. We propose cautious use of these results in counseling patients. We agree with the authors’ conclusion that further studies with larger cohorts will be needed to clarify the true effect size for predicting seizure freedom.