Ellen Lee

The statistical interpretation of pilot trials: should significance thresholds be reconsidered?

BackgroundIn an evaluation of a new health technology, a pilot trial may be undertaken prior to a trial that makes a definitive assessment of benefit. The objective of pilot studies is to provide sufficient evidence that a larger definitive trial can be undertaken and, at times, to provide a preliminary assessment of benefit.MethodsWe describe significance thresholds, confidence intervals and surrogate markers in the context of pilot studies and how Bayesian methods can be used in pilot trials. We use a worked example to illustrate the issues raised.ResultsWe show how significance levels other than the traditional 5% should be considered to provide preliminary evidence for efficacy and how estimation and confidence intervals should be the focus to provide an estimated range of possible treatment effects. We also illustrate how Bayesian methods could also assist in the early assessment of a health technology.ConclusionsWe recommend that in pilot trials the focus should be on descriptive statistics and estimation, using confidence intervals, rather than formal hypothesis testing and that confidence intervals other than 95% confidence intervals, such as 85% or 75%, be used for the estimation. The confidence interval should then be interpreted with regards to the minimum clinically important difference. We also recommend that Bayesian methods be used to assist in the interpretation of pilot trials. Surrogate endpoints can also be used in pilot trials but they must reliably predict the overall effect on the clinical outcome.

The use of non-invasive ventilation at end of life in patients with motor neurone disease: A qualitative exploration of family carer and health professional experiences

Background: Non-invasive ventilation improves quality and quantity of life in patients with motor neurone disease who have respiratory failure. Use of non-invasive ventilation may, however, result in complex clinical issues for end-of-life care, with concerns as to whether and how it should be withdrawn. Aim: This study aimed to describe carer and health professional experiences of end-of-life care of motor neurone disease patients using non-invasive ventilation. Design/participants: This article reports data from qualitative interviews with family carers and professionals following the death of patients with motor neurone disease who were using non-invasive ventilation in the final phase of the disease. Results: Ten of the 20 patients initiated on non-invasive ventilation were using it in the end-of-life phase of their disease, with 5 using it for 24 h/day. Interviews were carried out with nine family carers and 15 professionals. Nine recurring themes were identified in the data. Both carers and health-care professionals perceived that the terminal phase of motor neurone disease was unexpectedly rapid and that this often led to unplanned interactions with the emergency services. Carers of patients who used non-invasive ventilation perceived non-invasive ventilation as aiding patient comfort and anxiety at the end of life. Conclusions: The use of non-invasive ventilation was described as beneficial and was not perceived by carers or most professionals to have adversely impacted patient’s end-of-life experience. This study highlights variation in patient wishes regarding usage towards the end of life, uncertainty regarding appropriate management among professionals and the importance of disseminating end-of-life wishes.

The initiation of non-invasive ventilation for patients with motor neuron disease: Patient and carer perceptions of obstacles and outcomes

Abstract This study aimed to examine the experiences of patients with motor neuron disease and their carers following the recommendation to use non-invasive ventilation (NIV). Qualitative interviews were carried out with 20 patients and 17 carers within one month of NIV being initiated. The study identified a range of potential barriers to usage including: adverse impressions of the technology; sleep disturbance; the sensation of pressure and pulsing; dry mouth; and mask design issues. Patients/carers perceived benefits related to: increased energy; improved sleeping; enhanced carer well-being; improved breathing and increased speech clarity. A key factor described by patients and carers was the need to persevere to overcome the challenges associated with early NIV use. The study highlights the importance of patient perceptions of gains as a factor in their NIV usage decisions. While recognizing that older individuals with limb-onset disease were over-represented in the sample, key recommendations from the study are: availability of easily accessible in-person support for patients; for clinicians to pre-empt potential obstacles by discussing options such as humidification or alternative mask interfaces; the importance of discussing potential benefits in detail with patients; and optimization of secretion management prior to NIV trial.

Effect of lipid profile on prognosis in the patients with amyotrophic lateral sclerosis: Insights from the olesoxime clinical trial

Patients with ALS may have insufficientenergy substrates, due to dysphagia and hypermetabolism, which adversely affects the prognosis. Hyperlipidaemia has been reported to be associated with ALS and to represent a significant prognostic factor for survival in ALS. The aim of this study was to examine the prevalence of dyslipidaemia among a cohort of patients with ALS and how the lipid profile of patients with ALS influenced the prognosis. This was a prospective observational cohort study comprising 512 ALS patients, recruited for the TRO19622 (Olesoxime) investigational medicinal product trial. Fasting serum concentrations of total cholesterol (TC), low density lipoprotein cholesterol (LDL-C) and high density lipoprotein cholesterol (HDL-C) were measured as part of routine monitoring in the trial. Seventy-three percent of the participants had hypercholesterolaemia (defined as total cholesterol ≥ 5.1 mmol/l) at the screening visit. The prevalence of hypercholesterolaemia decreased with time and was 64% at 18 months follow-up. On unadjusted analysis total cholesterol, LDL-C and LDL/HDL ratio had a statistically significant effect on survival (p = 0.015, 0.003 and 0.027, respectively). When adjusted for prognostic covariates, however, none of the lipid measures was found to have a statistically significant effect on survival. In conclusion, prognosis in ALS is not influenced by the lipid profile of patients.

Relative effectiveness of insulin pump treatment over multiple daily injections and structured education during flexible intensive insulin treatment for type 1 diabetes: cluster randomised trial (REPOSE)

Objective To compare the effectiveness of insulin pumps with multiple daily injections for adults with type 1 diabetes, with both groups receiving equivalent training in flexible insulin treatment. Design Pragmatic, multicentre, open label, parallel group, cluster randomised controlled trial (Relative Effectiveness of Pumps Over MDI and Structured Education (REPOSE) trial). Setting Eight secondary care centres in England and Scotland. Participants Adults with type 1 diabetes who were willing to undertake intensive insulin treatment, with no preference for pumps or multiple daily injections. Participants were allocated a place on established group training courses that taught flexible intensive insulin treatment (“dose adjustment for normal eating,” DAFNE). The course groups (the clusters) were then randomly allocated in pairs to either pump or multiple daily injections. Interventions Participants attended training in flexible insulin treatment (using insulin analogues) structured around the use of pump or injections, followed for two years. Main outcome measures The primary outcomes were a change in glycated haemoglobin (HbA1c) values (%) at two years in participants with baseline HbA1c value of ≥7.5% (58 mmol/mol), and the proportion of participants achieving an HbA1c value of <7.5%. Secondary outcomes included body weight, insulin dose, and episodes of moderate and severe hypoglycaemia. Ancillary outcomes included quality of life and treatment satisfaction. Results 317 participants (46 courses) were randomised (156 pump and 161 injections). 267 attended courses and 260 were included in the intention to treat analysis, of which 235 (119 pump and 116 injection) had baseline HbA1c values of ≥7.5%. Glycaemic control and rates of severe hypoglycaemia improved in both groups. The mean change in HbA1c at two years was −0.85% with pump treatment and −0.42% with multiple daily injections. Adjusting for course, centre, age, sex, and accounting for missing values, the difference was −0.24% (−2.7 mmol/mol) in favour of pump users (95% confidence interval −0.53 to 0.05, P=0.10). Most psychosocial measures showed no difference, but pump users showed greater improvement in treatment satisfaction and some quality of life domains (dietary freedom and daily hassle) at 12 and 24 months. Conclusions Both groups showed clinically relevant and long lasting decreases in HbA1c, rates of severe hypoglycaemia, and improved psychological measures, although few participants achieved glucose levels currently recommended by national and international guidelines. Adding pump treatment to structured training in flexible intensive insulin treatment did not substantially enhance educational benefits on glycaemic control, hypoglycaemia, or psychosocial outcomes in adults with type 1 diabetes. These results do not support a policy of providing insulin pumps to adults with poor glycaemic control until the effects of training on participants’ level of engagement in intensive self management have been determined. Trial registration Current Controlled Trials ISRCTN61215213.

The Impact on the Family Carer of Motor Neurone Disease and Intervention with Noninvasive Ventilation

BACKGROUND The diagnosis of motor neurone disease (MND) has a profound effect on the functioning and well-being of both the patient and their family, with studies describing an increase in carer burden and depression as the disease progresses. AIM This study aimed to assess whether patient use of noninvasive ventilation (NIV) impacted on their family carer, and to explore other sources of carer burden. DESIGN The study used qualitative interviews and scaled measures of carer health and well-being completed at three monthly intervals until patient end of life. PARTICIPANTS Sixteen family carers were followed up over a period ranging from one month to two years. RESULTS NIV was perceived as having little impact on carer burden. The data however highlighted a range of sources of other burdens relating to the physical strain of caring. The Medical Outcomes Study Short Form (SF-36 Health Survey) Physical Component Summary (PCS) scores were considerably below that of the Mental Component Summary (MCS) score at baseline and at all following time points. Carers described the physical effort associated with patient care and role change; the challenge inherent in having time away; and problems relating to the timing of equipment and service delivery. CONCLUSIONS NIV can be recommended to patients without concerns regarding increasing carer burden. The predominant source of burden described related to the physical impact of caring for a patient with MND. Services face challenges if this physical burden is to be reduced by providing equipment at an optimal time and successfully coordinating their input.

A cluster randomised trial, cost-effectiveness analysis and psychosocial evaluation of insulin pump therapy compared with multiple injections during flexible intensive insulin therapy for type 1 diabetes: the REPOSE Trial

BACKGROUND Insulin is generally administered to people with type 1 diabetes mellitus (T1DM) using multiple daily injections (MDIs), but can also be delivered using infusion pumps. In the UK, pumps are recommended for patients with the greatest need and adult use is less than in comparable countries. Previous trials have been small, of short duration and have failed to control for training in insulin adjustment. OBJECTIVE To assess the clinical effectiveness and cost-effectiveness of pump therapy compared with MDI for adults with T1DM, with both groups receiving equivalent structured training in flexible insulin therapy. DESIGN Pragmatic, multicentre, open-label, parallel-group cluster randomised controlled trial, including economic and psychosocial evaluations. After participants were assigned a group training course, courses were randomly allocated in pairs to either pump or MDI. SETTING Eight secondary care diabetes centres in the UK. PARTICIPANTS Adults with T1DM for > 12 months, willing to undertake intensive insulin therapy, with no preference for pump or MDI, or a clinical indication for pumps. INTERVENTIONS Pump or MDI structured training in flexible insulin therapy, followed up for 2 years. MDI participants used insulin analogues. Pump participants used a Medtronic Paradigm® VeoTM (Medtronic, Watford, UK) with insulin aspart (NovoRapid, Novo Nordisk, Gatwick, UK). MAIN OUTCOME MEASURES Primary outcome - change in glycated haemoglobin (HbA1c) at 2 years in participants whose baseline HbA1c was ≥ 7.5% (58 mmol/mol). Key secondary outcome - proportion of participants with HbA1c ≤ 7.5% at 2 years. Other outcomes at 6, 12 and 24 months - moderate and severe hypoglycaemia; insulin dose; body weight; proteinuria; diabetic ketoacidosis; quality of life (QoL); fear of hypoglycaemia; treatment satisfaction; emotional well-being; qualitative interviews with participants and staff (2 weeks), and participants (6 months); and ICERs in trial and modelled estimates of cost-effectiveness. RESULTS We randomised 46 courses comprising 317 participants: 267 attended a Dose Adjustment For Normal Eating course (132 pump; 135 MDI); 260 were included in the intention-to-treat analysis, of which 235 (119 pump; 116 MDI) had baseline HbA1c of ≥ 7.5%. HbA1c and severe hypoglycaemia improved in both groups. The drop in HbA1c% at 2 years was 0.85 on pump and 0.42 on MDI. The mean difference (MD) in HbA1c change at 2 years, at which the baseline HbA1c was ≥ 7.5%, was -0.24% [95% confidence interval (CI) -0.53% to 0.05%] in favour of the pump (p = 0.098). The per-protocol analysis showed a MD in change of -0.36% (95% CI -0.64% to -0.07%) favouring pumps (p = 0.015). Pumps were not cost-effective in the base case and all of the sensitivity analyses. The pump group had greater improvement in diabetes-specific QoL diet restrictions, daily hassle plus treatment satisfaction, statistically significant at 12 and 24 months and supported by qualitative interviews. LIMITATION Blinding of pump therapy was not possible, although an objective primary outcome was used. CONCLUSION Adding pump therapy to structured training in flexible insulin therapy did not significantly enhance glycaemic control or psychosocial outcomes in adults with T1DM. RESEARCH PRIORITY To understand why few patients achieve a HbA1c of < 7.5%, particularly as glycaemic control is worse in the UK than in other European countries. TRIAL REGISTRATION Current Controlled Trials ISRCTN61215213. FUNDING This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 21, No. 20. See the NIHR Journals Library website for further project information.

Creatine kinase enzyme level correlates positively with serum creatinine and lean body mass, and is a prognostic factor for survival in amyotrophic lateral sclerosis

Amyotrophic lateral sclerosis (ALS) is a fatal neurodegenerative condition for which there is no single diagnostic test or biomarker. The level of the creatine kinase (CK) enzyme in serum may be mild to moderately elevated in some patients with ALS, the precise cause of which and its behaviour with disease progression is unknown. The aim of this study was to examine the usefulness of monitoring CK serially during the ALS disease trajectory and to determine whether CK levels mirror disease progression.

A rapid review indicated higher recruitment rates in treatment trials than in prevention trials

OBJECTIVES To test the hypothesis that the percentage of patients screened that randomize differs between prevention and therapy trials. STUDY DESIGN AND SETTING Rapid review of randomized controlled trials (RCTs) identified through published systematic reviews in August 2013. Individually randomized, parallel group controlled RCTs were eligible if they evaluated metformin monotherapy or exercise for the prevention or treatment of type 2 diabetes. Numbers of patients screened and randomized were extracted by a single reviewer. Percentages were calculated for each study for those randomized: as a function of those approached, screened, and eligible. Percentages (95% confidence intervals) from each individual study were weighted according to the denominator and pooled rates calculated. Statistical heterogeneity was assessed using I(2). RESULTS The percentage of those screened who subsequently randomized was 6.2% (6.0%, 6.4%; 3 studies, I(2) = 100.0%) for metformin prevention trials; 50.7% (49.9%, 51.4%; 21 studies, I(2) = 99.6%) for metformin treatment trials; 4.8% (4.7%, 4.8%; 14 studies, I(2) = 99.9%) for exercise prevention trials; and 43.3% (42.6%, 43.9%; 28 studies, I(2) = 99.8%) for exercise treatment trials. CONCLUSION This study provides qualified support for the hypothesis that prevention trials recruit a smaller proportion of those screened than treatment trials. Statistical heterogeneity associated with pooled estimates and other study limitations is discussed.

Aquatic therapy for children with Duchenne muscular dystrophy: a pilot feasibility randomised controlled trial and mixed-methods process evaluation.

BACKGROUND Duchenne muscular dystrophy (DMD) is a rare disease that causes the progressive loss of motor abilities such as walking. Standard treatment includes physiotherapy. No trial has evaluated whether or not adding aquatic therapy (AT) to land-based therapy (LBT) exercises helps to keep muscles strong and children independent. OBJECTIVES To assess the feasibility of recruiting boys with DMD to a randomised trial evaluating AT (primary objective) and to collect data from them; to assess how, and how well, the intervention and trial procedures work. DESIGN Parallel-group, single-blind, randomised pilot trial with nested qualitative research. SETTING Six paediatric neuromuscular units. PARTICIPANTS Children with DMD aged 7-16 years, established on corticosteroids, with a North Star Ambulatory Assessment (NSAA) score of 8-34 and able to complete a 10-m walk without aids/assistance. Exclusions: > 20% variation between baseline screens 4 weeks apart and contraindications. INTERVENTIONS Participants were allocated on a 1 : 1 ratio to (1) optimised, manualised LBT (prescribed by specialist neuromuscular physiotherapists) or (2) the same plus manualised AT (30 minutes, twice weekly for 6 months: active assisted and/or passive stretching regime; simulated or real functional activities; submaximal exercise). Semistructured interviews with participants, parents (n = 8) and professionals (n = 8) were analysed using Framework analysis. An independent rater reviewed patient records to determine the extent to which treatment was optimised. A cost-impact analysis was performed. Quantitative and qualitative data were mixed using a triangulation exercise. MAIN OUTCOME MEASURES Feasibility of recruiting 40 participants in 6 months, participant and therapist views on the acceptability of the intervention and research protocols, clinical outcomes including NSAA, independent assessment of treatment optimisation and intervention costs. RESULTS Over 6 months, 348 children were screened - most lived too far from centres or were enrolled in other trials. Twelve (30% of target) were randomised to AT (n = 8) or control (n = 4). People in the AT (n = 8) and control (n = 2: attrition because of parental report) arms contributed outcome data. The mean change in NSAA score at 6 months was -5.5 [standard deviation (SD) 7.8] for LBT and -2.8 (SD 4.1) in the AT arm. One boy suffered pain and fatigue after AT, which resolved the same day. Physiotherapists and parents valued AT and believed that it should be delivered in community settings. The independent rater considered AT optimised for three out of eight children, with other children given programmes that were too extensive and insufficiently focused. The estimated NHS costs of 6-month service were between £1970 and £2734 per patient. LIMITATIONS The focus on delivery in hospitals limits generalisability. CONCLUSIONS Neither a full-scale frequentist randomised controlled trial (RCT) recruiting in the UK alone nor a twice-weekly open-ended AT course delivered at tertiary centres is feasible. Further intervention development research is needed to identify how community-based pools can be accessed, and how families can link with each other and community physiotherapists to access tailored AT programmes guided by highly specialised physiotherapists. Bayesian RCTs may be feasible; otherwise, time series designs are recommended. TRIAL REGISTRATION Current Controlled Trials ISRCTN41002956. FUNDING This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 21, No. 27. See the NIHR Journals Library website for further project information.