Karla Hemming

The stepped wedge cluster randomised trial : rationale, design, analysis, and reporting

The stepped wedge cluster randomised trial is a novel research study design that is increasingly being used in the evaluation of service delivery type interventions. The design involves random and sequential crossover of clusters from control to intervention until all clusters are exposed. It is a pragmatic study design which can reconcile the need for robust evaluations with political or logistical constraints. While not exclusively for the evaluation of service delivery interventions, it is particularly suited to evaluations that do not rely on individual patient recruitment. As in all cluster trials, stepped wedge trials with individual recruitment and without concealment of allocation (or blinding of the intervention) are at risk of selection biases. In a stepped wedge design more clusters are exposed to the intervention towards the end of the study than in its early stages. This implies that the effect of the intervention might be confounded with any underlying temporal trend. A result that initially might seem suggestive of an effect of the intervention may therefore transpire to be the result of a positive underlying temporal trend. Sample size calculations and analysis must make allowance for both the clustered nature of the design and the confounding effect of time. The stepped wedge cluster randomised trial is an alternative to traditional parallel cluster studies, in which the intervention is delivered in only half the clusters with the remainder functioning as controls. When the clusters are relatively homogeneous (that is, the intra-cluster correlation is small), parallel studies tend to deliver better statistical performance than a stepped wedge trial. However, if substantial cluster-level effects are present (that is, larger intra-cluster correlations) or the clusters are large, the stepped wedge design will be more powerful than a parallel design, even one in which the intervention is preceded by a period of baseline control observations.

Percutaneous vesicoamniotic shunting versus conservative management for fetal lower urinary tract obstruction (PLUTO): a randomised trial.

Summary Background Fetal lower urinary tract obstruction (LUTO) is associated with high perinatal and long-term childhood mortality and morbidity. We aimed to assess the effectiveness of vesicoamniotic shunting for treatment of LUTO. Methods In a randomised trial in the UK, Ireland, and the Netherlands, women whose pregnancies with a male fetus were complicated by isolated LUTO were randomly assigned by a central telephone and web-based randomisation service to receive either the intervention (placement of vesicoamniotic shunt) or conservative management. Allocation could not be masked from clinicians or participants because of the invasive nature of the intervention. Diagnosis was by prenatal ultrasound. The primary outcome was survival of the baby to 28 days postnatally. All primary analyses were done on an intention-to-treat basis, but these results were compared with those of an as-treated analysis to investigate the effect of a fairly large proportion of crossovers. We used Bayesian methods to estimate the posterior probability distribution of the effectiveness of vesicoamniotic shunting at 28 days. The study is registered with the ISRCTN Register, number ISRCTN53328556. Findings 31 women with singleton pregnancies complicated by LUTO were included in the trial and main analysis, with 16 allocated to the vesicoamniotic shunt group and 15 to the conservative management group. The study closed early because of poor recruitment. There were 12 livebirths in each group. In the vesicoamniotic shunt group one intrauterine death occurred and three pregnancies were terminated. In the conservative management group one intrauterine death occurred and two pregnancies were terminated. Of the 16 pregnancies randomly assigned to vesicoamniotic shunting, eight neonates survived to 28 days, compared with four from the 15 pregnancies assigned to conservative management (intention-to-treat relative risk [RR] 1·88, 95% CI 0·71–4·96; p=0·27). Analysis based on treatment received showed a larger effect (3·20, 1·06–9·62; p=0·03). All 12 deaths were caused by pulmonary hypoplasia in the early neonatal period. Sensitivity analysis in which non-treatment-related terminations of pregnancy were excluded made some slight changes to point estimates only. Bayesian analysis in which the trial data were combined with elicited priors from experts suggested an 86% probability that vesicoamniotic shunting increased survival at 28 days and a 25% probability that it had a large, clinically important effect (defined as a relative increase of 55% or more in the proportion of neonates who survived). There was substantial short-term and long-term morbidity in both groups, including poor renal function—only two babies (both in the shunt group) survived to 2 years with normal renal function. Seven complications occurred in six fetuses from the shunt group, including spontaneous ruptured membranes, shunt blockage, and dislodgement. These complications resulted in four pregnancy losses. Interpretation Survival seemed to be higher in the fetuses receiving vesicoamniotic shunting, but the size and direction of the effect remained uncertain, such that benefit could not be conclusively proven. Our results suggest that the chance of newborn babies surviving with normal renal function is very low irrespective of whether or not vesicoamniotic shunting is done. Funding UK National Institute of Health Research, Wellbeing of Women, Hannah Eliza Guy Charity (Birmingham Childrens Hospital Charity).

Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation

Objectives To conduct an independent evaluation of the first phase of the Health Foundation’s Safer Patients Initiative (SPI), and to identify the net additional effect of SPI and any differences in changes in participating and non-participating NHS hospitals. Design Mixed method evaluation involving five substudies, before and after design. Setting NHS hospitals in the United Kingdom. Participants Four hospitals (one in each country in the UK) participating in the first phase of the SPI (SPI1); 18 control hospitals. Intervention The SPI1 was a compound (multi-component) organisational intervention delivered over 18 months that focused on improving the reliability of specific frontline care processes in designated clinical specialties and promoting organisational and cultural change. Results Senior staff members were knowledgeable and enthusiastic about SPI1. There was a small (0.08 points on a 5 point scale) but significant (P<0.01) effect in favour of the SPI1 hospitals in one of 11 dimensions of the staff questionnaire (organisational climate). Qualitative evidence showed only modest penetration of SPI1 at medical ward level. Although SPI1 was designed to engage staff from the bottom up, it did not usually feel like this to those working on the wards, and questions about legitimacy of some aspects of SPI1 were raised. Of the five components to identify patients at risk of deterioration—monitoring of vital signs (14 items); routine tests (three items); evidence based standards specific to certain diseases (three items); prescribing errors (multiple items from the British National Formulary); and medical history taking (11 items)—there was little net difference between control and SPI1 hospitals, except in relation to quality of monitoring of acute medical patients, which improved on average over time across all hospitals. Recording of respiratory rate increased to a greater degree in SPI1 than in control hospitals; in the second six hours after admission recording increased from 40% (93) to 69% (165) in control hospitals and from 37% (141) to 78% (296) in SPI1 hospitals (odds ratio for “difference in difference” 2.1, 99% confidence interval 1.0 to 4.3; P=0.008). Use of a formal scoring system for patients with pneumonia also increased over time (from 2% (102) to 23% (111) in control hospitals and from 2% (170) to 9% (189) in SPI1 hospitals), which favoured controls and was not significant (0.3, 0.02 to 3.4; P=0.173). There were no improvements in the proportion of prescription errors and no effects that could be attributed to SPI1 in non-targeted generic areas (such as enhanced safety culture). On some measures, the lack of effect could be because compliance was already high at baseline (such as use of steroids in over 85% of cases where indicated), but even when there was more room for improvement (such as in quality of medical history taking), there was no significant additional net effect of SPI1. There were no changes over time or between control and SPI1 hospitals in errors or rates of adverse events in patients in medical wards. Mortality increased from 11% (27) to 16% (39) among controls and decreased from 17% (63) to 13% (49) among SPI1 hospitals, but the risk adjusted difference was not significant (0.5, 0.2 to 1.4; P=0.085). Poor care was a contributing factor in four of the 178 deaths identified by review of case notes. The survey of patients showed no significant differences apart from an increase in perception of cleanliness in favour of SPI1 hospitals. Conclusions The introduction of SPI1 was associated with improvements in one of the types of clinical process studied (monitoring of vital signs) and one measure of staff perceptions of organisational climate. There was no additional effect of SPI1 on other targeted issues nor on other measures of generic organisational strengthening.

Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase.

Objective To independently evaluate the impact of the second phase of the Health Foundation’s Safer Patients Initiative (SPI2) on a range of patient safety measures. Design A controlled before and after design. Five substudies: survey of staff attitudes; review of case notes from high risk (respiratory) patients in medical wards; review of case notes from surgical patients; indirect evaluation of hand hygiene by measuring hospital use of handwashing materials; measurement of outcomes (adverse events, mortality among high risk patients admitted to medical wards, patients’ satisfaction, mortality in intensive care, rates of hospital acquired infection). Setting NHS hospitals in England. Participants Nine hospitals participating in SPI2 and nine matched control hospitals. Intervention The SPI2 intervention was similar to the SPI1, with somewhat modified goals, a slightly longer intervention period, and a smaller budget per hospital. Results One of the scores (organisational climate) showed a significant (P=0.009) difference in rate of change over time, which favoured the control hospitals, though the difference was only 0.07 points on a five point scale. Results of the explicit case note reviews of high risk medical patients showed that certain practices improved over time in both control and SPI2 hospitals (and none deteriorated), but there were no significant differences between control and SPI2 hospitals. Monitoring of vital signs improved across control and SPI2 sites. This temporal effect was significant for monitoring the respiratory rate at both the six hour (adjusted odds ratio 2.1, 99% confidence interval 1.0 to 4.3; P=0.010) and 12 hour (2.4, 1.1 to 5.0; P=0.002) periods after admission. There was no significant effect of SPI for any of the measures of vital signs. Use of a recommended system for scoring the severity of pneumonia improved from 1.9% (1/52) to 21.4% (12/56) of control and from 2.0% (1/50) to 41.7% (25/60) of SPI2 patients. This temporal change was significant (7.3, 1.4 to 37.7; P=0.002), but the difference in difference was not significant (2.1, 0.4 to 11.1; P=0.236). There were no notable or significant changes in the pattern of prescribing errors, either over time or between control and SPI2 hospitals. Two items of medical history taking (exercise tolerance and occupation) showed significant improvement over time, across both control and SPI2 hospitals, but no additional SPI2 effect. The holistic review showed no significant changes in error rates either over time or between control and SPI2 hospitals. The explicit case note review of perioperative care showed that adherence rates for two of the four perioperative standards targeted by SPI2 were already good at baseline, exceeding 94% for antibiotic prophylaxis and 98% for deep vein thrombosis prophylaxis. Intraoperative monitoring of temperature improved over time in both groups, but this was not significant (1.8, 0.4 to 7.6; P=0.279), and there were no additional effects of SPI2. A dramatic rise in consumption of soap and alcohol hand rub was similar in control and SPI2 hospitals (P=0.760 and P=0.889, respectively), as was the corresponding decrease in rates of Clostridium difficile and meticillin resistant Staphylococcus aureus infection (P=0.652 and P=0.693, respectively). Mortality rates of medical patients included in the case note reviews in control hospitals increased from 17.3% (42/243) to 21.4% (24/112), while in SPI2 hospitals they fell from 10.3% (24/233) to 6.1% (7/114) (P=0.043). Fewer than 8% of deaths were classed as avoidable; changes in proportions could not explain the divergence of overall death rates between control and SPI2 hospitals. There was no significant difference in the rate of change in mortality in intensive care. Patients’ satisfaction improved in both control and SPI2 hospitals on all dimensions, but again there were no significant changes between the two groups of hospitals. Conclusions Many aspects of care are already good or improving across the NHS in England, suggesting considerable improvements in quality across the board. These improvements are probably due to contemporaneous policy activities relating to patient safety, including those with features similar to the SPI, and the emergence of professional consensus on some clinical processes. This phenomenon might have attenuated the incremental effect of the SPI, making it difficult to detect. Alternatively, the full impact of the SPI might be observable only in the longer term. The conclusion of this study could have been different if concurrent controls had not been used.

Sample size calculations for cluster randomised controlled trials with a fixed number of clusters

AbstractBackgroundCluster randomised controlled trials (CRCTs) are frequently used in health service evaluation. Assuming an average cluster size, required sample sizes are readily computed for both binary and continuous outcomes, by estimating a design effect or inflation factor. However, where the number of clusters are fixed in advance, but where it is possible to increase the number of individuals within each cluster, as is frequently the case in health service evaluation, sample size formulae have been less well studied.MethodsWe systematically outline sample size formulae (including required number of randomisation units, detectable difference and power) for CRCTs with a fixed number of clusters, to provide a concise summary for both binary and continuous outcomes. Extensions to the case of unequal cluster sizes are provided.ResultsFor trials with a fixed number of equal sized clusters (k), the trial will be feasible provided the number of clusters is greater than the product of the number of individuals required under individual randomisation (nI ) and the estimated intra-cluster correlation (ρ). So, a simple rule is that the number of clusters (k) will be sufficient provided: Where this is not the case, investigators can determine the maximum available power to detect the pre-specified difference, or the minimum detectable difference under the pre-specified value for power.ConclusionsDesigning a CRCT with a fixed number of clusters might mean that the study will not be feasible, leading to the notion of a minimum detectable difference (or a maximum achievable power), irrespective of how many individuals are included within each cluster.

Evaluating policy and service interventions: framework to guide selection and interpretation of study end points

The effect of many cost effective policy and service interventions cannot be detected at the level of the patient. This new framework could help improve the design (especially choice of primary end point) and interpretation of evaluative studies

Prevalence of visual field loss following exposure to vigabatrin therapy: A systematic review

Purpose:  Vigabatrin is an efficacious antiepileptic drug licensed as add‐on therapy in refractory epilepsy and used in infantile spasms. Eight years after licensing, there emerged a strong and possibly causative association with bilateral visual field loss. We report a systematic review ascertaining the magnitude of risk of vigabatrin associated visual field loss (VAVFL) and any clinical predictors of risk.

Stepped-wedge cluster randomised controlled trials: a generic framework including parallel and multiple-level designs

Stepped-wedge cluster randomised trials (SW-CRTs) are being used with increasing frequency in health service evaluation. Conventionally, these studies are cross-sectional in design with equally spaced steps, with an equal number of clusters randomised at each step and data collected at each and every step. Here we introduce several variations on this design and consider implications for power. One modification we consider is the incomplete cross-sectional SW-CRT, where the number of clusters varies at each step or where at some steps, for example, implementation or transition periods, data are not collected. We show that the parallel CRT with staggered but balanced randomisation can be considered a special case of the incomplete SW-CRT. As too can the parallel CRT with baseline measures. And we extend these designs to allow for multiple layers of clustering, for example, wards within a hospital. Building on results for complete designs, power and detectable difference are derived using a Wald test and obtaining the variance–covariance matrix of the treatment effect assuming a generalised linear mixed model. These variations are illustrated by several real examples. We recommend that whilst the impact of transition periods on power is likely to be small, where they are a feature of the design they should be incorporated. We also show examples in which the power of a SW-CRT increases as the intra-cluster correlation (ICC) increases and demonstrate that the impact of the ICC is likely to be smaller in a SW-CRT compared with a parallel CRT, especially where there are multiple levels of clustering. Finally, through this unified framework, the efficiency of the SW-CRT and the parallel CRT can be compared.

Hypoglycaemia is associated with increased length of stay and mortality in people with diabetes who are hospitalized.

Diabet. Med. 29, e445–e448 (2012)

Does exercise improve glycaemic control in type 1 diabetes? A systematic review and meta-analysis.

Objective Whilst regular exercise is advocated for people with type 1 diabetes, the benefits of this therapy are poorly delineated. Our objective was to review the evidence for a glycaemic benefit of exercise in type 1 diabetes. Research Design and Methods Electronic database searches were carried out in MEDLINE, Embase, Cochrane’s Controlled Trials Register and SPORTDiscus. In addition, we searched for as yet unpublished but completed trials. Glycaemic benefit was defined as an improvement in glycosylated haemoglobin (HbA1c). Both randomised and non-randomised controlled trials were included. Results Thirteen studies were identified in the systematic review. Meta-analysis of twelve of these (including 452 patients) demonstrated an HbA1c reduction but this was not statistically significant (standardised mean difference (SMD) −0.25; 95% CI, −0.59 to 0.09). Conclusions This meta-analysis does not reveal evidence for a glycaemic benefit of exercise as measured by HbA1c. Reasons for this finding could include increased calorie intake, insulin dose reductions around the time of exercise or lack of power. We also suggest that HbA1c may not be a sensitive indicator of glycaemic control, and that improvement in glycaemic variability may not be reflected in this measure. Exercise does however have other proven benefits in type 1 diabetes, and remains an important part of its management.