Sandy Oliver

Randomised controlled trials for policy interventions: a review of reviews and meta-regression.

OBJECTIVES To determine whether randomised controlled trials (RCTs) lead to the same effect size and variance as non-randomised studies (NRSs) of similar policy interventions, and whether these findings can be explained by other factors associated with the interventions or their evaluation. DATA SOURCES Two RCTs were resampled to compare randomised and non-randomised arms. Comparable field trials were identified from a series of health promotion systematic reviews and a systematic review of transition for youths with disabilities. Previous methodological studies were sought from 14 electronic bibliographic databases (Applied Social Sciences Index and Abstracts, Australian Education Index, British Education Index, CareData, Dissertation Abstracts, EconLIT, Educational Resources Information Centre, International Bibliography of the Sociological Sciences, ISI Proceedings: Social Sciences and Humanities, PAIS International, PsycINFO, SIGLE, Social Science Citation Index, Sociological Abstracts) in June and July 2004. These were supplemented by citation searching for key authors, contacting review authors and searching key internet sites. REVIEW METHODS Analyses of previous resampling studies, replication studies, comparable field studies and meta-epidemiology investigated the relationship between randomisation and effect size of policy interventions. New resampling studies and new analyses of comparable field studies and meta-epidemiology were strengthened by testing pre-specified associations supported by carefully argued hypotheses. RESULTS Resampling studies offer no evidence that the absence of randomisation directly influences the effect size of policy interventions in a systematic way. Prior methodological reviews and meta-analyses of existing reviews comparing effects from RCTs and non-randomised controlled trials (nRCTs) suggested that effect sizes from RCTs and nRCTs may indeed differ in some circumstances and that these differences may well be associated with factors confounded with design. No consistent explanations were found for randomisation being associated with changes in effect sizes of policy interventions in field trials. CONCLUSIONS From the resampling studies we have no evidence that the absence of randomisation directly influences the effect size of policy interventions in a systematic way. At the level of individual studies, non-randomised trials may lead to different effect sizes, but this is unpredictable. Many of the examples reviewed and the new analyses in the current study reveal that randomisation is indeed associated with changes in effect sizes of policy interventions in field trials. Despite extensive analysis, we have identified no consistent explanations for these differences. Researchers mounting new evaluations need to avoid, wherever possible, allocation bias. New policy evaluations should adopt randomised designs wherever possible.

Structured, intensive education maximising engagement, motivation and long-term change for children and young people with diabetes: a cluster randomised controlled trial with integral process and economic evaluation - the CASCADE study.

BACKGROUND Type 1 diabetes (T1D) in children and young people is increasing worldwide with a particular increase in children under the age of 5 years. Fewer than one in six children and young people achieve glycosylated fraction of haemoglobin (HbA1c) values in the range identified as providing best future outcomes. There is an urgent need for clinic-based pragmatic, feasible and effective interventions that improve both glycaemic control and quality of life (QoL). The intervention offers both structured education, to ensure young people know what they need to know, and a delivery model designed to motivate self-management. OBJECTIVE To assess the feasibility of providing a clinic-based structured educational group programme incorporating psychological approaches to improve long-term glycaemic control, QoL and psychosocial functioning in a diverse range of young people. DESIGN The study was a pragmatic, cluster randomised control trial with integral process and economic evaluation. SETTING Twenty-eight paediatric diabetes services across London, south-east England and the Midlands. RANDOMISATION Minimised by clinic size, age (paediatric or adolescent) and specialisation (district general hospital clinic or teaching hospital/tertiary clinic). ALLOCATION Half of the sites were randomised to the intervention arm and half to the control arm. Allocation was concealed until after clinics had consented and the first participant was recruited. Where possible, families were blind to allocation until recruitment finished. PARTICIPANTS Forty-three health-care practitioners (14 teams) were trained in the intervention. The study recruited 362 children aged 8-16 years, diagnosed with T1D for > 12 months, with a mean 12-month HbA1c level of ≥ 8.5%. INTERVENTION Two 1-day workshops taught intervention delivery. A detailed manual and resources were provided. The intervention consists of four group education sessions led by a paediatric diabetes specialist nurse with another team member. OUTCOMES The primary outcome was glycaemic control, assessed at the individual level using venous HbA1c values, measured at baseline, 12 and 24 months. Secondary outcomes were directly and indirectly related to diabetes management, including hypoglycaemic episodes, hospital admissions, diabetes regimen, knowledge, skills and responsibility for diabetes management, intervention compliance, clinic utilisation, emotional and behavioural adjustment, and general and diabetes-specific QoL. PROCESS EVALUATION Questionnaires, semistructured interviews, informal discussion following observation sessions, fieldwork notes and case note review were used to collect qualitative and quantitative data from key stakeholder groups at specific time points in the trial. STATISTICAL ANALYSES Primary and secondary analyses were intention-to-treat comparisons of outcomes at 12 and 24 months, using analysis of covariance with a random effect for clinic. Prespecified subgroup analyses based on age, gender, initial HbA1c value and socioeconomic status were estimated from models that included an interaction term. The economic analysis compared long-term costs and predicted quality-adjusted life-years (QALYs). RESULTS The intervention did not improve HbA1c at 12 months [intervention effect 0.11; 95% confidence interval (CI) -0.28 to 0.50; p = 0.584] or 24 months (intervention effect 0.03; 95% CI -0.36 to 0.41; p = 0.891). A total of 298/362 patients (82.3%) provided blood samples at 12-month follow-up, and 284/362 (78.5%) provided blood samples at 24-month follow-up. Follow-up questionnaires were completed by 307 patients (85.3%) at 12 months and by 295 patients (81.5%) at 24 months. Intervention group parents at 12 months (95% CI 0.74; 0.03 to 1.52) and young people at 24 months (0.85; 95% CI 0.03 to 1.61) had higher scores on the diabetes family responsibility questionnaire. Young people reported reduced happiness with body weight at 12 months (-0.56; 95% CI -1.03 to -0.06). Only 68% of groups were run. Of the 180 families recruited, 96 (53%) attended at least one module. Reasons for low uptake included difficulties organising groups, and work and school commitments. Young people with higher HbA1c levels were less likely to attend. Parents and young people who attended groups described improved family relationships, improved knowledge and understanding, greater confidence and increased motivation to manage diabetes. Twenty-four months after the intervention, nearly half of the young people reported that the groups had made them want to try harder and that they had carried on trying. A high-quality, complex, pragmatic trial of structured education can be delivered alongside standard care in NHS diabetes clinics. Health-care providers benefited from behaviour change skill training and can deliver pragmatic aspects of a National Institute for Health and Care Excellence (NICE)-compliant structured education programme after relatively brief training. The process evaluation provides insight into aspects of the model, and highlights strengths and aspects that may have contributed to the failure to influence primary and secondary outcomes. Current NHS practice dominates CASCADE (Child and Adolescent Structured Competencies Approach to Diabetes Education) in that it achieves the same number of QALYs at a lower cost. The mean cost of providing the intervention was £5098 per site or £683 per child. Members of paediatric diabetes services trained to deliver the CASCADE structured education package using behaviour change techniques did not improve glycaemic control in patients compared with control subjects 1 and 2 years after the intervention. The training workshops for practitioners were well evaluated; however, more intensive training was needed. The intervention cost £683 per patient but was not cost-effective because it did not improve metabolic control. CONCLUSIONS A high-quality, complex, pragmatic trial of structured education can be successfully conducted alongside standard care in NHS diabetes clinics. Pragmatic components of a NICE-compliant structured education programme can be successfully delivered following a relatively brief 2-day training while paediatric health-care professionals benefit from training in behaviour change skills. The study provides invaluable information on barriers and opportunities regarding future, similar interventions. A low dropout rate and good attendance for the subgroup that attended the intervention suggests there might be improved uptake if offered to young people with lower HbA1c. Testing whether this approach can be more successful with a robust ongoing supervisory element should be a target of further research. TRIAL REGISTRATION Current Controlled Trials ISRCTN52537669. FUNDING This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 18, No. 20. See the NIHR Journals Library website for further project information.

Broadening public participation in systematic reviews: a case example involving young people in two configurative reviews

Background Arguments supporting the involvement of users in research have even more weight when involving the public in systematic reviews of research. We aimed to explore the potential for public involvement in systematic reviews of observational and qualitative studies. Methods Two consultative workshops were carried out with a group of young people (YP) aged 12–17 years to examine two ongoing reviews about obesity: one about childrens views and one on the link between obesity and educational attainment. YP were invited to comment on the credibility of themes, to propose elements of interventions, to suggest links between educational attainment and obesity and to comment on their plausibility. Results Researchers had more confidence in review findings, after checking that themes identified as important by YP were emphasised appropriately. Researchers were able to use factors linking obesity and attainment identified as important by YP to identify limitations in the scope of extant research. Conclusion Consultative workshops helped researchers draw on the perspectives of YP when interpreting and reflecting upon two systematic reviews. Involving users in judging synthesis credibility and identifying concepts was easier than involving them in interpreting findings. Involvement activities for reviews should be designed with review stage, purpose and group in mind.