Shaun Treweek

Improving the reporting of pragmatic trials: an extension of the CONSORT statement

Background The CONSORT statement is intended to improve reporting of randomised controlled trials and focuses on minimising the risk of bias (internal validity). The applicability of a trial’s results (generalisability or external validity) is also important, particularly for pragmatic trials. A pragmatic trial (a term first used in 1967 by Schwartz and Lellouch) can be broadly defined as a randomised controlled trial whose purpose is to inform decisions about practice. This extension of the CONSORT statement is intended to improve the reporting of such trials and focuses on applicability. Methods At two, two-day meetings held in Toronto in 2005 and 2008, we reviewed the CONSORT statement and its extensions, the literature on pragmatic trials and applicability, and our experiences in conducting pragmatic trials. Recommendations We recommend extending eight CONSORT checklist items for reporting of pragmatic trials: the background, participants, interventions, outcomes, sample size, blinding, participant flow, and generalisability of the findings. These extensions are presented, along with illustrative examples of reporting, and an explanation of each extension. Adherence to these reporting criteria will make it easier for decision makers to judge how applicable the results of randomised controlled trials are to their own conditions. Empirical studies are needed to ascertain the usefulness and comprehensiveness of these CONSORT checklist item extensions. In the meantime we recommend that those who support, conduct, and report pragmatic trials should use this extension of the CONSORT statement to facilitate the use of trial results in decisions about health care. Pragmatic trials are designed to inform decisions about practice, but poor reporting can reduce their usefulness. The CONSORT and Practihc groups describe modifications to the CONSORT guidelines to help readers assess the applicability of the results

Development of a theory of implementation and integration: Normalization Process Theory

BackgroundTheories are important tools in the social and natural sciences. The methods by which they are derived are rarely described and discussed. Normalization Process Theory explains how new technologies, ways of acting, and ways of working become routinely embedded in everyday practice, and has applications in the study of implementation processes. This paper describes the process by which it was built.MethodsBetween 1998 and 2008, we developed a theory. We derived a set of empirical generalizations from analysis of data collected in qualitative studies of healthcare work and organization. We developed an applied theoretical model through analysis of empirical generalizations. Finally, we built a formal theory through a process of extension and implication analysis of the applied theoretical model.ResultsEach phase of theory development showed that the constructs of the theory did not conflict with each other, had explanatory power, and possessed sufficient robustness for formal testing. As the theory developed, its scope expanded from a set of observed regularities in data with procedural explanations, to an applied theoretical model, to a formal middle-range theory.ConclusionNormalization Process Theory has been developed through procedures that were properly sceptical and critical, and which were opened to review at each stage of development. The theory has been shown to merit formal testing.

Understanding the implementation of complex interventions in health care: The normalization process model

BackgroundThe Normalization Process Model is a theoretical model that assists in explaining the processes by which complex interventions become routinely embedded in health care practice. It offers a framework for process evaluation and also for comparative studies of complex interventions. It focuses on the factors that promote or inhibit the routine embedding of complex interventions in health care practice.MethodsA formal theory structure is used to define the model, and its internal causal relations and mechanisms. The model is broken down to show that it is consistent and adequate in generating accurate description, systematic explanation, and the production of rational knowledge claims about the workability and integration of complex interventions.ResultsThe model explains the normalization of complex interventions by reference to four factors demonstrated to promote or inhibit the operationalization and embedding of complex interventions (interactional workability, relational integration, skill-set workability, and contextual integration).ConclusionThe model is consistent and adequate. Repeated calls for theoretically sound process evaluations in randomized controlled trials of complex interventions, and policy-makers who call for a proper understanding of implementation processes, emphasize the value of conceptual tools like the Normalization Process Model.

Normalisation process theory: a framework for developing, evaluating and implementing complex interventions

BackgroundThe past decade has seen considerable interest in the development and evaluation of complex interventions to improve health. Such interventions can only have a significant impact on health and health care if they are shown to be effective when tested, are capable of being widely implemented and can be normalised into routine practice. To date, there is still a problematic gap between research and implementation. The Normalisation Process Theory (NPT) addresses the factors needed for successful implementation and integration of interventions into routine work (normalisation).DiscussionIn this paper, we suggest that the NPT can act as a sensitising tool, enabling researchers to think through issues of implementation while designing a complex intervention and its evaluation. The need to ensure trial procedures that are feasible and compatible with clinical practice is not limited to trials of complex interventions, and NPT may improve trial design by highlighting potential problems with recruitment or data collection, as well as ensuring the intervention has good implementation potential.SummaryThe NPT is a new theory which offers trialists a consistent framework that can be used to describe, assess and enhance implementation potential. We encourage trialists to consider using it in their next trial.

Making trials matter: pragmatic and explanatory trials and the problem of applicability

Randomised controlled trials are the best research design for decisions about the effect of different interventions but randomisation does not, of itself, promote the applicability of a trials results to situations other than the precise one in which the trial was done. While methodologists and trialists have rightly paid great attention to internal validity, much less has been given to applicability.This narrative review is aimed at those planning to conduct trials, and those aiming to use the information in them. It is intended to help the former group make their trials more widely useful and to help the latter group make more informed decisions about the wider use of existing trials. We review the differences between the design of most randomised trials (which have an explanatory attitude) and the design of trials more able to inform decision making (which have a pragmatic attitude) and discuss approaches used to assert applicability of trial results.If we want evidence from trials to be used in clinical practice and policy, trialists should make every effort to make their trial widely applicable, which means that more trials should be pragmatic in attitude.

The PRECIS-2 tool: designing trials that are fit for purpose

PRECIS is a tool to help trialists make design decisions consistent with the intended purpose of their trial. This paper gives guidance on how to use an improved, validated version, PRECIS-2, which has been developed with the help of over 80 international trialists, clinicians, and policymakers. Keeping the original simple wheel format, PRECIS-2 has nine domains—eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis—scored from 1 (very explanatory) to 5 (very pragmatic) to facilitate domain discussion and consensus. It is hoped PRECIS-2 will be valuable in supporting the explicit matching of design decisions to how the trial results are intended to be used

Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis

This review is an abridged version of a Cochrane Review previously published in the Cochrane Database of Systematic Reviews 2010, Issue 4, Art. No.: MR000013 DOI: 10.1002/14651858.MR000013.pub5 (see www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as new evidence emerges and in response to feedback, and Cochrane Database of Systematic Reviews should be consulted for the most recent version of the review. Objective To identify interventions designed to improve recruitment to randomised controlled trials, and to quantify their effect on trial participation. Design Systematic review. Data sources The Cochrane Methodology Review Group Specialised Register in the Cochrane Library, MEDLINE, EMBASE, ERIC, Science Citation Index, Social Sciences Citation Index, C2-SPECTR, the National Research Register and PubMed. Most searches were undertaken up to 2010; no language restrictions were applied. Study selection Randomised and quasi-randomised controlled trials, including those recruiting to hypothetical studies. Studies on retention strategies, examining ways to increase questionnaire response or evaluating the use of incentives for clinicians were excluded. The study population included any potential trial participant (eg, patient, clinician and member of the public), or individual or group of individuals responsible for trial recruitment (eg, clinicians, researchers and recruitment sites). Two authors independently screened identified studies for eligibility. Results 45 trials with over 43 000 participants were included. Some interventions were effective in increasing recruitment: telephone reminders to non-respondents (risk ratio (RR) 1.66, 95% CI 1.03 to 2.46; two studies, 1058 participants), use of opt-out rather than opt-in procedures for contacting potential participants (RR 1.39, 95% CI 1.06 to 1.84; one study, 152 participants) and open designs where participants know which treatment they are receiving in the trial (RR 1.22, 95% CI 1.09 to 1.36; two studies, 4833 participants). However, the effect of many other strategies is less clear, including the use of video to provide trial information and interventions aimed at recruiters. Conclusions There are promising strategies for increasing recruitment to trials, but some methods, such as open-trial designs and opt-out strategies, must be considered carefully as their use may also present methodological or ethical challenges. Questions remain as to the applicability of results originating from hypothetical trials, including those relating to the use of monetary incentives, and there is a clear knowledge gap with regard to effective strategies aimed at recruiters.

A gender-sensitised weight loss and healthy living programme for overweight and obese men delivered by Scottish Premier League football clubs (FFIT): a pragmatic randomised controlled trial

BACKGROUND The prevalence of male obesity is increasing but few men take part in weight loss programmes. We assessed the effect of a weight loss and healthy living programme on weight loss in football (soccer) fans. METHODS We did a two-group, pragmatic, randomised controlled trial of 747 male football fans aged 35-65 years with a body-mass index (BMI) of 28 kg/m(2) or higher from 13 Scottish professional football clubs. Participants were randomly assigned with SAS (version 9·2, block size 2-9) in a 1:1 ratio, stratified by club, to a weight loss programme delivered by community coaching staff in 12 sessions held every week. The intervention group started a weight loss programme within 3 weeks, and the comparison group were put on a 12 month waiting list. All participants received a weight management booklet. Primary outcome was mean difference in weight loss between groups at 12 months, expressed as absolute weight and a percentage of their baseline weight. Primary outcome assessment was masked. Analyses were based on intention to treat. The trial is registered with Current Controlled Trials, number ISRCTN32677491. FINDINGS 374 men were allocated to the intervention group and 374 to the comparison group. 333 (89%) of the intervention group and 355 (95%) of the comparison group completed 12 month assessments. At 12 months the mean difference in weight loss between groups, adjusted for baseline weight and club, was 4·94 kg (95% CI 3·95-5·94) and percentage weight loss, similarly adjusted, was 4·36% (3·64-5·08), both in favour of the intervention (p<0·0001). Eight serious adverse events were reported, five in the intervention group (lost consciousness due to drugs for pre-existing angina, gallbladder removal, hospital admission with suspected heart attack, ruptured gut, and ruptured Achilles tendon) and three in the comparison group (transient ischaemic attack, and two deaths). Of these, two adverse events were reported as related to participation in the programme (gallbladder removal and ruptured Achilles tendon). INTERPRETATION The FFIT programme can help a large proportion of men to lose a clinically important amount of weight; it offers one effective strategy to challenge male obesity. FUNDING Scottish Government and The UK Football Pools funded delivery of the programme through a grant to the Scottish Premier League Trust. The National Institute for Health Research Public Health Research Programme funded the assessment (09/3010/06).

Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting

BackgroundProcess evaluations are recommended to open the ‘black box’ of complex interventions evaluated in trials, but there is limited guidance to help researchers design process evaluations. Much current literature on process evaluations of complex interventions focuses on qualitative methods, with less attention paid to quantitative methods. This discrepancy led us to develop our own framework for designing process evaluations of cluster-randomised controlled trials.MethodsWe reviewed recent theoretical and methodological literature and selected published process evaluations; these publications identified a need for structure to help design process evaluations. We drew upon this literature to develop a framework through iterative exchanges, and tested this against published evaluations.ResultsThe developed framework presents a range of candidate approaches to understanding trial delivery, intervention implementation and the responses of targeted participants. We believe this framework will be useful to others designing process evaluations of complex intervention trials. We also propose key information that process evaluations could report to facilitate their identification and enhance their usefulness.ConclusionThere is no single best way to design and carry out a process evaluation. Researchers will be faced with choices about what questions to focus on and which methods to use. The most appropriate design depends on the purpose of the process evaluation; the framework aims to help researchers make explicit their choices of research questions and methods.Trial registrationClinicaltrials.gov NCT01425502

Supporting the delivery of cost-effective interventions in primary health-care systems in low-income and middle-income countries: an overview of systematic reviews

Strengthening health systems is a key challenge to improving the delivery of cost-effective interventions in primary health care and achieving the vision of the Alma-Ata Declaration. Effective governance, financial and delivery arrangements within health systems, and effective implementation strategies are needed urgently in low-income and middle-income countries. This overview summarises the evidence from systematic reviews of health systems arrangements and implementation strategies, with a particular focus on evidence relevant to primary health care in such settings. Although evidence is sparse, there are several promising health systems arrangements and implementation strategies for strengthening primary health care. However, their introduction must be accompanied by rigorous evaluations. The evidence base needs urgently to be strengthened, synthesised, and taken into account in policy and practice, particularly for the benefit of those who have been excluded from the health care advances of recent decades.