Mark Petticrew

Developing and evaluating complex interventions: the new Medical Research Council guidance

Evaluating complex interventions is complicated. The Medical Research Councils evaluation framework (2000) brought welcome clarity to the task. Now the council has updated its guidance

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement

Systematic reviews should build on a protocol that describes the rationale, hypothesis, and planned methods of the review; few reviews report whether a protocol exists. Detailed, well-described protocols can facilitate the understanding and appraisal of the review methods, as well as the detection of modifications to methods and selective reporting in completed reviews. We describe the development of a reporting guideline, the Preferred Reporting Items for Systematic reviews and Meta-Analyses for Protocols 2015 (PRISMA-P 2015). PRISMA-P consists of a 17-item checklist intended to facilitate the preparation and reporting of a robust protocol for the systematic review. Funders and those commissioning reviews might consider mandating the use of the checklist to facilitate the submission of relevant protocol information in funding applications. Similarly, peer reviewers and editors can use the guidance to gauge the completeness and transparency of a systematic review protocol submitted for publication in a journal or other medium.

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation

Protocols of systematic reviews and meta-analyses allow for planning and documentation of review methods, act as a guard against arbitrary decision making during review conduct, enable readers to assess for the presence of selective reporting against completed reviews, and, when made publicly available, reduce duplication of efforts and potentially prompt collaboration. Evidence documenting the existence of selective reporting and excessive duplication of reviews on the same or similar topics is accumulating and many calls have been made in support of the documentation and public availability of review protocols. Several efforts have emerged in recent years to rectify these problems, including development of an international register for prospective reviews (PROSPERO) and launch of the first open access journal dedicated to the exclusive publication of systematic review products, including protocols (BioMed Central’s Systematic Reviews). Furthering these efforts and building on the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines, an international group of experts has created a guideline to improve the transparency, accuracy, completeness, and frequency of documented systematic review and meta-analysis protocols—PRISMA-P (for protocols) 2015. The PRISMA-P checklist contains 17 items considered to be essential and minimum components of a systematic review or meta-analysis protocol. This PRISMA-P 2015 Explanation and Elaboration paper provides readers with a full understanding of and evidence about the necessity of each item as well as a model example from an existing published protocol. This paper should be read together with the PRISMA-P 2015 statement. Systematic review authors and assessors are strongly encouraged to make use of PRISMA-P when drafting and appraising review protocols.

Evidence, hierarchies, and typologies: horses for courses

Debate is ongoing about the nature and use of evidence in public health decision making, and there seems to be an emerging consensus that the “hierarchy of evidence” may be difficult to apply in other settings. It may be unhelpful however to simply abandon the hierarchy without having a framework or guide to replace it. One such framework is discussed. This is based around a matrix, and emphasises the need to match research questions to specific types of research. This emphasis on methodological appropriateness, and on typologies rather than hierarchies of evidence may be helpful in organising and appraising public health evidence.

Using natural experiments to evaluate population health interventions: new Medical Research Council guidance

Natural experimental studies are often recommended as a way of understanding the health impact of policies and other large scale interventions. Although they have certain advantages over planned experiments, and may be the only option when it is impossible to manipulate exposure to the intervention, natural experimental studies are more susceptible to bias. This paper introduces new guidance from the Medical Research Council to help researchers and users, funders and publishers of research evidence make the best use of natural experimental approaches to evaluating population health interventions. The guidance emphasises that natural experiments can provide convincing evidence of impact even when effects are small or take time to appear. However, a good understanding is needed of the process determining exposure to the intervention, and careful choice and combination of methods, testing of assumptions and transparent reporting is vital. More could be learnt from natural experiments in future as experience of promising but lesser used methods accumulates.

What types of interventions generate inequalities? Evidence from systematic reviews

Background Some effective public health interventions may increase inequalities by disproportionately benefiting less disadvantaged groups (‘intervention-generated inequalities’ or IGIs). There is a need to understand which types of interventions are likely to produce IGIs, and which can reduce inequalities. Methods We conducted a rapid overview of systematic reviews to identify evidence on IGIs by socioeconomic status. We included any review of non-healthcare interventions in high-income countries presenting data on differential intervention effects on any health status or health behaviour outcome. Results were synthesised narratively. Results The following intervention types show some evidence of increasing inequalities (IGIs) between socioeconomic status groups: media campaigns; and workplace smoking bans. However, for many intervention types, data on potential IGIs are lacking. By contrast, the following show some evidence of reducing health inequalities: structural workplace interventions; provision of resources; and fiscal interventions, such as tobacco pricing. Conclusion Our findings are consistent with the idea that ‘downstream’ preventive interventions are more likely to increase health inequalities than ‘upstream’ interventions. More consistent reporting of differential intervention effectiveness is required to help build the evidence base on IGIs.

Promoting walking and cycling as an alternative to using cars: systematic review

Abstract Objectives To assess what interventions are effective in promoting a population shift from using cars towards walking and cycling and to assess the health effects of such interventions. Data sources Published and unpublished reports in any language identified from electronic databases, bibliographies, websites, and reference lists. Review methods Systematic search and appraisal to identify experimental or observational studies with a prospective or controlled retrospective design that evaluated any intervention applied to an urban population or area by measuring outcomes in members of the local population. Results 22 studies met the inclusion criteria. We found some evidence that targeted behaviour change programmes can change the behaviour of motivated subgroups, resulting (in the largest study) in a shift of around 5% of all trips at a population level. Single studies of commuter subsidies and a new railway station also showed positive effects. The balance of best available evidence about publicity campaigns, engineering measures, and other interventions suggests that they have not been effective. Participants in trials of active commuting experienced short term improvements in certain measures of health and fitness, but we found no good evidence on effects on health of any effective intervention at population level. Conclusions The best available evidence of effectiveness in promoting a modal shift is for targeted behaviour change programmes, but the social distribution of their effects is unclear and some other types of intervention have yet to be rigorously evaluated.

Systematic reviews from astronomy to zoology: myths and misconceptions

Systematic literature reviews are widely used as an aid to evidence based decision making. For example, reviews of randomised controlled trials are regularly used to answer questions about the effectiveness of healthcare interventions. The high profile of systematic reviews as a cornerstone of evidence based medicine, however, has led to several misconceptions about their purpose and methods. Among these is the belief that systematic reviews are applicable only to randomised controlled trials and that they are incapable of dealing with other forms of evidence, such as from non-randomised studies or qualitative research. The systematic literature review is a method of locating, appraising, and synthesising evidence. The value of regularly updated systematic reviews in the assessment of effectiveness of healthcare interventions was dramatically illustrated by Antman and colleagues, who showed that review articles failed to mention advances in treatment identified by an updated systematic review.1 It is nearly a quarter of a century since Gene Glass coined the term “meta-analysis” to refer to the quantitative synthesis of the results of primary studies.2 The importance of making explicit efforts to limit bias in the review of literature, however, has been emphasised by social scientists at least since the 1960s.3 In recent years systematic reviews have found an important role in health services research, and the growing interest in evidence based approaches to decision making makes it likely that their use will increase. Not everybody accepts that systematic reviews are necessary or desirable, and as one moves further away from the clinical applications of systematic reviews cynicism about their utility grows. Several arguments are commonly used to reject a wider role for systematic reviews, and these arguments are often based on major misconceptions about the history, purpose, methods, and uses of systematic reviews. I have examined eight common myths about …

Health effects of housing improvement: systematic review of intervention studies.

Abstract Objective: To review the evidence on the effects of interventions to improve housing on health. Design: Systematic review of experimental and non-experimental housing intervention studies that measured quantitative health outcomes. Data sources: Studies dating from 1887, in any language or format, identified from clinical, social science, and grey literature databases, personal collections, expert consultation, and reference lists. Main outcome measures: Socioeconomic change and health, illness, and social measures. Results: 18 completed primary intervention studies were identified. 11 studies were prospective, of which six had control groups. Three of the seven retrospective studies used a control group. The interventions included rehousing, refurbishment, and energy efficiency measures. Many studies showed health gains after the intervention, but the small study populations and lack of controlling for confounders limit the generalisability of these findings. Conclusions: The lack of evidence linking housing and health may be attributable to pragmatic difficulties with housing studies as well as the political climate in the United Kingdom. A holistic approach is needed that recognises the multifactorial and complex nature of poor housing and deprivation. Large scale studies that investigate the wider social context of housing interventions are required. What is already known on this topic Many epidemiological studies have described associations between poor housing and health What this study adds 18 studies were reviewed that studied the health effects of housing improvements Most studies found some health gains Small populations and lack of control for confounders limits the generalisability of the findings More large scale, controlled studies of housing interventions are needed to give qualitative and quantitative data

Evidence for public health policy on inequalities: 1: The reality according to policymakers

Objective: To explore with UK and international policy advisors how research evidence influences public health policy making, and how its relevance and utility could be improved, with specific reference to the evidence on the production and reduction of health inequalities. Design, setting, and participants: Qualitative residential workshop involving senior policy advisors with a substantive role in policy development across a range of sectors (mainly public health, but also including education, social welfare, and health services). In four in depth sessions, facilitated by the authors, focused questions were presented to participants. Their responses were then analysed thematically to identify key themes, relating to the availability and utility of existing evidence on health inequalities. Main results: The lack of an equity dimension in much aetiological and evaluative research was highlighted by participants. Much public health research was also felt to have weak underlying theoretical underpinnings. As well as evaluations of the effectiveness and cost-effectiveness of policy and other interventions, they identified a need for predictive research, and for methodological research to further develop methods for assessing the impact on health of clusters of interventions. Conclusions: This study reinforces the view that there is a lack of information on the effectiveness and cost-effectiveness of policies, and it uncovered additional gaps in the health inequalities evidence base. A companion paper discusses researchers’ views of how the production of more relevant public health evidence can be stimulated.