Kathryn Oliver

A systematic review of barriers to and facilitators of the use of evidence by policymakers

BackgroundThe gap between research and practice or policy is often described as a problem. To identify new barriers of and facilitators to the use of evidence by policymakers, and assess the state of research in this area, we updated a systematic review.MethodsSystematic review. We searched online databases including Medline, Embase, SocSci Abstracts, CDS, DARE, Psychlit, Cochrane Library, NHSEED, HTA, PAIS, IBSS (Search dates: July 2000 - September 2012). Studies were included if they were primary research or systematic reviews about factors affecting the use of evidence in policy. Studies were coded to extract data on methods, topic, focus, results and population.Results145 new studies were identified, of which over half were published after 2010. Thirteen systematic reviews were included. Compared with the original review, a much wider range of policy topics was found. Although still primarily in the health field, studies were also drawn from criminal justice, traffic policy, drug policy, and partnership working. The most frequently reported barriers to evidence uptake were poor access to good quality relevant research, and lack of timely research output. The most frequently reported facilitators were collaboration between researchers and policymakers, and improved relationships and skills. There is an increasing amount of research into new models of knowledge transfer, and evaluations of interventions such as knowledge brokerage.ConclusionsTimely access to good quality and relevant research evidence, collaborations with policymakers and relationship- and skills-building with policymakers are reported to be the most important factors in influencing the use of evidence. Although investigations into the use of evidence have spread beyond the health field and into more countries, the main barriers and facilitators remained the same as in the earlier review. Few studies provide clear definitions of policy, evidence or policymaker. Nor are empirical data about policy processes or implementation of policy widely available. It is therefore difficult to describe the role of evidence and other factors influencing policy. Future research and policy priorities should aim to illuminate these concepts and processes, target the factors identified in this review, and consider new methods of overcoming the barriers described.

Attitudes to walking and cycling among children, young people and parents: a systematic review

Background: Promoting walking and cycling as a part of everyday activity is seen as a strategy for tackling obesity and chronic disease. Policy needs to be based on evidence about people’s views of walking and cycling as well as effects of interventions. Methods: Studies of the views of children, young people and parents about walking and cycling were searched for systematically, and a framework analysis applied. The findings were synthesised and compared with the findings of an effectiveness review of interventions for encouraging walking and cycling as an alternative to motorised transport. Results: The synthesis of views described a culture of car use, fed by a fear and dislike of local environments and parental responses that emphasised children’s safety at the expense of developing their independence, despite children expressing responsible attitudes towards transport choices. Comparison with effectiveness literature found that most evaluated interventions targeted only the public’s fear and dislike of local environments. Conclusion: Interventions need to address pedestrian and cyclist safety, perceptions of risk, and parental norms regarding children’s independence.

Patients’ and clinicians’ research priorities

Background  If research addresses the questions of relevance to patients and clinicians, decision‐makers will be better equipped to design and deliver health services which meet their needs. To this end, a number of initiatives have engaged patients and clinicians in setting research agendas. This paper aimed to scope the research literature addressing such efforts.

The views of young children in the UK about obesity, body size, shape and weight: a systematic review

BackgroundThere are high levels of concern about childhood obesity, with obese children being at higher risk of poorer health both in the short and longer terms. Childrens attitudes to, and beliefs about, their bodies have also raised concern. Children themselves have a stake in this debate; their perspectives on this issue can inform the ways in which interventions aim to work.This systematic review of qualitative and quantitative research aimed to explore the views of UK children about the meanings of obesity and body size, shape or weight and their own experiences of these issues.MethodsWe conducted sensitive searches of electronic databases and specialist websites, and contacted experts. We included studies published from the start of 1997 which reported the perspectives of UK children aged 4-11 about obesity or body size, shape or weight, and which described key aspects of their methods. Included studies were coded and quality-assessed by two reviewers independently.Findings were synthesised in two analyses: i) an interpretive synthesis of findings from open-ended questions; and ii) an aggregative synthesis of findings from closed questions. We juxtaposed the findings from the two syntheses. The effect of excluding the lowest quality studies was explored. We also consulted young people to explore the credibility of a subset of findings.ResultsWe included 28 studies. Instead of a focus on health, children emphasised the social impact of body size, describing experiences and awareness of abuse and isolation for children with a greater weight. Body size was seen as under the individuals control and children attributed negative characteristics to overweight people. Children actively assessed their own size; many wished their bodies were different and some were anxious about their shape.Reviewers judged that childrens engagement and participation in discussion had only rarely been supported in the included studies, and few study findings had depth or breadth.ConclusionsInitiatives need to consider the social aspects of obesity, in particular unhelpful beliefs, attitudes and discriminatory behaviours around body size. Researchers and policy-makers should involve children actively and seek their views on appropriate forms of support around this issue.

Randomised controlled trials for policy interventions: a review of reviews and meta-regression.

OBJECTIVES To determine whether randomised controlled trials (RCTs) lead to the same effect size and variance as non-randomised studies (NRSs) of similar policy interventions, and whether these findings can be explained by other factors associated with the interventions or their evaluation. DATA SOURCES Two RCTs were resampled to compare randomised and non-randomised arms. Comparable field trials were identified from a series of health promotion systematic reviews and a systematic review of transition for youths with disabilities. Previous methodological studies were sought from 14 electronic bibliographic databases (Applied Social Sciences Index and Abstracts, Australian Education Index, British Education Index, CareData, Dissertation Abstracts, EconLIT, Educational Resources Information Centre, International Bibliography of the Sociological Sciences, ISI Proceedings: Social Sciences and Humanities, PAIS International, PsycINFO, SIGLE, Social Science Citation Index, Sociological Abstracts) in June and July 2004. These were supplemented by citation searching for key authors, contacting review authors and searching key internet sites. REVIEW METHODS Analyses of previous resampling studies, replication studies, comparable field studies and meta-epidemiology investigated the relationship between randomisation and effect size of policy interventions. New resampling studies and new analyses of comparable field studies and meta-epidemiology were strengthened by testing pre-specified associations supported by carefully argued hypotheses. RESULTS Resampling studies offer no evidence that the absence of randomisation directly influences the effect size of policy interventions in a systematic way. Prior methodological reviews and meta-analyses of existing reviews comparing effects from RCTs and non-randomised controlled trials (nRCTs) suggested that effect sizes from RCTs and nRCTs may indeed differ in some circumstances and that these differences may well be associated with factors confounded with design. No consistent explanations were found for randomisation being associated with changes in effect sizes of policy interventions in field trials. CONCLUSIONS From the resampling studies we have no evidence that the absence of randomisation directly influences the effect size of policy interventions in a systematic way. At the level of individual studies, non-randomised trials may lead to different effect sizes, but this is unpredictable. Many of the examples reviewed and the new analyses in the current study reveal that randomisation is indeed associated with changes in effect sizes of policy interventions in field trials. Despite extensive analysis, we have identified no consistent explanations for these differences. Researchers mounting new evaluations need to avoid, wherever possible, allocation bias. New policy evaluations should adopt randomised designs wherever possible.

Adverse effects of public health interventions: a conceptual framework

Public health interventions may have a range of adverse effects. However, there is limited guidance as to how evaluations should address the possibility of adverse effects. This discussion paper briefly presents a framework for thinking about the potential harms of public health interventions, focusing on the following categories: direct harms; psychological harms; equity harms; group and social harms; and opportunity harms. We conclude that the possibility of adverse effects needs to be taken into account by those implementing and evaluating interventions, and requires a broad perspective on the potential impacts of public health strategies.

Evidence-based policymaking is not like evidence-based medicine, so how far should you go to bridge the divide between evidence and policy?

There is extensive health and public health literature on the ‘evidence-policy gap’, exploring the frustrating experiences of scientists trying to secure a response to the problems and solutions they raise and identifying the need for better evidence to reduce policymaker uncertainty. We offer a new perspective by using policy theory to propose research with greater impact, identifying the need to use persuasion to reduce ambiguity, and to adapt to multi-level policymaking systems.We identify insights from secondary data, namely systematic reviews, critical analysis and policy theories relevant to evidence-based policymaking. The studies are drawn primarily from countries such as the United States, United Kingdom, Canada, Australia and New Zealand. We combine empirical and normative elements to identify the ways in which scientists can, do and could influence policy.We identify two important dilemmas, for scientists and researchers, that arise from our initial advice. First, effective actors combine evidence with manipulative emotional appeals to influence the policy agenda – should scientists do the same, or would the reputational costs outweigh the policy benefits? Second, when adapting to multi-level policymaking, should scientists prioritise ‘evidence-based’ policymaking above other factors? The latter includes governance principles such the ‘co-production’ of policy between local public bodies, interest groups and service users. This process may be based primarily on values and involve actors with no commitment to a hierarchy of evidence.We conclude that successful engagement in ‘evidence-based policymaking’ requires pragmatism, combining scientific evidence with governance principles, and persuasion to translate complex evidence into simple stories. To maximise the use of scientific evidence in health and public health policy, researchers should recognise the tendency of policymakers to base judgements on their beliefs, and shortcuts based on their emotions and familiarity with information; learn ‘where the action is’, and be prepared to engage in long-term strategies to be able to influence policy; and, in both cases, decide how far you are willing to go to persuade policymakers to act and secure a hierarchy of evidence underpinning policy. These are value-driven and political, not just ‘evidence-based’, choices.

Who runs public health? A mixed-methods study combining qualitative and network analyses

BACKGROUND Persistent health inequalities encourage researchers to identify new ways of understanding the policy process. Informal relationships are implicated in finding evidence and making decisions for public health policy (PHP), but few studies use specialized methods to identify key actors in the policy process. METHODS We combined network and qualitative data to identify the most influential individuals in PHP in a UK conurbation and describe their strategies to influence policy. Network data were collected by asking for nominations of powerful and influential people in PHP (n = 152, response rate 80%), and 23 semi-structured interviews were analysed using a framework approach. RESULTS The most influential PHP makers in this conurbation were mid-level managers in the National Health Service and local government, characterized by managerial skills: controlling policy processes through gate keeping key organizations, providing policy content and managing selected experts and executives to lead on policies. Public health professionals and academics are indirectly connected to policy via managers. CONCLUSIONS The most powerful individuals in public health are managers, not usually considered targets for research. As we show, they are highly influential through all stages of the policy process. This study shows the importance of understanding the daily activities of influential policy individuals.

Broadening public participation in systematic reviews: a case example involving young people in two configurative reviews

Background Arguments supporting the involvement of users in research have even more weight when involving the public in systematic reviews of research. We aimed to explore the potential for public involvement in systematic reviews of observational and qualitative studies. Methods Two consultative workshops were carried out with a group of young people (YP) aged 12–17 years to examine two ongoing reviews about obesity: one about childrens views and one on the link between obesity and educational attainment. YP were invited to comment on the credibility of themes, to propose elements of interventions, to suggest links between educational attainment and obesity and to comment on their plausibility. Results Researchers had more confidence in review findings, after checking that themes identified as important by YP were emphasised appropriately. Researchers were able to use factors linking obesity and attainment identified as important by YP to identify limitations in the scope of extant research. Conclusion Consultative workshops helped researchers draw on the perspectives of YP when interpreting and reflecting upon two systematic reviews. Involving users in judging synthesis credibility and identifying concepts was easier than involving them in interpreting findings. Involvement activities for reviews should be designed with review stage, purpose and group in mind.

Improving the identification and management of chronic kidney disease in primary care: lessons from a staged improvement collaborative

Quality problem Undiagnosed chronic kidney disease (CKD) contributes to a high cost and care burden in secondary care. Uptake of evidence-based guidelines in primary care is inconsistent, resulting in variation in the detection and management of CKD. Initial assessment Routinely collected general practice data in one UK region suggested a CKD prevalence of 4.1%, compared with an estimated national prevalence of 8.5%. Of patients on CKD registers, ∼30% were estimated to have suboptimal management according to Public Health Observatory analyses. Choice of solution An evidence-based framework for implementation was developed. This informed the design of an improvement collaborative to work with a sample of 30 general practices. Implementation A two-phase collaborative was implemented between September 2009 and March 2012. Key elements of the intervention included learning events, improvement targets, Plan-Do-Study-Act cycles, benchmarking of audit data, facilitator support and staff time reimbursement. Evaluation Outcomes were evaluated against two indicators: number of patients with CKD on practice registers; percentage of patients achieving evidence-based blood pressure (BP) targets, as a marker for CKD care. In Phase 1, recorded prevalence of CKD in collaborative practices increased ∼2-fold more than that in comparator local practices; in Phase 2, this increased to 4-fold, indicating improved case identification. Management of BP according to guideline recommendations also improved. Lessons learned An improvement collaborative with tailored facilitation support appears to promote the uptake of evidence-based guidance on the identification and management of CKD in primary care. A controlled evaluation study is needed to rigorously evaluate the impact of this promising improvement intervention.