Lucie Rychetnik

Criteria for evaluating evidence on public health interventions

Public health interventions tend to be complex, programmatic, and context dependent. The evidence for their effectiveness must be sufficiently comprehensive to encompass that complexity. This paper asks whether and to what extent evaluative research on public health interventions can be adequately appraised by applying well established criteria for judging the quality of evidence in clinical practice. It is adduced that these criteria are useful in evaluating some aspects of evidence. However, there are other important aspects of evidence on public health interventions that are not covered by the established criteria. The evaluation of evidence must distinguish between the fidelity of the evaluation process in detecting the success or failure of an intervention, and the success or failure of the intervention itself. Moreover, if an intervention is unsuccessful, the evidence should help to determine whether the intervention was inherently faulty (that is, failure of intervention concept or theory), or just badly delivered (failure of implementation). Furthermore, proper interpretation of the evidence depends upon the availability of descriptive information on the intervention and its context, so that the transferability of the evidence can be determined. Study design alone is an inadequate marker of evidence quality in public health intervention evaluation.

A glossary for evidence based public health

This glossary seeks to define and explain some of the main concepts underpinning evidence based public health. It draws on the published literature, experience gained over several years analysis of the topic, and discussions with public health colleagues, including researchers, practitioners, policy makers, and students.

Increasing the use of evidence in health policy: practice and views of policy makers and researchers

BackgroundBetter communication is often suggested as fundamental to increasing the use of research evidence in policy, but little is known about how researchers and policy makers work together or about barriers to exchange. This study explored the views and practice of policy makers and researchers regarding the use of evidence in policy, including: (i) current use of research to inform policy; (ii) dissemination of and access to research findings for policy; (iii) communication and exchange between researchers and policy makers; and (iv) incentives for increasing the use of research in policy.MethodsSeparate but similar interview schedules were developed for policy makers and researchers. Senior policy makers from NSW Health and senior researchers from public health and health service research groups in NSW were invited to participate. Consenting participants were interviewed by an independent research company.ResultsThirty eight policy makers (79% response rate) and 41 researchers (82% response rate) completed interviews. Policy makers reported rarely using research to inform policy agendas or to evaluate the impact of policy; research was used more commonly to inform policy content. Most researchers reported that their research had informed local policy, mainly by increasing awareness of an issue. Policy makers reported difficulty in accessing useful research syntheses, and only a third of researchers reported developing targeted strategies to inform policy makers of their findings. Both policy makers and researchers wanted more exchange and saw this as important for increasing the use of research evidence in policy; however, both groups reported a high level of involvement by policy makers in research.ConclusionPolicy makers and researchers recognise the potential of research to contribute to policy and are making significant attempts to integrate research into the policy process. These findings suggest four strategies to assist in increasing the use of research in policy: making research findings more accessible to policy makers; increasing opportunities for interaction between policy makers and researchers; addressing structural barriers such as research receptivity in policy agencies and a lack of incentives for academics to link with policy; and increasing the relevance of research to policy.

Reduction of diabetes risk in routine clinical practice: are physical activity and nutrition interventions feasible and are the outcomes from reference trials replicable? A systematic review and meta-analysis

BackgroundThe clinical effectiveness of intensive lifestyle interventions in preventing or delaying diabetes in people at high risk has been established from randomised trials of structured, intensive interventions conducted in several countries over the past two decades. The challenge is to translate them into routine clinical settings. The objective of this review is to determine whether lifestyle interventions delivered to high-risk adult patients in routine clinical care settings are feasible and effective in achieving reductions in risk factors for diabetes.MethodsData sources: MEDLINE (PubMed), EMBASE, CINAHL, The Cochrane Library, Google Scholar, and grey literature were searched for English-language articles published from January 1990 to August 2009. The reference lists of all articles collected were checked to ensure that no relevant suitable studies were missed. Study selection: We included RCTs, before/after evaluations, cohort studies with or without a control group and interrupted time series analyses of lifestyle interventions with the stated aim of diabetes risk reduction or diabetes prevention, conducted in routine clinical settings and delivered by healthcare providers such as family physicians, practice nurses, allied health personnel, or other healthcare staff associated with a health service. Outcomes of interest were weight loss, reduction in waist circumference, improvement of impaired fasting glucose or oral glucose tolerance test (OGTT) results, improvements in fat and fibre intakes, increased level of engagement in physical activity and reduction in diabetes incidence.ResultsTwelve from 41 potentially relevant studies were included in the review. Four studies were suitable for meta-analysis. A significant positive effect of the interventions on weight was reported by all study types. The meta-analysis showed that lifestyle interventions achieved weight and waist circumference reductions after one year. However, no clear effects on biochemical or clinical parameters were observed, possibly due to short follow-up periods or lack of power of the studies meta-analysed. Changes in dietary parameters or physical activity were generally not reported. Most studies assessing feasibility were supportive of implementation of lifestyle interventions in routine clinical care.ConclusionLifestyle interventions for patients at high risk of diabetes, delivered by a variety of healthcare providers in routine clinical settings, are feasible but appear to be of limited clinical benefit one year after intervention. Despite convincing evidence from structured intensive trials, this systematic review showed that translation into routine practice has less effect on diabetes risk reduction.

Evidence, Ethics, and Values: A Framework for Health Promotion

We propose a new approach to guide health promotion practice. Health promotion should draw on 2 related systems of reasoning: an evidential system and an ethical system. Further, there are concepts, values, and procedures inherent in both health promotion evidence and ethics, and these should be made explicit. We illustrate our approach with the exemplar of intervention in weight, and use a specific mass-media campaign to show the real-world dangers of intervening with insufficient attention to ethics and evidence. Both researchers and health promotion practitioners should work to build the capacities required for evidential and ethical deliberation in the health promotion profession.

Translating research for evidence-based public health: key concepts and future directions

Applying research to guide evidence-based practice is an ongoing and significant challenge for public health. Developments in the emerging field of ‘translation’ have focused on different aspects of the problem, resulting in competing frameworks and terminology. In this paper the scope of ‘translation’ in public health is defined, and four related but conceptually different ‘translation processes’ that support evidence-based practice are outlined: (1) reviewing the transferability of evidence to new settings, (2) translation research, (3) knowledge translation, and (4) knowledge translation research. Finally, an integrated framework is presented to illustrate the relationship between these domains, and priority areas for further development and empirical research are identified.

The Australian experience of nurses' preparedness for evidence-based practice

AIM This study aimed to determine current knowledge and attitudes towards evidence-based practice (EBP) among pre- and post-registration nurses in New South Wales (NSW), Australia. BACKGROUND Educational and clinical strategies for EBP in nursing assume a readiness to interpret and integrate evidence into clinical care despite continued reports of low levels of understanding and skill in this area. METHOD Perceptions of EBP were examined through a self-complete, anonymous postal survey distributed to 677 (post-registration) clinical nurses and to 1134 final year (pre-registration) nursing students during 2002 and 2003. RESULTS A completed survey was returned by 126 post-registration and 257 final year nursing students (combined 21% response rate). Both pre- and post-registration nurses had a welcoming attitude towards EBP. Pre-registration nurses expressed more confidence in their EBP skills but self-rated knowledge and skill were low to moderate in both groups. CONCLUSION Nurses in Australia are clearly supportive of EBP but it is incorrect to assume that even recent graduates have a level of knowledge and skill that is sufficient to permit direct engagement in evidence implementation. IMPLICATIONS FOR NURSING MANAGEMENT Among a range of clinical supports, nurse managers and leaders can contribute to evidence-based health care by understanding the EBP knowledge and skills of their workforce and demanding a more practical approach in nursing education towards evidence-based guidelines and summaries appropriate to the clinical context.

Psychosocial aspects of post-treatment follow-up for stage I/II melanoma: a systematic review of the literature.

Patients treated for melanoma are advised to have lifelong full body skin examinations. Extended intervals between examinations have been proposed, but although this may be clinically effective, psychosocial aspects of follow‐up are not well understood. This systematic review summarised patient and clinician preferences, experiences and adherence with recommended follow‐up of stage I/II melanoma.

Policy and practice impacts of applied research: a case study analysis of the New South Wales Health Promotion Demonstration Research Grants Scheme 2000–2006

BackgroundIntervention research provides important information regarding feasible and effective interventions for health policy makers, but few empirical studies have explored the mechanisms by which these studies influence policy and practice. This study provides an exploratory case series analysis of the policy, practice and other related impacts of the 15 research projects funded through the New South Wales Health Promotion Demonstration Research Grants Scheme during the period 2000 to 2006, and explored the factors mediating impacts.MethodsData collection included semi-structured interviews with the chief investigators (n = 17) and end-users (n = 29) of each of the 15 projects to explore if, how and under what circumstances the findings had been used, as well as bibliometric analysis and verification using documentary evidence. Data analysis involved thematic coding of interview data and triangulation with other data sources to produce case summaries of impacts for each project. Case summaries were then individually assessed against four impact criteria and discussed at a verification panel meeting where final group assessments of the impact of research projects were made and key influences of research impact identified.ResultsFunded projects had variable impacts on policy and practice. Project findings were used for agenda setting (raising awareness of issues), identifying areas and target groups for interventions, informing new policies, and supporting and justifying existing policies and programs across sectors. Reported factors influencing the use of findings were: i) nature of the intervention; ii) leadership and champions; iii) research quality; iv) effective partnerships; v) dissemination strategies used; and, vi) contextual factors.ConclusionsThe case series analysis provides new insights into how and under what circumstances intervention research is used to influence real world policy and practice. The findings highlight that intervention research projects can achieve the greatest policy and practice impacts if they address proximal needs of the policy context by engaging end-users from the inception of projects and utilizing existing policy networks and structures, and using a range of strategies to disseminate findings that go beond traditional peer review publications.

A community jury on PSA screening: What do well-informed men want the government to do about prostate cancer screening - A qualitative analysis

Objective Cancer screening policies and programmes should take account of public values and concerns. This study sought to determine the priorities, values and concerns of men who were ‘fully informed’ about the benefits and harms of prostate-specific antigen (PSA) screening; and empirically examine the value of a community jury in eliciting public values on PSA screening. Setting Community jury was convened on the Gold Coast, Queensland (Australia) to consider PSA screening benefits and harms, and whether government campaigns on PSA screening should be conducted. Participants 27 men (volunteers) aged 50–70 with no personal history of prostate cancer and willing to attend jury 6–7 April 2013: 12 were randomly allocated to jury (11 attended). Outcome measures A qualitative analysis was conducted of the jury deliberations (audio-recorded and transcribed) to elicit the jurys views and recommendations. A survey determined the impact of the jury process on participants’ individual testing decisions compared with control group. Results The jury concluded governments should not invest in programmes focused on PSA screening directed at the public because the PSA test did not offer sufficient reassurance or benefit and could raise unnecessary alarm. It recommended an alternative programme to support general practitioners to provide patients with better quality and more consistent information about PSA screening. After the jury, participants were less likely to be tested in the future compared with the controls, but around half said they would still consider doing so. Conclusions The jurys unanimous verdict about government programmes was notable in the light of their divergent views on whether or not they would be screened themselves in the future. Community juries provide valuable insights into the priorities and concerns of men weighing up the benefits and harms of PSA screening. It will be important to assess the degree to which the findings are generalisable to other settings.