Robyn Newson

Increasing the scale and adoption of population health interventions: experiences and perspectives of policy makers, practitioners, and researchers.

BackgroundDecisions to scale up population health interventions from small projects to wider state or national implementation is fundamental to maximising population-wide health improvements. The objectives of this study were to examine: i) how decisions to scale up interventions are currently made in practice; ii) the role that evidence plays in informing decisions to scale up interventions; and iii) the role policy makers, practitioners, and researchers play in this process.MethodsInterviews with an expert panel of senior Australian and international public health policy-makers (n = 7), practitioners (n = 7), and researchers (n = 7) were conducted in May 2013 with a participation rate of 84%.ResultsScaling up decisions were generally made through iterative processes and led by policy makers and/or practitioners, but ultimately approved by political leaders and/or senior executives of funding agencies. Research evidence formed a component of the overall set of information used in decision-making, but its contribution was limited by the paucity of relevant intervention effectiveness research, and data on costs and cost effectiveness. Policy makers, practitioners/service managers, and researchers had different, but complementary roles to play in the process of scaling up interventions.ConclusionsThis analysis articulates the processes of how decisions to scale up interventions are made, the roles of evidence, and contribution of different professional groups. More intervention research that includes data on the effectiveness, reach, and costs of operating at scale and key service delivery issues (including acceptability and fit of interventions and delivery models) should be sought as this has the potential to substantially advance the relevance and ultimately usability of research evidence for scaling up population health action.

Policy and practice impacts of applied research: a case study analysis of the New South Wales Health Promotion Demonstration Research Grants Scheme 2000–2006

BackgroundIntervention research provides important information regarding feasible and effective interventions for health policy makers, but few empirical studies have explored the mechanisms by which these studies influence policy and practice. This study provides an exploratory case series analysis of the policy, practice and other related impacts of the 15 research projects funded through the New South Wales Health Promotion Demonstration Research Grants Scheme during the period 2000 to 2006, and explored the factors mediating impacts.MethodsData collection included semi-structured interviews with the chief investigators (n = 17) and end-users (n = 29) of each of the 15 projects to explore if, how and under what circumstances the findings had been used, as well as bibliometric analysis and verification using documentary evidence. Data analysis involved thematic coding of interview data and triangulation with other data sources to produce case summaries of impacts for each project. Case summaries were then individually assessed against four impact criteria and discussed at a verification panel meeting where final group assessments of the impact of research projects were made and key influences of research impact identified.ResultsFunded projects had variable impacts on policy and practice. Project findings were used for agenda setting (raising awareness of issues), identifying areas and target groups for interventions, informing new policies, and supporting and justifying existing policies and programs across sectors. Reported factors influencing the use of findings were: i) nature of the intervention; ii) leadership and champions; iii) research quality; iv) effective partnerships; v) dissemination strategies used; and, vi) contextual factors.ConclusionsThe case series analysis provides new insights into how and under what circumstances intervention research is used to influence real world policy and practice. The findings highlight that intervention research projects can achieve the greatest policy and practice impacts if they address proximal needs of the policy context by engaging end-users from the inception of projects and utilizing existing policy networks and structures, and using a range of strategies to disseminate findings that go beond traditional peer review publications.

A guide to scaling up population health interventions

INTRODUCTION In Australia, adult immunisation coverage is primarily monitored via periodic telephone surveys that rely on self-reported immunisation status. All Australian residents are eligible for Medicare, so we examined the feasibility of using immunisation-specific Medicare Benefits Schedule (MBS) item numbers to monitor and estimate adult influenza immunisation coverage. METHODS Baseline questionnaire data from 267 129 participants from the 45 and Up Study, a prospective cohort study, were linked to data containing information on individual MBS immunisation-specific items from 2006 to 2011. Temporal trends in recording of these items were examined. Self-reported influenza immunisation status obtained from a follow-up questionnaire from 27 036 participants was then compared with the MBS immunisation records. RESULTS From 2006 to 2011, the pattern of MBS immunisation claims was consistent with self-reported influenza immunisation trends, with annual peaks occurring from March to May. There was fair agreement between MBS immunisation records and self-reported influenza immunisation: 48.8% (95% CI 47.1, 50.4) of participants who self-reported influenza immunisation had a matching MBS record, and 79.6% (95% CI 78.8, 80.4) who reported never having influenza immunisation had no matching MBS record. However, compared with self-reported influenza vaccination for people aged ≥65 years from the 2009 Adult Immunisation Telephone Survey (74.6%), the proportion of participants aged >65 years with an MBS immunisation record was low, with an annual median of 39.3% (range 28.3%-62.1%). CONCLUSION MBS immunisation item data are readily available and may be useful to monitor trends in adult influenza immunisation, but they are likely to substantially underestimate coverage. Other approaches, such as capture of general practitioner-delivered influenza vaccine doses or a whole-of-life immunisation register, are needed to comprehensively monitor and estimate adult immunisation coverage for influenza and other recommended adult vaccines.

Utilization of a population health survey in policy and practice: a case study

BackgroundThere is growing interest by funding bodies and researchers in assessing the impact of research on real world policy and practice. Population health monitoring surveys provide an important source of data on the prevalence and patterns of health problems, but few empirical studies have explored if and how such data is used to influence policy or practice decisions. Here we provide a case study analysis of how the findings from an Australian population monitoring survey series of children’s weight and weight-related behaviors (Schools Physical Activity and Nutrition Survey (SPANS)) have been used, and the key facilitators and barriers to their utilization.MethodsData collection included semi-structured interviews with the chief investigators (n = 3) and end-users (n = 9) of SPANS data to explore if, how and under what circumstances the survey findings had been used, bibliometric analysis and verification using documentary evidence. Data analysis involved thematic coding of interview data and triangulation with other data sources to produce case summaries of policy and practice impacts for each of the three survey years (1997, 2004, 2010). Case summaries were then reviewed and discussed by the authors to distil key themes on if, how and why the SPANS findings had been used to guide policy and practice.ResultsWe found that the survey findings were used for agenda setting (raising awareness of issues), identifying areas and target groups for interventions, informing new policies, and supporting and justifying existing policies and programs across a range of sectors. Reported factors influencing use of the findings were: i) the perceived credibility of survey findings; ii) dissemination strategies used; and, iii) a range of contextual factors.ConclusionsUsing a novel approach, our case study provides important new insights into how and under what circumstances population health monitoring data can be used to influence real world policy and practice. The findings highlight the importance of population monitoring programs being conducted by independent credible agencies, researchers engaging end-users from the inception of survey programs and utilizing existing policy networks and structures, and using a range of strategies to disseminate the findings that go beyond traditional peer review publications.

Tracking funded health intervention research

Objective: To describe the research publication outputs from intervention research funded by Australias National Health and Medical Research Council (NHMRC).

A mixed methods study of the factors that influence whether intervention research has policy and practice impacts: perceptions of Australian researchers.

Objectives To investigate researchers’ perceptions about the factors that influenced the policy and practice impacts (or lack of impact) of one of their own funded intervention research studies. Design Mixed method, cross-sectional study. Setting Intervention research conducted in Australia and funded by Australias National Health and Medical Research Council between 2003 and 2007. Participants The chief investigators from 50 funded intervention research studies were interviewed to determine if their study had achieved policy and practice impacts, how and why these impacts had (or had not) occurred and the approach to dissemination they had employed. Results We found that statistically significant intervention effects and publication of results influenced whether there were policy and practice impacts, along with factors related to the nature of the intervention itself, the researchers’ experience and connections, their dissemination and translation efforts, and the postresearch context. Conclusions This study indicates that sophisticated approaches to intervention development, dissemination actions and translational efforts are actually widespread among experienced researches, and can achieve policy and practice impacts. However, it was the links between the intervention results, further dissemination actions by researchers and a variety of postresearch contextual factors that ultimately determined whether a study had policy and practice impacts. Given the complicated interplay between the various factors, there appears to be no simple formula for determining which intervention studies should be funded in order to achieve optimal policy and practice impacts.

Does citation matter? Research citation in policy documents as an indicator of research impact – an Australian obesity policy case-study

BackgroundCitation of research in policy documents has been suggested as an indicator of the potential longer-term impacts of research. We investigated the use of research citations in childhood obesity prevention policy documents from New South Wales (NSW), Australia, considering the feasibility and value of using research citation as a proxy measure of research impact.MethodsWe examined childhood obesity policy documents produced between 2000 and 2015, extracting childhood obesity-related references and coding these according to reference type, geographical origin and type of research. A content analysis of the policy documents examined where and how research was cited in the documents and the context of citation for individual research publications.ResultsOver a quarter (28%) of the policy documents (n = 86) were not publicly available, almost two-thirds (63%) contained references, half (47%) cited obesity-related research and over a third (41%) of those containing references used unorthodox referencing styles, making reference extraction laborious. No patterns, in terms of the types of documents more likely to cite research, were observed and the number of obesity research publications cited per document was highly variable. In total, 263 peer-reviewed and 94 non-peer-reviewed obesity research publications were cited. Research was most commonly cited to support a policy argument or choice of solution. However, it was not always possible to determine how or why individual publications were cited or whether the cited research itself had influenced the policy process. Content analysis identified circumstances where research was mentioned or considered, but not directly cited.ConclusionsCitation of research in policy documents in this case did not always provide evidence that the cited research had influenced the policy process, only that it was accessible and relevant to the content of the policy document. Research citation across these public health policy documents varied greatly and is unlikely to be an accurate reflection of actual research use by the policy agencies involved. The links between citation and impact may be more easily drawn in specific policy areas or types of documents (e.g. clinical guidelines), where research appraisal feeds directly into policy recommendations.

Looking both ways: a review of methods for assessing research impacts on policy and the policy utilisation of research

BackgroundMeasuring the policy and practice impacts of research is becoming increasingly important. Policy impacts can be measured from two directions – tracing forward from research and tracing backwards from a policy outcome. In this review, we compare these approaches and document the characteristics of studies assessing research impacts on policy and the policy utilisation of research.MethodsKeyword searches of electronic databases were conducted in December 2016. Included studies were published between 1995 and 2016 in English and reported methods and findings of studies measuring policy impacts of specified health research, or research use in relation to a specified health policy outcome, and reviews reporting methods of research impact assessment. Using an iterative data extraction process, we developed a framework to define the key elements of empirical studies (assessment reason, assessment direction, assessment starting point, unit of analysis, assessment methods, assessment endpoint and outcomes assessed) and then documented the characteristics of included empirical studies according to this framework.ResultsWe identified 144 empirical studies and 19 literature reviews. Empirical studies were derived from two parallel streams of research of equal size, which we termed ‘research impact assessments’ and ‘research use assessments’. Both streams provided insights about the influence of research on policy and utilised similar assessment methods, but approached measurement from opposite directions. Research impact assessments predominantly utilised forward tracing approaches while the converse was true for research use assessments. Within each stream, assessments focussed on narrow or broader research/policy units of analysis as the starting point for assessment, each with associated strengths and limitations. The two streams differed in terms of their relative focus on the contributions made by specific research (research impact assessments) versus research more generally (research use assessments) and the emphasis placed on research and the activities of researchers in comparison to other factors and actors as influencers of change.ConclusionsThe Framework presented in this paper provides a mechanism for comparing studies within this broad field of research enquiry. Forward and backward tracing approaches, and their different ways of ‘looking’, tell a different story of research-based policy change. Combining approaches may provide the best way forward in terms of linking outcomes to specific research, as well as providing a realistic picture of research influence.