Eleanor Jackson-Bowers

The feasibility of determining the impact of primary health care research projects using the payback framework

BackgroundPrimary health care research is under pressure to be accountable to funders in terms of benefits for practice and policy. However, methods to assess the impact of primary health care research must be appropriate to use with the diverse topics, settings and approaches of this sector. This project explored the feasibility of using the Buxton and Hanney Payback Framework to determine the impact of a stratified random sample (n = 4) of competitively funded, primary health care research projects.MethodsThe project conducted telephone interviews based on the Payback Framework with leaders of the research teams and nominated users of their research, used bibliometric methods for assessing impact through publication outputs and obtained documentary evidence of impact where possible. The purpose was to determine the effectiveness of the data collection methods and the applicability of the Payback Framework, and any other issues which arose around the assessment of impact of primary health care research.Results and discussionThe thirteen interviews were resource intensive to organise conduct and analyse but provided better information about impact than bibliometric analysis or documentary analysis. Bibliometric analysis of the papers published from the four projects was hampered by the inclusion of only one of the journals in major citation indexes. Document analysis provided more evidence of dissemination than of impact.The payback framework and logic model were a sound basis for assessing impact. Chief investigators and nominated users of research provided substantial information relevant to the impact categories closest to their spheres of influence and awareness, but less about the impact their research had on the wider health sector, population health or economic benefits. An additional category of impact emerged from the interviews, that of strengthening research networks which could enhance the impact of later work. The framework provided rich information about the pathways to impact, better understanding of which may enhance impact.ConclusionIt is feasible to use the Buxton and Hanney Payback framework and logic model to determine the proximal impacts of primary health care research. Though resource intensive, telephone interviews of chief investigators and nominated users provided rich information.

Pathways to research impact in primary healthcare: What do Australian primary healthcare researchers believe works best to facilitate the use of their research findings?

BackgroundPrimary healthcare researchers are under increasing pressure to demonstrate measurable and lasting improvement in clinical practice and healthcare policy as a result of their work. It is therefore important to understand the effectiveness of the research dissemination strategies used. The aim of this paper is to describe the pathways for research impact that have been achieved across several government-funded primary healthcare projects, and the effectiveness of these methods as perceived by their Chief Investigators.MethodsThe project used an online survey to collect information about government-funded primary healthcare research projects. Chief Investigators were asked how they disseminated their findings and how this achieved impact in policy and practice. They were also asked to express their beliefs regarding the most effective means of achieving research impact and describe how this occurred.ResultsChief Investigators of 17 projects indicated that a number of dissemination strategies were used but that professional networks were the most effective means of promoting uptake of their research findings. Utilisation of research findings for clinical practice was most likely to occur in organisations or among individual practitioners who were most closely associated with the research team, or when research findings were included in educational programmes involving clinical practice. Uptake of both policy- and practice-related research was deemed most successful if intermediary organisations such as formal professional networks were engaged in the research. Successful primary healthcare researchers had developed critical relationships with intermediary organisations within primary healthcare before the initiation of the research and had also involved them in the design. The scale of research impact was influenced by the current policy environment, the type and significance of the results, and the endorsement (or lack thereof) of professional bodies.ConclusionsChief Investigators believed that networks were the most effective means of research dissemination. Researchers who were embedded in professional, clinical or policy-focussed intermediary organisations, or had developed partnerships with clinical services, which had a vested interest in the research findings, were more able to describe a direct impact of their research. This suggests that development of these relationships and engagement of these stakeholders by primary healthcare researchers is a vital step for optimal research utilisation in the primary healthcare setting.