Pam Carter

How to study improvement interventions: a brief overview of possible study types

Improvement (defined broadly as purposive efforts to secure positive change) has become an increasingly important activity and field of inquiry within healthcare. This article offers an overview of possible methods for the study of improvement interventions. The choice of available designs is wide, but debates continue about how far improvement efforts can be simultaneously practical (aimed at producing change) and scientific (aimed at producing new knowledge), and whether the distinction between the practical and the scientific is a real and useful one. Quality improvement projects tend to be applied and, in some senses, self-evaluating. They are not necessarily directed at generating new knowledge, but reports of such projects if well conducted and cautious in their inferences may be of considerable value. They can be distinguished heuristically from research studies, which are motivated by and set out explicitly to test a hypothesis, or otherwise generate new knowledge, and from formal evaluations of improvement projects. We discuss variants of trial designs, quasi-experimental designs, systematic reviews, programme evaluations, process evaluations, qualitative studies, and economic evaluations. We note that designs that are better suited to the evaluation of clearly defined and static interventions may be adopted without giving sufficient attention to the challenges associated with the dynamic nature of improvement interventions and their interactions with contextual factors. Reconciling pragmatism and research rigour is highly desirable in the study of improvement. Trade-offs need to be made wisely, taking into account the objectives involved and inferences to be made.

The social licence for research: why care.data ran into trouble

In this article we draw on the concept of a social licence to explain public concern at the introduction of care.data, a recent English initiative designed to extract data from primary care medical records for commissioning and other purposes, including research. The concept of a social licence describes how the expectations of society regarding some activities may go beyond compliance with the requirements of formal regulation; those who do not fulfil the conditions for the social licence (even if formally compliant) may experience ongoing challenge and contestation. Previous work suggests that peoples cooperation with specific research studies depends on their perceptions that their participation is voluntary and is governed by values of reciprocity, non-exploitation and service of the public good. When these conditions are not seen to obtain, threats to the social licence for research may emerge. We propose that care.data failed to adequately secure a social licence because of: (i) defects in the warrants of trust provided for care.data, (ii) the implied rupture in the traditional role, expectations and duties of general practitioners, and (iii) uncertainty about the status of care.data as a public good. The concept of a social licence may be useful in explaining the specifics of care.data, and also in reinforcing the more general lesson for policy-makers that legal authority does not necessarily command social legitimacy.

Patient and public involvement in primary care research - an example of ensuring its sustainability

BackgroundThe international literature on patient and public involvement (PPI) in research covers a wide range of issues, including active lay involvement throughout the research cycle; roles that patients/public can play; assessing impact of PPI and recommendations for good PPI practice. One area of investigation that is less developed is the sustainability and impact of PPI beyond involvement in time-limited research projects.MethodsThis paper focuses on the issues of sustainability, the importance of institutional leadership and the creation of a robust infrastructure in order to achieve long-term and wide-ranging PPI in research strategy and programmes.ResultsWe use the case of a Primary Care Research Centre to provide a historical account of the evolution of PPI in the Centre and identified a number of key conceptual issues regarding infrastructure, resource allocation, working methods, roles and relationships.ConclusionsThe paper concludes about the more general applicability of the Centre’s model for the long-term sustainability of PPI in research.

How collaborative are quality improvement collaboratives: a qualitative study in stroke care

BackgroundQuality improvement collaboratives (QICs) continue to be widely used, yet evidence for their effectiveness is equivocal. We sought to explain what happened in Stroke 90:10, a QIC designed to improve stroke care in 24 hospitals in the North West of England. Our study drew in part on the literature on collective action and inter-organizational collaboration. This literature has been relatively neglected in evaluations of QICs, even though they are founded on principles of co-operation and sharing.MethodsWe interviewed 32 professionals in hospitals that participated in Stroke 90:10, conducted a focus group with the QIC faculty team, and reviewed purposively sampled documents including reports and newsletters. Analysis was based on a modified form of Framework Analysis, combining sensitizing constructs derived from the literature and new, empirically derived thematic categories.ResultsImprovements in stroke care were attributed to QIC participation by many professionals. They described how the QIC fostered a sense of community and increased attention to stroke care within their organizations. However, participants’ experiences of the QIC varied. Starting positions were different; some organizations were achieving higher levels of performance than others before the QIC began, and some had more pre-existing experience of quality improvement methods. Some participants had more to learn, others more to teach. Some evidence of free-riding was found. Benchmarking improvement was variously experienced as friendly rivalry or as time-consuming and stressful. Participants’ competitive desire to demonstrate success sometimes conflicted with collaborative aims; some experienced competing organizational pressures or saw the QIC as duplication of effort. Experiences of inter-organizational collaboration were influenced by variations in intra-organizational support.ConclusionsCollaboration is not the only mode of behavior likely to occur within a QIC. Our study revealed a mixed picture of collaboration, free-riding and competition. QICs should learn from work on the challenges of collective action; set realistic goals; account for context; ensure sufficient time and resources are made available; and carefully manage the collaborative to mitigate the risks of collaborative inertia and unhelpful competitive or anti-cooperative behaviors. Individual organizations should assess the costs and benefits of collaboration as a means of attaining quality improvement.

Mobilising the experiential knowledge of clinicians, patients and carers for applied health-care research

This article demonstrates the benefits of combining various types of knowledge for applied health research. Funding is available for health research despite these being ‘austere times’ for public services and international policy shifts recognise the role that patients, carers and the public can play in research. In England the National Institute for Health Research, Research Design Service (RDS) was created to ensure that the experiential knowledge of clinicians working in the National Health Service is informed by methodological expertise to achieve relevant research outcomes. The RDS also facilitates patient and public involvement in research, framed as ‘PPI’. This raises the question of how PPI impacts on research design and funding and which patients or members of the public should be involved in which aspects of research. To answer these questions we present case studies that draw on the expertise of academics, clinicians, patients and the public in applied health research. These cases demonstrate that where patients with direct experience of the condition that is to be studied are actively involved as advisers early on in applied health research, this can enhance the likelihood of successful funding applications, ethical aspects of research and the relevance of questionnaires and interventions to patients. For comparative purposes, we give an example of an unsuccessful research proposal. We contribute to theoretical development through refining the conceptualisation of PPI by unpicking the different roles that members of the public play as lay people, distinguishing this from the specific expertise that comes from direct experience of being a service user, carer or patient. We conclude that different types of knowledge are required for applied health research: methodological expertise, practice-based expertise, and the experiential expertise of patients or carers. While there are no guarantees, the scrutiny function performed by lay involvement in research funding panels can challenge the balance of power.

Sustaining patient and public involvement in research: A case study of a research centre

The literature on patient and public involvement (PPI) in research covers a wide range of topics. However, one area of investigation that appears under developed is the sustainability and impact of PPI beyond involvement in time-limited research projects. This paper presents a case study of PPI development in one primary care research centre in England, and its approach to making this sustainable using documentary sources and material from a formal evaluation. We provide narrative accounts of the set-up, operation and main processes of PPI, and its perceived impact. PPI requires a long-term perspective with participation and trust growing over time, and both users and researchers learning what approaches work best. PPI is a complex interplay of clarity of purpose, defined roles and relationships, organised support (paid PPI staff) and a well-funded infrastructure. ‘Soft systems’ are equally important such as flexible and informal approaches to meetings, adapting timetables and environments to meet the needs of lay members and to create spaces for relationships to develop between researchers and lay members that are based on mutual trust and respect. This case study highlights that the right combination of ethos, flexible working practices, leadership, and secure funding goes a long way to embedding PPI beyond ad hoc involvement. This allows PPI in research to be integrated in the infrastructure and sustainable.

Public priorities for joint pain research: results from a general population survey

Objective. We aimed to identify the priorities for joint pain research from a large general population survey and identify characteristics associated with these priorities. Methods. A question about research priorities was developed in collaboration with the Arthritis Research UK Primary Care Centre’s Research Users’ Group. The question was embedded in a postal survey to an existing cohort of adults with self-reported joint pain, aged ≥56 years, in North Staffordshire. Respondents were asked to rank their top three priorities for research. Factor mixture modelling was used to determine subgroups of priorities. Results. In all, 1756 (88%) people responded to the survey. Of these, 1356 (77%) gave three priorities for research. Keeping active was rated the top priority by 38%, followed by research around joint replacement (9%) and diet/weight loss (9%). Two clusters of people were identified: 62% preferred lifestyle/self-management topics (e.g. keeping active, weight loss) and 38% preferred medical intervention topics (e.g. joint replacement, tablets). Those who preferred the medical options tended to be older and have hip or foot pain. Conclusion. This study has provided population data on priorities for joint pain research expressed by a large cohort of older people who report joint pain. The most popular topics for research were linked to lifestyle and self-management opportunities. Pharmaceutical and invasive interventions, despite being common topics of research, are of less importance to these respondents than non-medical topics. Specific research questions will be generated from this study with collaboration of the patient’s group.

Governing welfare reform symbolically: evidence based or iconic policy?

This article reports findings from an ethnographic study of welfare reform in which the discursive negotiation of policy implementation at the local level was key to understanding the phenomenon of unintended consequences. Using policy give-aways or ‘freebies’ as a primary source of data, the article demonstrates how, despite the rhetoric of evidence based policy and practice, the meanings of policy are open to interpretation. The artifacts brand, materialize, reify, and attempt to discursively govern a range of somewhat abstract or paradoxical policy ideas in the course of implementing welfare reform. Whilst at first sight these hyper-visible manifestations of public policy may appear to be ephemeral data, on closer examination they turn out to be highly significant. They symbolize the commodification of public services, the fluid nature of policy, the uneven course of reform and the challenges of policy implementation.

The problem of men: a reply to Keith Pringle, CSP Issue 36

Keith Pringle’s willingness to name the problem of masculinity in welfare work is to be applauded. He is not of course the first to do so (Campbell, 1988; Macleod and Saraga, 1988; Macleod and Saraga, 1991; Parton, 1990) but our attention needs to be drawn back again and again to the overwhelming evidence that male power is central to sexual abuse. Voices naming the problem are frequently drowned by a cacophony of renewed denials and diversions. We are told to pay attention instead to more serious social problems; or we are reminded that women abuse too; or that we are forgetting the fact that males are also victims of abuse and rape. Those who articulate this final view will also imply that the effect on male victims of abuse is somehow worse than it is for women. That men are overwhelmingly the perpetrators of abuse against males is overlooked in the rush to deflect our gaze from the centrality of masculinity in relation to sexual violence of all kinds. The difficulty which Keith Pringle’s article presents is neither in his naming of the problem nor even his attempt to create a debate about the place of men in welfare work. My concern here is to examine the discourses of sexuality from which he draws and, through his writing, reinforces. I argue that these discourses do not allow a sufficiently nuanced discussion to progress what I recognise as a crucial debate.

Republished: How to study improvement interventions: a brief overview of possible study types

Improvement (defined broadly as purposive efforts to secure positive change) has become an increasingly important activity and field of inquiry within healthcare. This article offers an overview of possible methods for the study of improvement interventions. The choice of available designs is wide, but debates continue about how far improvement efforts can be simultaneously practical (aimed at producing change) and scientific (aimed at producing new knowledge), and whether the distinction between the practical and the scientific is a real and useful one. Quality improvement projects tend to be applied and, in some senses, self-evaluating. They are not necessarily directed at generating new knowledge, but reports of such projects if well conducted and cautious in their inferences may be of considerable value. They can be distinguished heuristically from research studies, which are motivated by and set out explicitly to test a hypothesis, or otherwise generate new knowledge, and from formal evaluations of improvement projects. We discuss variants of trial designs, quasi-experimental designs, systematic reviews, programme evaluations, process evaluations, qualitative studies, and economic evaluations. We note that designs that are better suited to the evaluation of clearly defined and static interventions may be adopted without giving sufficient attention to the challenges associated with the dynamic nature of improvement interventions and their interactions with contextual factors. Reconciling pragmatism and research rigour is highly desirable in the study of improvement. Trade-offs need to be made wisely, taking into account the objectives involved and inferences to be made.