Martin Marshall

Promotion of improvement as a science

The remarkable achievements of modern health care make the defi ciencies associated with delivery of care all the more noticeable. Health care is a major source of avoidable harm, and patients are routinely exposed to wide and inexplicable variations in the quality of care that they receive. These variations are very diffi cult to address, despite good intentions, policy focus, ambitious improvement programmes, and investment of resources. A major reason for this diffi culty is that the urge to act can easily overwhelm the need for evidence to inform that action, to the extent that much quality improvement work is unscientifi c—it is neither informed by high-quality evidence, nor is it subject to rigorous assessment to establish its eff ectiveness, costs, and risks. Ironically, this absence of a scientifi c approach might lead to outcomes that are exactly the opposite of what is intended by improvement eff orts: resources can be wasted, energy and enthusiasm are dissipated, the side-eff ects of interventions are ignored, and little positive change is evident. Adoption of a more scientifi c approach to improvement has great potential to enhance the ability of health systems to provide high-quality care and optimise resource use. We believe that the science of improvement is in what Kuhn calls the pre-paradigm phase of the emergence of a new discipline, one of the characteristics of which is the absence of an agreed defi nition. Attempts to create a defi nition can trigger intense debate, and the term improvement science has so far eluded consensus. Don Berwick and the Institute of Healthcare Improvement in Boston (MA, USA) have infl uenced one of the most common uses of this term, which is to describe applications of the approach developed by W Edwards Deming to reduce variation in industrial processes. The quality improvement move ment associated with these methods has made a valuable contribution, not least in challenging overly techno cratic, managerialist, or regulatory-focused approaches to change. However, this conceptualisation of a science of improvement is, by itself, too narrow and restrictive to address the challenges that face the health sector, and too often under-emphasises robust assessment. Percep tion of improvement solely in terms of imple mentation and comparative eff ectiveness research is similarly inadequate. An example of a benefi t of a broader way of thinking is the reduction of central line bloodstream infections in intensive-care units. These infections are a substantial and expensive clinical problem, and are often fatal. A cohort study of 103 intensive-care units in Michigan, USA, showed that a multicomponent programme of evidence-based technical interventions, adaptive interventions targeting culture and systems, and a centralised data collection and feedback system resulted in a substantial reduction in rates of these infections. These reductions were sustained over time and were associated with reduced mortality in participating intensive-care units compared with control units. The results have been replicated in 22 additional US states. Follow-up work investigated the mechanisms through which the programme worked, and generated a theory of change that could inform, and be tested in, subsequent repe titions of the programme. Another example of a benefi t of this broader way of thinking is the increase in identifi cation and referral of victims of domestic violence. Domestic violence is often managed inadequately, despite the major health and social implications of unidentifi ed and unmanaged abuse. In a cluster-randomised controlled trial in 51 primary care clinics in two UK cities (London and Bristol), investigators tested a range of evidence-based interventions to increase rates of primary care identifi cation and referral to specialist advocacy services of women subjected to domestic violence. The results showed three-times more recorded disclosures of domestic violence and a 22-times higher number of referrals to specialist services in the intervention group. The interventions, in the form of targeted educational programmes, are now being adopted in several places in England. These examples show some distinguishing features of a broader defi nition of improvement science. Improvement science aims to create practical learning that can make a timely diff erence to patient care. It is characterised by its large domain of interest, its applied nature, and its commitment to generation of practical learning that can be applied in real-life situations. Improvement science recognises and integrates many contributions, similar to the way that engineering science uses scientifi c knowledge and theories to address real-life problems. Improvement science also aims to generate local wisdom and generalisable or transferable knowledge, with robust, well established research methods applied in highly pragmatic ways. Improvement science enables local improvement and, crucially, produces knowledge with external validity. Thus, work classifi ed as improvement science should be of suffi cient quality to be published as a long-lasting, widely disseminated record in well-respected journals. Development of local knowledge based on a strong sense of ownership and a willingness to adapt improvement activities is clearly important, but too many quality improvement projects rely on contemporaneous, non-standardised, unverifi ed data to make judgments about their eff ectiveness. By contrast, the science of improvement is, based on moral and practical requirements, characterised by its commitment to rigorous and credible assessment methods Lancet 2013; 381: 419–21

An open letter to The BMJ editors on qualitative research

Seventy six senior academics from 11 countries invite The BMJ ’s editors to reconsider their policy of rejecting qualitative research on the grounds of low priority. They challenge the journal to develop a proactive, scholarly, and pluralist approach to research that aligns with its stated mission

Did a quality improvement collaborative make stroke care better? A cluster randomized trial

BackgroundStroke can result in death and long-term disability. Fast and high-quality care can reduce the impact of stroke, but UK national audit data has demonstrated variability in compliance with recommended processes of care. Though quality improvement collaboratives (QICs) are widely used, whether a QIC could improve reliability of stroke care was unknown.MethodsTwenty-four NHS hospitals in the Northwest of England were randomly allocated to participate either in Stroke 90:10, a QIC based on the Breakthrough Series (BTS) model, or to a control group giving normal care. The QIC focused on nine processes of quality care for stroke already used in the national stroke audit. The nine processes were grouped into two distinct care bundles: one relating to early hours care and one relating to rehabilitation following stroke. Using an interrupted time series design and difference-in-difference analysis, we aimed to determine whether hospitals participating in the QIC improved more than the control group on bundle compliance.ResultsData were available from nine interventions (3,533 patients) and nine control hospitals (3,059 patients). Hospitals in the QIC showed a modest improvement from baseline in the odds of average compliance equivalent to a relative improvement of 10.9% (95% CI 1.3%, 20.6%) in the Early Hours Bundle and 11.2% (95% CI 1.4%, 21.5%) in the Rehabilitation Bundle. Secondary analysis suggested that some specific processes were more sensitive to an intervention effect.ConclusionsSome aspects of stroke care improved during the QIC, but the effects of the QIC were modest and further improvement is needed. The extent to which a BTS QIC can improve quality of stroke care remains uncertain. Some aspects of care may respond better to collaboratives than others.Trial registrationISRCTN13893902.

What we know about designing an effective improvement intervention (but too often fail to put into practice)

It is temptingly easy to treat improvement interventions as if they are drugs—technical, stable and uninfluenced by the environment in which they work. Doing so makes life so much easier for everyone. It allows improvement practitioners to plan their work with a high degree of certainty, funders to be confident that they know what they are buying and evaluators to focus on what really matters—whether or not ‘it’ works. But of course most people know that life is not as simple as that. Experienced improvers have long recognised that interventions—the specific tools and activities introduced into a healthcare system with the aim of changing its performance for the better1—flex and morph. Clever improvers watch and describe how this happens. Even more clever improvers plan and actively manage the process in a way that optimises the impact of the improvement initiative. The challenge is that while most improvers (the authors included) appreciate the importance of carefully designing an improvement intervention, they (we) rarely do so in a sufficiently clever way. In this article, we describe our attempts as an experienced team of practitioners, improvers, commissioners and evaluators to design an effective intervention to improve the safety of people living in care homes in England. We highlight how the design of the intervention, as described in the original grant proposal, changed significantly throughout the initiative. We outline how the changes that were made resulted in a more effective intervention but how our failure to design a better intervention from the start reduced the overall impact of the project. Drawing on the rapidly expanding literature in the field and our own experience, we reflect on what we would do differently if we could have our time again. A growing number of vulnerable older people are living in care homes and are …

Doctors and quality improvement

Doctors are trained to diagnose and treat ill health. Within the confines of the one‐to‐one relationship with patients behind the closed door of the consulting room most clinicians fulfil this role highly successfully. But the traditional role of the doctor is carried out within a broader historical, organizational, social and political context – where the diagnosis and treatment of system failures can be as important as clinical interactions with individual patients. Unless doctors are willing to understand and influence this wider context, their ability to improve health outcomes in an increasingly complex environment will be challenged. This understanding can be achieved by engaging with the emerging science of Quality Improvement. In this paper we describe the medical professions current approach to improving quality and assess where this approach has got us. We then compare current practice with an alternative approach which focuses not only on patients but also on the wider health system and we examine the opportunities and challenges that this presents to the medical profession.

Increasing the impact of health services research on service improvement: the researcher-in-residence model

Martin Marshall, Laura Eyre, Mirza Lalani, Salmaan Khan, Susan Mann, Debi de Silva and Jonathan Shapiro Department of Primary Care and Population Health, UCL, Sir Ludwig Guttman Centre, London, UK Department of Primary Care and Population Health, UCL, Sir Ludwig Guttman Centre, London, UK NIHR Knowledge Mobilisation and Research Fellow, Institute for Women’s Health, UCL, London, UK The Evidence Centre, London, UK Independent consultant Corresponding author: Martin Marshall. Email: martin.marshall@ucl.ac.uk

Developing a science of improvement

Clinicians are used to using scientific evidence when they make decisions about the care they provide for patients: they have a good idea of what blood pressure they should be aiming to achieve for patients with diabetes, and there is enough evidence about the outcomes of knee joint replacement surgery to enable an informed conversation with patients presenting with osteoarthritis. Information to guide clinical decisions about what to do, or what to advise patients to do, is available, comprehensible and increasingly easy to access.

An evolution of professionalism

The model of financing and service delivery for the Royal Automobile Club (RAC, England) has not changed substantially for the last 50 years. Yet this belies a seismic change in how those working for the RAC are treated, what they do and how they are incentivized. The most important change has been in the complexity of the machines that they are working with. Twenty years ago, professional patrolmen could take pride in fixing most automobile emergencies, at least temporarily, by the roadside. This is no longer the case. The move towards computer-controlled cars with intrinsically complex electronic software means that increasingly patrolmen are diagnosticians who tow cars to specialized garages. This has impacted hugely on the motivation of patrolmen. Whereas previous satisfaction came from fixing the problem and required great technical skill, satisfaction now comes from guiding the problem to the right place for fixing.1 Analogies can be taken too far but the pace of change witnessed in healthcare, with the requirement for autonomous health professionals to adapt their clinical practice accordingly, bears similarities to the experience of the RAC patrolmen. Five years ago, the Royal College of Physicians undertook an inquiry into the meaning of medical professionalism. Following extensive consultation, this report defined professionalism as ‘a set of values, behaviors and relationships that underpins the trust the public has in doctors’. It concluded that building and maintaining trust was the most important aim of professionalism.2 The ensuing consensus statement, supported by the four Chief Medical Officers of the United Kingdom, on the role of the doctor begins: ‘Doctors alone amongst healthcare professionals must be capable of regularly taking ultimate responsibility for difficult decisions in situations of clinical complexity and uncertainty, drawing on their scientific knowledge and well developed clinical judgment. The doctor’s role must be defined by what is in the best interest of patients and of the population served’.3 At the time of publishing the above statements, it was recognized that the role of the doctor, alongside concepts of professionalism, would continue to evolve. In part this evolution is driven by internal changes to the practice of medicine such as technology; in part due to a more multidisciplinary approach to healthcare delivery; and in part as a response to the public consensus on what is expected from a clinician. Our current preferred model of new professionalism has a number of components. It places a stronger emphasis on accountability, recognizing the benefits of creating a different dynamic between patients and professionals, and assuming a stronger sense of responsibility for the ways in which the wider health system works and for all dimensions of quality. It promotes a desire constantly to improve what clinicians do, accepting change as an asset rather than a threat. It commits to using a range of different approaches to developing and mobilizing knowledge about how to improve care and to building the formal evidence base underpinning improvement. Finally, it emphasizes the importance of clinicians working in multidisciplinary teams across organizational boundaries. Most fundamentally, no longer is a commitment to improving the quality of patient care an ‘add-on’. It is a central part of the role of a clinician and a core value of the new model of professionalism. This push towards a new model of professionalism is more than a social construct. It is underpinned by a new science which is complementary to the established ways of knowing which have dominated the delivery of healthcare for over 150 years. Over the last five years, the science of quality improvement (QI) has grown in both clinical popularity and academic recognition. In the last 12 months, PubMed records 360 English published articles containing the key words ‘quality improvement’. Within the UK, there is now an Improvement Faculty for Patient Safety and Quality of Care.4 The Health Foundation, an independent charity devoted to continuously improving the quality of healthcare throughout the UK, supports hundreds of projects as part of its improvement programmes. Yet more enthusiasm and commitment for QI is found within Deaneries and the Royal Colleges. These organizations are undertaking pilot studies of trainees in quality improvement projects, replacing the traditional obligatory and unpopular ‘clinical audits’. On an international stage, the Institute for Healthcare Improvement (IHI) has pioneered improvement initiatives such as the 5 Million Lives Campaign in the US.5 IHI is now collaborating with tens of thousands of healthcare organizations in countries including England, Scotland, Ghana, Malawi and South Africa. This global push for quality improvement creates momentum for the transition to the new model of professionalism presented here. Further reinforcement for this model lies in the urgent need, due to the stringent financial conditions, for improved efficiency and productivity. It is widely acknowledged that to achieve this will need clinical leadership by those who understand quality improvement.6,7 This renewed focus on the clinician developing and actively altering the service they work in to ensure better patient care leads to a new facet of the compact between the public and clinicians around the nature of professionalism.8 This does not detract from the primacy of the individual doctor–patient relationship, rather there is an additional caveat – helping the patient is achieved not just by the one to one interaction but the ‘backstage’ changes that are enacted. This understanding of the importance of influencing the environment within which care is provided has long been recognized by the public health community but now needs to become a core body of knowledge and skill for all clinicians. The importance of training in quality improvement and making quality improvement part of the everyday delivery of healthcare requires training from an undergraduate level. However, the work of Friedson9 identifies that professional norms are adopted via modeling from role models in the workplace. The challenge for the NHS, therefore, is that many senior clinicians are unfamiliar with the science of quality improvement and ultimately ill-equipped to teach it. Rather than relying on the traditional gradient of senior to junior transmission of knowledge, we propose that the diffusion of knowledge about quality improvement as core to new professionalism may be best disseminated across networks, formal and informal, online and offline. Furthermore, the locus of knowledge may be required to extend beyond the traditional bastions of professionalism such as the Royal Colleges to include organizations such as IHI and The Health Foundation. For many industries, including automobile and healthcare, the changing world is complex and often intimidating to professional norms. We encourage clinicians to integrate active involvement in service improvement as part of their professional identity. Because a commitment to improving quality in healthcare is no longer a choice. It is a core value of new professionalism.

What can a participatory approach to evaluation contribute to the field of integrated care

Better integration of care within the health sector and between health and social care is seen in many countries as an essential way of addressing the enduring problems of dwindling resources, changing demographics and unacceptable variation in quality of care. Current research evidence about the effectiveness of integration efforts supports neither the enthusiasm of those promoting and designing integrated care programmes nor the growing efforts of practitioners attempting to integrate care on the ground. In this paper we present a methodological approach, based on the principles of participatory research, that attempts to address this challenge. Participatory approaches are characterised by a desire to use social science methods to solve practical problems and a commitment on the part of researchers to substantive and sustained collaboration with relevant stakeholders. We describe how we applied an emerging practical model of participatory research, the researcher-in-residence model, to evaluate a large-scale integrated care programme in the UK. We propose that the approach added value to the programme in a number of ways: by engaging stakeholders in using established evidence and with the benefits of rigorously evaluating their work, by providing insights for local stakeholders that they were either not familiar with or had not fully considered in relation to the development and implementation of the programme and by challenging established mindsets and norms. While there is still much to learn about the benefits and challenges of applying participatory approaches in the health sector, we demonstrate how using such approaches have the potential to help practitioners integrate care more effectively in their daily practice and help progress the academic study of integrated care.

Protocol for a process-oriented qualitative evaluation of the Waltham Forest and East London Collaborative (WELC) integrated care pioneer programme using the Researcher-in-Residence model.

Introduction The integration of health and social care in England is widely accepted as the answer to fragmentation, financial concerns and system inefficiencies, in the context of growing and ageing populations with increasingly complex needs. Despite an expanding body of literature, there is little evidence yet to suggest that integrated care can achieve the benefits that its advocates claim for it. Researchers have often adopted rationalist and technocratic approaches to evaluation, treating integration as an intervention rather than a process. Results have usually been of limited use to practitioners responsible for health and social care integration. There is, therefore, a need to broaden the evidence base, exploring not only what works but also how integrated care can most successfully be implemented and delivered. For this reason, we are carrying out a formative evaluation of the Waltham Forest and East London Collaborative (WELC) integrated care pioneer programme. Our expectation is that this will add value to the literature by focusing on the processes by which the vision and objectives of integrated care are translated through phases of development, implementation and delivery from a central to a local perspective, and from a strategic to an operational perspective. Methods and analysis The qualitative and process-oriented evaluation uses an innovative participative approach—the Researcher-in-Residence model. The evaluation is underpinned by a critical ontology, an interpretive epistemology and a critical discourse analysis methodology. Data will be generated using interviews, observations and documentary gathering. Ethics and dissemination Emerging findings will be interpreted and disseminated collaboratively with stakeholders, to enable the research to influence and optimise the effective implementation of integrated care across WELC. Presentations and publications will ensure that learning is shared as widely as possible. The study has received ethical approval from University College Londons Research Ethics Committee and has all appropriate NHS governance clearances.