Thomas Woodcock

How to study improvement interventions: a brief overview of possible study types

Improvement (defined broadly as purposive efforts to secure positive change) has become an increasingly important activity and field of inquiry within healthcare. This article offers an overview of possible methods for the study of improvement interventions. The choice of available designs is wide, but debates continue about how far improvement efforts can be simultaneously practical (aimed at producing change) and scientific (aimed at producing new knowledge), and whether the distinction between the practical and the scientific is a real and useful one. Quality improvement projects tend to be applied and, in some senses, self-evaluating. They are not necessarily directed at generating new knowledge, but reports of such projects if well conducted and cautious in their inferences may be of considerable value. They can be distinguished heuristically from research studies, which are motivated by and set out explicitly to test a hypothesis, or otherwise generate new knowledge, and from formal evaluations of improvement projects. We discuss variants of trial designs, quasi-experimental designs, systematic reviews, programme evaluations, process evaluations, qualitative studies, and economic evaluations. We note that designs that are better suited to the evaluation of clearly defined and static interventions may be adopted without giving sufficient attention to the challenges associated with the dynamic nature of improvement interventions and their interactions with contextual factors. Reconciling pragmatism and research rigour is highly desirable in the study of improvement. Trade-offs need to be made wisely, taking into account the objectives involved and inferences to be made.

Designing and implementing a COPD discharge care bundle

National surveys have revealed significant differences in patient outcomes following admission to hospital with acute exacerbation of COPD which are likely to be due to variations in care. We developed a care bundle, comprising a short list of evidence-based practices to be implemented prior to discharge for all patients admitted with this condition, based on a review of national guidelines and other relevant literature, expert opinion and patient consultation. Implementation was then piloted using action research methodologies with patient input. Actively involving staff was vital to ensure that the changes introduced were understood and the process followed. Implementation of a care bundle has the potential to produce a dramatic improvement in compliance with optimum health care practice.

Making change last: applying the NHS institute for innovation and improvement sustainability model to healthcare improvement.

The implementation of evidence-based treatments to deliver high-quality care is essential to meet the healthcare demands of aging populations. However, the sustainable application of recommended practice is difficult to achieve and variable outcomes well recognised. The NHS Institute for Innovation and Improvement Sustainability Model (SM) was designed to help healthcare teams recognise determinants of sustainability and take action to embed new practice in routine care. This article describes a formative evaluation of the application of the SM by the National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care for Northwest London (CLAHRC NWL).Data from project teams’ responses to the SM and formal reviews was used to assess acceptability of the SM and the extent to which it prompted teams to take action. Projects were classified as ‘engaged,’ ‘partially engaged’ and ‘non-engaged.’ Quarterly survey feedback data was used to explore reasons for variation in engagement. Score patterns were compared against formal review data and a ‘diversity of opinion’ measure was derived to assess response variance over time.Of the 19 teams, six were categorized as ‘engaged,’ six ‘partially engaged,’ and seven as ‘non-engaged.’ Twelve teams found the model acceptable to some extent. Diversity of opinion reduced over time. A minority of teams used the SM consistently to take action to promote sustainability but for the majority SM use was sporadic. Feedback from some team members indicates difficulty in understanding and applying the model and negative views regarding its usefulness.The SM is an important attempt to enable teams to systematically consider determinants of sustainability, provide timely data to assess progress, and prompt action to create conditions for sustained practice. Tools such as these need to be tested in healthcare settings to assess strengths and weaknesses and findings disseminated to aid development. This study indicates the SM provides a potentially useful approach to measuring teams’ views on the likelihood of sustainability and prompting action. Securing engagement of teams with the SM was challenging and redesign of elements may need to be considered. Capacity building and facilitation appears necessary for teams to effectively deploy the SM.

Designing quality improvement initiatives: the action effect method, a structured approach to identifying and articulating programme theory

Background The identification and articulation of programme theory can support effective design, execution and evaluation of quality improvement (QI) initiatives. Programme theory includes an agreed aim, potential interventions to achieve this aim, anticipated cause/effect relationships between the interventions and the aim and measures to monitor improvement. This paper outlines the approach used in a research and improvement programme to support QI initiatives in identifying and articulating programme theory: the action effect method. Background to method development Building on a previously used QI method, the driver diagram, the action effect method was developed using co-design and iteration over four annual rounds of improvement initiatives. This resulted in a specification of the elements required to fully articulate the programme theory of a QI initiative. The action effect method The action effect method is a systematic and structured process to identify and articulate a QI initiatives programme theory. The method connects potential interventions and implementation activities with an overall improvement aim through a diagrammatic representation of hypothesised and evidenced cause/effect relationships. Measure concepts, in terms of service delivery and patient and system outcomes, are identified to support evaluation. Discussion and conclusions The action effect method provides a framework to guide the execution and evaluation of a QI initiative, a focal point for other QI methods and a communication tool to engage stakeholders. A clear definition of what constitutes a well-articulated programme theory is provided to guide the use of the method and assessment of the fidelity of its application.

The effect of applying NICE guidelines for the investigation of stable chest pain on out-patient cardiac services in the UK

BACKGROUNDnThe National Institute for Health and Clinical Excellence (NICE) recently released guidelines for the investigation of chest pain of recent onset. There is no published data regarding their impact on out-patient cardiac services.nnnAIMnThis study was undertaken to assess the likelihood of coronary artery disease (CAD) in Rapid Access Chest Pain Clinic (RACPC) patients and the resultant investigation burden if NICE guidance was applied.nnnMETHODSnFive hundred and ninety-five consecutive patients attending two RACPCs over 6 months preceding release of the NICE guidelines [51% male; median age 55 (range 22-94) years] were risk stratified using NICE criteria and the resultant investigations evaluated.nnnRESULTSnOne hundred and six (18%) patients had a likelihood of CAD <10%, 123 (21%) between 10% and 29%, 175 (29%) between 30% and 60%, 141 (24%) between 61% and 90% and 50 (8%) >90%. NICE would have recommended 443 (74%) patients for no cardiac investigation, 10 (2%) for cardiac computed tomography (CCT), 69 (12%) for functional cardiac testing and 73 (12%) for invasive angiography. Relative to existing practice, there would have been a trend towards reduced functional cardiac testing (-24%, Pu2009=u20090.06), no significant change in CCT (43%, Pu2009=u20090.436) and a significant increase in invasive angiography (508%, Pu2009<u20090.001). The cost of investigations recommended by NICE would have been £15,881 greater than existing practice.nnnCONCLUSIONnThis study suggests patients attending RACPC will have a greater likelihood of CAD than predicted by NICE. Differences between recommended investigations and existing practice will guide investment in cardiac services. Individual hospitals should assess their RACPC cohorts prior to implementing the NICE guidelines.

Quantifying the prevalence of frailty in English hospitals

Objectives Population ageing has been associated with an increase in comorbid chronic disease, functional dependence, disability and associated higher health care costs. Frailty Syndromes have been proposed as a way to define this group within older persons. We explore whether frailty syndromes are a reliable methodology to quantify clinically significant frailty within hospital settings, and measure trends and geospatial variation using English secondary care data set Hospital Episode Statistics (HES). Setting National English Secondary Care Administrative Data HES. Participants All 50u2005540u2005141 patient spells for patients over 65u2005years admitted to acute provider hospitals in England (January 2005—March 2013) within HES. Primary and secondary outcome measures We explore the prevalence of Frailty Syndromes as coded by International Statistical Classification of Diseases, Injuries and Causes of Death (ICD-10) over time, and their geographic distribution across England. We examine national trends for admission spells, inpatient mortality and 30-day readmission. Results A rising trend of admission spells was noted from January 2005 to March 2013(daily average admissions for month rising from over 2000 to over 4000). The overall prevalence of coded frailty is increasing (64u2005559 spells in January 2005 to 150u2005085 spells by Jan 2013). The majority of patients had a single frailty syndrome coded (10.2% vs total burden of 13.9%). Cognitive impairment and falls (including significant fracture) are the most common frailty syndromes coded within HES. Geographic variation in frailty burden was in keeping with known distribution of prevalence of the English elderly population and location of National Health Service (NHS) acute provider sites. Overtime, in-hospital mortality has decreased (>65u2005years) whereas readmission rates have increased (esp.>85u2005years). Conclusions This study provides a novel methodology to reliably quantify clinically significant frailty. Applications include evaluation of health service improvement over time, risk stratification and optimisation of services.

Model-driven approach to data collection and reporting for quality improvement

Graphical abstract

Statistical process control for data without inherent order

BackgroundThe XmR chart is a powerful analytical tool in statistical process control (SPC) for detecting special causes of variation in a measure of quality. In this analysis a statistic called the average moving range is used as a measure of dispersion of the data. This approach is correct for data with natural underlying order, such as time series data. There is however conflict in the literature over the appropriateness of the XmR chart to analyse data without an inherent ordering.MethodsWe derive the maxima and minima for the average moving range in data without inherent ordering, and show how to calculate this for any data set. We permute a real world data set and calculate control limits based on these extrema.ResultsIn the real world data set, permuting the order of the data affected an absolute difference of 109 percent in the width of the control limits.DiscussionWe prove quantitatively that XmR chart analysis is problematic for data without an inherent ordering, and using real-world data, demonstrate the problem this causes for calculating control limits. The resulting ambiguity in the analysis renders it unacceptable as an approach to making decisions based on data without inherent order.ConclusionThe XmR chart should only be used for data endowed with an inherent ordering, such as a time series. To detect special causes of variation in data without an inherent ordering we suggest that one of the many well-established approaches to outlier analysis should be adopted. Furthermore we recommend that in all SPC analyses authors should consistently report the type of control chart used, including the measure of variation used in calculating control limits.

Republished: How to study improvement interventions: a brief overview of possible study types

Improvement (defined broadly as purposive efforts to secure positive change) has become an increasingly important activity and field of inquiry within healthcare. This article offers an overview of possible methods for the study of improvement interventions. The choice of available designs is wide, but debates continue about how far improvement efforts can be simultaneously practical (aimed at producing change) and scientific (aimed at producing new knowledge), and whether the distinction between the practical and the scientific is a real and useful one. Quality improvement projects tend to be applied and, in some senses, self-evaluating. They are not necessarily directed at generating new knowledge, but reports of such projects if well conducted and cautious in their inferences may be of considerable value. They can be distinguished heuristically from research studies, which are motivated by and set out explicitly to test a hypothesis, or otherwise generate new knowledge, and from formal evaluations of improvement projects. We discuss variants of trial designs, quasi-experimental designs, systematic reviews, programme evaluations, process evaluations, qualitative studies, and economic evaluations. We note that designs that are better suited to the evaluation of clearly defined and static interventions may be adopted without giving sufficient attention to the challenges associated with the dynamic nature of improvement interventions and their interactions with contextual factors. Reconciling pragmatism and research rigour is highly desirable in the study of improvement. Trade-offs need to be made wisely, taking into account the objectives involved and inferences to be made.

Improving mental health outcomes: achieving equity through quality improvement

Objective To investigate equity of patient outcomes in a psychological therapy service, following increased access achieved by a quality improvement (QI) initiative. Design Retrospective service evaluation of health outcomes; data analysed by ANOVA, chi-squared and Statistical Process Control. Setting A psychological therapy service in Westminster, London, UK. Participants People living in the Borough of Westminster, London, attending the service (from either healthcare professional or self-referral) between February 2009 and May 2012. Intervention(s) Social marketing interventions were used to increase referrals, including the promotion of the service through local media and through existing social networks. Main Outcome Measure(s) (i) Severity of depression on entry using Patient Health Questionnaire-9 (PHQ9). (ii) Changes to severity of depression following treatment (ΔPHQ9). (iii) Changes in attainment of a meaningful improvement in condition assessed by a key performance indicator. Results Patients from areas of high deprivation entered the service with more severe depression (M = 15.47, SD = 6.75), compared with patients from areas of low (M = 13.20, SD = 6.75) and medium (M = 14.44, SD = 6.64) deprivation. Patients in low, medium and high deprivation areas attained similar changes in depression score (ΔPHQ9: M = −6.60, SD = 6.41). Similar proportions of patients achieved the key performance indicator across initiative phase and deprivation categories. Conclusions QI methods improved access to mental health services; this paper finds no evidence for differences in clinical outcomes in patients, regardless of level of deprivation, interpreted as no evidence of inequity in the service with respect to this outcome.