A Mohammed

Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma

The history of monitoring the outcomes of health care by external agencies can be traced to ancient times. However, the danger, now as then, is that in the search for improvement, comparative measures of mortality and morbidity are often overinterpreted, resulting in judgments about the underlying quality of care. Such judgments can translate into performance management strategies in the form of capricious sanctions (such as star ratings) and unjustified rewards (such as special freedoms or financial allocations). The resulting risk of stigmatising an entire institution injects huge tensions into health-care organisations and can divert attention from genuine improvement towards superficial improvement or even gaming behaviour (ie, manipulating the system). These dangers apply particularly to measures of outcome and throughput. We argue that comparative outcome data (league tables) should not be used by external agents to make judgments about quality of hospital care. Although they might provide a reasonable measure of quality in some high-risk surgical situations, they have little validity in acute medical settings. Their use to support a system of reward and punishment is unfair and, unsurprisingly, often resisted by clinicians and managers. We argue further that although outcome data are useful for research and monitoring trends within an organisation, those who wish to improve care for patients and not penalise doctors and managers, should concentrate on direct measurement of adherence to clinical and managerial standards.

Bristol, Shipman, and clinical governance: Shewhart's forgotten lessons

During the past century, manufacturing industry has achieved great success in improving the quality of its products. An essential factor in this success has been the use of Walter A Shewharts pioneering work in the economic control of variation, which culminated in the development of a simple yet powerful graphical method known as the control chart. This chart classifies variation as having a common cause or special cause and thus guides the user to the most appropriate action to effect improvement. Using six case studies, including the excess deaths after paediatric cardiac surgery seen in Bristol, UK, and the activities of general practitioner turned murderer Harold Shipman, we show a central role for Shewharts approach in turning the rhetoric of clinical governance into a reality.

Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals

Objective To assess the validity of case mix adjustment methods used to derive standardised mortality ratios for hospitals, by examining the consistency of relations between risk factors and mortality across hospitals. Design Retrospective analysis of routinely collected hospital data comparing observed deaths with deaths predicted by the Dr Foster Unit case mix method. Setting Four acute National Health Service hospitals in the West Midlands (England) with case mix adjusted standardised mortality ratios ranging from 88 to 140. Participants 96 948 (April 2005 to March 2006), 126 695 (April 2006 to March 2007), and 62 639 (April to October 2007) admissions to the four hospitals. Main outcome measures Presence of large interaction effects between case mix variable and hospital in a logistic regression model indicating non-constant risk relations, and plausible mechanisms that could give rise to these effects. Results Large significant (P≤0.0001) interaction effects were seen with several case mix adjustment variables. For two of these variables—the Charlson (comorbidity) index and emergency admission—interaction effects could be explained credibly by differences in clinical coding and admission practices across hospitals. Conclusions The Dr Foster Unit hospital standardised mortality ratio is derived from an internationally adopted/adapted method, which uses at least two variables (the Charlson comorbidity index and emergency admission) that are unsafe for case mix adjustment because their inclusion may actually increase the very bias that case mix adjustment is intended to reduce. Claims that variations in hospital standardised mortality ratios from Dr Foster Unit reflect differences in quality of care are less than credible.

Performance league tables: the NHS deserves better

League tables are frequently used to depict comparative performance in sport and commerce. However, extension of their use to rank services provided by healthcare agencies has attracted resistance, criticism, and anxiety. In this article we discuss the benefits and drawbacks of league tables and suggest that an alternative technique, based on statistical process control, could be introduced in their place. We believe that this technique would have the dual advantage of being less threatening to providers of health services and would be more easily understood and correctly interpreted by patients, auditors, and commissioners of services. ### Summary points League tables are an established technique for displaying the comparative ranking of organisations in terms of their performance League tables provoke anxiety and concern among health service providers for several reasons, including concerns over adjustment for case mix and the role of chance in determining their rank Control charts, used for monitoring and control of variation in the manufacturing industry, overcome these problems by displaying performance without ranking and helping to differentiate between random variation and that due to special causes League tables are useful for comparing quality or outputs from different systems, whereas control charts are more useful for comparison of units within a single system, such as the NHS Control charts avoid stigmatising “poor performers” and promote the use of a systems approach to quality improvement For many years league tables have been used to rank the quality of goods or services provided by competing organisations. They are commonly published in the popular press and magazines, specialist journals, and the internet. These tables range from those that simply rank crude performance on indicators to those that report sophisticated comparisons of summary adjusted statistics (such as those with uncertainty intervals around the rank). The public is prepared to pay intermediaries, such as financial advisers, …

What is the empirical evidence that hospitals with higher-risk adjusted mortality rates provide poorer quality care? A systematic review of the literature

BackgroundDespite increasing interest and publication of risk-adjusted hospital mortality rates, the relationship with underlying quality of care remains unclear. We undertook a systematic review to ascertain the extent to which variations in risk-adjusted mortality rates were associated with differences in quality of care.MethodsWe identified studies in which risk-adjusted mortality and quality of care had been reported in more than one hospital. We adopted an iterative search strategy using three databases – Medline, HealthSTAR and CINAHL from 1966, 1975 and 1982 respectively. We identified potentially relevant studies on the basis of the title or abstract. We obtained these papers and included those which met our inclusion criteria.ResultsFrom an initial yield of 6,456 papers, 36 studies met the inclusion criteria. Several of these studies considered more than one process-versus-risk-adjusted mortality relationship. In total we found 51 such relationships in a widen range of clinical conditions using a variety of methods. A positive correlation between better quality of care and risk-adjusted mortality was found in under half the relationships (26/51 51%) but the remainder showed no correlation (16/51 31%) or a paradoxical correlation (9/51 18%).ConclusionThe general notion that hospitals with higher risk-adjusted mortality have poorer quality of care is neither consistent nor reliable.

Development and validation of an electronic frailty index using routine primary care electronic health record data

Background: frailty is an especially problematic expression of population ageing. International guidelines recommend routine identification of frailty to provide evidence-based treatment, but currently available tools require additional resource. Objectives: to develop and validate an electronic frailty index (eFI) using routinely available primary care electronic health record data. Study design and setting: retrospective cohort study. Development and internal validation cohorts were established using a randomly split sample of the ResearchOne primary care database. External validation cohort established using THIN database. Participants: patients aged 65–95, registered with a ResearchOne or THIN practice on 14 October 2008. Predictors: we constructed the eFI using the cumulative deficit frailty model as our theoretical framework. The eFI score is calculated by the presence or absence of individual deficits as a proportion of the total possible. Categories of fit, mild, moderate and severe frailty were defined using population quartiles. Outcomes: outcomes were 1-, 3- and 5-year mortality, hospitalisation and nursing home admission. Statistical analysis: hazard ratios (HRs) were estimated using bivariate and multivariate Cox regression analyses. Discrimination was assessed using receiver operating characteristic (ROC) curves. Calibration was assessed using pseudo-R2 estimates. Results: we include data from a total of 931,541 patients. The eFI incorporates 36 deficits constructed using 2,171 CTV3 codes. One-year adjusted HR for mortality was 1.92 (95% CI 1.81–2.04) for mild frailty, 3.10 (95% CI 2.91–3.31) for moderate frailty and 4.52 (95% CI 4.16–4.91) for severe frailty. Corresponding estimates for hospitalisation were 1.93 (95% CI 1.86–2.01), 3.04 (95% CI 2.90–3.19) and 4.73 (95% CI 4.43–5.06) and for nursing home admission were 1.89 (95% CI 1.63–2.15), 3.19 (95% CI 2.73–3.73) and 4.76 (95% CI 3.92–5.77), with good to moderate discrimination but low calibration estimates. Conclusions: the eFI uses routine data to identify older people with mild, moderate and severe frailty, with robust predictive validity for outcomes of mortality, hospitalisation and nursing home admission. Routine implementation of the eFI could enable delivery of evidence-based interventions to improve outcomes for this vulnerable group.

The measurement of active errors: methodological issues

The value of research in any topic area turns on its validity. Patient safety research has revealed—or, at least, given renewed urgency to—a raft of methodological issues. The meaning and thus the value of empirical research in this field is contingent on getting the methodology right. The need for good methods for the measurement of error is necessary whenever an inference is intended and, since inferences lie at the heart of research and management, there is a huge need to understand better how to make measurements that are meaningful, precise, and accurate. In this paper we consider issues relating to the measurement of error and the need for more research.

The effect of walking on risk factors for cardiovascular disease: An updated systematic review and meta-analysis of randomised control trials

OBJECTIVE To conduct a systematic review and meta-analysis of randomised control trials that examined the effect of walking on risk factors for cardiovascular disease. METHODS Four electronic databases and reference lists were searched (Jan 1971-June 2012). Two authors identified randomised control trials of interventions ≥ 4 weeks in duration that included at least one group with walking as the only treatment and a no-exercise comparator group. Participants were inactive at baseline. Pooled results were reported as weighted mean treatment effects and 95% confidence intervals using a random effects model. RESULTS 32 articles reported the effects of walking interventions on cardiovascular disease risk factors. Walking increased aerobic capacity (3.04 mL/kg/min, 95% CI 2.48 to 3.60) and reduced systolic (-3.58 mm Hg, 95% CI -5.19 to -1.97) and diastolic (-1.54 mm Hg, 95% CI -2.83 to -0.26) blood pressure, waist circumference (-1.51 cm, 95% CI -2.34 to -0.68), weight (-1.37 kg, 95% CI -1.75 to -1.00), percentage body fat (-1.22%, 95% CI -1.70 to -0.73) and body mass index (-0.53 kg/m(2), 95% CI -0.72 to -0.35) but failed to alter blood lipids. CONCLUSIONS Walking interventions improve many risk factors for cardiovascular disease. This underscores the central role of walking in physical activity for health promotion.

Weekend admission to hospital has a higher risk of death in the elective setting than in the emergency setting: a retrospective database study of national health service hospitals in England

BackgroundAlthough acute hospitals offer a twenty-four hour seven day a week service levels of staffing are lower over the weekends and some health care processes may be less readily available over the weekend. Whilst it is thought that emergency admission to hospital on the weekend is associated with an increased risk of death, the extent to which this applies to elective admissions is less well known. We investigated the risk of death in elective and elective patients admitted over the weekend versus the weekdays.MethodsRetrospective statistical analysis of routinely collected acute hospital admissions in England, involving all patient discharges from all acute hospitals in England over a year (April 2008-March 2009), using a logistic regression model which adjusted for a range of patient case-mix variables, seasonality and admission over a weekend separately for elective and emergency (but excluding zero day stay emergency admissions discharged alive) admissions.ResultsOf the 1,535,267 elective admissions, 91.7% (1,407,705) were admitted on the weekday and 8.3% (127,562) were admitted on the weekend. The mortality following weekday admission was 0.52% (7,276/1,407,705) compared with 0.77% (986/127,562) following weekend admission. Of the 3,105,249 emergency admissions, 76.3% (2,369,316) were admitted on the weekday and 23.7% (735,933) were admitted on the weekend. The mortality following emergency weekday admission was 6.53% (154,761/2,369,316) compared to 7.06% (51,922/735,933) following weekend admission. After case-mix adjustment, weekend admissions were associated with an increased risk of death, especially in the elective setting (elective Odds Ratio: 1.32, 95% Confidence Interval 1.23 to 1.41); vs emergency Odds Ratio: 1.09, 95% Confidence Interval 1.05 to 1.13).ConclusionsWeekend admission appears to be an independent risk factor for dying in hospital and this risk is more pronounced in the elective setting. Given the planned nature of elective admissions, as opposed to the unplanned nature of emergency admissions, it would seem less likely that this increased risk in the elective setting is attributable to unobserved patient risk factors. Further work to understand the relationship between weekend processes of care and mortality, especially in the elective setting, is required.

Impact of introducing an electronic physiological surveillance system on hospital mortality

Background Avoidable hospital mortality is often attributable to inadequate patient vital signs monitoring, and failure to recognise or respond to clinical deterioration. The processes involved with vital sign collection and charting; their integration, interpretation and analysis; and the delivery of decision support regarding subsequent clinical care are subject to potential error and/or failure. Objective To determine whether introducing an electronic physiological surveillance system (EPSS), specifically designed to improve the collection and clinical use of vital signs data, reduced hospital mortality. Methods A pragmatic, retrospective, observational study of seasonally adjusted in-hospital mortality rates in three main hospital specialties was undertaken before, during and after the sequential deployment and ongoing use of a hospital-wide EPSS in two large unconnected acute general hospitals in England. The EPSS, which uses wireless handheld computing devices, replaced a paper-based vital sign charting and clinical escalation system. Results During EPSS implementation, crude mortality fell from a baseline of 7.75% (2168/27 959) to 6.42% (1904/29 676) in one hospital (estimated 397 fewer deaths), and from 7.57% (1648/21 771) to 6.15% (1614/26 241) at the second (estimated 372 fewer deaths). At both hospitals, multiyear statistical process control analyses revealed abrupt and sustained mortality reductions, coincident with the deployment and increasing use of the system. The cumulative total of excess deaths reduced in all specialties with increasing use of the system across the hospital. Conclusions The use of technology specifically designed to improve the accuracy, reliability and availability of patients’ vital signs and early warning scores, and thereby the recognition of and response to patient deterioration, is associated with reduced mortality in this study.