Gavin Rudge

Evidence of methodological bias in hospital standardised mortality ratios: retrospective database study of English hospitals

Objective To assess the validity of case mix adjustment methods used to derive standardised mortality ratios for hospitals, by examining the consistency of relations between risk factors and mortality across hospitals. Design Retrospective analysis of routinely collected hospital data comparing observed deaths with deaths predicted by the Dr Foster Unit case mix method. Setting Four acute National Health Service hospitals in the West Midlands (England) with case mix adjusted standardised mortality ratios ranging from 88 to 140. Participants 96 948 (April 2005 to March 2006), 126 695 (April 2006 to March 2007), and 62 639 (April to October 2007) admissions to the four hospitals. Main outcome measures Presence of large interaction effects between case mix variable and hospital in a logistic regression model indicating non-constant risk relations, and plausible mechanisms that could give rise to these effects. Results Large significant (P≤0.0001) interaction effects were seen with several case mix adjustment variables. For two of these variables—the Charlson (comorbidity) index and emergency admission—interaction effects could be explained credibly by differences in clinical coding and admission practices across hospitals. Conclusions The Dr Foster Unit hospital standardised mortality ratio is derived from an internationally adopted/adapted method, which uses at least two variables (the Charlson comorbidity index and emergency admission) that are unsafe for case mix adjustment because their inclusion may actually increase the very bias that case mix adjustment is intended to reduce. Claims that variations in hospital standardised mortality ratios from Dr Foster Unit reflect differences in quality of care are less than credible.

Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation

Objectives To conduct an independent evaluation of the first phase of the Health Foundation’s Safer Patients Initiative (SPI), and to identify the net additional effect of SPI and any differences in changes in participating and non-participating NHS hospitals. Design Mixed method evaluation involving five substudies, before and after design. Setting NHS hospitals in the United Kingdom. Participants Four hospitals (one in each country in the UK) participating in the first phase of the SPI (SPI1); 18 control hospitals. Intervention The SPI1 was a compound (multi-component) organisational intervention delivered over 18 months that focused on improving the reliability of specific frontline care processes in designated clinical specialties and promoting organisational and cultural change. Results Senior staff members were knowledgeable and enthusiastic about SPI1. There was a small (0.08 points on a 5 point scale) but significant (P<0.01) effect in favour of the SPI1 hospitals in one of 11 dimensions of the staff questionnaire (organisational climate). Qualitative evidence showed only modest penetration of SPI1 at medical ward level. Although SPI1 was designed to engage staff from the bottom up, it did not usually feel like this to those working on the wards, and questions about legitimacy of some aspects of SPI1 were raised. Of the five components to identify patients at risk of deterioration—monitoring of vital signs (14 items); routine tests (three items); evidence based standards specific to certain diseases (three items); prescribing errors (multiple items from the British National Formulary); and medical history taking (11 items)—there was little net difference between control and SPI1 hospitals, except in relation to quality of monitoring of acute medical patients, which improved on average over time across all hospitals. Recording of respiratory rate increased to a greater degree in SPI1 than in control hospitals; in the second six hours after admission recording increased from 40% (93) to 69% (165) in control hospitals and from 37% (141) to 78% (296) in SPI1 hospitals (odds ratio for “difference in difference” 2.1, 99% confidence interval 1.0 to 4.3; P=0.008). Use of a formal scoring system for patients with pneumonia also increased over time (from 2% (102) to 23% (111) in control hospitals and from 2% (170) to 9% (189) in SPI1 hospitals), which favoured controls and was not significant (0.3, 0.02 to 3.4; P=0.173). There were no improvements in the proportion of prescription errors and no effects that could be attributed to SPI1 in non-targeted generic areas (such as enhanced safety culture). On some measures, the lack of effect could be because compliance was already high at baseline (such as use of steroids in over 85% of cases where indicated), but even when there was more room for improvement (such as in quality of medical history taking), there was no significant additional net effect of SPI1. There were no changes over time or between control and SPI1 hospitals in errors or rates of adverse events in patients in medical wards. Mortality increased from 11% (27) to 16% (39) among controls and decreased from 17% (63) to 13% (49) among SPI1 hospitals, but the risk adjusted difference was not significant (0.5, 0.2 to 1.4; P=0.085). Poor care was a contributing factor in four of the 178 deaths identified by review of case notes. The survey of patients showed no significant differences apart from an increase in perception of cleanliness in favour of SPI1 hospitals. Conclusions The introduction of SPI1 was associated with improvements in one of the types of clinical process studied (monitoring of vital signs) and one measure of staff perceptions of organisational climate. There was no additional effect of SPI1 on other targeted issues nor on other measures of generic organisational strengthening.

Multiple component patient safety intervention in English hospitals: controlled evaluation of second phase.

Objective To independently evaluate the impact of the second phase of the Health Foundation’s Safer Patients Initiative (SPI2) on a range of patient safety measures. Design A controlled before and after design. Five substudies: survey of staff attitudes; review of case notes from high risk (respiratory) patients in medical wards; review of case notes from surgical patients; indirect evaluation of hand hygiene by measuring hospital use of handwashing materials; measurement of outcomes (adverse events, mortality among high risk patients admitted to medical wards, patients’ satisfaction, mortality in intensive care, rates of hospital acquired infection). Setting NHS hospitals in England. Participants Nine hospitals participating in SPI2 and nine matched control hospitals. Intervention The SPI2 intervention was similar to the SPI1, with somewhat modified goals, a slightly longer intervention period, and a smaller budget per hospital. Results One of the scores (organisational climate) showed a significant (P=0.009) difference in rate of change over time, which favoured the control hospitals, though the difference was only 0.07 points on a five point scale. Results of the explicit case note reviews of high risk medical patients showed that certain practices improved over time in both control and SPI2 hospitals (and none deteriorated), but there were no significant differences between control and SPI2 hospitals. Monitoring of vital signs improved across control and SPI2 sites. This temporal effect was significant for monitoring the respiratory rate at both the six hour (adjusted odds ratio 2.1, 99% confidence interval 1.0 to 4.3; P=0.010) and 12 hour (2.4, 1.1 to 5.0; P=0.002) periods after admission. There was no significant effect of SPI for any of the measures of vital signs. Use of a recommended system for scoring the severity of pneumonia improved from 1.9% (1/52) to 21.4% (12/56) of control and from 2.0% (1/50) to 41.7% (25/60) of SPI2 patients. This temporal change was significant (7.3, 1.4 to 37.7; P=0.002), but the difference in difference was not significant (2.1, 0.4 to 11.1; P=0.236). There were no notable or significant changes in the pattern of prescribing errors, either over time or between control and SPI2 hospitals. Two items of medical history taking (exercise tolerance and occupation) showed significant improvement over time, across both control and SPI2 hospitals, but no additional SPI2 effect. The holistic review showed no significant changes in error rates either over time or between control and SPI2 hospitals. The explicit case note review of perioperative care showed that adherence rates for two of the four perioperative standards targeted by SPI2 were already good at baseline, exceeding 94% for antibiotic prophylaxis and 98% for deep vein thrombosis prophylaxis. Intraoperative monitoring of temperature improved over time in both groups, but this was not significant (1.8, 0.4 to 7.6; P=0.279), and there were no additional effects of SPI2. A dramatic rise in consumption of soap and alcohol hand rub was similar in control and SPI2 hospitals (P=0.760 and P=0.889, respectively), as was the corresponding decrease in rates of Clostridium difficile and meticillin resistant Staphylococcus aureus infection (P=0.652 and P=0.693, respectively). Mortality rates of medical patients included in the case note reviews in control hospitals increased from 17.3% (42/243) to 21.4% (24/112), while in SPI2 hospitals they fell from 10.3% (24/233) to 6.1% (7/114) (P=0.043). Fewer than 8% of deaths were classed as avoidable; changes in proportions could not explain the divergence of overall death rates between control and SPI2 hospitals. There was no significant difference in the rate of change in mortality in intensive care. Patients’ satisfaction improved in both control and SPI2 hospitals on all dimensions, but again there were no significant changes between the two groups of hospitals. Conclusions Many aspects of care are already good or improving across the NHS in England, suggesting considerable improvements in quality across the board. These improvements are probably due to contemporaneous policy activities relating to patient safety, including those with features similar to the SPI, and the emergence of professional consensus on some clinical processes. This phenomenon might have attenuated the incremental effect of the SPI, making it difficult to detect. Alternatively, the full impact of the SPI might be observable only in the longer term. The conclusion of this study could have been different if concurrent controls had not been used.

Weekend admission to hospital has a higher risk of death in the elective setting than in the emergency setting: a retrospective database study of national health service hospitals in England

BackgroundAlthough acute hospitals offer a twenty-four hour seven day a week service levels of staffing are lower over the weekends and some health care processes may be less readily available over the weekend. Whilst it is thought that emergency admission to hospital on the weekend is associated with an increased risk of death, the extent to which this applies to elective admissions is less well known. We investigated the risk of death in elective and elective patients admitted over the weekend versus the weekdays.MethodsRetrospective statistical analysis of routinely collected acute hospital admissions in England, involving all patient discharges from all acute hospitals in England over a year (April 2008-March 2009), using a logistic regression model which adjusted for a range of patient case-mix variables, seasonality and admission over a weekend separately for elective and emergency (but excluding zero day stay emergency admissions discharged alive) admissions.ResultsOf the 1,535,267 elective admissions, 91.7% (1,407,705) were admitted on the weekday and 8.3% (127,562) were admitted on the weekend. The mortality following weekday admission was 0.52% (7,276/1,407,705) compared with 0.77% (986/127,562) following weekend admission. Of the 3,105,249 emergency admissions, 76.3% (2,369,316) were admitted on the weekday and 23.7% (735,933) were admitted on the weekend. The mortality following emergency weekday admission was 6.53% (154,761/2,369,316) compared to 7.06% (51,922/735,933) following weekend admission. After case-mix adjustment, weekend admissions were associated with an increased risk of death, especially in the elective setting (elective Odds Ratio: 1.32, 95% Confidence Interval 1.23 to 1.41); vs emergency Odds Ratio: 1.09, 95% Confidence Interval 1.05 to 1.13).ConclusionsWeekend admission appears to be an independent risk factor for dying in hospital and this risk is more pronounced in the elective setting. Given the planned nature of elective admissions, as opposed to the unplanned nature of emergency admissions, it would seem less likely that this increased risk in the elective setting is attributable to unobserved patient risk factors. Further work to understand the relationship between weekend processes of care and mortality, especially in the elective setting, is required.

Weekend specialist intensity and admission mortality in acute hospital trusts in England: a cross-sectional study

Summary Background Increased mortality rates associated with weekend hospital admission (the so-called weekend effect) have been attributed to suboptimum staffing levels of specialist consultants. However, evidence for a causal association is elusive, and the magnitude of the weekend specialist deficit remains unquantified. This uncertainty could hamper efforts by national health systems to introduce 7 day health services. We aimed to examine preliminary associations between specialist intensity and weekend admission mortality across the English National Health Service. Methods Eligible hospital trusts were those in England receiving unselected emergency admissions. On Sunday June 15 and Wednesday June 18, 2014, we undertook a point prevalence survey of hospital specialists (consultants) to obtain data relating to the care of patients admitted as emergencies. We defined specialist intensity at each trust as the self-reported estimated number of specialist hours per ten emergency admissions between 0800 h and 2000 h on Sunday and Wednesday. With use of data for all adult emergency admissions for financial year 2013–14, we compared weekend to weekday admission risk of mortality with the Sunday to Wednesday specialist intensity ratio within each trust. We stratified trusts by size quintile. Findings 127 of 141 eligible acute hospital trusts agreed to participate; 115 (91%) trusts contributed data to the point prevalence survey. Of 34 350 clinicians surveyed, 15 537 (45%) responded. Substantially fewer specialists were present providing care to emergency admissions on Sunday (1667 [11%]) than on Wednesday (6105 [42%]). Specialists present on Sunday spent 40% more time caring for emergency patients than did those present on Wednesday (mean 5·74 h [SD 3·39] vs 3·97 h [3·31]); however, the median specialist intensity on Sunday was only 48% (IQR 40–58) of that on Wednesday. The Sunday to Wednesday intensity ratio was less than 0·7 in 104 (90%) of the contributing trusts. Mortality risk among patients admitted at weekends was higher than among those admitted on weekdays (adjusted odds ratio 1·10, 95% CI 1·08–1·11; p<0·0001). There was no significant association between Sunday to Wednesday specialist intensity ratios and weekend to weekday mortality ratios (r −0·042; p=0·654). Interpretation This cross-sectional analysis did not detect a correlation between weekend staffing of hospital specialists and mortality risk for emergency admissions. Further investigation is needed to evaluate whole-system secular change during the implementation of 7 day services. Policy makers should exercise caution before attributing the weekend effect mainly to differences in specialist staffing. Funding National Institute for Health Research Health Services and Delivery Research Programme.

A study of childhood attendance at emergency departments in the West Midlands region

Introduction: Research into childhood attendance at EDs in the UK has focused mainly on injury rather than medical conditions and studies have been relatively small. This study looks at all types of ED attendance by children across a large population. Data and methods: Routine data on all new attendances by children under 16 years were available for 12 EDs in the West Midlands (period: 1 April 2002 to 31 March 2004, 365 695 records). The data were split into four age groups (<1, 1–4, 5–9, and 10–15 years). Results: Injury related conditions increased with age (with the exception of head injury). Respiratory and gastrointestinal were the most common medical conditions decreased with age. 11.5% of children were admitted to hospital and this varied from 8.2% (10–15 years) to 24.2% (<1 year). Conclusions: This study has shown substantial variations in ED attendance by age and has given an insight into the variation among hospitals. This is the largest study of childhood ED attendance undertaken in the UK, and it is hoped that the questions raised will prompt more research in this field.

The combined influence of distance and neighbourhood deprivation on Emergency Department attendance in a large English population: a retrospective database study.

The frequency of visits to Emergency Departments (ED) varies greatly between populations. This may reflect variation in patient behaviour, need, accessibility, and service configuration as well as the complex interactions between these factors. This study investigates the relationship between distance, socio-economic deprivation, and proximity to an alternative care setting (a Minor Injuries Unit (MIU)), with particular attention to the interaction between distance and deprivation. It is set in a population of approximately 5.4 million living in central England, which is highly heterogeneous in terms of ethnicity, socio-economics, and distance to hospital. The study data set captured 1,413,363 ED visits made by residents of the region to National Health Service (NHS) hospitals during the financial year 2007/8. Our units of analysis were small units of census geography having an average population of 1,545. Separate regression models were made for children and adults. For each additional kilometre of distance from a hospital, predicted child attendances fell by 2.2% (1.7%–2.6% p<0.001) and predicted adult attendances fell by 1.5% (1.2% –1.8%, p<0.001). Compared to the least deprived quintile, attendances in the most deprived quintile more than doubled for children (incident rate ratio (IRR)  = 2.19, (1.90–2.54, p<0.001)) and adults (IRR 2.26, (2.01–2.55, p<0.001)). Proximity of an MIU was significant and both adult and child attendances were greater in populations who lived further away from them, suggesting that MIUs may reduce ED demand. The interaction between distance and deprivation was significant. Attendance in deprived neighbourhoods reduces with distance to a greater degree than in less deprived ones for both adults and children. In conclusion, ED use is related to both deprivation and distance, but the effect of distance is modified by deprivation.

Which is more useful in predicting hospital mortality - dichotomised blood test results or actual test values? a retrospective study in two hospitals

Background Routine blood tests are an integral part of clinical medicine and in interpreting blood test results clinicians have two broad options. (1) Dichotomise the blood tests into normal/abnormal or (2) use the actual values and overlook the reference values. We refer to these as the “binary” and the “non-binary” strategy respectively. We investigate which strategy is better at predicting the risk of death in hospital based on seven routinely undertaken blood tests (albumin, creatinine, haemoglobin, potassium, sodium, urea, and white blood cell count) using tree models to implement the two strategies. Methodology A retrospective database study of emergency admissions to an acute hospital during April 2009 to March 2010, involving 10,050 emergency admissions with routine blood tests undertaken within 24 hours of admission. We compared the area under the Receiver Operating Characteristics (ROC) curve for predicting in-hospital mortality using the binary and non-binary strategy. Results The mortality rate was 6.98% (701/10050). The mean predicted risk of death in those who died was significantly (p-value <0.0001) lower using the binary strategy (risk = 0.181 95%CI: 0.193 to 0.210) versus the non-binary strategy (risk = 0.222 95%CI: 0.194 to 0.251), representing a risk difference of 28.74 deaths in the deceased patients (n = 701). The binary strategy had a significantly (p-value <0.0001) lower area under the ROC curve of 0.832 (95% CI: 0.819 to 0.845) versus the non-binary strategy (0.853 95% CI: 0.840 to 0.867). Similar results were obtained using data from another hospital. Conclusions Dichotomising routine blood test results is less accurate in predicting in-hospital mortality than using actual test values because it underestimates the risk of death in patients who died. Further research into the use of actual blood test values in clinical decision making is required especially as the infrastructure to implement this potentially promising strategy already exists in most hospitals.

Does population mixing measure infectious exposure in children at the community level

Epidemiological studies focusing on the etiology of childhood chronic diseases have used population mixing as a proxy for the level of infection circulating in a community. We compared different measures of population mixing (based on residential migration and commuting) and other demographic variables, derived from the United Kingdom Census, with hospital inpatient data on infections from two Government Office Regions in England (Eastern and the West Midlands) to inform the development of an infectious disease proxy for future epidemiological studies. The association between rates of infection and the population mixing measures was assessed, using incidence rate ratios across census areas, from negative binomial regression. Commuting distance demonstrated the most consistent association with admissions for infections across the two regions; areas with a higher median distance travelled by commuters leaving the area having a lower rate of hospital admissions for infections. Deprived areas and densely populated areas had a raised rate of admissions for infections. Assuming hospital admissions are a reliable indicator of common infection rates, the results from this study suggest that commuting distance is a consistent measure of population mixing in relation to infectious disease and deprivation and population density are reliable demographic proxies for infectious exposure. Areas that exhibit high levels of population mixing do not necessarily possess raised rates of hospital admissions for infectious disease.

Index Blood Tests and National Early Warning Scores within 24 Hours of Emergency Admission Can Predict the Risk of In-Hospital Mortality: A Model Development and Validation Study

Background We explored the use of routine blood tests and national early warning scores (NEWS) reported within ±24 hours of admission to predict in-hospital mortality in emergency admissions, using empirical decision Tree models because they are intuitive and may ultimately be used to support clinical decision making. Methodology A retrospective analysis of adult emergency admissions to a large acute hospital during April 2009 to March 2010 in the West Midlands, England, with a full set of index blood tests results (albumin, creatinine, haemoglobin, potassium, sodium, urea, white cell count and an index NEWS undertaken within ±24 hours of admission). We developed a Tree model by randomly splitting the admissions into a training (50%) and validation dataset (50%) and assessed its accuracy using the concordance (c-) statistic. Emergency admissions (about 30%) did not have a full set of index blood tests and/or NEWS and so were not included in our analysis. Results There were 23248 emergency admissions with a full set of blood tests and NEWS with an in-hospital mortality of 5.69%. The Tree model identified age, NEWS, albumin, sodium, white cell count and urea as significant (p<0.001) predictors of death, which described 17 homogeneous subgroups of admissions with mortality ranging from 0.2% to 60%. The c-statistic for the training model was 0.864 (95%CI 0.852 to 0.87) and when applied to the testing data set this was 0.853 (95%CI 0.840 to 0.866). Conclusions An easy to interpret validated risk adjustment Tree model using blood test and NEWS taken within ±24 hours of admission provides good discrimination and offers a novel approach to risk adjustment which may potentially support clinical decision making. Given the nature of the clinical data, the results are likely to be generalisable but further research is required to investigate this promising approach.