Jon Helgeland

Comparing hospital mortality – how to count does matter for patients hospitalized for acute myocardial infarction (AMI), stroke and hip fracture

BackgroundMortality is a widely used, but often criticised, quality indicator for hospitals. In many countries, mortality is calculated from in-hospital deaths, due to limited access to follow-up data on patients transferred between hospitals and on discharged patients. The objectives were to: i) summarize time, place and cause of death for first time acute myocardial infarction (AMI), stroke and hip fracture, ii) compare case-mix adjusted 30-day mortality measures based on in-hospital deaths and in-and-out-of hospital deaths, with and without patients transferred to other hospitals.MethodsNorwegian hospital data within a 5-year period were merged with information from official registers. Mortality based on in-and-out-of-hospital deaths, weighted according to length of stay at each hospital for transferred patients (W30D), was compared to a) mortality based on in-and-out-of-hospital deaths excluding patients treated at two or more hospitals (S30D), and b) mortality based on in-hospital deaths (IH30D). Adjusted mortalities were estimated by logistic regression which, in addition to hospital, included age, sex and stage of disease. The hospitals were assigned outlier status according to the Z-values for hospitals in the models; low mortality: Z-values below the 5-percentile, high mortality: Z-values above the 95-percentile, medium mortality: remaining hospitals.ResultsThe data included 48 048 AMI patients, 47 854 stroke patients and 40 142 hip fracture patients from 55, 59 and 58 hospitals, respectively. The overall relative frequencies of deaths within 30 days were 19.1% (AMI), 17.6% (stroke) and 7.8% (hip fracture). The cause of death diagnoses included the referral diagnosis for 73.8-89.6% of the deaths within 30 days. When comparing S30D versus W30D outlier status changed for 14.6% (AMI), 15.3% (stroke) and 36.2% (hip fracture) of the hospitals. For IH30D compared to W30D outlier status changed for 18.2% (AMI), 25.4% (stroke) and 27.6% (hip fracture) of the hospitals.ConclusionsMortality measures based on in-hospital deaths alone, or measures excluding admissions for transferred patients, can be misleading as indicators of hospital performance. We propose to attribute the outcome to all hospitals by fraction of time spent in each hospital for patients transferred between hospitals to reduce bias due to double counting or exclusion of hospital stays.

Survival curves to support quality improvement in hospitals with excess 30-day mortality after acute myocardial infarction, cerebral stroke and hip fracture: a before-after study.

Objectives To evaluate survival curves (Kaplan-Meier) as a means of identifying areas in the clinical pathway amenable to quality improvement. Design Observational before–after study. Setting In Norway, annual public reporting of nationwide 30-day in-and-out-of-hospital mortality (30D) for three medical conditions started in 2011: first time acute myocardial infarction (AMI), stroke and hip fracture; reported for 2009. 12 of 61 hospitals had statistically significant lower/higher mortality compared with the hospital mean. Participants Three hospitals with significantly higher mortality requested detailed analyses for quality improvement purposes: Telemark Hospital Trust Skien (AMI and stroke), Østfold Hospital Trust Fredrikstad (stroke), Innlandet Hospital Trust Gjøvik (hip fracture). Outcome measures Survival curves, crude and risk-adjusted 30D before (2008–2009) and after (2012–2013). Interventions Unadjusted survival curves for the outlier hospitals were compared to curves based on pooled data from the other hospitals for the 30-day period 2008–2009. For patients admitted with AMI (Skien), stroke (Fredrikstad) and hip fracture (Gjøvik), the curves suggested increased mortality from the initial part of the clinical pathway. For stroke (Skien), increased mortality appeared after about 8 days. The curve profiles were thought to reflect suboptimal care in various phases in the clinical pathway. This informed improvement efforts. Results For 2008–2009, hospital-specific curves differed from other hospitals: borderline significant for AMI (p=0.064), highly significant (p≤0.005) for the remainder. After intervention, no difference was found (p>0.188). Before–after comparison of the curves within each hospital revealed a significant change for Fredrikstad (p=0.006). For the three hospitals, crude 30D declined and they were non-outliers for risk-adjusted 30D for 2013. Conclusions Survival curves as a supplement to 30D may be useful for identifying suboptimal care in the clinical pathway, and thus informing design of quality improvement projects.

An observational study: associations between nurse-reported hospital characteristics and estimated 30-day survival probabilities

Background There is a growing body of evidence for associations between the work environment and patient outcomes. A good work environment may maximise healthcare workers’ efforts to avoid failures and to facilitate quality care that is focused on patient safety. Several studies use nurse-reported quality measures, but it is uncertain whether these outcomes are correlated with clinical outcomes. The aim of this study was to determine the correlations between hospital-aggregated, nurse-assessed quality and safety, and estimated probabilities for 30-day survival in and out of hospital. Methods In a multicentre study involving almost all Norwegian hospitals with more than 85 beds (sample size=30, information about nurses’ perceptions of organisational characteristics were collected. Subscales from this survey were used to describe properties of the organisations: quality system, patient safety management, nurse–physician relationship, staffing adequacy, quality of nursing and patient safety. The average scores for these organisational characteristics were aggregated to hospital level, and merged with estimated probabilities for 30-day survival in and out of hospital (survival probabilities) from a national database. In this observational, ecological study, the relationships between the organisational characteristics (independent variables) and clinical outcomes (survival probabilities) were examined. Results Survival probabilities were correlated with nurse-assessed quality of nursing. Furthermore, the subjective perception of staffing adequacy was correlated with overall survival. Conclusions This study showed that perceived staffing adequacy and nurses’ assessments of quality of nursing were correlated with survival probabilities. It is suggested that the way nurses characterise the microsystems they belong to, also reflects the general performance of hospitals.

30-Day Survival Probabilities as a Quality Indicator for Norwegian Hospitals: Data Management and Analysis.

Background The Norwegian Knowledge Centre for the Health Services (NOKC) reports 30-day survival as a quality indicator for Norwegian hospitals. The indicators have been published annually since 2011 on the website of the Norwegian Directorate of Health (www.helsenorge.no), as part of the Norwegian Quality Indicator System authorized by the Ministry of Health. Openness regarding calculation of quality indicators is important, as it provides the opportunity to critically review and discuss the method. The purpose of this article is to describe the data collection, data pre-processing, and data analyses, as carried out by NOKC, for the calculation of 30-day risk-adjusted survival probability as a quality indicator. Methods and Findings Three diagnosis-specific 30-day survival indicators (first time acute myocardial infarction (AMI), stroke and hip fracture) are estimated based on all-cause deaths, occurring in-hospital or out-of-hospital, within 30 days counting from the first day of hospitalization. Furthermore, a hospital-wide (i.e. overall) 30-day survival indicator is calculated. Patient administrative data from all Norwegian hospitals and information from the Norwegian Population Register are retrieved annually, and linked to datasets for previous years. The outcome (alive/death within 30 days) is attributed to every hospital by the fraction of time spent in each hospital. A logistic regression followed by a hierarchical Bayesian analysis is used for the estimation of risk-adjusted survival probabilities. A multiple testing procedure with a false discovery rate of 5% is used to identify hospitals, hospital trusts and regional health authorities with significantly higher/lower survival than the reference. In addition, estimated risk-adjusted survival probabilities are published per hospital, hospital trust and regional health authority. The variation in risk-adjusted survival probabilities across hospitals for AMI shows a decreasing trend over time: estimated survival probabilities for AMI in 2011 varied from 80.6% (in the hospital with lowest estimated survival) to 91.7% (in the hospital with highest estimated survival), whereas it ranged from 83.8% to 91.2% in 2013. Conclusions Since 2011, several hospitals and hospital trusts have initiated quality improvement projects, and some of the hospitals have improved the survival over these years. Public reporting of survival/mortality indicators are increasingly being used as quality measures of health care systems. Openness regarding the methods used to calculate the indicators are important, as it provides the opportunity of critically reviewing and discussing the methods in the literature. In this way, the methods employed for establishing the indicators may be improved.

Classifying nursing organization in wards in Norwegian hospitals: self-identification versus observation

BackgroundThe organization of nursing services could be important to the quality of patient care and staff satisfaction. However, there is no universally accepted nomenclature for this organization. The objective of the current study was to classify general hospital wards based on data describing organizational practice reported by the ward nurse managers, and then to compare this classification with the name used in the wards to identify the organizational model (self-identification).MethodsIn a cross-sectional postal survey, 93 ward nurse managers in Norwegian hospitals responded to questions about nursing organization in their wards, and what they called their organizational models. K-means cluster analysis was used to classify the wards according to the pattern of activities attributed to the different nursing roles and discriminant analysis was used to interpret the solutions. Cross-tabulation was used to validate the solutions and to compare the classification obtained from the cluster analysis with that obtained by self-identification. The bootstrapping technique was used to assess the generalizability of the cluster solution.ResultsThe cluster analyses produced two alternative solutions using two and three clusters, respectively. The three-cluster solution was considered to be the best representation of the organizational models: 32 team leader-dominated wards, 23 primary nurse-dominated wards and 38 wards with a hybrid or mixed organization. There was moderate correspondence between the three-cluster solution and the models obtained by self-identification. Cross-tabulation supported the empirical classification as being representative for variations in nursing service organization. Ninety-four per cent of the bootstrap replications showed the same pattern as the cluster solution in the study sample.ConclusionsA meaningful classification of wards was achieved through an empirical cluster solution; this was, however, only moderately consistent with the self-identification. This empirical classification is an objective approach to variable construction and can be generally applied across Norwegian hospitals. The classification procedure used in the study could be developed into a standardized method for classifying hospital wards across health systems and over time.

Agreement between referral information and discharge diagnoses according to Norwegian elective treatment guidelines - a cross-sectional study

BackgroundNorway introduced 32 priority guidelines for elective health treatment in the specialist health service in the period 2008-9. The guidelines were intended to reduce large differences in waiting times among hospitals, streamline referrals and ensure that patients accessed the necessary healthcare to which they were entitled for certain conditions. Referral information guided the priorities. As the referral information was key to future evaluation of the guidelines, this study validates the referral information in hospital patient records against discharge diagnoses, because only the discharge diagnosis is recorded in the Norwegian Patient Register (NPR) database, which is used in the main evaluation.MethodsOf the specific conditions from 10 priority guidelines, 20 were selected for review for the period 2008-9 at 4 hospitals in Norway. The ICD-10 diagnoses per disease or condition were given in retrospect by clinicians who participated in the expert groups developing the priority guidelines. Reasons for deviations between referral information and discharge diagnoses were coded into four categories, according to the degree of precision of the former compared with the latter.ResultsIn all, 1854 medical records were available for review. The diagnostic precision of the referrals differed significantly between hospitals, and across the 2 years 2008 and 2009. The overall sensitivity was 0.93 (95% confidence interval 0.92-0.94). For the separate conditions, sensitivity was in the range 0.60-1.00. Experience showed that it was necessary to pay careful attention to the selection of ICD-10 diagnoses for identifying patients. The medical records of psychiatry patients were unavailable in some cases and for certain conditions some were unavailable after use of our record extraction algorithm.ConclusionThe sensitivity of the referral information on diagnosis or condition was high compared with the discharge diagnosis for the 20 selected conditions from the 10 priority guidelines. Although the review assessed a limited number of the total, we consider the results sufficiently representative and, hence, they will allow use of the NPR data for analyses of the introduction and follow-up of the 32 priority guidelines.

Observed to expected or logistic regression to identify hospitals with high or low 30-day mortality?

Introduction A common quality indicator for monitoring and comparing hospitals is based on death within 30 days of admission. An important use is to determine whether a hospital has higher or lower mortality than other hospitals. Thus, the ability to identify such outliers correctly is essential. Two approaches for detection are: 1) calculating the ratio of observed to expected number of deaths (OE) per hospital and 2) including all hospitals in a logistic regression (LR) comparing each hospital to a form of average over all hospitals. The aim of this study was to compare OE and LR with respect to correctly identifying 30-day mortality outliers. Modifications of the methods, i.e., variance corrected approach of OE (OE-Faris), bias corrected LR (LR-Firth), and trimmed mean variants of LR and LR-Firth were also studied. Materials and methods To study the properties of OE and LR and their variants, we performed a simulation study by generating patient data from hospitals with known outlier status (low mortality, high mortality, non-outlier). Data from simulated scenarios with varying number of hospitals, hospital volume, and mortality outlier status, were analysed by the different methods and compared by level of significance (ability to falsely claim an outlier) and power (ability to reveal an outlier). Moreover, administrative data for patients with acute myocardial infarction (AMI), stroke, and hip fracture from Norwegian hospitals for 2012–2014 were analysed. Results None of the methods achieved the nominal (test) level of significance for both low and high mortality outliers. For low mortality outliers, the levels of significance were increased four- to fivefold for OE and OE-Faris. For high mortality outliers, OE and OE-Faris, LR 25% trimmed and LR-Firth 10% and 25% trimmed maintained approximately the nominal level. The methods agreed with respect to outlier status for 94.1% of the AMI hospitals, 98.0% of the stroke, and 97.8% of the hip fracture hospitals. Conclusion We recommend, on the balance, LR-Firth 10% or 25% trimmed for detection of both low and high mortality outliers.

Effects of external inspection on sepsis detection and treatment: A study protocol for a quasiexperimental study with a stepped-wedge design

Introduction Inspections are widely used in health care as a means to improve the health services delivered to patients. Despite their widespread use, there is little evidence of their effect. The mechanisms for how inspections can promote change are poorly understood. In this study, we use a national inspection campaign of sepsis detection and initial treatment in hospitals as case to: (1) Explore how inspections affect the involved organizations. (2) Evaluate what effect external inspections have on the process of delivering care to patients, measured by change in indicators reflecting how sepsis detection and treatment is carried out. (3) Evaluate whether external inspections affect patient outcomes, measured as change in the 30-day mortality rate and length of hospital stay. Methods and analysis The intervention that we study is inspections of sepsis detection and treatment in hospitals. The intervention will be rolled out sequentially during 12 months to 24 hospitals. Our effect measures are change on indicators related to the detection and treatment of sepsis, the 30-day mortality rate and length of hospital stay. We collect data from patient records at baseline, before the inspections, and at 8 and 14 months after the inspections. We use logistic regression models and linear regression models to compare the various effect measurements between the intervention and control periods. All the models will include time as a covariate to adjust for potential secular changes in the effect measurements during the study period. We collect qualitative data before and after the inspections, and we will conduct a thematic content analysis to explore how inspections affect the involved organisations. Ethics and dissemination The study has obtained ethical approval by the Regional Ethics Committee of Norway Nord and the Norwegian Data Protection Authority. It is registered at www.clinicaltrials.gov (Identifier: NCT02747121). Results will be reported in international peer-reviewed journals. Trial Registration NCT02747121; Pre-results.

Variation between Hospitals with Regard to Diagnostic Practice, Coding Accuracy, and Case-Mix. A Retrospective Validation Study of Administrative Data versus Medical Records for Estimating 30-Day Mortality after Hip Fracture

Background The purpose of this study was to assess the validity of patient administrative data (PAS) for calculating 30-day mortality after hip fracture as a quality indicator, by a retrospective study of medical records. Methods We used PAS data from all Norwegian hospitals (2005–2009), merged with vital status from the National Registry, to calculate 30-day case-mix adjusted mortality for each hospital (n = 51). We used stratified sampling to establish a representative sample of both hospitals and cases. The hospitals were stratified according to high, low and medium mortality of which 4, 3, and 5 hospitals were sampled, respectively. Within hospitals, cases were sampled stratified according to year of admission, age, length of stay, and vital 30-day status (alive/dead). The final study sample included 1043 cases from 11 hospitals. Clinical information was abstracted from the medical records. Diagnostic and clinical information from the medical records and PAS were used to define definite and probable hip fracture. We used logistic regression analysis in order to estimate systematic between-hospital variation in unmeasured confounding. Finally, to study the consequences of unmeasured confounding for identifying mortality outlier hospitals, a sensitivity analysis was performed. Results The estimated overall positive predictive value was 95.9% for definite and 99.7% for definite or probable hip fracture, with no statistically significant differences between hospitals. The standard deviation of the additional, systematic hospital bias in mortality estimates was 0.044 on the logistic scale. The effect of unmeasured confounding on outlier detection was small to moderate, noticeable only for large hospital volumes. Conclusions This study showed that PAS data are adequate for identifying cases of hip fracture, and the effect of unmeasured case mix variation was small. In conclusion, PAS data are adequate for calculating 30-day mortality after hip-fracture as a quality indicator in Norway.