Claudia Fischer

Is the readmission rate a valid quality indicator? A review of the evidence

Introduction Hospital readmission rates are increasingly used for both quality improvement and cost control. However, the validity of readmission rates as a measure of quality of hospital care is not evident. We aimed to give an overview of the different methodological aspects in the definition and measurement of readmission rates that need to be considered when interpreting readmission rates as a reflection of quality of care. Methods We conducted a systematic literature review, using the bibliographic databases Embase, Medline OvidSP, Web-of-Science, Cochrane central and PubMed for the period of January 2001 to May 2013. Results The search resulted in 102 included papers. We found that definition of the context in which readmissions are used as a quality indicator is crucial. This context includes the patient group and the specific aspects of care of which the quality is aimed to be assessed. Methodological flaws like unreliable data and insufficient case-mix correction may confound the comparison of readmission rates between hospitals. Another problem occurs when the basic distinction between planned and unplanned readmissions cannot be made. Finally, the multi-faceted nature of quality of care and the correlation between readmissions and other outcomes limit the indicators validity. Conclusions Although readmission rates are a promising quality indicator, several methodological concerns identified in this study need to be addressed, especially when the indicator is intended for accountability or pay for performance. We recommend investing resources in accurate data registration, improved indicator description, and bundling outcome measures to provide a more complete picture of hospital care.

Health services research related to performance indicators and benchmarking in Europe

Objective: Measuring quality of care through performance indicators and subsequently using these to compare, learn, and improve (benchmarking) has become a central component of health care policy. This paper aims to identify the main themes of health services research in this area and focuses on opportunities for improving the evidence underpinning performance indicators. Methods: A literature survey was carried out to identify research activities and main research themes in Europe in the years 2000-09. Identified literature was categorized into sub-topics and for each topic the main methodological issues were identified and discussed. Experts validated the findings and explored the potential for related further European research. Results: The distribution of research on performance and benchmarking across EU member states varies in time, scope and settings with a large amount of studies focusing on hospitals. Eight specific fields of research were identified (research on concepts and performance frameworks; performance indicators and benchmarking using mortality data; performance indicators and benchmarking related to cancer care; performance indicators and benchmarking on care delivered in hospitals; patient safety indicators; performance indicators in primary care; patient experience; research on the practice of benchmarking and performance improvement). Expert discussions confirmed that research on performance indicators and benchmarking should focus on the development of indicators, as well as their use. The research should involve the potential users and incorporate scientific approaches from biomedicine and epidemiology as well as the social sciences. Further progress is hampered by data availability. Issues which need to be addressed include the use of unique patient identifiers (UPIs) to facilitate linkages between separate databases; standardized measurement of the experiences of patients and others; and deepening collaboration between Eurostat, the World Health Organization (WHO), and the Organization for Economic Co-operation and Development (OECD) to facilitate the availability of internationally comparable performance information. Conclusions: This study suggests a number of themes for future research. These include testing and improving: the validity and reliability of performance indicators, especially related to avoidable mortality and other outcome indicators; the effectiveness and efficiency of embedding performance indicators in the various governance, monitoring and management models, and their effect on health systems, services and professionals; and the effectiveness and efficiency of linking performance indicators to other national and international strategies and policies such as accreditation and certification, practice guidelines, audits, quality systems, patient safety strategies, national standards on volume and/or quality, public reporting, pay-for-performance and patient/consumer involvement. The field would benefit from strengthening the clearinghouse function for research findings, training of researchers and appropriate scientific publication media. Results should be systematically shared with policy-makers and managers, and networking stimulated between the growing number of regional and national institutes involved in quality measurement and reporting.

The validity of indicators for assessing quality of care: a review of the European literature on hospital readmission rate

BACKGROUND Quality indicators are increasingly being implemented in Europe for policy and management purposes. Many of these indicators were initially developed and implemented in the USA. However, the suitability of directly adopting indicators that have been developed in a different health care system can be questioned. Therefore, we investigate the validity behind the readmission rate indicator in the European setting. METHODS A systematic literature study was conducted to identify the status of scientific research on the validity of this indicator (January 1999 and April 2010). Descriptive information as well as information on the data source, indicator definition, risk adjustment factors, and conclusions was assessed. RESULTS The majority of the 486 included studies focused on the actual use of the indicator as an outcome measure in European countries. Only 21 studies specifically addressed its validity, or important prerequisites of validity. There is little consensus over the time-frame used to calculate the indicator, the type of readmission that is included, and the case-mix adjustment applied. CONCLUSIONS Despite the increase in Europe of the use of the readmission rate as a measure of quality of care, the amount of research performed on its validity is scarce. Those studies that report on validity replicate earlier, mainly US findings (<1999) of methodological problems and express reservations on its large-scale use. The readmission rate as an indicator should be used with care. Users should address issues related to definition, time-frame and case-mix adjustment as part of the process to enhance validity in the European settings.

Differences in fruit and vegetable intake and their determinants among 11-year-old schoolchildren between 2003 and 2009

BackgroundFruit and vegetable (FV) intake in children in the Netherlands is much lower than recommended. Recurrent appraisal of intake levels is important for detecting changes in intake over time and to inform future interventions and policies. The aim of the present study was to investigate differences in fruit and vegetable intake, and whether these could be explained by differences in potential determinants of FV intake in 11-year-old Dutch schoolchildren, by comparing two school samples assessed in 2003 and 2009.MethodsFor 1105 children of the Pro Children study in 2003 and 577 children of the Pro Greens study in 2009 complete data on intake and behavioural determinants were available. The self-administered questionnaire included questions on childrens ethnicity, usual fruit and vegetable intake, mothers educational level, and important potential determinants of fruit and vegetable intake.Multiple regression analysis was applied to test for differences in intake and determinants between study samples. Mediation analyses were used to investigate whether the potential mediators explained the differences in intake between the two samples.ResultsIn 2009, more children complied with the World Health Organization recommendation of 400 g fruit and vegetables per day (17.0%) than in 2003 (11.8%, p = 0.004). Fruit consumption was significantly higher in the sample of 2009 than in the sample of 2003 (difference = 23.8 (95%CI: 8.1; 39.5) grams/day). This difference was mainly explained by a difference in the parental demand regarding their childs intake (23.6%), followed by the childs knowledge of the fruit recommendation (14.2%) and parental facilitation of consumption (18.5%). Vegetable intake was lower in the 2009 sample than in the 2003 sample (12.3 (95%CI -21.0; -3.6). This difference could not be explained by the assessed mediators.ConclusionsThe findings indicate that fruit intake among 11-year-olds improved somewhat between 2003 and 2009. Vegetable intake, however, appears to have declined somewhat between 2003 and 2009. Since a better knowledge of the recommendation, parental demand and facilitation explained most of the observed fruit consumption difference, future interventions may specifically address these potential mediators. Further, the provision of vegetables in the school setting should be considered in order to increase childrens vegetable intake.

Risk adjustment models for short-term outcomes after surgical resection for oesophagogastric cancer.

Outcomes for oesophagogastric cancer surgery are compared with the aim of benchmarking quality of care. Adjusting for patient characteristics is crucial to avoid biased comparisons between providers. The study objective was to develop a case‐mix adjustment model for comparing 30‐ and 90‐day mortality and anastomotic leakage rates after oesophagogastric cancer resections.

Influences of hospital information systems, indicator data collection and computation on reported Dutch hospital performance indicator scores

BackgroundFor health care performance indicators (PIs) to be reliable, data underlying the PIs are required to be complete, accurate, consistent and reproducible. Given the lack of regulation of the data-systems used in the Netherlands, and the self-report based indicator scores, one would expect heterogeneity with respect to the data collection and the ways indicators are computed. This might affect the reliability and plausibility of the nationally reported scores.MethodsWe aimed to investigate the extent to which local hospital data collection and indicator computation strategies differ and how this affects the plausibility of self-reported indicator scores, using survey results of 42 hospitals and data of the Dutch national quality database.ResultsThe data collection and indicator computation strategies of the hospitals were substantially heterogenic. Moreover, the Hip and Knee replacement PI scores can be regarded as largely implausible, which was, to a great extent, related to a limited (computerized) data registry. In contrast, Breast Cancer PI scores were more plausible, despite the incomplete data registry and limited data access. This might be explained by the role of the regional cancer centers that collect most of the indicator data for the national cancer registry, in a standardized manner. Hospitals can use cancer registry indicator scores to report to the government, instead of their own locally collected indicator scores.ConclusionsIndicator developers, users and the scientific field need to focus more on the underlying (heterogenic) ways of data collection and conditional data infrastructures. Countries that have a liberal software market and are aiming to implement a self-report based performance indicator system to obtain health care transparency, should secure the accuracy and precision of the heath care data from which the PIs are calculated. Moreover, ongoing research and development of PIs and profound insight in the clinical practice of data registration is warranted.

Comparing colon cancer outcomes: The impact of low hospital case volume and case-mix adjustment.

OBJECTIVE When comparing performance across hospitals it is essential to consider the noise caused by low hospital case volume and to perform adequate case-mix adjustment. We aimed to quantify the role of noise and case-mix adjustment on standardized postoperative mortality and anastomotic leakage (AL) rates. METHODS We studied 13,120 patients who underwent colon cancer resection in 85 Dutch hospitals. We addressed differences between hospitals in postoperative mortality and AL, using fixed (ignoring noise) and random effects (incorporating noise) logistic regression models with general and additional, disease specific, case-mix adjustment. RESULTS Adding disease specific variables improved the performance of the case-mix adjustment models for postoperative mortality (c-statistic increased from 0.77 to 0.81). The overall variation in standardized mortality ratios was similar, but some individual hospitals changed considerably. For the standardized AL rates the performance of the adjustment models was poor (c-statistic 0.59 and 0.60) and overall variation was small. Most of the observed variation between hospitals was actually noise. CONCLUSION Noise had a larger effect on hospital performance than extended case-mix adjustment, although some individual hospital outcome rates were affected by more detailed case-mix adjustment. To compare outcomes between hospitals it is crucial to consider noise due to low hospital case volume with a random effects model.

Using quality measures for quality improvement: the perspective of hospital staff.

Research objective This study examines the perspectives of a range of key hospital staff on the use, importance, scientific background, availability of data, feasibility of data collection, cost benefit aspects and availability of professional personnel for measurement of quality indicators among Iranian hospitals. The study aims to facilitate the use of quality indicators to improve quality of care in hospitals. Study design A cross-sectional study was conducted over the period 2009 to 2010. Staff at Iranian hospitals completed a self-administered questionnaire eliciting their views on organizational, clinical process, and outcome (clinical effectiveness, patient safety and patient centeredness) indicators. Population studied 93 hospital frontline staff including hospital/nursing managers, medical doctors, nurses, and quality improvement/medical records officers in 48 general and specialized hospitals in Iran. Principal findings On average, only 69% of respondents reported using quality indicators in practice at their affiliated hospitals. Respondents varied significantly in their reported use of organizational, clinical process and outcome quality indicators. Overall, clinical process and effectiveness indicators were reported to be least used. The reported use of indicators corresponded with their perceived level of importance. Quality indicators were reported to be used among clinical staff significantly more than among managerial staff. In total, 74% of the respondents reported to use obligatory indicators, while this was 68% for voluntary indicators (p<0.05). Conclusions There is a general awareness of the importance and usability of quality indicators among hospital staff in Iran, but their use is currently mostly directed towards external accountability purposes. To increase the formative use of quality indicators, creation of a common culture and feeling of shared ownership, alongside an increased uptake of clinical process and effectiveness indicators is needed to support internal quality improvement processes at hospital level.

Volume-outcome revisited: The effect of hospital and surgeon volumes on multiple outcome measures in oesophago-gastric cancer surgery

Background Most studies showing a volume outcome effect in resection surgery for oesophago-gastric cancer were conducted before the centralisation of clinical services. This study evaluated the relation between hospital- and surgeon volume and different risk-adjusted outcomes after oesophago-gastric (OG) cancer surgery in England between 2011 and 2013. Methods In data from the National Oesophago-Gastric Cancer Audit from the UK, multivariable random-effects logistic regression models were used to quantify the effect of surgeon and hospital volume on three outcomes: 30-day and 90-day mortality and anastomotic leakage. The models included patient risk factors to adjust for differences in case-mix among hospitals and surgeons. The between-cluster heterogeneity was estimated with the median odds ratio (MOR). Results The study included patients treated at 42 hospitals and 329 surgeons. The median (interquartile range) of the annual hospital and surgeon volumes were 110 patients (82 to 137) and 13 patients (8 to 19), respectively. The overall rates for 30-day and 90-day mortality were 2.3% and 4.4% respectively, and the anastomotic leakage was 6.3%. Higher hospital volume was associated with lower 30-day mortality (OR: 0.94; 95% CI: 0.91–0.98) and lower anastomotic leakage rates (OR: 0.96; 95% CI: 0.93–0.98) but not 90-day mortality. Higher surgeon volume was only associated with lower anastomotic leakage rates (OR: 0.81; 95% CI: 0.72–0.92). Hospital volume explained a part of the between-hospital variation in 30-day mortality whereas surgeon volume explained part of the between-hospital variation in anastomotic leakage. Conclusions In the setting of centralized O-G cancer surgery in England, we could still observe an effect of volume on short-term outcomes. However, the effect is inconsistent, depending on the type of outcome measure under consideration, and much smaller than in previous studies. Efforts to centralise O-G cancer services further should carefully address the effects of both hospital and surgeon volume on the range of outcome measures that are relevant to patients.

Are performance indicators used for hospital quality management: a qualitative interview study amongst health professionals and quality managers in The Netherlands

BackgroundHospitals are under increasing pressure to share indicator-based performance information. These indicators can also serve as a means to promote quality improvement and boost hospital performance. Our aim was to explore hospitals’ use of performance indicators for internal quality management activities.MethodsWe conducted a qualitative interview study among 72 health professionals and quality managers in 14 acute care hospitals in The Netherlands. Concentrating on orthopaedic and oncology departments, our goal was to gain insight into data collection and use of performance indicators for two conditions: knee and hip replacement surgery and breast cancer surgery. The semi-structured interviews were recorded and summarised. Based on the data, themes were synthesised and the analyses were executed systematically by two analysts independently. The findings were validated through comparison.ResultsThe hospitals we investigated collect data for performance indicators in different ways. Similarly, these hospitals have different ways of using such data to support their quality management, while some do not seem to use the data for this purpose at all. Factors like ‘linking pin champions’, pro-active quality managers and engaged medical specialists seem to make a difference. In addition, a comprehensive hospital data infrastructure with electronic patient records and robust data collection software appears to be a prerequisite to produce reliable external performance indicators for internal quality improvement.ConclusionsHospitals often fail to use performance indicators as a means to support internal quality management. Such data, then, are not used to its full potential. Hospitals are recommended to focus their human resource policy on ‘linking pin champions’, the engagement of professionals and a pro-active quality manager, and to invest in a comprehensive data infrastructure. Furthermore, the differences in data collection processes between Dutch hospitals make it difficult to draw comparisons between outcomes of performance indicators.