Oliver Groene

An international review of projects on hospital performance assessment

BACKGROUND Assessing the quality of health care has become increasingly important in health care in response to growing demands from purchasers, providers, clinicians and the public. Given the increase in projects and programs to assess performance in health care in the last 15 years, the purpose of this paper is to review current indicator projects for hospital performance assessment and compare them to the Performance Assessment Tool for Quality Improvement in Hospitals (PATH), an initiative by the WHO Regional Office for Europe. METHODOLOGY We identified current indicator projects through a systematic literature search and through contact with experts. Using an inductive approach based on a review of the literature, we identified 10 criteria for the comparison of indicator projects. We extracted data and contacted the coordinators of each indicator project to validate this information. In addition, we carried out interviews with coordinators to gather additional information on the evaluation of the respective projects. RESULTS We included 11 projects that appear to have adopted a common methodology for the design and selection of indicators; however, major differences exist with regard to the philosophy, scope and coverage of the projects. This relates in particular to criteria such as participation, disclosure of results and dimensions of hospital performance assessed. CONCLUSION Hospital performance assessment projects have become common worldwide, and initiatives such as the WHO PATH project need to be well coordinated with existing projects. Our review raised questions regarding the impact of hospital performance assessment that should be pursued in further research.

Accreditation and ISO certification: do they explain differences in quality management in European hospitals?

BACKGROUND Hospital accreditation and International Standardisation Organisation (ISO) certification offer alternative mechanisms for improving safety and quality, or as a mark of achievement. There is little published evidence on their relative merits. OBJECTIVE To identify systematic differences in quality management between hospitals that were accredited, or certificated, or neither. Research design ANALYSIS of compliance with measures of quality in 89 hospitals in six countries, as assessed by external auditors using a standardized tool, as part of the EC-funded METHODS of Assessing Response to Quality Improvement Strategies project. MAIN OUTCOME MEASURES Compliance scores in six dimensions of each hospital-grouped according to the achievement of accreditation, certification or neither. RESULTS Of the 89 hospitals selected for external audit, 34 were accredited (without ISO certification), 10 were certificated under ISO 9001 (without accreditation) and 27 had neither accreditation nor certification. Overall percentage scores for 229 criteria of quality and safety were 66.9, 60.0 and 51.2, respectively. Analysis confirmed statistically significant differences comparing mean scores by the type of external assessment (accreditation, certification or neither); however, it did not substantially differentiate between accreditation and certification only. Some of these associations with external assessments were confounded by the country in which the sample hospitals were located. CONCLUSIONS It appears that quality and safety structures and procedures are more evident in hospitals with either the type of external assessment and suggest that some differences exist between accredited versus certified hospitals. Interpretation of these results, however, is limited by the sample size and confounded by variations in the application of accreditation and certification within and between countries.

Is patient-centredness in European hospitals related to existing quality improvement strategies? Analysis of a cross-sectional survey (MARQuIS study)

Background: There is growing recognition of patients’ contributions to setting objectives for their own care, improving health outcomes and evaluating care. Objective: To quantify the extent to which European hospitals have implemented strategies to promote a patient-centred approach, and to assess whether these strategies are associated with hospital characteristics and the development of the hospital’s quality improvement system. Design: Cross-sectional survey of 351 European hospital managers and professionals. Main outcome measures: Patients’ rights, patient information and empowerment, patient involvement in quality management, learning from patients, and patient hotel services at the hospital and ward level were assessed. The hypothesis that the implementation of strategies to improve patient-centredness is associated with hospital characteristics, including maturity of the hospital’s quality management system, was tested using binary logistic regression. Results: In general, hospitals reported high implementation of policies for patients’ rights (85.5%) and informed consent (93%), whereas strategies to involve patients (71%) and learn from their experience (66%) were less frequently implemented. For 13 out of 18 hospital strategies, institutions with a more developed quality improvement system consistently reported better results (percentage differences within maturity classification ranged from 12.4% to 46.6%). The strength of association between implementation of patient-centredness strategies and the quality improvement system, however, seemed lower at the ward than at the hospital level. Some associations (OR 2.1 to 5.1) disappeared or were weaker after adjustment for potential confounding variables (OR 2.2 to 3.7). Conclusions: Although quality improvement systems seem to be effective with regard to the implementation of selected patient-centredness strategies, they seem to be insufficient to ensure widespread implementation of patient-centredness throughout the organisation.

The World Health Organization Performance Assessment Tool for Quality Improvement in Hospitals (PATH): An Analysis of the Pilot Implementation in 37 Hospitals

OBJECTIVE To evaluate the pilot implementation of the World Health Organization Performance Assessment Tool for Quality Improvement in hospitals (PATH). DESIGN Semi-structured interviews with regional/country coordinators and Internet-based survey distributed to hospital coordinators. SETTING A total of 37 hospitals in six regions/countries (Belgium, Ontario (Canada), Denmark, France, Slovakia, KwaZulu Natal (South Africa)). PARTICIPANTS Six PATH regional/country coordinators and 37 PATH hospital coordinators. INTERVENTION Implementation of a hospital performance assessment pilot project. OUTCOME MEASURE Experience of regional/country coordinators (structured interviews) and experience of hospital coordinators (survey) with the pilot implementation. RESULTS The main achievement has been the collection and analysis of data on a set of indicators for comprehensive performance assessment in hospitals in regions and countries with different cultures and resource availability. Both regional/country coordinators and hospital coordinators required seed funding and technical support during data collection for implementation. Based on the user evaluation, we identified the following research and development tasks: further standardization and improved validity of indicators, increased use of routine data, more timely feedback with a stronger focus on international benchmarking and further support on interpretation of results. CONCLUSIONS Key to successful implementation was the embedding of PATH in existing performance measurement initiatives while acknowledging the core objective of the project as a self-improvement tool. The pilot test raised a number of organizational and methodological challenges in the design and implementation of international research on hospital performance assessment. Moreover, the process of evaluating PATH resulted in interesting learning points for other existing and newly emerging quality indicator projects.

Deepening our understanding of quality improvement in Europe (DUQuE): overview of a study of hospital quality management in seven countries

Introduction and Objective This paper provides an overview of the DUQuE (Deepening our Understanding of Quality Improvement in Europe) project, the first study across multiple countries of the European Union (EU) to assess relationships between quality management and patient outcomes at EU level. The paper describes the conceptual framework and methods applied, highlighting the novel features of this study. Design DUQuE was designed as a multi-level cross-sectional study with data collection at hospital, pathway, professional and patient level in eight countries. Setting and Participants We aimed to collect data for the assessment of hospital-wide constructs from up to 30 randomly selected hospitals in each country, and additional data at pathway and patient level in 12 of these 30. Main outcome measures A comprehensive conceptual framework was developed to account for the multiple levels that influence hospital performance and patient outcomes. We assessed hospital-specific constructs (organizational culture and professional involvement), clinical pathway constructs (the organization of care processes for acute myocardial infarction, stroke, hip fracture and deliveries), patient-specific processes and outcomes (clinical effectiveness, patient safety and patient experience) and external constructs that could modify hospital quality (external assessment and perceived external pressure). Results Data was gathered from 188 hospitals in 7 participating countries. The overall participation and response rate were between 75% and 100% for the assessed measures. Conclusions This is the first study assessing relation between quality management and patient outcomes at EU level. The study involved a large number of respondents and achieved high response rates. This work will serve to develop guidance in how to assess quality management and makes recommendations on the best ways to improve quality in healthcare for hospital stakeholders, payers, researchers, and policy makers throughout the EU.

“It's like two worlds apart”: an analysis of vulnerable patient handover practices at discharge from hospital

Background Handover practices at hospital discharge are relatively under-researched, particularly as regards the specific risks and additional requirements for handovers involving vulnerable patients with limited language, cognitive and social resources. Objective To explore handover practices at discharge and to focus on the patients’ role in handovers and on the potential additional risks for vulnerable patients. Methods We conducted qualitative interviews with patients, hospital professionals and primary care professionals in two hospitals and their associated primary care centres in Catalonia, Spain. Results We identified handover practices at discharge that potentially put patients at risk. Patients did not feel empowered in the handover but were expected to transfer information between care providers. Professionals identified lack of medication reconciliation at discharge, loss of discharge information, and absence of plans for follow-up care in the community as quality and safety problems for discharge handovers. These occurred for all patients, but appeared to be more frequent and have a greater negative effect in patients with limited language comprehension and/or lack of family and social support systems. Conclusions Discharge handovers are often haphazard. Healthcare professionals do not consider current handover practices safe, with patients expected to transfer information without being empowered to understand and act on it. This can lead to misinformation, omission or duplication of tests or interventions and, potentially, patient harm. Vulnerable patients may be at greater risk given their limited language, cognitive and social resources. Patient safety at discharge could benefit from strategies to enhance patient education and promote empowerment.

The effect of certification and accreditation on quality management in 4 clinical services in 73 European hospitals

Objective To investigate the relationship between ISO 9001 certification, healthcare accreditation and quality management in European hospitals. Design A mixed method multi-level cross-sectional design in seven countries. External teams assessed clinical services on the use of quality management systems, illustrated by four clinical pathways. Setting and Participants Seventy-three acute care hospitals with a total of 291 services managing acute myocardial infarction (AMI), hip fracture, stroke and obstetric deliveries, in Czech Republic, France, Germany, Poland, Portugal, Spain and Turkey. Main Outcome Measure Four composite measures of quality and safety [specialized expertise and responsibility (SER), evidence-based organization of pathways (EBOP), patient safety strategies (PSS) and clinical review (CR)] applied to four pathways. Results Accreditation in isolation showed benefits in AMI and stroke more than in deliveries and hip fracture; the greatest significant association was with CR in stroke. Certification in isolation showed little benefit in AMI but had more positive association with the other conditions; greatest significant association was in PSS with stroke. The combination of accreditation and certification showed least benefit in EBOP, but significant benefits in SER (AMI), in PSS (AMI, hip fracture and stroke) and in CR (AMI and stroke). Conclusions Accreditation and certification are positively associated with clinical leadership, systems for patient safety and clinical review, but not with clinical practice. Both systems promote structures and processes, which support patient safety and clinical organization but have limited effect on the delivery of evidence-based patient care. Further analysis of DUQuE data will explore the association of certification and accreditation with clinical outcomes.

The influence of context on the effectiveness of hospital quality improvement strategies: a review of systematic reviews

BackgroundIt is now widely accepted that the mixed effect and success rates of strategies to improve quality and safety in health care are in part due to the different contexts in which the interventions are planned and implemented. The objectives of this study were to (i) describe the reporting of contextual factors in the literature on the effectiveness of quality improvement strategies, (ii) assess the relationship between effectiveness and contextual factors, and (iii) analyse the importance of contextual factors.MethodsWe conducted an umbrella review of systematic reviews searching the following databases: PubMed, Cochrane Database of Systematic Reviews, Embase and CINAHL. The search focused on quality improvement strategies included in the Cochrane Effective Practice and Organisation of Care Group taxonomy. We extracted data on quality improvement effectiveness and context factors. The latter were categorized according to the Model for Understanding Success in Quality tool.ResultsWe included 56 systematic reviews in this study of which only 35 described contextual factors related with the effectiveness of quality improvement interventions. The most frequently reported contextual factors were: quality improvement team (n = 12), quality improvement support and capacity (n = 11), organization (n = 9), micro-system (n = 8), and external environment (n = 4). Overall, context factors were poorly reported. Where they were reported, they seem to explain differences in quality improvement effectiveness; however, publication bias may contribute to the observed differences.ConclusionsContextual factors may influence the effectiveness of quality improvement interventions, in particular at the level of the clinical micro-system. Future research on the implementation and effectiveness of quality improvement interventions should emphasize formative evaluation to elicit information on context factors and report on them in a more systematic way in order to better appreciate their relative importance.

Implementation of patient safety strategies in European hospitals

Context: This study is part of the Methods of Assessing Response to Quality Improvement Strategies (MARQuIS) research project on cross-border care, investigating quality improvement strategies in healthcare systems across the European Union (EU). Aim: To explore to what extent a sample of acute care European hospitals have implemented patient safety strategies and mechanisms and whether the implementation is related to the type of hospital. Methods: Data were collected on patient safety structures and mechanisms in 389 acute care hospitals in eight EU countries using a web-based questionnaire. Subsequently, an on-site audit was carried out by independent surveyors in 89 of these hospitals to assess patient safety outputs. This paper presents univariate and bivariate statistics on the implementation and explores the associations between implementation of patient safety strategies and hospital type using the χ2 test and Fisher exact test. Results: Structures and plans for safety (including responsibilities regarding patient safety management) are well developed in most of the hospitals that participated in this study. The study found greater variation regarding the implementation of mechanisms or activities to promote patient safety, such as electronic drug prescription systems, guidelines for prevention of wrong patient, wrong site and wrong surgical procedure, and adverse events reporting systems. In the sample of hospitals that underwent audit, a considerable proportion do not comply with basic patient safety strategies—for example, using bracelets for adult patient identification and correct labelling of medication.

High performing hospitals: a qualitative systematic review of associated factors and practical strategies for improvement

BackgroundHigh performing hospitals attain excellence across multiple measures of performance and multiple departments. Studying high performing hospitals can be valuable if factors associated with high performance can be identified and applied. Factors leading to high performance are complex and an exclusive quantitative approach may fail to identify richly descriptive or relevant contextual factors. The objective of this study was to undertake a systematic review of qualitative literature to identify methods used to identify high performing hospitals, the factors associated with high performers, and practical strategies for improvement.MethodsMethods used to collect and summarise the evidence contributing to this review followed the ‘enhancing transparency in reporting the synthesis of qualitative research’ protocol. Peer reviewed studies were identified through Medline, Embase and Cinahl (Jan 2000-Feb 2014) using specified key words, subject terms, and medical subject headings. Eligible studies required the use of a quantitative method to identify high performing hospitals, and qualitative methods or tools to identify factors associated with high performing hospitals or hospital departments. Title, abstract, and full text screening was undertaken by four reviewers, and inter-rater reliability statistics were calculated for each review phase. Risk of bias was assessed. Following data extraction, thematic syntheses identified contextual factors important for explaining success. Practical strategies for achieving high performance were then mapped against the identified themes.ResultsA total of 19 studies from a possible 11,428 were included in the review. A range of process, output, outcome and other indicators were used to identify high performing hospitals. Seven themes representing factors associated with high performance (and 25 sub-themes) emerged from the thematic syntheses: positive organisational culture, senior management support, effective performance monitoring, building and maintaining a proficient workforce, effective leaders across the organisation, expertise-driven practice, and interdisciplinary teamwork. Fifty six practical strategies for achieving high performance were catalogued.ConclusionsThis review provides insights into methods used to identify high performing hospitals, and yields ideas about the factors important for success. It highlights the need to advance approaches for understanding what constitutes high performance and how to harness factors associated with high performance.