Kevin Forde

Health services accreditation: what is the evidence that the benefits justify the costs?

PURPOSE To identify and analyse research on the use of economic evaluation in health services accreditation. DATA SOURCES Seven online health and economic databases, and key accreditation agency and health department websites were searched between June and December 2011. STUDY SELECTION The selection criteria were English language and published empirical research studies on the topic of economic evaluation of health service accreditation. No formal economic evaluation of health services accreditation has been carried out to date. Empirical data on costs and benefits were analysed in 6 and 15 studies, respectively. Data extraction Meta-analysis was unsuitable due to output variability. Attributes relating to STUDY DESIGN scalability and independence of outcome data were collected. For the benefit studies, we also assessed the strength of claim that accreditation improved patient safety and quality, and sources of potential bias. RESULTS OF DATA SYNTHESIS The incremental costs ranged from 0.2 to 1.7% of total costs averaged over the accreditation cycle. The benefit studies were inconclusive in terms of showing clear evidence that accreditation improves patient safety and quality of care. CONCLUSION The lack of formal economic appraisal makes it difficult to evaluate accreditation in comparison to other methods to improve patient safety and quality of care. The lack of a clear relationship between accreditation and the outcomes measured in the benefit studies makes it difficult to design and conduct such appraisals without a more robust and explicit understanding of the costs and benefits involved.

Economic evaluation of Australian acute care accreditation (ACCREDIT-CBA (Acute)): study protocol for a mixed-method research project

Introduction The Accreditation Collaborative for the Conduct of Research, Evaluation and Designated Investigations through Teamwork—Cost–Benefit Analysis (ACCREDIT-CBA (Acute)) study is designed to determine and make explicit the costs and benefits of Australian acute care accreditation and to determine the effectiveness of acute care accreditation in improving patient safety and quality of care. The cost–benefit analysis framework will be provided in the form of an interactive model for industry partners, health regulators and policy makers, accreditation agencies and acute care service providers. Methods and design The study will use a mixed-method approach to identify, quantify and monetise the costs and benefits of accreditation. Surveys, expert panels, focus groups, interviews and primary and secondary data analysis will be used in cross-sectional and case study designs. Ethics and dissemination The University of New South Wales Human Research Ethics Committee has approved this project (approval number HREC 10274). The results of the study will be reported via peer-reviewed publications, conferences and seminar resentations and will form part of a doctoral thesis.

Disentangling quality and safety indicator data: A longitudinal, comparative study of hand hygiene compliance and accreditation outcomes in 96 Australian hospitals

Objectives The study aims are twofold. First, to investigate the suitability of hand hygiene as an indicator of accreditation outcomes and, second, to test the hypothesis that hospitals with better accreditation outcomes achieve higher hand hygiene compliance rates. Design A retrospective, longitudinal, multisite comparative survey. Setting Acute public hospitals in New South Wales, Australia. Participants 96 acute hospitals with accreditation survey results from two surveys during 2009–2012 and submitted data for more than four hand hygiene audits between 2010 and 2013. Outcomes Our primary outcome comprised observational hand hygiene compliance data from eight audits during 2010–2013. The explanatory variables in our multilevel regression model included: accreditation outcomes and scores for the infection control standard; timing of the surveys; and hospital size and activity. Results Average hand hygiene compliance rates increased from 67.7% to 80.3% during the study period (2010–2013), with 46.7% of hospitals achieving target compliance rates of 70% in audit 1, versus 92.3% in audit 8. Average hand hygiene rates at small hospitals were 7.8 percentage points (pp) higher than those at the largest hospitals (p<0.05). The association between hand hygiene rates, accreditation outcomes and infection control scores is less clear. Conclusions Our results indicate that accreditation outcomes and hand hygiene audit data are measuring different parts of the quality and safety spectrum. Understanding what is being measured when selecting indicators to assess the impact of accreditation is critical as focusing on accreditation results would discount successful hand hygiene implementation by smaller hospitals. Conversely, relying on hand hygiene results would discount the infection control related research and leadership investment by larger hospitals. Our hypothesis appears to be confounded by an accreditation programme that makes it more difficult for smaller hospitals to achieve high infection control scores.

Is accreditation linked to hospital infection rates? A 4-year, data linkage study of Staphylococcus aureus rates and accreditation scores in 77 Australian acute hospitals

OBJECTIVES To test our hypothesis that hospitals with higher accreditation scores, specifically in infection control, would be associated with lower Staphylococcus aureus bacteraemia (SAB) rates. DESIGN A retrospective cohort study. SETTING AND PARTICIPANTS Acute public hospitals (n = 77) in New South Wales, Australia, with reported SAB rates, results from two accreditation surveys and results from at least four hand hygiene audits. METHODS We linked three separate data sets comprising SAB rates, accreditation scores and hand hygiene rates. SAB rates were regressed against accreditation scores, hand hygiene audit rates and hospital demographics using a generalized linear model to account for the non-linear nature of our outcome variable. RESULTS Significant (P < 0.05) findings included the following: SAB rates across all hospitals fell from 1.34 per 10 000 bed days in 2009 to 0.77 per 10 000 bed days in 2012; mean SAB rates in small hospitals (0.62/10 000 bed days) over the study period were lower than those for principal referral hospitals (1.52/10 000 bed days); smaller hospitals with higher accreditation scores had lower SAB rates, but larger hospitals with higher overall accreditation scores had higher SAB rates, although the effect size was small (<2%). CONCLUSIONS There is a strong evidence base for using SAB rates to measure the impact of infection control programs that are assessed during accreditation. However, there is less evidence to support whether accreditation scores accurately reflect implementation of the infection control accreditation standards. This impacts identification of indicators to measure patient safety and quality of care, especially in ensuring these are appropriate across a range of hospital size and activities.

Counting the costs of accreditation in acute care: An activity-based costing approach

Objectives To assess the costs of hospital accreditation in Australia. Design Mixed methods design incorporating: stakeholder analysis; survey design and implementation; activity-based costs analysis; and expert panel review. Setting Acute care hospitals accredited by the Australian Council for Health Care Standards. Participants Six acute public hospitals across four States. Results Accreditation costs varied from 0.03% to 0.60% of total hospital operating costs per year, averaged across the 4-year accreditation cycle. Relatively higher costs were associated with the surveys years and with smaller facilities. At a national level these costs translate to

The evolution of Australian intensive care and its related costs: A narrative review

A36.83 million, equivalent to 0.1% of acute public hospital recurrent expenditure in the 2012 fiscal year. Conclusions This is the first time accreditation costs have been independently evaluated across a wide range of hospitals and highlights the additional cost burden for smaller facilities. A better understanding of the costs allows policymakers to assess alternative accreditation and other quality improvement strategies, and understand their impact across a range of facilities. This methodology can be adapted to assess international accreditation programmes.

Development and application of an indicator assessment tool for measuring health services accreditation programs.

OBJECTIVE To conduct a narrative review on the evolution of intensive care and the cost of intensive care services in Australia. REVIEW METHOD A narrative review using a search of online medical databases and grey literature with keyword verification via Delphi-technique. DATA SOURCES Using Medical Subject Headings and keywords (intensive care, critical care, mechanical ventilation, renal replacement therapy, extracorporeal membrane oxygenation, monitoring, staffing, cost, cost analysis) we searched MEDLINE, PubMed, CINAHL, Embase, Google and Google Scholar. RESULTS The search yielded 30 articles from which we provide a narrative synthesis on the evolving intensive care practice in relation to key service elements and therapies. For the review of costs, we found five relevant publications and noted significant variation in methods used to cost ICU. Notwithstanding the limitations of the methods used to cost all publications reported staffing as the primary cost driver, representing up to 71% of costs. CONCLUSION Intensive care is a highly specialised medical field, which has developed rapidly and plays an increasingly important role in the provision of hospital care. Despite the increasing importance of the specialty and the known resource intensity there is a paucity of data on the cost of providing this service. In Australia, staffing costs consistently represent the majority of costs associated with operating an ICU. This finding should be interpreted cautiously given the variation of methods used to cost ICU services and the limited number of available studies. Developing standardised methods to consistently estimate ICU costs which can be incorporated in research into the cost-effectiveness of alternate practice is an important step to ensuring cost-effective care.

Microbial Keratitis: Predisposing Factors and Morbidity

BackgroundHospital accreditation programs are internationally widespread and consume increasingly scarce health resources. However, we lack tools to consistently identify suitable indicators to assess and monitor accreditation outcomes. We describe the development and validation of such a tool.ResultsUsing Australian accreditation standards as our reference point we: reviewed the research evidence for potential indicators; looked for links with existing external indicators; and assessed relevant state and federal policies. We allocated provisional scores, on a five point Likert scale, to the five accountability criteria in the tool: research; accuracy; proximity; no adverse effects; and specificity. An expert panel validated the use of the purpose designed indicator assessment tool. The panel identified hand hygiene compliance rates as a suitable process indicator, and hospital acquired Staphylococcus aureus infection (SAB) rates as an outcome indicator, with the hypothesis that improved hand hygiene compliance rates and lower SAB rates would correlate with accreditation performance.ConclusionsThis new tool can be used to identify, analyse, and compare accreditation indicators. Using infection control indicators such as hand hygiene compliance and SAB rates to measure accreditation effectiveness has merit, and their efficacy can be determined by comparing accreditation scores with indicator outcomes. To verify the tool as a robust instrument, testing is needed in other health service domains, both in Australia and internationally. This tool provides health policy makers with an important means for assessing the accreditation programs which form a critical part of the national patient safety and quality framework.