Marjorie Pawsey

Health service accreditation as a predictor of clinical and organisational performance: a blinded, random, stratified study

Background Despite the widespread use of accreditation in many countries, and prevailing beliefs that accreditation is associated with variables contributing to clinical care and organisational outcomes, little systematic research has been conducted to examine its validity as a predictor of healthcare performance. Objective To determine whether accreditation performance is associated with self-reported clinical performance and independent ratings of four aspects of organisational performance. Design Independent blinded assessment of these variables in a random, stratified sample of health service organisations. Settings Acute care: large, medium and small health-service organisations in Australia. Study participants Nineteen health service organisations employing 16 448 staff treating 321 289 inpatients and 1 971 087 non-inpatient services annually, representing approximately 5% of the Australian acute care health system. Main measures Correlations of accreditation performance with organisational culture, organisational climate, consumer involvement, leadership and clinical performance. Results Accreditation performance was significantly positively correlated with organisational culture (rho=0.618, p=0.005) and leadership (rho=0.616, p=0.005). There was a trend between accreditation and clinical performance (rho=0.450, p=0.080). Accreditation was unrelated to organisational climate (rho=0.378, p=0.110) and consumer involvement (rho=0.215, p=0.377). Conclusions Accreditation results predict leadership behaviours and cultural characteristics of healthcare organisations but not organisational climate or consumer participation, and a positive trend between accreditation and clinical performance is noted.

Narrative synthesis of health service accreditation literature

Aims To systematically identify and synthesise health service accreditation literature. Methods A systematic identification and narrative synthesis of health service accreditation literature published prior to 2012 were conducted. The search identified 122 empirical studies that examined either the processes or impacts of accreditation programmes. Study components were recorded, including: dates of publication; research settings; levels of study evidence and quality using established rating frameworks; and key results. A content analysis was conducted to determine the frequency of key themes and subthemes examined in the literature and identify knowledge-gaps requiring research attention. Results The majority of studies (n=67) were published since 2006, occurred in the USA (n=60) and focused on acute care (n=79). Two thematic categories, that is, ‘organisational impacts’ and ‘relationship to quality measures’, were addressed 60 or more times in the literature. ‘Financial impacts’, ‘consumer or patient satisfaction’ and ‘survey and surveyor issues’ were each examined fewer than 15 times. The literature is limited in terms of the level of evidence and quality of studies, but highlights potential relationships among accreditation programmes, high quality organisational processes and safe clinical care. Conclusions Due to the limitations of the literature, it is not prudent to make strong claims about the effectiveness of health service accreditation. Nonetheless, several critical issues and knowledge-gaps were identified that may help stimulate and inform discussion among healthcare stakeholders. Ongoing effort is required to build upon the accreditation evidence-base by using high quality experimental study designs to examine the processes, effectiveness and financial value of accreditation programmes and their critical components in different healthcare domains.

A prospective, multi-method, multi-disciplinary, multi-level, collaborative, social-organisational design for researching health sector accreditation [LP0560737]

BackgroundAccreditation has become ubiquitous across the international health care landscape. Award of full accreditation status in health care is viewed, as it is in other sectors, as a valid indicator of high quality organisational performance. However, few studies have empirically demonstrated this assertion. The value of accreditation, therefore, remains uncertain, and this persists as a central legitimacy problem for accreditation providers, policymakers and researchers. The question arises as to how best to research the validity, impact and value of accreditation processes in health care. Most health care organisations participate in some sort of accreditation process and thus it is not possible to study its merits using a randomised controlled strategy. Further, tools and processes for accreditation and organisational performance are multifaceted.Methods/designTo understand the relationship between them a multi-method research approach is required which incorporates both quantitative and qualitative data. The generic nature of accreditation standard development and inspection within different sectors enhances the extent to which the findings of in-depth study of accreditation process in one industry can be generalised to other industries. This paper presents a research design which comprises a prospective, multi-method, multi-level, multi-disciplinary approach to assess the validity, impact and value of accreditation.DiscussionThe accreditation program which assesses over 1,000 health services in Australia is used as an exemplar for testing this design. The paper proposes this design as a framework suitable for application to future international research into accreditation. Our aim is to stimulate debate on the role of accreditation and how to research it.

The standard of healthcare accreditation standards: a review of empirical research underpinning their development and impact

BackgroundHealthcare accreditation standards are advocated as an important means of improving clinical practice and organisational performance. Standard development agencies have documented methodologies to promote open, transparent, inclusive development processes where standards are developed by members. They assert that their methodologies are effective and efficient at producing standards appropriate for the health industry. However, the evidence to support these claims requires scrutiny. The study’s purpose was to examine the empirical research that grounds the development methods and application of healthcare accreditation standards.MethodsA multi-method strategy was employed over the period March 2010 to August 2011. Five academic health research databases (Medline, Psych INFO, Embase, Social work abstracts, and CINAHL) were interrogated, the websites of 36 agencies associated with the study topic were investigated, and a snowball search was undertaken. Search criteria included accreditation research studies, in English, addressing standards and their impact. Searching in stage 1 initially selected 9386 abstracts. In stage 2, this selection was refined against the inclusion criteria; empirical studies (n = 2111) were identified and refined to a selection of 140 papers with the exclusion of clinical or biomedical and commentary pieces. These were independently reviewed by two researchers and reduced to 13 articles that met the study criteria.ResultsThe 13 articles were analysed according to four categories: overall findings; standards development; implementation issues; and impact of standards. Studies have only occurred in the acute care setting, predominately in 2003 (n = 5) and 2009 (n = 4), and in the United States (n = 8). A multidisciplinary focus (n = 9) and mixed method approach (n = 11) are common characteristics. Three interventional studies were identified, with the remaining 10 studies having research designs to investigate clinical or organisational impacts. No study directly examined standards development or other issues associated with their progression. Only one study noted implementation issues, identifying several enablers and barriers. Standards were reported to improve organisational efficiency and staff circumstances. However, the impact on clinical quality was mixed, with both improvements and a lack of measurable effects recorded.ConclusionStandards are ubiquitous within healthcare and are generally considered to be an important means by which to improve clinical practice and organisational performance. However, there is a lack of robust empirical evidence examining the development, writing, implementation and impacts of healthcare accreditation standards.

Stakeholder perspectives on implementing accreditation programs: a qualitative study of enabling factors

BackgroundAccreditation programs are complex, system-wide quality and safety interventions. Despite their international popularity, evidence of their effectiveness is weak and contradictory. This may be due to variable implementation in different contexts. However, there is limited research that informs implementation strategies. We aimed to advance knowledge in this area by identifying factors that enable effective implementation of accreditation programs across different healthcare settings.MethodsWe conducted 39 focus groups and eight interviews between 2011 and 2012, involving 258 diverse healthcare stakeholders from every Australian State and Territory. Interviews were semi-structured and focused on the aims, implementation and consequences of three prominent accreditation programs in the aged, primary and acute care sectors. Data were thematically analysed to distil and categorise facilitators of effective implementation.ResultsFour factors were identified as critical enablers of effective implementation: the accreditation program is collaborative, valid and uses relevant standards; accreditation is favourably received by health professionals; healthcare organisations are capable of embracing accreditation; and accreditation is appropriately aligned with other regulatory initiatives and supported by relevant incentives.ConclusionsStrategic implementation of accreditation programs should target the four factors emerging from this study, which may increase the likelihood of accreditation being implemented successfully.

Evaluation of current Australian health service accreditation processes (ACCREDIT-CAP): protocol for a mixed-method research project

Introduction Accreditation programmes aim to improve the quality and safety of health services, and have been widely implemented. However, there is conflicting evidence regarding the outcomes of existing programmes. The Accreditation Collaborative for the Conduct of Research, Evaluation and Designated Investigations through Teamwork-Current Accreditation Processes (ACCREDIT-CAP) project is designed to address key gaps in the literature by evaluating the current processes of three accreditation programmes used across Australian acute, primary and aged care services. Methods and design The project comprises three mixed-method studies involving documentary analyses, surveys, focus groups and individual interviews. Study samples will comprise stakeholders from across the Australian healthcare system: accreditation agencies; federal and state government departments; consumer advocates; professional colleges and associations; and staff of acute, primary and aged care services. Sample sizes have been determined to ensure results allow robust conclusions. Qualitative information will be thematically analysed, supported by the use of textual grouping software. Quantitative data will be subjected to a variety of analytical procedures, including descriptive and comparative statistics. The results are designed to inform health system policy and planning decisions in Australia and internationally. Ethics and dissemination The project has been approved by the University of New South Wales Human Research Ethics Committee (approval number HREC 10274). Results will be reported to partner organisations, healthcare consumers and other stakeholders via peer-reviewed publications, conference and seminar presentations, and a publicly accessible website.

Distributed leadership to mobilise capacity for accreditation research.

PURPOSE Inquiries into healthcare organisations have highlighted organisational or system failure, attributed to poor responses to early warning signs. One response, and challenge, is for professionals and academics to build capacity for quality and safety research to provide evidence for improved systems. However, such collaborations and capacity building do not occur easily as there are many stakeholders. Leadership is necessary to unite differences into a common goal. The lessons learned and principles arising from the experience of providing distributed leadership to mobilise capacity for quality and safety research when researching health care accreditation in Australia are presented. DESIGN/METHODOLOGY/APPROACH A case study structured by temporal bracketing that presents a narrative account of multi-stakeholder perspectives. Data are collected using in-depth informal interviews with key informants and ethno-document analysis. FINDINGS Distributed leadership enabled a collaborative research partnership to be realised. The leadership harnessed the relative strengths of partners and accounted for, and balanced, the interests of stakeholder participants involved. Across three phases, leadership and the research partnership was enacted: identifying partnerships, bottom-up engagement and enacting the research collaboration. PRACTICAL IMPLICATIONS Two principles to maximise opportunities to mobilise capacity for quality and safety research have been identified. First, successful collaborations, particularly multi-faceted inter-related partnerships, require distributed leadership. Second, the leadership-stakeholder enactment can promote reciprocity so that the collaboration becomes mutually reinforcing and beneficial to partners. ORIGINALITY/VALUE The paper addresses the need to understand the practice and challenges of distributed leadership and how to replicate positive practices to implement patient safety research.

A multimethod research investigation of consumer involvement in Australian health service accreditation programmes: the ACCREDIT-SCI study protocol

Introduction Health service accreditation programmes are a regulatory mechanism adopted to drive improvements inpatient safety and quality. Research investigating the benefits or limitations, of consumer involvement in accreditation programmes is negligible. To develop our knowledge in this area the ACCREDIT collaboration (Accreditation Collaborative for the Conduct of Research, Evaluation and Designated Investigations through Teamwork) has developed a research plan, known as the ACCREDIT-SCI (Standards of Consumer Involvement) study protocol. Two complementary studies have been designed: one, to examine the effectiveness of a standard for consumer participation and two, to explore how patient experiences vary across a range of settings with differing accreditation results. Methods and design The research setting is the Australian healthcare system, and the two studies focus on three accreditation programmes in the primary, acute and aged care domains. The studies will use multimethods: document analysis; interviews and surveys. Participants will be stakeholders across the three domains including: policy officers; frontline healthcare professionals; accreditation agency personnel, including surveyors and healthcare consumers. Drawing on previous experience, the research team has developed purpose-designed tools. Data will be analysed using thematic, narrative and statistical (descriptive and inferential) procedures. Ethics and dissemination The University of New South Wales Human Research Ethics Committee has approved the two studies (HREC 10274). Findings will be disseminated through seminars, conference presentations, academic publications and research partner websites. The findings will be formulated to facilitate uptake by policy and accreditation agency professionals, researchers and academics, and consumers, nationally and internationally.

An empirical test of accreditation patient journey surveys: randomized trial.

OBJECTIVE To evaluate the effectiveness of utilizing the patient journey survey (PJS) method in healthcare accreditation processes. DESIGN Randomized trial of the PJS method in parallel with the current accreditation survey (CAS) method of the Australian Council on Healthcare Standards (ACHS). SETTING Acute healthcare organizations in Australia. PARTICIPANTS Seventeen organizations, 28 organizational staff, nine surveyors and 38 patients. MAIN OUTCOME MEASURES The results of each surveying method were compared. Participants provided feedback, via 18 interviews and 40 questionnaire surveys, about the benefits and disadvantages of a PJS compared to a CAS. RESULTS The PJS method is not as comprehensive as the CAS method for accreditation assessment. In matched assessments the majority of items were rated lower by the PJS method than by the CAS. PJSs were shown to be appropriate for assessing mandatory clinical criteria, but were less effective for assessing corporate and support criteria. The two methods diverged in their final assessments of which organizations met the accreditation threshold. Participants endorsed the use of PJSs within accreditation processes. CONCLUSIONS The PJS methodology complements but is not a substitute for existing accreditation methods. There is significant stakeholder support for the inclusion of the PJS method within the current accreditation programme.

Analysing 'big picture' policy reform mechanisms : the Australian health service safety and quality accreditation scheme

Agencies promoting national health‐care accreditation reform to improve the quality of care and safety of patients are largely working without specific blueprints that can increase the likelihood of success.