Christian D. Helfrich

Organizational readiness to change assessment (ORCA): development of an instrument based on the Promoting Action on Research in Health Services (PARIHS) framework.

BackgroundThe Promoting Action on Research Implementation in Health Services, or PARIHS, framework is a theoretical framework widely promoted as a guide to implement evidence-based clinical practices. However, it has as yet no pool of validated measurement instruments that operationalize the constructs defined in the framework. The present article introduces an Organizational Readiness to Change Assessment instrument (ORCA), organized according to the core elements and sub-elements of the PARIHS framework, and reports on initial validation.MethodsWe conducted scale reliability and factor analyses on cross-sectional, secondary data from three quality improvement projects (n = 80) conducted in the Veterans Health Administration. In each project, identical 77-item ORCA instruments were administered to one or more staff from each facility involved in quality improvement projects. Items were organized into 19 subscales and three primary scales corresponding to the core elements of the PARIHS framework: (1) Strength and extent of evidence for the clinical practice changes represented by the QI program, assessed with four subscales, (2) Quality of the organizational context for the QI program, assessed with six subscales, and (3) Capacity for internal facilitation of the QI program, assessed with nine subscales.ResultsCronbachs alpha for scale reliability were 0.74, 0.85 and 0.95 for the evidence, context and facilitation scales, respectively. The evidence scale and its three constituent subscales failed to meet the conventional threshold of 0.80 for reliability, and three individual items were eliminated from evidence subscales following reliability testing. In exploratory factor analysis, three factors were retained. Seven of the nine facilitation subscales loaded onto the first factor; five of the six context subscales loaded onto the second factor; and the three evidence subscales loaded on the third factor. Two subscales failed to load significantly on any factor. One measured resources in general (from the context scale), and one clinical champion role (from the facilitation scale).ConclusionWe find general support for the reliability and factor structure of the ORCA. However, there was poor reliability among measures of evidence, and factor analysis results for measures of general resources and clinical champion role did not conform to the PARIHS framework. Additional validation is needed, including criterion validation.

A Guide for applying a revised version of the PARIHS framework for implementation

BackgroundBased on a critical synthesis of literature on use of the Promoting Action on Research Implementation in Health Services (PARIHS) framework, revisions and a companion Guide were developed by a group of researchers independent of the original PARIHS team. The purpose of the Guide is to enhance and optimize efforts of researchers using PARIHS in implementation trials and evaluations.MethodsAuthors used a planned, structured process to organize and synthesize critiques, discussions, and potential recommendations for refinements of the PARIHS framework arising from a systematic review. Using a templated form, each author independently recorded key components for each reviewed paper; that is, study definitions, perceived strengths/limitations of PARIHS, other observations regarding key issues and recommendations regarding needed refinements. After reaching consensus on these key components, the authors summarized the information and developed the Guide.ResultsA number of revisions, perceived as consistent with the PARIHS frameworks general nature and intent, are proposed. The related Guide is composed of a set of reference tools, provided in Additional files. Its core content is built upon the basic elements of PARIHS and current implementation science.ConclusionsWe invite researchers using PARIHS for targeted evidence-based practice (EBP) implementations with a strong task-orientation to use this Guide as a companion and to apply the revised framework prospectively and comprehensively. Researchers also are encouraged to evaluate its use relative to perceived strengths and issues. Such evaluations and critical reflections regarding PARIHS and our Guide could thereby promote the frameworks continued evolution.

Determinants of implementation effectiveness: adapting a framework for complex innovations.

Many innovations in the health sector are complex, requiring coordinated use by multiple organizational members to achieve benefits. Often, complex innovations are adopted with great anticipation only to fail during implementation. The health services literature provides limited conceptual guidance to researchers and practitioners about implementation of complex innovations. In the present study, we adapt an organizational framework of innovation implementation developed and validated in a manufacturing setting and explore the extent to which it aptly characterizes implementation in health sector organizations. Through comparative case studies of four cancer clinical research networks, we illustrate how this conceptual framework captures key determinants of the implementation of new programs in cancer prevention and control (CP/C) research and helps explain observed differences in implementation effectiveness. Key determinants include management support and innovation-values fit, which contribute to an organizational “climate” for implementation. We explore the implications for researchers and managers.

Implementation of the Patient-Centered Medical Home in the Veterans Health Administration: Associations With Patient Satisfaction, Quality of Care, Staff Burnout, and Hospital and Emergency Department Use

IMPORTANCE In 2010, the Veterans Health Administration (VHA) began implementing the patient-centered medical home (PCMH) model. The Patient Aligned Care Team (PACT) initiative aims to improve health outcomes through team-based care, improved access, and care management. To track progress and evaluate outcomes at all VHA primary care clinics, we developed and validated a method to assess PCMH implementation. OBJECTIVES To create an index that measures the extent of PCMH implementation, describe variation in implementation, and examine the association between the implementation index and key outcomes. DESIGN, SETTING, AND PARTICIPANTS We conducted an observational study using data on more than 5.6 million veterans who received care at 913 VHA hospital-based and community-based primary care clinics and 5404 primary care staff from (1) VHA clinical and administrative databases, (2) a national patient survey administered to a weighted random sample of veterans who received outpatient care from June 1 to December 31, 2012, and (3) a survey of all VHA primary care staff in June 2012. Composite scores were constructed for 8 core domains of PACT: access, continuity, care coordination, comprehensiveness, self-management support, patient-centered care and communication, shared decision making, and team-based care. MAIN OUTCOMES AND MEASURES Patient satisfaction, rates of hospitalization and emergency department use, quality of care, and staff burnout. RESULTS Fifty-three items were included in the PACT Implementation Progress Index (Pi2). Compared with the 87 clinics in the lowest decile of the Pi2, the 77 sites in the top decile exhibited significantly higher patient satisfaction (9.33 vs 7.53; P < .001), higher performance on 41 of 48 measures of clinical quality, lower staff burnout (Maslach Burnout Inventory emotional exhaustion subscale, 2.29 vs 2.80; P = .02), lower hospitalization rates for ambulatory care-sensitive conditions (4.42 vs 3.68 quarterly admissions for veterans 65 years or older per 1000 patients; P < .001), and lower emergency department use (188 vs 245 visits per 1000 patients; P < .001). CONCLUSIONS AND RELEVANCE The extent of PCMH implementation, as measured by the Pi2, was highly associated with important outcomes for both patients and providers. This measure will be used to track the effectiveness of implementing PACT over time and to elucidate the correlates of desired health outcomes.

Assessing an organizational culture instrument based on the Competing Values Framework: Exploratory and confirmatory factor analyses

BackgroundThe Competing Values Framework (CVF) has been widely used in health services research to assess organizational culture as a predictor of quality improvement implementation, employee and patient satisfaction, and team functioning, among other outcomes. CVF instruments generally are presented as well-validated with reliable aggregated subscales. However, only one study in the health sector has been conducted for the express purpose of validation, and that study population was limited to hospital managers from a single geographic locale.MethodsWe used exploratory and confirmatory factor analyses to examine the underlying structure of data from a CVF instrument. We analyzed cross-sectional data from a work environment survey conducted in the Veterans Health Administration (VHA). The study population comprised all staff in non-supervisory positions. The survey included 14 items adapted from a popular CVF instrument, which measures organizational culture according to four subscales: hierarchical, entrepreneurial, team, and rational.ResultsData from 71,776 non-supervisory employees (approximate response rate 51%) from 168 VHA facilities were used in this analysis. Internal consistency of the subscales was moderate to strong (α = 0.68 to 0.85). However, the entrepreneurial, team, and rational subscales had higher correlations across subscales than within, indicating poor divergent properties. Exploratory factor analysis revealed two factors, comprising the ten items from the entrepreneurial, team, and rational subscales loading on the first factor, and two items from the hierarchical subscale loading on the second factor, along with one item from the rational subscale that cross-loaded on both factors. Results from confirmatory factor analysis suggested that the two-subscale solution provides a more parsimonious fit to the data as compared to the original four-subscale model.ConclusionThis study suggests that there may be problems applying conventional CVF subscales to non-supervisors, and underscores the importance of assessing psychometric properties of instruments in each new context and population to which they are applied. It also further highlights the challenges management scholars face in assessing organizational culture in a reliable and comparable way. More research is needed to determine if the emergent two-subscale solution is a valid or meaningful alternative and whether these findings generalize beyond VHA.

Are you ready? How health professionals can comprehensively conceptualize readiness for change.

One important factor influencing the successful implementation of system-wide change is initial readiness. Readiness is defined as the degree to which those involved are individually and collectively primed, motivated, and technically capable of executing the change. We present a conceptual framework that highlights three broad areas to be considered if health-care professionals are to comprehensively evaluate readiness that includes psychological factors (i.e., characteristics of those being asked to change), structural factors (i.e., circumstances under which the change is occurring) as well as the level of analysis (i.e., individual and organizational levels). We also describe more specific dimensions within each of these broad categories that have both empirical and theoretical support, presenting several valid and reliable survey instruments that measure key dimensions of readiness quantitatively.

Strategies from a Nationwide Health Information Technology Implementation: The VA CART STORY

The VA Cardiovascular Assessment, Reporting, and Tracking (CART) system is a customized electronic medical record system which provides standardized report generation for cardiac catheterization procedures, serves as a national data repository, and is the centerpiece of a national quality improvement program. Like many health information technology projects, CART implementation did not proceed without some barriers and resistance. We describe the nationwide implementation of CART at the 77 VA hospitals which perform cardiac catheterizations in three phases: (1) strategic collaborations; (2) installation; and (3) adoption. Throughout implementation, success required a careful balance of technical, clinical, and organizational factors. We offer strategies developed through CART implementation which are broadly applicable to technology projects aimed at improving the quality, reliability, and efficiency of health care.

Predicting implementation from organizational readiness for change: a study protocol

BackgroundThere is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment.ObjectivesOur objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias.Methods and DesignWe will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique.DiscussionWe propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs.

Declining mortality following acute myocardial infarction in the Department of Veterans Affairs Health Care System

BackgroundMortality from acute myocardial infarction (AMI) is declining worldwide. We sought to determine if mortality in the Veterans Health Administration (VHA) has also been declining.MethodsWe calculated 30-day mortality rates between 2004 and 2006 using data from the VHA External Peer Review Program (EPRP), which entails detailed abstraction of records of all patients with AMI. To compare trends within VHA with other systems of care, we estimated relative mortality rates between 2000 and 2005 for all males 65 years and older with a primary diagnosis of AMI using administrative data from the VHA Patient Treatment File and the Medicare Provider Analysis and Review (MedPAR) files.ResultsUsing EPRP data on 11,609 patients, we observed a statistically significant decline in adjusted 30-day mortality following AMI in VHA from 16.3% in 2004 to 13.9% in 2006, a relative decrease of 15% and a decrease in the odds of dying of 10% per year (p = .011). Similar declines were found for in-hospital and 90-day mortality.Based on administrative data on 27,494 VHA patients age 65 years and older and 789,400 Medicare patients, 30-day mortality following AMI declined from 16.0% during 2000-2001 to 15.7% during 2004-June 2005 in VHA and from 16.7% to 15.5% in private sector hospitals. After adjusting for patient characteristics and hospital effects, the overall relative odds of death were similar for VHA and Medicare (odds ratio 1.02, 95% C.I. 0.96-1.08).ConclusionMortality following AMI within VHA has declined significantly since 2003 at a rate that parallels that in Medicare-funded hospitals.

1-year risk-adjusted mortality and costs of percutaneous coronary intervention in the Veterans Health Administration: insights from the VA CART Program.

BACKGROUND There is significant interest in measuring health care value, but this concept has not been operationalized in specific patient cohorts. The longitudinal outcomes and costs for patients after percutaneous coronary intervention (PCI) provide an opportunity to measure an aspect of health care value. OBJECTIVES This study evaluated variations in 1-year outcomes (risk-adjusted mortality) and risk-standardized costs of care for all patients undergoing PCI in the Veterans Affairs (VA) system from 2007 to 2010. METHODS This retrospective cohort study evaluated all veterans undergoing PCI at any of 60 hospitals in the VA health care system, using data from the national VA Clinical Assessment, Reporting, and Tracking (CART) program. Primary outcomes were 1-year mortality and costs following PCI. Risk-standardized mortality and cost ratios were calculated, adjusting for cardiac and noncardiac comorbidities. RESULTS A median of 261 PCIs were performed in the 60 hospitals during the study period. Median 1-year unadjusted hospital mortality rate was 6.13%. Four hospitals were significantly above the 1-year risk-standardized median mortality rate, with median mortality ratios ranging from 1.23 to 1.28. No hospitals were significantly below median mortality. Median 1-year total unadjusted hospital costs were