Hildi Hagedorn

Role of "external facilitation" in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration

BackgroundFacilitation has been identified in the literature as a potentially key component of successful implementation. It has not, however, either been well-defined or well-studied. Significant questions remain about the operational definition of facilitation and about the relationship of facilitation to other interventions, especially to other change agent roles when used in multi-faceted implementation projects.Researchers who are part of the Quality Enhancement Research Initiative (QUERI) are actively exploring various approaches and processes, including facilitation, to enable implementation of best practices in the Veterans Health Administration health care system – the largest integrated healthcare system in the United States. This paper describes a systematic, retrospective evaluation of implementation-related facilitation experiences within QUERI, a quality improvement program developed by the US Department of Veterans Affairs.MethodsA post-hoc evaluation was conducted through a series of semi-structured interviews to examine the concept of facilitation across several multi-site QUERI implementation studies. The interview process is based on a technique developed in the field of education, which systematically enhances learning through experience by stimulating recall and reflection regarding past complex activities. An iterative content analysis approach relative to a set of conceptually-based interview questions was used for data analysis.FindingsFindings suggest that facilitation, within an implementation study initiated by a central change agency, is a deliberate and valued process of interactive problem solving and support that occurs in the context of a recognized need for improvement and a supportive interpersonal relationship. Facilitation was described primarily as a distinct role with a number of potentially crucial behaviors and activities. Data further suggest that external facilitators were likely to use or integrate other implementation interventions, while performing this problem-solving and supportive role.Preliminary ConclusionsThis evaluation provides evidence to suggest that facilitation could be considered a distinct implementation intervention, just as audit and feedback, educational outreach, or similar methods are considered to be discrete interventions. As such, facilitation should be well-defined and explicitly evaluated for its perceived usefulness within multi-intervention implementation projects. Additionally, researchers should better define the specific contribution of facilitation to the success of implementation in different types of projects, different types of sites, and with evidence and innovations of varying levels of strength and complexity.

A Guide for applying a revised version of the PARIHS framework for implementation

BackgroundBased on a critical synthesis of literature on use of the Promoting Action on Research Implementation in Health Services (PARIHS) framework, revisions and a companion Guide were developed by a group of researchers independent of the original PARIHS team. The purpose of the Guide is to enhance and optimize efforts of researchers using PARIHS in implementation trials and evaluations.MethodsAuthors used a planned, structured process to organize and synthesize critiques, discussions, and potential recommendations for refinements of the PARIHS framework arising from a systematic review. Using a templated form, each author independently recorded key components for each reviewed paper; that is, study definitions, perceived strengths/limitations of PARIHS, other observations regarding key issues and recommendations regarding needed refinements. After reaching consensus on these key components, the authors summarized the information and developed the Guide.ResultsA number of revisions, perceived as consistent with the PARIHS frameworks general nature and intent, are proposed. The related Guide is composed of a set of reference tools, provided in Additional files. Its core content is built upon the basic elements of PARIHS and current implementation science.ConclusionsWe invite researchers using PARIHS for targeted evidence-based practice (EBP) implementations with a strong task-orientation to use this Guide as a companion and to apply the revised framework prospectively and comprehensively. Researchers also are encouraged to evaluate its use relative to perceived strengths and issues. Such evaluations and critical reflections regarding PARIHS and our Guide could thereby promote the frameworks continued evolution.

A guiding framework and approach for implementation research in substance use disorders treatment.

This paper introduces readers to the concepts of implementation science, implementation theory, and implementation frameworks and models. A wide range of models has been published in the literature related to implementation. The paper will present an overview of the Consolidated Framework for Implementation Research (CFIR), which is a comprehensive typology that unifies and consolidates the array of constructs that influence implementation from the perspective of these models. The CFIR is then used to evaluate implementation models used in studies of substance use disorder (SUD) treatments. Implementation research is scarce, with few prospective studies of theory-driven implementation. We assert that future research in SUD needs to meet three overarching objectives to promote wider implementation of evidence-based practices: (a) differentiation of core versus adaptable components of evidence-based interventions need; (b) development of methods to design implementation strategies, effectively adapted to the broad context; and (c) design and testing of predictive models to assess likelihood of effective implementation and prospects for sustainability while taking into account salient contextual factors. A recommended strategy for accomplishing these objectives is described.

An introduction to implementation science for the non-specialist

BackgroundThe movement of evidence-based practices (EBPs) into routine clinical usage is not spontaneous, but requires focused efforts. The field of implementation science has developed to facilitate the spread of EBPs, including both psychosocial and medical interventions for mental and physical health concerns.DiscussionThe authors aim to introduce implementation science principles to non-specialist investigators, administrators, and policymakers seeking to become familiar with this emerging field. This introduction is based on published literature and the authors’ experience as researchers in the field, as well as extensive service as implementation science grant reviewers.Implementation science is “the scientific study of methods to promote the systematic uptake of research findings and other EBPs into routine practice, and, hence, to improve the quality and effectiveness of health services.” Implementation science is distinct from, but shares characteristics with, both quality improvement and dissemination methods. Implementation studies can be either assess naturalistic variability or measure change in response to planned intervention. Implementation studies typically employ mixed quantitative-qualitative designs, identifying factors that impact uptake across multiple levels, including patient, provider, clinic, facility, organization, and often the broader community and policy environment. Accordingly, implementation science requires a solid grounding in theory and the involvement of trans-disciplinary research teams.SummaryThe business case for implementation science is clear: As healthcare systems work under increasingly dynamic and resource-constrained conditions, evidence-based strategies are essential in order to ensure that research investments maximize healthcare value and improve public health. Implementation science plays a critical role in supporting these efforts.

The relationship between baseline Organizational Readiness to Change Assessment subscale scores and implementation of hepatitis prevention services in substance use disorders treatment clinics: a case study

BackgroundThe Organizational Readiness to Change Assessment (ORCA) is a measure of organizational readiness for implementing practice change in healthcare settings that is organized based on the core elements and sub-elements of the Promoting Action on Research Implementation in Health Services (PARIHS) framework. General support for the reliability and factor structure of the ORCA has been reported. However, no published study has examined the utility of the ORCA in a clinical setting. The purpose of the current study was to examine the relationship between baseline ORCA scores and implementation of hepatitis prevention services in substance use disorders (SUD) clinics.MethodsNine clinic teams from Veterans Health Administration SUD clinics across the United States participated in a six-month training program to promote evidence-based practices for hepatitis prevention. A representative from each team completed the ORCA evidence and context subscales at baseline.ResultsEight of nine clinics reported implementation of at least one new hepatitis prevention practice after completing the six-month training program. Clinic teams were categorized by level of implementation-high (n = 4) versus low (n = 5)-based on how many hepatitis prevention practices were integrated into their clinics after completing the training program. High implementation teams had significantly higher scores on the patient experience and leadership culture subscales of the ORCA compared to low implementation teams. While not reaching significance in this small sample, high implementation clinics also had higher scores on the research, clinical experience, staff culture, leadership behavior, and measurement subscales as compared to low implementation clinics.ConclusionsThe results of this study suggest that the ORCA was able to measure differences in organizational factors at baseline between clinics that reported high and low implementation of practice recommendations at follow-up. This supports the use of the ORCA to describe factors related to implementing practice recommendations in clinical settings. Future research utilizing larger sample sizes will be essential to support these preliminary findings.

Lessons Learned about Implementing Research Evidence into Clinical Practice

The mission of the Veterans Health Administration’s (VHA) quality enhancement research initiative (QUERI) is to enhance the quality of VHA health care by implementing clinical research findings into routine care. This paper presents lessons that QUERI investigators have learned through their initial attempts to pursue the QUERI mission. The lessons in this paper represent those that were common across multiple QUERI projects and were mutually agreed on as having substantial impact on the success of implementation. While the lessons are consistent with commonly recognized ingredients of successful implementation efforts, the examples highlight the fact that, even with a through knowledge of the literature and thoughtful planning, unexpected circumstances arise during implementation efforts that require flexibility and adaptability. The findings stress the importance of utilizing formative evaluation techniques to identify barriers to successful implementation and strategies to address these barriers.

Pharmacotherapy for alcohol dependence: perceived treatment barriers and action strategies among Veterans Health Administration service providers.

Although access to and consideration of pharmacological treatments for alcohol dependence are consensus standards of care, receipt of these medications by patients is generally rare and highly variable across treatment settings. The goal of the present project was to survey and interview the clinicians, managers, and pharmacists affiliated with addiction treatment programs within Veterans Health Administration (VHA) facilities to learn about their perceptions of barriers and facilitators regarding greater and more reliable consideration of pharmacological treatments for alcohol dependence. Fifty-nine participants from 19 high-adopting and 11 low-adopting facilities completed the survey (facility-level response rate = 50%) and 23 participated in a structured interview. The top 4 barriers to increased consideration and use of pharmacotherapy for alcohol dependence were consistent across high- and low-adopting facilities and included perceived low patient demand, pharmacy procedures or formulary restrictions, lack of provider skills or knowledge regarding pharmacotherapy for alcohol dependence, and lack of confidence in treatment effectiveness. Low patient demand was rated as the most important barrier for oral naltrexone and disulfiram, whereas pharmacy or formulary restrictions were rated as the most important barrier for acamprosate and extended-release naltrexone. The 4 strategies rated across low- and high-adopting facilities as most likely to facilitate consideration and use of pharmacotherapy for alcohol dependence were more education to patients about existing medications, more education to health care providers about medications, increased involvement of physicians in treatment for alcohol dependence, and more compelling research on existing medications. This knowledge provides a foundation for designing, deploying, and evaluating targeted implementation efforts.

Predicting implementation from organizational readiness for change: a study protocol

BackgroundThere is widespread interest in measuring organizational readiness to implement evidence-based practices in clinical care. However, there are a number of challenges to validating organizational measures, including inferential bias arising from the halo effect and method bias - two threats to validity that, while well-documented by organizational scholars, are often ignored in health services research. We describe a protocol to comprehensively assess the psychometric properties of a previously developed survey, the Organizational Readiness to Change Assessment.ObjectivesOur objective is to conduct a comprehensive assessment of the psychometric properties of the Organizational Readiness to Change Assessment incorporating methods specifically to address threats from halo effect and method bias.Methods and DesignWe will conduct three sets of analyses using longitudinal, secondary data from four partner projects, each testing interventions to improve the implementation of an evidence-based clinical practice. Partner projects field the Organizational Readiness to Change Assessment at baseline (n = 208 respondents; 53 facilities), and prospectively assesses the degree to which the evidence-based practice is implemented. We will conduct predictive and concurrent validities using hierarchical linear modeling and multivariate regression, respectively. For predictive validity, the outcome is the change from baseline to follow-up in the use of the evidence-based practice. We will use intra-class correlations derived from hierarchical linear models to assess inter-rater reliability. Two partner projects will also field measures of job satisfaction for convergent and discriminant validity analyses, and will field Organizational Readiness to Change Assessment measures at follow-up for concurrent validity (n = 158 respondents; 33 facilities). Convergent and discriminant validities will test associations between organizational readiness and different aspects of job satisfaction: satisfaction with leadership, which should be highly correlated with readiness, versus satisfaction with salary, which should be less correlated with readiness. Content validity will be assessed using an expert panel and modified Delphi technique.DiscussionWe propose a comprehensive protocol for validating a survey instrument for assessing organizational readiness to change that specifically addresses key threats of bias related to halo effect, method bias and questions of construct validity that often go unexplored in research using measures of organizational constructs.

Psychiatric illness among drug court probationers.

The purpose of this study was to identify the level of psychiatric symptoms reported by probationers involved with a drug court in Hennepin County, Minnesota. Sixty probationers completed a brief demographic interview, the Beck Depression and Anxiety Inventories (BDI and BAI) and a measure of medical quality of life. Fifteen participants completed a structured interview for psychiatric diagnosis (SCID‐I). The sample was predominantly male, African American, and unemployed. Over 40% had received treatment for psychiatric problems, including 20% who reported a history of inpatient psychiatric admission and 15% currently taking a psychotropic medication. More than 1/3 of BDI and BAI scores were moderate to severe. The mean Short Form (SF)‐36 scores were significantly lower than in the general population. Trends suggested more distress associated with: Caucasian race, female gender, less education, unemployment, and less previous legal involvement. Of 15 participants that completed a SCID‐I, 13 participants met lifetime diagnostic criteria for at least one psychiatric disorder. The most common diagnoses were major depressive disorder and posttraumatic stress disorder (PTSD). Three participants met diagnostic criteria for current psychotic disorder. Half of participants who currently met criteria for a disorder reported that they had never received psychiatric treatment. Results indicate participants currently were experiencing high rates of emotional symptoms. Serious mental illness was common. Many of these individuals had not been identified previously as needing psychiatric treatment. More frequent and thorough screening for psychiatric illness in drug court settings is necessary to identify serious psychiatric illnesses.

Telephone Smoking-Cessation Counseling for Smokers in Mental Health Clinics: A Patient-Randomized Controlled Trial.

INTRODUCTION People with a mental health diagnosis have high rates of tobacco use and encounter limited availability of tobacco treatment targeted to their needs. This study compared the effectiveness of a specialized telephone smoking-cessation intervention developed for mental health patients with standard state quit-line counseling. DESIGN RCT. SETTING/PARTICIPANTS The study was conducted at six Veterans Health Administration facilities in the Northeast U.S. Participants were 577 mental health clinic patients referred by their providers for smoking-cessation treatment. INTERVENTION From 2010 to 2012, the study implemented a telephone program that included patient referral from a mental health provider, mailed cessation medications, and telephone counseling. Participants were randomized to receive a specialized multisession telephone counseling protocol (n=270) or transfer to their states quit-line for counseling (n=307). MAIN OUTCOME MEASURES Participants completed telephone surveys at baseline, 2 months, and 6 months. The studys primary outcome was self-reported 30-day abstinence at 6 months. Secondary outcomes were self-reported 30-day abstinence, counseling satisfaction and counseling content at 2 months, and self-reported use of cessation treatment and quit attempts at 6 months. Logistic regression was used to compare treatment groups on outcomes, controlling for baseline cigarettes per day and site. Inverse probability weighting and multiple imputation were used to handle missing abstinence outcomes. Data were analyzed in 2014-2015. RESULTS At 6 months, participants in the specialized counseling arm were more likely to report 30-day abstinence (26% vs 18%, OR=1.62, 95% CI=1.24, 2.11). There was no significant group difference in abstinence at 2 months (18% vs 14%, OR=1.31, 95% CI=0.49, 3.49). Participants in the specialized arm were more likely to be assisted with developing a quit plan; receive follow-up calls after quitting; and receive counseling on several domains, including motivation, confidence, smoking triggers, coping with urges, and mental health symptoms (all p<0.05). Specialized counseling participants were more satisfied with treatment and more likely to find the counseling useful (p<0.05). CONCLUSIONS The specialized counseling intervention was more effective at helping patients quit than transfer to a state quit-line. Patients were more satisfied with the specialized counseling program. TRIAL REGISTRATION This study is registered at www.clinicaltrials.gov NCT00724308.