Janet E. Squires

Knowledge translation of research findings

BackgroundOne of the most consistent findings from clinical and health services research is the failure to translate research into practice and policy. As a result of these evidence-practice and policy gaps, patients fail to benefit optimally from advances in healthcare and are exposed to unnecessary risks of iatrogenic harms, and healthcare systems are exposed to unnecessary expenditure resulting in significant opportunity costs. Over the last decade, there has been increasing international policy and research attention on how to reduce the evidence-practice and policy gap. In this paper, we summarise the current concepts and evidence to guide knowledge translation activities, defined as T2 research (the translation of new clinical knowledge into improved health). We structure the article around five key questions: what should be transferred; to whom should research knowledge be transferred; by whom should research knowledge be transferred; how should research knowledge be transferred; and, with what effect should research knowledge be transferred?DiscussionWe suggest that the basic unit of knowledge translation should usually be up-to-date systematic reviews or other syntheses of research findings. Knowledge translators need to identify the key messages for different target audiences and to fashion these in language and knowledge translation products that are easily assimilated by different audiences. The relative importance of knowledge translation to different target audiences will vary by the type of research and appropriate endpoints of knowledge translation may vary across different stakeholder groups. There are a large number of planned knowledge translation models, derived from different disciplinary, contextual (i.e., setting), and target audience viewpoints. Most of these suggest that planned knowledge translation for healthcare professionals and consumers is more likely to be successful if the choice of knowledge translation strategy is informed by an assessment of the likely barriers and facilitators. Although our evidence on the likely effectiveness of different strategies to overcome specific barriers remains incomplete, there is a range of informative systematic reviews of interventions aimed at healthcare professionals and consumers (i.e., patients, family members, and informal carers) and of factors important to research use by policy makers.SummaryThere is a substantial (if incomplete) evidence base to guide choice of knowledge translation activities targeting healthcare professionals and consumers. The evidence base on the effects of different knowledge translation approaches targeting healthcare policy makers and senior managers is much weaker but there are a profusion of innovative approaches that warrant further evaluation.

Study protocol for the translating research in elder care (TREC): building context – an organizational monitoring program in long-term care project (project one)

BackgroundWhile there is a growing awareness of the importance of organizational context (or the work environment/setting) to successful knowledge translation, and successful knowledge translation to better patient, provider (staff), and system outcomes, little empirical evidence supports these assumptions. Further, little is known about the factors that enhance knowledge translation and better outcomes in residential long-term care facilities, where care has been shown to be suboptimal. The project described in this protocol is one of the two main projects of the larger five-year Translating Research in Elder Care (TREC) program.AimsThe purpose of this project is to establish the magnitude of the effect of organizational context on knowledge translation, and subsequently on resident, staff (unregulated, regulated, and managerial) and system outcomes in long-term care facilities in the three Canadian Prairie Provinces (Alberta, Saskatchewan, Manitoba).Methods/DesignThis study protocol describes the details of a multi-level – including provinces, regions, facilities, units within facilities, and individuals who receive care (residents) or work (staff) in facilities – and longitudinal (five-year) research project. A stratified random sample of 36 residential long-term care facilities (30 urban and 6 rural) from the Canadian Prairie Provinces will comprise the sample. Caregivers and care managers within these facilities will be asked to complete the TREC survey – a suite of survey instruments designed to assess organizational context and related factors hypothesized to be important to successful knowledge translation and to achieving better resident, staff, and system outcomes. Facility and unit level data will be collected using standardized data collection forms, and resident outcomes using the Resident Assessment Instrument-Minimum Data Set version 2.0 instrument. A variety of analytic techniques will be employed including descriptive analyses, psychometric analyses, multi-level modeling, and mixed-method analyses.DiscussionThree key challenging areas associated with conducting this project are discussed: sampling, participant recruitment, and sample retention; survey administration (with unregulated caregivers); and the provision of a stable set of study definitions to guide the project.

A systematic review of the psychometric properties of self-report research utilization measures used in healthcare.

BackgroundIn healthcare, a gap exists between what is known from research and what is practiced. Understanding this gap depends upon our ability to robustly measure research utilization.ObjectivesThe objectives of this systematic review were: to identify self-report measures of research utilization used in healthcare, and to assess the psychometric properties (acceptability, reliability, and validity) of these measures.MethodsWe conducted a systematic review of literature reporting use or development of self-report research utilization measures. Our search included: multiple databases, ancestry searches, and a hand search. Acceptability was assessed by examining time to complete the measure and missing data rates. Our approach to reliability and validity assessment followed that outlined in the Standards for Educational and Psychological Testing.ResultsOf 42,770 titles screened, 97 original studies (108 articles) were included in this review. The 97 studies reported on the use or development of 60 unique self-report research utilization measures. Seven of the measures were assessed in more than one study. Study samples consisted of healthcare providers (92 studies) and healthcare decision makers (5 studies). No studies reported data on acceptability of the measures. Reliability was reported in 32 (33%) of the studies, representing 13 of the 60 measures. Internal consistency (Cronbachs Alpha) reliability was reported in 31 studies; values exceeded 0.70 in 29 studies. Test-retest reliability was reported in 3 studies with Pearsons r coefficients > 0.80. No validity information was reported for 12 of the 60 measures. The remaining 48 measures were classified into a three-level validity hierarchy according to the number of validity sources reported in 50% or more of the studies using the measure. Level one measures (n = 6) reported evidence from any three (out of four possible) Standards validity sources (which, in the case of single item measures, was all applicable validity sources). Level two measures (n = 16) had evidence from any two validity sources, and level three measures (n = 26) from only one validity source.ConclusionsThis review reveals significant underdevelopment in the measurement of research utilization. Substantial methodological advances with respect to construct clarity, use of research utilization and related theory, use of measurement theory, and psychometric assessment are required. Also needed are improved reporting practices and the adoption of a more contemporary view of validity (i.e., the Standards) in future research utilization measurement studies.

Systematic reviews of complex interventions: framing the review question.

The first and most important decision in preparing any systematic review is to clearly frame the question the review team seeks to answer. However, this is not always straightforward, particularly if synthesis teams are interested in the effects of complex interventions. In this article, we discuss how to formulate good systematic review questions of complex interventions. We describe the rationale for developing well-formulated review questions and review the existing guidance on formulating review questions. We discuss that complex interventions can contain a mix of effective and ineffective (or even harmful) actions, which may interact synergistically or dysynergistically or be interdependent, and how these interactions and interdependencies need to be considered when formulating systematic review questions. We discuss complexity specifically in terms of how it relates to the type of question, the scope of the review (i.e., lumping vs. splitting debate), and specification of the intervention. We offer several recommendations to assist review authors in developing a definition for their complex intervention of interest, which is an essential first step in formulating the review question. We end by identifying areas in which future methodological research aimed at improving question formulation, especially as it relates to complex interventions, is needed.

Effects of shift length on quality of patient care and health provider outcomes: systematic review

Background: Healthcare providers work increasingly under a variety of shift work systems to cover the continuous care required by patients. However, the effects of shift work on patient and provider outcomes in healthcare settings has not been systematically evaluated. Objective: To identify and analyse the available evidence on the effect of shift length (8-h vs 12-h shifts) on quality of patient care and healthcare provider outcomes. Methods: Systematic searching of eight online databases, key governmental/organisational websites and academic journals with ancestry search of relevant articles (limited to articles published in English and Spanish). Results: Of 562 articles that were retrieved from 20 446 titles identified through database and manual searches, 27 satisfied the inclusion criteria, of which 15 were rejected because of low methodological quality. The 12 final studies included cross-sectional/survey (7), before–after (3) and prospective cohort (2) designs. The main primary outcomes evaluated were: (1) quality of patient care and (2) healthcare provider outcomes. The results were equivocal. With respect to the effect of shift length on quality of patient care, two studies found that errors and near errors were associated with working longer shifts, and another study reported decreased patient complications and length of stay with longer shifts. Specific healthcare provider outcomes such as health complaints, well-being, drug and alcohol consumption, stress and job satisfaction were mostly evaluated by single studies and therefore there was insufficient evidence from which to draw conclusions. Conclusions: Methodological quality of the studies generally was low and results equivocal with insufficient evidence to determine the effects of shift length on quality of patient care and healthcare provider outcomes. Clearly, robust well-designed studies are needed to examine the effect of shift length on patient and healthcare provider outcomes.

Translating research in elder care: an introduction to a study protocol series

BackgroundThe knowledge translation field is undermined by two interrelated gaps – underdevelopment of the science and limited use of research in health services and health systems decision making. The importance of context in theory development and successful translation of knowledge has been identified in past research. Additionally, examination of knowledge translation in the long-term care (LTC) sector has been seriously neglected, despite the fact that aging is increasingly identified as a priority area in health and health services research.AimsThe aims of this study are: to build knowledge translation theory about the role of organizational context in influencing knowledge use in LTC settings and among regulated and unregulated caregivers, to pilot knowledge translation interventions, and to contribute to enhanced use of new knowledge in LTC.DesignThis is a multi-level and longitudinal program of research comprising two main interrelated projects and a series of pilot studies. An integrated mixed method design will be used, including sequential and simultaneous phases to enable the projects to complement and inform one another. Inferences drawn from the quantitative and qualitative analyses will be merged to create meta-inferences.OutcomesOutcomes will include contributions to (knowledge translation) theory development, progress toward resolution of major conceptual issues in the field, progress toward resolution of methodological problems in the field, and advances in the design of effective knowledge translation strategies. Importantly, a better understanding of the contextual influences on knowledge use in LTC will contribute to improving outcomes for residents and providers in LTC settings.

Advancing the argument for validity of the Alberta Context Tool with healthcare aides in residential long-term care

BackgroundOrganizational context has the potential to influence the use of new knowledge. However, despite advances in understanding the theoretical base of organizational context, its measurement has not been adequately addressed, limiting our ability to quantify and assess context in healthcare settings and thus, advance development of contextual interventions to improve patient care. We developed the Alberta Context Tool (the ACT) to address this concern. It consists of 58 items representing 10 modifiable contextual concepts. We reported the initial validation of the ACT in 2009. This paper presents the second stage of the psychometric validation of the ACT.MethodsWe used the Standards for Educational and Psychological Testing to frame our validity assessment. Data from 645 English speaking healthcare aides from 25 urban residential long-term care facilities (nursing homes) in the three Canadian Prairie Provinces were used for this stage of validation. In this stage we focused on: (1) advanced aspects of internal structure (e.g., confirmatory factor analysis) and (2) relations with other variables validity evidence. To assess reliability and validity of scores obtained using the ACT we conducted: Cronbachs alpha, confirmatory factor analysis, analysis of variance, and tests of association. We also assessed the performance of the ACT when individual responses were aggregated to the care unit level, because the instrument was developed to obtain unit-level scores of context.ResultsItem-total correlations exceeded acceptable standards (> 0.3) for the majority of items (51 of 58). We ran three confirmatory factor models. Model 1 (all ACT items) displayed unacceptable fit overall and for five specific items (1 item on adequate space for resident care in the Organizational Slack-Space ACT concept and 4 items on use of electronic resources in the Structural and Electronic Resources ACT concept). This prompted specification of two additional models. Model 2 used the 7 scaled ACT concepts while Model 3 used the 3 count-based ACT concepts. Both models displayed substantially improved fit in comparison to Model 1. Cronbachs alpha for the 10 ACT concepts ranged from 0.37 to 0.92 with 2 concepts performing below the commonly accepted standard of 0.70. Bivariate associations between the ACT concepts and instrumental research utilization levels (which the ACT should predict) were statistically significant at the 5% level for 8 of the 10 ACT concepts. The majority (8/10) of the ACT concepts also showed a statistically significant trend of increasing mean scores when arrayed across the lowest to the highest levels of instrumental research use.ConclusionsThe validation process in this study demonstrated additional empirical support for construct validity of the ACT, when completed by healthcare aides in nursing homes. The overall pattern of the data was consistent with the structure hypothesized in the development of the ACT and supports the ACT as an appropriate measure for assessing organizational context in nursing homes. Caution should be applied in using the one space and four electronic resource items that displayed misfit in this study with healthcare aides until further assessments are made.

Assessment of variation in the alberta context tool: the contribution of unit level contextual factors and specialty in Canadian pediatric acute care settings

BackgroundThere are few validated measures of organizational context and none that we located are parsimonious and address modifiable characteristics of context. The Alberta Context Tool (ACT) was developed to meet this need. The instrument assesses 8 dimensions of context, which comprise 10 concepts. The purpose of this paper is to report evidence to further the validity argument for ACT. The specific objectives of this paper are to: (1) examine the extent to which the 10 ACT concepts discriminate between patient care units and (2) identify variables that significantly contribute to between-unit variation for each of the 10 concepts.Methods859 professional nurses (844 valid responses) working in medical, surgical and critical care units of 8 Canadian pediatric hospitals completed the ACT. A random intercept, fixed effects hierarchical linear modeling (HLM) strategy was used to quantify and explain variance in the 10 ACT concepts to establish the ACTs ability to discriminate between units. We ran 40 models (a series of 4 models for each of the 10 concepts) in which we systematically assessed the unique contribution (i.e., error variance reduction) of different variables to between-unit variation. First, we constructed a null model in which we quantified the variance overall, in each of the concepts. Then we controlled for the contribution of individual level variables (Model 1). In Model 2, we assessed the contribution of practice specialty (medical, surgical, critical care) to variation since it was central to construction of the sampling frame for the study. Finally, we assessed the contribution of additional unit level variables (Model 3).ResultsThe null model (unadjusted baseline HLM model) established that there was significant variation between units in each of the 10 ACT concepts (i.e., discrimination between units). When we controlled for individual characteristics, significant variation in the 10 concepts remained. Assessment of the contribution of specialty to between-unit variation enabled us to explain more variance (1.19% to 16.73%) in 6 of the 10 ACT concepts. Finally, when we assessed the unique contribution of the unit level variables available to us, we were able to explain additional variance (15.91% to 73.25%) in 7 of the 10 ACT concepts.ConclusionThe findings reported here represent the third published argument for validity of the ACT and adds to the evidence supporting its use to discriminate patient care units by all 10 contextual factors. We found evidence of relationships between a variety of individual and unit-level variables that explained much of this between-unit variation for each of the 10 ACT concepts. Future research will include examination of the relationships between the ACTs contextual factors and research utilization by nurses and ultimately the relationships between context, research utilization, and outcomes for patients.

Validation of the conceptual research utilization scale: an application of the standards for educational and psychological testing in healthcare.

BackgroundThere is a lack of acceptable, reliable, and valid survey instruments to measure conceptual research utilization (CRU). In this study, we investigated the psychometric properties of a newly developed scale (the CRU Scale).MethodsWe used the Standards for Educational and Psychological Testing as a validation framework to assess four sources of validity evidence: content, response processes, internal structure, and relations to other variables. A panel of nine international research utilization experts performed a formal content validity assessment. To determine response process validity, we conducted a series of one-on-one scale administration sessions with 10 healthcare aides. Internal structure and relations to other variables validity was examined using CRU Scale response data from a sample of 707 healthcare aides working in 30 urban Canadian nursing homes. Principal components analysis and confirmatory factor analyses were conducted to determine internal structure. Relations to other variables were examined using: (1) bivariate correlations; (2) change in mean values of CRU with increasing levels of other kinds of research utilization; and (3) multivariate linear regression.ResultsContent validity index scores for the five items ranged from 0.55 to 1.00. The principal components analysis predicted a 5-item 1-factor model. This was inconsistent with the findings from the confirmatory factor analysis, which showed best fit for a 4-item 1-factor model. Bivariate associations between CRU and other kinds of research utilization were statistically significant (p < 0.01) for the latent CRU scale score and all five CRU items. The CRU scale score was also shown to be significant predictor of overall research utilization in multivariate linear regression.ConclusionsThe CRU scale showed acceptable initial psychometric properties with respect to responses from healthcare aides in nursing homes. Based on our validity, reliability, and acceptability analyses, we recommend using a reduced (four-item) version of the CRU scale to yield sound assessments of CRU by healthcare aides. Refinement to the wording of one item is also needed. Planned future research will include: latent scale scoring, identification of variables that predict and are outcomes to conceptual research use, and longitudinal work to determine CRU Scale sensitivity to change.

Methods for designing interventions to change healthcare professionals’ behaviour: a systematic review

BackgroundSystematic reviews consistently indicate that interventions to change healthcare professional (HCP) behaviour are haphazardly designed and poorly specified. Clarity about methods for designing and specifying interventions is needed. The objective of this review was to identify published methods for designing interventions to change HCP behaviour.MethodsA search of MEDLINE, Embase, and PsycINFO was conducted from 1996 to April 2015. Using inclusion/exclusion criteria, a broad screen of abstracts by one rater was followed by a strict screen of full text for all potentially relevant papers by three raters. An inductive approach was first applied to the included studies to identify commonalities and differences between the descriptions of methods across the papers. Based on this process and knowledge of related literatures, we developed a data extraction framework that included, e.g. level of change (e.g. individual versus organization); context of development; a brief description of the method; tasks included in the method (e.g. barrier identification, component selection, use of theory).Results3966 titles and abstracts and 64 full-text papers were screened to yield 15 papers included in the review, each outlining one design method. All of the papers reported methods developed within a specific context. Thirteen papers included barrier identification and 13 included linking barriers to intervention components; although not the same 13 papers. Thirteen papers targeted individual HCPs with only one paper targeting change across individual, organization, and system levels. The use of theory and user engagement were included in 13/15 and 13/15 papers, respectively.ConclusionsThere is an agreement across methods of four tasks that need to be completed when designing individual-level interventions: identifying barriers, selecting intervention components, using theory, and engaging end-users. Methods also consist of further additional tasks. Examples of methods for designing the organisation and system-level interventions were limited. Further analysis of design tasks could facilitate the development of detailed guidelines for designing interventions.