Jo Rick

A conceptual framework for implementation fidelity.

BackgroundImplementation fidelity refers to the degree to which an intervention or programme is delivered as intended. Only by understanding and measuring whether an intervention has been implemented with fidelity can researchers and practitioners gain a better understanding of how and why an intervention works, and the extent to which outcomes can be improved.DiscussionThe authors undertook a critical review of existing conceptualisations of implementation fidelity and developed a new conceptual framework for understanding and measuring the process. The resulting theoretical framework requires testing by empirical research.SummaryImplementation fidelity is an important source of variation affecting the credibility and utility of research. The conceptual framework presented here offers a means for measuring this variable and understanding its place in the process of intervention implementation.

Workplace involvement improves return to work rates among employees with back pain on long-term sick leave: a systematic review of the effectiveness and cost-effectiveness of interventions

Purpose. Long-term sickness absence among workers is a major problem in industrialised countries. The aim of the review is to determine whether interventions involving the workplace are more effective and cost-effective at helping employees on sick leave return to work than those that do not involve the workplace at all. Methods. A systematic review of controlled intervention studies and economic evaluations. Sixteen electronic databases and grey literature sources were searched, and reference and citation tracking was performed on included publications. A narrative synthesis was performed. Results. Ten articles were found reporting nine trials from Europe and Canada, and four articles were found evaluating the cost-effectiveness of interventions. The population in eight trials suffered from back pain and related musculoskeletal conditions. Interventions involving employees, health practitioners and employers working together, to implement work modifications for the absentee, were more consistently effective than other interventions. Early intervention was also found to be effective. The majority of trials were of good or moderate quality. Economic evaluations indicated that interventions with a workplace component are likely to be more cost effective than those without. Conclusion. Stakeholder participation and work modification are more effective and cost effective at returning to work adults with musculoskeletal conditions than other workplace-linked interventions, including exercise.

Validation of the General Health Questionnaire-12: Using a sample of employees from England's health care services.

Psychiatric interviews were carried out to validate the General Health Questionnaire-12 (GHQ-12; Goldberg, 1972) for use with staff of Englands National Health Service (NHS), and to determine the appropriate threshold score to identify probable cases. In a sample of 551 NHS staff, the correlation between the GHQ-12 and the Clinical Interview Schedule-Revised (CIS-R) was found to be .70. The receiver operating characteristic showed that a 3/4 threshold, higher than used in all but one previous study, gave the best conservative estimate of minor psychiatric morbidity. This threshold gave an estimated sensitivity of the GHQ-12 of.69 and specificity of.88. It reduced case rates by between 8% and 17% as compared with lower possible thresholds. Convergent and discriminant validation of the GHQ-12 was demonstrated through comparison with other measures of mental and physical health.

Measures of perceived work characteristics for health services research: Test of a measurement model and normative data

Objectives. The aim of this paper is to describe the development and refinement of nine measures of perceived work characteristics for use in studies of the psychological wellbeing of health services employees. The constructs measured are: autonomy/control, feedback, influence, leader support, professional compromise, role clarity, role conflict, peer support and work demands. Design. The study was a large scale cross-sectional survey. Methods. National Health Service Trust staff were selected from seven major occupational groups (nurses, doctors, administrative staff, managers, professions allied to medicine, professional and technical staff, and ancillary staff) within a sample of Trusts chosen to be representative of those in England. Questionnaires incorporating the scale items were completed by over 9,000 staff. The fit with the a priori nine-dimensional measurement model was tested using confirmatory factor analysis. Scale internal reliabilities and norms were calculated, and aspects of construct validity were examined. Results. The findings show a good fit to the measurement model for the sample as a whole and across occupational groups. All the scales exhibit acceptable internal reliabilities; and normative data is provided for all seven occupations (with norms for more specific job categories available on request). Analyses suggest good construct validity, showing the scales discriminate as anticipated across occupational groups and job categories, and they have the expected relationships with job satisfaction and psychological strain. Conclusions. The measures developed are usable for research purposes across the major occupational groups, with the possible exception of ancillary staff. Further work is desirable to confirm the measurement properties and extend the normative database.

“Best fit” framework synthesis: refining the method

BackgroundFollowing publication of the first worked example of the “best fit” method of evidence synthesis for the systematic review of qualitative evidence in this journal, the originators of the method identified a need to specify more fully some aspects of this particular derivative of framework synthesis.Methods and ResultsWe therefore present a second such worked example in which all techniques are defined and explained, and their appropriateness is assessed. Specified features of the method include the development of new techniques to identify theories in a systematic manner; the creation of an a priori framework for the synthesis; and the “testing” of the synthesis. An innovative combination of existing methods of quality assessment, analysis and synthesis is used to complete the process. This second worked example was a qualitative evidence synthesis of employees’ views of workplace smoking cessation interventions, in which the “best fit” method was found to be practical and fit for purpose.ConclusionsThe method is suited to producing context-specific conceptual models for describing or explaining the decision-making and health behaviours of patients and other groups. It offers a pragmatic means of conducting rapid qualitative evidence synthesis and generating programme theories relating to intervention effectiveness, which might be of relevance both to researchers and policy-makers.

Systematic review of the links between human resource management practices and performance

BACKGROUND In recent years human resource management (HRM) has been seen as an important factor in the successful realisation of organisational change programmes. The UK NHS is undergoing substantial organisational change and there is a need to establish which human resource (HR) initiatives may be most effective. OBJECTIVES To assess the results from a wide-ranging series of systematic reviews of the evidence on HRM and performance. The first part assesses evidence on use of HRM in the UK and fidelity of practice implemented. The second part considers evidence for the impact of HRM practices on intermediate outcomes, which can impact on final outcomes, such as organisational performance or patient care. DATA SOURCES The following databases were searched: Applied Social Sciences Index and Abstracts (ASSIA), British Nursing Index (BNI), Business Source Premier, Campbell Collaboration, Cochrane Central Register of Controlled Trials (CENTRAL), Cochrane Database of Systematic Reviews (CDSR), Cumulative Index to Nursing and Allied Health Literature (CINAHL), Database of Abstracts of Reviews of Effectiveness (DARE), DH-Data, EMBASE, Health Management Information Consortium (HMIC), International Bibliography of the Social Sciences (IBSS), Kings Fund database, MEDLINE, NHS Economic Evaluation Database (NHS EED), National Research Register (NRR), PREMEDLINE, PsycINFO, ReFeR, Social Sciences Citation Index (SSCI) and Science Citation Index (SCI). The searches were conducted in May/June 2006. REVIEW METHODS Broad categories of HRM interventions and intermediate outcomes were generated: 10 HRM categories and 12 intermediate outcome categories. Seven patient final outcomes were derived from the NHS Performance Indicators and the NHS Improvement Plan. The quality criteria used to select papers incorporated a longitudinal study design filter to provide evidence of the causal direction of relationships between HRM and relevant outcomes. Single HRM practices were considered. Within the health-specific literature, focus was on the impact of HRM on patient outcomes. Information is presented on the reliability of measures in each of the intermediate outcome areas. RESULTS Work design practices that enhance employee autonomy and control influenced a number of outcomes and there was consistent evidence for the positive impact of increased job control on employee outcomes, such as job satisfaction, absence and health. For employee participation, the small number of studies reviewed supported the involvement of employees in design/implementation of changes that affect their work. In health literature in particular, employee involvement through quality improvement teams resulted in improved patient outcomes. Findings were positive for the impact of training on the intended outcomes of the initiatives. Support for the impact of performance management practices was apparent, in particular feedback on performance outcomes and the use of participative goal setting. Strong associations were found among all intermediate outcomes, and the relationship between most intermediate behaviours and outcomes were significant. LIMITATIONS Limited evidence was available on the use of HRM and on the implementation of policy. Also, the specific practices studied within each HRM category differ so there was little evidence to show whether similar practices have the same effects in health and non-health settings. CONCLUSIONS Some potentially effective practices for both health and non-health areas were identified, and HRM methods could be used to support change processes within the NHS; the findings relating to work organisation are particularly promising with regard to changes in methods of service delivery. Using training to support the implementation of change is highlighted. However, future multilevel studies that embrace the individual, team and organisational level are needed. Studies should look into interventions aimed at improving HR outcomes and performance, and allow for pre- and post-intervention measurement of practices and outcomes.

Psychometric properties of the patient assessment of chronic illness care measure: acceptability, reliability and validity in United Kingdom patients with long-term conditions

BackgroundThe Patient Assessment of Chronic Illness Care (PACIC) is a US measure of chronic illness quality of care, based on the influential Chronic Care Model (CCM). It measures a number of aspects of care, including patient activation; delivery system design and decision support; goal setting and tailoring; problem-solving and contextual counselling; follow-up and coordination. Although there is developing evidence of the utility of the scale, there is little evidence about its performance in the United Kingdom (UK). We present preliminary data on the psychometric performance of the PACIC in a large sample of UK patients with long-term conditions.MethodWe collected PACIC, demographic, clinical and quality of care data from patients with long-term conditions across 38 general practices, as part of a wider longitudinal study. We assess rates of missing data, present descriptive and distributional data, assess internal consistency, and test validity through confirmatory factor analysis, and through associations between PACIC scores, patient characteristics and related measures.ResultsThere was evidence that rates of missing data were high on PACIC (9.6% - 15.9%), and higher than on other scales used in the same survey. Most PACIC sub-scales showed reasonable levels of internal consistency (alpha = 0.68 – 0.94), responses did not demonstrate high skewness levels, and floor effects were more frequent (up to 30.4% on the follow up and co-ordination subscale) than ceiling effects (generally <5%). PACIC demonstrated preliminary evidence of validity in terms of measures of long-term condition care. Confirmatory factor analysis suggested that the five factor PACIC structure proposed by the scale developers did not fit the data: reporting separate factor scores may not always be appropriate.ConclusionThe importance of improving care for long-term conditions means that the development and validation of measures is a priority. The PACIC scale has demonstrated potential utility in this regard, but further assessment is required to assess low levels of completion of the scale, and to explore the performance of the scale in predicting outcomes and assessing the effects of interventions.

Systematic techniques for assisting recruitment to trials (START): study protocol for embedded, randomized controlled trials.

BackgroundRandomized controlled trials play a central role in evidence-based practice, but recruitment of participants, and retention of them once in the trial, is challenging. Moreover, there is a dearth of evidence that research teams can use to inform the development of their recruitment and retention strategies. As with other healthcare initiatives, the fairest test of the effectiveness of a recruitment strategy is a trial comparing alternatives, which for recruitment would mean embedding a recruitment trial within an ongoing host trial. Systematic reviews indicate that such studies are rare. Embedded trials are largely delivered in an ad hoc way, with interventions almost always developed in isolation and tested in the context of a single host trial, limiting their ability to contribute to a body of evidence with regard to a single recruitment intervention and to researchers working in different contexts.Methods/DesignThe Systematic Techniques for Assisting Recruitment to Trials (START) program is funded by the United Kingdom Medical Research Council (MRC) Methodology Research Programme to support the routine adoption of embedded trials to test standardized recruitment interventions across ongoing host trials. To achieve this aim, the program involves three interrelated work packages: (1) methodology - to develop guidelines for the design, analysis and reporting of embedded recruitment studies; (2) interventions - to develop effective and useful recruitment interventions; and (3) implementation - to recruit host trials and test interventions through embedded studies.DiscussionSuccessful completion of the START program will provide a model for a platform for the wider trials community to use to evaluate recruitment interventions or, potentially, other types of intervention linked to trial conduct. It will also increase the evidence base for two types of recruitment intervention.Trial registrationThe START protocol covers the methodology for embedded trials. Each embedded trial is registered separately or as a substudy of the host trial.

The influences of gender and grade on perceived work stress and job satisfaction in white collar employees

Abstract A self-completion questionnaire survey was carried out, resulting in a sample of 675 white collar public sector employees. The survey examined the frequency of occurrence of work-based stressors and perceived control, and included a general measure of job satisfaction. The results revealed that higher levels of job satisfaction were reported by employees in higher grades. It was also observed that higher grades perceived more control within their working environment. No gender differenccs were found concerning reported stress problems, although significant differences were observed across grades in relation to role differentiation. Multivariate analyses revealed that grade effects were largely accounted for by differences in perceived control. Perceived control, role-based and organizational stressors as well as gender were the strongest contributors in predicting reported job satisfaction.

How do nurses, midwives and health visitors contribute to protocol-based care? A synthesis of the UK literature

OBJECTIVES To explore how nurses, midwives and health visitors contribute to the development, implementation and audit of protocol-based care. Protocol-based care refers to the use of documents that set standards for clinical care processes with the intent of reducing unacceptable variations in practice. Documents such as protocols, clinical guidelines and care pathways underpin evidence-based practice throughout the world. METHODS An interpretative review using the five-stage systematic literature review process. The data sources were the British Nursing Index, CINAHL, EMBASE, MEDLINE and Web of Science from onset to 2005. The Journal of Integrated Care Pathways was hand searched (1997-June 2006). Thirty three studies about protocol-based care in the United Kingdom were appraised using the Qualitative Assessment and Review Instrument (QARI version 2). The literature was synthesized inductively and deductively, using an official 12-step guide for development as a framework for the deductive synthesis. RESULTS Most papers were descriptive, offering practitioner knowledge and positive findings about a locally developed and owned protocol-based care. The majority were instigated in response to clinical need or service re-design. Development of protocol-based care was a non-linear, idiosyncratic process, with steps omitted, repeated or completed in a different order. The context and the multiple purposes of protocol-based care influenced the development process. Implementation and sustainability were rarely mentioned, or theorised as a change. The roles and activities of nurses were so understated as to be almost invisible. There were notable gaps in the literature about the resource use costs, the engagement of patients in the decision-making process, leadership and the impact of formalisation and new roles on inter-professional relations. CONCLUSIONS Documents that standardise clinical care are part of the history of nursing as well as contemporary evidence-based care and expanded roles. Considering the proliferation and contested nature of protocol-based care, the dearth of literature about the contribution, experience and outcomes for nurses, midwives and health visitors is noteworthy and requires further investigation.