H.G.A. Mokkink

Attributes of clinical guidelines that influence use of guidelines in general practice: observational study

Abstract Objective: To determine which attributes of clinical practice guidelines influence the use of guidelines in decision making in clinical practice.. Design: Observational study relating the use of 47 different recommendations from 10 national clinical guidelines to 12 different attributes of clinical guidelines—for example, evidence based, controversial, concrete. Setting: General practice in the Netherlands. Subjects: 61 general practitioners who made 12 880 decisions in their contacts with patients. Main outcome measures: Compliance of decisions with clinical guidelines according to the attribute of the guideline. Results: Recommendations were followed in, on average, 61% (7915/12 880) of the decisions. Controversial recommendations were followed in 35% (886/2497) of decisions and non-controversial recommendations in 68% (7029/10 383) of decisions. Vague and non-specific recommendations were followed in 36% (826/2280) of decisions and clear recommendations in 67% (7089/10 600) of decisions. Recommendations that demanded a change in existing practice routines were followed in 44% (1278/2912) of decisions and those that did not in 67% (6637/9968) of decisions. Evidence based recommendations were used more than recommendations for practice that were not based on research evidence (71% (2745/3841) v 57% (5170/9039)). Conclusions: People and organisations setting evidence based clinical practice guidelines should take into account some of the other important attributes of effective recommendations for clinical practice.

Safety of telephone triage in general practitioner cooperatives: do triage nurses correctly estimate urgency?

Background: In recent years, there has been a growth in the use of triage nurses to decrease general practitioner (GP) workloads and increase the efficiency of telephone triage. The actual safety of decisions made by triage nurses has not yet been assessed. Objectives: To investigate whether triage nurses accurately estimate the urgency level of health complaints when using the national telephone guidelines, and to examine the relationship between the performance of triage nurses and their education and training. Method: A cross-sectional, multicentre, observational study employing five mystery (simulated) patients who telephoned triage nurses in four GP cooperatives. The mystery patients played standardised roles. Each role had one of four urgency levels as determined by experts. The triage nurses called were asked to estimate the level of urgency after the contact. This level of urgency was compared with a gold standard. Results: Triage nurses estimated the level of urgency of 69% of the 352 contacts correctly and underestimated the level of urgency of 19% of the contacts. The sensitivity and specificity of the urgency estimates provided by the triage nurses were found to be 0.76 and 0.95, respectively. The positive and negative predictive values of the urgency estimates were 0.83 and 0.93, respectively. A significant correlation was found between correct estimation of urgency and specific training on the use of the guidelines. The educational background (primary or secondary care) of the nurses had no significant relationship with the rate of underestimation. Conclusion: Telephone triage by triage nurses is efficient but possibly not safe, with potentially severe consequences for the patient. An educational programme for triage nurses is recommended. Also, a direct second safety check of all cases by a specially trained GP telephone doctor is advisable.

In-training assessment using direct observation of single-patient encounters: a literature review

We reviewed the literature on instruments for work-based assessment in single clinical encounters, such as the mini-clinical evaluation exercise (mini-CEX), and examined differences between these instruments in characteristics and feasibility, reliability, validity and educational effect. A PubMed search of the literature published before 8 January 2009 yielded 39 articles dealing with 18 different assessment instruments. One researcher extracted data on the characteristics of the instruments and two researchers extracted data on feasibility, reliability, validity and educational effect. Instruments are predominantly formative. Feasibility is generally deemed good and assessor training occurs sparsely but is considered crucial for successful implementation. Acceptable reliability can be achieved with 10 encounters. The validity of many instruments is not investigated, but the validity of the mini-CEX and the ‘clinical evaluation exercise’ is supported by strong and significant correlations with other valid assessment instruments. The evidence from the few studies on educational effects is not very convincing. The reports on clinical assessment instruments for single work-based encounters are generally positive, but supporting evidence is sparse. Feasibility of instruments seems to be good and reliability requires a minimum of 10 encounters, but no clear conclusions emerge on other aspects. Studies on assessor and learner training and studies examining effects beyond ‘happiness data’ are badly needed.

Patients either contacting a general practice cooperative or accident and emergency department out of hours: a comparison

Introduction: Lack of collaboration between general practice (GP) cooperatives and accident and emergency (A&E) departments and many self referrals may lead to inefficient out-of-hours care. Methods: We retrospectively analysed the records of all patients contacting the GP cooperative and all patients self referring to the A&E department out of hours in a region in the Netherlands. Results: 258 patients contacted the GP cooperative and 43 self referred to the A&E department per 1000 patients per year. A wide range of problems were seen in the GP cooperative, mainly related to infections (26.2%). The A&E department had a smaller range of problems, mainly related to trauma (66.1%). Relatively few urgent problems were seen in the GP cooperative (4.6%) or for self referrals in the A&E department (6.1%). Women, children, the elderly, and rural patients chose the GP cooperative significantly more often, as did men and patients with less urgent complaints, infections, and heart and airway problems. Discussion: The contact frequency of self referrals to the A&E department is much lower than that at the GP cooperative. Care is complementary: the A&E department focuses on trauma while the GP cooperative deals with a wide range of problems. The self referrals concern mostly minor, non-urgent problems and can generally be treated by the general practitioner, by a nurse, or by advice over the telephone, particularly in the case of optimal collaboration in an integrated care facility of GP cooperatives and A&E departments with one access point to medical care for all patients.

Changing preventive practice: a controlled trial on the effects of outreach visits to organise prevention of cardiovascular disease.

OBJECTIVES: To assess the effects of outreach visits by trained nurse facilitators on the organisation of services used to prevent cardiovascular disease. To identify the characteristics of general practices that determined success. DESIGN: A non-randomised controlled trial of two methods of implementing guidelines to organise prevention of cardiovascular disease: an innovative outreach visit method compared with a feedback method. The results in both groups were compared with data from a control group. SETTING AND SUBJECTS: 95 general practices in two regions in The Netherlands. INTERVENTIONS: Trained nurse facilitators visited practices, focusing on solving problems in the organisation of prevention. They applied a four step model in each practice. The number of visits depended on the needs of the practice team. The feedback method consisted of the provision of a feedback report with advice specific to each practice and standardised instructions. MAIN OUTCOME MEASURES: The proportion of practices adhering to 10 different guidelines. Guidelines were on the detection of patients at risk, their follow up, the registration of preventive activities, and teamwork within the practice. RESULTS: Outreach visits were more effective than feedback in implementing guidelines to organise prevention. Within the group with outreach visits, the increase in the number of practices adhering to the guidelines was significant for six out of 10 guidelines. Within the feedback group, a comparison of data before and after intervention showed no significant differences. Partnerships and practices with a computer changed more. CONCLUSION: Outreach visits by trained nurse facilitators proved to be effective in implementing guidelines within general practices, probably because their help was practical and designed for the individual practice, guided by the wishes and capabilities of the practice team.

The process of feedback in workplace-based assessment: organisation, delivery, continuity

Medical Education 2012: 46:604–612

Ethnic specific recommendations in clinical practice guidelines: a first exploratory comparison between guidelines from the USA, Canada, the UK, and the Netherlands

Objectives: To investigate whether clinical practice guidelines in different countries take ethnic differences between patients into consideration and to assess the scientific foundation of such ethnic specific recommendations. Design: Analysis of the primary care sections of clinical practice guidelines. Setting: Primary care practice guidelines for type 2 diabetes mellitus, hypertension, and asthma developed in the USA, Canada, the UK, and the Netherlands. Main outcome measures: Enumeration of the ethnic specific information and recommendations in the guidelines, and the scientific basis and strength of this evidence. Results: Different guidelines do address ethnic differences between patients, but to a varying extent. The USA guidelines contained the most ethnic specific statements and the Dutch guidelines the least. Most ethnic specific statements were backed by scientific evidence, usually arising from descriptive studies or narrative reviews. Conclusion: The attention given to ethnic differences between patients in clinical guidelines varies between countries. Guideline developers should be aware of the potential problems of ignoring differences in ethnicity.

Effects of the vocational training of general practice consultation skills and medical performance

Summary. The effects of the vocational training of general practitioners in the Netherlands on the consultation skills and medical performance of junior doctors were studied. Results obtained at a training institute providing systematic training in these skills (Nijmegen) were compared with those at an institute taking a problem‐based learning approach (Groningen). Trainees (n =63) audiotaped consultations and recorded their medical performance at the start and at completion of training. The skills were evaluated with the aid of validated criteria and medical ‘protocols’. Data on 631 pre‐training and 624 post‐training consultations were compared. Changes in consultation skills and medical performance occurred at both institutes and proved more marked at the institute providing systematic training. Improved medical performance was found to be associated with improved consultation skills. Enhanced clinical knowledge was found to be related to improved medical performance and consultation skills. The most profound changes were found in junior doctors who had started at a lower level of consultation skills and medical performance.

Reflection as a component of formative assessment appears to be instrumental in promoting the use of feedback; an observational study

Background: Although the literature suggests that reflection has a positive impact on learning, there is a paucity of evidence to support this notion. Aim: We investigated feedback and reflection in relation to the likelihood that feedback will be used to inform action plans. We hypothesised that feedback and reflection present a cumulative sequence (i.e. trainers only pay attention to trainees’ reflections when they provided specific feedback) and we hypothesised a supplementary effect of reflection. Method: We analysed copies of assessment forms containing trainees’ reflections and trainers’ feedback on observed clinical performance. We determined whether the response patterns revealed cumulative sequences in line with the Guttman scale. We further examined the relationship between reflection, feedback and the mean number of specific comments related to an action plan (ANOVA) and we calculated two effect sizes. Results: Both hypotheses were confirmed by the results. The response pattern found showed an almost perfect fit with the Guttman scale (0.99) and reflection seems to have supplementary effect on the variable action plan. Conclusions: Reflection only occurs when a trainer has provided specific feedback; trainees who reflect on their performance are more likely to make use of feedback. These results confirm findings and suggestions reported in the literature.

Quality of written narrative feedback and reflection in a modified mini-clinical evaluation exercise: an observational study

BackgroundResearch has shown that narrative feedback, (self) reflections and a plan to undertake and evaluate improvements are key factors for effective feedback on clinical performance. We investigated the quantity of narrative comments comprising feedback (by trainers), self-reflections (by trainees) and action plans (by trainer and trainee) entered on a mini-CEX form that was modified for use in general practice training and to encourage trainers and trainees to provide narrative comments. In view of the importance of specificity as an indicator of feedback quality, we additionally examined the specificity of the comments.MethodWe collected and analysed modified mini-CEX forms completed by GP trainers and trainees. Since each trainee has the same trainer for the duration of one year, we used trainer-trainee pairs as the unit of analysis. We determined for all forms the frequency of the different types of narrative comments and rated their specificity on a three-point scale: specific, moderately specific, not specific. Specificity was compared between trainee-trainer pairs.ResultsWe collected 485 completed modified mini-CEX forms from 54 trainees (mean of 8.8 forms per trainee; range 1–23; SD 5.6). Trainer feedback was more frequently provided than trainee self-reflections, and action plans were very rare. The comments were generally specific, but showed large differences between trainee-trainer pairs.ConclusionThe frequency of self-reflection and action plans varied, all comments were generally specific and there were substantial and consistent differences between trainee-trainer pairs in the specificity of comments. We therefore conclude that feedback is not so much determined by the instrument as by the users. Interventions to improve the educational effects of the feedback procedure should therefore focus more on the users than on the instruments.