Andrea R. Horvath

Harmonising evidence-based medicine teaching : a study of the outcomes of e-learning in five European countries

BackgroundWe developed and evaluated the outcomes of an e-learning course for evidence based medicine (EBM) training in postgraduate medical education in different languages and settings across five European countries.MethodsWe measured changes in knowledge and attitudes with well-developed assessment tools before and after administration of the course. The course consisted of five e-learning modules covering acquisition (formulating a question and search of the literature), appraisal, application and implementation of findings from systematic reviews of therapeutic interventions, each with interactive audio-visual learning materials of 15 to 20 minutes duration. The modules were prepared in English, Spanish, German and Hungarian. The course was delivered to 101 students from different specialties in Germany (psychiatrists), Hungary (mixture of specialties), Spain (general medical practitioners), Switzerland (obstetricians-gynaecologists) and the UK (obstetricians-gynaecologists). We analysed changes in scores across modules and countries.ResultsOn average across all countries, knowledge scores significantly improved from pre- to post-course for all five modules (p < 0.001). The improvements in scores were on average 1.87 points (14% of total score) for module 1, 1.81 points (26% of total score) for module 2, 1.9 points (11% of total score) for module 3, 1.9 points (12% of total score) for module 4 and 1.14 points (14% of total score) for module 5. In the country specific analysis, knowledge gain was not significant for module 4 in Spain, Switzerland and the UK, for module 3 in Spain and Switzerland and for module 2 in Spain. Compared to pre-course assessment, after completing the course participants felt more confident that they can assess research evidence and that the healthcare system in their country should have its own programme of research about clinical effectiveness.ConclusionE-learning in EBM can be harmonised for effective teaching and learning in different languages, educational settings and clinical specialties, paving the way for development of an international e-EBM course.

Effectiveness of an e-learning course in evidence-based medicine for foundation (internship) training

Summary Aim To evaluate the educational effectiveness of a clinically integrated e-learning course for teaching basic evidence-based medicine (EBM) among postgraduate medical trainees compared to a traditional lecture-based course of equivalent content. Methods We conducted a cluster randomized controlled trial to compare a clinically integrated e-learning EBM course (intervention) to a lecture-based course (control) among postgraduate trainees at foundation or internship level in seven teaching hospitals in the UK West Midlands region. Knowledge gain among participants was measured with a validated instrument using multiple choice questions. Change in knowledge was compared between groups taking into account the cluster design and adjusted for covariates at baseline using generalized estimating equations (GEE) model. Results There were seven clusters involving teaching of 237 trainees (122 in the intervention and 115 in the control group). The total number of postgraduate trainees who completed the course was 88 in the intervention group and 72 in the control group. After adjusting for baseline knowledge, there was no difference in the amount of improvement in knowledge of EBM between the two groups. The adjusted post course difference between the intervention group and the control group was only 0.1 scoring points (95% CI −1.2–1.4). Conclusion An e-learning course in EBM was as effective in improving knowledge as a standard lecture-based course. The benefits of an e-learning approach need to be considered when planning EBM curricula as it allows standardization of teaching materials and is a potential cost-effective alternative to standard lecture-based teaching.

The effectiveness of a clinically integrated e-learning course in evidence-based medicine: A cluster randomised controlled trial

BackgroundTo evaluate the educational effects of a clinically integrated e-learning course for teaching basic evidence-based medicine (EBM) among postgraduates compared to a traditional lecture-based course of equivalent content.MethodsWe conducted a cluster randomised controlled trial in the Netherlands and the UK involving postgraduate trainees in six obstetrics and gynaecology departments. Outcomes (knowledge gain and change in attitude towards EBM) were compared between the clinically integrated e-learning course (intervention) and the traditional lecture based course (control). We measured change from pre- to post-intervention scores using a validated questionnaire assessing knowledge (primary outcome) and attitudes (secondary outcome).ResultsThere were six clusters involving teaching of 61 postgraduate trainees (28 in the intervention and 33 in the control group). The intervention group achieved slightly higher scores for knowledge gain compared to the control, but these results were not statistically significant (difference in knowledge gain: 3.5 points, 95% CI -2.7 to 9.8, p = 0.27). The attitudinal changes were similar for both groups.ConclusionA clinically integrated e-learning course was at least as effective as a traditional lecture based course and was well accepted. Being less costly than traditional teaching and allowing for more independent learning through materials that can be easily updated, there is a place for incorporating e-learning into postgraduate EBM curricula that offer on-the-job training for just-in-time learning.Trial registrationTrial registration number: ACTRN12609000022268.

A clinically integrated curriculum in evidence-based medicine for just-in-time learning through on-the-job training: the EU-EBM project.

BackgroundOver the last years key stake holders in the healthcare sector have increasingly recognised evidence based medicine (EBM) as a means to improving the quality of healthcare. However, there is considerable uncertainty about the best way to disseminate basic knowledge of EBM. As a result, huge variation in EBM educational provision, setting, duration, intensity, content, and teaching methodology exists across Europe and worldwide. Most courses for health care professionals are delivered outside the work context (stand alone) and lack adaptation to the specific needs for EBM at the learners workplace. Courses with modern adaptive EBM teaching that employ principles of effective continuing education might fill that gap. We aimed to develop a course for post-graduate education which is clinically integrated and allows maximum flexibility for teachers and learners.MethodsA group of experienced EBM teachers, clinical epidemiologists, clinicians and educationalists from institutions from eight European countries participated. We used an established methodology of curriculum development to design a clinically integrated EBM course with substantial components of e-learning. An independent European steering committee provided input into the process.ResultsWe defined explicit learning objectives about knowledge, skills, attitudes and behaviour for the five steps of EBM. A handbook guides facilitator and learner through five modules with clinical and e-learning components. Focussed activities and targeted assignments round off the learning process, after which each module is formally assessed.ConclusionThe course is learner-centred, problem-based, integrated with activities in the workplace and flexible. When successfully implemented, the course is designed to provide just-in-time learning through on-the-job-training, with the potential for teaching and learning to directly impact on practice.

Teaching trainers to incorporate evidence-based medicine (EBM) teaching in clinical practice: the EU-EBM project.

BackgroundEvidence based medicine (EBM) is considered an integral part of medical training, but integration of teaching various EBM steps in everyday clinical practice is uncommon. Currently EBM is predominantly taught through theoretical courses, workshops and e-learning. However, clinical teachers lack confidence in teaching EBM in workplace and are often unsure of the existing opportunities for teaching EBM in the clinical setting. There is a need for continuing professional development (CPD) courses that train clinical trainers to teach EBM through on-the-job training by demonstration of applied EBM real time in clinical practice. We developed such a course to encourage clinically relevant teaching of EBM in post-graduate education in various clinical environments.MethodsWe devised an e-learning course targeting trainers with EBM knowledge to impart educational methods needed to teach application of EBM teaching in commonly used clinical settings. The curriculum development group comprised experienced EBM teachers, clinical epidemiologists, clinicians and educationalists from institutions in seven European countries. The e-learning sessions were designed to allow participants (teachers) to undertake the course in the workplace during short breaks within clinical activities. An independent European steering committee provided input into the process.ResultsThe curriculum defined specific learning objectives for teaching EBM by exploiting educational opportunities in six different clinical settings. The e-modules incorporated video clips that demonstrate practical and effective methods of EBM teaching in everyday clinical practice. The course encouraged focussed teaching activities embedded within a trainers personal learning plan and documentation in a CPD portfolio for reflection.ConclusionThis curriculum will help senior clinicians to identify and make the best use of available opportunities in everyday practice in clinical situations to teach various steps of EBM and demonstrate their applicability to clinical practice. Once fully implemented, the ultimate outcome of this pilot project will be a European qualification in teaching EBM, which will be used by doctors, hospitals, professional bodies responsible for postgraduate qualifications and continuing medical education.

Do Guidelines for the Diagnosis and Monitoring of Diabetes Mellitus Fulfill the Criteria of Evidence-Based Guideline Development?

BACKGROUNDnAlthough the methodological quality of therapeutic guidelines (GLs) has been criticized, little is known regarding the quality of GLs that make diagnostic recommendations. Therefore, we assessed the methodological quality of GLs providing diagnostic recommendations for managing diabetes mellitus (DM) and explored several reasons for differences in quality across these GLs.nnnMETHODSnAfter systematic searches of published and electronic resources dated between 1999 and 2007, 26 DM GLs, published in English, were selected and scored for methodological quality using the AGREE Instrument. Subgroup analyses were performed based on the source, scope, length, origin, and date and type of publication of GLs. Using a checklist, we collected laboratory-specific items within GLs thought to be important for interpretation of test results.nnnRESULTSnThe 26 diagnostic GLs had significant shortcomings in methodological quality according to the AGREE criteria. GLs from agencies that had clear procedures for GL development, were longer than 50 pages, or were published in electronic databases were of higher quality. Diagnostic GLs contained more preanalytical or analytical information than combined (i.e., diagnostic and therapeutic) recommendations, but the overall quality was not significantly different. The quality of GLs did not show much improvement over the time period investigated.nnnCONCLUSIONSnThe methodological shortcomings of diagnostic GLs in DM raise questions regarding the validity of recommendations in these documents that may affect their implementation in practice. Our results suggest the need for standardization of GL terminology and for higher-quality, systematically developed recommendations based on explicit guideline development and reporting standards in laboratory medicine.

How far did we get? : How far to go? : A European survey on postgraduate courses in evidence-based medicine

BACKGROUNDnOver the past decade, evidence-based medicine (EBM) has gained recognition as a means to improve the quality of health care provision. However, little is known about learning opportunities to acquire EBM-associated skills. The EUebm-Unity partnership explored current educational activities for EBM practice for doctors across Europe.nnnMETHODSnWe surveyed organizations offering postgraduate EBM courses across Europe inquiring about their course programme, teaching content and strategies, and interest in a Europe-wide curriculum in EBM.nnnRESULTSnOne hundred and fifty-six organizers in eight European countries reported 403 courses that had started first-time from 1996 to 2006. Despite a steady increase, in absolute terms, the frequency of courses was low and varied from 1 first-time offering of a course per 640 doctors (Spain) to 1 first-time offering per 5600 doctors (Austria) over 10 years. Most adopted the McMaster EBM teaching concept of small group, problem-based learning focussing on interventions, diagnostic tests and guidelines, and included efforts to link EBM to patient care. Teaching staff consisted of doctors from academic and non-academic settings, supported by methodologists. Efforts to formally integrate EBM in postgraduate activities were only partially successful. Most organizations welcomed a standardized European qualification in EBM. A limitation of the survey is the lack of follow-up information about the continuation of courses following the first-time offering.nnnCONCLUSIONSnAll countries offer some EBM courses with varying teaching intensity. Learning opportunities are insufficient to ensure widespread dissemination of knowledge and skills. Most countries welcome more efforts to develop inexpensive and feasible educational activities at a postgraduate level.

Quality of Guidelines for the Laboratory Management of Diabetes Mellitus

BACKGROUND: There is increasing concern about the quality and reliability of practice guidelines, especially in the field of laboratory medicine, as most recommendations are developed by clinical specialty societies, often without involving laboratory professionals. Little information is available on the methodological quality of guidelines for the use of laboratory investigations in the care of specific diseases. We describe a pilot assessment of the most well‐known guidelines for the diagnosis and monitoring of diabetes mellitus (DM). METHODS: Practice guidelines on DM published in English between 1999 and 2005 April were identified by systematic searching in Medline and international guideline databases. Fifty four DM guidelines were retrieved, of which 29 met our inclusion criteria. The four most widely used international guidelines (WHO, ADA, NACB, NICE) were selected for a critical appraisal of their methodological quality. This was carried out by seven independent assessors using a validated checklist, the AGREE Instrument. Twenty three guideline attributes arranged in six independent domains were investigated and the mean scores of assessors for each attribute and the aggregated scores for each domain were calculated. Cronbachs alpha and interclass correlations were calculated to measure internal consistency and reliability within each domain. The four guidelines were compared using one‐way ANOVA and ANOVA using repeated measurements. RESULTS: The selected four guidelines on DM have significant shortcomings in demonstrating and/or reporting multidisciplinary stakeholder involvement in the guideline development process, evidence‐based methodology for formulating recommendations, applicability of statements, and disclosing any conflicts of interest or reporting editorial independence. CONCLUSIONS: Poor quality and lack of explicitness of recommendations in laboratory medicine call for methodological standards of guideline development and reporting, and for an international collaboration of guideline development activities, to increase the internal and external validity of recommendations in laboratory practice.

Quality assessment of clinical practice guidelines for adaptation in burn injury

OBJECTIVESnA number of clinical practice guidelines (CPGs) are available for managing burn injury patients but clinical practice is highly variable. We report the first steps to trans-contextual adaptation of international burn CPGs to local settings.nnnMETHODSnKey clinical topics and questions to be covered in the final guideline were defined and prioritized. Systematic search between 1990 and 2008 retrieved 546 citations, of which 24 were CPGs on the general and intensive care of burn patients. Assessment of the clinical content of CPGs was carried out. Methodological quality of CPGs was evaluated using the AGREE instrument.nnnRESULTSnOf the 24 CPGs evaluated, 10 (42%) were evidence-based. All major burn topics were covered by at least one CPG, but no single CPG addressed all areas important in terms of outcomes. According to the AGREE criteria, 2 CPGs (8%) were strongly recommended, 14 with provisos or alterations (58%) and the rest were not recommended for adaptation.nnnCONCLUSIONSnAlthough existing CPGs for the management of burn may accurately reflect agreed clinical practice, most performed poorly when evaluated for methodological quality. Future CPG efforts addressing these methodological shortcomings would add substantially to the improved management of burned patients.

How are "teaching the teachers" courses in evidence based medicine evaluated? A systematic review

BackgroundTeaching of evidence-based medicine (EBM) has become widespread in medical education. Teaching the teachers (TTT) courses address the increased teaching demand and the need to improve effectiveness of EBM teaching. We conducted a systematic review of assessment tools for EBM TTT courses. To summarise and appraise existing assessment methods for teaching the teachers courses in EBM by a systematic review.MethodsWe searched PubMed, BioMed, EmBase, Cochrane and Eric databases without language restrictions and included articles that assessed its participants. Study selection and data extraction were conducted independently by two reviewers.ResultsOf 1230 potentially relevant studies, five papers met the selection criteria. There were no specific assessment tools for evaluating effectiveness of EBM TTT courses. Some of the material available might be useful in initiating the development of such an assessment tool.ConclusionThere is a need for the development of educationally sound assessment tools for teaching the teachers courses in EBM, without which it would be impossible to ascertain if such courses have the desired effect.