Alison Dobbie

Web-based learning: sound educational method or hype? A review of the evaluation literature.

The first reports of Web-based medical education appear in 1992, building on 30 years of computer-assisted instruction. Proponents claimed computer-assisted instruction is superior to text-based, lecture, and traditional educational methods for reasons that include control by the learner over content, time, and place of learning; enhancement of learning, reasoning, and efficiency; and cost savings. Many studies that reported advantages contained methodologic flaws and reported advantages unrelated to computer-specific features. Web-based learning (WBL) represents a further evolution of computer-assisted instruction. Technical advantages of WBL include universal accessibility, ease in updating content, and hyperlink functions that permit cross-referencing to other resources. These technical advances, specifically hyperlink and searching capabilities, fit the constructivist learning theory, where learners search out and create their own knowledge bases. However, as was evident with computer-assisted instruction, potential advantages may not translate into significant improvements in educational outcomes. With the widespread adoption of WBL, it is critical that medical educators have evidence regarding its performance as a learning medium. This paper reviews the medical, dental, and nursing WBL evaluation literature to: (1) identify which facets of WBL have been evaluated, (2) describe the evaluation strategies used, (3) synthesize the findings, and (4) discuss educational implications and future research directions.

What evidence supports teaching evidence-based medicine?

Evidence-based teaching and learning are ‘‘hot topics’’ in medical education. Teaching critical thinking and appraisal skills to learners should give them a current knowledge base, a constantly questioning attitude, and the tools for lifelong learning. However, what is the evidence that teaching evidence-based medicine (EBM) actually changes learners’ behaviors and that such changes eventually translate into better patient care and outcomes? Ironically, few published studies have evaluated the teaching of evidencebased medicine, and two recent critical reviews of EBM curricula offer disappointing conclusions as to the effectiveness of such programs. Norman and Shannon found evidence that teaching critical appraisal skills can increase students’, but not residents’, knowledge of epidemiology. Green reviewed published reports on 18 teaching programs and concluded that most of the studies had poor teaching methods and inadequate evaluation methods. He concluded that, ‘‘in those studies that were methodologically rigorous, the curricula’s effectiveness in improving knowledge and skills was modest.’’ There are thus few good tools for measuring shortterm outcomes of evidence-based teaching and learning (for example, how well learners acquire the basic knowledge and skills of EBM), and fewer yet to measure whether learners’ behaviors change or are maintained over time. It is even more difficult to determine whether teaching EBM techniques benefits patients in terms of reduced morbidity and mortality. In fact, some authors have postulated that using EBM can adversely affect patient care, by devaluing the ‘‘non-evidentiary aspects of medical practice,’’ such as clinical judgment and expert opinion. We conducted a small-group discussion session entitled ‘‘How Can We Best Evaluate The Teaching of Evidence-based Medicine?’’ at the 1999 annual meeting of the Association of American Medical Colleges. Approximately 40 MD and non-MD faculty at all levels of seniority participated, representing many different specialties and medical schools across the country. Most were currently involved in the development and administration of EBM teaching programs for students and residents at their home institutions. The discussion centered on the following four questions: (1) What is the evidence that teaching evidence-based learning techniques changes learners’ behaviors? (2) How can we collaborate to produce this evidence? (3) What new tools and strategies can we invent as a group? (4) Is there any valid and reliable way to measure the effect of EBM teaching and learning on patient outcomes? This paper presents a summary of the group’s discussion, with suggestions for future work needed to evaluate the outcomes of EBM teaching programs.

A validated cultural competence curriculum for US pediatric clerkships

OBJECTIVE A 2006 national survey of pediatric clerkship directors revealed that only 25% taught cultural competence, but 81% expressed interest in a validated cultural competence curriculum. The authors designed and evaluated a multi-modality cultural competence curriculum for pediatric clerkships including a validated cultural knowledge test. METHODS Curriculum content included two interactive workshops, multimedia web cases, and a Cultural and Linguistic Competence Pocket Guide. Evaluation included a student satisfaction survey, a Nominal Technique Focus Group, and a validated knowledge test. The knowledge test comprised 6 case studies with 49 multiple choice items covering the curricular content. RESULTS Of 149/160 (93%) students who completed satisfaction surveys using a 5-point Likert scale, >82% strongly agreed or agreed that the curricular intervention was a meaningful experience (93%), increased their understanding of the culture of medicine (91%), increased their knowledge of racial and ethnic disparities (89%) and core cultural issues (91%), and improved their skills in working with interpreters (90%) and cross-cultural communication (82%). Top strengths identified by a focus group (34 students) included learning about interpreters, examples of cultural practices, and raised cultural awareness. Pre- and post-knowledge test scores improved by 17% (p<.0001). After six administrations, the test achieved the target reliability of .7. CONCLUSIONS The authors successfully designed and validated a practical cultural competence curriculum for pediatric clerkships that meets the need demonstrated in the 2006 national survey. PRACTICE IMPLICATIONS This curriculum will enable pediatric clerkship directors to equip more graduates to provide culturally sensitive pediatric care to an increasingly diverse US population.