Denise M. Dupras

Internet-based learning in the health professions:a meta-analysis

CONTEXT The increasing use of Internet-based learning in health professions education may be informed by a timely, comprehensive synthesis of evidence of effectiveness. OBJECTIVES To summarize the effect of Internet-based instruction for health professions learners compared with no intervention and with non-Internet interventions. DATA SOURCES Systematic search of MEDLINE, Scopus, CINAHL, EMBASE, ERIC, TimeLit, Web of Science, Dissertation Abstracts, and the University of Toronto Research and Development Resource Base from 1990 through 2007. STUDY SELECTION Studies in any language quantifying the association of Internet-based instruction and educational outcomes for practicing and student physicians, nurses, pharmacists, dentists, and other health care professionals compared with a no-intervention or non-Internet control group or a preintervention assessment. DATA EXTRACTION Two reviewers independently evaluated study quality and abstracted information including characteristics of learners, learning setting, and intervention (including level of interactivity, practice exercises, online discussion, and duration). DATA SYNTHESIS There were 201 eligible studies. Heterogeneity in results across studies was large (I(2) > or = 79%) in all analyses. Effect sizes were pooled using a random effects model. The pooled effect size in comparison to no intervention favored Internet-based interventions and was 1.00 (95% confidence interval [CI], 0.90-1.10; P < .001; n = 126 studies) for knowledge outcomes, 0.85 (95% CI, 0.49-1.20; P < .001; n = 16) for skills, and 0.82 (95% CI, 0.63-1.02; P < .001; n = 32) for learner behaviors and patient effects. Compared with non-Internet formats, the pooled effect sizes (positive numbers favoring Internet) were 0.10 (95% CI, -0.12 to 0.32; P = .37; n = 43) for satisfaction, 0.12 (95% CI, 0.003 to 0.24; P = .045; n = 63) for knowledge, 0.09 (95% CI, -0.26 to 0.44; P = .61; n = 12) for skills, and 0.51 (95% CI, -0.24 to 1.25; P = .18; n = 6) for behaviors or patient effects. No important treatment-subgroup interactions were identified. CONCLUSIONS Internet-based learning is associated with large positive effects compared with no intervention. In contrast, effects compared with non-Internet instructional methods are heterogeneous and generally small, suggesting effectiveness similar to traditional methods. Future research should directly compare different Internet-based interventions.

Instructional design variations in internet-based learning for health professions education: A systematic review and meta-analysis

Purpose A recent systematic review (2008) described the effectiveness of Internet-based learning (IBL) in health professions education. A comprehensive synthesis of research investigating how to improve IBL is needed. This systematic review sought to provide such a synthesis. Method The authors searched MEDLINE, CINAHL, EMBASE, Web of Science, Scopus, ERIC, TimeLit, and the University of Toronto Research and Development Resource Base for articles published from 1990 through November 2008. They included all studies quantifying the effect of IBL compared with another Internet-based or computer-assisted instructional intervention on practicing and student physicians, nurses, pharmacists, dentists, and other health professionals. Reviewers working independently and in duplicate abstracted information, coded study quality, and grouped studies according to inductively identified themes. Results From 2,705 articles, the authors identified 51 eligible studies, including 30 randomized trials. The pooled effect size (ES) for learning outcomes in 15 studies investigating high versus low interactivity was 0.27 (95% confidence interval, 0.08–0.46; P = .006). Also associated with higher learning were practice exercises (ES 0.40 [0.08–0.71; P = .01]; 10 studies), feedback (ES 0.68 [0.01–1.35; P = .047]; 2 studies), and repetition of study material (ES 0.19 [0.09–0.30; P < .001]; 2 studies). The ES was 0.26 (−0.62 to 1.13; P = .57) for three studies examining online discussion. Inconsistency was large (I2 ≥89%) in most analyses. Meta-analyses for other themes generally yielded imprecise results. Conclusions Interactivity, practice exercises, repetition, and feedback seem to be associated with improved learning outcomes, although inconsistency across studies tempers conclusions. Evidence for other instructional variations remains inconclusive.

A Practical Guide To Developing Effective Web‐based Learning

OBJECTIVE: Online learning has changed medical education, but many “educational” websites do not employ principles of effective learning. This article will assist readers in developing effective educational websites by integrating principles of active learning with the unique features of the Web.DESIGN: Narrative review.RESULTS: The key steps in developing an effective educational website are: Perform a needs analysis and specify goals and objectives; determine technical resources and needs; evaluate preexisting software and use it if it fully meets your needs; secure commitment from all participants and identify and address potential barriers to implementation; develop content in close coordination with website design (appropriately use multimedia, hyperlinks, and online communication) and follow a timeline; encourage active learning (self-assessment, reflection, self-directed learning, problem-based learning, learner interaction, and feedback); facilitate and plan to encourage use by the learner (make website accessible and user-friendly, provide time for learning, and motivate learners); evaluate learners and course; pilot the website before full implementation; and plan to monitor online communication and maintain the site by resolving technical problems, periodically verifying hyperlinks, and regularly updating content.CONCLUSION: Teaching on the Web involves more than putting together a colorful webpage. By consistently employing principles of effective learning, educators will unlock the full potential of Web-based medical education.

What do we mean by web‐based learning? A systematic review of the variability of interventions

Medical Education 2010: 44: 765–774

Web-based learning in residents' continuity clinics: a randomized, controlled trial.

Purpose To determine whether internal medicine residents prefer learning from Web-based (WB) modules or printed material, and to compare the effect of these teaching formats on knowledge. Method The authors conducted a randomized, controlled, crossover study in the internal medicine resident continuity clinics of the Mayo School of Graduate Medical Education during the 2002–03 academic year. Participants studied two topics of ambulatory medicine using WB modules and two topics using paper practice guidelines in randomly assigned sequences. Primary outcomes were format preference (assessed by an end-of-course questionnaire) and score changes from pre- to postintervention tests of knowledge. Results A total of 109 consented and 75 (69%) completed the postintervention test. Fifty-seven of 73 (78% “95% CI, 67–86%”) preferred the WB format (p < .001). Test scores improved for both formats (67.7% to 75.0% for WB, 66.0% to 73.3% for paper), but score change was not different between formats both before (p = .718) and after (p = .080) adjusting for topic, clinic site, study group, postgraduate year, and gender. Residents spent less time on WB modules (mean = 47 ± 26 minutes) than paper (mean = 59 ± 35, p = .024). Difficulties with passwords limited their use of WB modules for 71% (59–80%) of residents. Conclusion No difference was found between WB and paper-based formats in knowledge-test score change, but residents preferred learning with WB modules and spent less time doing so. Passwords appeared to impede use of WB modules. WB learning is effective, well accepted, and efficient. Research should focus on aspects of WB instruction that will enhance its power as a learning tool and better define its role in specific settings.

Effect of Rater Training on Reliability and Accuracy of Mini-CEX Scores: A Randomized, Controlled Trial

BackgroundMini-CEX scores assess resident competence. Rater training might improve mini-CEX score interrater reliability, but evidence is lacking.ObjectiveEvaluate a rater training workshop using interrater reliability and accuracy.DesignRandomized trial (immediate versus delayed workshop) and single-group pre/post study (randomized groups combined).SettingAcademic medical center.ParticipantsFifty-two internal medicine clinic preceptors (31 randomized and 21 additional workshop attendees).InterventionThe workshop included rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion. Delayed group received no intervention until after posttest.MeasurementsMini-CEX ratings at baseline (just before workshop for workshop group), and four weeks later using videotaped resident–patient encounters; mini-CEX ratings of live resident–patient encounters one year preceding and one year following the workshop; rater confidence using mini-CEX.ResultsAmong 31 randomized participants, interrater reliabilities in the delayed group (baseline intraclass correlation coefficient [ICC] 0.43, follow-up 0.53) and workshop group (baseline 0.40, follow-up 0.43) were not significantly different (p = 0.19). Mean ratings were similar at baseline (delayed 4.9 [95% confidence interval 4.6–5.2], workshop 4.8 [4.5–5.1]) and follow-up (delayed 5.4 [5.0–5.7], workshop 5.3 [5.0–5.6]; p = 0.88 for interaction). For the entire cohort, rater confidence (1 = not confident, 6 = very confident) improved from mean (SD) 3.8 (1.4) to 4.4 (1.0), p = 0.018. Interrater reliability for ratings of live encounters (entire cohort) was higher after the workshop (ICC 0.34) than before (ICC 0.18) but the standard error of measurement was similar for both periods.ConclusionsRater training did not improve interrater reliability or accuracy of mini-CEX scores.Clinical trials registrationclinicaltrials.gov identifier NCT00667940

Warfarin Sensitivity Genotyping: A Review of the Literature and Summary of Patient Experience

The antithrombotic benefits of warfarin are countered by a narrow therapeutic index that contributes to excessive bleeding or cerebrovascular clotting and stroke in some patients. This article reviews the current literature describing warfarin sensitivity genotyping and compares the results of that review to the findings of our study in 189 patients at Mayo Clinic conducted between June 2001 and April 2003. For the review of the literature, we identified relevant peer-reviewed articles by searching the Web of Knowledge using key word warfarin-related adverse event. For the 189 Mayo Clinic patients initiating warfarin therapy to achieve a target international normalized ratio (INR) in the range of 2.0 to 3.5, we analyzed the CYP2C9 (cytochrome P450 2C9) and VKORC1 (vitamin K epoxide reductase complex, subunit 1) genetic loci to study the relationship among the initial warfarin dose, steady-state dose, time to achieve steady-state dose, variations in INR, and allelic variance. Results were compared with those previously reported in the literature for 637 patients. The relationships between allelic variants and warfarin sensitivity found in our study of Mayo Clinic patients are fundamentally the same as in those reported by others. The Mayo Clinic population is predominantly white and shows considerable allelic variability in CYP2C9 and VKORC1. Certain of these alleles are associated with increased sensitivity to warfarin. Polymorphisms in CYP2C9 and VKORC1 have a considerable effect on warfarin dose in white people. A correlation between steady-state warfarin dose and allelic variants of CYP2C9 and VKORC1 has been demonstrated by many previous reports and is reconfirmed in this report. The allelic variants found to most affect warfarin sensitivity are CYP2C9*1*1-VKORC1BB (less warfarin sensitivity than typical); CYP2C9*1*1-VKORC1AA (considerable variance in INR throughout initiation); CYP2C9*1*2-VKORC1AB (more sensitivity to warfarin than typical); CYP2C9*1*3-VKORC1AB (much more sensitivity to warfarin than typical); CYP2C9*1*2-VKORC1AB (much more sensitivity to warfarin than typical); CYP2C9*1*3-VKORC1AA (much more sensitivity to warfarin than typical); and CYP2C9*2*2-VKORC1AB (much more sensitivity to warfarin than typical). Although we were unable to show an association between allelic variants and initial warfarin dose or dose escalation, an association was seen between allelic variant and steady-state warfarin dose. White people show considerable variance in CYP2C9 allele types, whereas people of Asian or African descent infrequently carry CYP2C9 allelic variants. The VKORC1AA allele associated with high warfarin sensitivity predominates in those of Asian descent, whereas white people and those of African descent show diversity, carrying either the VKORC1BB, an allele associated with low warfarin sensitivity, or VKORC1AB or VKORC1AA, alleles associated with moderate and high warfarin sensitivity, respectively.

General Medicine vs Subspecialty Career Plans Among Internal Medicine Residents

CONTEXT Current medical training models in the United States are unlikely to produce sufficient numbers of general internists and primary care physicians. Differences in general internal medicine (GIM) career plans between internal medicine residency program types and across resident demographics are not well understood. OBJECTIVE To evaluate the general medicine career plans of internal medicine residents and how career plans evolve during training. DESIGN, SETTING, AND PARTICIPANTS A study of US internal medicine residents using an annual survey linked to the Internal Medicine In-Training Examination taken in October of 2009-2011 to evaluate career plans by training program, sex, and medical school location. Of 67,207 US eligible categorical and primary care internal medicine residents, 57,087 (84.9%) completed and returned the survey. Demographic data provided by the National Board of Medical Examiners were available for 52,035 (77.4%) of these residents, of whom 51,390 (76.5%) responded to all survey items and an additional 645 (1.0%) responded to at least 1 survey item. Data were analyzed from the 16,781 third-year residents (32.2%) in this sample. MAIN OUTCOME MEASURES Self-reported ultimate career plans of internal medicine residents. RESULTS A GIM career plan was reported by 3605 graduating residents (21.5%). A total of 562 primary care program (39.6%) and 3043 categorical (19.9%) residents reported GIM as their ultimate career plan (adjusted odds ratio [AOR], 2.76; 99% CI, 2.35-3.23; P < .001). Conversely, 10 008 categorical (65.3%) and 745 primary care program (52.5%) residents reported a subspecialty career plan (AOR, 1.90; 99% CI, 1.62-2.23; P < .001). GIM career plans were reported more frequently by women than men (26.7% vs 17.3%, respectively; AOR, 1.69; 99% CI, 1.53-1.87; P < .001). US medical graduates were slightly more likely to report GIM career plans than international medical graduates (22.0% vs 21.1%, respectively; AOR, 1.76; 99% CI, 1.50-2.06; P < .001). Within primary care programs, US medical graduates were much more likely to report GIM career plans than international medical graduates (57.3% vs 27.3%, respectively; AOR, 3.48; 99% CI, 2.58-4.70; P < .001). Compared with their counterparts, maintaining a first-year GIM career plan over the course of their training was more likely among primary care program residents (68.2% vs 52.3%; AOR, 1.81; 99% CI, 1.25-2.64; P < .001), women (62.4% vs 47.2%; AOR, 1.75; 99% CI, 1.34-2.29; P < .001), and US medical graduates (60.9% vs 49.2%; AOR, 1.48; 99% CI, 1.13-1.93; P < .001). CONCLUSION Reported GIM career plans were markedly less common than subspecialty career plans among internal medicine residents, including those in primary care training programs, and differed according to resident sex, medical school location, and program type.

Searching the Medical Literature Using PubMed: A Tutorial

Staying current with advances in medicine is becoming a major challenge for clinicians. Access to updated repositories of medical information and the expertise to locate relevant information within them are becoming necessary clinical skills. PubMed (http://www.pubmedgov) provides free access to the largest biomedical resource available and is updated daily. Clinicians can use this resource to find answers to focused clinical questions quickly and efficiently. The purpose of this article is to assist clinicians in the development of the basic skills required to use PubMed to make informed clinical decisions.

Problem residents: prevalence, problems and remediation in the era of core competencies.

a S A g The American Board of Internal Medicine (ABIM) has defined the “problem resident” as a learner who demonstrates problem behaviors significant enough to require intervention by program leadership, typically the residency program director or chief resident. It has been over a decade since Yao and Wright’s report on the prevalence of “problem residents” in internal medicine residency training programs. Their survey of program directors reported performance problems in 6.9% of residents. We are unaware of any subsequent large studies in internal medicine of this important topic. Although the term “problem resident” has been used frequently, we will refer to these individuals as “residents in difficulty.” The purpose of this study was to assess internal medicine program director experiences with residents in difficulty in the era of Accreditation Council for Graduate Medical Education (ACGME) competencies.