B. Price Kerfoot

Online spaced education generates transfer and improves long-term retention of diagnostic skills: a randomized controlled trial.

BACKGROUND Retention of learning from surgical training is often limited, especially if the knowledge and skills are used infrequently. Using histopathology diagnostic skills as an experimental system, we compared knowledge transfer and retention between bolus Web-based teaching (WBT) modules and online spaced education, a novel email-based method of online education founded on the spacing effect. STUDY DESIGN All US urology residents were eligible to participate. Enrollees were randomized to 1 of 2 cohorts. Cohort 1 residents received 3 cycles/repetitions of spaced education on prostate-testis histopathology (weeks 1 to 16) and 3 WBT modules on bladder-kidney (weeks 14 to 16). Cohort 2 residents received 3 cycles of spaced education on bladder-kidney (weeks 1 to 16) and 3 WBT modules on prostate-testis (weeks 14 to 16). Each daily spaced education email presented a clinical scenario with histopathology image and asked for a diagnosis. Participants received immediate feedback after submitting their answers. Each cycle/repetition was 4 weeks long and consisted of 20 questions with unique images. WBT used the identical content and delivery system, with questions aggregated into three 20-question modules. Long-term retention of all 4 topics was assessed during weeks 18 to 45. RESULTS Seven-hundred and twenty-four urology residents enrolled. Spaced education and WBT were completed by 77% and 66% of residents, respectively. Spaced education and WBT generated mean long-term score increases of 15.2% (SD 15.3%) and 3.4% (SD 16.3%), respectively (p < 0.01). Spaced education increased long-term learning efficiency 4-fold. CONCLUSIONS Online spaced education generates transfer of histopathology diagnostic skills and substantially improves their long-term retention. Additional research is needed to determine how spaced education can optimize learning, transfer, and retention of surgical skills.

A Multi-institutional Randomized Controlled Trial of Adjuvant Web-based Teaching to Medical Students

Purpose To investigate the impact of an adjuvant Web-based teaching program on medical students’ learning during clinical rotations. Method From April 2003 to May 2004, 351 students completing clinical rotations in surgery–urology at four U.S. medical schools were invited to volunteer for the study. Web-based teaching cases were developed covering four core urologic topics. Students were block randomized to receive Web-based teaching on two of the four topics. Before and after a designated duration at each institution (ranging one to three weeks), students completed a validated 28-item Web-based test (Cronbach’s alpha = .76) covering all four topics. The test was also administered to a subset of students at one school at the conclusion of their third-year to measure long-term learning. Results Eighty-one percent of all eligible students (286/351) volunteered to participate in the study, 73% of whom (210/286) completed the Web-based program. Compared to controls, Web-based teaching significantly increased test scores in the four topics at each medical school (p < .001, mixed analysis of variance), corresponding to a Cohen’s d effect size of 1.52 (95% confidence interval [CI], 1.23–1.80). Learning efficiency was increased three-fold by Web-based teaching (Cohen’s d effect size 1.16; 95% CI 1.13–1.19). Students who were tested a median of 4.8 months later demonstrated significantly higher scores for Web-based teaching compared to non-Web-based teaching (p = .007, paired t-test). Limited learning was noted in the absence of Web-based teaching. Conclusions This randomized controlled trial provides Class I evidence that Web-based teaching as an adjunct to clinical experiences can significantly and durably improve medical students’ learning.

Interactive spaced education to assess and improve knowledge of clinical practice guidelines: a randomized controlled trial.

Objective:To determine whether Interactive Spaced Education (ISE) is an effective and acceptable form of graduate and continuing medical education (GME/CME), using clinical practice guideline (CPG) education as an experimental system. Summary Background Data:ISE is a novel form of online education, which combines the pedagogical merits of the spacing and testing effects. Its efficacy for GME and CME is not known. Methods:One-hundred sixty urologists and 320 urology residents were randomized to 1 of 2 cohorts. We developed and validated 48 ISE items (questions and answers) on 5 urology CPGs (hematuria and priapism [HP]; staghorn calculi, infertility, and antibiotic use [SIA]). Physicians were sent 3 emails a week, each containing 2 questions. Content was repeated 3 times over 20 weeks. Cohort A physicians received the 3-cycle ISE course on HP, with 24 control items on SIA in cycle 3. Cohort B physicians received the 3-cycle ISE course on SIA, with 24 control items on HP in cycle 3. Results:The ISE program was completed by 71% urologists and 83% residents. Cohort A scores on HP increased from mean 44.9% in cycle 1% to 75.7% in cycle 3, a 57% relative increase compared with controls (P < 0.001; Cohen effect size, 2.2). Similarly, cohort B scores on SIA increased from 45.2% in cycle 1% to 69.5% in cycle 3, a 56% relative increase compared with controls (P < 0.001; effect size, 2.2). Eighty-four percent of all participants requested to enroll in further ISE programs. Conclusions:ISE is an effective and well-accepted form of GME and CME and is a promising new methodology to improve CPG knowledge.

Learning Benefits of On-Line Spaced Education Persist for 2 Years

PURPOSE A total of 537 urology residents participated in a randomized trial of on-line spaced education in 2005 that used 96 American Urological Association Self-Assessment Study Program questions as educational material. I investigated whether the learning gains generated by the spaced education program could be detected 2 years later. MATERIALS AND METHODS A test instrument was constructed with 60 of the 96 Self-Assessment Study Program questions from the 2005 trial. These multiple choice questions were delivered to residents from September to November 2007 via daily interactive e-mails. Residents submitted answers on line and were included in analysis if they completed 85% or greater of the questions. RESULTS Of the 537 residents in the 2005 trial 206 (38%) were still in residency and volunteered to complete the test. Of those residents 104 (50%) were randomized to the spaced education cohort, 102 (50%) were randomized to the bolus cohort and 147 (71%) submitted answers to 85% or greater of the test questions. There were no significant differences in age, gender, degree or training year between the cohorts. Residents in the spaced education cohort had significantly greater test scores than residents in the bolus cohort (mean +/- SD 70.2% +/- 9.0% vs 66.8% +/- 10.6%, effect size 0.35, p = 0.03). CONCLUSIONS On-line spaced education can generate improvements in learning that are retained 2 years later. Although the effect size is modest, the persistence of detectable knowledge differences between educational interventions after such a long duration is exceedingly unusual. Further research is needed to determine how spaced education can best be used to optimize lifelong learning for urologists.

Patient Safety Knowledge and Its Determinants in Medical Trainees

BackgroundPatient safety is a core educational topic for medical trainees.ObjectivesTo determine the current level and determinants of patient safety knowledge in medical trainees.DesignMulti-institutional cross-sectional assessment of patient safety knowledge.ParticipantsResidents and medical students from seven Harvard-affiliated residencies and two Harvard Medical School courses.MeasurementsParticipants were administered a 14-item validated test instrument developed based on the patient safety curriculum of the Risk Management Foundation (Cambridge, MA). The primary outcome measure was the amount of patient safety knowledge demonstrated by trainees on the validated test instrument. The secondary outcome measure was their subjective perceptions as to their baseline knowledge level in this domain.ResultsNinety-two percent (640/693) of residents and medical students completed the patient safety test. Participants correctly answered a mean 58.4% of test items (SD 15.5%). Univariate analyses show that patient safety knowledge levels varied significantly by year of training (p = 0.001), degree program (p < 0.001), specialty (p < 0.001), country of medical school (p = 0.006), age (p < 0.001), and gender (p = 0.050); all but the latter two determinants remained statistically significant in multivariate models. In addition, trainees were unable to assess their own knowledge deficiencies in this domain.ConclusionsPatient safety knowledge is limited among medical trainees across a broad range of training levels, degrees, and specialties. Effective educational interventions that target deficiencies in patient safety knowledge are greatly needed.

Influence of new educational technology on problem‐based learning at Harvard Medical School

Purpose  Computers with 50‐inch, wall‐mounted plasma screens and broadband Internet access were installed in all small group tutorial rooms at Harvard Medical School. This study examines how the introduction of this educational technology impacted on the problem‐based learning tutorials.

Adaptive Spaced Education Improves Learning Efficiency: A Randomized Controlled Trial

PURPOSE Spaced education is a novel form of online education that harnesses the 2 psychology research findings of spacing and testing effects. Spaced education is delivered by daily emails containing clinically relevant multiple choice questions. To take advantage of the spacing effect the questions are repeated at fixed intervals for a fixed number of repetitions. An adaptive spaced education system was developed to customize spacing intervals and the number of repetitions based on learner knowledge level. To determine whether this system improves learning efficiency I performed a randomized trial to compare the learning efficiency of adaptive vs nonadaptive spaced education systems among surgery students at 2 medical schools. MATERIALS AND METHODS A total of 62 year 3 students were randomized to identical course content in adaptive or nonadaptive spaced education formats. The course consisted of 40 validated, spaced education items on the 4 urology topics benign prostatic hyperplasia, erectile dysfunction, prostate cancer and prostate specific antigen screening. The nonadaptive cohort received daily emails containing 2 questions with a linear review of the material 20 days after initial presentation. The adaptive cohort received daily emails via an adaptive algorithm that limited the repetition of mastered content. Each cohort completed a validated end of course test. RESULTS The adaptive cohort answered significantly fewer spaced education items than the nonadaptive cohort (p = 0.001) but achieved comparable end of course test scores (p = 0.37). The adaptive algorithm increased learning efficiency by 38%. CONCLUSIONS Adaptive spaced education boosts learning efficiency.

An Online Spaced-Education Game to Teach and Assess Medical Students: A Multi- Institutional Prospective Trial

Purpose To investigate whether a spaced-education (SE) game can be an effective means of teaching core content to medical students and a reliable and valid method of assessing their knowledge. Method This nine-month trial (2008–2009) enrolled students from three U.S. medical schools. The SE game consisted of 100 validated multiple-choice questions–explanations in preclinical/clinical domains. Students were e-mailed two questions daily. Adaptive game mechanics re-sent questions in three or six weeks if answered, respectively, incorrectly or correctly. Questions expired if not answered on time (appointment dynamic). Students retired questions by answering each correctly twice consecutively (progression dynamic). Posting of relative performance fostered competition. Main outcome measures were baseline and completion scores. Results Seven-hundred thirty-one students enrolled. Median baseline score was 53% (interquartile range [IQR] 16) and varied significantly by year (P < .001, dmax = 2.08), school (P < .001, dmax = 0.75), and gender (P < .001, d = 0.38). Median completion score was 93% (IQR 12) and varied significantly by year (P = .001, dmax = 1.12), school (P < .001, dmax = 0.34), and age (P = .019, dmax = 0.43). Scores did not differ significantly between years 3 and 4. Seventy percent of enrollees (513/731) requested to participate in future SE games. Conclusions An SE game is an effective and well-accepted means of teaching core content and a reliable and valid method to assess student knowledge. SE games may be valuable tools to identify and remediate students who could benefit from additional educational support.

Interactive Spaced Education Versus Web Based Modules for Teaching Urology to Medical Students: A Randomized Controlled Trial

PURPOSE This study is an assessment of the acceptability and short-term educational efficacy of interactive spaced education compared to web based teaching cases within the compact time frame of a clinical clerkship. MATERIALS AND METHODS All 237 third-year students completing their 3-month surgery clerkships at 2 medical schools were asked to complete a urology online-education program covering 4 core topics of benign prostatic hyperplasia, erectile dysfunction, prostate cancer and screening with prostate specific antigen. Students were stratified by clinical site and randomized to 1 of 2 cohorts. Students in cohort A received interactive spaced education on prostate cancer/prostate specific antigen and web based teaching on benign prostatic hyperplasia/erectile dysfunction. Students in cohort B received interactive spaced education on benign prostatic hyperplasia/erectile dysfunction and web based teaching on prostate cancer/prostate specific antigen. A validated 28-item test on all 4 topics was administered at the end of the 10-week program. RESULTS No statistically significant differences in end-of-program test scores were observed between cohorts in the topics of prostate cancer/prostate specific antigen with 87.6% (SD 12.9) for cohort A (interactive spaced education) and 82.4% (SD 19.6) for cohort B (web based teaching) (p = 0.25). Similarly there was also no statistically significant difference in test scores in the topics of benign prostatic hyperplasia/erectile dysfunction with 79.5% (SD 15.9) for cohort A (web based teaching) and 82.1% (SD 14.7) for cohort B (interactive spaced education, p = 0.28). When students were asked which format they would prefer if they were to receive all their urology online education in a single format, 55% of students (109 of 198 respondents) preferred interactive spaced education while 45% (89 of 109) preferred web based teaching (p = 0.16). CONCLUSIONS Within the compact time frame of a clinical clerkship interactive spaced education is equivalent to web based teaching in short-term learning gains and in acceptability by medical students.

A Faculty Development Program to Train Tutors to Be Discussion Leaders Rather Than Facilitators

Purpose During 2003, 2004, and 2005, the role of 70 tutors was changed from that of facilitator to discussion leader, in a preclinical PBL learning course, Gastrointestinal Pathophysiology, by use of three key business school teaching strategies: questions, summaries, and schematics. The purpose of this study was to learn what difference this new approach made. Method During each of the three study years, 171 (2003), 167 (2004), and 170 (2005) students were given Likert-scale attitudinal questionnaires to rate whether their tutors encouraged student direction of the tutorials and whether the summaries and closure schematics benefited their learning. Students’ overall course evaluations and mean USMLE scores were quantitatively analyzed, pre- and postintervention. A variety of statistical tests were used to assess the statistical significance of means at the confidence level of .05. Results In the third year of the program, student ratings indicated that their tutors were significantly better at encouraging student direction of the tutorials than in the first year (P < .05). The students reported that the tutorial made a more important contribution to their learning (P < .05), and the course objectives were better stated (P = .038) and better met (P = .007). Overall satisfaction with the course also improved significantly (P = .006). Part I gastrointestinal system mean scores of the USMLE showed a statistically significant increase in 2005 compared with 2001 or 2002. Conclusions The tutor as a discussion leader who questions, summarizes, and uses schematics to illustrate concepts had a significant and positive impact on learning in tutorials, achieving course objectives, improving overall course satisfaction, and increasing a standardized national exam’s mean score.