Douglas P. Larsen

Test-enhanced learning in medical education

Context  In education, tests are primarily used for assessment, thus permitting teachers to assess the efficacy of their curriculum and to assign grades. However, research in cognitive psychology has shown that tests can also directly affect learning by promoting better retention of information, a phenomenon known as the testing effect.

Repeated testing improves long-term retention relative to repeated study: a randomised controlled trial.

Context  Laboratory studies in cognitive psychology with relatively brief final recall intervals suggest that repeated retrieval in the form of tests may result in better retention of information compared with repeated study.

Comparative effects of test‐enhanced learning and self‐explanation on long‐term retention

Educators often encourage students to engage in active learning by generating explanations for the material being learned, a method called self‐explanation. Studies have also demonstrated that repeated testing improves retention. However, no studies have directly compared the two learning methods.

The effects of test-enhanced learning on long-term retention in AAN annual meeting courses

Objective: We measured the long-term retention of knowledge gained through selected American Academy of Neurology annual meeting courses and compared the effects of repeated quizzing (known as test-enhanced learning) and repeated studying on that retention. Methods: Participants were recruited from 4 annual meeting courses. All participants took a pretest. This randomized, controlled trial utilized a within-subjects design in which each participant experienced 3 different postcourse activities with each activity performed on different material. Each key information point from the course was randomized in a counterbalanced fashion among participants to one of the 3 activities: repeated short-answer quizzing, repeated studying, and no further exposure to the materials. A final test covering all information points from the course was taken 5.5 months after the course. Results: Thirty-five participants across the 4 courses completed the study. Average score on the pretest was 36%. Performance on the final test showed that repeated quizzing led to significantly greater long-term retention relative to both repeated studying (55% vs 46%; t[34] = 3.28, SEM = 0.03, p = 0.01, d = 0.49) and no further exposure (55% vs 44%; t[34] = 3.16, SEM = 0.03, p = 0.01, d = 0.58). Relative to the pretest baseline, repeated quizzing helped participants to retain almost twice as much of the knowledge acquired from the course compared to repeated studying or no further exposure. Conclusions: Whereas annual meeting continuing medical education (CME) courses lead to long-term gains in knowledge, when repeated quizzing is added, retention is significantly increased. CME planners may consider adding repeated quizzing to increase the impact of their courses.

Effects of Team-Based Learning on short-term and long-term retention of factual knowledge.

Abstract Purpose: We investigated the effect of Team-Based Learning (TBL) on long-term retention of knowledge in comparison to a traditional curriculum. Methods: As TBL was incorporated into our curriculum in the 2008–2009 academic year, students were compared with those who received the traditional curriculum the year prior. Students in both the groups completed multiple-choice knowledge test at four time points spanning two years. Test performance was compared at each time point to assess changes in knowledge retention as a function of time. Results: Baseline knowledge did not differ significantly between the TBL and control groups [51% versus 46%; t(84) = 0.91, p = 0.37, d = 0.20]. Performance improved after the course for both the groups, but was significantly higher in the TBL group [79% versus 59%; t(84) = 4.96, p = 0.000004, d = 0.95]. However, when assessed prior to the pediatrics clerkship, learning gains from TBL had largely disappeared and the small difference in performance was not significant [57% versus 51%; t(84) = 1.51, p = 0.14, d = 0.32]. Conclusion: Incorporating TBL into the pre-clinical pediatrics curriculum led to large gains in knowledge over the short-term, but these gains did not persist. Further research should focus on extending the impact of TBL on long-term knowledge retention.

Tying knots: an activity theory analysis of student learning goals in clinical education

Learning goal programmes are often created to help students develop self‐regulated learning skills; however, these programmes do not necessarily consider the social contexts surrounding learning goals or how they fit into daily educational practice.

Student-directed retrieval practice is a predictor of medical licensing examination performance.

IntroductionA large body of evidence indicates that retrieval practice (test-enhanced learning) and spaced repetition increase long-term information retention. Implementation of these strategies in medical curricula is unfortunately limited. However, students may choose to apply them autonomously when preparing for high-stakes, cumulative assessments, such as the United States Medical Licensing Examination Step 1. We examined the prevalence of specific self-directed methods of testing, with or without spaced repetition, among preclinical students and assessed the relationship between these methods and licensing examination performance.MethodsSeventy-two medical students at one institution completed a survey concerning their use of user-generated (Anki) or commercially-available (Firecracker) flashcards intended for spaced repetition and of boards-style multiple-choice questions (MCQs). Other information collected included Step 1 score, past academic performance (Medical College Admission Test [MCAT] score, preclinical grades), and psychological factors that may have affected exam preparation or performance (feelings of depression, burnout, and test anxiety).ResultsAll students reported using practice MCQs (mean 3870, SD 1472). Anki and Firecracker users comprised 31 and 49 % of respondents, respectively. In a multivariate regression model, significant independent predictors of Step 1 score included MCQs completed (unstandardized beta coefficient [B] = 2.2 × 10− 3, p < 0.001), unique Anki flashcards seen (B = 5.9 × 10− 4, p = 0.024), second-year honours (B = 1.198, p = 0.002), and MCAT score (B = 1.078, p = 0.003). Test anxiety was a significant negative predictor (B= − 1.986, p < 0.001). Unique Firecracker flashcards seen did not predict Step 1 score. Each additional 445 boards-style practice questions or 1700 unique Anki flashcards was associated with an additional point on Step 1 when controlling for other academic and psychological factors.ConclusionsMedical students engage extensively in self-initiated retrieval practice, often with spaced repetition. These practices are associated with superior performance on a medical licensing examination and should be considered for formal support by educators.

When I say … test-enhanced learning

Anyone over 30 years of age has probably had the experience of trying to learn a phone number. This is a dying art in the age of smartphones; if you haven’t had the experience, just try to imagine it. When you tried to learn a new phone number, did you re-read it again and again hoping that you would remember it? Of course not. Rather, you said it over and over again until you used it. To learn the phone number, you used a technique known as ‘retrieval practice’ (although you probably didn’t know what it was called).

Quizzes and conversations: exploring the role of retrieval in medical education

Editor’s note: This article is part of a series in Medical Education entitled ‘Dialogue’. Each publication in the series will be a transcription of an e-mail discussion about a current issue in the field held by two scholars who have approached the issue from different perspectives. For further details, see the editorial published in Med Educ 2012;46(9):826–7. In this issue, Douglas P Larsen, Director for Medical Student Education for the Division of Paediatric Neurology, Washington University in St Louis, and Tim Dornan, Professor of Medical Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, discuss the roles that testing, broadly defined, and social interaction play in the acquisition and retention of medical knowledge.

Student views on the role of self-regulated learning in a surgery clerkship

BACKGROUND Self-regulated learning, including student-generated learning goals and flexibility in the learning structure are increasingly being used to enhance medical education. The role of these practices in surgical education of medical students has not been studied. MATERIALS AND METHODS We administered an 18-question electronic survey to all third-year medical students at Washington University in St. Louis School of Medicine. Of the 126 students invited, 64 responded and 56 were included in the analysis. RESULTS We found that third-year medical students develop learning goals at the beginning of the surgery clerkship. Although these learning goals theoretically can be a mechanism for enhanced student-faculty engagement, students are not aware of formal mechanisms for sharing these goals with faculty members. Furthermore, students report a lack of flexibility within the surgery clerkship and discomfort with requesting specific learning opportunities. Finally, students report that they believe increased flexibility could improve student engagement, learning, and the overall clerkship experience. CONCLUSIONS We therefore propose that a mechanism for students to share their learning goals with faculty and an infrastructure in which student learning experiences can be tailored to fit with these individualized goals would enhance student surgical learning.