S. Beth Bierer

Peer assessment of professional competence.

Background  Current assessment formats for medical students reliably test core knowledge and basic skills. Methods for assessing other important domains of competence, such as interpersonal skills, humanism and teamwork skills, are less well developed. This study describes the development, implementation and results of peer assessment as a measure of professional competence of medical students to be used for formative purposes.

Psychometric Properties of an Instrument Designed to Measure the Educational Quality of Graduate Training Programs

To assess the quality of residency education programs at an academic medical center for purposes of enhancing individual graduate medical education programs, we asked residents and fellows (N = 419) to evaluate their training programs using a Web-based questionnaire (response rate = 70%). Kruskal-Wallis tests, factor analysis, correlations, generalizability/decision studies, and mean plots were used to examine trainee responses and to assess the questionnaire’s measurement properties. Exploratory factor analysis indicated that the instrument had a threefactor structure that correlated highly with overall program rating. Cronbach’s alpha exceeded .80 for all factors, and decision studies revealed that 13 to 23 raters were needed to obtain G-coefficients greater than .70. Mean plots showed that the instrument could discriminate within and among training programs at the item level and the factor level, which should help target improvements across graduate training programs within large institutions.

The Learning Environment Counts: Longitudinal Qualitative Analysis of Study Strategies Adopted by First-Year Medical Students in a Competency-Based Educational Program.

Purpose The move toward competency-based education will require medical schools and postgraduate training programs to restructure learning environments to motivate trainees to take personal ownership for learning. This qualitative study explores how medical students select and implement study strategies while enrolled in a unique, nontraditional program that emphasizes reflection on performance and competence rather than relying on high-stakes examinations or grades to motivate students to learn and excel. Method Fourteen first-year medical students volunteered to participate in three, 45-minute interviews (42 overall) scheduled three months apart during 2013–2014. Two medical educators used structured interview guides to solicit students’ previous assessment experiences, preferred learning strategies, and performance monitoring processes. Interviews were digitally recorded and transcribed verbatim. Participants confirmed accuracy of transcripts. Researchers independently read transcripts and met regularly to discuss transcripts and judge when themes achieved saturation. Results Medical students can adopt an assessment for learning mind-set with faculty guidance and implement appropriate study strategies for mastery-learning demands. Though students developed new strategies at different rates during the year, they all eventually identified study and performance monitoring strategies to meet learning needs. Students who had diverse learning experiences in college embraced mastery-based study strategies sooner than peers after recognizing that the learning environment did not reward performance-based strategies. Conclusions Medical students can take ownership for their learning and implement specific strategies to regulate behavior when learning environments contain building blocks emphasized in self-determination theory. Findings should generalize to educational programs seeking strategies to design learning environments that promote self-regulated learning.

Case Western Reserve University School of Medicine and Cleveland Clinic.

University-based program (University track) was established in 1843 and Cleveland Clinic-based Cleveland Clinic Lerner College of Medicine (College track) was established in 2002. University track implemented a major curriculum revision in 2006, focusing on development of lifelong learners and incorporating themes of scholarship, clinical mastery, leadership, and civic professionalism. A dedicated four-month research requirement culminating in an MD thesis was incorporated into the four-year curriculum. A hybrid of active, student-centered PBL groups and teacher-centered large and medium group sessions form the backbone of learning venues. Five-year competency-based College track is designed to train graduates with excellent clinical skills, expertise in research, and a passion for scientific inquiry. Students complete a master’s level thesis and graduate with an MD degree with Special Qualification in Biomedical Research. While course format, assessment approaches, and curricular topics beyond core content of University and College tracks differ, clinical rotations for both tracks were designed collaboratively and implemented jointly.

Tool for Diagnosing Gaps Within a Competency-Based Assessment System

Tool for Diagnosing Gaps Within a Competency-Based Assessment System S. Bierer;Colleen Colbert;Cecile Foshee;Judith French;Lily Pien; Academic Medicine