Debra L. Klamen

Designing a Patient Safety Undergraduate Medical Curriculum: The Telluride Interdisciplinary Roundtable Experience

Purpose: Patient safety has emerged as a global concern in the provision of quality health care, and yet, to date, few medical schools have created and/or implemented patient safety curricula. The purpose of this article is to introduce readers to one model of a patient safety undergraduate medical curriculum, as designed by a group of experts attending an annual interdisciplinary roundtable assembled for this purpose. Summary: The Annual Telluride Interdisciplinary Roundtable met in 2005 and 2006 to design what it considered to be a comprehensive patient safety curriculum for medical students. Invited members included stakeholders from a variety of fields, including health care providers, senior health care administration, students, residents, patient advocacy leaders, and curriculum development/assessment experts. The group developed a list of general curricular principles, followed by 11 specific elements felt to be essential to an effective patient safety curriculum for undergraduate medical education students. It also identified a number of challenges to implementing such a curriculum. Conclusions: A patient safety curriculum, developed by a group of experts for an undergraduate medical education population, was successfully developed over a two-year period of time. Future meetings of the Telluride Roundtable group have centered on evaluation and refinement of these curricular elements as pilots occur in a number of medical schools, and new curricular ideas continue to be developed. Continued interprofessional dialogue and collaborative research will enable the development and implementation of a standardized longitudinal patient safety student curriculum.

Forecasting residents' performance--partly cloudy.

The authors offer a practical guide for improving the appraisal of a residents performance. They identify six major factors that compromise the process of observing, measuring, and characterizing a residents current performance, forecasting future performance, and making decisions about the residents progress. Factors that compromise any of these steps lead to individual and collective uncertainty and decrease faculty confidence when making decisions on a residents progress. The six factors, addressed in order of importance, are inaccuracies due to (1) incomplete sampling of performance, (2) rater memory constraints, (3) hidden performance deficits of the resident, (4) lack of performance benchmarks, (5) faculty members’ hesitancy to act on negative performance information, and (6) systematic rater error. The description of each factor is followed by a number of specific suggestions on what residency programs can do to eliminate or minimize the impact of these factors. While this article is couched in the context of the performance evaluation of residents, everything included pertains to measuring and appraising medical students’ and practicing physicians’ clinical performance as well.

Tracking development of clinical reasoning ability across five medical schools using a progress test.

Purpose Little is known about the acquisition of clinical reasoning skills in medical school, the development of clinical reasoning over the medical curriculum as a whole, and the impact of various curricular methodologies on these skills. This study investigated (1) whether there are differences in clinical reasoning skills between learners at different years of medical school, and (2) whether there are differences in performance between students at schools with various curricular methodologies. Method Students (n = 2,394) who had completed zero to three years of medical school at five U.S. medical schools participated in a cross-sectional study in 2008. Students took the same diagnostic pattern recognition (DPR) and clinical data interpretation (CDI) tests. Percent correct scores were used to determine performance differences. Data from all schools and students at all levels were aggregated for further analysis. Results Student performance increased substantially as a result of each year of training. Gains in DPR and CDI performance during the third year of medical school were not as great as in previous years across the five schools. CDI performance and performance gains were lower than DPR performance and gains. Performance gains attributable to training at each of the participating medical schools were more similar than different. Conclusions Years of training accounted for most of the variation in DPR and CDI performance. As a rule, students at higher training levels performed better on both tests, though the expected larger gains during the third year of medical school did not materialize.

Pursuing Professional Accountability An Evidence-Based Approach to Addressing Residents With Behavioral Problems

OBJECTIVE To develop an evidence-based approach to the identification, prevention, and management of surgical residents with behavioral problems. DESIGN The American College of Surgeons and Southern Illinois University Department of Surgery hosted a 1-day think tank to develop strategies for early identification of problem residents and appropriate interventions. Participants read a selection of relevant literature before the meeting and reviewed case reports. SETTING American College of Surgeons headquarters, Chicago, Illinois. PARTICIPANTS Medical and nursing leaders in the field of resident education; individuals with expertise in dealing with academic law, mental health issues, learning deficiencies, and disruptive physicians; and surgical residents. MAIN OUTCOME MEASURES Evidence-based strategies for the identification, prevention, and management of problem residents. RESULTS Recommendations based on the literature and expert opinions have been made for the identification, remediation, and reassessment of problem residents. CONCLUSIONS It is essential to set clear expectations for professional behavior with faculty and residents. A notice of deficiency should define the expected acceptable behavior, timeline for improvement, and consequences for noncompliance. Faculty should note and address systems problems that unintentionally reinforce and thus enable unprofessional behavior. Complaints, particularly by new residents, should be investigated and addressed promptly through a process that is transparent, fair, and reasonable. The importance of early intervention is emphasized.

See one, do one, teach one--exploring the core teaching beliefs of medical school faculty.

This paper explores the core teaching beliefs of medical school faculty and establishes whether these beliefs differ among basic science, clinical, and instruction specialist faculty. One hundred and twenty-five medical school teachers who were members of professional organizations dedicated to the improvement of medical school teaching completed a Q-sort of 56 statements reflecting their core teaching beliefs. The statements described beliefs about motivation, knowledge and skill acquisition, retention, feedback, transfer, teacher characteristics, and teaching strategies. Q-sorts were completed by 37 basic scientists (30% of respondents), 59 clinicians (47%) and 29 instruction specialists (23%) working in medical schools. Fifty-two participants were classroom teachers (42%), 66 were classroom and clinical teachers (53%), and seven reported that they do not teach (6%). The Q-sort results indicate how medical school faculty members differ in their core beliefs about teaching and learning. Thirty-two respondents (26%) focused on the student as a person first. Eight (6%) were content oriented. Thirty-four (27%) were performance oriented; their focus was on having students learn and apply knowledge and skills to accomplish clinical tasks. Fifty-one respondents (41%) were found to have a blend of these viewpoints. Respondents’ type of training or type of teaching did not provide a reliable indication of core teaching beliefs classification.

Students learning handovers in a simulated in-patient unit.

Context and setting History-taking skills have undisputable relevance in medical student education. Experiential methods, such as videotaped interviews and simulated patient encounters with feedback, have been shown in evidence-based reviews to be effective for communication skills learning. Why the idea was necessary Many medical schools lack resources such as videotaping facilities and access to standardised patients, and history-taking skills are taught using lectures and practice sessions with real patients. Although students may enjoy practising with real patients, encounters are usually unobserved and may cause discomfort to both participants. What was done With the aim of enhancing active learning on history taking, a structured, three-way, role-play activity was devised and is being successfully utilised within an elective course. This course is offered to junior medical students who have just completed a compulsory course on basic clinical skills. The course programme consists of selected readings and discussions on factors affecting doctor– patient communication, as well as practical activities consisting mainly of real patient interviews followed by group discussions. The role-play activity starts with the teacher explaining the activity and emphasising the three different roles of doctor, patient and observer. Special attention is given to the observer role as this person is in charge of delivering structured feedback based on checklist-documented behaviours. Students then memorise individually the features of a common case from a series of previously constructed baseline patients. Groups of nine to 12 students are split into subgroups of three students, who rotate in the roles of doctor, patient and observer. Observed patient–doctor encounters then take place for 5–10 minutes. Thereafter, the observer delivers feedback on the doctor role-player performance, which is followed by comments from the patient and doctor players. Roles are rotated and, after three interviews, individual members successively rotate through the different subgroups, so that each student can play each role and obtain the relevant feedback. The final part of the session includes comments and feedback from the teacher to the players, as well as a general discussion on what was learned. For three consecutive years, students (n = 21) evaluated the role-play activity and various aspects of course using a 5-point scale (1 = poor, 5 = excellent) and open questions. Evaluation of results and impact All students gave the course an overall grade of 4 or 5. Although 20 of 21 students reported that both the role-play activity and real patient interviews were good or excellent, the proportion of students who rated the role-play activity as excellent was significantly greater than the proportion of those who rated the interviews with real patients as excellent (18 ⁄ 21 versus 10 ⁄ 21; P = 0.02). Narrative evaluations highlighted the perceived advantages afforded by the opportunity for extensive practice with common cases and the receipt of immediate feedback from both patient and observer players. Comments on patient and observer roleplaying indicate that this may also contribute to reflection on the learning process. Finally, students indicated in a nearly unanimous manner that the role-play activity should be introduced as compulsory for all students. Further systematic utilisation and evaluation of this structured educational activity may provide evidence of a positive impact on student performance in history taking.

Improving resident performance assessment data: Numeric precision and narrative specificity

Purpose To evaluate the use of a systems approach for diagnosing performance assessment problems in surgery residencies, and intervene to improve the numeric precision of global rating scores and the behavioral specificity of narrative comments. Method Faculty and residents at two surgery programs participated in parallel before-and-after trials. During the baseline year, quality assurance data were gathered and problems were identified. During two subsequent intervention years, an educational specialist at each program intervened with an organizational change strategy to improve information feedback loops. Three quality-assurance measures were analyzed: (1) percentage return rate of forms, (2) generalizability coefficients and 95% confidence intervals of scores, and (3) percentage of forms with behaviorally specific narrative comments. Results Median return rates of forms increased significantly from baseline to intervention Year 1 at Site A (71% to 100%) and Site B (75% to 100%), and then remained stable during Year 2. Generalizability coefficients increased between baseline and intervention Year 1 at Site A (0.65 to 0.85) and Site B (0.58 to 0.79), and then remained stable. The 95% confidence interval around resident mean scores improved at Site A from baseline to intervention Year 1 (0.78 to 0.58) and then remained stable; at Site B, it remained constant throughout (0.55 to 0.56). The median percentage of forms with behaviorally specific narrative comments at Site A increased significantly from baseline to intervention Years 1 and 2 (50%, 57%, 82%); at Site B, the percentage increased significantly in intervention Year 1, and then remained constant (50%, 60%, 67%). Conclusions Diagnosing performance assessment system problems and improving information feedback loops improved the quality of resident performance assessment data at both programs.

Examining the Diagnostic Justification Abilities of Fourth-Year Medical Students

Purpose Fostering ability to organize and use medical knowledge to guide data collection, make diagnostic decisions, and defend those decisions is at the heart of medical training. However, these abilities are not systematically examined prior to graduation. This study examined diagnostic justification (DXJ) ability of medical students shortly before graduation. Method All senior medical students in the Classes of 2011 (n = 67) and 2012 (n = 70) at Southern Illinois University were required to take and pass a 14-case, standardized patient examination prior to graduation. For nine cases, students were required to write a free-text response indicating how they used patient data to move from their differential to their final diagnosis. Two physicians graded each DXJ response. DXJ scores were compared with traditional standardized patient examination (SCCX) scores. Results The average intraclass correlation between raters’ rankings of DXJ responses was 0.75 and 0.64 for the Classes of 2011 and 2012, respectively. Student DXJ scores were consistent across the nine cases. Using SCCX and DXJ scores led to the same pass–fail decision in a majority of cases. However, there were many cases where discrepancies occurred. In a majority of those cases, students would fail using the DXJ score but pass using the SCCX score. Common DXJ errors are described. Conclusions Commonly used standardized patient examination component scores (history/physical examination checklist score, findings, differential diagnosis, diagnosis) are not direct, comprehensive measures of DXJ ability. Critical deficiencies in DXJ abilities may thus go undiscovered.

Competencies, milestones, and EPAs – Are those who ignore the past condemned to repeat it?

Abstract Background: The idea of competency-based education sounds great on paper. Who wouldn’t argue for a standardized set of performance-based assessments to assure competency in graduating students and residents? Even so, conceptual concerns have already been raised about this new system and there is yet no evidence to refute their veracity. Aims: We argue that practical concerns deserve equal consideration, and present evidence strongly suggesting these concerns should be taken seriously. Method: Specifically, we share two historical examples that illustrate what happened in two disparate contexts (K-12 education and the Department of Defense [DOD]) when competency (or outcomes-based) assessment frameworks were implemented. We then examine how observation and assessment of clinical performance stands currently in medical schools and residencies, since these methodologies will be challenged to a greater degree by expansive lists of competencies and milestones. Results/Conclusions: We conclude with suggestions as to a way forward, because clearly the assessment of competency and the ability to guarantee that graduates are ready for medical careers is of utmost importance. Hopefully the headlong rush to competencies, milestones, and core entrustable professional activities can be tempered before even more time, effort, frustration and resources are invested in an endeavor which history suggests will collapse under its own weight.

The Efficacy of a Targeted Remediation Process for Students Who Fail Standardized Patient Examinations

Background: Current remediation strategies for students failing standardized patient examinations represent poorly targeted approaches since the specific nature of clinical performance weaknesses has not been defined. Purpose: The purpose is to determine the impact of a specifically targeted clinical performance course required of students who failed a clinical performance examination. Methods: A month-long clinical performance course, targeted to treat specific types of clinical performance deficiencies, was designed to remediate students failing standardized patient examinations in 2007 (n = 8) and 2008 (n = 5). Participating students were assessed on pre- and postperformance measures, including multiple-choice tests that measured diagnostic pattern recognition and clinical data interpretation and clinical performance measures using standardized clinical encounters. Comparisons between average pre- and postintervention performance scores were computed using paired sample t tests. Results were adjusted for regression toward the mean. Results: In both 2007 and 2008, the mean preintervention clinical data interpretation and standardized patient examination scores were below the criterion referenced passing standard set for the clinical competency exam. In both years the mean postintervention scores for the participants were above the passing standard for these two examinations. Pre- and postintervention differences were statistically significant in both cases. Conclusions: This study provides insight into the reasons that students fail clinical performance examinations and elucidates one method by which such students may be successfully remediated.