Reena Karani

Direct Observation in Medical Education: A Review of the Literature and Evidence for Validity

In 2000, the Accreditation Council for Medical Education introduced a new initiative that substantively changed the method by which residency programs are evaluated. In this new competency-based approach to residency education, assessment of performance became a main area of interest, and direct observation was offered as a tool to assess knowledge and skills. Despite being an inherent part of medical education as faculty and learners work together in clinical experiences, direct observation has traditionally been an informal and underused assessment method across all specialties. Residents and students report rarely being observed during their educational process, even though they value the experience. Reasons for this include a lack of faculty time, a lack of faculty skills, a potential stressful effect on the learner, and a perceived lack of validation of the assessment. This article examines the literature regarding the use of direct observation in medical education with a focus on validity evidence. We performed a PubMed search of articles pertaining to direct observation, using key words such as direct observation, performance observation, clinical observation, students, and residents. A subsequent search was conducted in known articles, focusing on variations of the term observation in the titles of articles and introducing the concept of clinical competence. In conclusion, direct observation is a unique and useful tool in the assessment of medical students and residents. Assessing learners in natural settings offers the opportunity to see beyond what they know and into what they actually do, which is fundamentally essential to training qualified physicians. Although the literature identifies several threats to its validity as an assessment, it also demonstrates methods to minimize those threats. Based on the current recommendations and need for performance assessment in education and with attention paid to the development and design, direct observation can and should be included in medical education curricula.

Readability of Patient Education Materials Available at the Point of Care

BackgroundMany patient education materials (PEMs) available on the internet are written at high school or college reading levels, rendering them inaccessible to the average US resident, who reads at or below an 8th grade level. Currently, electronic health record (EHR) providers partner with companies that produce PEMs, allowing clinicians to access PEMs at the point of care.ObjectiveTo assess the readability of PEMs provided by a popular EHR vendor as well as the National Library of Medicine (NLM).DesignWe included PEMs from Micromedex, EBSCO, and MedlinePlus. Micromedex and EBSCO supply PEMs to Meditech, a popular EHR supplier in the US. MedlinePlus supplies the NLM. These PEM databases have high market penetration and accessibility.MeasurementsGrade reading level of the PEMs was calculated using three validated indices: Simple Measure of Gobbledygook (SMOG), Gunning Fog (GFI), and Flesch–Kincaid (FKI). The percentage of documents above target readability and average readability scores from each database were calculated.ResultsWe randomly sampled 100 disease-matched PEMs from three databases (n = 300 PEMs). Depending on the readability index used, 30-100% of PEMs were written above the 8th grade level. The average reading level for MedlinePlus, EBSCO, and Micromedex PEMs was 10.2 (1.9), 9.7 (1.3), and 8.6 (0.9), respectively (p ≤ 0.000) as estimated by the GFI. Estimates of readability using SMOG and FKI were similar.ConclusionsThe majority of PEMS available through the NLM and a popular EHR were written at reading levels considerably higher than that of the average US adult.

How medical students learn from residents in the workplace: A qualitative study

Purpose To explore what third-year medical students learn from residents and which teaching strategies are used by excellent resident teachers in their interactions with students in the clinical workplace environment. Method In this multi-institutional qualitative study between January and March 2012, the authors conducted focus groups with medical students who were midway through their third year. Qualitative analysis was used to identify themes. Results Thirty-seven students participated. Students contributed 228 comments related to teaching methods used by residents. The authors categorized these into 20 themes within seven domains: role-modeling, focusing on teaching, creating a safe learning environment, providing experiential learning opportunities, giving feedback, setting expectations, and stimulating learning. Role-modeling, the most frequently classified method of teaching in this study, was not included in three popular “Resident-as-Teacher” (RAT) models. Strategies including offering opportunities for safe practice, involving students in the team, and providing experiential learning opportunities were not emphasized in these models either. Almost 200 comments representing the knowledge and skills students learned from residents were categorized into 33 themes within nine domains: patient care, communication, navigating the system, adaptability, functioning as a student/resident, lifelong learning, general comments, career/professional development, and medical content. Most of these areas are not emphasized in popular RAT models. Conclusions Residents serve as critically important teachers of students in the clinical workplace. Current RAT models are based largely on the teaching behaviors of faculty. The content and teaching strategies identified by students in this study should serve as the foundation for future RAT program development.

Impact of a 360-degree professionalism assessment on faculty comfort and skills in feedback delivery.

BackgroundProfessionalism is identified as a competency of resident education. Best approaches to teaching and evaluating professionalism are unknown, but feedback about professionalism is necessary to change practice and behavior. Faculty discomfort with professionalism may limit their delivery of feedback to residents.ObjectivesA pilot program to implement a 360-degree evaluation of observable professionalism behaviors and determine how its use impacts faculty feedback to residents.DesignInternal Medicine (IM) residents were evaluated during ambulatory rotations using a 360-degree assessment of professional behaviors developed by the National Board of Medical Examiners®. Faculty used evaluation results to provide individual feedback to residents.Patients/ParticipantsFifteen faculty members.Measurements and Main ResultsFaculty completed pre- and post-intervention surveys. Using a 7-point Likert scale, faculty reported increased skill in giving general feedback (4.85 vs 4.36, p < .05) and feedback about professionalism (4.71 vs 3.57, p < .01) after the implementation of the 360-degree evaluation. They reported increased comfort giving feedback about professionalism (5.07 vs 4.35, p < .05) but not about giving feedback in general (5.43 vs 5.50).ConclusionsA 360-degree professionalism evaluation instrument used to guide feedback to residents improves faculty comfort and self-assessed skill in giving feedback about professionalism.

Succeeding as a Clinician Educator: Useful Tips and Resources

Clinician Educators (CEs) play an essential role in the education and patient care missions of academic medical centers. Despite their crucial role, academic advancement is slower for CEs than for other faculty. Increased clinical productivity demands and financial stressors at academic medical centers add to the existing challenges faced by CEs. This perspective seeks to provide a framework for junior CEs to consider with the goal of maximizing their chance of academic success. We discuss six action areas that we consider central to flourishing at academic medical centers: 1. Clarify what success means and define goals; 2. Seek mentorship and be a responsible mentee; 3. Develop a niche and engage in relevant professional development; 4. Network; 5. Transform educational activities into scholarship; and 6. Seek funding and other resources.

Audit and feedback: an intervention to improve discharge summary completion.

Discharge summaries (DS) communicate important clinical information from inpatient to outpatient settings. Previous studies noted increased adverse events and rehospitalization due to poor DS quality. We postulated that an audit and feedback intervention of DS completed by geriatric medicine fellows would improve the completeness of their summaries. We conducted a preintervention post intervention study. In phase 1 (AUDIT #1 and FEEDBACK) we scored all DS (n = 89) completed by first year fellows between July 2006 to December 2006 using a 21-item checklist. Individual performance scores were reviewed with each fellow in 30-minute feedback sessions. In phase 2 (AUDIT #2) we scored all DS (n = 79) completed after the first phase between February 2007 to July 2007 using the same checklist. Data were analyzed using generalized estimating equations. Fellows were more likely to complete all required DS data after feedback when compared with prior to feedback (91% vs. 71%, P < 0.001). Feedback was also associated with improved admission (93% vs. 70%, P < 0.001), duration of hospitalization (93% vs 78%, P < 0.001), discharge planning (93% vs. 18%, P < 0.02) and postdischarge care (83% vs. 57%., P < 0.001) section-specific information. In conclusion, audit and feedback sessions were associated with better DS completeness in areas of particular importance to geriatric care.

Medical student documentation in the medical record: is it a liability?

Medical students have routinely documented patient encounters in both inpatient and outpatient care venues. This hands-on experience has provided a way for students to reflect on patient encounters, learn proper documentation skills, and attain a sense of being actively involved in and responsible for the care of patients. Over the last several years, the practice of student note writing has come into question. Institutional disincentives to student documentation include insurance regulations that restrict student documentation from substantiating billing claims, concerns about the legal status of student notes, and implementation of electronic medical records that do not allow or restrict student access. The increased scrutiny of the medical record from pay-for-performance programs and other quality measures will likely add to the pressure to exclude students from writing notes. This trend in limiting medical student documentation may have wide-ranging consequences for student education, from delaying the learning of proper documentation skills to limiting training opportunities. This article reviews the educational value of student note writing, the factors that have made student documentation problematic, and the potential educational impact of limiting student documentation. In addition, it offers some suggestions for future research to guide policy in this area.

Medical students' experiences with authorship in biomedical research: a national survey.

Purpose To explore authorship issues related to medical students’ primary research projects, assess medical students’ knowledge about authorship issues in biomedical research, and determine their interest in learning about authorship guidelines. Method In 2011, the authors developed and conducted an electronic survey of 243 U.S. medical students who attended an educational event at the National Institutes of Health as part of their funded, yearlong research fellowship programs. The authors then analyzed the results using descriptive statistics. Results Of 243 students, 152 (63%) responded. Most (120/151; 79%) had completed or were in the process of writing a manuscript based on their projects. Of these, most (95/119; 80%) wrote the entire manuscript independently or with guidance. Whereas almost two-thirds (99/152; 65%) indicated that expectations and criteria for authorship were clarified for them, 26% (40/152) indicated that they were not. Most students (108/118; 92%) were in the authorship position they expected and had no concerns about who the other authors were (91/119; 77%). Of those with concerns, 52% (11/21) did not raise the issue for fear of challenging their mentor. Two-thirds (95/145; 66%) never received formal training in authorship guidelines, and 41% (42/103) believed such training would be valuable. Conclusions Although a majority of students had conversations about authorship and were clear about the guidelines for ethical authorship, additional work is needed. The authors recommend that academic institutions develop a menu of options for teaching students about this important area in research ethics.

Seeing Is Believing: Evaluating a Point-of-Care Ultrasound Curriculum for 1st-Year Medical Students.

ABSTRACT Problem: Point-of-care ultrasound has been a novel addition to undergraduate medical education at a few medical schools. The impact is not fully understood, and few rigorous assessments of educational outcomes exist. This study assessed the impact of a point-of-care ultrasound curriculum on image acquisition, interpretation, and student and faculty perceptions of the course. Intervention: All 142 first-year medical students completed a curriculum on ultrasound physics and instrumentation, cardiac, thoracic, and abdominal imaging. A flipped classroom model of preclass tutorials and tests augmenting live, hands-on scanning sessions was incorporated into the physical examination course. Students and faculty completed surveys on impressions of the curriculum, and all students under-went competency assessments with standardized patients. Context: The curriculum was a mandatory part of the physical examination course and was taught by experienced clinician-sonographers as well as faculty who do not routinely perform sonography in their clinical practice. Outcome: Students and faculty agreed that the physical examination course was the right time to introduce ultrasound (87% and 80%). Students demonstrated proper use of the ultrasound machine functions (M score = 91.55), and cardiac, thoracic, and abdominal system assessments (M score = 80.35, 79.58, and 71.57, respectively). Students and faculty valued the curriculum, and students demonstrated basic competency in performance and interpretation of ultrasound. Further study is needed to determine how to best incorporate this emerging technology into a robust learning experience for medical students.

A Patient Safety and Transitions of Care Curriculum for Third-Year Medical Students

The elderly are the most vulnerable to adverse events during and after hospitalization. This study sought to evaluate the effectiveness of a curriculum on patient safety and transitions of care for medical students during an Internal Medicine-Geriatrics Clerkship on students’ knowledge, skills, and attitudes. The curriculum included didactics on patient safety, health literacy, discharge planning and transitions of care, and postdischarge visits to patients. Analysis of pre- and postassessments showed afterwards students were significantly more comfortable assessing a patient’s health literacy and confident performing a medication reconciliation, providing education regarding medications, and identifying barriers during transitions. More students were able to identify the most common source of adverse events after discharge (86% vs. 62% before), risk factors for low health literacy (28% vs. 14%), and ways to assess a patient’s health literacy (14% vs. 2%). It was feasible to implement a postdischarge visit assignment in an urban tertiary care setting and only required on average of approximately an one and one half hours for students to complete.