Robert Treat

Medical student case presentation performance and perception when using mobile learning technology in the emergency department

Abstract Background Hand-held mobile learning technology provides opportunities for clinically relevant self-instructional modules to augment traditional bedside teaching. Using this technology as a teaching tool has not been well studied. We sought to evaluate medical students’ case presentation performance and perception when viewing short, just-in-time mobile learning videos using the iPod touch prior to patient encounters. Methods Twenty-two fourth-year medical students were randomized to receive or not to receive instruction by video, using the iPod Touch, prior to patient encounters. After seeing a patient, they presented the case to their faculty, who completed a standard data collection sheet. Students were surveyed on their perceived confidence and effectiveness after using these videos. Results Twenty-two students completed a total of 67 patient encounters. There was a statistically significant improvement in presentations when the videos were viewed for the first time (p = 0.032). There was no difference when the presentations were summed for the entire rotation (p = 0.671). The reliable (alpha = 0.97) survey indicated that the videos were a useful teaching tool and gave students more confidence in their presentations. Conclusions Medical student patient presentations were improved with the use of mobile instructional videos following first time use, suggesting mobile learning videos may be useful in medical student education. If direct bedside teaching is unavailable, just-in-time iPod touch videos can be an alternative instructional strategy to improve first-time patient presentations by medical students.

Outcomes Results From the Evaluation of the APA/HRSA Faculty Scholars Program

BACKGROUND The goal of the Ambulatory Pediatric Association/Health Resources and Services Administration National Faculty Development Scholars Program was to improve primary care education in the pediatric setting. The program evaluation focused on four stake-holder objectives: 1) increase the educational skills of community and generalist faculty; 2) create pediatric leadership focused on changing the culture within the medical community to support primary care education; 3) develop an infrastructure that supports sustained faculty development efforts at the local, regional, and national level; and 4) include content areas consistent with Health Resources and Services Administration contract requirements. METHODS A multimethod evaluation plan, focused on the 107 completing scholars, was implemented utilizing six evaluation instruments. RESULTS Key outcomes from both quantitative and qualitative outcome measures reveal that all evaluation objectives were achieved. Scholars presented 438 local workshops and 161 regional/national workshops focused on pediatric education with a combined attendance of 7939 participants. More than half of the scholars have now assumed a leadership position in education associated with program participation. Ninety-three percent of the scholars reported organizational/infrastructure changes associated with their program participation ranging from increased numbers of community teaching sites to specific resource allocations to support of faculty development. CONCLUSIONS The outcomes of this evaluation reveal that the faculty development program achieved its objectives, with participants leading workshops, impacting faculty development infrastructure, advancing their own careers, and being strategically positioned in leadership roles with the skills to improve primary care education in the ambulatory setting.

No relationship between measures of clinical efficiency and teaching effectiveness for emergency medicine faculty

Objectives Emergency medicine (EM) doctors affiliated with academic institutions experience professional tension between providing excellent, timely care for patients and high-quality bedside instruction for residents and medical students. The goal of this study was to assess the relationship between measures of faculty clinical efficiency and teaching effectiveness. Methods This was a retrospective review of data from a single academic institution with an annual census of 55 000. Faculty clinical efficiency was measured by two variables: the relative value unit (RVU)/h ratio and average ‘door to discharge’ time. Teaching effectiveness was estimated by determining the average ‘overall teaching’ scores derived from anonymous EM resident and senior medical student evaluations. Relationships were assessed using the Spearmans correlation coefficient. Results There was no statistically significant relationship (p>0.050) between measures of faculty clinical efficiency and teaching effectiveness. Conclusion These data replicate previous findings that clinical productivity has no correlation with teaching effectiveness for emergency medicine faculty doctors.

Have clinical teaching effectiveness ratings changed with the Medical College of Wisconsin's entry into the health care marketplace?

Medical schools, as competitors in today’s health care marketplace, have the challenge of training future physicians while increasingly relying on clinical revenues. Is teaching compatible with competitive managed care in the future of health care? Skeff, Bowen, and Irby argue that teaching takes time and that its values must be re-emphasized as a core mission of medical schools. Medical education researchers have reported diminishing amounts of time available for physicians’ educational responsibilities to both residents and medical students. Student evaluations reveal that there has been less time available for them in more recent years. Thus, time impacts on education have been documented, but the critical issue to be investigated is whether the quality of teaching has been compromised. As a large, private medical school, the Medical College of Wisconsin (MCW) has not escaped the grasp of today’s competitive health care environment. On December 31, 1995, the John L. Doyne Hospital (JLDH), formerly Milwaukee County General Hospital, was closed. While this facility (a primary practice and clinical teaching site) was purchased by a private adult not-for-profit hospital, it’s sale nonetheless serves as a major demarcation point in MCW’s transition into today’s health care marketplace. Indigent care was now provided on a competitive contract basis. Our faculty formed a clinical practice group to enhance their competitive position in this evolving health care environment. Declining federal support for graduate medical education led to decreased positions in selected specialties and their associated support of medical student education. While the multi-dimensional impact of these changes on medical education, at MCW and elsewhere, will take years to analyze, preliminary analysis can reveal whether the quality of clinical teaching has changed during this time period. This study, therefore, examined whether there have been changes in clinical teaching effectiveness ratings as clinicians at MCW compete for patients and revenue.

Surgery Residency Curriculum Examination Scores Predict Future American Board of Surgery In-Training Examination Performance

IMPORTANCE A protected block curriculum (PBC) with postcurriculum examinations for all surgical residents has been provided to assure coverage of core curricular topics. Biannual assessment of resident competency will soon be required by the Next Accreditation System. OBJECTIVE To identify opportunities for early medical knowledge assessment and interventions, we examined whether performance in postcurriculum multiple-choice examinations (PCEs) is predictive of performance in the American Board of Surgery In-Training Examination (ABSITE) and clinical service competency assessments. DESIGN Retrospective single-institutional education research study. SETTING Academic general surgery residency program. PARTICIPANTS A total of 49 surgical residents. INTERVENTION Data for PGY1 and PGY2 residents participating in the 2008 to 2012 PBC are included. Each resident completed 6 PCEs during each year. MAIN OUTCOME MEASURES The results of 6 examinations were correlated to percentage-correct ABSITE scores and clinical assessments based on the 6 Accreditation Council for Graduate Medical Education core competencies. Individual ABSITE performance was compared between PGY1 and PGY2. Statistical analysis included multivariate linear regression and bivariate Pearson correlations. RESULTS A total of 49 residents completed the PGY1 PBC and 36 completed the PGY2 curriculum. Linear regression analysis of percentage-correct ABSITE and PCE scores demonstrated a statistically significant correlation between the PGY1 PCE 1 score and the subsequent PGY1 ABSITE score (p = 0.037, β = 0.299). Similarly, the PGY2 PCE 1 score predicted performance in the PGY2 ABSITE (p = 0.015, β = 0.383). The ABSITE scores correlated between PGY1 and PGY2 with statistical significance, r = 0.675, p = 0.001. Performance on the 6 Accreditation Council for Graduate Medical Education core competencies correlated between PGY1 and PGY2, r = 0.729, p = 0.001, but did not correlate with PCE scores during either years. CONCLUSIONS AND RELEVANCE Within a mature PBC, early performance in a PGY1 and PGY2 PCE is predictive of performance in the respective ABSITE. This information can be used for formative assessment and early remediation of residents who are predicted to be at risk for poor performance in the ABSITE.

Impact of group size on the effectiveness of a resuscitation simulation curriculum for medical students.

Background: Simulation requires involvement from participants. However, it is unknown to what extent simulation effectiveness is a function of the number of participants. Purpose: This study assessed the impact of varying group size on medical students’ subjective experience of simulation and on postsimulation exam performance. Methods: Medical students were randomly assigned to groups of 2, 3, or 4. Retrospective assessment was done through a survey assessing confidence and knowledge as it relates to resuscitation and statements related to group size. Performance on a postsimulation exam was analyzed. Results: There were significant increases in students’ confidence and knowledge following simulation. There were no significant differences in student perception of the effectiveness or realism of the simulation or in performance on the postsimulation exam as a function of group size. Conclusions: Students feel that simulation is an effective way to learn medical knowledge. Varying group size had no effect on students’ subjective experience or exam performance.

Evaluating handoffs in the context of a communication framework

Background. The implementation of mandated restrictions in resident duty hours has led to increased handoffs for patient care and thus more opportunities for errors during transitions of care. Much of the current handoff literature is empiric, with experts recommending the study of handoffs within an established framework. Methods. A prospective, single‐institution study was conducted evaluating the process of handoffs for the care of surgical patients in the context of a published communication framework. Evaluation tools for the source, receiver, and observer were developed to identify factors impacting the handoff process, and inter‐rater correlations were assessed. Data analysis was generated with Pearson/Spearman correlations and multivariate linear regressions. Rater consistency was assessed with intraclass correlations. Results. A total of 126 handoffs were observed. Evaluations were completed by 1 observer (N = 126), 2 observers (N = 23), 2 receivers (N = 39), 1 receiver (N = 82), and 1 source (N = 78). An average (±standard deviation) service handoff included 9.2 (±4.6) patients, lasted 9.1 (±5.4) minutes, and had 4.7 (±3.4) distractions recorded by the observer. The source and receiver(s) recognized distractions in >67% of handoffs, with the most common internal and external distractions being fatigue (60% of handoffs) and extraneous staff entering/exiting the room (31%), respectively. Teams with more patients spent less time per individual patient handoff (r = −0.298; P = .001). Statistically significant intraclass correlations (P ≤ .05) were moderate between observers (r ≥ 0.4) but not receivers (r < 0.4). Intraclass correlation values between different types of raters were inconsistent (P > .05). The quality of the handoff process was affected negatively by presence of active electronic devices (&bgr; = −0.565; P = .005), number of teaching discussions (&bgr; = −0.417; P = .048), and a sense of hierarchy between source and receiver (&bgr; = −0.309; P = .002). Conclusion. Studying the handoff process within an established framework highlights factors that impair communication. Internal and external distractions are common during handoffs and along with the working relationship between the source and receiver impact the quality of the handoff process. This information allows further study and targeted interventions of the handoff process to improve overall effectiveness and patient safety of the handoff.

Medical Student Communication Skills and Specialty Choice

ObjectiveThe aim of this study was to determine if communication skills differ for medical students entering person or technique-oriented specialties.MethodsCommunication ratings by clerkship preceptors on an institutionally required end of clerkship medical student performance evaluation (SPE) form were compiled for 2011/2012 academic year (Class of 2013). M3 clerkships and the Class of 2013 match appointments were categorized as person or technique-oriented clerkships/specialties. Mean differences in SPE communication scores were determined by analyses of variance (ANOVA) and independent t tests. Score associations were determined by Pearson correlations. Inter-item reliability was reported with Cronbach alpha.ResultsThe Class of 2013 match appointments were as follows: person-oriented (N = 91) and technique-oriented (N = 91) residency specialties. There was no significant difference in mean communication scores for medical students who entered person-oriented (mean 7.8, SD 0.4) versus technique-oriented (mean 7.9, SD 0.4) specialties (p = 0.258) or for person-oriented clerkship (mean 7.8, SD 0.4) versus technique-oriented clerkship (mean 7.9, SD 0.6) ratings for medical students who matched into person-oriented specialties (p = 0.124). Medical students who matched into technique-oriented specialties (mean 8.1, SD 0.5) received significantly higher (p = 0.001) communication ratings as compared with those matching into person-oriented specialties (mean 7.8, SD 0.5) from technique-oriented clerkships.ConclusionsCommunication with patients and families is a complex constellation of specific abilities that appear to be influenced by the rater’s specialty. Further study is needed to determine if technique-oriented specialties communication skill rating criteria differ from those used by raters from person-oriented specialties.

Individual learning plans foster self-directed learning skills and contribute to improved educational outcomes in the surgery clerkship

BACKGROUND The Individual Learning Plan (ILP) is a newly implemented curricular element designed to foster self-directed learning (SDL) skills among medical students during our surgery clerkship. Our aim was to determine the impact of ILPs on educational outcomes and acquisition of SDL skills. METHODS Students were surveyed regarding the educational value of ILPs, their acquisition of SDL skills, and the impact of the surgery clerkship on fostering these skills. Mean scores for the NBME surgery subject exam (SSE) were compared before and after implementation of the ILP requirement. RESULTS Students perceived the ILP requirement as having strong educational value. Mean scores for the SSE increased significantly in the year following ILP implementation (74.9 vs 76.6; p = .042; d = 0.21). Students reported successful acquisition and frequent utilization of SDL skills. CONCLUSIONS SDL exercises, such as the ILP requirement, lead to improved education outcomes while fostering the acquisition of SDL skills.

Team-based learning on a third-year pediatric clerkship improves NBME subject exam blood disorder scores

Purpose At our institution, speculation amongst medical students and faculty exists as to whether team-based learning (TBL) can improve scores on high-stakes examinations over traditional didactic lectures. Faculty with experience using TBL developed and piloted a required TBL blood disorders (BD) module for third-year medical students on their pediatric clerkship. The purpose of this study is to analyze the BD scores from the NBME subject exams before and after the introduction of the module. Methods We analyzed institutional and national item difficulties for BD items from the NBME pediatrics content area item analysis reports from 2011 to 2014 before (pre) and after (post) the pilot (October 2012). Total scores of 590 NBME subject examination students from examinee performance profiles were analyzed pre/post. t-Tests and Cohens d effect sizes were used to analyze item difficulties for institutional versus national scores and pre/post comparisons of item difficulties and total scores. Results BD scores for our institution were 0.65 (±0.19) compared to 0.62 (±0.15) nationally (P=0.346; Cohens d=0.15). The average of post-consecutive BD scores for our students was 0.70(±0.21) compared to examinees nationally [0.64 (±0.15)] with a significant mean difference (P=0.031; Cohens d=0.43). The difference in our institutions pre [0.65 (±0.19)] and post [0.70 (±0.21)] BD scores trended higher (P=0.391; Cohens d=0.27). Institutional BD scores were higher than national BD scores for both pre and post, with an effect size that tripled from pre to post scores. Institutional BD scores increased after the use of the TBL module, while overall exam scores remained steadily above national norms. Conclusions Institutional BD scores were higher than national BD scores for both pre and post, with an effect size that tripled from pre to post scores. Institutional BD scores increased after the use of the TBL module, while overall exam scores remained steadily above national norms.