J. Jon Veloski

The predictive validity of three versions of the MCAT in relation to performance in medical school, residency, and licensing examinations: a longitudinal study of 36 classes of Jefferson Medical College.

Purpose The Medical College Admission Test (MCAT) has undergone several revisions for content and validity since its inception. With another comprehensive review pending, this study examines changes in the predictive validity of the MCATs three recent versions. Method Study participants were 7,859 matriculants in 36 classes entering Jefferson Medical College between 1970 and 2005; 1,728 took the pre-1978 version of the MCAT; 3,032 took the 1978–1991 version, and 3,099 took the post-1991 version. MCAT subtest scores were the predictors, and performance in medical school, attrition, scores on the medical licensing examinations, and ratings of clinical competence in the first year of residency were the criterion measures. Results No significant improvement in validity coefficients was observed for performance in medical school or residency. Validity coefficients for all three versions of the MCAT in predicting Part I/Step 1 remained stable (in the mid-0.40s, P < .01). A systematic decline was observed in the validity coefficients of the MCAT versions in predicting Part II/Step 2. It started at 0.47 for the pre-1978 version, decreased to between 0.42 and 0.40 for the 1978–1991 versions, and to 0.37 for the post-1991 version. Validity coefficients for the MCAT versions in predicting Part III/Step 3 remained near 0.30. These were generally larger for women than men. Conclusions Although the findings support the short- and long-term predictive validity of the MCAT, opportunities to strengthen it remain. Subsequent revisions should increase the tests ability to predict performance on United States Medical Licensing Examination Step 2 and must minimize the differential validity for gender.

A comparison of medical students’ self-reported empathy with simulated patients’ assessments of the students’ empathy

Background: Empathy is necessary for communication between patients and physicians to achieve optimal clinical outcomes. Aim: To examine associations between Simulated Patients’ (SPs) assessment of medical students’ empathy and the students’ self-reported empathy. Methods: A total of 248 third-year medical students completed the Jefferson Scale of Physician Empathy (JSPE). SPs completed the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE), and a global rating of empathy in 10 objective clinical skills examination encounters during a comprehensive end of third-year clinical skills examination. Results: High correlation was found between the scores on the JSPPPE and the global ratings of empathy completed by the SPs (r = 0.87, p < 0.01). A moderate but statistically significant correlation was observed between scores of the JSPE and the JSPPPE (r = 0.19, p < 0.05). Significant differences were observed on the JSPE and global ratings of empathy among top, middle and low scorers on the JSPPPE in the expected direction. Conclusions: While significant associations exist between students’ self-reported scores on the JSPE and SPs’ evaluations of students’ empathy, the associations are not large enough to conclude that the two evaluations are redundant.

Standardized patient assessment of medical student empathy: ethnicity and gender effects in a multi-institutional study.

Purpose To examine, primarily, the effects of ethnicity and gender, which could introduce bias into scoring, on standardized patient (SP) assessments of medical students and, secondarily, to examine medical students’ self-reported empathy for ethnicity and gender effects so as to compare self-perception with the perceptions of SPs. Method Participants were 577 students from four medical schools in 2012: 373 (65%) were white, 79 (14%) black/African American, and 125 (22%) Asian/Pacific Islander. These students were assessed by 84 SPs: 62 (74%) were white and 22 (26%) were black/African American. SPs completed the Jefferson Scale of Patient Perceptions of Physician Empathy (JSPPPE) and the Global Ratings of Empathy tool. Students completed the Jefferson Scale of Empathy and two Interpersonal Reactivity Index subscales. The investigators used 2,882 student–SP encounters in their analyses. Results Analyses of SPs’ assessments of students’ empathy indicated significant interaction effects of gender and ethnicity. Female students, regardless of ethnicity, obtained significantly higher mean JSPPPE scores than men. Female black/African American, female white, and female Asian/Pacific Islander students scored significantly higher on the JSPPPE than their respective male counterparts. Male black/African American students obtained the lowest SP assessment scores of empathy regardless of SP ethnicity. Black/African American students obtained the highest mean scores on self-reported empathy. Conclusions The significant interaction effects of ethnicity and gender in clinical encounters, plus the inconsistencies observed between SPs’ assessments of students’ empathy and students’ self-reported empathy, raise questions about possible ethnicity and gender biases in the SPs’ assessments of medical students’ clinical skills.

Development and Evaluation of a Program to Strengthen First Year Residents’ Proficiency in Leading End-of-Life Discussions

Purpose: Multiple interventions have been developed to teach and improve internal medicine residents’ end-of-life communication skills, but have not been easily adaptable to other institutions. The purpose of this study was to develop and evaluate a program to enhance physicians’ end-of-life communication with families of dying patients using a format that could be incorporated into an existing curriculum for first-year internal medicine residents. Methods: An end-of-life educational program was developed and evaluated in the context of educating first-year residents at an urban academic medical center during the 2008-2009 academic year. The program consisted of three sessions including an interactive workshop flanked by pre- and post-workshop evaluations in simulated encounter and clinical vignette formats. Simulated encounters were recorded on video and residents’ performances were rated by two independent observers using a 23 point checklist. Results: Complete data were available for 24 (73%) of 33 residents who participated in the program. The residents’ checklist scores increased significantly from a mean of 48.1 at baseline to 73.9 at follow-up. The increase in the scores on the clinical vignettes was also statistically significant, but of lesser magnitude. Conclusions: A short, focused intervention can have significant impact on residents’ communication skills in the setting of an end-of-life objective structured clinical examination (OSCE).

Can empathy, other personality attributes, and level of positive social influence in medical school identify potential leaders in medicine?

Purpose To test the hypotheses that medical students recognized by peers as the most positive social influencers would score (1) high on measures of engaging personality attributes that are conducive to relationship building (empathy, sociability, activity, self-esteem), and (2) low on disengaging personality attributes that are detrimental to interpersonal relationships (loneliness, neuroticism, aggression-hostility, impulsive sensation seeking). Method The study included 666 Jefferson Medical College students who graduated in 2011–2013. Students used a peer nomination instrument to identify classmates who had a positive influence on their professional and personal development. At matriculation, these students had completed a survey that included the Jefferson Scale of Empathy and Zuckerman–Kuhlman Personality Questionnaire short form and abridged versions of the Rosenberg Self-Esteem Scale and UCLA Loneliness Scale. In multivariate analyses of variance, the method of contrasted groups was used to compare the personality attributes of students nominated most frequently by their peers as positive influencers (top influencers [top 25% in their class distribution], n = 176) with those of students nominated least frequently (bottom influencers [bottom 25%], n = 171). Results The top influencers scored significantly higher on empathy, sociability, and activity and significantly lower on loneliness compared with the bottom influencers. However, the effect size estimates of the differences were moderate at best. Conclusions The research hypotheses were partially confirmed. Positive social influencers appear to possess personality attributes conducive to relationship building, which is an important feature of effective leadership. The findings have implications for identifying and training potential leaders in medicine.

Measuring Medical Students' Orientation Toward Lifelong Learning: A Psychometric Evaluation

Background The principle of lifelong learning is pervasive in regulations governing medical education and medical practice; yet, tools to measure lifelong learning are lagging in development. This study evaluates the Jefferson Scale of Physician Lifelong Learning (JeffSPLL) adapted for administration to medical students. Method The Jefferson Scale of Physician Lifelong Learning–Medical Students (JeffSPLL-MS) was administered to 732 medical students in four classes. Factor analysis and t tests were performed to investigate its construct validity. Results Maximum likelihood factor analysis identified a three-factor solution explaining 46% of total variance. Mean scores of clinical and preclinical students were compared; clinical students scored significantly higher in orientation toward lifelong learning (P < .001). Conclusions The JeffSPLL-MS presents findings consistent with key concepts of lifelong learning. Results from use of the JeffSPLL-MS may reliably inform curriculum design and education policy decisions that shape the careers of physicians.

AM last page. The Jefferson Longitudinal Study of medical education.

AM Last Page: The Jefferson Longitudinal Study of Medical Education Joseph S. Gonnella;Mohammadreza Hojat;Jon Veloski; Academic Medicine

Evaluation of the use of patient-focused simulation for student assessment in a surgery clerkship.

BACKGROUND The purpose of this study was to evaluate the use of simulated patients in conjunction with anatomic and tissue task-training models to assess skills. METHODS Faculty reviewed the objectives of the clerkship to identify skills to be acquired. Three cases were developed related to rectal examination, suturing, and inserting intravenous lines and nasogastric tubes. Student scores were based on their ability to gather data from simulated patients and perform procedures on simulation models. RESULTS A total of 670 students were assessed between 2006 and 2009. Alpha reliability coefficients were .97 for Communication/Interpersonal Skills, .71 for Procedures, and .58 for Data Gathering. Students receiving low ratings from faculty in the clerkship had significantly (P < .001) lower simulation scores. There were significant (P < .001) relationships between scores and grades in other clerkships. CONCLUSIONS The combination of simulated patients and simulation models yielded reliable scores for procedural and interpersonal skills, and evidence of validity related to clinical ratings.

Equivalence of students' scores on timed and untimed anatomy practical examinations

Untimed examinations are popular with students because there is a perception that first impressions may be incorrect, and that difficult questions require more time for reflection. In this report, we tested the hypothesis that timed anatomy practical examinations are inherently more difficult than untimed examinations. Students in the Doctor of Physical Therapy program at Thomas Jefferson University were assessed on their understanding of anatomic relationships using multiple‐choice questions. For the class of 2012 (n = 46), students were allowed to circulate freely among 40 testing stations during the 40‐minute testing session. For the class of 2013 (n = 46), students were required to move sequentially through the 40 testing stations (one minute per item). Students in both years were given three practical examinations covering the back/upper limb, lower limb, and trunk. An identical set of questions was used for both groups of students (untimed and timed examinations). Our results indicate that there is no significant difference between student performance on untimed and timed examinations (final percent scores of 87.3 and 88.9, respectively). This result also held true for students in the top and bottom 20th percentiles of the class. Moreover, time limits did not lead to errors on even the most difficult, higher‐order questions (i.e., items with P‐values < 0.70). Thus, limiting time at testing stations during an anatomy practical examination does not adversely affect student performance. Anat Sci Educ 6: 281–285.

Medical School Anatomy and Pathology Workshops for High School Students Enhance Learning and Provide Inspiration for Careers in Medicine

“Anatomy and Pathology Workshop” is a cadaver-based outreach program that models medical education to large groups of high school students. This study was designed to evaluate the impact of this program on students’ knowledge of anatomy and interest in biomedical science. A total of 144 high school students participated in the workshop in 2015. Preworkshop and postworkshop assessments were administered to assess students’ learning. A postworkshop survey was conducted to solicit students’ reflections and feedback. It was found that student performance in the postworkshop examination (mean 78%) had significantly improved when compared to the performance in the preexamination (mean 54%), indicating that this program enhances learning. Students were also inspired to consider opportunities in medicine and allied health professions—97% indicated that they had a better understanding of medical education; 95% agreed that they had better understanding of the human body; 84% thought anatomy was interesting and exciting; and 62% of the students indicated that they looked forward to studying medicine or another health profession. Students rated the instructors highly—95% agreed that the instructors were professional and served as role models. Medical/graduate student instructors were also highly regarded by the high school students—96% thought it was valuable to have student instructors and 94% thought that student instructors were caring and enthusiastic about teaching. In summary, this study demonstrates that outreach programs provided by medical schools help young adults during their formative years by modeling professionalism, providing role models, enhancing learning, and encouraging many to consider opportunities in the health professions.