Mary L. Zanetti

Using standardized patients to assess professionalism: a generalizability study

Background: Assessment of professionalism in undergraduate medical education is challenging. One approach that has not been well studied in this context is performance-based examinations. Purpose: This study sought to investigate the reliability of standardized patients’ scores of students’ professionalism in performance-based examinations. Methods: Twenty students were observed on 4 simulated cases involving professional challenges; 9 raters evaluated each encounter on 21 professionalism items. Correlational and multivariate generalizability (G) analyses were conducted. Results: G coefficients were .75, .53, and .68 for physicians, standardized patients (SPs), and lay raters, respectively. Composite G coefficient for all raters reached acceptable level of .86. Results indicated SP raters were more variable than other rater types in severity with which they rated students, although rank ordering of students was consistent among SPs. Conclusions: SPs’ ratings were less reliable and consistent than physician or lay ratings, although the SPs rank ordered students more consistently than the other rater types.

Tracking the Longitudinal Stability of Medical Students’ Perceptions Using the AAMC Graduation Questionnaire and Serial Evaluation Surveys

Background. This study examined the longitudinal stability of students’ perceptions by comparing ratings on similar survey items in three sequential evaluations: end-of-clerkship (EOC), AAMC graduation questionnaire (GQ), and a postgraduate survey (PGY1). Method. For the classes of 2000 and 2001, ratings were compiled from EOC evaluations and comparable items from the GQ. For both cohorts, selected GQ items were included in the PGY1 survey and these ratings were compiled. Matched responses from EOC versus GQ and PGY1 versus GQ were compared. Results. Proportions of “excellent” ratings were consistent across EOC and GQ surveys for all clerkships. Comparison of GQ and PGY1 ratings revealed significant differences in only seven of 31 items. Conclusion. Student perceptions as measured by GQ ratings are notably consistent across the clinical years and internship. This longitudinal stability supports the usefulness of the GQ in programmatic assessment and reinforces its value as a measure of student satisfaction.

Recent and Anticipated Changes in Postsecondary Admissions: A Survey of New England Colleges and Universities

In this article, the authors report the results of a survey of postsecondary institutions in New England regarding recent changes in their admissions processes and the factors that influenced those changes. Their results indicate that traditional admissions criteria continue to be weighted heavily in the admissions process and that recruiting and retaining underrepresented minority students remains a concern of many institutions. The implications of these findings for public, private, two-year, and four-year institutions are discussed.

A longitudinal study of multicultural curriculum in medical education

Objectives To evaluate impact a multicultural interclerkship had on students’ perception of knowledge, interview skills, and empathy towards serving culturally diverse populations and role student demographics played in learning. Methods Data extracted from students’ self-reported course evaluations and pre/post questionnaires during multiculturalism interclerkship across 11 academic years. Inquired students’ opinion about four areas: effectiveness, small group leaders, usefulness, and overall experience. Subscale and item ratings were compared using trend tests including multivariate analyses. Results During studied years, 883 students completed course evaluation with high overall mean rating of 3.08 (SD=0.45) and subscale mean scores ranging from 3.03 to 3.30. Trends in three of four subscales demonstrated clear uptrend (p<0.0001). Positive correlations between ratings of leaders and “usefulness” were observed (p<0.0001). Pre/post matched dataset (n=967) indicated majority of items (19/23) had statistically significant higher post interclerkship ratings compared to pre scores with nine of 19 having statistically significant magnitudes of change. Questionnaire had high overall reliability (Cronbach alpha=0.8), and item-to-group correlations ranged from 0.40 to 0.68 (p <0.0001). Conclusions By increasing students’ exposure and interaction with diverse patients, their knowledge, attitude, and skills were increased and expanded in positive manner. These findings might inform those who are interested in enhancing this important competence. This is especially true given increasing scrutiny this global topic is receiving within and across healthcare professions around the world.

Global Longitudinal Pathway: Has Medical Education Curriculum Influenced Medical Students’ Skills and Attitudes Toward Culturally Diverse Populations?

Background: The Pathway represents a longitudinal program for medical students, consisting of both domestic and international experiences with poor populations. A previous study reported no significant attitudinal changes toward the medically indigent between Pathway and non-Pathway students. Purpose: The purpose of this study was to investigate and differentiate the skills and attitudes of Pathway and non-Pathway students in working with culturally diverse populations by conducting quantitative and qualitative analyses. Methods: Selected items from a cultural assessment were analyzed using independent t-tests and a proportional analysis using approximation of the binomial distribution. In addition, a qualitative assessment of non-Pathway and Pathway students was conducted. Results: A statistically significant difference was found at the end of Years 2, 3, and 4 regarding student confidence ratings, and qualitative results had similar findings. Conclusions: Clear and distinct differences between the two studied groups were found indicating the root of this increased confidence may have developed due to exposure to the Pathway program.