Richard Arnett

Is a picture worth a thousand words: an analysis of the difficulty and discrimination parameters of illustrated vs. text-alone vignettes in histology multiple choice questions

BackgroundAdvances in cognitive load theory have led to greater understanding of how we process verbal and visual material during learning, but the evidence base with regard to the use of images within written assessments is still sparse. This study examines whether the inclusion of images within the stimulus format of multiple choice questions (MCQs) has a predictable or consistent influence on psychometric item properties, such as difficulty or discrimination.MethodsItem analysis data from three consecutive years of histology multiple choice examinations were included in this study. All items were reviewed and categorised according to whether their stem, or stimulus format, was purely textual or included an associated image.ResultsA total of 195 MCQs were identified for inclusion and analysed using classical test theory; 95 used text alone and 100 included an image within the question stem. The number of students per examination ranged from 277 to 347, with a total of 60,850 student-question interactions. We initially examined whether the inclusion of an image within the item stem altered the item difficulty using Mann–Whitney U. The median item difficulty for images with purely textual stems was 0.77, while that for items incorporating an appropriate image was 0.80; this difference was not significant (0.77 vs. 0.80; p = 0.862, Mann–Whitney-U = 4818.5). Mean values showed that the Item Discrimination Index appeared unaffected by the inclusion of an image within the stem, and Item point biserial correlation also showed no difference in means between these two groups (Independent samples t-test; 2-tailed).ConclusionWe demonstrate that the addition of illustrations within undergraduate histology Multiple Choice Question stems has no overall influence on item difficulty, or measures of item discrimination. We conclude that the use of images in this context is statistically uncritical, and suggest that their inclusion within item stems should be based upon the principles of constructive alignment. However, further research with respect to the effect of images within item stems on cognitive processing, particularly with regard to image complexity or type, would enable the development of more informed guidelines for their use.

Comparison of performance in a four year graduate entry medical programme and a traditional five/six year programme

BackgroundIn 2006 the Royal College of Surgeons in Ireland, (RCSI), introduced the first four year Graduate Entry Programme (GEP) in medicine in Ireland in line with national policy to broaden access to medical education. One concern considered at the time, was whether the GEP students could be trained to the same standard as their undergraduate Direct Entry Programme (DEP, five/six year duration) counterparts in the shorter time frame. Since students from both cohorts undertake the same examinations in the final two years, it is possible to directly compare GEP vs DEP outcomes. The primary aim of the current study was to analyse the comparative performance of GEP and DEP students undergoing these examinations between 2008 and 2013.MethodsScores from five assessments performed during the final two years were transformed to z scores for each student and 4 scores for the penultimate year were summed to create a unit weighted composite score. The resultant scores for each of the two years were used to assess the comparative performance of GEP vs DEP cohorts and to perform sub-cohort analyses of GEP outcomes.ResultsIn all cohorts/years examined, evidence demonstrated significantly better assessment outcomes for the GEP group for the final two years’ examinations as compared with the DEP group. In all but one cohort examined, this advantage was retained when nationality factors were excluded. Further analyses showed no difference in outcomes between GEP students having science vs. non-science backgrounds and/or between those from EU vs non-EU backgrounds. Finally, data suggested weak correlations between total composite scores and entry scores in American (r = 0.15) and Australian (r = 0.08) medical school admissions tests.ConclusionsWe have shown for the first time in Ireland, that graduate-entry students perform at least as well, or even better, than a corresponding undergraduate-entry group. Moreover, having a scientific background on entry to the GEP confers no advantage in final assessments. These data provide evidence of the viability of the graduate entry route into medical education in Ireland.

Graduate medical education in Ireland: a profile of the first cohort of students

BackgroundThe first graduate-entry programmes to Irish medicine were established at the Royal College of Surgeons in Ireland (RCSI) and the University of Limerick (UL) in 2007. There were over 400 applications across both institutions and 306 people sat a special aptitude test (GAMSAT) in Ireland in 2007. Ultimately, 61 Irish/EU students were admitted to one or other programme.Aims and methodsWe describe the demographic profile, academic background and aggregated GAMSAT performance of 306 people who sat GAMSAT in Ireland in 2007 and of the 61 people admitted to the RCSI/UL programmes.ResultsWhile more females than males sat GAMSAT, slightly more males were admitted. Over 90% of those admitted were aged in their 20s, almost 20% had a higher degree and they came from a wide range of academic backgrounds.ConclusionsAmong others, this information should be of interest to prospective students and to government policy makers.

Changing Objective Structured Clinical Examinations Stations at Lunchtime During All Day Postgraduate Surgery Examinations Improves Examiner Morale and Stress

BACKGROUND Human factors are important causes of error, but little is known about their possible effect during objective structured clinical examinations (OSCE). We have previously identified stress and pressure in OSCE examiners in the postgraduate intercollegiate Membership of the Royal College of Surgeons (MRCS) examination. After modifying examination delivery by changing OSCE stations at lunchtime with no demonstrable effect on candidate outcome, we resurveyed examiners to ascertain whether examiner experience was improved. METHOD Examiners (n = 180) from all 4 surgical colleges in the United Kingdom and Ireland were invited to complete the previously validated human factors questionnaire used in 2014. Aggregated scores for each of 4 previously identified factors were compared with the previous data. Unit-weighted z-scores and nonparametric Kruskal-Wallis methods were used to test the hypothesis that there was no difference among the median factor z-scores for each college. Individual Mann-Whitney-Wilcoxon tests (with appropriate Bonferonni corrections) were used to determine any differences between factors and the respective colleges. RESULTS 141 Completed questionnaires were evaluated (78% response rate) and compared with 108 responses (90%) from the original study. Analysis was based on 26 items common to both studies. In 2014, the college with the highest candidate numbers (England) was significantly different in 1 factor (stress and pressure), compared with Edinburgh (Mann-Whitney-Wilcoxon: W = 1524, p < 0.001) and Glasgow colleges (Mann-Whitney-Wilcoxon: W = 104, p = 0.004). No differences were found among colleges in the same factor in 2016, Kruskall-Wallis: (χ2 (3) = 1.73, p = 0.63). Analysis of responses found inconsistency among examiners regarding mistakes or omissions made when candidates were performing well. CONCLUSION After making changes to OSCE delivery, factor scores relating to examiner stress and pressure are now improved and consistent across the surgical colleges. Stress and pressure can occur in OSCE examiners and examination delivery should ideally minimize these issues, thereby improving morale is also likely to benefit candidates.

Push and stay factors affecting Irish medical student migration intentions.

BackgroundA significant proportion of medical students in Ireland have demonstrated strong intentions to migrate following their graduation. Factors influencing these intentions are poorly understood. Our study aimed to investigate ‘push’ and ‘stay’ factors that may influence the migration plans of medical students.MethodsCross-sectional survey of 2273 medical students in Ireland using a mixed methods approach. Survey was completed by 2273 medical students, of which 263 provided free text data that are analysed in this paper.ResultsIdentified push factors include negative perception regarding career opportunities, working conditions and lifestyle, while family was the only identified strong stay factor. Qualitative analysis of free text responses revealed themes that included training, career, personal and financial factors.ConclusionThis study has provided insight into the factors that influence medical student migration intentions. There is a strong need for all stakeholders to collectively discuss and engage in possible solutions.

Investigating the Irish brain drain: factors influencing migration intentions among medical students

Background Ireland has the highest level of medical emigration in Europe with an increasing demand for physicians worldwide [1,2]. This has received considerable public and political interest. However, few studies have described the migration intentions of medical students at the undergraduate level [3]. Our study aimed to describe the migration intentions of Irish medical students by nationality and identify unique factors, “push factors”, that influence their decisions.