Karen M. Stegers-Jager

Motivation, learning strategies, participation and medical school performance

Medical Education 2012: 46:678–688

Ethnic disparities in undergraduate pre-clinical and clinical performance

Medical Education 2012: 46:575–585

Academic and non-academic selection criteria in predicting medical school performance

Background: A two-step selection procedure, consisting of a non-academic and an academic step, was recently shown to select students with a 2.6 times lower risk of early dropout and a higher clerkship Grade Point Average (GPA) than lottery-admitted controls. Aim: To determine the relative contribution of the non-academic and academic steps to differences found in student performance. Method: Lottery-admitted students (n = 653) and three groups of selection procedure participants were compared on early dropout rate and clerkship GPA: (1) all participants (n = 1676), (2) participants who passed step 1, and (3) participants who passed step 2. Results: Selection procedure participation resulted in a 4.4% lower dropout rate than lottery admission and this difference increased to 5.2% after step 1 and to 8.7% after step 2. Clerkship GPA was significantly higher for participants who passed step 1 than for their lottery-admitted controls. This difference remained significant after the rejection of students on academic criteria in step 2. Conclusion: The lower dropout rate of selected students is related to both self-selection of participants before the start of the selection procedure and the academic part of the selection procedure. The higher clerkship GPA of selected students is almost exclusively related to the non-academic selection criteria.

Academic dismissal policy for medical students: Effect on study progress and help-seeking behaviour

Medical Education 2011: 45: 987–994

The effect of a short integrated study skills programme for first-year medical students at risk of failure: A randomised controlled trial

Background: There is a need for outcome-based studies on strategies for supporting at-risk medical students that use long-term follow-up and contemporaneous controls. Aim: To measure the effect of a short integrated study skills programme (SSP) on the study progress of at-risk medical students. Methods: First-year students identified as at-risk of academic failure at 7 months after enrolment were invited to participate in the randomised controlled trial. Participants were randomly assigned to the SSP group or to a control group receiving standard academic support. Effects of SSP were measured on the short (passed first exam after intervention), medium (obtained enough credits to proceed to second year) and long term (completed first-year curriculum within 2 years). Results: SSP participants (n = 43) more often passed the first exam after the intervention than controls (n = 41; 30% versus 12%; X2(1) = 4.06, p < 0.005, effect size = 0.22), in particular those who had previously passed at least one exam. No medium or long-term effect was found. Participants who had attended four or five SSP sessions outperformed those who had attended fewer sessions on all outcome measures. Conclusion: A short, integrated SSP benefited some, but not all students. Our advice is to focus support efforts on at-risk students who have demonstrated commitment and academic potential.

Predicting performance: Relative importance of students' background and past performance

Despite evidence for the predictive value of both pre‐admission characteristics and past performance at medical school, their relative contribution to predicting medical school performance has not been thoroughly investigated.

Ethnic and social disparities in performance on medical school selection criteria.

Medical schools in Western societies seek measures to increase the diversity of their student bodies with respect to ethnicity and social background. Currently, little is known about the effects of different selection procedures on student diversity.

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

Situational Judgment Tests (SJTs) are increasingly used for medical school selection. Scoring an SJT is more complicated than scoring a knowledge test, because there are no objectively correct answers. The scoring method of an SJT may influence the construct and concurrent validity and the adverse impact with respect to non-traditional students. Previous research has compared only a small number of scoring methods and has not studied the effect of scoring method on internal consistency reliability. This study compared 28 different scoring methods for a rating SJT on internal consistency reliability, adverse impact and correlation with personality. The scoring methods varied on four aspects: the way of controlling for systematic error, and the type of reference group, distance and central tendency statistic. All scoring methods were applied to a previously validated integrity-based SJT, administered to 931 medical school applicants. Internal consistency reliability varied between .33 and .73, which is likely explained by the dependence of coefficient alpha on the total score variance. All scoring methods led to significantly higher scores for the ethnic majority than for the non-Western minorities, with effect sizes ranging from 0.48 to 0.66. Eighteen scoring methods showed a significant small positive correlation with agreeableness. Four scoring methods showed a significant small positive correlation with conscientiousness. The way of controlling for systematic error was the most influential scoring method aspect. These results suggest that the increased use of SJTs for selection into medical school must be accompanied by a thorough examination of the scoring method to be used.

Dealing with diversity in medical education.

A second challenge refers to the underperformance of ethnic minority medical students. Although research from numerous medical schools has shown that students from ethnic minorities underperform compared with those from the ethnic majority, little is known about why this underperformance occurs and what might be done about it. 2

Lessons learned from 15 years of non-grades-based selection for medical school

Thirty years ago, it was suggested in the Edinburgh Declaration that medical school applicants should be selected not only on academic, but also on non‐academic, attributes. The main rationale behind extending medical school selection procedures with the evaluation of (non‐academic) personal qualities is that this will lead to the selection of students who will perform better as a doctor than those who are selected on the basis of academic measures only. A second rationale is the expectation that this will lead to a representative health workforce as a result of reduced adverse impact. The aims of this paper are (i) to describe what can be learned about the use of selection criteria other than grades from over 15 years of Dutch experience and (ii) to summarise current knowledge on the issue of adverse impact in relation to non‐grades‐based selection.