Annette Mercer

Admission selection criteria as predictors of outcomes in an undergraduate medical course: A prospective study

Background: In 1998, a new selection process which utilised an aptitude test and an interview in addition to previous academic achievement was introduced into an Australian undergraduate medical course. Aims: To test the outcomes of the selection criteria over an 11-year period. Methods: 1174 students who entered the course from secondary school and who enrolled in the MBBS from 1999 through 2009 were studied in relation to specific course outcomes. Regression analyses using entry scores, sex and age as independent variables were tested for their relative value in predicting subsequent academic performance in the 6-year course. The main outcome measures were assessed by weighted average mark for each academic year level; together with results in specific units, defined as either ‘knowledge’-based or ‘clinically’ based. Results: Previous academic performance and female sex were the major independent positive predictors of performance in the course. The interview score showed positive predictive power during the latter years of the course and in a range of ‘clinically’ based units. This relationship was mediated predominantly by the score for communication skills. Conclusions: Results support combining prior academic achievement with the assessment of communication skills in a structured interview as selection criteria into this undergraduate medical course.

Rediscovering nursing: A study of overseas nurses working in Western Australia

The shortage of nurses worldwide has taken its toll on the Australian healthcare system and, as a result, there is an increased migration of overseas-qualified nurses, some of them with a non-English-speaking background. Despite Australias regulations that assess the eligibility for nursing registration, many migrant nurses who have been successful in gaining their nursing license feel only partially prepared to work. This article presents the findings of a study, based on Husserlian phenomenology, that describes the work experience of 13 female nurses who were working in Western Australia, Australia. The participants, who could recognize the core components of nursing, were taken aback by the way that nursing is practised in Western Australia. The major differences that they encountered were related to clinical skills, holistic care, the work dynamic with doctors and patients, and the overall societal status of the nursing profession. As a result, they had to adjust their practice to conform to the new work environment. In this study, the participants elaborated on some positive and some not-so-positive aspects of their experiences in their endeavor to integrate into the Western Australian metropolitan hospital setting.

Predicting academic outcomes in an Australian graduate entry medical programme

BackgroundPredictive validity studies for selection criteria into graduate entry courses in Australia have been inconsistent in their outcomes. One of the reasons for this inconsistency may have been failure to have adequately considered background disciplines of the graduates as well as other potential confounding socio-demographic variables that may influence academic performance.MethodsGraduate entrants into the MBBS at The University of Western Australia between 2005 and 2012 were studied (N = 421). They undertook a 6-month bridging course, before joining the undergraduate-entry students for Years 3 through 6 of the medical course. Students were selected using their undergraduate Grade Point Average (GPA), Graduate Australian Medical School Admissions Test scores (GAMSAT) and a score from a standardised interview. Students could apply from any background discipline and could also be selected through an alternative rural entry pathway again utilising these 3 entry scores. Entry scores, together with age, gender, discipline background, rural entry status and a socioeconomic indicator were entered into linear regression models to determine the relative influence of each predictor on subsequent academic performance in the course.ResultsBackground discipline, age, gender and selection through the rural pathway were variously related to each of the 3 entry criteria. Their subsequent inclusion in linear regression models identified GPA at entry, being from a health/allied health background and total GAMSAT score as consistent independent predictors of stronger academic performance as measured by the weighted average mark for the core units completed throughout the course. The Interview score only weakly predicted performance later in the course and mainly in clinically-based units. The association of total GAMSAT score with academic performance was predominantly dictated by the score in GAMSAT Section 3 (Reasoning in the biological and physical sciences) with Section 1 (Reasoning in the humanities and social sciences) and Section 2 (Written communication) also contributing either later or early in the course respectively. Being from a more disadvantaged socioeconomic background predicted weaker academic performance early in the course. Being an older student at entry or from a humanities background also predicted weaker academic performance.ConclusionsThis study confirms that both GPA at entry and the GAMSAT score together predict outcomes not only in the early stages of a graduate-entry medical programme but throughout the course. It also indicates that a comprehensive evaluation of the predictive validity of GAMSAT scores, interview scores and undergraduate academic performance as valid selection processes for graduate entry into medical school needs to simultaneously consider the potential confounding influence of graduate discipline background and other socio-demographic factors on both the initial selection parameters themselves as well as subsequent academic performance.

Potential influence of selection criteria on the demographic composition of students in an Australian medical school

BackgroundPrior to 1999 students entering our MBBS course were selected on academic performance alone. We have now evaluated the impact on the demographics of subsequent cohorts of our standard entry students (those entering directly from high school) of the addition to the selection process of an aptitude test (UMAT), a highly structured interview and a rural incentive program.MethodsStudents entering from 1985 to 1998, selected on academic performance alone (N = 1402), were compared to those from 1999 to 2011, selected on the basis of a combination of academic performance, interview score, and UMAT score together with the progressive introduction of a rural special entry pathway (N = 1437).ResultsMales decreased from 57% to 45% of the cohort, students of NE or SE Asian origin decreased from 30% to 13%, students born in Oceania increased from 52% to 69%, students of rural origin from 5% to 21% and those from independent high schools from 56% to 66%. The proportion of students from high schools with relative socio-educational disadvantage remained unchanged at approximately 10%. The changes reflect in part increasing numbers of female and independent high school applicants and the increasing rural quota. However, they were also associated with higher interview scores in females vs males and lower interview scores in those of NE and SE Asian origin compared to those born in Oceania or the UK. Total UMAT scores were unrelated to gender or region of origin.ConclusionsThe revised selection processes had no impact on student representation from schools with relative socio-educational disadvantage. However, the introduction of special entry quotas for students of rural origin and a structured interview, but not an aptitude test, were associated with a change in gender balance and ethnicity of students in an Australian undergraduate MBBS course.

Selection of medical students: an Australian perspective

Recent articles appearing in this journal indicate that the selection of students into medical courses remains a major issue for medical schools and a controversial one. 1–4 Admissions policies attempt to achieve a number of objectives, not all of which can easily be reconciled. These include: • The selection of medical students with the cognitive skills and other qualities to enable them to cope with the demands of a lengthy and intellectually challenging period of training, both before and after completion of the medical course • The selection of applicants with the potential to become ‘good doctors’ who possess the personal and professional qualities which enable them to sustain a career in medicine without experiencing undue stress leading to burnout • The development of selection procedures which are equitable, consistent and transparent. This last requirement is frequently complicated by the need to exercise a degree of affirmative action in favour of disadvantaged minorities, or to meet some other perceived community need, for example to redress the shortage of doctors prepared to work in rural or remote areas. Within Australasia and in other parts of the world, major changes in the methods of selection into medical courses have occurred over the last 10–15 years. There has been widespread acceptance that entry to medical school should not be based solely on academic criteria. One reason for this change has been the introduction of the managed care model of health-care delivery. 5 This developed in the 1990s and the emphasis on seeing patients quickly created the dilemma of upholding the values of the profession while responding to the needs of institutions and businesses. This places new challenges on medical schools and teaching hospitals since this shift, brought about by market forces, will require doctors of the future to possess a range of new skills, and it has been proposed that the selection process should become more thoughtful and rich, not more simplistic. 5

Socio-economic predictors of performance in the Undergraduate Medicine and Health Sciences Admission Test (UMAT)

BackgroundEntry from secondary school to Australian and New Zealand undergraduate medical schools has since the late 1990’s increasingly relied on the Undergraduate Medicine and Health Sciences Admission Test (UMAT) as one of the selection factors. The UMAT consists of 3 sections – logical reasoning and problem solving (UMAT-1), understanding people (UMAT-2) and non-verbal reasoning (UMAT-3). One of the goals of using this test has been to enhance equity in the selection of students with the anticipation of an increase in the socioeconomic diversity in student cohorts. However there has been limited assessment as to whether UMAT performance itself might be influenced by socioeconomic background.MethodsBetween 2000 and 2012, 158,909 UMAT assessments were completed. From these, 118,085 cases have been identified where an Australian candidate was sitting for the first time during that period. Predictors of the total UMAT score, UMAT-1, UMAT-2 and UMAT-3 scores were entered into regression models and included gender, age, school type, language used at home, deciles for the Index of Relative Socioeconomic Advantage and Disadvantage score, the Accessibility/Remoteness Index of Australia (ARIA), self-identification as being of Aboriginal or Torres Strait Islander origin (ATSI) and current Australian state or territory of abode.ResultsA lower UMAT score was predicted by living in an area of relatively higher social disadvantage and lower social advantage. Other socioeconomic indicators were consistent with this observation with lower scores in those who self-identified as being of ATSI origin and higher scores evident in those from fee-paying independent school backgrounds compared to government schools. Lower scores were seen with increasing age, female gender and speaking any language other than English at home. Divergent effects of rurality were observed, with increased scores for UMAT-1 and UMAT-2, but decreasing UMAT-3 scores with increasing ARIA score. Significant state-based differences largely reflected substantial socio-demographic differences across Australian states and territories.ConclusionsBetter performance by Australian candidates in the UMAT is linked to an increase in socio-economic advantage and reduced disadvantage.This observation provides a firm foundation for selection processes at medical schools in Australia that have incorporated affirmative action pathways to quarantine places for students from areas of socio-economic disadvantage.

Medical student selection criteria as predictors of intended rural practice following graduation

BackgroundRecruiting medical students from a rural background, together with offering them opportunities for prolonged immersion in rural clinical training environments, both lead to increased participation in the rural workforce after graduation. We have now assessed the extent to which medical students’ intentions to practice rurally may also be predicted by either medical school selection criteria and/or student socio-demographic profiles.MethodsThe study cohort included 538 secondary school-leaver entrants to The University of Western Australia Medical School from 2006 to 2011. On entry they completed a questionnaire indicating intention for either urban or rural practice following graduation. Selection factors (standardised interview score, percentile score from the Undergraduate Medicine and Health Sciences Admission Test (UMAT) and prior academic performance (Australian Tertiary Admissions Rank), together with socio-demographic factors (age, gender, decile for the Index of Relative Socioeconomic Advantage and Disadvantage (IRSAD) and an index of rurality) were examined in relation to intended rural or urban destination of practice.ResultsIn multivariate logistic regression, students from a rural background had a nearly 8-fold increase in the odds of intention to practice rurally after graduation compared to those from urban backgrounds (OR 7.84, 95% CI 4.10, 14.99, P < 0.001). Those intending to be generalists rather than specialists had a more than 4-fold increase in the odds of intention to practice rurally (OR 4.36, 95% CI 1.69, 11.22, P < 0.001). After controlling for these 2 factors, those with rural intent had significantly lower academic entry scores (P = 0.002) and marginally lower interview scores (P = 0.045). UMAT percentile scores were no different. Those intending to work in a rural location were also more likely to be female (OR 1.93, 95% CI 1.08, 3.48, P = 0.027), to come from the lower eight IRSAD deciles (OR 2.52, 95% CI 1.47, 4.32, P = 0.001) and to come from Government vs independent schools (OR 2.02, 95% CI 1.15, 3.55, P = 0.015).ConclusionsVery high academic scores generally required for medical school entry may have the unintended consequence of selecting fewer graduates interested in a rural practice destination. Increased efforts to recruit students from lower socioeconomic backgrounds may be beneficial in terms of an ultimate intended rural practice destination.

A seven-year retrospective analysis of students entering medicine via a Rural Student Recruitment program in Western Australia

OBJECTIVE This research aimed to evaluate the effectiveness of the Rural Student Recruitment (RSR) program. This program was an initiative to address the low number of rural students enrolled in medicine at the University of Western Australia. RSR identifies students throughout rural and remote areas of Australia interested in pursuing a career in medicine. The program provides support to these students through the various stages of the selection process and subsequently through the course. SETTING Medical School, the University of Western Australia. PARTICIPANTS Rural students enrolled in medicine at the University of Western Australia. RESULTS Of the 1591 participants in the RSR program, 11.6% have been successful in being offered a place. Participation was consistently higher for women, although men were proportionately more successful at gaining entry (14.5% versus 10.4%). It was found that the distribution of successful students in the RSR program generally reflects population density across rural Western Australia, with the majority of students coming from the South West, and the minority from the Pilbara and Kimberley. However, over the last three years (2006-2008) an increase in access from very remote regions was noted. This has been associated with a modification to the entry process that now includes a remoteness weighting for the secondary school attended. CONCLUSIONS The conclusion from this analysis was that the RSR program in concert with refinements in entry criteria has been effective in increasing the number of medical students from a rural background.

Practice effects in medical school entrance testing with the undergraduate medicine and health sciences admission test (UMAT)

BackgroundThe UMAT is widely used for selection into undergraduate medical and dental courses in Australia and New Zealand (NZ). It tests aptitudes thought to be especially relevant to medical studies and consists of 3 sections – logical reasoning and problem solving (UMAT-1), understanding people (UMAT-2) and non-verbal reasoning (UMAT-3). A substantial proportion of all candidates re-sit the UMAT. Re-sitting raises the issue as to what might be the precise magnitude and determinants of any practice effects on the UMAT and their implications for equity in subsequent selection processes.MethodsBetween 2000 and 2012, 158,909 UMAT assessments were completed. From these, 135,833 cases were identified where a candidate had sat once or more during that period with 117,505 cases (86.5%) having sat once, 14,739 having sat twice (10.9%), 2,752 thrice (2%) and 837, 4 or more times (0.6%). Subsequent analyses determined predictors of multiple re-sits as well as the magnitude and socio-demographic determinants of any practice effects.ResultsIncreased likelihood of re-sitting the UMAT twice or more was predicted by being male, of younger age, being from a non-English language speaking background and being from NZ and for Australian candidates, being urban rather than rurally based. For those who sat at least twice, the total UMAT score between a first and second attempt improved by 10.7 ± 0.2 percentiles, UMAT-1 by 8.3 ± 0.2 percentiles, UMAT-2 by 8.3 ± 0.2 percentiles and UMAT-3 by 7.7 ± 0.2 percentiles. An increase in total UMAT percentile score on re-testing was predicted by a lower initial score and being a candidate from NZ rather than from Australia while a decrease was related to increased length of time since initially sitting the test, older age and non-English language background.ConclusionsRe-sitting the UMAT augments performance in each of its components together with the total UMAT percentile score. Whether this increase represents just an improvement in performance or an improvement in understanding of the variables and therefore competence needs to be further defined. If only the former, then practice effects may be introducing inequity in student selection for medical or dental schools in Australia or NZ.

Admission interview scores are associated with clinical performance in an undergraduate physiotherapy course: an observational study

OBJECTIVE The purpose of this study was to determine if there is an association between admission interview score and subsequent academic and clinical performance, in a four-year undergraduate physiotherapy course. DESIGN Retrospective observational study. PARTICIPANTS 141 physiotherapy students enrolled in two entry year groups. OUTCOME MEASURES Individual student performance in all course units, practical examinations, clinical placements as well as year level and overall Grade Point Average. Predictor variables included admission interview scores, admission academic scores and demographic data (gender, age and entry level). RESULTS Interview score demonstrated a significant association with performance in three of six clinical placements through the course. This association was stronger than for any other admission criterion although effect sizes were small to moderate. Further, it was the only admission score to have a significant association with overall Clinical Grade Point Average for the two year groups analysed (r=0.322). By contrast, academic scores on entry showed significant associations with all year level Grade Point Averages except Year 4, the clinical year. CONCLUSIONS This is the first study to review the predictive validity of an admission interview for entry into a physiotherapy course in Australia. The results show that performance in this admission interview is associated with overall performance in clinical placements through the course, while academic admission scoring is not. These findings suggest that there is a role for both academic and non-academic selection processes for entry into physiotherapy.