David Powis

Assessment for selection for the health care professions and specialty training : consensus statement and recommendations from the Ottawa 2010 Conference

Assessment for selection in medicine and the health professions should follow the same quality assurance processes as in-course assessment. The literature on selection is limited and is not strongly theoretical or conceptual. For written testing, there is evidence of the predictive validity of Medical College Admission Test (MCAT) for medical school and licensing examination performance. There is also evidence for the predictive validity of grade point average, particularly in combination with MCAT for graduate entry but little evidence about the predictive validity of school leaver scores. Interviews have not been shown to be robust selection measures. Studies of multiple mini-interviews have indicated good predictive validity and reliability. Of other measures used in selection, only the growing interest in personality testing appears to warrant future work. Widening access to medical and health professional programmes is an increasing priority and relates to the social accountability mandate of medical and health professional schools. While traditional selection measures do discriminate against various population groups, there is little evidence on the effect of non-traditional measures in widening access. Preparation and outreach programmes show most promise. In summary, the areas of consensus for assessment for selection are small in number. Recommendations for future action focus on the adoption of principles of good assessment and curriculum alignment, use of multi-method programmatic approaches, development of interdisciplinary frameworks and utilisation of sophisticated measurement models. The social accountability mandate of medical and health professional schools demands that social inclusion, workforce issues and widening of access are embedded in the principles of good assessment for selection.

Intellectual aptitude tests and A levels for selecting UK school leaver entrants for medical school

An extension of A level grades is the most promising alternative to intellectual aptitude tests for selecting students for medical school

Assessment of personal qualities in relation to admission to medical school

Background  Recently there has been much scrutiny of the medical school admissions process by universities, the General Medical Council and the public. Improved objectivity, fairness and effectiveness of selection procedures are desirable. The ultimate outcome sought is the graduation of competent doctors who reflect the values of and are in tune with the communities they serve.

The objective structured interview for medical student selection

An objective structured interview is an integral part of the process of selecting and admitting applicants to study medicine at this university. During the nine years (to the end of 1986) that the interview has been used 1600 candidates were interviewed out of roughly 13 000 applicants, and from these, 584 students were admitted to the course. Analysis of the interview data was carried out based on two aspects of student progress: graduation with honours and failure to complete the course of study. The interview as a whole, and especially some of the subscales, appears to identify students who may fail to complete the course: it may also help to predict which students are likely to graduate with honours.

Personality factors in professional ethical behaviour: Studies of empathy and narcissism

An investigation into personality factors possibly underlying ethical behaviour in medical students and doctors indicated the importance of two primary dimensions, empathy and narcissism. Experimental questionnaires based on these dimensions were constructed and administered to large samples of medical school applicants. Factor analyses suggested the presence of four factors, labelled Narcissism, Aloofness, Empathy and Confidence. These were combined in a 100-item instrument intended for screening purposes. This paper reports on two construct validation studies for the intrument, with New Zealand medical students (n = 237) and Scottish medical school applicants (n = 510), using overlapping batteries of personality scales. The validity coefficients for the samples indicate similar constructs for narcissism (related to disagreeableness, aggressiveness, aloofness from others, sensitivity to rewards, and anxiety), and empathy (related positively to emotional intelligence, extraversion, open-mindedness, compli...

A comprehensive model for the selection of medical students

Background: Medical schools have a need to select their students from an excess of applicants. Selection procedures have evolved piecemeal: Academic thresholds have risen, written tests have been incorporated and interview protocols are developed. Aim: To develop and offer for critical review and, ultimately, present for adoption by medical schools, an evidence-based and defensible model for medical student selection. Methods: We have described here a comprehensive model for selecting medical students which is grounded on the theoretical and empirical selection and assessment literature, and has been shaped by our own research and experience. Results: The model includes the following selection criteria: Informed self-selection, academic achievement, general cognitive ability (GCA) and aspects of personality and interpersonal skills. A psychometrically robust procedure by which cognitive and non-cognitive test scores can be used to make selection decisions is described. Using de-identified data (n = 1000) from actual selection procedures, we demonstrate how the model and the procedure can be used in practice. Conclusion: The model presented is based on a currently best-practice approach and uses measures and methods that maximise the probability of making accurate, fair and defensible selection decisions.

Time for a review of admission to medical school

Appropriate selection of medical students is a fundamental prerequisite if medical schools are to produce competent and caring doctors. The selection criteria for entry to the medical degree course at the University of Newcastle, New South Wales, are unique in Australia. The purpose of this study was to identify admission criteria that may predict performance in the first postgraduate (intern) year. Performance ratings were obtained from the clinical supervisors of two graduating classes of Newcastle University medical students during their five terms in internship (first postgraduate year). At least one rating was obtained for 93% of interns. A subset analysis of interns with multiple ratings (57%) showed that combining previous study in both humanities and science before medical school entry was predictive of higher intern performance ratings. These interns were rated more favourably than those who had studied science alone. Moreover, students who had earlier studied both humanities and science were twice as likely to complete their medical degree as those who had studied science alone. Age, gender, admission interview results, written psychometric test scores, academic marks, and whether previous tertiary study had been undertaken prior to medical school entry were not predictive of intern performance ratings. Subject spread, including a background in humanities, is important for effective medical practice, at least in the immediate postgraduate period. Perhaps it is time to evaluate the admission criteria by which medical students are selected.

Should medical students be selected only from recent school-leavers who have studied science?

Summary. This study was undertaken to investigate the apparently widespread belief that in order to be successful at medical school, aspiring candidates must have a sound academic background based predominantly on the study of the natural sciences, be school‐leavers aged about 18 (in the non‐college educational environments such as Australia or the United Kingdom) and preferably be men. The demographic background and prior academic achievement of individual students entering the University of Newcastle Medical School between 1978 and 1989 has been reviewed and compared with their progress in medical studies. The data show that in the Newcastle medical school environment there is no significant correlation between outcome and previous study of any of the natural sciences. However, significant correlations between outcome and performance in the humanities in general, and English in particular, were found. In this regard a weak background in these subjects correlated with an increased tendency not to complete the medical course. There were no significant correlations found between outcome and age at entry, sex or levels of prior academic achievement (within the top 10% achievement band studied). In summary, the study offers no support for the current selection criteria dominating medical school admissions processes.

Selection of medical students according to their moral orientation

Introduction  Consideration has been given to the use of tests of moral reasoning in the selection procedure for medical students. We argue that moral orientation, rather than moral reasoning, might be more efficacious in minimising the likelihood of inappropriate ethical behaviour in medicine. A conceptualisation and measure of moral orientation are presented, together with findings from 11 samples of medical school applicants and students.

Clinical competence of interns

A clinical supervisors rating form addressing 13 competencies was used to assess the clinical competence of graduates one year after qualification in New South Wales (NSW), Australia. Data from 485 interns (97.2%) showed that graduates from the problem‐based medical school were rated significantly better than their peers with respect to their interpersonal relationships, ‘reliability’ and ‘self‐directed learning’. Interns from one of the two traditional NSW medical schools had significantly higher ratings on ‘teaching’, ‘diagnostic skills’ and ‘understanding of basic mechanisms’. Graduates from international medical schools performed worse than their peers on all competencies. These results were adjusted for age and gender. Additionally, women graduates and younger interns tended to have better ratings. Junior doctors have differing educational and other background experiences and their performance should be monitored.