David L Garne

A longitudinal integrated placement and medical students' intentions to practise rurally.

Medical Education 2012: 46 : 179–191

Improving assessment practice through cross-institutional collaboration: An exercise on the use of OSCEs

Abstract Background: This study was undertaken to improve assessment practice on OSCEs through collaboration across geographically dispersed medical schools in Australia. Methods: A total of eleven OSCE stations were co-developed by four medical schools and used in summative 2011 and 2012 examinations for the assessment of clinical performance in the early clinical and exit OSCEs in each school’s medical course. Partial Credit Rasch Model was used to evaluate the psychometric properties of the shared OSCE data. Evaluation of the quality assurance reports was used to determine the beneficial impact of the collaborative benchmarking exercise on learning and teaching outcomes. Results: The data for each examination demonstrated sufficient fit to the Rasch model with infit mean square values ranging from 0.88 to 0.99. Person separation (1.25–1.63) indices indicated good reliability. Evaluation of perceived benefits showed that the benchmarking process was successful as it highlighted common curriculum areas requiring specific focus and provided comparable data on the quality of teaching at the participating medical schools. Conclusion: This research demonstrates the validity of the psychometric data and benefits of evaluating clinical competence across medical schools without the enforcement of a prescriptive national curriculum or assessment.

A collaborative comparison of Objective Structured Clinical Examination (OSCE) standard setting methods at Australian medical schools

Abstract Background: A key issue underpinning the usefulness of the OSCE assessment to medical education is standard setting, but the majority of standard-setting methods remain challenging for performance assessment because they produce varying passing marks. Several studies have compared standard-setting methods; however, most of these studies are limited by their experimental scope, or use data on examinee performance at a single OSCE station or from a single medical school. This collaborative study between 10 Australian medical schools investigated the effect of standard-setting methods on OSCE cut scores and failure rates. Methods: This research used 5256 examinee scores from seven shared OSCE stations to calculate cut scores and failure rates using two different compromise standard-setting methods, namely the Borderline Regression and Cohen’s methods. Results: The results of this study indicate that Cohen’s method yields similar outcomes to the Borderline Regression method, particularly for large examinee cohort sizes. However, with lower examinee numbers on a station, the Borderline Regression method resulted in higher cut scores and larger difference margins in the failure rates. Conclusion: Cohen’s method yields similar outcomes as the Borderline Regression method and its application for benchmarking purposes and in resource-limited settings is justifiable, particularly with large examinee numbers.

Block versus longitudinal integrated clerkships: students’ views of rural clinical supervision

Medical students undertaking longitudinal integrated clerkships (LICs) train in multiple disciplines concurrently, compared with students in block rotations who typically address one medical discipline at a time. Current research suggests that LICs afford students increased access to patients and continuity of clinical supervision. However, these factors are less of an issue in rural placements where there are fewer learners. The aim of this study was to compare rural LIC and rural block rotation students’ reported experiences of clinical supervision.