Sydney Smee

Large-scale high-stakes testing with an OSCE: report from the Medical Council of Canada.

No abstract available.

Guidelines for Estimating the Real Cost of an Objective Structured Clinical Examination.

A major impediment to the use of the objective structured clinical examination (OSCE) is that it is a labor-intensive and costly form of assessment. The cost of an OSCE is highly dependent on the particular model used, the extent to which hidden costs are reported, and the purpose of the examination. The authors detail hypothetical costs of running a four-hour OSCE for 120 medical students at one medical school. Costs are reported for four phases of this process: development, production, administration, and post-examination reporting and analysis. Costs are reported at two ends of the spectrum: the high end, where it is assumed that little is paid for by the institution and that faculty receive honoraria for work put into the examination; and the low end, where it is assumed that the sponsoring institution defrays basic costs and that faculty do not receive honoraria for their participation. The total costs reported for a first-time examination were

Doctor scores on national qualifying examinations predict quality of care in future practice

104,400 and

Influence of Physicians' Management and Communication Ability on Patients' Persistence With Antihypertensive Medication

59,460 (Canadian dollars) at the high and low ends, respectively. These translate to per-student costs of

Using the Judgments of Physician Examiners in Setting the Standards for a National Multi-center High Stakes OSCE

870 and

A Comparison of Physician Examiners and Trained Assessors in a High-stakes Osce Setting

496. The cost of running an OSCE is high. However, the OSCE is uniquely capable of assessing many fundamental clinical skills that are presently not being assessed in a rigorous way in most medical schools.

Core principles of assessment in competency-based medical education

Objectives  This study aimed to determine if national licensing examinations that measure medical knowledge (QE1) and clinical skills (QE2) predict the quality of care delivered by doctors in future practice.

A sequenced OSCE for licensure: Administrative issues, results and myths

BACKGROUND Less than 75% of people prescribed antihypertensive medication are still using treatment after 6 months. Physicians determine treatment, educate patients, manage side effects, and influence patient knowledge and motivation. Although physician communication ability likely influences persistence, little is known about the importance of medical management skills, even though these abilities can be enhanced through educational and practice interventions. The purpose of this study was to determine whether a physicians medical management and communication ability influence persistence with antihypertensive treatment. METHODS This was a population-based study of 13,205 hypertensive patients who started antihypertensive medication prescribed by a cohort of 645 physicians entering practice in Quebec, Canada, between 1993 and 2007. Medical Council of Canada licensing examination scores were used to assess medical management and communication ability. Population-based prescription and medical services databases were used to assess starting therapy, treatment changes, comorbidity, and persistence with antihypertensive treatment in the first 6 months. RESULTS Within 6 months after starting treatment, 2926 patients (22.2%) had discontinued all antihypertensive medication. The risk of nonpersistence was reduced for patients who were treated by physicians with better medical management (odds ratio per 2-SD increase in score, 0.74; 95% confidence interval, 0.63-0.87) and communication (0.88; 0.78-1.00) ability and with early therapy changes (odds ratio, 0.45; 95% confidence interval, 0.37-0.54), more follow-up visits, and nondiuretics as the initial choice of therapy. Medical management ability was responsible for preventing 15.8% (95% confidence interval, 7.5%-23.3%) of nonpersistence. CONCLUSION Better clinical decision-making and data collection skills and early modifications in therapy improve persistence with antihypertensive therapy.

A method for identifying extreme OSCE examiners

In 1994 and 1995, the Medical Council of Canada used an innovative approach to set the pass mark on its large scale, multi-center national OSCE which is designed to assess basic clinical and communication skills in physicians in Canada after 15 months of post-graduate medical training. The goal of this article is to describe the new approach and to present the experience with the method during its first two years of operation. The approach utilizes the global judgments of the physician examiners at each station to identify the candidates with borderline performances. The scores of the candidates whose performances are judged to be borderline are summed for each station, yielding an initial passing score for all stations and then the examination as a whole. The latter score is then adjusted upward one standard error of measurement for the final passing score and is used as one of the criteria to pass the examination. Based on the results to date, the new approach has worked well. The advantages, disadvantages and areas of possible refinement for the approach are reviewed.

Does the gender of the standardised patient influence candidate performance in an objective structured clinical examination

Background The Medical Council of Canada (MCC) administers an objective structured clinical examination for licensure. Traditionally, physician examiners (PE) have evaluated these examinees. Recruitment of physicians is becoming more difficult. Determining if alternate scorers can be used is of increasing importance. Method In 2003, the MCC ran a study using trained assessors (TA) simultaneously with PEs. Four examination centers and three history-taking stations were selected. Health care workers were recruited as the TAs. Results A 3 × 2 × 4 mixed analyses of variance indicated no significant difference between scorers (F1,462 = .01, p = .94). There were significant interaction effects, which were, localized to site 1/station 3, site 3/station 2, and site 4/station1. Pass/fail decisions would have misclassified 14.4–25.01% of examinees. Conclusion Trained assessors may be a valid alternative to PE for completing checklists in history-taking stations, but their role in completing global ratings is not supported by this study.