Christine Beyeler

The new licencing examination for human medicine: from concept to implementation

A new Swiss federal licencing examination for human medicine (FLE) was developed and released in 2011. This paper describes the process from concept design to the first results obtained on implementation of the new examination. The development process was based on the Federal Act on University Medical Professions and involved all national stakeholders in this venture. During this process questions relating to the assessment aims, the assessment formats, the assessment dimensions, the examination content and necessary trade-offs were clarified. The aims were to create a feasible, fair, valid and psychometrically sound examination in accordance with international standards, thereby indicating the expected knowledge and skills level at the end of undergraduate medical education. Finally, a centrally managed and locally administered examination comprising a written multiple-choice element and a practical “clinical skills” test in the objective structured clinical examination (OSCE) format was developed. The first two administrations of the new FLE show that the examination concept could be implemented as intended. The anticipated psychometric indices were achieved and the results support the validity of the examination. Possible changes to the format or content in the future are discussed.

The influence of students’ prior clinical skills and context characteristics on mini-CEX scores in clerkships – a multilevel analysis

BackgroundIn contrast to objective structured clinical examinations (OSCEs), mini-clinical evaluation exercises (mini-CEXs) take place at the clinical workplace. As both mini-CEXs and OSCEs assess clinical skills, but within different contexts, this study aims at analyzing to which degree students’ mini-CEX scores can be predicted by their recent OSCE scores and/or context characteristics.MethodsMedical students participated in an end of Year 3 OSCE and in 11 mini-CEXs during 5 different clerkships of Year 4. The students’ mean scores of 9 clinical skills OSCE stations and mean ‘overall’ and ‘domain’ mini-CEX scores, averaged over all mini-CEXs of each student were computed. Linear regression analyses including random effects were used to predict mini-CEX scores by OSCE performance and characteristics of clinics, trainers, students and assessments.ResultsA total of 512 trainers in 45 clinics provided 1783 mini-CEX ratings for 165 students; OSCE results were available for 144 students (87 %). Most influential for the prediction of ‘overall’ mini-CEX scores was the trainers’ clinical position with a regression coefficient of 0.55 (95 %-CI: 0.26–0.84; p < .001) for residents compared to heads of department. Highly complex tasks and assessments taking place in large clinics significantly enhanced ‘overall’ mini-CEX scores, too. In contrast, high OSCE performance did not significantly increase ‘overall’ mini-CEX scores.ConclusionIn our study, Mini-CEX scores depended rather on context characteristics than on students’ clinical skills as demonstrated in an OSCE. Ways are discussed which focus on either to enhance the scores’ validity or to use narrative comments only.

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace‐based assessment. In this study, we aimed to analyse how often trainer–student pairs identified corresponding LNs in mini‐clinical evaluation exercise (mini‐CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini‐CEX was conducted.

The new final Clinical Skills examination in human medicine in Switzerland: Essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group.

Objective: Since 2011, the new national final examination in human medicine has been implemented in Switzerland, with a structured clinical-practical part in the OSCE format. From the perspective of the national Working Group, the current article describes the essential steps in the development, implementation and evaluation of the Federal Licensing Examination Clinical Skills (FLE CS) as well as the applied quality assurance measures. Finally, central insights gained from the last years are presented. Methods: Based on the principles of action research, the FLE CS is in a constant state of further development. On the foundation of systematically documented experiences from previous years, in the Working Group, unresolved questions are discussed and resulting solution approaches are substantiated (planning), implemented in the examination (implementation) and subsequently evaluated (reflection). The presented results are the product of this iterative procedure. Results: The FLE CS is created by experts from all faculties and subject areas in a multistage process. The examination is administered in German and French on a decentralised basis and consists of twelve interdisciplinary stations per candidate. As important quality assurance measures, the national Review Board (content validation) and the meetings of the standardised patient trainers (standardisation) have proven worthwhile. The statistical analyses show good measurement reliability and support the construct validity of the examination. Among the central insights of the past years, it has been established that the consistent implementation of the principles of action research contributes to the successful further development of the examination. Conclusion: The centrally coordinated, collaborative-iterative process, incorporating experts from all faculties, makes a fundamental contribution to the quality of the FLE CS. The processes and insights presented here can be useful for others planning a similar undertaking.

Eidgenössische Prüfung Clinical Skills: Die neue nationale klinisch-praktische Schlussprüfung in Humanmedizin der Schweiz

Bitte zitieren als: Schurter T, König B, Vichard E, van Gessel E, Bonvin R, Maier V, Keller U, Kropf R, Beyeler C, Guttormsen S, Huwendiek S. Zentrale Erkenntnisse der Qualitätssicherung des schriftlichen Teils der Schweizer Schlussprüfung Humanmedizin nach 6 Jahren. In: Jahrestagung der Gesellschaft für Medizinische Ausbildung (GMA). Bern, 14.-17.09.2016. Düsseldorf: German Medical Science GMS Publishing House; 2016. DocV33-224. DOI: 10.3205/16gma179, URN: urn:nbn:de:0183-16gma1799 Frei verfügbar unter: http://www.egms.de/en/meetings/gma2016/16gma179.shtml

Alignment between learning needs and learning goals of Mini-CEX in clerkships

Background: Defining learning goals (LG) in alignment with learning needs (LN) is one of the key purposes of formative workplace-based assessment, but studies about this topic are scarce. Summary of Work: We analysed quantitatively and qualitatively how often trainer-student pairs identified the same LN during Mini Clinical Evaluation Exercises (Mini-CEX) in clerkships and to what degree those LNs were in line with the recorded LGs. Multilevel logistic regression models were used to predict LGs by identified LNs, controlling for context variables. Summary of Results: 512 trainers and 165 students conducted 1783 Mini-CEX (98% completion rate). Concordantly, trainer-student pairs most often identified LNs in the domains ‘clinical reasoning’ (23% of 1167 complete forms), ‘organisation / efficiency’ (20%) and ‘physical examination’ (20%). At least one ‘defined’ LG was noted on 313 student forms (18% of 1710), with a total of 446 LGs. Of these, the most frequent LGs were ‘physical examination’ (49% of 446 LGs) and ‘history taking’ (21%); corresponding LNs as well as context variables (e.g. clinic size) were found to be predictors of these LGs. Discussion and Conclusions: Although trainer-student pairs often agreed in their identified LNs, many assessments did not result in an aligned LG or a LG at all. Interventions are needed to enhance the proportion of (aligned) LGs in Mini-CEX in order to tap into its full potential not only as a ‘diagnostic’ but also as an ‘educational tool’. Take-home messages: The sparseness of LGs, their dependency on context variables and their partial non-alignment with students’ LNs raise the question of how the effectiveness of Mini-CEX can be further enhanced.

Die Entwicklung und Implementierung einer neuen staatlichen Schlussprüfung Humanmedizin am Beispiel Schweiz – Ergebnisse und Daten zur Prüfungsqualität der ersten 2 Kohorten sowie Überlegungen zur Qualitätssicherung und zur Weiterentwicklung der Prüfung

Hintergrund: Im Rahmen des neuen nationalen Medizinalberufegesetzes [http://www.admin.ch/ch/d/as/2007/4031.pdf], [http://www.bag.admin.ch/themen/berufe/07918/07919/index.html], der Entwicklung hin zu Kompetenz-basierten Curricula [1] und der Einfuhrung der Bologna-Reform in den medizinischen Studiengangen [2] wurde in der Schweiz eine neue eidgenossische Schlussprufung Humanmedizin unter Aufsicht des Bundes und in Zusammenarbeit mit den medizinischen Fakultaten in zwei Sprachen (D/F) entwickelt und 2011 erstmals durchgefuhrt. Projektbeschreibung: Im vorliegenden Beitrag werden die Rahmenbedingungen fur die Implementierung aufgezeigt und die Entwicklung der Gesamtprufung als Pass/Fail-Prufung einschliesslich ihrer 2 Einzelprufungen beschrieben. Die 1. Einzelprufung besteht aus einer schriftlichen Prufung (MCQ) an 2 Prufungstagen zu je 4.5 h mit je 150 interdisziplinaren, taxonomisch auf Anwendungswissen ausgerichteten Fragen. Die 2. Einzelprufung umfasst eine strukturierte, klinisch-praktische CS-Prufung (OSCE) mit insgesamt 12 Rotationsposten uber je 13 min Dauer und je 2 min Rotationszeit zwischen den Posten. Zur Qualitatssicherung wurden zahlreiche Massnahmen ergriffen wie z.B. die Schulung der standardisierten Patienten anhand zentraler Standardisierungsvorlagen. Der Gesamtblueprint ist abgestimmt auf den Schweizer Lernzielkatalog Humanmedizin [http://sclo.smifk.ch] und beinhaltet die 2 Hauptdimensionen „General Objectives/CanMed Roles“ und „Problems as Starting Points“. Ergebnisse: Die Prufung wurde an allen 5 Standorten 2011 und 2012 erfolgreich durchgefuhrt. Die Prufungsresultate der ersten 2 Kohorten differenziert nach Gesamtprufung und Einzelprufungen zeigen in etwa die erwarteten Werte hinsichtlich der Bestehensquote. Die Metadaten zur Prufungsqualitat zeigen fur beide Jahre, dass die angestrebte Messzuverlassigkeit der Prufung mit einem Cronbach Alpha als Mass fur die Reliabilitat von im Mittel α=0.9 fur die MCQ Einzelprufung und von im Mittel α>0.8 fur die CS-Einzelprufung erreicht wurde. Diskussion und Schlussfolgerungen: Basierend auf den Erfahrungen und Daten der ersten 2 Prufungskohorten kann gesagt werden, dass die Implementierung einer neuen nationalen Prufung, die neben der neu ausgerichteten MCQ-Einzelprufung erstmals mit einem strukturierten, objektivierbaren und national standardisierten Instrument klinische Fahigkeiten und Fertigkeiten misst, grundsatzlich gelungen ist. In diesem Kontext muss die Relevanz der intensiven Koordination und Abstimmung von der Gesetzgebung und den Verordnungsvorgaben bis hin zum Lernzielkatalog und dem korrespondierenden Gesamtblueprint der Prufung hervorgehoben werden. Bezuglich der zukunftigen Entwicklung werden Aspekte der Qualitatssicherung und der Weiterentwicklung der Gesamtprufung auch im Sinne von erganzenden Prufungsformaten diskutiert werden.

Ziel und Nutzen des Arbeitsplatz-basierten Assessments im klinischen Alltag

Das Arbeitsplatz-basierte Assessment hat das Ziel, den Lernprozess des Arztes in Weiterbildung zu unterstutzen.