Christoph Berendonk

The new licencing examination for human medicine: from concept to implementation

A new Swiss federal licencing examination for human medicine (FLE) was developed and released in 2011. This paper describes the process from concept design to the first results obtained on implementation of the new examination. The development process was based on the Federal Act on University Medical Professions and involved all national stakeholders in this venture. During this process questions relating to the assessment aims, the assessment formats, the assessment dimensions, the examination content and necessary trade-offs were clarified. The aims were to create a feasible, fair, valid and psychometrically sound examination in accordance with international standards, thereby indicating the expected knowledge and skills level at the end of undergraduate medical education. Finally, a centrally managed and locally administered examination comprising a written multiple-choice element and a practical “clinical skills” test in the objective structured clinical examination (OSCE) format was developed. The first two administrations of the new FLE show that the examination concept could be implemented as intended. The anticipated psychometric indices were achieved and the results support the validity of the examination. Possible changes to the format or content in the future are discussed.

The influence of students’ prior clinical skills and context characteristics on mini-CEX scores in clerkships – a multilevel analysis

BackgroundIn contrast to objective structured clinical examinations (OSCEs), mini-clinical evaluation exercises (mini-CEXs) take place at the clinical workplace. As both mini-CEXs and OSCEs assess clinical skills, but within different contexts, this study aims at analyzing to which degree students’ mini-CEX scores can be predicted by their recent OSCE scores and/or context characteristics.MethodsMedical students participated in an end of Year 3 OSCE and in 11 mini-CEXs during 5 different clerkships of Year 4. The students’ mean scores of 9 clinical skills OSCE stations and mean ‘overall’ and ‘domain’ mini-CEX scores, averaged over all mini-CEXs of each student were computed. Linear regression analyses including random effects were used to predict mini-CEX scores by OSCE performance and characteristics of clinics, trainers, students and assessments.ResultsA total of 512 trainers in 45 clinics provided 1783 mini-CEX ratings for 165 students; OSCE results were available for 144 students (87 %). Most influential for the prediction of ‘overall’ mini-CEX scores was the trainers’ clinical position with a regression coefficient of 0.55 (95 %-CI: 0.26–0.84; p < .001) for residents compared to heads of department. Highly complex tasks and assessments taking place in large clinics significantly enhanced ‘overall’ mini-CEX scores, too. In contrast, high OSCE performance did not significantly increase ‘overall’ mini-CEX scores.ConclusionIn our study, Mini-CEX scores depended rather on context characteristics than on students’ clinical skills as demonstrated in an OSCE. Ways are discussed which focus on either to enhance the scores’ validity or to use narrative comments only.

The mini-clinical evaluation exercise during medical clerkships: are learning needs and learning goals aligned?

The generation of learning goals (LGs) that are aligned with learning needs (LNs) is one of the main purposes of formative workplace‐based assessment. In this study, we aimed to analyse how often trainer–student pairs identified corresponding LNs in mini‐clinical evaluation exercise (mini‐CEX) encounters and to what degree these LNs aligned with recorded LGs, taking into account the social environment (e.g. clinic size) in which the mini‐CEX was conducted.

Factors influencing the educational impact of Mini-CEX and DOPS: A qualitative synthesis

Abstract Introduction: The educational impact of Mini-CEX and DOPS varies greatly and can be influenced by several factors. However, there is no comprehensive analysis and synthesis of the described influencing factors. Methods: To fill this gap, we chose a two-step approach. First, we performed a systematic literature review and selected articles describing influencing factors on the educational impact of Mini-CEX and DOPS. Second, we performed a qualitative synthesis of these factors. Results: Twelve articles were included, which revealed a model consisting of four themes and nine subthemes as influencing factors. The theme context comprises “time for Mini-CEX/DOPS” and “usability of the tools”, and influences the users. The theme users comprises “supervisors’ knowledge about how to use Mini-CEX/DOPS”, “supervisors’ attitude to Mini-CEX/DOPS”, “trainees’ knowledge about Mini-CEX/DOPS”, and “trainees’ perception of Mini-CEX/DOPS”. These influence the implementation of Mini-CEX and DOPS, including “observation” and “feedback”. The theme implementation directly influences the theme outcome, which, in addition to the educational impact, encompasses “trainees’ appraisal of feedback”. Conclusions: Our model of influencing factors might help to further improve the use of Mini-CEX and DOPS and serve as basis for future research.

The new final Clinical Skills examination in human medicine in Switzerland: Essential steps of exam development, implementation and evaluation, and central insights from the perspective of the national Working Group.

Objective: Since 2011, the new national final examination in human medicine has been implemented in Switzerland, with a structured clinical-practical part in the OSCE format. From the perspective of the national Working Group, the current article describes the essential steps in the development, implementation and evaluation of the Federal Licensing Examination Clinical Skills (FLE CS) as well as the applied quality assurance measures. Finally, central insights gained from the last years are presented. Methods: Based on the principles of action research, the FLE CS is in a constant state of further development. On the foundation of systematically documented experiences from previous years, in the Working Group, unresolved questions are discussed and resulting solution approaches are substantiated (planning), implemented in the examination (implementation) and subsequently evaluated (reflection). The presented results are the product of this iterative procedure. Results: The FLE CS is created by experts from all faculties and subject areas in a multistage process. The examination is administered in German and French on a decentralised basis and consists of twelve interdisciplinary stations per candidate. As important quality assurance measures, the national Review Board (content validation) and the meetings of the standardised patient trainers (standardisation) have proven worthwhile. The statistical analyses show good measurement reliability and support the construct validity of the examination. Among the central insights of the past years, it has been established that the consistent implementation of the principles of action research contributes to the successful further development of the examination. Conclusion: The centrally coordinated, collaborative-iterative process, incorporating experts from all faculties, makes a fundamental contribution to the quality of the FLE CS. The processes and insights presented here can be useful for others planning a similar undertaking.

The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis

Introduction Mini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences. Methods We searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr’s adaptation of Kirkpatrick’s four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher’s exact test for count data. Results We identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners’ reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees’ performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, “quality” and “participant responsiveness”, to be associated with the educational impact. Conclusions Despite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.

The Authors reply: Factors influencing the educational impact of mini-CEX and DOPS

We thank Evans and Lambrinudi for sharing their thoughts and ideas on our article “Factors influencing the educational impact of Mini-CEX and DOPS: A qualitative synthesis” (L€ orwald et al. 2017). One aspect in their letter struck us as particularly noteworthy. The authors observe how in the program in which they are enrolled, students regularly reflect on their clinical encounters from their first year of training, until this reflection seemingly becomes almost a second nature to them. They then hypothesize that this “entrenched need to reflect on all experiences” improves their attitudes and perceptions toward completing assessments such as Mini-CEX and DOPS. There are, in fact, hints in the literature that support their hypothesis. Back in 1986, Dweck studied how motivational processes affect learning (Dweck 1986). She found that one’s goal orientation determines one’s learning behavior. She distinguished between two kinds of goal orientation: people with a performance goal orientation aim to receive positive judgments and avoid criticism; people with a learning goal orientation instead aim to increase their competence. This translates to the assumption that students with a learning goal orientation persistently seek new challenges, while students with a performance goal orientation might prefer easy tasks for their workplace-based assessment in order to demonstrate their competency and avoid new challenges. Reflecting upon clinical encounters might foster students’ learning goal orientation, encourage them to seek new challenges and welcome feedback, and increase the impact of Mini-CEX and DOPS (Sargeant et al. 2009; Ramani et al. 2018). We would like to congratulate the authors, their teachers and their program directors for having achieved a program that fosters reflection and nourishes students’ feedback-seeking behavior.

Cognitive load in internal medicine: What every clinical teacher should know about cognitive load theory

Internal medicine is an appropriate example of specialties in which to teach learners clinical reasoning skills, decision-making, and analytical thinking, as well as evidence-based, patient-oriented medicine. During daily clinical work, general internists always encounter a multitude of situations that lend themselves to educating medical trainees in ambulatory and inpatient settings. Application of existing learning theories to teaching has been shown to optimize teaching ability and to maximize the efficiency of teaching efforts. Cognitive Load Theory explains learning according to three important aspects: the types of memory (working and long-term memory), the learning process and the forms of cognitive load that affect our learning. The aim of this paper is to show the main perspectives and implications of the Cognitive Load Theory on clinical educational practices. It is important to give the right amount of information in the most effective way to learners, thereby making this information more useful. This article presents a concise overview of the basis of the Cognitive Load Theory in its first part, and, in its second part, it exposes the practical applications of this theory with examples. This learning theory will encourage clinical teachers to reflect on how to foster learning in medical trainees in the more effective way.

Stabile Antwortmuster bei Script Concordance Test Fragen in der Schweizer Facharztprüfung Allgemeine Innere Medizin

Fragestellung: Mit dem Script Concordance Test (SCT) soll die Fähigkeit zu klinischem Denken (clinical reasoning) geprüft werden [1], [2]. Jedoch wird das Fragenformat u.a. kritisiert, weil die Messzuverlässigkeit schwierig zu überprüfen ist. Insbesondere fehlen Angaben zu test-retest Reliabilitäten [3]. Ziel der vorliegenden Studie ist es, die Stabilität der Antwortmuster auf SCT Fragen zu untersuchen.

Eidgenössische Prüfung Clinical Skills: Die neue nationale klinisch-praktische Schlussprüfung in Humanmedizin der Schweiz

Bitte zitieren als: Schurter T, König B, Vichard E, van Gessel E, Bonvin R, Maier V, Keller U, Kropf R, Beyeler C, Guttormsen S, Huwendiek S. Zentrale Erkenntnisse der Qualitätssicherung des schriftlichen Teils der Schweizer Schlussprüfung Humanmedizin nach 6 Jahren. In: Jahrestagung der Gesellschaft für Medizinische Ausbildung (GMA). Bern, 14.-17.09.2016. Düsseldorf: German Medical Science GMS Publishing House; 2016. DocV33-224. DOI: 10.3205/16gma179, URN: urn:nbn:de:0183-16gma1799 Frei verfügbar unter: http://www.egms.de/en/meetings/gma2016/16gma179.shtml