Michaela Zupanic

May student examiners be reasonable substitute examiners for faculty in an undergraduate OSCE on medical emergencies

Abstract Objectives: To compare the effect of student examiners (SE) to that of faculty examiners (FE) on examinee performance in an OSCE as well as on post-assessment evaluation in the area of emergency medicine management. Methods: An OSCE test-format (seven stations: Advanced Cardiac Life Support (ACLS), Basic Life Support (BLS), Trauma-Management (TM), Pediatric-Emergencies (PE), Acute-Coronary-Syndrome (ACS), Airway-Management (AM), and Obstetrical-Emergencies (OE)) was administered to 207 medical students in their third year of training after they had received didactics in emergency medicine management. Participants were randomly assigned to one of the two simultaneously run tracks: either with SE (n = 110) or with FE (n = 98). Students were asked to rate each OSCE station and to provide their overall OSCE perception by means of a standardized questionnaire. The independent samples t-test was used and effect sizes were calculated (Cohens d). Results: Students achieved significantly higher scores for the OSCE stations “TM”, “AM”, and “OE” as well as “overall OSCE score” in the SE track, whereas the station score for “PE” was significantly higher for students in the FE track. Mostly small effect sizes were reported. In the post-assessment evaluation portion of the study, students gave significant higher ratings for the ACS station and “overall OSCE evaluation” in the FE track; also with small effect sizes. Conclusion: It seems quite admissible and justified to encourage medical students to officiate as examiners in undergraduate emergency medicine OSCE formative testing, but not necessarily in summative assessment evaluations.

The Questionnaire D-RECT German: Adaptation and testtheoretical properties of an instrument for evaluation of the learning climate in medical specialist training.

Aim: Boor et al [1] developed and validated the questionnaire D-RECT (Dutch Residency Educational Climate Test ) to measure the clinical learning environment within the medical specialist training. In this study, a German version of this questionnaire (D-RECT German) is analyzed regarding testtheoretical properties. Problem: Are the results of Boor et al replicable as a proof for validity of the questionnaire D-RECT? Material & Methods: The study was performed as online survey using the questionnaire D-RECT German (50 items in 11 subscales). To determine item characteristics and internal consistency (Cronbach’s α), item- and reliability analyses were performed. Furthermore, a confirmatory factor analysis was performed using a model for maximum-likelihood estimation to evaluate validity. Results: This replication study on the psychometric properties of the D-RECT with 255 residents at 17 German hospitals revealed heterogeneous discriminatory power for all items and an internal consistency of Cronbach’s α between 0.57 and 0.85. Within the confirmatory factor analysis, 6 items showed standardized regression coeffizients <0.5, two of them in the subscale “Attendings role”. Furthermore, strong interdependencies (>0.7) were found between the subscales “Supervision”, “Coaching” and “Attendings role”. Conclusion: The present replication study with the D-RECT German showed structural differences with respect to factorial validity underpinning the need of further validation studies.

The Questionnaire "SFDP26-German": a reliable tool for evaluation of clinical teaching?

Aims: Evaluation of the effectiveness of clinical teaching is an important contribution for the quality control of medical teaching. This should be evaluated using a reliable instrument in order to be able to both gauge the status quo and the effects of instruction. In the Stanford Faculty Development Program (SFDP), seven categories have proven to be appropriate: Establishing the Learning Climate, Controlling a Teaching Session, Communication of Goals, Encouraging Understanding and Retention, Evaluation, Feedback and Self-directed Learning. Since 1998, the SFDP26 questionnaire has established itself as an evaluation tool in English speaking countries. To date there is no equivalent German-language questionnaire available which evaluates the overall effectiveness of teaching. Question: Development and theoretical testing of a German-language version of SFDP26 (SFDP26-German), Check the correlation of subscale of SFDPGerman against overall effectiveness of teaching. Methods: 19 anaesthetists (7 female, 12 male) from the University of Lübeck were evaluated at the end of a teaching seminar on emergency medical care using SFDP-German. The sample consisted of 173 medical students (119 female (68.8%) and 54 male (31.2%), mostly from the fifth semester (6.6%) and sixth semester (80.3%). The mean age of the students was 23±3 years. Results: The discriminatory power of all items ranged between good and excellent (rit=0.48-0.75). All subscales displayed good internal consistency (α=0.69-0.92) and significant positive inter-scale correlations (r=0.40-0.70). The subscales and “overall effectiveness of teaching” showed significant correlation, with the highest correlation for the subscale “communication of goals (p< 0.001; r = 0.61). Conclusion: The analysis of SFDP26-German confirms high internal consistency. Future research should investigate the effectiveness of the individual categories on the overall effectiveness of teaching and validate according to external criteria.

Students benefit from developing their own emergency medicine OSCE stations: a comparative study using the matched-pair method.

BackgroundStudents can improve the learning process by developing their own multiple choice questions. If a similar effect occurred when creating OSCE (objective structured clinical examination) stations by themselves it could be beneficial to involve them in the development of OSCE stations. This study investigates the effect of students developing emergency medicine OSCE stations on their test performance.MethodIn the 2011/12 winter semester, an emergency medicine OSCE was held for the first time at the Faculty of Medicine at the University of Leipzig. When preparing for the OSCE, 13 students (the intervention group) developed and tested emergency medicine examination stations as a learning experience. Their subsequent OSCE performance was compared to that of 13 other students (the control group), who were parallelized in terms of age, gender, semester and level of previous knowledge using the matched-pair method. In addition, both groups were compared to 20 students who tested the OSCE prior to regular emergency medicine training (test OSCE group).ResultsThere were no differences between the three groups regarding age (24.3 ± 2.6; 24.2 ± 3.4 and 24 ± 2.3 years) or previous knowledge (29.3 ± 3.4; 29.3 ± 3.2 and 28.9 ± 4.7 points in the multiple choice [MC] exam in emergency medicine). Merely the gender distribution differed (8 female and 5 male students in the intervention and control group vs. 3 males and 17 females in the test OSCE group).In the exam OSCE, participants in the intervention group scored 233.4 ± 6.3 points (mean ± SD) compared to 223.8 ± 9.2 points (p < 0.01) in the control group. Cohen’s effect size was d = 1.24. The students of the test OSCE group scored 223.2 ± 13.4 points.ConclusionsStudents who actively develop OSCE stations when preparing for an emergency medicine OSCE achieve better exam results.

Enhancing medical students’ reflectivity in mentoring groups for professional development – a qualitative analysis

BackgroundProfessional competence is important in delivering high quality patient care, and it can be enhanced by reflection and reflective discourse e.g. in mentoring groups. However, students are often reluctant though to engage in this discourse. A group mentoring program involving all preclinical students as well as faculty members and co-mentoring clinical students was initiated at Witten-Herdecke University. This study explores both the attitudes of those students towards such a program and factors that might hinder or enhance how students engage in reflective discourse.MethodsA qualitative design was applied using semi-structured focus group interviews with preclinical students and semi-structured individual interviews with mentors and co-mentors. The interview data were analyzed using thematic content analysis.ResultsStudents’ attitudes towards reflective discourse on professional challenges were diverse. Some students valued the new program and named positive outcomes regarding several features of professional development. Enriching experiences were described. Others expressed aversive attitudes. Three reasons for these were given: unclear goals and benefits, interpersonal problems within the groups hindering development and intrapersonal issues such as insecurity and traditional views of medical education. Participants mentioned several program setup factors that could enhance how students engage in such groups: explaining the program thoroughly, setting expectations and integrating the reflective discourse in a meaningful way into the curriculum, obliging participation without coercion, developing a sense of security, trust and interest in each other within the groups, randomizing group composition and facilitating group moderators as positive peer and faculty role models and as learning group members.ConclusionsA well-designed and empathetic setup of group mentoring programs can help raise openness towards engaging in meaningful reflective discourse. Reflection on and communication of professional challenges can, in turn, improve professional development, which is essential for high quality patient care.

Development of an Online-System for Assessing the Progress of Knowledge Acquisition in Psychology Students.

Results of summative examinations represent most often only a snapshot of the knowledge of students over a part of the curriculum and do not provide valid information on whether a long-term retention of knowledge and knowledge growth takes place during the course of studies. Progress testing allows the repeated formative assessment of students’ functional knowledge and consists of questions covering all domains of relevant knowledge from a given curriculum. This article describes the development and structure of an online platform for progress testing in psychology at the Witten/Herdecke University. The Progress Test Psychology (PTP) was developed in 2015 in the Department of Psychology and Psychotherapy at Witten/Herdecke University and consists of 100 confidence-weighted true-/false-items (sure / unsure / don’t know). The Online-System for implementation of the PTP was developed based on XAMPP including an Apache Server, a MySQL-Database, PHP and JavaScript. First results of a longitudinal survey show the increase in student’s knowledge in the course of studies also reliably reflects the course of the curriculum. Thus, content validity of the PTP could be confirmed. Apart from directly measuring the long-term retention of knowledge the use of the PTP in the admission of students applying for a Master’s program is discussed.

Implementation and Empirical Evaluation of a Case-based, Interactive e-Learning Module with X-ray Tooth Prognosis.

The prognosis estimation of teeth based on radiographs is a subordinate but relevant target in many dental medicine curricula in Germany. Empirical data on the integration of e-learning material into dental curricula are rare. We aimed at developing and implementing a radiological pillar diagnostics online-course in the dental curriculum at the University of Witten/Herdecke. This online course was developed on the CASUS web-based learning platform and implemented in a blended learning approach. Results showed an easy creation of learning cases (virtual patients), higher utilization for the intervention group regarding the number of cases revised, time-on-task, and student acceptance. Dental students experienced improved learning efficacy, higher long time knowledge retention and significantly better results in case based assessment. The usability of the CASUS learning Platform therefore can be regarded as high and further studies using this e-learning approach are recommended.

Can today's house officers teach effectively? An assessment in undergraduate emergency training.

Background Previous publications describe house officers (HOs) as unaware of their ineffective teaching skills. Objective The aim of this study was to evaluate the role of teaching seniority in the comparison of teaching skills between HOs and faculty. Materials and methods Ten HOs (F: n=4, M: n=6, age: 35.1±6.8 years) and nine faculty (F: n=3, M: n=6, age: 41.4±4.9 years) who actively teach undergraduate emergency medicine were immediately evaluated at the end of the course by their students using the questionnaire SFDP26. The questionnaire consists of one item on ‘overall teaching effectiveness’ (OTE) (1=very poor to 5=excellent) and 25 items measured on a five-point Likert scale (1–5=strongly disagree to strongly agree) divided into seven subscales: 1, ‘establishing the learning climate’ (LC); 2, ‘control of session’ (CS); 3, ‘communication of goals’ (CG); 4, ‘facilitating understanding and retention’ (UR); 5, ‘evaluation’ (E); 6, ‘feedback’ (FB) and 7, ‘promoting self-directed learning’ (SL). The sample included 173 medical students in their third year of training who were randomly assigned to the instructors. A Mann–Whitney U-test was used to calculate group-related differences (resident vs. teaching faculty). For significant differences, effect size was calculated (r=Z/√N). Results No sex-related differences were found. Significantly better ratings for HOs were found in subscales: 1, ‘LC’ (P=0.001; r=0.20); 2, ‘CS’ (P=0.037; r=0.15); 5, ‘E’ (P=0.007; r=0.20); 6, ‘FB’ (P=0.001; r=0.23); 7, ‘SL’ (P=0.004; r=0.24) and ‘OTE’ (P=0.027; r=0.26). Conclusion From a learner’s perspective, the quality of teaching provided by HOs was rated at least similar and mostly better overall than that provided by faculty. These findings contradict results from previous studies on the quality of HO teaching and therefore warrant further assessment.