Jasmina Sterz

Examiner effect on the objective structured clinical exam - a study at five medical schools

BackgroundThe Objective Structured Clinical Examination (OSCE) is increasingly used at medical schools to assess practical competencies. To compare the outcomes of students at different medical schools, we introduced standardized OSCE stations with identical checklists.MethodsWe investigated examiner bias at standardized OSCE stations for knee- and shoulder-joint examinations, which were implemented into the surgical OSCE at five different medical schools. The checklists for the assessment consisted of part A for knowledge and performance of the skill and part B for communication and interaction with the patient. At each medical faculty, one reference examiner also scored independently to the local examiner. The scores from both examiners were compared and analysed for inter-rater reliability and correlation with the level of clinical experience. Possible gender bias was also evaluated.ResultsIn part A of the checklist, local examiners graded students higher compared to the reference examiner; in part B of the checklist, there was no trend to the findings. The inter-rater reliability was weak, and the scoring correlated only weakly with the examiner’s level of experience. Female examiners rated generally higher, but male examiners scored significantly higher if the examinee was female.ConclusionsThese findings of examiner effects, even in standardized situations, may influence outcome even when students perform equally well. Examiners need to be made aware of these biases prior to examining.

Undergraduate medical students need more training in craniomaxillofacial surgery: a comparative study between medical and dental students

Abstract Purpose: To compare the performance in oral, craniomaxillofacial, and facial plastic surgery (CMF)-specific surgical skills between medical students (MS) and dental students (DS) and hence adjust the current CMF training to student-specific needs. The investigators hypothesized that there would be no performance differences between MS and DS. Methods: The investigators implemented a comparative retrospective item-based analysis of student performance in a CMF-specific objective structured clinical examination (OSCE) from 2008 to 2015. The sample was composed of 1010 MS and 225 DS who completed a standardized CMF training and OSCE. Three OSCE scenarios [management mandible fracture (MMF), management zygomatic fracture (MZF), and structured facial examination (SFE)] were included in the study because learning objectives were equal. Descriptive and bivariate statistics were computed and the p value was set at 0.05. Results: In all of the analyzed OSCE scenarios, DS significantly outperformed MS (MMF p<0.001; MZF p=0.013; SFE p<0.001). DS especially appeared to be better in the correct interpretation of radiological findings (five of seven items, MMF) and the correct allocation of anatomical structures (four of five items, MZF) as well as the symptom-oriented examination of the eye (three of three items, SFE). Discussion and conclusion: DS overall seem to be perform better in typical CMF skills. The reasons for this performance gap could be a more profound knowledge of the facial anatomy as well as a higher awareness for CMF as a related specialty to dentistry. CMF should be included in medical curricula in a larger scale, and possible career paths should be highlighted to MS and DS to raise attraction for the specialty. Further studies should focus on the implementation of modern teaching methods in CMF education.

Does quantity ensure quality? Standardized OSCE-stations for outcome-oriented evaluation of practical skills at different medical faculties

BACKGROUND Practical skills are often assessed using Objective Structured Clinical Skill Exams (OSCE). Nevertheless, in Germany, interchange and agreement between different medical faculties or a general agreement on the minimum standard for passing is lacking. METHODS We developed standardized OSCE-stations for assessing structured clinical examination of knee and shoulder joint with identical checklists and evaluation standards. These were implemented into the OSCE-course at five different medical faculties. Learning objectives for passing the stations were agreed beforehand. At each faculty, one reference examiner scored independently of the local examiner. Outcome of the students at the standardized station was compared between faculties and correlated to their total outcome at the OSCE, to their results at the Part One of the National Medical Licensing Examination as a reference test during medical studies and to their previous amount of lessons in examining joints. RESULTS Comparing the results of the reference examiner, outcome at the station differed significantly between some of the participating medical faculties. Depending on the faculty, mean total results at the knee-examination-station differed from 64.4% to 77.9% and at the shoulder-examination-station from 62.6% to 79.2%. Differences were seen in knowledge-based items and also in competencies like communication and professional manner. There was a weak correlation between outcome at the joint-examination-OSCE-station and Part One of the National Medical Licensing Examination, and a modest correlation between outcome at the joint-examination-station and total OSCE-result. Correlation to the previous amount of lessons in examining joint was also weak. CONCLUSION Although addressing approved learning objectives, different outcomes were achieved when testing a clinical skill at different medical faculties with a standardized OSCE-station. Results can be used as a tool for evaluating lessons, training and curricula at the different sites. Nevertheless, this study shows the importance of information exchange and agreement upon certain benchmarks and evaluation standards when assessing practical skills.

Effects and Consequences of Being an OSCE Examiner in Surgery—A Qualitative Study

OBJECTIVE Even though objective structured clinical examination (OSCE) is a well-investigated format for competency-based practical examination, only a few studies have explored the motivations of OSCE examiners and their opinions, both positive and negative, toward being an examiner. The aim of this study was to gain insights into the views of OSCE examiners using semi-structured interviews. DESIGN Surgical OSCE examiners were queried at two medical faculties in Germany via semi-structured interviews. The interviews were transcribed verbatim and analyzed using the techniques of structured qualitative content analysis. SETTING This study was conducted at the medical faculties of the Goethe University, Frankfurt, Germany and of the Otto-von-Guericke University, Magdeburg, Germany. PARTICIPANTS All of the study participants were surgeons working at the university hospital of one of the faculties. RESULTS A total of 29 examiners were queried until a saturation of content was achieved. A critical reflection of ones own teaching was described as a major benefit by most participants. Furthermore, they noted that the standards and competences examined during the OSCE boosted the detail of their teaching sessions in the wards. However, the examiners criticized missed operations due the examination and were not appreciated by superiors for being an examiner. Most of the examiners (22/29) preferred to be an examiner themselves rather than appointing student peer examiners. If they had appointed someone else, that would mean they would miss valuable experiences useful for their own teaching. CONCLUSIONS Being an OSCE examiner confers several advantages, notably the reflection of ones own teaching, which was described as highly valuable by the examiners.

Young surgeons’ challenges at the start of their clinical residency: a semi-qualitative study

Abstract Introduction According to German regulations on licensing to practice medicine, the aim of undergraduate medical training is to produce a scientifically and practically trained physician who is able to work independently. More precisely, medical training has to impart the required knowledge and skills in diagnostics, therapy, health promotion, prevention, and rehabilitation. From the young residents’ point of view, this aim is not achieved, and they do not feel prepared to be a doctor. However, the literature on this subject relies mostly on data based on surveys, and there is a lack of deep analysis of the specific details of the topic. The aim of this study was to analyze in depth how junior doctors in their first and second years felt about their preparation for clinical practice as a doctor from their undergraduate training, as well as which teaching formats and factors influence their preparedness. Methods This semi-qualitative study is based on recorded interviews conducted using a structured interview manual. This serves to limit the subject matter of the interview and to target the topics. The study participants were 35 residents of general and visceral surgery, trauma surgery, and urology in their first and second years of medical specialty training. The number of participants was defined by the concept of saturation of the content. Basic data regarding age and the location and length of study were collected using a questionnaire. The audio recordings were transcribed word by word and analyzed with structured qualitative content analysis techniques. Results Only 43% (n=15) of the 35 participating residents stated they were sufficiently prepared to be a doctor from undergraduate medical training, and 22.9% stated that they were not prepared for their work as a resident (8/35). However, 34.3% of the residents stated that undergraduate medical training did prepare them for some of the parts they were expected to master in daily clinical practice, but not other parts. Most of the participants described their first weeks as doctors as particularly stressful and exhausting. As major hurdles during their daily clinical work, participants described knowledge gaps regarding organizational and administrative pathways (71%), deficits in linking knowledge to clinical reasoning (71%), decision making (54%), and therapy planning (51%). Most participants stated that the practical placements during the semester, the clinical clerkships, and the last year internship were most effective as preparation for clinical residency. To be better prepared for clinical practice, participants suggested providing a clearer structure and that the course subjects bear better relations to each other. Nearly all participants proposed increasing patient encounters directly from the beginning of medical training as a longitudinal approach. Discussion Even though we were able to demonstrate an increase in residents’ preparedness, 57% of the study participants still felt unprepared for their job to some extent. One might argue that starting a new profession will always result in a feeling of being unprepared to some extent. However, this unpreparedness can increase the risk for patients’ well being due to medical errors, which actually represents the third leading cause of death in the US after malignant tumors and cardiovascular diseases. Structured on-the-job adjustment, structured qualification training, and guided professional training are becoming increasingly important for future doctors as selection criteria for career choice and choice of employer. Thus, the surgical disciplines that are struggling with new young residents have to improve their concepts.

Emotion recognition and extraversion of medical students interact to predict their empathic communication perceived by simulated patients.

BackgroundThis study assessed the impact of medical students’ emotion recognition ability and extraversion on their empathic communication, as perceived by simulated patients in a training context.MethodsThis study used a crossed-effect data structure and examined 245 students in their fourth year of medical school. The students’ personality traits were assessed based on a self-assessment questionnaire of the short form of the Big Five Inventory; their emotion recognition ability was measured using a performance test (Diagnostic Analysis of Nonverbal Accuracy-2, Adult Facial Expressions). Simulated patients evaluated the medical students’ empathic communication.ResultsStudents with a combination of high emotion recognition ability and extraversion received more positive ratings from simulated patients than their fellow students with a combination of emotion recognition ability and low extraversion. The main effects of emotion recognition or extraversion were not sufficient to yield similar effects. There were no other effects related to the remaining Big Five variables.ConclusionsThe results support the hypothesis that to build rapport with patients, medical staff need to combine emotional capabilities with a dispositional interest in interpersonal encounters.