Bernd Bender

Examiner effect on the objective structured clinical exam - a study at five medical schools

BackgroundThe Objective Structured Clinical Examination (OSCE) is increasingly used at medical schools to assess practical competencies. To compare the outcomes of students at different medical schools, we introduced standardized OSCE stations with identical checklists.MethodsWe investigated examiner bias at standardized OSCE stations for knee- and shoulder-joint examinations, which were implemented into the surgical OSCE at five different medical schools. The checklists for the assessment consisted of part A for knowledge and performance of the skill and part B for communication and interaction with the patient. At each medical faculty, one reference examiner also scored independently to the local examiner. The scores from both examiners were compared and analysed for inter-rater reliability and correlation with the level of clinical experience. Possible gender bias was also evaluated.ResultsIn part A of the checklist, local examiners graded students higher compared to the reference examiner; in part B of the checklist, there was no trend to the findings. The inter-rater reliability was weak, and the scoring correlated only weakly with the examiner’s level of experience. Female examiners rated generally higher, but male examiners scored significantly higher if the examinee was female.ConclusionsThese findings of examiner effects, even in standardized situations, may influence outcome even when students perform equally well. Examiners need to be made aware of these biases prior to examining.

Undergraduate medical students need more training in craniomaxillofacial surgery: a comparative study between medical and dental students

Abstract Purpose: To compare the performance in oral, craniomaxillofacial, and facial plastic surgery (CMF)-specific surgical skills between medical students (MS) and dental students (DS) and hence adjust the current CMF training to student-specific needs. The investigators hypothesized that there would be no performance differences between MS and DS. Methods: The investigators implemented a comparative retrospective item-based analysis of student performance in a CMF-specific objective structured clinical examination (OSCE) from 2008 to 2015. The sample was composed of 1010 MS and 225 DS who completed a standardized CMF training and OSCE. Three OSCE scenarios [management mandible fracture (MMF), management zygomatic fracture (MZF), and structured facial examination (SFE)] were included in the study because learning objectives were equal. Descriptive and bivariate statistics were computed and the p value was set at 0.05. Results: In all of the analyzed OSCE scenarios, DS significantly outperformed MS (MMF p<0.001; MZF p=0.013; SFE p<0.001). DS especially appeared to be better in the correct interpretation of radiological findings (five of seven items, MMF) and the correct allocation of anatomical structures (four of five items, MZF) as well as the symptom-oriented examination of the eye (three of three items, SFE). Discussion and conclusion: DS overall seem to be perform better in typical CMF skills. The reasons for this performance gap could be a more profound knowledge of the facial anatomy as well as a higher awareness for CMF as a related specialty to dentistry. CMF should be included in medical curricula in a larger scale, and possible career paths should be highlighted to MS and DS to raise attraction for the specialty. Further studies should focus on the implementation of modern teaching methods in CMF education.

Does quantity ensure quality? Standardized OSCE-stations for outcome-oriented evaluation of practical skills at different medical faculties

BACKGROUND Practical skills are often assessed using Objective Structured Clinical Skill Exams (OSCE). Nevertheless, in Germany, interchange and agreement between different medical faculties or a general agreement on the minimum standard for passing is lacking. METHODS We developed standardized OSCE-stations for assessing structured clinical examination of knee and shoulder joint with identical checklists and evaluation standards. These were implemented into the OSCE-course at five different medical faculties. Learning objectives for passing the stations were agreed beforehand. At each faculty, one reference examiner scored independently of the local examiner. Outcome of the students at the standardized station was compared between faculties and correlated to their total outcome at the OSCE, to their results at the Part One of the National Medical Licensing Examination as a reference test during medical studies and to their previous amount of lessons in examining joints. RESULTS Comparing the results of the reference examiner, outcome at the station differed significantly between some of the participating medical faculties. Depending on the faculty, mean total results at the knee-examination-station differed from 64.4% to 77.9% and at the shoulder-examination-station from 62.6% to 79.2%. Differences were seen in knowledge-based items and also in competencies like communication and professional manner. There was a weak correlation between outcome at the joint-examination-OSCE-station and Part One of the National Medical Licensing Examination, and a modest correlation between outcome at the joint-examination-station and total OSCE-result. Correlation to the previous amount of lessons in examining joint was also weak. CONCLUSION Although addressing approved learning objectives, different outcomes were achieved when testing a clinical skill at different medical faculties with a standardized OSCE-station. Results can be used as a tool for evaluating lessons, training and curricula at the different sites. Nevertheless, this study shows the importance of information exchange and agreement upon certain benchmarks and evaluation standards when assessing practical skills.

Effects and Consequences of Being an OSCE Examiner in Surgery—A Qualitative Study

OBJECTIVE Even though objective structured clinical examination (OSCE) is a well-investigated format for competency-based practical examination, only a few studies have explored the motivations of OSCE examiners and their opinions, both positive and negative, toward being an examiner. The aim of this study was to gain insights into the views of OSCE examiners using semi-structured interviews. DESIGN Surgical OSCE examiners were queried at two medical faculties in Germany via semi-structured interviews. The interviews were transcribed verbatim and analyzed using the techniques of structured qualitative content analysis. SETTING This study was conducted at the medical faculties of the Goethe University, Frankfurt, Germany and of the Otto-von-Guericke University, Magdeburg, Germany. PARTICIPANTS All of the study participants were surgeons working at the university hospital of one of the faculties. RESULTS A total of 29 examiners were queried until a saturation of content was achieved. A critical reflection of ones own teaching was described as a major benefit by most participants. Furthermore, they noted that the standards and competences examined during the OSCE boosted the detail of their teaching sessions in the wards. However, the examiners criticized missed operations due the examination and were not appreciated by superiors for being an examiner. Most of the examiners (22/29) preferred to be an examiner themselves rather than appointing student peer examiners. If they had appointed someone else, that would mean they would miss valuable experiences useful for their own teaching. CONCLUSIONS Being an OSCE examiner confers several advantages, notably the reflection of ones own teaching, which was described as highly valuable by the examiners.

Emotion recognition and extraversion of medical students interact to predict their empathic communication perceived by simulated patients.

BackgroundThis study assessed the impact of medical students’ emotion recognition ability and extraversion on their empathic communication, as perceived by simulated patients in a training context.MethodsThis study used a crossed-effect data structure and examined 245 students in their fourth year of medical school. The students’ personality traits were assessed based on a self-assessment questionnaire of the short form of the Big Five Inventory; their emotion recognition ability was measured using a performance test (Diagnostic Analysis of Nonverbal Accuracy-2, Adult Facial Expressions). Simulated patients evaluated the medical students’ empathic communication.ResultsStudents with a combination of high emotion recognition ability and extraversion received more positive ratings from simulated patients than their fellow students with a combination of emotion recognition ability and low extraversion. The main effects of emotion recognition or extraversion were not sufficient to yield similar effects. There were no other effects related to the remaining Big Five variables.ConclusionsThe results support the hypothesis that to build rapport with patients, medical staff need to combine emotional capabilities with a dispositional interest in interpersonal encounters.