Andrea Carolin Lörwald

Factors influencing the educational impact of Mini-CEX and DOPS: A qualitative synthesis

Abstract Introduction: The educational impact of Mini-CEX and DOPS varies greatly and can be influenced by several factors. However, there is no comprehensive analysis and synthesis of the described influencing factors. Methods: To fill this gap, we chose a two-step approach. First, we performed a systematic literature review and selected articles describing influencing factors on the educational impact of Mini-CEX and DOPS. Second, we performed a qualitative synthesis of these factors. Results: Twelve articles were included, which revealed a model consisting of four themes and nine subthemes as influencing factors. The theme context comprises “time for Mini-CEX/DOPS” and “usability of the tools”, and influences the users. The theme users comprises “supervisors’ knowledge about how to use Mini-CEX/DOPS”, “supervisors’ attitude to Mini-CEX/DOPS”, “trainees’ knowledge about Mini-CEX/DOPS”, and “trainees’ perception of Mini-CEX/DOPS”. These influence the implementation of Mini-CEX and DOPS, including “observation” and “feedback”. The theme implementation directly influences the theme outcome, which, in addition to the educational impact, encompasses “trainees’ appraisal of feedback”. Conclusions: Our model of influencing factors might help to further improve the use of Mini-CEX and DOPS and serve as basis for future research.

The educational impact of Mini-Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) and its association with implementation: A systematic review and meta-analysis

Introduction Mini Clinical Evaluation Exercise (Mini-CEX) and Direct Observation of Procedural Skills (DOPS) are used as formative assessments worldwide. Since an up-to-date comprehensive synthesis of the educational impact of Mini-CEX and DOPS is lacking, we performed a systematic review. Moreover, as the educational impact might be influenced by characteristics of the setting in which Mini-CEX and DOPS take place or their implementation status, we additionally investigated these potential influences. Methods We searched Scopus, Web of Science, and Ovid, including All Ovid Journals, Embase, ERIC, Ovid MEDLINE(R), and PsycINFO, for original research articles investigating the educational impact of Mini-CEX and DOPS on undergraduate and postgraduate trainees from all health professions, published in English or German from 1995 to 2016. Educational impact was operationalized and classified using Barr’s adaptation of Kirkpatrick’s four-level model. Where applicable, outcomes were pooled in meta-analyses, separately for Mini-CEX and DOPS. To examine potential influences, we used Fisher’s exact test for count data. Results We identified 26 articles demonstrating heterogeneous effects of Mini-CEX and DOPS on learners’ reactions (Kirkpatrick Level 1) and positive effects of Mini-CEX and DOPS on trainees’ performance (Kirkpatrick Level 2b; Mini-CEX: standardized mean difference (SMD) = 0.26, p = 0.014; DOPS: SMD = 3.33, p<0.001). No studies were found on higher Kirkpatrick levels. Regarding potential influences, we found two implementation characteristics, “quality” and “participant responsiveness”, to be associated with the educational impact. Conclusions Despite the limited evidence, the meta-analyses demonstrated positive effects of Mini-CEX and DOPS on trainee performance. Additionally, we revealed implementation characteristics to be associated with the educational impact. Hence, we assume that considering implementation characteristics could increase the educational impact of Mini-CEX and DOPS.

Modern Medical Moulage in Health Professions Education

Medical moulage is the art of reproducing a healthy or afflicted body part as macroscopic model for the purpose of documentation, training or assessment. Casts, additionally modified by colouring or other techniques, and wax moulages have served as models since ancient times until photography was established. Modern moulage includes a complex combination of silicone casts, temporary tattoos, professional make-up, and other techniques to depict particular signs of disease on a healthy person or on a mannequin in simulation-based education. In the literature a comprehensive overview about the use of modern medical moulage in health professions education is missing. We aim to provide medical educators with such an overview of this promising aspect of simulation in education, describing how the moulages are being used in health professions education and assessment, and how the use of moulage is evaluated regarding its effectiveness. Hits for the search term “moulage” in the journal websites’ search option of InCites Journal Citation Report listed journals of dermatology (all, n=63) and education journals (health related, including nursing and military medicine, n=22) constituted an initial data basis and included publications up until December 2017. Original research articles in English and German describing the use of modern medical moulage in health professions education (teaching and assessment) or evaluating such use were included. Based on these search results, iterative forward search (sources citing an included hit) as well as iterative backward search (sources cited within an included hit) was performed until no more articles matching inclusion criteria were found. For inclusion, articles had to specify either the depicted condition, information on whom the moulages were used, the training’s or assessment’s target population, the educational scenario in which moulages were used, or the mode of moulage evaluation. In sum, 50 articles were included in this review. The use of modern moulage included the simulation of signs of trauma like hematoma, abrasions, lacerations, burns, open and closed fractures, evisceration, or amputations and, in a more dermatological setting, integumentary diseases such as melanoma, herpes, psoriasis, diabetic ulcer or Ebola rashes. Moulage was either applied to mannequins or simulated patients. Target populations covered undergraduate and postgraduate learners in health professions including nursing students and registered nurses, medical students, residents, physicians, pharmacy students, emergency medical technicians, firemen, first responders, and army and navy personnel. 39 studies reported the use of moulage in teaching settings. These were predominantly instances of clinical skills teaching, but a different, noteworthy application was to facilitate affective learning objectives by applying the moulage to the students themselves and letting them experience certain illnesses. Eleven studies described the use of moulage for assessment purposes. In both, teaching and assessment, moulages were evaluated regarding the extent to which they looked realistic and the extent to which their use enhanced the credibility of the simulation. For teaching, moulage was additionally evaluated regarding its usability and teaching effectiveness. Different teaching modalities (with or without or with different moulages) were compared regarding their effect on learners’ self-efficacy and performance. In assessment, moulages were evaluated indirectly by the extent to which simulated lesions (like incidental melanoma) were correctly detected by the examinees. In this review, we analysed and describe how modern moulage is used in health professions education. A certain degree of validity can be argued for both teaching and assessment settings. The range of signs and symptoms being simulated with moulage is probably wider then could be reported here, since the details of moulage simulation are often not elaborated upon in literature. As in other simulation settings, there is little understanding so far, as to how much simulation authenticity is necessary, i.e. when moulage application is worth the effort and makes a meaningful difference. On a more practical side, studies have demonstrated that moulage produced by professional make-up artists is of higher quality than health professions educators’ own creations. Therefore, we suggest there should be more opportunities for health professions educators to professionalize themselves and we highlight the value of exchanging recipes and techniques of preparing modern medical moulage. Lastly, we emphasize the educational potential of modern medical moulage, as it offers possibilities well beyond clinical skills training.

The Authors reply: Factors influencing the educational impact of mini-CEX and DOPS

We thank Evans and Lambrinudi for sharing their thoughts and ideas on our article “Factors influencing the educational impact of Mini-CEX and DOPS: A qualitative synthesis” (L€ orwald et al. 2017). One aspect in their letter struck us as particularly noteworthy. The authors observe how in the program in which they are enrolled, students regularly reflect on their clinical encounters from their first year of training, until this reflection seemingly becomes almost a second nature to them. They then hypothesize that this “entrenched need to reflect on all experiences” improves their attitudes and perceptions toward completing assessments such as Mini-CEX and DOPS. There are, in fact, hints in the literature that support their hypothesis. Back in 1986, Dweck studied how motivational processes affect learning (Dweck 1986). She found that one’s goal orientation determines one’s learning behavior. She distinguished between two kinds of goal orientation: people with a performance goal orientation aim to receive positive judgments and avoid criticism; people with a learning goal orientation instead aim to increase their competence. This translates to the assumption that students with a learning goal orientation persistently seek new challenges, while students with a performance goal orientation might prefer easy tasks for their workplace-based assessment in order to demonstrate their competency and avoid new challenges. Reflecting upon clinical encounters might foster students’ learning goal orientation, encourage them to seek new challenges and welcome feedback, and increase the impact of Mini-CEX and DOPS (Sargeant et al. 2009; Ramani et al. 2018). We would like to congratulate the authors, their teachers and their program directors for having achieved a program that fosters reflection and nourishes students’ feedback-seeking behavior.