Jan-Joost Rethans

The use of simulated patients in medical education: AMEE Guide No 42

Medical training has traditionally depended on patient contact. However, changes in healthcare delivery coupled with concerns about lack of objectivity or standardization of clinical examinations lead to the introduction of the ‘simulated patient’ (SP). SPs are now used widely for teaching and assessment purposes. SPs are usually, but not necessarily, lay people who are trained to portray a patient with a specific condition in a realistic way, sometimes in a standardized way (where they give a consistent presentation which does not vary from student to student). SPs can be used for teaching and assessment of consultation and clinical/physical examination skills, in simulated teaching environments or in situ. All SPs play roles but SPs have also been used successfully to give feedback and evaluate student performance. Clearly, given this potential level of involvement in medical training, it is critical to recruit, train and use SPs appropriately. We have provided a detailed overview on how to do so, for both teaching and assessment purposes. The contents include: how to monitor and assess SP performance, both in terms of validity and reliability, and in terms of the impact on the SP; and an overview of the methods, staff costs and routine expenses required for recruiting, administrating and training an SP bank, and finally, we provide some intercultural comparisons, a ‘snapshot’ of the use of SPs in medical education across Europe and Asia, and briefly discuss some of the areas of SP use which require further research.

Assessing simulated patients in an educational setting: the MaSP (Maastricht Assessment of Simulated Patients)

Context  For more than two decades the Medical School in Maastricht, the Netherlands, has used simulated patients (SPs) to provide students with opportunities to practise their skills in communication and physical examination. In this educational setting a student meets a SP in a videotaped session. Feedback by the SP to the student at the end of the session is considered an important educational feature. We found no instruments to assess individual SP performance during those sessions.

Students' views on the use of real patients and simulated patients in undergraduate medical education.

Purpose To determine students’ views about the strengths and weaknesses of real patient interactions as opposed to simulated patient (SP) interactions in the undergraduate medical curriculum in order to evaluate how their strengths can be optimally used and weaknesses remedied. Method Five focus-group interviews were conducted among fourth- and fifth-year medical students at Maastricht University in 2007, using a preestablished interview guide. The interviews were recorded, transcribed, and analyzed using qualitative methods. Results In general, the 38 participants considered real patient encounters more instructive and more authentic than SP encounters. However, students identified several strengths of SP encounters compared with real patient encounters. For example, SP interactions were helpful in preparing students for real patient interactions (particularly with regard to communication skills and self-confidence), in the teaching of “intimate” physical examination skills, such as gynecological examination skills, and in giving constructive feedback on communication skills. In contrast to what we had anticipated, taking a time-out was considered easier in real patient interactions. Conclusions Both real patient interactions and SP interactions are considered indispensable to undergraduate medical education. Each encounter has unique strengths and weaknesses from the perspectives of students. On the basis of strengths and weaknesses that were identified, suggestions were made for the use of real patients and SPs in undergraduate medical education.

Feedback by simulated patients in undergraduate medical education: a systematic review of the literature.

Objectives  Although the importance of feedback by simulated patients (SPs) is generally recognised, knowledge is scarce about the most effective ways in which SPs can provide feedback. In addition, little is known about how SPs are trained to provide feedback. This study aimed to provide a systematic overview of the ways in which SPs provide feedback to undergraduate medical students, the domains in which SPs provide feedback and the ways in which SPs are trained to provide feedback.

Assessment in general practice: the predictive value of written-knowledge tests and a multiple-station examination for actual medical performance in daily practice.

This study compares the predictive values of written‐knowledge tests and a standardized multiple‐station examination for the actual medical performance of general practitioners (GPs) in order to select effective assessment methods to be used in quality‐improvement activities.

Unannounced standardised patients in real practice: a systematic literature review.

Background No attempt has been made to give a systematic overview of the innovative use of unannounced or incognito standardised patients.

Assessment for simulation learning outcomes: A comparison of knowledge and self-reported confidence with observed clinical performance

BACKGROUND With extensive use of simulation in nursing education, researchers around the world are evaluating learning outcomes from simulation. Numerous studies reported the use of knowledge tests and self-reported measures to evaluate simulation outcomes. AIM To determine whether self-reported confidence and knowledge measures are indicators of clinical performance observed in a simulation-based assessment. METHOD Thirty-one third year nursing students were randomized into intervention and control group. The intervention group received a six hour simulation-based programme in care of a patient with physiological deterioration. Pre and post-tests using knowledge test, confidence scale and simulation-based assessment were conducted immediately before and after the simulation program. RESULTS The intervention group had a significantly higher post-test mean score than the control group for knowledge and clinical performances. Both groups demonstrated a significant improvement on post-test scores from pre-test scores for self-confidence with no significant differences detected among the two groups. Correlation tests indicated no significant correlation between self-confidence and clinical performance, and between knowledge and clinical performance. CONCLUSION The study did not provide evidence to support the validity of the knowledge test and self-confidence measures for predicting clinical performance. Most importantly, it revealed potential danger of a simulation-based assessment that could lead toward overestimation of self-confidence.

Assessment of competence in technical clinical skills of general practitioners

Technical clinical procedures constitute an important part of the work of general practitioners. Assessment of competence in the relevant skills is important from the perspective of quality assurance. In this study, the psychometric characteristics of three different methods for assessment of competence in technical clinical skills in general practice were evaluated. A performance‐based test (8 stations), a written knowledge test of skills (125 items) and a self‐assessment questionnaire (41 items) on technical clinical skills were administered to 49 GPs and 47 trainees in general practice. The mean scores on the performance‐based test and the written knowledge test of skills showed no substantial differences between GPs and trainees, whereas the GPs scored higher on the self‐assessment questionnaire. While the correlation of the score on the knowledge test of skills with the score on the performance‐based test was moderately high, the score on the self‐assessment questionnaire showed a rather low correlation with the performance‐based test. Although performance‐based testing is obviously the best method to assess proficiency in hands‐on skills, a written test can serve as a reasonable alternative, particularly for screening and research purposes.

Strengths and Weaknesses of Simulated and Real Patients in the Teaching of Skills to Medical Students: A Review

The aim of this review was to identify the strengths and weaknesses of the roles of real and simulated patients in undergraduate medical education. The literature was reviewed in relation to four patient roles: real patients as educational “resource” (passive role), real patients as teachers (active role), and simulated patients as educational resource and teachers. Each of the four patient roles was found to have specific advantages and disadvantages from the perspectives of teachers, students, and patients. For example, advantages of real patients as educational resource were patient-centered learning and high patient satisfaction. Disadvantages were their limited availability and the variability in learning experiences among students. Despite the considerable amount of literature we found, many gaps in knowledge about patient roles in medical education remain and should be addressed by future studies.

Evaluation of cardiopulmonary resuscitation skills of general practitioners using different scoring methods

In this study we evaluated the practical performance of 70 general practitioners in cardiopulmonary resuscitation (CPR) before and after instruction and compared checklist-based scores to mechanical recording scores in order to investigate which scoring method is preferable. Both checklist and recording strip-based scores showed significant improvement after instruction, but only 37% were judged proficient according to the American Heart Association standards (checklist scoring), and 47% according to the recording print-based scoring system, while rates judged 97% as satisfactory by general impression. Interrater reliability was highest for the recording print (0.97) and lower for the checklist (0.79), especially for CPR performance (0.56). Comparison of checklist and recording print showed that the checklist was specific but not very sensitive in identifying poor performance for cardiac compression rate, since observers overestimated performance. The correlation for CPR performance between checklist score and recording strip score was low (0.45), indicating that candidates were ranked differently. The correlation between diagnosis and performance score was low for checklist as well as recording print (0.22), indicating that the score on diagnosis was a poor predictor for the score on performance of CPR. These results support the use of the recording manikin as compared with the use of a checklist for formative evaluation of basic life support skills. However, as proficiency in diagnosis and performance in CPR are poorly correlated, assessment of diagnosis using a checklist must be included. Therefore we strongly recommend the combination of assessment by observers using a checklist for diagnostic procedures and the recording strip of the manikin for performance of CPR, as employed in most evaluation schemes.