Daniel V. Girzadas

High fidelity simulation can discriminate between novice and experienced residents when assessing competency in patient care

Background: High fidelity (HF) simulators have become more common in residency training programs. Aims: We hypothesized that high fidelity (HF) simulation-based assessment of patient care competency could differentiate novice from experienced residents. Methods: Prospective study of 44 emergency medicine residents. A simulated case of anaphylactic shock was administered to each participating resident utilizing a HF patient simulator. Management of the case required epinephrine, airway management including a surgical airway, and IV fluids. Data was recorded using a standardized form and stop watch. Primary outcome: Time to completion of surgical airway. Secondary outcomes: Times to administration of epinephrine/attempt intubation/start surgical airway/complete case; checklist items: epinephrine as first action, pre-oxygenation, cricoid pressure and type of surgical airway. Results: Novice residents took significantly longer than the experienced residents to achieve our primary outcome, time to completion of surgical airway (621/512 sec; p = 0.03). The novice residents took significantly longer to acheive three of our secondary outcomes: time to start of surgical airway (534 versus 442 sec; p = 0.04), time to case completion (650 versus 513 sec; p = 0.006), and epinephrine as a first action (73% versus 100%; p = 0.02). Conclusions: HF simulation-based assessment using objective measures, particularly time to action, discerned our novice from our experienced residents. Practice points An assessment tool that can differentiate between different levels of competency demonstrates construct validity. Reports of the validity and reliability of simulator-based assessments of resident performance are limited and call for further research in this sector of medical education. The Patient Care competency encompasses data gathering, integration of diagnostic information, implementation of an effective patient management plan, competent performance of procedures and the ability to prioritize and stabilize multiple patients. Because of the complexity, Patient Care competency may be more difficult to measure than other competencies such as Medical Knowledge. HF simulation may be a useful tool for evaluating this competency. Our study demonstrated that a HF simulation-based asssessment using objective measures, particularly time to action, discerned our novice from our experienced residents. This finding of construct validity adds to the growing body of evidence that HF simulation may become a valid method of assessing Patient Care competency in residents.

Measures of stress and learning seem to be equally affected among all roles in a simulation scenario.

Introduction: Simulation scenarios provide a realistic, stressful environment in which participants can gain new clinical knowledge. It is unclear whether the role a participant plays in a scenario affects these goals. We measured heart rate, self-perceived stress and learning value, and objective written test results of participants in two simulation scenarios. Our hypothesis was that measurements of participants’ stress and learning would be similar among all team members in our simulated scenarios. Methods: Residents and medical students were prospectively randomized to take part in two difficult airway scenarios in the roles of team leader, procedure chief, or team member. Heart rate was recorded using pulse oximetry preprocedure and at the critical airway intervention. After debriefing, participants completed a data collection form that included visual analog scales for perceived stress and learning and objective questions related to scenario teaching points. Results: We obtained 53 measurements during a single day. Heart rates increased during the airway intervention (median 4 beats per minute, P = 0.04). Self-reported learning values increased with self-reported stress level (rs = 0.373, P = 0.01); however, no correlation was found between a participant’s role in the scenario and heart rate, test score, stress level, or perceived learning benefit. Conclusions: In our limited scenarios, measurements of stress and learning did not differ by role. Our results support the concept that all participants may benefit from simulation scenarios.