James R. Korndorffer

Construct and face validity of MIST-VR, Endotower, and CELTS Are we ready for skills assessment using simulators?

BackgroundVideo trainers may best offer visually realistic laparoscopic simulation, whereas virtual reality (VR) modules may best provide multidimensional objective measures of performance. This study compares the construct and face validity of three different laparoscopic simulators.MethodsSubjects were voluntarily enrolled at the Learning Center during the 2004 SAGES annual meeting. Each subject completed two repetitions of a single task on each of three simulators, MIST-VR, Endotower, and CELTS; performance scores were automatically generated and recorded. Scores of individuals with various levels of experience were compared to determine construct validity for each simulator. Experience was defined according to four parameters: (a) PGY level, (b) fellowship training, (c) basic laparoscopic cases, and (d) advanced laparoscopic cases. Subjects rated each simulator regarding six face validity (realism of simulation) parameters using a 10-point Likert scale (10 = best rating) and participant scores were compared to previously established expert scores (proficiency goals for training).ResultsNinety-one attendees completed the study. Construct validity was demonstrated for all three simulators; significant differences in scores were detected according to one parameter for MIST-VR, two parameters for Endotower, and all four parameters for CELTS. Face validity was rated as good to excellent for all three simulators (7.0 ± 0.3 for MIST-VR, 7.9 ± 0.3 for Endotower [p < 0.001 vs MIST-VR], and 8.7 ± 0.1 for CELTS [p = 0.001 vs MIST-VR, p = 0.01 vs Endotower]); 6%, 0%, and 36% of “expert” participants obtained expert scores on MIST-VR, Endotower, and CELTS, respectively.ConclusionsAll three simulators demonstrated significant construct and reasonable face validity. Although virtual reality holds great promise to expand the scope of laparoscopic simulation, current interfaces may limit their utility for assessment. Computer-enhanced video trainers may offer an improved interface while incorporating useful multidimensional metrics. Further work is needed to establish standards for appropriate skills assessment methods and performance levels using simulators.

Initial Laparoscopic Basic Skills Training Shortens the Learning Curve of Laparoscopic Suturing and Is Cost-Effective

BACKGROUND Laparoscopic suturing is an advanced skill that is difficult to acquire. Simulator-based skills curricula have been developed that have been shown to transfer to the operating room. Currently available skills curricula need to be optimized. We hypothesized that mastering basic laparoscopic skills first would shorten the learning curve of a more complex laparoscopic task and reduce resource requirements for the Fundamentals of Laparoscopic Surgery suturing curriculum. STUDY DESIGN Medical students (n = 20) with no previous simulator experience were enrolled in an IRB-approved protocol, pretested on the Fundamentals of Laparoscopic Surgery suturing model, and randomized into 2 groups. Group I (n = 10) trained (unsupervised) until proficiency levels were achieved on 5 basic tasks; Group II (n = 10) received no basic training. Both groups then trained (supervised) on the Fundamentals of Laparoscopic Surgery suturing model until previously reported proficiency levels were achieved. Two weeks later, they were retested to evaluate their retention scores, training parameters, instruction requirements, and cost between groups using t-test. RESULTS Baseline characteristics and performance were similar for both groups, and 9 of 10 subjects in each group achieved the proficiency levels. The initial performance on the simulator was better for Group I after basic skills training, and their suturing learning curve was shorter compared with Group II. In addition, Group I required less active instruction. Overall time required to finish the curriculum was similar for both groups; but the Group I training strategy cost less, with a savings of

Development and transferability of a cost-effective laparoscopic camera navigation simulator

148 per trainee. CONCLUSIONS Teaching novices basic laparoscopic skills before a more complex laparoscopic task produces substantial cost savings. Additional studies are needed to assess the impact of such integrated curricula on ultimate educational benefit.

SAGES guideline for laparoscopic appendectomy

BackgroundLaparoscopic camera navigation (LCN) is vital for the successful performance of laparoscopic operations, yet little time is spent on training. This study aimed to develop an inexpensive LCN simulator, to design a structured curriculum, and to determine the transferability of skills acquired.MethodsIn this study, 0° and 30° LCN simulators were developed for use on a videotrainer platform. Transferability was tested by enrolling 20 medical students in an institutional review board-approved, randomized, controlled, blinded protocol. Subjects viewed a video tutorial and were pretested in LCN on a porcine Nissen model. Procedures were videotaped and the LCN performance was scored by a blinded rater according to the number of standardized verbal cues required and the percentage of time an optimal surgical view (%OSV) was obtained. Procedure time also was recorded. Subjects were stratified and randomized. The trained group practiced on the LCN simulator until competency was demonstrated. The control group received no training. Both groups were posttested on the porcine Nissen model.ResultsThe constructed simulators required 35 man hours for development, cost

Research regarding methods of assessing learning outcomes.

25 per board for materials, and proved to be durable. The trained group demonstrated significant improvement in verbal cues (p = 0.001), %OSV (p < 0.001), and procedure time (p = 0.001), whereas the control group showed improvement only in verbal cues (p < 0.02). At posttesting, the training group demonstrated significantly better scores for verbal cues (2.1 vs 8.0; p = 0.02) and %OSV (64% vs 45% p = 0.01) than the control group.ConclusionThese data suggest that the LCN simulator is cost effective and provides trainees with skills that translate to the operating room.

Do metrics matter? Time versus motion tracking for performance assessment of proficiency-based laparoscopic skills training

The laparoscopic approach to appendectomy has gained wide acceptance over the last 15 years as a means of improved diagnostic accuracy and wound complication rate over the open procedure. Despite a breadth of data and widespread adoption of the technique, there continues to be controversy regarding the advantages of this approach in hastening postoperative recovery, as well as its use in the management of complicated appendicitis. The following guidelines provide recommendations to surgeons for the laparoscopic management of patients with both simple and complicated appendicitis.

Effective home laparoscopic simulation training: a preliminary evaluation of an improved training paradigm

As the use of simulation-based assessment expands for healthcare workers, there is a growing need for research to quantify the psychometric properties of the associated process and outcome measures.

The American College of Surgeons/Association of Program Directors in Surgery National Skills Curriculum: adoption rate, challenges and strategies for effective implementation into surgical residency programs.

Introduction: The purpose of this study was to compare the added value of motion metrics in determining training completion during a proficiency-based simulator curriculum compared with traditional metrics (time). Methods: Novices (n = 16) practiced on a basic laparoscopic task of a hybrid simulator until expert-derived proficiency levels for time, path length, and smoothness were achieved on two consecutive attempts. The order by which proficiency in each metric was achieved was recorded and correlated to baseline characteristics. Motion metrics were considered valuable if their incorporation led to extension of training duration. Results: Compared with baseline participant performance improved at training completion according to all metrics (time 67 ± 17 to 20 ± 6 seconds; P < 0.001, pathlength 5326 ± 1444 to 2339 ± 545 cm; P < 0.001, and smoothness from 529 ± 185 to 133 ± 59; P < 0.001). Pathlength was the easiest metric to reach the proficiency level and time the most difficult. Four (33%) participants benefited from the motion metrics as their training was prolonged by an average of 50% compared with using time alone. Baseline characteristics did not correlate to the order of achievement of these metrics. Conclusions: Time may be superior to motion tracking metrics for performance assessment during proficiency-based simulator training. Nevertheless, in this study one third of trainees benefited from motion analysis metrics by having their training duration extended. Further study is needed to establish the value of motion metrics during simulator training and their impact on operating room performance improvement.

Fundamentals of endoscopic surgery cognitive examination: Development and validity evidence

BACKGROUND Laparoscopic simulation training has proven to be effective in developing skills but requires expensive equipment, is a challenge to integrate into a work-hour restricted surgical residency, and may use nonoptimal practice schedules. The purpose of this study was to evaluate the efficacy of laparoscopic skills training at home using inexpensive trainer boxes. METHODS Residents (n = 20, postgraduate years 1-5) enrolled in an institutional review board-approved laparoscopic skills training protocol. An instructional video was reviewed, and baseline testing was performed using the fundamentals of laparoscopic surgery (FLS) peg transfer and suturing tasks. Participants were randomized to home training with inexpensive, self-contained trainer boxes or to simulation center training using standard video trainers. Discretionary, goal-directed training of at least 1 hour per week was encouraged. A posttest and retention test were performed. Intragroup and intergroup comparisons as well as the relationship between the suture score and the total training sessions, the time in training, and attempts were studied. RESULTS Intragroup comparisons showed significant improvement from baseline to the posttest and the retention test. No differences were shown between the groups. The home-trained group practiced more, and the number of sessions correlated with suture retention score (r(2) = .54, P < .039). CONCLUSIONS Home training results in laparoscopic skill acquisition and retention. Training is performed in a more distributed manner and trends toward improved skill retention.

Simulation-Based Surgical EducationThe American College of Surgeons/Association of Program Directors in Surgery National Skills Curriculum: Adoption rate, challenges and strategies for effective implementation into surgical residency programs

BACKGROUND The American College of Surgeons/Association of Program Directors in Surgery (ACS/APDS) National Skills Curriculum is a 3-phase program targeting technical and nontechnical skills development. Few data exist regarding the adoption of this curriculum by surgical residencies. This study attempted to determine the rate of uptake and identify implementation enablers/barriers. METHODS A web-based survey was developed by an international expert panel of surgical educators (5 surgeons and 1 psychologist). After piloting, the survey was sent to all general surgery program directors via email link. Descriptive statistics were used to determine the residency program characteristics and perceptions of the curriculum. Implementation rates for each phase and module were calculated. Adoption barriers were identified quantitatively and qualitatively using free text responses. Standardized qualitative methodology of emergent theme analysis was used to identify strategies for success and details of support required for implementation. RESULTS Of the 238 program directors approached, 117 (49%) responded to the survey. Twenty-one percent (25/117) were unaware of the ACS/APDS curriculum. Implementation rates for were 36% for phase I, 19% for phase II, and 16% for phase III. The most common modules adopted were the suturing, knot-tying, and chest tube modules of phase I. Over 50% of respondents identified lack of faculty protected time, limited personnel, significant costs, and resident work-hour restrictions as major obstacles to implementation. Strategies for effective uptake included faculty incentives, adequate funding, administrative support, and dedicated time and resources. CONCLUSION Despite the availability of a comprehensive curriculum, its diffusion into general surgery residency programs remains low. Obstacles related to successful implementation include personnel, learner, and administrative issues. Addressing these issues may improve the adoption rate of the curriculum.