Donna Stanbridge

Proving the value of simulation in laparoscopic surgery

Objective:To assess the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) physical laparoscopic simulator for construct and predictive validity and for its educational utility. Summary Background Data:MISTELS is the physical simulator incorporated by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in their Fundamentals of Laparoscopic Surgery (FLS) program. MISTELS’ metrics have been shown to have high interrater and test-retest reliability and to correlate with skill in animal surgery. Methods:Over 200 surgeons and trainees from 5 countries were assessed using MISTELS in a series of experiments to assess the validity of the system and to evaluate whether practicing MISTELS basic skills (transferring) would result in skill acquisition transferable to complex laparoscopic tasks (suturing). Results:Face validity was confirmed through questioning 44 experienced laparoscopic surgeons using global rating scales. MISTELS scores increased progressively with increasing laparoscopic experience (n = 215, P < 0.0001), and residents followed over time improved their scores (n = 24, P < 0.0001), evidence of construct validity. Results in the host institution did not differ from 5 beta sites (n = 215, external validity). MISTELS scores correlated with a highly reliable validated intraoperative rating of technical skill during laparoscopic cholecystectomy (n = 19, r = 0.81, P < 0.0004; concurrent validity). Novice laparoscopists were randomized to practice/no practice of the transfer drill for 4 weeks. Improvement in intracorporeal suturing skill was significantly related to practice but not to baseline ability, career goals, or gender (P < 0.001). Conclusion:MISTELS is a practical and inexpensive inanimate system developed to teach and measure technical skills in laparoscopy. This system is reliable, valid, and a useful educational tool.

Evaluating laparoscopic skills

Background: The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) was developed to assess laparoscopic skills and to score them objectively. This system has been described previously. The purpose of the current study was to determine a pass/fail threshold. Methods: In this study. 165 individuals were tested and grouped according to their clinical competency in laparoscopic surgery. The noncompetent group consisted of medical students and surgical residents in their first 2 years of training (n = 83). The competent group consisted of chief general surgical residents in their last year of training, laparoscopy fellows. and practicing laparoscopic surgeons (n = 82). The Mann-Whitney U test was used to evaluate differences in task performance between the two groups. Results: There was a significant difference in total scores and individual MISTELS task scores between the noncompetent and competent laparoscopic surgeons (189 vs 372.5; p < 0.0001). By setting specific pass/fail total score thresholds (cutoff scores), competent surgeons can be discriminated from noncompetent surgeons. Conclusion: An objective pass/fail evaluation can be given to individuals tested with the MISTELS system.

The MISTELS program to measure technical skill in laparoscopic surgery

BackgroundThe McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) is a series of five tasks with an objective scoring system. The purpose of this study was to estimate the interrater and test–retest reliability of the MISTELS metrics and to assess their internal consistency.MethodsTo determine interrater reliability, two trained observers scored 10 subjects, either live or on tape. Test–retest reliability was assessed by having 12 subjects perform two tests, the second immediately following the first. Interrater and test–retest reliability were assessed using intraclass correlation coefficients. Internal consistency between tasks was estimated using Cronbach’s alpha.ResultsThe interrater and test–retest reliabilities for the total scores were both excellent at 0.998 [95% confidence interval (CI), 0.985–1.00] and 0.892 (95% CI, 0.665–0.968), respectively. Cronbach’s alpha for the first assessment of the test–retest was 0.86.ConclusionsThe MISTELS metrics have excellent reliability, which exceeds the threshold level of 0.8 required for high-stakes evaluations. These findings support the use of MISTELS for evaluation in many different settings, including residency training programs.

FLS simulator performance predicts intraoperative laparoscopic skill

IntroductionSimulators are being used more and more for teaching and testing laparoscopic skills. However, it has yet to be firmly established that simulator performance reflects operative laparoscopic skill. The study reported here was designed to test the hypothesis that laparoscopic simulator performance predicts intraoperative laparoscopic skill.MethodsA review of our prospectively maintained database identified 40 subjects who underwent Fundamentals of Lapraoscopic Surgery (FLS) skills testing and objective intraoperative assessments within the same 6-month period. Subjects consisted of 22 novice (postgraduate year [PGY] 1–2), 10 intermediate (PGY 3–4), and 8 experienced (PGY 5, fellows, and attendings) laparoscopic surgeons. Laparoscopic performance was objectively assessed in the operating room using the previously validated Global Operative Assessment of Laparoscopic Skill (GOALS). Analysis of variance (ANOVA) was used to compare mean FLS scores and mean GOALS scores across experience levels. The relationship between individual FLS scores and GOALS scores was assessed with linear regression analysis. A multivariate analysis evaluated FLS score and surgeon experience as predictors of intraoperative GOALS score. A receiver-operator curve (ROC) was constructed in order to define an FLS cutoff score that predicts intraoperative performance at or above the level of experienced surgeons. Significance was defined as p < 0.05.ResultsMean FLS scores and mean GOALS scores increased with increasing experience. Individual FLS scores correlated significantly with intraoperative GOALS scores (0.77, p < 0.001). Multivariate analysis confirmed that FLS score is an independent predictor of intraoperative GOALS scores. The ROC identified an FLS cutoff score of 70 with optimal sensitivity (91%) and specificity (86%) for predicting a GOALS score at or above the level of experienced surgeons.ConclusionsIn this study sample, FLS simulator scores were independently predictive of intraoperative laparoscopic performance as measured by GOALS. More precisely, an FLS cutoff score of 70 optimized sensitivity and specificity for expert intraoperative performance. A larger prospective study is justified to validate these findings.

Assessing the learning curve for the acquisition of laparoscopic skills on a virtual reality simulator

BackgroundThe aim of this study was to develop summary metrics and assess the construct validity for a virtual reality laparoscopic simulator (LapSim) by comparing the learning curves of three groups with different levels of laparoscopic expertise.MethodsThree groups of subjects (‘expert’, ‘junior’, and ‘naïve’) underwent repeated trials on three LapSim tasks. Formulas were developed to calculate scores for efficiency (‘time-error’) and economy of ‘motion’ (‘motion’) using metrics generated by the software after each drill. Data (mean ± SD) were evaluated by analysis of variance (ANOVA). Significance was set at p < 0.05.ResultsAll three groups improved significantly from baseline to final for both ‘time-error’ and ‘motion’ scores. There were significant differences between groups in time error performances at baseline and final, due to higher scores in the ‘expert’ group. A significant difference in ‘motion’ scores was seen only at baseline.ConclusionWe have developed summary metrics for the LapSim that differentiate among levels of laparoscopic experience. This study also provides evidence of construct validity for the LapSim.

Should laparoscopic paraesophageal hernia repair be abandoned in favor of the open approach

BackgroundThe most appropriate approach to the repair of large paraesophageal hernias remains controversial. Despite early results of excellent outcomes after laparoscopic repair, recent reports of high recurrence require that this approach be reevaluated.MethodsFor this study, 60 primary paraesophageal hernias consecutively repaired at one institution from 1990 to 2002 were reviewed. These 25 open transabdominal and 35 laparoscopic repairs were compared for operative, short-, and long-term outcomes on the basis of quality-of -life questionnaires and radiographs.ResultsNo difference in patient characteristics was detected. Laparoscopic repair resulted in lower blood loss, fewer intraoperative complications, and a shorter length of hospital stay. No difference in general or disease-specific quality-of-life was documented. Radiographic follow-up was available for 78% open and 91% laparoscopic repairs, showing anatomic recurrence rates of 44% and 23%, respectively (p = 0.11).ConclusionsLaparoscopic repair should remain in the forefront for the management of paraesophageal hernias. However, there is considerable room for improvement in reducing the incidence of recurrence.

Characterizing the learning curve for a basic laparoscopic drill

Background:The psychomotor challenges inherent in laparoscopic surgery are evident by the steep procedural “learning curves” documented throughout the literature. Few methods have been described to evaluate learning curves. The cumulative summation (CUSUM) method is a criterion-based evaluation of the learning process. The purpose of this study is to describe the CUSUM learning curves for a simple task for individuals and for a group of novice laparoscopists.Methods:Sixteen medical students undertook four weekly sessions of 10 laparoscopic pegboard transfers in the MISTELS system. Their performance was scored and recorded for each trial. CUSUM learning curves were constructed based on the goal of achieving mean scores for senior, intermediate, or junior laparoscopists ≥95% of the time.Results:Based on senior criteria, one student achieved the goal by the 40th peg transfer trial. Based on intermediate criteria, three students achieved the goal by their 40th trial (trials 21 and 36), and for junior criteria, 10 students achieved the acceptable success rate by their 40th trial (range, 26–40).Conclusion:CUSUM analysis suggests criterion-based practice is useful for novice laparoscopists. It allows educators to track the progress of an individual toward target criteria for each MISTELS task, to more logically allocate time for training and set attainable goals, to objectively evaluate trainee acquisition of basic laparoscopic skills, and to identify trainees who need remediation.

Evaluating intraoperative laparoscopic skill: direct observation versus blinded videotaped performances.

The Global Operative Assessment of Laparoscopic Skill (GOALS) has been shown to meet high standards for direct observation. The purpose of this study was to investigate the reliability and validity of GOALS when applied to blinded, videotaped performances. Five novice surgeons and 5 experienced surgeons were each evaluated by 2 observers during a laparoscopic cholecystectomy. Subsequently, 4 laparoscopists (V1 to V4) evaluated the videotaped procedures using GOALS. Two of the raters (V1 and V3) had prior experience using GOALS. The interrater reliabilities between video raters (VRs) and between VRs and direct raters (DRs) were calculated using the intraclass correlation coefficient. Construct validity was assessed using 2-way analysis of variance. Interrater reliability between the 4 VRs and the 2 DRs was 0.72. The intraclass correlation coefficient for the 4 VRs was 0.68 and for each VR compared with the mean DR was 0.86, 0.39, 0.94, and 0.76, respectively. All raters, except V2, differentiated between novice and experienced groups (P values ranged from .01 to .05). These data suggest that GOALS can be used to assess laparoscopic skill based on videotaped performances but that rater training may play an important role in ensuring the reliability and validity of the instrument. Experience with the tool in the operating room may improve the reliability of video rating and could be of value in training evaluators.

Relationship between subjective and objective outcome measures after Heller myotomy and Dor fundoplication for achalasia.

BackgroundThe purpose of this study is to assess how subjective evaluation (heartburn, dysphagia, quality of life, and satisfaction) correlates with objective data after Heller myotomy and Dor fundoplication for achalasia.MethodsA total of 53 consecutive patients with achalasia undergoing laparoscopic Heller myotomy and Dor fundoplication were studied prospectively. Subjective evaluation was done preop and postop using the Gastroesophageal Reflux Disease Health-Related Quality of Life instrument (GERD-HRQL; 0 = best, 45 = worse), 4-point dysphagia and heartburn scales (0 = best, 3 = worst), patient satisfaction scale (0 = very satisfied, 5 = incapacitated), and the SF-12 general health-related quality-of-life score. At 3 months postop, patients were asked to undergo objective evaluation with 24-h pH testing, manometry, and endoscopy. Data are expressed as median (interquartile range) and analyzed by Wilcoxon signed rank test or Mann–Whitney U test.ResultsForty-nine patients were more than 3 months postsurgery. Comparing preop to postop, improvements were found in dysphagia [3 (2–3) to 0 (0–1)], heartburn [1 (0–2) to 0 (0–1)], GERD-HRQL [13.5 (6.3–22.5) to 2 (0–5)], satisfaction [3 (3–4) to 1 (0–1)], and SF-12 mental component summary [46 (37–56) to 58 (50–59)] and physical component summary [46 (36–53) to 55 (48–56)] scores (p < 0.0001 for all). Thirty-eight patients (78%) agreed to undergo objective testing, and complete data were available for 32 (65%). Four of 32 patients (12.5%) had evidence of reflux based on 24-h pH testing. Of nine patients with GERD-HRQL >5, only two had positive pH test (22%). Of 23 patients with GERD-HRQL <5, two had positive pH test (7%). Of four tested patients with moderate to severe heartburn, two had an abnormal pH test. There was no significant relationship between GERD-HRQL score and pH test results. Lower esophageal sphincter pressure (LESP) decreased from 24 (16–35) to 13 mmHg (11–17) (p < 0.001). There was no relationship between dysphagia score and postop absolute LESP or a decrease in LESP after operation.ConclusionsLaparoscopic Heller myotomy and Dor fundoplication is an effective treatment for achalasia. Subjective evaluation can document patient satisfaction and health-related quality of life but does not accurately reflect postop reflux. Twenty-four-hour pH study is required to accurately assess reflux disease.

Optimization of cardiac preload during laparoscopic donor nephrectomy: A preliminary study of central venous pressure versus esophageal doppler monitoring

Background: While the popularity of laparoscopic donor nephrectomy (LDN) has increased, concern persists about the potential deleterious effects of pneumoperitoneum on renal function. Thus, preload optimization with vigorous intravenous hydration has been recommended. The purpose of this study was to compare central venous pressure (CVP) monitoring with a noninvasive measure of cardiac preload (esophageal Doppler) during LDN. Methods: Thirteen patients were studied. Following induction of general anesthesia, a Doppler probe was inserted in the lower third of the esophagus to measure flow time corrected for heart rate (FTc), which is an index of preload. In 10 patients, a catheter was placed in the right internal jugular vein and CVP measured. CVP and FTc were measured at baseline in the supine and right lateral decubitus positions, then 15 and 60 min after the establishment of CO2 pneumoperitoneum (12–15 mmHg). IV fluids were increased if the FTc fell below 300 msec. Results are expressed as means (±SD). Data were analyzed using repeated measures ANOVA. Results: Lateral positioning and pneumoperitoneum significantly increased CVP from baseline (p < 0.01), while the FTc did not change (p = 0.57). After 60 min of pneumoperitoneum, the FTc was <300 msec in only one patient. Conclusion: CVP is not an accurate guide for administration of IV fluids during LDN. Esophageal Doppler monitoring can be used to noninvasively follow changes in preload during LDN and is worthy of further study.