Shannon A. Fraser

Proving the value of simulation in laparoscopic surgery

Objective:To assess the McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) physical laparoscopic simulator for construct and predictive validity and for its educational utility. Summary Background Data:MISTELS is the physical simulator incorporated by the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) in their Fundamentals of Laparoscopic Surgery (FLS) program. MISTELS’ metrics have been shown to have high interrater and test-retest reliability and to correlate with skill in animal surgery. Methods:Over 200 surgeons and trainees from 5 countries were assessed using MISTELS in a series of experiments to assess the validity of the system and to evaluate whether practicing MISTELS basic skills (transferring) would result in skill acquisition transferable to complex laparoscopic tasks (suturing). Results:Face validity was confirmed through questioning 44 experienced laparoscopic surgeons using global rating scales. MISTELS scores increased progressively with increasing laparoscopic experience (n = 215, P < 0.0001), and residents followed over time improved their scores (n = 24, P < 0.0001), evidence of construct validity. Results in the host institution did not differ from 5 beta sites (n = 215, external validity). MISTELS scores correlated with a highly reliable validated intraoperative rating of technical skill during laparoscopic cholecystectomy (n = 19, r = 0.81, P < 0.0004; concurrent validity). Novice laparoscopists were randomized to practice/no practice of the transfer drill for 4 weeks. Improvement in intracorporeal suturing skill was significantly related to practice but not to baseline ability, career goals, or gender (P < 0.001). Conclusion:MISTELS is a practical and inexpensive inanimate system developed to teach and measure technical skills in laparoscopy. This system is reliable, valid, and a useful educational tool.

Evaluating laparoscopic skills

Background: The McGill Inanimate System for Training and Evaluation of Laparoscopic Skills (MISTELS) was developed to assess laparoscopic skills and to score them objectively. This system has been described previously. The purpose of the current study was to determine a pass/fail threshold. Methods: In this study. 165 individuals were tested and grouped according to their clinical competency in laparoscopic surgery. The noncompetent group consisted of medical students and surgical residents in their first 2 years of training (n = 83). The competent group consisted of chief general surgical residents in their last year of training, laparoscopy fellows. and practicing laparoscopic surgeons (n = 82). The Mann-Whitney U test was used to evaluate differences in task performance between the two groups. Results: There was a significant difference in total scores and individual MISTELS task scores between the noncompetent and competent laparoscopic surgeons (189 vs 372.5; p < 0.0001). By setting specific pass/fail total score thresholds (cutoff scores), competent surgeons can be discriminated from noncompetent surgeons. Conclusion: An objective pass/fail evaluation can be given to individuals tested with the MISTELS system.

Characterizing the learning curve for a basic laparoscopic drill

Background:The psychomotor challenges inherent in laparoscopic surgery are evident by the steep procedural “learning curves” documented throughout the literature. Few methods have been described to evaluate learning curves. The cumulative summation (CUSUM) method is a criterion-based evaluation of the learning process. The purpose of this study is to describe the CUSUM learning curves for a simple task for individuals and for a group of novice laparoscopists.Methods:Sixteen medical students undertook four weekly sessions of 10 laparoscopic pegboard transfers in the MISTELS system. Their performance was scored and recorded for each trial. CUSUM learning curves were constructed based on the goal of achieving mean scores for senior, intermediate, or junior laparoscopists ≥95% of the time.Results:Based on senior criteria, one student achieved the goal by the 40th peg transfer trial. Based on intermediate criteria, three students achieved the goal by their 40th trial (trials 21 and 36), and for junior criteria, 10 students achieved the acceptable success rate by their 40th trial (range, 26–40).Conclusion:CUSUM analysis suggests criterion-based practice is useful for novice laparoscopists. It allows educators to track the progress of an individual toward target criteria for each MISTELS task, to more logically allocate time for training and set attainable goals, to objectively evaluate trainee acquisition of basic laparoscopic skills, and to identify trainees who need remediation.

Evaluating intraoperative laparoscopic skill: direct observation versus blinded videotaped performances.

The Global Operative Assessment of Laparoscopic Skill (GOALS) has been shown to meet high standards for direct observation. The purpose of this study was to investigate the reliability and validity of GOALS when applied to blinded, videotaped performances. Five novice surgeons and 5 experienced surgeons were each evaluated by 2 observers during a laparoscopic cholecystectomy. Subsequently, 4 laparoscopists (V1 to V4) evaluated the videotaped procedures using GOALS. Two of the raters (V1 and V3) had prior experience using GOALS. The interrater reliabilities between video raters (VRs) and between VRs and direct raters (DRs) were calculated using the intraclass correlation coefficient. Construct validity was assessed using 2-way analysis of variance. Interrater reliability between the 4 VRs and the 2 DRs was 0.72. The intraclass correlation coefficient for the 4 VRs was 0.68 and for each VR compared with the mean DR was 0.86, 0.39, 0.94, and 0.76, respectively. All raters, except V2, differentiated between novice and experienced groups (P values ranged from .01 to .05). These data suggest that GOALS can be used to assess laparoscopic skill based on videotaped performances but that rater training may play an important role in ensuring the reliability and validity of the instrument. Experience with the tool in the operating room may improve the reliability of video rating and could be of value in training evaluators.

Validation of the da Vinci Surgical Skill Simulator across three surgical disciplines

OBJECTIVE In this paper, we evaluate face, content and construct validity of the da Vinci Surgical Skills Simulator (dVSSS) across 3 surgical disciplines. METHODS In total, 48 participants from urology, gynecology and general surgery participated in the study as novices (0 robotic cases performed), intermediates (1-74) or experts (≥75). Each participant completed 9 tasks (Peg board level 2, match board level 2, needle targeting, ring and rail level 2, dots and needles level 1, suture sponge level 2, energy dissection level 1, ring walk level 3 and tubes). The Mimic Technologies software scored each task from 0 (worst) to 100 (best) using several predetermined metrics. Face and content validity were evaluated by a questionnaire administered after task completion. Wilcoxon test was used to perform pair wise comparisons. RESULTS The expert group comprised of 6 attending surgeons. The intermediate group included 4 attending surgeons, 3 fellows and 5 residents. The novices included 1 attending surgeon, 1 fellow, 13 residents, 13 medical students and 2 research assistants. The median number of robotic cases performed by experts and intermediates were 250 and 9, respectively. The median overall realistic score (face validity) was 8/10. Experts rated the usefulness of the simulator as a training tool for residents (content validity) as 8.5/10. For construct validity, experts outperformed novices in all 9 tasks (p < 0.05). Intermediates outperformed novices in 7 of 9 tasks (p < 0.05); there were no significant differences in the energy dissection and ring walk tasks. Finally, experts scored significantly better than intermediates in only 3 of 9 tasks (matchboard, dots and needles and energy dissection) (p < 0.05). CONCLUSIONS This study confirms the face, content and construct validities of the dVSSS across urology, gynecology and general surgery. Larger sample size and more complex tasks are needed to further differentiate intermediates from experts.

Identification of Quality of Care Deficiencies in Elderly Surgical Patients by Measuring Adherence to Process-Based Quality Indicators

BACKGROUND The ability to measure surgical quality of care is important and can lead to improvements in patient safety. As such, processes should be carried out in an identical fashion for all patients, regardless of how vulnerable or complex they are. Our objectives were to assess quality of surgical care delivered to elderly patients and to determine the association between patient characteristics and quality of care. STUDY DESIGN This is a retrospective pilot cohort study, conducted in a single university-affiliated hospital. Using the institutions National Surgical Quality Improvement Program (NSQIP) database (2009 to 2010), 143 consecutive patients 65 years or older, undergoing elective major abdominal surgery, were selected. Adherence to 15 process-based quality indicators (QIs) was measured, and a pass rate was calculated for each individual QI. The association between patient characteristics (age, sex, Charlson Comorbidity Index, functional status, wound class) and patient quality score was assessed using multiple linear regression. RESULTS Quality indicators with the lowest pass rates included postoperative delirium screening (0%), level of care documentation (0.7%), cognition and functional assessment at discharge (4.9%), oral intake documentation (12.6%), and pressure ulcer risk assessment (35.0%). The mean patient quality score was 46.8% ± 10.7% (range 16.7% to 75.0%). No association was found between patient characteristics and patient quality score. CONCLUSIONS Quality of care delivered to elderly patients undergoing major surgery at our institution was generally poor and independent of patient characteristics. Although quality appears to be uniform across different patients, these results provide targets for quality improvement initiatives.

Does aggressive hydration reverse the effects of pneumoperitoneum on renal perfusion

BackgroundAlthough pneumoperitoneum (PP) decreases renal blood flow, it remains unclear whether this impacts renal function. Our aim was to characterize the effects of PP on renal perfusion and function using two fluid strategies for intravenous fluid administration.MethodsTwelve 30-kg pigs were randomized into two groups: maintenance (3 cc/kg/h of normal saline (NS)) and bolus (15 cc/kg/h + 20 cc/kg NaCl bolus prior to induction of PP). Pigs were studied in a blinded fashion for 30 min prior, 60 min during, and 30 min after release of 15 mmHg CO2 PP. Cardiac output (CO) and stroke volume (SV) were measured using an esophageal Doppler probe, renal cortical perfusion (RCP) was measured with a laser Doppler probe on the right kidney, and renal function was measured using the fractional excretion of sodium (FeNa) and urine output. Statistical analysis was performed with area-under-the-curve (AUC) analysis and analysis of varianceResultsAUC analysis revealed moderate effect size for CO (0.416) and small effect size for SV (0.366) and RCP (0.363), with decreases seen in the control group but not the bolus group. During PP, urine output increased in the bolus group (p = 0.04) but not in the control group; there was no difference in FeNa in either group.ConclusionAggressive fluid hydration during CO2 PP of 15 mmHg preserves CO, SV, and RCP while increasing urine output. No effect on renal function as measured by FeNa was observed in either group.

Formal research training during surgical residency: scaffolding for academic success

BACKGROUND The purpose of this study was to determine the impact of a formal surgical research program (leading to a postgraduate degree) during residency, on future research productivity. METHODS We surveyed all North American graduates of the McGill University general surgery residency program between 1987 and 2005. The survey included questions on research involvement before, during, and after general surgery residency. This was combined with a literature search revealing all research publications of the participants. Outcomes were the yearly average of publications and awarded funding as faculty members. RESULTS Seventy-five of 119 graduates (63%) responded. Staff physicians who had participated in formal research programs during residency (n = 35), compared with those who had not (n = 40), produced more publications per year (2.8 ± 2.3 vs 1.1 ± 1.2, P < .01) and had greater funding success (81% vs 55%, P = .03). CONCLUSIONS Residents who had participated in formal research programs during residency were more likely to have greater academic success.

Laparoscopic Splenectomy: Learning Curve Comparison Between Benign and Malignant Disease

Background. New surgical techniques should be formally evaluated for feasibility and safety. As a model for this evaluation, this study examines the authors’ institution’s experience with splenectomy for benign and malignant hematologic disease since the introduction of laparoscopic splenectomy (LS) in 1996. The authors present the evaluation of the recognized surgeon/institutional learning curve using CUSUM (cumulative sum) analysis. Methods. This is a single institution retrospective chart review of consecutive splenectomies for hematologic disease performed between 1996 and 2008. The primary outcome was conversion to open splenectomy. The learning curve for LS was evaluated using CUSUM analysis. Results. A total of 123 splenectomies were performed for benign (51.2%) or malignant (48.7%) hematologic disease. 58% of patients underwent planned LS, with a 21% conversion rate. The surgeon’s overall learning curves for LS, as well as that for malignant disease, were maintained within acceptable conversion thresholds. However, the learning curve for benign disease did cross the unacceptable conversion threshold at case 29. With additional experience, the curve again approached the acceptable conversion threshold. Patients with malignant disease were significantly older (P = .0004), had larger spleens (P = .0004), were more likely to undergo open splenectomy (P = .001), and had longer lengths of stay (P = .01). However, there was no significant difference in operative time, transfusion requirements, morbidity rates, or mortality rates between patients with benign and malignant disease. Conclusion: LS, for benign or for malignant hematologic disease, is associated with a significant learning curve. This evaluation model illustrates that careful patient selection and ongoing quality assessment is essential when introducing a new technique.

Knowing the operative game plan: a novel tool for the assessment of surgical procedural knowledge

BACKGROUND What is the source of inadequate performance in the operating room? Is it a lack of technical skills, poor judgment or a lack of procedural knowledge? We created a surgical procedural knowledge (SPK) assessment tool and evaluated its use. METHODS We interviewed medical students, residents and training program staff on SPK assessment tools developed for 3 different common general surgery procedures: inguinal hernia repair with mesh in men, laparoscopic cholecystectomy and right hemicolectomy. The tools were developed as a step-wise assessment of specific surgical procedures based on techniques described in a current surgical text. We compared novice (medical student to postgraduate year [PGY]-2) and expert group (PGY-3 to program staff) scores using the Mann-Whitney U test. We calculated the total SPK score and defined a cut-off score using receiver operating characteristic analysis. RESULTS In all, 5 participants in 7 different training groups (n = 35) underwent an interview. Median scores for each procedure and overall SPK scores increased with experience. The median SPK for novices was 54.9 (95% confidence interval [CI] 21.6-58.8) compared with 98.05 (95% CP 94.1-100.0) for experts (p = 0.012). The SPK cut-off score of 93.1 discriminates between novice and expert surgeons. CONCLUSION Surgical procedural knowledge can reliably be assessed using our SPK assessment tool. It can discriminate between novice and expert surgeons for common general surgical procedures. Future studies are planned to evaluate its use for more complex procedures.