Krista Hardy

Establishing the concurrent validity of general and technique-specific skills assessments in surgical education

BACKGROUND Laparoscopic surgery entails a unique skill set, but it is unclear whether it requires a specific assessment form or whether more general assessment tools can be applied. The purpose of this study was to assess the concurrent validity of 2 previously validated assessment scales. One scale designed specifically to assess laparoscopic skills and the other to assess more general surgical skills. METHODS Postgraduate year 1-6 general surgery and urology residents (n = 33) performed a live laparoscopic cholecystectomy. Three surgeon raters scored their performance using previously validated objective structured assessment of technical skills (OSATS) and global operative assessment of laparoscopic skills rating scales. RESULTS Pearsons correlation coefficient between global operative assessment of laparoscopic skills and OSATS rating scales was .975 (P = .01). CONCLUSIONS The near total correlation between the 2 scales questions the need for separate laparoscopic assessment tools, highlighting the real strengths of OSATS, the use of which allows for more consistent nomenclature and standardized skills assessment across surgical platforms.

The Fundamentals of Laparoscopic Surgery and LapVR evaluation metrics may not correlate with operative performance in a novice cohort

Background Considerable resources have been invested in both low- and high-fidelity simulators in surgical training. The purpose of this study was to investigate if the Fundamentals of Laparoscopic Surgery (FLS, low-fidelity box trainer) and LapVR (high-fidelity virtual reality) training systems correlate with operative performance on the Global Operative Assessment of Laparoscopic Skills (GOALS) global rating scale using a porcine cholecystectomy model in a novice surgical group with minimal laparoscopic experience. Methods Fourteen postgraduate year 1 surgical residents with minimal laparoscopic experience performed tasks from the FLS program and the LapVR simulator as well as a live porcine laparoscopic cholecystectomy. Performance was evaluated using standardized FLS metrics, automatic computer evaluations, and a validated global rating scale. Results Overall, FLS score did not show an association with GOALS global rating scale score on the porcine cholecystectomy. None of the five LapVR task scores were significantly associated with GOALS score on the porcine cholecystectomy. Conclusions Neither the low-fidelity box trainer or the high-fidelity virtual simulator demonstrated significant correlation with GOALS operative scores. These findings offer caution against the use of these modalities for brief assessments of novice surgical trainees, especially for predictive or selection purposes.

The effect of blocked versus random task practice schedules on the acquisition and retention of surgical skills

BACKGROUND When learning multiple tasks, blocked or random training schedules may be used. We assessed the effects of blocked and random schedules on the acquisition and retention of laparoscopic skills. METHODS Thirty-six laparoscopic novices were randomized to practice laparoscopic tasks using blocked, random, or no additional training. Participants performed immediate post-tests, followed by retention tests 6 weeks later. Outcomes included previously validated Fundamentals of Laparoscopic Surgery (FLS) and hand-motion efficiency scores. RESULTS Both blocked and random groups had significantly higher FLS and hand-motion efficiency scores over baseline on post-tests for each task (P < .05) and higher overall FLS scores than controls on retention tests (P < .01). No difference was seen between the blocked and random groups in the amount of skill acquired or skill retained. CONCLUSIONS Both blocked and random training schedules can be considered as valid training options to allow programs and learners to tailor training to their individual needs.

Development of consensus-derived quality indicators for laparoscopic Roux-en-Y gastric bypass surgery.

BACKGROUND Synoptic operative reporting is a solution to the poor quality of narrative reports. OBJECTIVE To develop operative report quality indicators (QI) for laparoscopic Roux-en-Y gastric bypass (LRYGB) to generate validated parameters by which these reports can be evaluated and improved. SETTING University hospital in Canada. METHODS A Delphi protocol was used to determine QIs for LRYGB reporting. Bariatric surgeons across Canada were recruited along with key physician stakeholders to participate via a secure web-based platform. Participants initially submitted potential QIs. These were grouped by theme. Items were rated on 9-point Likert scales in subsequent rounds. Scores of 70% or greater were used for inclusion consensus, and 30% or less denoted exclusion. Elements scoring 30% to 70% were recirculated by runoff in subsequent rounds to generate the final list of QIs. RESULTS Four community and 4 academic bariatric surgeons were invited, representing all provinces performing LRYGB. The 4 multidisciplinary invitees included 1 minimally invasive/acute care surgeon, 1 tertiary abdominal radiologist, 1 gastroenterologist performing advanced endoscopy, and 1 general surgeon with expertise in synoptic reporting. Round 1 achieved an 83.3% (10/12) response and identified 91 potential items for consideration. Round 2 had a 100% response, and 69 items reached inclusion consensus. The third round achieved a 100% response and resulted in 75 QIs reaching final inclusion consensus. CONCLUSION This study established consensus-derived multidisciplinary QIs for LRYGB operative reports. This will allow further assessment of the quality of narrative reports and afford the development of a synoptic operative report that may ameliorate identified deficiencies.

Electronic synoptic reporting: assessing the completeness of synoptic and narrative reports for Roux-en-Y gastric bypass

BACKGROUND Operative reporting is the foundation of surgical communication. The quality indicators (QIs) contained in these reports can be used to document the performance of processes that affect patient care and may afford quality assurance with improvement in healthcare. OBJECTIVES We assessed the degree to which the electronic synoptic report (SR) documents the operative QIs compared with narrative reports (NR) for Roux-en-Y gastric bypass (RYGB). The time to report availability on patient chart was also identified as a secondary measure. SETTING Academic center, public hospital, Canada. METHODS A total of 40 prospectively collected RYGB synoptic reports and 40 case-matched historical NRs were compared against checklist QIs that were established by a national Delphi process. These checklist QIs are validated and have high interrater agreement at our institution. Time from dictation to report availability on patient chart was measured. RESULTS SR had a mean completion of 99.7% (±standard deviation [SD] 1.3%) compared with 64.0% (±SD 6.3%) for NR (t = 36.0, P<.0001). All subsections of SR were>99% complete and significantly higher than NR (P<.001). The mean time from NR dictation to report availability was 4.14 days (SD±3.17, range 0-10 d). All SRs were in the operative area after the procedure and were available immediately. CONCLUSION The RYGB synoptic report is superior to the narrative report for inclusion of accepted quality indicators and time to availability. Important elements, including process of care, demographic characteristics, and anatomic-related data, were often missing from the NR. SR is a promising method for improving documentation for RYGB.

Cognitive and Technical Skill Assessment in Surgical Education: a Changing Horizon

Assessment is an integral component of training and credentialing surgeons for practice. Traditional methods of cognitive and technical appraisal are well established but have clear shortcomings. This review outlines the components of the surgical care assessment model, identifies the deficits of current evaluation techniques, and discusses novel and emerging technologies that attempt to ameliorate this educational void.