Mitchell G. Goldenberg

Systematic review to establish absolute standards for technical performance in surgery

Standard setting allows educators to create benchmarks that distinguish between those who pass and those who fail an assessment. It can also be used to create standards in clinical and simulated procedural skill. The objective of this review was to perform a systematic review of the literature using absolute standard‐setting methodology to create benchmarks in technical performance.

Baseline Laparoscopic Skill May Predict Baseline Robotic Skill and Early Robotic Surgery Learning Curve

INTRODUCTION Robotic surgery is associated with a learning curve unique to each trainee. Knowledge about a trainees baseline skill and learning curve would facilitate the development of a more individualized training curriculum. The aim of our study was to determine whether baseline laparoscopic skill is predictive of ones baseline robotic skill and short-term learning curve. METHODS Trainees from four different surgical specialties were included in the study. Each trainee participated in a 4-week, simulation-based robotic surgery basic skills training course. Precourse, baseline laparoscopic and robotic skills were assessed using validated test tasks; a basic peg transfer (PT) and an advanced intracorporeal suturing and knot tying (ISKT) task. Trainee robotic skill was assessed again 1 week postcourse. Each task performance was video recorded and scored by two blinded expert surgeons. RESULTS A total of 32 trainees were included; 14 urology, 7 gynecology, 8 thoracic Sx, 3 general Sx. Most (91%) were senior residents or clinical fellows and 50% had no prior robotic experience. There were no differences in baseline laparoscopic and robotic skill related to reported prior robotic experience. Between specialties, no differences were seen on baseline laparoscopic skill and only small differences were seen on baseline robotic skill. Both baseline Lap PT (p = 0.01) and Lap ISKT (p = 0.01) performances correlated with baseline robotic ISKT performance, but not robotic PT scores. Only baseline Lap ISKT performance correlated with postcourse robotic PT (p = 0.01) and ISKT (p < 0.01) performance. Baseline robotic ISKT scores, but not PT scores, correlated with postcourse robotic performance (p = 0.02 for PT, p < 0.01 for ISKT). CONCLUSIONS In this study, a trainees baseline laparoscopic skill correlated with certain baseline robotic skills. Better baseline performance on an advanced, but not basic, laparoscopic and robotic skill task may correlate with a shorter learning curve for basic robotic skills. Further exploration of this finding may yield better training curricula.

Simulation-Based Laparoscopic Surgery Crisis Resource Management Training—Predicting Technical and Nontechnical Skills

OBJECTIVES To develop a unique simulation-based assessment using a laparoscopic inferior vena cava (IVC) injury scenario that allows for the safe assessment of urology residents technical and nontechnical skills, and investigate the effect of personality traits performance in a surgical crisis. METHODS Urology residents from our institution were recruited to participate in a simulation-based training laparoscopic nephrectomy exercise. Residents completed demographic and multidimensional personality questionnaires and were instructed to play the role of staff urologist. A vasovagal response to pneumoperitoneum and an IVC injury event were scripted into the scenario. Technical and nontechnical skills were assessed by expert laparoscopic surgeons using validated tools (task checklist, GOALS, and NOTSS). RESULTS Ten junior and five senior urology residents participated. Five residents were unable to complete the exercise safely. Senior residents outperformed juniors on technical (checklist score 15.1 vs 9.9, p < 0.01, GOALS score 18.0 vs 13.3, p < 0.01) and nontechnical performance (NOTSS score 13.8 vs 10.1, p = 0.03). Technical performance scores correlated with NOTSS scores (p < 0.01) and pass/fail rating correlated with technical performance (p < 0.01 for both checklist and GOALS), NOTSS score (p = 0.02), and blood loss (p < 0.01). Only the conscientiousness dimension of the big five inventory correlated with technical score (p = 0.03) and pass/fail rating (p = 0.04). CONCLUSIONS Resident level of training and laparoscopic experience correlated with technical performance during a simulation-based laparoscopic IVC injury crisis management scenario, as well as multiple domains of nontechnical performance. Personality traits of our surgical residents are similar and did not predict technical skill.

Using Data to Enhance Performance and Improve Quality and Safety in Surgery

What Is the Innovation? Errors resulting in adverse events are a common cause of morbidity in hospitalized patients. A significant portion of these errors occurs in the operating room (OR) and may be avoidable. A successful operative outcome reflects more than disease factors and postoperative management in isolation. Comprehensive assessment of operative quality is not possible with traditional postevent analysis. In response to this, our group developed and pilot tested a multiport synchronized data capture and analytic platform called the OR Black Box. Previous recording devices have limited the data capture to only video and audio, which restricts the opportunities for automated analysis. The OR Black Box continuously acquires various intraoperative data feeds, such as audiovisual data, physiological parameters from both patients and health care professionals, and multiple other sensors and devices (Figure). Video is captured using in-room wide-angle cameras, and intracorporeal video is collected from the laparoscope or robotic camera or from light-mounted or wearable cameras in open surgical procedures. All inputs are synchronized, encrypted, and stored on a secure server for further analysis. Expert analysts and software-based algorithms populate a procedural timeline using relevant data drawn from these inputs. Data points include procedural steps, disruptive environmental and organizational factors, OR team technical and nontechnical skills, surgeon physiological stress, and intraoperative errors, events, and rectification processes.

Enhancing Clinical Performance and Improving Patient Safety Using Digital Health

Patient confidentiality has remained a central issue in the current “big data” era of healthcare. Protections such as the Health Insurance Portability and Accountability Act of 1996 (HIPAA) exist to ensure that digital personal health information (PHI) are legally secure from threats and breaches that would threaten confidentiality. To be compliant with HIPAA regulations, steps must be taken by health care providers and digital health platforms, and these fall under the Privacy Rule, which outlines appropriate uses and disclosures of PHI, and the Security Rule, which lays out with granularity the steps that must be taken to adhere to the HIPAA regulations. Through deliberate design of secure digital health platforms, we can use technological advances in the collection, measurement, and delivery of health care to advance care and improve patient safety. Renewed efforts to optimize and standardize health care delivery has facilitated the implementation of electronic and digital health solutions that benefit medical and surgical training and efficiency while minimizing harm to patients. Cross-industry innovations such as the OR Black Box® will allow us to accomplish these lofty goals. Finally, we must strive to include patients in this digital health movement, as now more than ever we can create knowledge translation solutions that ensure that patients understand their health in a meaningful way.

Implementing assessments of robot-assisted technical skill in urological education: a systematic review and synthesis of the validity evidence

To systematically review and synthesise the validity evidence supporting intraoperative and simulation‐based assessments of technical skill in urological robot‐assisted surgery (RAS), and make evidence‐based recommendations for the implementation of these assessments in urological training.

The Future of Medical Education: Simulation-Based Assessment in a Competency-by-Design Curriculum

Competency-Based Medical Education (CBME) represents the biggest change to medical education since Halstead proposed his apprenticeship model over 100 years ago. This new paradigm has been crafted over nearly four decades (McGaghie 1978), and is built on the principle that trainee physicians and surgeons must fulfill core competencies, spanning professionalism to technical aptitude, prior to independent practice (Frank et al. 2015). This shift to CBME as an underpinning framework in medical education has brought about increased demand for assessment of trainee performance. Simulation has been identified as a means to increase trainee exposure to and experience with clinical tasks, without increasing the burden of patient harm (Griswold et al. 2012; Holmboe et al. 2010).

Understanding and Assessing Nontechnical Skills in Robotic Urological Surgery: A Systematic Review and Synthesis of the Validity Evidence

OBJECTIVE Robotic urological surgery (RUS) has seen widespread adoption across institutions in the last decade. To match this rapid growth, it is imperative to develop a structured RUS curriculum that addresses both technical and nontechnical competencies. Emerging evidence has shown that nontechnical skills form a critical component of RUS training. The purpose of this review is to examine the validity evidence of available nontechnical skills assessment tools in RUS. METHODS A literature search of MEDLINE, EMBASE, and PsycINFO was conducted to identify primary articles using nontechnical skills assessment tools in RUS. Messicks validity framework and the Medical Education Research Study Quality Instrument were utilized to evaluate the quality of the validity evidence of the abstracted articles. RESULTS Of the 566 articles identified, 12 used nontechnical skills assessment tools in RUS. The metrics used ranged from self-assessment using global rating scales, to objective measures such as electroencephalography. The setting of these evaluations ranged from immersive and virtual reality-based simulators to live surgery. CONCLUSIONS Limited effort has been made to develop nontechnical skills assessment tools in RUS. Recently, there has been a shift from subjective to objective measures of nontechnical performance, as well as the development of assessments specific to RUS. However, the validity evidence supporting these nontechnical assessments is limited at this time, including their relationship to technical skills, and their impact on surgical outcomes.

MP51-15 SURGICAL TECHNICAL PERFORMANCE IMPACTS PATIENT OUTCOMES IN ROBOTIC-ASSISTED RADICAL PROSTATECTOMY

INTRODUCTION AND OBJECTIVES: Defining competence in procedural training may prove difficult. Historically, this has been via direct supervision by senior surgeons. There has been a push for the development and use of validated curricula. We assess robotics curricula (RC) and surgical simulation (SS) in U.S. urology residency programs (URPs). METHODS: Of all URPs, 129 were contacted. Program directors (PDs) were queried on use of a formal or validated RC, use of virtual reality (VR), graduation requirements, availability of a SS laboratory, and chief resident comfort level with robotic surgery. RESULTS: Response rate was 26.3 %. Of PDs, 17 (50%) reported a formalized RC but 82.3% did not utilize validated RC. Physical consoles exist in 73.5% and VR trainers exist in 50% of programs. Simulation laboratories were reported in 85.2% of programs. Completion of a RC was required for graduation in 38.2% while it was suggested in 50% of programs. Six programs (17.6%) are undergoing curriculum development/validation. Figure 3 outlines PD perceptions of chief resident operative comfort. CONCLUSIONS: Our findings suggest that the majority (82.3%) of URPs do not employ validated RC. However, half of PDs report using VR modules, physical consoles, and online courses. By graduation, 91.1% of PDs reported that graduating chief residents are comfortable with robotic surgery. Source of Funding: none