I. Van Herzeele

Experienced Endovascular Interventionalists Objectively Improve their Skills by Attending Carotid Artery Stent Training Courses

OBJECTIVE Carotid artery stenting (CAS) is an advanced endovascular intervention with a steep learning curve. Virtual reality (VR) simulation has been proposed as a means to train and objectively assess technical performance. AIM To objectively assess psychomotor skills acquisition of experienced interventionalists attending a two-day CAS course, using a VR simulator. METHODS Both cognitive and technical skills of 11 interventionalists were trained in a two-day course using didactic sessions, case reviews, supervised VR simulation and live-cases. Pre- and post-course skills were assessed through performance on the same CAS procedure using metrics derived from the simulator. RESULTS Significant differences were noted between pre- and post-course performance for procedure (36 vs. 20min., p=0.005), X-ray (20 vs. 11min., p=0.016) and delivery-retrieval time of the embolic protection device (12 vs. 9min., p=0.007). Advancement of the guiding catheter without a leading wire occurred to a greater extent pre- versus post-course (199 vs. 152mm., p=0.050) as did spasm of the internal carotid artery (4 vs. 2, p=0.049). CONCLUSIONS This study has objectively proven a benefit for experienced interventionalists to attend CAS courses for skills acquisition measured by a VR simulator. These data can be used to offer participants an insight into their skills and objectively audit course efficacy.

Validation of Video-based Skill Assessment in Carotid Artery Stenting

OBJECTIVES To develop weighted error-based, generic and procedure-specific rating scales, to validate these scales for video-based assessment during virtual carotid artery stent (CAS) procedures and correlate them with simulator-derived metrics. METHODS A questionnaire was developed to assess the technique during live CAS procedures. Errors were rated from 1 (unimportant) to 5 (life-threatening) by 28 highly experienced CAS (>50 CAS) physicians. Virtual CAS procedure was performed by 21 interventionalists with varied CAS experience. Fluoroscopy screen and hand movements were video-taped, and simulator-derived metrics recorded. Experienced CAS practitioners then rated the video-taped performances using weighted error, generic and procedure-specific rating scales. RESULTS Of the 23 errors assessed, 12 were regarded as moderate (score 3), six serious (score 4) and four life-threatening (score 5). The generic rating scale was able to detect significant differences in performance between inexperienced and experienced CAS operators (score 25 vs. 32 respectively, P<0.01). All scoring systems demonstrated good inter-rater reliability (alpha=0.61-0.87). Significant correlations were observed between simulator-derived and video-based scores: weighted error-based score (r: 0.76, P<0.01), generic (r: 0.62, P<0.01) and procedure-specific (r: 0.76, P<0.01) rating scales. CONCLUSIONS The generic endovascular rating scale differentiated between levels of CAS experience among skilled interventionalists and correlated to simulator-based error scoring.

Patient-specific Rehearsal Prior to EVAR: A Pilot Study

OBJECTIVES This study aims to evaluate feasibility, face validity, influence on technical factors and subjective sense of utility of patient-specific rehearsal (PsR) prior to endovascular aortic aneurysm repair (EVAR). DESIGN A prospective, multicentre pilot study. METHODS Patients suitable for EVAR were enrolled and a three-dimensional (3D) model of the patients anatomy was generated. Less than 24 h prior to the real case, rehearsals were conducted in the laboratory or clinical angiosuite. Technical metrics were recorded during both procedures. A subjective questionnaire was used to evaluate realism, technical and human factor aspects (scale 1-5). RESULTS Ten patients were enrolled. In one case, the treatment plan was altered based on PsR. In 7/9 patients, the rehearsal significantly altered the optimal C-arm position for the proximal landing zone and an identical fluoroscopy angle was chosen in the real procedure. All team members found the rehearsal useful for selecting the optimal fluoroscopy angle (median 4). The realism of the EVAR procedure simulation was rated highly (median 4). All team members found the PsR useful to prepare the individual team members and the entire team (median 4). CONCLUSIONS PsR for EVAR permits creation of realistic case studies. Subjective evaluation indicates that it may influence optimal C-arm angles and be valuable to prepare the entire team. A randomised controlled trial (RCT) is planned to evaluate how this technology may influence technical and team performance, ultimately leading to improved patient safety.

Patient-specific Endovascular Simulation Influences Interventionalists Performing Carotid Artery Stenting Procedures

OBJECTIVE The ability to perform patient-specific simulated rehearsal of complex endovascular interventions is a technological advance with potential benefits to patient outcomes. This study aimed to evaluate whether patient-specific rehearsal of a carotid artery stenting (CAS) procedure has an influence on tool selection and the use of fluoroscopy. METHODS Following case note and computed tomography (CT) angiographic review of a real patient case, subjects performed the CAS procedure on a virtual reality simulator. Endovascular tool requirements and fluoroscopic angles were evaluated with a pre- and post-case questionnaire. Participants also rated the simulation from 1 (poor) to 5 (excellent). RESULTS Thirty-three endovascular physicians with varying degrees of CAS experience were recruited: inexperienced (5-20 CAS procedures) n = 11, moderately (21-50 CAS procedures) n = 7 or highly experienced (>50 CAS procedures) n = 15. For all participants, 96 of a possible 363 changes (26%) were observed from pre- to post-case questionnaires. This was most notable for optimal fluoroscopy C-arm position 15/33 (46%), choice of selective catheter 13/33 (39%), choice of sheath or guiding catheter 11/33 (33%) and balloon dilatation strategy 10/33 (30%). Experience with the CAS procedure did not influence the degree of change significantly (p > 0.05), and all groups exhibited a considerable modification in tool and fluoroscopy preference. The model was considered realistic and useful as a tool to practice a real case (median score 4/5). CONCLUSION Patient-specific simulated rehearsal of a complex endovascular procedure strongly influences tool selection and fluoroscopy preferences for the real case. Further research has to evaluate how this technology may transfer from in vitro to in vivo and if it can reduce the radiation dose and the number of endovascular tools used and improve outcomes for patients in the clinical setting.

Systematic review of e-learning for surgical training.

Internet and software‐based platforms (e‐learning) have gained popularity as teaching tools in medical education. Despite widespread use, there is limited evidence to support their effectiveness for surgical training. This study sought to evaluate the effectiveness of e‐learning as a teaching tool compared with no intervention and other methods of surgical training.

A Pilot Study of Video-motion Analysis in Endovascular Surgery: Development of Real-time Discriminatory Skill Metrics

OBJECTIVES Accurate assessment and credentialing of physicians is essential. Objective motion analysis of guide-wire/catheter manipulation to assess proficiency during endovascular interventions remains unexplored. This study aims to assess its feasibility and its role in evaluation of technical ability. MATERIALS AND METHODS A semi-automated catheter-tracking software was developed which allows for frame-by-frame motion analysis of fluoroscopic videos and calculation 2D catheter tip path-length. 21 interventionalists (6 cardiologists, 8 interventional radiologists, 7 vascular surgeons; 14/21 had performed >500 endovascular procedures) performed an identical carotid artery stenting procedure (CAS) on a VIST simulator (Mentice, Gothenburg, Sweden). Operators were sub-divided into four categories according to CAS experience: 6 inexperienced (0 CAS-group A), 3 low-volume (1-20 CAS-group B), 5 moderate-volume (21-50 CAS-group C) and 7 high-volume (>50 CAS-group D) CAS experience. Total PL was calculated for each case and comparisons made between groups. PL was correlated with: quantitative, simulator-derived metrics and qualitative performance scores (generic and procedure-specific) derived from post-hoc video analysis by three blinded observers. RESULTS Group D used 5160.3 (inter-quartile range- IQR 4046.4-7142.9) pixels of movement, compared to 6856.7 (5914.4-8106.9) for group A (p = 0.046); 10,905.1 (7851.1-14,381.5) for group B (p = 0.017); and 9482.6 (8663.5-13,847.6) for group C (p = 0.003). Statistically significant inverse correlations were seen between total PL and qualitative performance scores (rho = -0.519 for generic (p = 0.027) rho = -0.567 for procedure-specific (p = 0.014) scores). PL did not correlate with any of the simulator-derived metrics (errors, contrast volume, total procedure and fluoroscopy times, cine-loops used). CONCLUSION Endovascular instrument video motion analysis is feasible and may represent a valuable tool for the objective assessment of endovascular skill.

Role of patient-specific virtual reality rehearsal in carotid artery stenting

Recent advances in simulation science permit patient‐specific rehearsal of endovascular stenting procedures. This study aimed to evaluate how effectively real interventions are replicated by patient‐specific rehearsal technology, and to assess its value as a preparatory tool for the interventionalist and the operating team.

Content Validation and Evaluation of an Endovascular Teamwork Assessment Tool

OBJECTIVE/BACKGROUND To modify, content validate, and evaluate a teamwork assessment tool for use in endovascular surgery. METHODS A multistage, multimethod study was conducted. Stage 1 included expert review and modification of the existing Observational Teamwork Assessment for Surgery (OTAS) tool. Stage 2 included identification of additional exemplar behaviours contributing to effective teamwork and enhanced patient safety in endovascular surgery (using real-time observation, focus groups, and semistructured interviews of multidisciplinary teams). Stage 3 included content validation of exemplar behaviours using expert consensus according to established psychometric recommendations and evaluation of structure, content, feasibility, and usability of the Endovascular Observational Teamwork Assessment Tool (Endo-OTAS) by an expert multidisciplinary panel. Stage 4 included final team expert review of exemplars. RESULTS OTAS core team behaviours were maintained (communication, coordination, cooperation, leadership team monitoring). Of the 114 OTAS behavioural exemplars, 19 were modified, four removed, and 39 additional endovascular-specific behaviours identified. Content validation of these 153 exemplar behaviours showed that 113/153 (73.9%) reached the predetermined Item-Content Validity Index rating for teamwork and/or patient safety. After expert team review, 140/153 (91.5%) exemplars were deemed to warrant inclusion in the tool. More than 90% of the expert panel agreed that Endo-OTAS is an appropriate teamwork assessment tool with observable behaviours. Some concerns were noted about the time required to conduct observations and provide performance feedback. CONCLUSION Endo-OTAS is a novel teamwork assessment tool, with evidence for content validity and relevance to endovascular teams. Endo-OTAS enables systematic objective assessment of the quality of team performance during endovascular procedures.

Multicentre observational study of surgical system failures in aortic procedures and their effect on patient outcomes

Vascular surgical care has changed dramatically in recent years with little knowledge of the impact of system failures on patient safety. The primary aim of this multicentre observational study was to define the landscape of surgical system failures, errors and inefficiency (collectively termed failures) in aortic surgery. Secondary aims were to investigate determinants of these failures and their relationship with patient outcomes.

A Multicentre Trial of Patient specific Rehearsal Prior to EVAR : Impact on Procedural Planning and Team Performance

OBJECTIVE Patient specific rehearsal (PsR) prior to endovascular aneurysm repair (EVAR) enables the endovascular team to practice and evaluate the procedure prior to treating the real patient. This multicentre trial aimed to evaluate the utility of PsR prior to EVAR as a pre-operative planning and briefing tool. MATERIAL AND METHODS Patients with an aneurysm suitable for EVAR were randomised to pre-operative or post-operative PsR. Before and after the PsR, the lead implanter completed a questionnaire to identify any deviation from the initial treatment plan. All team members completed a questionnaire evaluating realism, technical issues, and human factor aspects pertinent to PsR. Technical and human factor skills, and technical and clinical success rates were compared between the randomised groups. RESULTS 100 patients were enrolled between September 2012 and June 2014. The plan to visualise proximal and distal landing zones was adapted in 27/50 (54%) and 38/50 (76%) cases, respectively. The choice of the main body, contralateral limb, or iliac extensions was adjusted in 8/50 (16%), 17/50 (34%), and 14/50 (28%) cases, respectively. At least one of the abovementioned parameters was changed in 44/50 (88%) cases. For 100 EVAR cases, 199 subjective questionnaires post-PsR were completed. PsR was considered to be useful for selecting the optimal C-arm angulation (median 4, IQR 4-5) and was recognised as a helpful tool for team preparation (median 4, IQR 4-4), to improve communication (median 4, IQR 3-4), and encourage confidence (median 4, IQR 3-4). Technical and human factor skills and technical and initial clinical success rates were similar between the randomisation groups. CONCLUSION PsR prior to EVAR has a significant impact on the treatment plan and may be useful as a pre-operative planning and briefing tool. Subjective ratings indicate that this technology may facilitate planning of optimal C-arm angulation and improve non-technical skills. TRIAL REGISTRATION URL://www.clinicaltrials.gov. Unique identifier: NCT01632631.