Barbara M. A. Schout

How Useful and Realistic is the Uro Trainer for Training Transurethral Prostate and Bladder Tumor Resection Procedures

PURPOSE We evaluated the face and content validity (novice and expert opinions of realism and usefulness) of the Uro Trainer (Karl Storz GmbH, Tuttlingen, Germany), a simulator for transurethral resection procedures, to ascertain whether it is justifiable to continue the validation process by performing prospective experimental studies. MATERIALS AND METHODS Between 2006 and 2008, 104 urologists and urology residents performed a transurethral bladder tumor resection and/or transurethral prostate resection procedure on the Uro Trainer, and rated simulator usefulness and realism on a 10-point scale (1-not at all useful/realistic/poor, 10-very useful/realistic/excellent). Participants were classified as experts (more than 50 procedures performed) or novices (50 or fewer procedures performed). Because the literature offered no guidelines for interpreting our data, we used criteria from other studies to interpret the results. RESULTS A total of 161 questionnaires were analyzed from 97 (21% experts, 79% novices) and 64 (30% experts, 70% novices) participants who performed transurethral prostate resection and transurethral bladder tumor resection procedures, respectively. Mean usefulness, realism and overall scores varied from 5.6 to 8.2 (SD 1.4-2.5). Measured by validity criteria from other studies, Uro Trainer face and content validity was unsatisfactory, with ratings on only 3%, 5% and 8% of the parameters interpreted as positive, moderately acceptable and good, respectively. CONCLUSIONS Measured against criteria from other validation studies, Uro Trainer face and content validity appears to be unsatisfactory. Modification of the simulator seems advisable before further experimental validation studies are initiated. The lack of general guidelines for establishing face and content validity suggests a need for consensus about appropriate methods for evaluating the validity of simulators.

Transfer of cysto-urethroscopy skills from a virtual-reality simulator to the operating room: a randomized controlled trial

Study Type – Therapy (RCT)
 Level of Evidence 1b

Results of the European Basic Laparoscopic Urological Skills Examination

BACKGROUND In 2011, the European Basic Laparoscopic Urological Skills (E-BLUS) examination was introduced as a pilot for the examination of final-year urologic residents. OBJECTIVE In this study, we aimed to answer the following research questions: What level of laparoscopic skills do final-year residents in urology have in Europe, and do the participants of the E-BLUS pass the examination according to the validated criteria? DESIGN, SETTING, AND PARTICIPANTS Participants of the examination were final-year urology residents from different European countries taking part in the European Urology Residents Education Program in 2011 and 2012. SURGICAL PROCEDURE The E-BLUS exam consists of five tasks validated for the training of basic urologic laparoscopic skills. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS Performances of the tasks were recorded on DVD and analysed by an objective rater. Time and number of errors made in tasks 1-4 were noted. Furthermore, all expert laparoscopic urologists were asked to score participants on a global rating scale (1-5) based on three items: depth perception, bimanual dexterity, and efficiency. Participants were asked to complete a questionnaire on prior training and laparoscopic experience. RESULTS AND LIMITATIONS Seventy DVD recordings were analysed. Most participants did not pass the time criteria on task 4 (90%), task 2 (85.7%), task 1 (74.3%), and task 5 (71.4%). Task 3 was passed by 84.3%. The overall quality score was passed by 64%. When combining time and quality, only three participants (4.2%) passed the examination according to the validated criteria. According to the questionnaire, 61% did not have the opportunity to train in laparoscopic skills. CONCLUSIONS The results of the E-BLUS examination show that the level of basic laparoscopic skills among European residents is low. Although quality of performance is good, most residents do not pass the validated time criteria. Regular laparoscopic training or a dedicated fellowship should improve the laparoscopic level of residents in urology.

Acquisition of flexible cystoscopy skills on a virtual reality simulator by experts and novices

Study Type – Therapy (case control)
Level of Evidence 3b

The Virtual Reality Endourologic Simulator Is Realistic and Useful for Educational Purposes

PURPOSE To examine the educational value of URO Mentor, a virtual reality simulator for endourologic procedures, by establishing its face and content validity. MATERIALS AND METHODS Eighty-nine urologists and residents in urology performed a urethrocystoscopy task (bladder inspection, biopsy, and coagulation) or a ureterorenoscopy task (manipulation of a distal ureter stone) using the URO Mentor. They completed an evaluation questionnaire about these tasks. Questionnaire bias related to task performance on URO Mentor was analyzed as well (Hawthorne effect). RESULTS The overall rating of the URO Mentor was 7.3 on a 10-point scale (1 = poor, 10 = excellent). Regression analysis showed that ratings were not related to previous experience or task performance using the URO Mentor. Of all subjects, 25% rated the realism of URO Mentor as >or=3.5 on a five-point scale, and 82% rated its usefulness as an educational tool as >or=3.5 on a five-point scale. More than 73% of all participants said they would purchase a URO Mentor if financial means were available; participants who needed more time to complete the task were less positive in their answers to this question. CONCLUSIONS URO Mentor appears to be a realistic and useful training model for endourologic procedures.

Designing simulator-based training: An approach integrating cognitive task analysis and four-component instructional design

Most studies of simulator-based surgical skills training have focused on the acquisition of psychomotor skills, but surgical procedures are complex tasks requiring both psychomotor and cognitive skills. As skills training is modelled on expert performance consisting partly of unconscious automatic processes that experts are not always able to explicate, simulator developers should collaborate with educational experts and physicians in developing efficient and effective training programmes. This article presents an approach to designing simulator-based skill training comprising cognitive task analysis integrated with instructional design according to the four-component/instructional design model. This theory-driven approach is illustrated by a description of how it was used in the development of simulator-based training for the nephrostomy procedure.

Program for laparoscopic urological skills assessment: setting certification standards for residents

Abstract Aim: There is growing pressure from the government and the public to define proficiency standards for surgical skills. Aim of this study was to estimate the reliability of the Program for Laparoscopic Urological Skills (PLUS) assessment and to set a certification standard for second-year urological residents. Methods: Fifty participants were assessed on performance time and performance quality to investigate the reliability of the PLUS assessment. Generalisability coefficient of 0.8, on a scale of 0 to 1.0, was considered to indicate good reliability for assessment purposes. Pass/fail standards were based on laparoscopic experience: Novices, intermediates, and experts (>100 procedures). The pass/fail standards were investigated for the PLUS performances of 33 second-year urological residents. Results: Fifteen novices, twenty-three intermediates and twelve experts were included. An inter-trial reliability of >0.80 was reached with two trials for each task. Inter-rater reliability of the quality measurements was 0.79 for two judges. Pass/fail scores were determined for the novice/intermediate boundary and the intermediate/expert boundary. Pass rates for second-year residents were 63.64% and 9.09%, respectively. Conclusion: The PLUS assessment is reliable for setting a certification standard for second-year urological residents that serves as a starting point for residents to proceed to the next level of laparoscopic competency.

The effect of a low-fidelity model on cystoscopic skill training: a single-blinded randomized controlled trial.

Introduction: Models for training urological procedures without burdening patients are available at varying costs. We examined the value of training on a low-fidelity model in addition to training on a high-fidelity simulator in a cystoscopy training program. Methods: Thirty-two medical students were randomized to an intervention and a control group. The former started by performing cystoscopy on a low-cost, low-fidelity, glass globe model before moving on to training on the URO Mentor (UM), a computerized simulator. The control group took part in the same UM training program but not in the low-fidelity training. Performance on UM was assessed by a global rating score, percentage of correctly inspected areas of the bladder (% inspected areas), time, and number of traumas caused. Results: The intervention group had generally higher scores. Its global rating score on task 1 was significantly higher than that of the control group (Mann-Whitney U test, P = 0.046, effect size 0.6) and the group also scored higher, albeit not significantly, on time and % inspected areas. All students said they valued training with UM, but the appreciation of the intervention group was stronger (mean 8.9 vs. 8.1 on a scale from 1 to 10, P = 0.017, effect size 1.8). Conclusion: A low-fidelity glass globe model seemed to be an inexpensive educational tool to practice the first steps of cystoscopy. It may reduce training time on the UM simulator. The combined use of a low- and high-fidelity training model may provide an optimal learning effect.

The Simbla TURBT Simulator in Urological Residency Training: From Needs Analysis to Validation

OBJECTIVE To investigate the value of the physical Simbla Transurethral Resection of a Bladder Tumor (TURBT) simulator as an educational tool within urological residency training, by means of a training needs analysis (TNA) and assessment of its feasibility, acceptability, and face, content, and construct validity. METHODS To analyze the training needs for TURBT, procedural steps and pitfalls were identified and the TNA was completed during an expert consensus meeting. Participants (n = 76) were divided into three groups based on their experience in TURBT: novices, intermediates, and experts. Participants performed two standardized TURBT procedures on the simulator. Face validity and content validity, as well as feasibility and acceptability, were assessed with a quantitative survey. Construct validity was assessed by comparing the performance of novices, intermediates, and experts on resection time, quality of tumor resection, and overall performance. RESULTS Of the 21 procedural steps and 17 pitfalls defined in TNA, 13 steps and 8 pitfalls were covered by the Simbla. Participants rated the Simblas overall realism (face validity) with a score of 8 of 10 (range 6-9). The simulator was judged to be most useful (content validity) for learning eye-hand coordination: score 8 (6-10). All aspects regarding realism and usefulness were rated above the acceptability threshold of 6/10. Intermediates (100%) and experts (96%) considered the Simbla to be a useful educational tool within the urological curriculum. Resection time was longer for novices than for experts (p < 0.05; construct validity). In addition, the overall performance of novices was rated lower compared with intermediates and experts, and novices showed more irradical resections and bladder perforations (all p < 0.05). CONCLUSIONS The Simbla TURBT simulator is a valid, feasible, and acceptable educational tool for training procedural skills and may be implemented in the urological curriculum to complement learning in clinical practice. TNA is valuable in defining training objectives and evaluating the educational value of a simulator.

A Simulator for Teaching Transrectal Ultrasound Procedures How Useful and Realistic Is It

Introduction: We describe a new simulator for teaching transrectal ultrasound (TRUS) and present the results of a preliminary evaluation of the simulators realism and usefulness for training. Methods: A simulator for abdominal ultrasound was adjusted by the developer to enable simulation of TRUS by providing an opening for inserting a dummy rectal probe. To enable TRUS simulation, data from ultrasound prostate imaging of eight real patients obtained with our regular ultrasound machine were transferred to the simulator by connecting the computer of the simulator to the ultrasound machine. These data were used to create images in the TRUS simulator. Residents and urologists used the simulator to perform TRUS in one of the eight patient cases and judged the simulators realism and usefulness. Results: We were able to construct an initial urological module for the TRUS simulator. The images shown on the monitor of the simulator are quite realistic. The simulator can be used without difficulty to collect data, to create cases, and to perform TRUS. The absence of an option for prostate biopsy and the lack of tissue resistance were mentioned as two important shortcomings. Forty-seven participants rated the simulators overall realism and usefulness for training purposes as 3.8 (standard deviation: 0.7) and 4.0 (standard deviation: 0.8) on a five-point Likert scale, respectively. Conclusions: The simulator we describe can be used as a training tool for TRUS. It enables training with different patient cases and minimizes the burden to patients. Simulation of prostate biopsies should be added to increase the models usefulness.