Vivian E. Ekkelenkamp

Training and competence assessment in GI endoscopy: a systematic review

Introduction Training procedural skills in GI endoscopy once focused on threshold numbers. As threshold numbers poorly reflect individual competence, the focus gradually shifts towards a more individual approach. Tools to assess and document individual learning progress are being developed and incorporated in dedicated training curricula. However, there is a lack of consensus and training guidelines differ worldwide, which reflects uncertainties on optimal set-up of a training programme. Aims The primary aim of this systematic review was to evaluate the currently available literature for the use of training and assessment methods in GI endoscopy. Second, we aimed to identify the role of simulator-based training as well as the value of continuous competence assessment in patient-based training. Third, we aimed to propose a structured training curriculum based on the presented evidence. Methods A literature search was carried out in the available medical and educational literature databases. The results were systematically reviewed and studies were included using a predefined protocol with independent assessment by two reviewers and a final consensus round. Results The literature search yielded 5846 studies. Ninety-four relevant studies on simulators, assessment methods, learning curves and training programmes for GI endoscopy met the inclusion criteria. Twenty-seven studies on simulator validation were included. Good validity was demonstrated for four simulators. Twenty-three studies reported on simulator training and learning curves, including 17 randomised control trials. Increased performance on a virtual reality (VR) simulator was shown in all studies. Improved performance in patient-based assessment was demonstrated in 14 studies. Four studies reported on the use of simulators for assessment of competence levels. Current simulators lack the discriminative power to determine competence levels in patient-based endoscopy. Eight out of 14 studies on colonoscopy, endoscopic retrograde cholangiopancreatography and endosonography reported on learning curves in patient-based endoscopy and proved the value of this approach for measuring performance. Ten studies explored the numbers needed to gain competence, but the proposed thresholds varied widely between them. Five out of nine studies describing the development and evaluation of assessment tools for GI endoscopy provided insight into the performance of endoscopists. Five out of seven studies proved that intense training programmes result in good performance. Conclusions The use of validated VR simulators in the early training setting accelerates the learning of practical skills. Learning curves are valuable for the continuous assessment of performance and are more relevant than threshold numbers. Future research will strengthen these conclusions by evaluating simulation-based as well as patient-based training in GI endoscopy. A complete curriculum with the assessment of competence throughout training needs to be developed for all GI endoscopy procedures.

Patient comfort and quality in colonoscopy.

AIM To explore the relationship of patient comfort and experience to commonly used performance indicators for colonoscopy. METHODS All colonoscopies performed in our four endoscopy centres are recorded in two reporting systems that log key performance indicators. From 2008 to 2011, all procedures performed by qualified endoscopists were evaluated; procedures performed by trainees were excluded. The following variables were measured: Caecal intubation rate (CIR), nurse-reported comfort levels (NRCL) on a scale from 1 to 5, polyp detection rate (PDR), patient experience of the procedure (worse than expected, as expected, better than expected), and use of sedation and analgesia. Pearsons correlation coefficient was used to identify relationships between performance indicators. RESULTS A total of 17027 colonoscopies were performed by 23 independent endoscopists between 2008 and 2011. Caecal intubation rate varied from 79.0% to 97.8%, with 18 out of 23 endoscopists achieving a CIR of > 90%. The percentage of patients experiencing significant discomfort during their procedure (defined as NRCL of 4 or 5) ranged from 3.9% to 19.2% with an average of 7.7%. CIR was negatively correlated with NRCL-45 (r = -0.61, P < 0.005), and with poor patient experience (r = -0.54, P < 0.01). The average dose of midazolam (mean 1.9 mg, with a range of 1.1 to 3.5 mg) given by the endoscopist was negatively correlated with CIR (r = -0.59, P < 0.01). CIR was positively correlated with PDR (r = 0.44, P < 0.05), and with the numbers of procedures performed by the endoscopists (r = 0.64, P < 0.01). CONCLUSION The best colonoscopists have a higher CIR, use less sedation, cause less discomfort and find more polyps. Measuring patient comfort is valuable in monitoring performance.

Competence development in ERCP: The learning curve of novice trainees

BACKGROUND AND STUDY AIM Measures for competence in endoscopic retrograde cholangiopancreatography (ERCP) during training are poorly defined. Currently, various training and accreditation programs base verification of competence on performance of a minimum number of procedures. There is a general awareness that procedural competence certification should be based on objective performance criteria. Continuous self-assessment using a Rotterdam Assessment Form for ERCP (RAF-E) can provide insight into trainee performance. The study aim was to express development in ERCP competence as a learning curve. METHODS ERCP trainees at a tertiary referral center in the Netherlands were invited to participate. Performed procedures were appraised using RAF-E. Indication for each ERCP and presence of a virgin papilla were documented. Complexity was graded on a 3-point scale. The primary outcome parameter was common bile duct (CBD) cannulation success rate. Success of the intended therapeutic interventions was additionally expressed as a learning curve. RESULTS 15 trainees were included. 1541 ERCPs (624 procedures in native papillary anatomy) were assessed through RAF-E. Unassisted CBD cannulation success rate improved from 36 % at baseline to 85 % after 200 procedures (P < 0.001), and in 624 patients with a virgin papilla from 22 % at baseline to 68 % after 180 procedures (P < 0.001). Learning curves for therapeutic interventions showed significant improvements for successful sphincterotomy (P = 0.01) and stent placement (P < 0.001). CONCLUSIONS Learning curves are a valuable means for assessing competence in ERCP. Differences in learning curves can be shown with RAF-E. Verification of competence should be based on actual performance, instead of minimum numbers.

Quality evaluation through self-assessment: a novel method to gain insight into ERCP performance

Background The American Society for Gastrointestinal Endoscopy Committee on Outcomes Research has recommended monitoring nine endoscopic retrograde cholangiopancreatography (ERCP)-specific quality indicators for quality assurance in ERCP. With the development of a self-assessment tool for ERCP (Rotterdam Assessment Form for ERCP—RAF-E), key indicators can easily be assessed. Objective The aim of this study was to test in daily practice an easy-to-use form for assessment of procedural quality in ERCP and to determine ERCP quality outcomes in a tertiary referral hospital. Design This was a prospective study carried out in a tertiary referral hospital. In January 2008, a quality self-assessment programme was started. Five qualified endoscopists participated in this study. All ERCPs were appraised using RAF-E. Primary parameters were common bile duct (CBD) cannulation rate and procedural success. The indication was classified and procedural difficulty was graded; success rates of therapeutic interventions were measured for all different difficulty degrees. Results A total number of 1691 ERCPs were performed. 1515 (89.6%) of these were appraised using RAF-E. Median CBD cannulation success rate was 94.1%. Successful sphincterotomy was accomplished in almost all patients (median 100%; range 98.2–100%). Stent placement was successful in 97.8% and complete stone extraction, if indicated, was achieved in 86.8%. Conclusions Quality indicators for ERCP can be measured using the Rotterdam self-assessment programme for ERCP. Outcome data in ERCPs obtained with this RAF-E provide insight into the quality of individual as well as group performance and can be used to assess and set standards for quality control in ERCP.

Comfort scores in colonoscopy performance

Introduction The principle indicator for assessing competence in colonoscopy is caecal intubation rate (CIR). Comfort is a key auditable outcome for colonoscopy but there are no standards for patient comfort during colonoscopy and no reports of comfort scores in relation to other quality indicators. The aim of this study is to analyze the role of different factors in determining an individuals performance in colonoscopy and to explore the significance of patient comfort scores in colonoscopist performance. Methods All colonoscopies performed in our endoscopy centres are recorded in customised reporting systems (SQLscope and Unisoft), which log all key performance indicators. Data was extracted between 2007 and 2010. The following variables were measured: CIR, nurse-reported comfort levels (NRCL) on a 5-point scale (1 = no discomfort, 2 = minimal discomfort, 3 = mild discomfort, 4 = moderate discomfort, 5 = severe discomfort), polyp detection rate (PDR) (hyperplastic and adenomatous), patients experience (PE) of the procedure (better than expected, as expected, worse than expected) and use of sedation. Significant discomfort was defined as a NRCL of 4 or 5 or a PE of worse than expected. Results A total of 12561 colonoscopies were recorded with NRCL and PE. NRCL of 4 or 5 was measured in 1181 cases (9.4%). The average number of procedures performed per endoscopist per year was 146 (range 11–483). Figure 1 shows the relation between CIR and NRCL (4–5). There was a significant negative correlation (R = −0.57 ; p < 0.005). A positive correlation was found between PDR and CIR (R = 0.57; p < 0.005). The amount of midazolam given during the procedure was negatively correlated with CIR (R = −0.39; p = 0.055). Finally, fewer than one in 20 patients rated their experience worse than expected and a worse than expected PE of colonoscopy showed a negative correlation with CIR (R = -0.54; p < 0.01). Table 1 shows the outcome of the different variables per year (2007 and 2010 are only partial years). CIR, PDR and NRCL have collectively improved year on year. Figure 1 OC-088 Relation between CIR and NRCL 4-5 Table 1 OC-088 Performance per year Year Number of colonoscopies CIR (%) NCRL 4–5 (%) PDR (%) PE worse than expected (%) 2007 1328 90.7 12.7 20.5 4.7 2008 3966 92.2 10.5 28.4 5.5 2009 4235 93.1 7.9 27.3 4.2 2010 3032 94.5 8.6 30 4.3 Conclusion This study shows that endoscopists with a high CIR perform colonoscopies with less patient discomfort than those with lower CIRs, use less midazolam and see and remove more polyps. Thus achieving a high CIR and high PDR does not need to be associated with more pain and more sedation. Comfort scores should be included in the assessment of overall performance in colonoscopy to provide a fuller picture of performance.

Is Textbook Outcome a valuable composite measure for short-term outcomes of gastrointestinal treatments in the Netherlands using hospital information system data? A retrospective cohort study

Objective To develop a feasible model for monitoring short-term outcome of clinical care trajectories for hospitals in the Netherlands using data obtained from hospital information systems for identifying hospital variation. Study design Retrospective analysis of collected data from hospital information systems combined with clinical indicator definitions to define and compare short-term outcomes for three gastrointestinal pathways using the concept of Textbook Outcome. Setting 62 Dutch hospitals. Participants 45 848 unique gastrointestinal patients discharged in 2015. Main outcome measure A broad range of clinical outcomes including length of stay, reintervention, readmission and doctor–patient counselling. Results Patients undergoing endoscopic retrograde cholangiopancreatography (ERCP) for gallstone disease (n=4369), colonoscopy for inflammatory bowel disease (IBD; n=19 330) and colonoscopy for colorectal cancer screening (n=22 149) were submitted to five suitable clinical indicators per treatment. The percentage of all patients who met all five criteria was 54%±9% (SD) for ERCP treatment. For IBD this was 47%±7% of the patients, and for colon cancer screening this number was 85%±14%. Conclusion This study shows that reusing data obtained from hospital information systems combined with clinical indicator definitions can be used to express short-term outcomes using the concept of Textbook Outcome without any excess registration. This information can provide meaningful insight into the clinical care trajectory on the level of individual patient care. Furthermore, this concept can be applied to many clinical trajectories within gastroenterology and beyond for monitoring and improving the clinical pathway and outcome for patients.