Kulsoom Laeeq

Pilot testing of an assessment tool for competency in mastoidectomy

To determine the feasibility, validity, and reliability of an evaluation tool for the assessment of competency in mastoid surgery. This study tests the hypothesis that residents of dissimilar training levels differ in their technical performance as measured by this tool.

Development and pilot-testing of a feasible, reliable, and valid operative competency assessment tool for endoscopic sinus surgery.

Background Otolaryngology residency programs are required by the Accreditation Council for Graduate Medical Education to evaluate residents’ operative competency. Many such tools based on the model of objective structured assessment of technical skills (OSATSs) have been developed in other surgical specialties, but no such instruments exist for otolaryngologic procedures except for tonsillectomy. Endoscopic sinus surgery (ESS) is among the most common rhinologic procedure and lends itself to objective evaluation of operative competency. The purpose of this study was to develop and test a new tool for ESS, focusing on feasibility, content and construct validity, and interrater agreement that can be used for such assessment in the operating room and the cadaver dissection course. Methods Faculty input via the modified Delphi technique helped develop the content of a new OSATS-based instrument. The instrument underwent serial improvements based on 3 years of endoscopic sinus surgery (ESS) cadaver courses. All evaluations were used to calculate construct validity while paired observations were used to determine interrater agreement. Regional and national faculty input was incorporated for increasing generalizability. Internal consistency was calculated using Cronbachs alpha. Results A total of 51 assessments were completed for 28 residents who were evaluated by 15 faculty members as they performed ESS on cadavers over a period of 3 years. A high degree of internal consistency (0.99) and feasibility was noted for the instrument, which took 7 minutes to complete. The interrater agreement improved with focused faculty development for the 3rd year of the course. Conclusion Our results and experience suggest that a feasible, reliable, and valid instrument for objective evaluation of operative competency can be developed for ESS. Further experience at other otolaryngology programs and efforts focused on faculty development will be needed to enhance faculty buy-in. The instrument can be used for formative and summative feedback as well as for identifying residents needing remediation.

Objective assessment of mastoidectomy skills in the operating room

Objective(s): To determine the feasibility and validity of an objective assessment tool designed to measure the development of mastoidectomy skills by resident trainees in the operating room. Study Design: Prospective longitudinal validation study. Setting: Tertiary referral center and residency training program. Subjects: Otolaryngology residents. Main Outcome Measure: Technical performance as measured over time using Task-Based Checklist (TBC) and Global Rating Scale (GRS) developed for assessment of mastoidectomy skills. Results: Seventy pairs of evaluations were completed on 15 residents, showing strong correlation between both instruments (r = 0.93; p < 0.0001). Our instrument demonstrated construct validity for both TBC and GRS, showing higher scores with increasing surgical experience in otology. Both instruments showed high interitem reliability with Cronbach &agr; coefficients of 0.98 and 0.95 for TBC and GRS, respectively. Regression analysis showed that thinning posterior external auditory canal (p < 0.05) and opening antrum to deepen dissection at sinodural angle (p < 0.05) were the strongest predictors of overall surgical performance. Conclusion: Our assessment tool is a feasible and valid method of evaluating acquisition of mastoidectomy skills in the operating room. It can be integrated into surgical teaching in the operating room and yields information for direct formative feedback.

Video-based assessment of operative competency in endoscopic sinus surgery

Background The Accreditation Council for Graduate Medical Education requires that residency programs have an effective plan for assessing their residents’ competence. Current evaluation tools used for the assessment of operative competency are subjective, recall based, and do not allow providing formative feedback to the resident. The purpose of this study was to evaluate the feasibility and validity of video-based assessment of endoscopic sinus surgery (ESS) skills. Methods Five junior (postgraduate year levels 1–3[PGY-1 to -3]) and three senior (PGY-4 and -5) residents were videotaped as they performed ESS on cadavers in the laboratory. Five faculty members watched the recorded videotapes and rated performance using global and checklist assessment tools for ESS. Evaluators were blinded to the identity of the performers. Rating was made feasible by allowing the raters to fast-forward through the tapes when appropriate. Construct validity was calculated by comparing total scores and score on each item of the instrument across the training levels. For all statistical purposes p < 0.05 was considered significant. Results Results showed construct validity with senior residents performing better than junior residents. It took an average of 20 minutes (range, 7- 39 minutes) to watch and evaluate a video. Interrater reliability, as measured by interclass correlation coefficient across evaluators, was 0.62. Conclusion The use of videotapes combined with the objective assessment tool is a feasible and valid method for evaluating surgical skills. It has implications for residency programs, because it will increase faculty buy-in and reduce the potential for biased assessment.

Defining milestones toward competency in mastoidectomy using a skills assessment paradigm.

To establish milestones toward achievement of surgical competency by using an objective assessment tool designed to measure the development of mastoidectomy skills in the operating room (OR).

Learning curve for competency in flexible laryngoscopy.

The purpose of our study was to identify the number of attempts required to attain competency in performing flexible laryngoscopy.

Assessment of improvement of trainee surgical skills in the operating room for tonsillectomy

To determine the feasibility, reliability, and validity of an objective surgical assessment tool designed to measure the development of tonsillectomy skills by resident trainees in the operating room. A tonsillectomy evaluation instrument created previously serves as a basis for this tool, which incorporates many similar concepts but was optimized to maximize feasibility and reliability with simplified anchors and inclusion of global and task‐specific sections.

In-training assessment and predictors of competency in endoscopic sinus surgery†‡§

To evaluate the reliability and validity of the assessment tool for endoscopic sinus surgery when used in the operating room for in‐training assessment of operative competency; to identify the tasks that may serve as the best indicators for overall surgical performance.

Learning styles in two otolaryngology residency programs.

Kolb portrays four learning styles depending on how an individual grasps or transforms experience: accommodating, assimilating, diverging, and converging. Past studies in surgery, medicine, and anesthesia identified the predominant learning style in each of these specialties. The prevalence of different learning styles and existence of a predominant style, if any, has not been reported for otolaryngology residency programs. The purpose of our study was to determine if otolaryngology residents have a preferred learning style that is different from the predominant learning styles reported for other specialties.

Achievement of competency in endoscopic sinus surgery of otolaryngology residents

The goal of our study is to identify the number of Endoscopic Sinus Surgery (ESS) cases required to obtain competency in ESS, using a previously validated assessment tool.