The peritoneal dialysis orders objective structured clinical examination (OSCE): A formative assessment for nephrology fellows

Abstract

Background: Peritoneal dialysis (PD) management is a fundamental nephrology skill, especially with the recent emphasis on home dialysis. We report a prospective multicentre cohort study of a formative objective structured clinical examination (OSCE) assessing competence in managing PD-associated bacterial peritonitis, using the unified model of construct validity. Methods: The OSCE was developed by the principal investigators and reviewed by two subject matter experts. The test committee (eight nephrologists and one PD nurse) assessed test item difficulty/relevance and determined passing score. There were 22 test items (7 evidence-based/standard-of-care questions). Passing score was 16/22 (73%). No item had median relevance less than ‘important’, and all were easy to medium difficulty. Content validity index was 0.91. Preliminary validation (16 board-certified volunteers): mean score was 19 ± 2, with 94% (15/16) passing. Kappa = 0.85 [95% confidence interval (CI) 0.77–0.94]. Cronbach’s α = 0.70. Results: Eighty-seven fellows (16 programmes) were tested; 67% passed. Fellows scored significantly less than validators: 17 ± 3 versus 19 ± 2, p < 0.001 [95% CI 1.2–3.6]. Eighty-six per cent of evidence-based/standard-of-care questions were answered correctly by validators versus 54% by fellows; p < 0.001. Ninety-three per cent of fellows recognized that sufficient criteria were present to diagnose peritonitis, but only 17% correctly indicated all three. Seventy-seven per cent recognized peritonitis-associated ultrafiltration failure, but only 17% prescribed 21 days of antibiotic treatment for gram-negative peritonitis. Eighty-five per cent of fellows surveyed agreed/strongly agreed that the OSCE was useful in self-assessing proficiency. Second-year in-training examination and OSCE scores were positively correlated (Pearson’s r = 0.57, p < 0.00). Conclusions: The OSCE may be used to formatively assess fellow proficiency in managing PD-associated peritonitis.