Do Resident Surgical Volumes and Level of Training Correlate with Improved Performance on Psychomotor Skills Tasks: Construct Validity Testing of an ASSH Training Platform (STEP)?

Abstract

Background: The Surgical Training and Educational Platform (STEP) was developed by the American Society for Surgery of the Hand (ASSH) as a cost-effective set of surgical simulation modules designed to assess critical skills in hand surgery. Previous study demonstrated that STEP can differentiate between novice trainees and board-certified, certificate of added qualification hand surgeons. The purpose of this study was to assess construct validity of STEP by testing its ability to differentiate psychomotor skill level among intermediate trainees. Methods: We evaluated 30 residents from 2 orthopaedic residency programs on 8 modules: (1) lag screw fixation, (2) depth of plunge during bicortical drilling, (3) flexor tendon repair, (4) phalangeal fracture pinning, (5) central axis scaphoid fixation, (6) full-thickness skin graft harvest, (7) microsurgery, and (8) wrist arthroscopy. Spearman correlation was used to correlate total and task-specific scores to case log numbers, months in training, and number of hand surgery rotations. Results: Senior residents had significantly higher mean number of total cases in their total case log (mean difference 96.2, 95% confidence interval [CI] 67.5-124.8, p < 0.01) and number of task-specific cases. Moderate correlation was observed between case log numbers and scaphoid fixation score (rs = 0.423, 95% CI 0.07-0.69) and total score (rs = 0.584, 95% CI 0.25-0.79). Moderate correlation was observed between months in training with: scaphoid fixation (rs = 0.377, 95% CI 0.01-0.66) and microsurgery (rs = 0.483, 95% CI 0.13-0.73); strong correlation was seen with total score (rs = 0.656, 95% CI 0.35-0.83). Moderate correlation with number of hand surgery rotations was observed with tendon repair (rs = 0.362, 95% CI −0.01 to 0.65), skin graft (rs = 0.385, 95% CI 0.01-0.66), wrist arthroscopy (rs = 0.391, 95% CI 0.02-0.67), microsurgery (rs = 0.461, 95% CI 0.10-0.71), and scaphoid fixation (rs = 0.578, 95% CI 0.25-0.79); and strong correlation was seen with total score (rs = 0.670, 95% CI 0.37-0.84). Discussion/Conclusion: The STEP is a validated ASSH education tool that provides a cost-effective simulation for the assessment of fundamental psychomotor skills in hand surgery. Total STEP score correlated with total task-related case volumes as well as months in training and number of hand rotations. Scoring could be modified to improve the fidelity of assessing surgical performance. And, while both time and human resource consuming to perform, administer, and score, this study demonstrates construct validity of STEP in assessing the progression of surgical skill through residency.