Evaluation of a novel universal robotic surgery virtual reality simulation proficiency index that will allow comparisons of users across any virtual reality simulation curriculum

Abstract

Evaluate a universal proficiency metric for Robotic Surgery Virtual Reality (VR) simulation that will allow comparison of all users across any VR curriculum. Retrospective analysis of VR Simulation metrics. Two training institutions. Residents, fellows and practicing surgeons. Analysis of the Mimic robotic Virtual Reality (VR)-Simulation database of over 600,000 sessions was utilized to calculate Mean scores for each exercise. Those Mean scores were then normalized to 100. Subject s scores were also averaged and normalized to 100. We called this Index score the MScore Proficiency Index (MPI©). Scores above 100 were better than average; Less than 100 were worse than average. Seventeen thousand six hundred and forty eight sessions were analyzed (2017–2020) comparing 77 students (residents to practicing surgeons) working in 7 different curriculums. On average, each student spent 8 h and 24 min on simulation, attempted 26.5 different exercises, and became proficient in 20.6 exercises per user. The MPI© mean score for all participants in all curricula was an MPI© of 104.9 (SD: 15.5). Thirteen students were 1 standard deviation below the norm with an average MPI© of 80.15. This group averaged 9 h 27 min each on the simulator attempting 23.46 exercises but becoming proficient in only 10.38 (47%) of them in 224 sessions. Twelve students were 1 standard deviation above the norm with an average MPI© of 127.05. This group averaged 6 h 31 min each on the simulator attempting 29.08 exercises but becoming proficient in 27.5 (95%) of them in 196 sessions. A universal skill-based performance index (MPI©) was calculated and found to be a reliable tool that could be used to identify relative proficiency among students in different robotic surgery VR Simulation curriculums. An individual user’s proficiency can be utilized to identify a student’s progress in a given curriculum. Future studies of MPI© will determine if machine learning can provide timely personalized feedback to the user.