Margaret A. Plymale

Assessment of residents' interpersonal skills by faculty proctors and standardized patients: a psychometric analysis.

The objective structured clinical examination (OSCE) has typically been found to be a valid and reliable method for assessing clinical knowledge and skills when evaluating performances of residents. For example, Sloan et al. found a 19-problem, 38-station OSCE to be reliable (rxx = .91) and valid in assessing the clinical skills of 56 surgical residents. Often, OSCE performance is summarized in an overall score, which may represent a combination of history, physical examination, interpersonal and communication skills, technical skills, and organization. Interpersonal skills scores are sometimes reported separately because of their importance in overall performance. Warf et al. found that when faculty judges evaluated general surgery residents’ performances on a neurosurgical station there was no statistically significant difference between the junior and senior residents in performing the neurologic examination. Since general surgery residents do not receive training in neurosurgery beyond their intern year, it was not unexpected that there was no significant difference between levels of training. However, the senior residents were judged to be competent significantly more frequently than were the junior residents. It was also found that interpersonal skills correlated significantly with both competence and level of training. This study suggested that interpersonal skills are a very important facet of clinical competence that differentiates between residents at different skill levels. Colliver et al. also found statistically significant correlations (in the .30 to .50 range) between interpersonal skills and clinical competence. Similarly, Sloan et al. found that global interpersonal skill judgments were moderately reliable and correlated highly with overall OSCE performance scores. Thus, it is clear that interpersonal skills are highly associated with the judged competency of medical students’ or residents’ performances. Several studies have raised the question of who should evaluate interpersonal skills, a faculty proctor (FP) or the standardized patient (SP). Given the increasing clinical demands on faculty time, it is important to know whether SPs can assess interpersonal skills as validly and reliably as faculty members. Cooper and Mira found that, on average, SPs gave more positive evaluations of communication skills of undergraduate medical students than did faculty members or other professional staff. They found that the communication scores derived from faculty’s ratings did not correlate with the scores derived from the SPs’ ratings. Finlay et al. assessed the communication skills of primary care physicians who had just received training in communication skills. Professional examiners and SPs evaluated the physicians’ communication skills by means of a checklist. The two sets of scores correlated between .40 and .50 on the different OSCE problems, indicating that the SPs’ evaluations cannot be used interchangeably with the faculty’s evaluations. In a test of the validity of eight faculty raters, Kalet et al. videotaped the performances of 21 year-two medical students. Faculty evaluated the interviewing skills of those students on two different occasions using a checklist. The correlations of the communication scores among faculty members were low. Furthermore, the correlations between a faculty member’s evaluations of the interviewing skills of the same students’ performances on two occasions were also low. A related question is whether checklist scores or global ratings provide more reliable and valid measures of performance. Regehr et al. compared the psychometric properties of checklists with those of global rating scales on an eight-problem OSCE given to residents at all levels of training. They found better reliability and construct validity for global rating scales than for checklists. On the other hand, Hodges et al. also evaluated the comparative reliability and validity of checklists and global ratings of communication skills. They found high correlations between global ratings and checklists. Based on this review of the literature, we conclude that interpersonal skills are an important component of clinical competence. Global ratings are at least as valid and reliable as checklist scores. However, the levels of reliability and validity of interpersonal-skills ratings have not been clearly established. Also, it is not clear whether faculty or SPs, provide the more valid and reliable evaluations. The purpose of this study was to determine the psychometric characteristics of global interpersonal skills ratings of faculty proctors (FPs) and SPs.

Enhancing the Clinical skills of surgical Residents through Structured Cancer education

Objective:To assess the short and long-term educational value of a highly structured, interactive Breast Cancer Structured Clinical Instruction Module (BCSCIM). Summary Background Data:Cancer education for surgical residents is generally unstructured, particularly when compared with surgical curricula like the Advanced Trauma Life Support (ATLS) course. Methods:Forty-eight surgical residents were randomly assigned to 1 of 4 groups. Two of the groups received the BCSCIM and 2 served as controls. One of the BCSCIM groups and 1 of the control groups were administered an 11-problem Objective Structured Clinical Examination (OSCE) immediately after the workshop; the other 2 groups were tested with the same OSCE 8 months later. The course was an intensive multidisciplinary, multistation workshop where residents rotated in pairs from station to station interacting with expert faculty members and breast cancer patients. Results:Residents who took the BCSCIM outperformed the residents in the control groups for each of the 7 performance measures at both the immediate and 8-month test times (P < 0.01). Although the residents who took the BCSCIM had higher competence ratings than the residents in the control groups, there was a decline in the faculty ratings of resident competence from the immediate test to the 8-month test (P < 0.004). Conclusions:This interactive patient-based workshop was associated with objective evidence of educational benefit as determined by a unique method of outcome assessment.

Faculty evaluation of surgery clerkship students: important components of written comments.

Quiet 0.96 Intelligence 0.93 Clinical reasoning skills 0.91 Fund of knowledge 0.91 Work ethic 0.89 Overall performance 0.88 Future life as a physician 0.85 Role on team 0.85 Will improve in time 0.83 Organization 0.81 Interpersonal skills with patients 0.79 Evidence of study 0.79 Written and/or oral skills 0.75 Professionalism 0.74 Motivation 0.73 Participates during rounds/conferences 0.68 Prepares for and participates in patient care activities 0.62 Initiative 0.60 Clinical skills 0.58 Interpersonal skills with staff 0.49 Miscellaneous; no category 0.20 General comments such as ‘‘nice guy’’ 0.17

Cancer pain education: a structured clinical instruction module for hospice nurses.

The learning experience with the Cancer Pain Structured Clinical Instruction Module (SCIM), a highly structured skills training course for medical students, has been reported favorably. The purpose of this study was to present the Cancer Pain SCIM to registered nurses employed in a hospice setting. The goal of the study was to pilot test a structured cancer pain educational program for hospice nurses and to determine the perceived effectiveness of this course on the participants’ cancer pain assessment and management skills. A multidisciplinary Cancer Pain SCIM was presented to 25 hospice nurses to improve their understanding of the management of cancer pain. The development group identified essential aspects of cancer pain management and then developed checklists defining specific station content. During the 2-hour Cancer Pain SCIM, nurses rotated through 8 stations in groups of 3, spending 15 minutes at each station. Eight instructors and 6 standardized patients, 5 of whom were survivors of cancer, participated in the course. All participants (students, instructors, and patients) evaluated the course, using a 5-point Likert scale (1 = strongly disagree; 5 = strongly agree). Nurses provided self-assessments of their perceived competence on important aspects of cancer pain management both before and after the SCIM. The self-assessment items used a 5-point scale ranging from 1 (not competent) to 5 (very competent). Twenty-five hospice nurses, averaging 4.1 years (range 1–30 years) postgraduation, participated in the Cancer Pain SCIM. Overall, nurses agreed that they improved on each of the 8 teaching items (P < 0.001). The average (SD) pretest score of 2.8 (0.72) improved to 3.8 (0.58) on the post-test (P < 0.001). Nurses believed that their mastery of specific clinical skills, taught in all 8 stations, improved as a result of participation in the course. Nurses strongly agreed (mean ± SD) that it was beneficial to use patients with cancer in the course (4.6 ± 0.82). Faculty members enjoyed participating in the course (4.9 ± 0.35) and indicated a willingness to participate in future courses (4.7 ± 0.49). Significant perceived learning among hospice nurses took place in all aspects of the Cancer Pain SCIM. Participating nurses, instructors, and patients with cancer appreciated the SCIM format. Nurses and faculty considered the participation of actual patients with cancer highly beneficial. The SCIM format has great potential to improve the quality of cancer pain education.

A Middle Fidelity Model Is Effective in Teaching and Retaining Skill Set Needed to Perform a Laparoscopic Pyloromyotomy

INTRODUCTION An inanimate technical skills trainer for laparoscopic pyloromyotomy (LP) has not been described. A middle fidelity model, reproducing the three consistent steps in LP, was developed as a component of a teaching module for surgical residents, and tested on medical students, residents, and pediatric surgeons. MATERIALS AND METHODS In the first phase of the study, a cohort of 29 pediatric surgeons used the LP model and completed questionnaires about the models realism and accuracy. Descriptive statistics were used to analyze questionnaire responses. Chi-square tests were performed to determine if level of experience influenced responses. For the second phase of the study, medical students and surgical residents individually participated in the training of cognitive knowledge about hypertrophic pyloric stenosis and skills acquisition for LP. Subject testing consisted of simulator task performance and multiple-choice quiz administration immediately after training and repeated at 8 weeks after training. Data were analyzed by using paired sample t-tests and one-way analyses of variance (ANOVA). RESULTS The pediatric surgeons agreed that the model accurately simulated essential components of the pyloromyotomy, and that the model would be an excellent tool to introduce surgeons to LP. A total of 26 students and early surgical residents completed the training and testing. Knowledge-based test performance improved from pre- to postinstruction by 17.45 [standard error of the mean (SEM) + 3.5] (P < 0.001) and from preinstruction to 8 weeks by 4.54 (SEM = 3.2) (P = 0.17). Mean improvement in time of simulator task performance was 85.2 +/- 75.4 seconds. Based on a one-way ANOVA, higher level of training was associated with decreased mean times (P = 0.04). CONCLUSIONS Face and content validities of the simulation were demonstrated by the pediatric surgeons. An effective training experience was demonstrated with medical students and residents. At 2 months, simulator task-completion rates and task-performance times showed technical skills were retained, whereas, based on test scores, cognitive knowledge was not as well retained.

Junior Medical Students’ Perceptions of an Introductory Hospice Experience

ABSTRACT Introduction: The importance of palliative care education in the medical school curriculum is becoming more recognized. The purpose of this study was to assess medical students’ perceptions of an introductory hospice experience. Methods: Forty-one second-year medical students took part in an introductory hospice experience in which they were acquainted with a wide range of hospice services provided to patients and families by an interdisciplinary team involved in hospice care. In addition, the students visited patients’ homes individually with an experienced hospice nurse or social worker. At the end of their experience, the students were asked to complete a multi-item evaluation questionnaire in order to share their perceptions of the hospice experience and their suggestions for improvement of the course. Results: The students spent an average of four hours on their introductory hospice experience, and they indicated that all of their personal goals for their experience had been met. Suggestions for improvement of the course were to increase the amount of course time allotted and to provide further opportunity to see more patients. Overall, the students rated their experience as “above average” to “excellent.” Conclusions: According to the medical students who participated, the introductory hospice experience was a worthwhile and valuable educational experience. An equal or increased amount of hospice time should be allotted in the education of future junior medical students. [Article copies available for a fee from The Haworth Document Delivery Service: 1-800-342-9678. E-mail address: Website:

Critical assessment of the head and neck clinical skills of general surgery residents.

Abstract. Head and neck surgery is an important part of general surgery. There is, however, little information about the quality of residents’ clinical skills in this important field. In an Objective Structured Clinical Examination (OSCE), residents encounter multiple patients with various clinical problems and are rated by faculty members using objective criteria. This study was undertaken to assess the head and neck surgery skills of a group of general surgical residents. Fifty-one general surgery residents examined the same nine patients with head and neck disease. Faculty members graded each clinical interaction according to preset objective criteria. Both actual (e.g., thyroid nodule, oral cancer follow-up examination) and simulated (e.g., dysphagia) patients were used in the OSCE. The reliability of the examination was assessed by coefficient α. The construct validity was determined by a two-way analysis of variance with one repeated measure. The reliability was 0.75 for the clinical examination. Performance varied by level of training: Residents performed at a higher level than interns (p< 0.0001), but overall scores were poor (mean score 55%). Important deficits in skills were identified at all levels of training. It is concluded that more attention should be focused on specific outcome assessments of surgical training programs and on strategies for upgrading the clinical skills of surgical residents.

360-Degree Evaluations of Plastic Surgery Resident Accreditation Council for Graduate Medical Education Competencies: Experience Using a Short Form

Background: The Accreditation Council for Graduate Medical Education has asked training programs to develop methods to evaluate resident performance, using competencies essential for outcomes. Methods: A two-page form was completed by 12 surgeons and 28 nurses and clinical staff directly involved in plastic surgery patient care (n = 40), evaluating University of Kentucky plastic surgery residents at each level of training (n = 6). There were eight groups of health care professionals among the 40. Six Accreditation Council for Graduate Medical Education competencies were rated, with technical/operative skills added as a subset of patient care. Hierarchical cluster analysis was used to determine similarity of rating profiles of the rating groups; Kruskal-Wallis analysis of variance delineated the way in which the participants used the competencies to make their selections by asking them whether they would choose the resident for future surgical care. Results: Rating profiles revealed two clusters of raters. In one cluster were nurses assigned to an ambulatory surgery center, faculty, residents, and an intern (the surgeons’ cluster; n = 15); in the second cluster were other nurses and clinical staff (nurses’ cluster; n = 25). The nurses’ cluster was found to rate residents more positively, and the surgeons’ cluster more often cited areas for improvement. Specific competencies deemed important to each group were identified. Conclusions: Resident performance is rated differently by health care professionals, in two distinct groups. Based on this clustered arrangement, the resident is able quarterly to enjoy two, independent, formative assessments, potentially over 6 years of integrated training.

Abdominal Wall Reconstruction: A Comparison of Totally Extraperitoneal and Transabdominal Preperitoneal Approaches.

BACKGROUND Abdominal wall reconstruction for complex ventral and incisional hernias is associated with significant complications. Commonly, the peritoneal cavity is opened and adhesiolysis is performed with the potential for enterotomy. A totally extraperitoneal (TE) approach to abdominal wall reconstruction is feasible in many ventral hernia repairs and can reduce visceral injuries without impacting other outcomes. This study compares outcomes after retro-rectus ventral hernia repairs with TE and transabdominal (TA) preperitoneal approaches. STUDY DESIGN An IRB-approved review of a prospective hernia database was performed for all ventral hernia repairs between 2009 and 2013. Preoperative patient characteristics, including demographics and comorbidities; operative variables, including surgical technique, operative duration, type/size/location of mesh, concomitant procedures, and incidence of inadvertent injury; and patient outcomes in terms of length of stay, wound and nonwound complications, and readmissions or returns to the operating room were obtained. Groups were compared using t-tests, Mann-Whitney U tests, chi-square tests, and Fishers exact tests as appropriate. Significance was set at p < .05. RESULTS One hundred and seventy-five complex abdominal wall reconstructions were performed between 2009 and 2013. Of those, 85 patients underwent hernia repair for CDC grade 1 hernias with retro-rectus mesh placement performed (n = 45 TA, n = 40 TE). Groups did not differ in age, BMI, sex, smoking status, hernia defect size, history of COPD, asthma, hypertension, cancer, or renal failure. More TA patients had diabetes (36% vs. 13%; p = 0.02) and previous hernia repair (73% vs. 45%; p = 0.01) than TE patients. Mesh size was larger in the TE group (625 ± 234 cm(2) vs. 424 ± 214 cm(2); p < .001). There was no difference in enterotomy between TA and TE groups (0% vs. 2%; p = 1.0). However, there was a reduced operative time with TE (170 ± 49 minutes vs. 212 ± 49 minutes; p < .001). CONCLUSIONS Abdominal wall reconstruction can be performed safely in a TE fashion. The extraperitoneal approach results in shorter operative duration, but had similar complications when compared with TA preperitoneal approach.

Early outcomes of an enhanced recovery protocol for open repair of ventral hernia

BackgroundEnhanced recovery after surgery (ERAS) protocols are evidence-based quality improvement pathways reported to be associated with improved patient outcomes. The purpose of this study was to compare short-term outcomes for open ventral hernia repair (VHR) before and after implementation of an ERAS protocol.MethodsAfter obtaining IRB approval, surgical databases were searched for VHR cases for two years prior and eleven months after protocol implementation for retrospective review. Groups were compared on perioperative characteristics and clinical outcomes using chi-square, Fisher’s exact, or Mann–Whitney U test, as appropriate.ResultsOne hundred and seventy-one patients underwent VHR (46 patients with ERAS protocol in place and 125 historic controls). Age, gender, ASA Class, comorbidities, and smoking status were similar between the two groups. Body mass index was lower among ERAS patients (p = .038). ERAS patients had earlier return of bowel function (median 3 vs. 4 days) (p = .003) and decreased incidence of superficial surgical site infection (SSI) (7 vs. 25%) (p = .008) than controls.ConclusionAn ERAS protocol for VHR demonstrated improved patient outcomes. A system-wide culture focused on enhanced recovery is needed to ensure improved patient outcomes.