Richard W. Schwartz

The objective structured clinical examination : the new gold standard for evaluating postgraduate clinical performance

ObjectiveThe authors determine the reliability, validity, and usefulness of the Objective Structured Clinical Examination (OSCE) in the evaluation of surgical residents. Summary Background DataInterest is increasing in using the OSCE as a measurement of clinical competence and as a certification tool. However, concerns exist about the reliability, feasibility, and cost of the OSCE. Experience with the OSCE in postgraduate training programs is limited. MethodsA comprehensive 38-station OSCE was administered to 56 surgical residents. Residents were grouped into three levels of training: interns, junior residents, and senior residents. The reliability of the examination was assessed by coefficient α; its validity, by the construct of experience. Differences between training levels and in performance on the various OSCE problems were determined by a three-way analysis of variance with two repeated measures and the Student-Newman-Keuls post hoc test. Pearson correlations were used to determine the relationship between OSCE and American Board of Surgery in-Training Examination (ABSITE) scores. ResultsThe reliability of the OSCE was very high (0.91). Performance varied significantly according to level of training (postgraduate year; p < 0.0001). Senior residents performed best, and interns performed worst. The OSCE problems differed significantly in difficulty (p < 0.0001). Overall scores were poor. Important and specific performance deficits were identified at all levels of training. The ABSTTE clinical scores, unlike the basic science scores, correlated modestly with the OSCE scores when level of training was held constant. ConclusionThe OSCE is a highly reliable and valid clinical examination that provides unique information about the performance of individual residents and the quality of postgraduate training programs.

Transfer of training in acquiring laparoscopic skills.

BACKGROUND Building on skills already learned in acquiring more complex or related skills is termed transfer of training (TOT). This study examined the TOT effects of previous open and laparoscopic surgical experience on a laparoscopic training module. STUDY DESIGN Intracorporeal knot tying was chosen for evaluating TOT among three groups of surgical residents: interns (n = 11) with limited open and laparoscopic surgical experience, junior residents (n = 9) with recent and ongoing open and laparoscopic surgical experience, and senior residents (n = 8) with remote and limited laparoscopic experience but ongoing open surgical experience. After receiving a lecture, demonstration, and written instructions on three knot-tying techniques, residents rotated through three performance stations, one for each technique, over 2 days. After 15 minutes of practice, the residents were videotaped completing a test knot. Time to completion and economy of motion were recorded and analyzed. RESULTS Junior residents had fewer performance errors than senior residents (reported as mean +/- standard error of the mean) and were significantly faster than interns. No significant differences between interns and senior residents for mean time or error performance were observed. Senior residents did not demonstrate TOT from open surgical experience to laparoscopic knot tying. No significant differences were obtained across the three sessions for errors or for time. CONCLUSION No evidence was found for TOT from open surgical experience to newly introduced laparoscopic knot-tying techniques or from one skill training session to a different skill session at least 4 hours later. This study indicates that specific minimally invasive surgery training is needed to develop laparoscopic surgery skills.

Controllable Lifestyle: A New Factor in Career Choice by Medical Students.

Abstract To determine whether control of work hours (controllable lifestyle) was becoming an increasingly important factor in choices of specialties by medical students, data from three medical schools over the past ten, ten, and six years, respectively, were reviewed for the types of specialty training entered by students in the top 15% of their classes. Since students in the upper 15% of the class are likely to obtain the specialties of their choice, any change in the pattern of their specialty preferences probably reflects a general trend. Specialties that feature a controllable lifestyle (CL) were defined as anesthesiology, dermatology, emergency medicine, neurology, ophthalmology, otolaryngology, pathology, psychiatry, and radiology. Non‐CL specialties were surgery, medicine, family practice, pediatrics, and obstetrics‐gynecology. The results showed that the percentages of students entering CL specialties increased significantly at all three schools, the percentages of students entering non‐CL specialties decreased significantly at all three schools, and there was no significant change in the percentage of students entering surgical specialties. Acad. Med. 64(1989):606–609.: To determine whether control of work hours (controllable lifestyle) was becoming an increasingly important factor in choices of specialties by medical students, data from three medical schools over the past ten, ten, and six years, respectively, were reviewed for the types of specialty training entered by students in the top 15% of their classes. Since students in the upper 15% of the class are likely to obtain the specialties of their choice, any change in the pattern of their specialty preferences probably reflects a general trend. Specialties that feature a controllable lifestyle (CL) were defined as anesthesiology, dermatology, emergency medicine, neurology, ophthalmology, otolaryngology, pathology, psychiatry, and radiology. Non-CL specialties were surgery, medicine, family practice, pediatrics, and obstetrics-gynecology. The results showed that the percentages of students entering CL specialties increased significantly at all three schools, the percentages of students entering non-CL specialties decreased significantly at all three schools, and there was no significant change in the percentage of students entering surgical specialties.

The Controllable Lifestyle Factor and Students' Attitudes about Specialty Selection.

Questionnaires were distributed to 346 fourth-year students in nine medical schools. The students were asked to state their selected specialty and to rank the importance that each of 25 influences, listed as questionnaire items, had had in making their choice of specialty. Factor analysis showed that particular items were significantly associated with particular factors. The first factor emphasized perceived lifestyle (items in this category gave importance to remuneration, personal time, and prestige); the second factor emphasized cerebral activities and a practice orientation; and the third factor stressed altruistic values and attitudes. The authors classified the selected specialties into three groups: those characterized as having a non-controllable lifestyle (NCL), those with a controllable lifestyle (CL), and surgery. (CL specialties were defined as those that allow the physician to control the number of hours devoted to practicing the specialty.) Data were analyzed using factor analysis, and analysis of variance, and the Scheffe method. Analysis indicated that the perceived lifestyle factor was most closely associated with the responses of those students choosing CL specialties. Furthermore, this factor received the highest total loading of the three factors from all the students, thus indicating the level of interest in lifestyle factors. Responses to items that defined the cerebral and practice factor were highest from the group of students choosing CL specialties and lowest from the group choosing NCL specialties. The NCL students scored highest in the altruism factor and the CL students scored the lowest. The surgery and NCL groups were similar in attitude patterns, and both were substantially different in attitude patterns from those of the CL groups. (ABSTRACT TRUNCATED AT 250 WORDS)

Laparoscopic pancreatic surgery

BACKGROUND Potential applications for laparoscopic surgery in pancreatic disease include (1) staging of pancreatic malignancies; (2) palliation of pancreatic malignancies; (3) pancreatic resections for benign and malignant disease; and (4) pancreatic drainage procedures. METHODS A review of the literature is presented. In addition, original data on a series of 5 laparoscopic pancreatic distal resections and 10 laparoscopic cystogastrostomies are presented. RESULTS AND CONCLUSIONS Laparoscopy may have a role in the staging of patients with pancreatic malignancies; however, with high-quality preoperative imaging, the percentage of patients who will benefit from laparoscopy may be as low as 5%. For palliation, both cholecystoenterostomy and choledochoenterostomy can be performed laparoscopically. The former is technically straightforward but has a higher failure rate; the latter is technically difficult and currently not suitable for widespread adoption. Laparoscopic gastroenterostomy is a straightforward means of palliating gastrointestinal obstruction. Patients appear to benefit from laparoscopic distal pancreatic resection but not from laparoscopic pancreaticoduodenectomy. Patients appear to benefit from laparoscopic pseudocyst decompression.

Physician leadership: essential skills in a changing environment

Precisely because they are at the center of clinical service delivery, physicians, especially surgeons, are the ideal leaders for health care in the 21st century. Although most physicians possess the traits essential for leadership, the vast majority lacks the technical skills necessary for major leadership/management roles that will both change and empower the local healthcare service delivery environment. Such skills include strategic and tactical planning, persuasive communication, negotiation, financial decision-making, team building, conflict resolution, and interviewing. Just like surgical training, these skills too require systematic training. With patients beginning to demand value-added service, it is important for healthcare executives to identify those physicians best suited to serve as leaders within the larger healthcare system and to deliberately nurture their growth in these administrative competencies.

Student perceptions of tutor effectiveness in a problem‐based surgery clerkship

This study identified the characteristics of effective tutors in a problem‐based learning (PBL) educational setting. Forty‐four junior medical students participated in two 6‐week PBL groups and evaluated their tutors based on a list of 12 characteristics. Statistical analyses of the students’ responses revealed that faculty members differed significantly in their possession of tutor skills, in the way they carried out the tutor skills, and in their performance of group‐management skills. Tutors were rated highest on participation in the sessions, enthusiasm, and level of comfort outside their area of expertise. They were rated lowest on providing feedback to the group and promoting psychosocial issues. The results indicate that students are highly satisfied with overall tutor performance despite significant differences among tutors. Two important characteristics of the effective tutor were identified: (a) helping students identify important issues and (b) providing feedback to students while encouraging f...

Cancer pain assessment and management by housestaff.

&NA; Pain control for cancer is a significant problem in health care, and lack of expertise by clinicians in assessing and managing cancer pain is an important cause of inadequate pain management. This study was designed to use performance‐based testing to evaluate the skills of resident physicians in assessing and managing the severe chronic pain of a cancer patient. Thirty‐three resident physicians (PGY 1–6) were presented with the same standardized severe cancer pain patient and asked to complete a detailed pain assessment. The residents then completed questions related to management of the cancer pain patient. In the cancer pain assessment, residents did well in assessing pain onset (70%), temporal pattern of pain (64%), and pain location (73%). However, only 33% and 45% physicians adequately assessed the pain description and pain intensity, respectively, and assessment of pain‐relieving factors, previous pain history, and psychosocial history was done poorly or not at all by 70%, 88%, and 94% of residents. Only 58% of the residents were judged to be competent in this clinical cancer pain assessment. In the cancer pain management section, opioid analgesic therapy was prescribed by 98% of residents, and 91% used the oral route. However, only 18% of prescriptions were for regular use and 88% of residents did not provide analgesics for breakthrough pain. A significant number of graduated physicians were judged to be not competent in the assessment and management of the severe pain of a standardized cancer patient. Opioids and NSAIDs were the analgesics of choice; however, most were prescribed on a PRN basis only. Co‐analgesics were rarely prescribed. Few physicians managed persistent, severe cancer pain according to the WHO guideline of increasing the opioid dose. The lack of significant difference in scores between junior and senior residents suggest that adequate cancer pain management is not being effectively taught in postgraduate training programs.

Concepts in service marketing for healthcare professionals.

Patients are becoming increasingly involved in making healthcare choices as their burden of healthcare costs continues to escalate. At the same time, healthcare has entered a tightened market economy. For these reasons, the marketing of healthcare services has become essential for the financial survival of physicians and healthcare organizations. Physicians can successfully use the fundamental service marketing principles proven by other service industries to win patient satisfaction and loyalty and remain competitive in todays market economy. Understanding concepts such as service quality zone of tolerance, levels of consumer satisfaction, the branding of services, patient participation, and service recovery can be useful in achieving these goals.

Assessing residents' clinical performance: Cumulative results of a four-year study with the Objective Structured Clinical Examination

BACKGROUND The Objective Structural Clinical Examination (OSCE) is an objective method for assessing clinical skills and can be used to identify deficits in clinical skill. During the past 5 years, we have administered 4 OSCEs to all general surgery residents and interns. METHODS Two OSCEs (1993 and 1994) were used as broad-based examinations of the core areas of general surgery; subsequent OSCEs (1995 and 1997) were used as needs assessments. For each year, the reliability of the entire examination was calculated with Cronbachs alpha. A reliability-based minimal competence score (MCS) was defined as the mean performance (in percent) minus the standard error of measurement for each group in 1997 (interns, junior residents, and senior residents). RESULTS The reliability of each OSCE was acceptable, ranging from 0.63 to 0.91. The MCS during the 4-year period ranged from 45% to 65%. In 1997, 4 interns, 2 junior residents, and 2 senior residents scored below their groups MCS. MCS for the groups increased across training levels in developmental fashion (P < .05). CONCLUSIONS Given the relatively stable findings observed, we conclude (1) the OSCE can be used to identify group and individual differences reliably in clinical skills, and (2) we continue to use this method to develop appropriate curricular remediation for deficits in both individuals and groups.