Analia Castiglioni

A Pilot Study Using Nominal Group Technique to Assess Residents’ Perceptions of Successful Attending Rounds

ABSTRACTBACKGROUNDWard attending rounds are fundamental for internal medicine residency training. An improved understanding of interns’ and residents’ perceptions of attending rounds should inform training programs and attending physicians.OBJECTIVESThe aim of this study was to assess residents’ perceptions of successful attending rounds.DESIGNWe convened two groups of interns and two groups of residents, to elicit their perceptions on attending rounds.SUBJECTSParticipants were recruited by e-mail and conference announcements from the 49 interns and 80 residents in the internal medicine and medicine-pediatrics residency programs.MEASUREMENTSThe nominal group technique (NGT) uses a structured group process to elicit and prioritize answers to a carefully articulated question.MAIN RESULTSSeven interns (14%) identified 27 success factors and ranked attending approachability and enthusiasm and high quality teaching as most important. A second group of six (12%) interns identified 40 detractors and ranked having “mean attendings,” receiving disrespectful comments, and too long or too short rounds as the most significant detractors. Nine (11%) residents identified 32 success factors and ranked attention to length of rounds, house staff autonomy, and establishing goals/expectations as the most important success factors. A second group of six (8%) residents identified 34 detractors and ranked very long rounds, interruptions and time constraints, and poor rapport between team members as the most significant detractors).CONCLUSIONSAlthough there was some overlap in interns’ and residents’ perceptions of attending rounds, interns identified interpersonal factors as the most important factors; whereas residents viewed structural factors as most important. These findings should assist attending physicians improve the way they conduct rounds targeting both interns and residents needs.

Program directors' views of the importance and prevalence of mentoring in internal medicine residencies.

Program directors establish priorities for residency programs. Their views of mentoring likely influence the tone about and availability of mentoring programs. This survey to all internal medicine program directors assessed attitudes toward mentoring, existence of formal mentoring programs, and features of the mentoring programs. The response was 60%. The majority (>60%) favored mentoring. Forty-nine percent of the residencies had structured mentoring programs. Programs differed in frequency of meetings, use of evaluations, and presence of curriculum. Overall, although program directors’ attitudes are favorable and at least half have formal mentoring programs, the programs are largely unstructured, loosely monitored, and underevaluated.

Succeeding as a Clinician Educator: Useful Tips and Resources

Clinician Educators (CEs) play an essential role in the education and patient care missions of academic medical centers. Despite their crucial role, academic advancement is slower for CEs than for other faculty. Increased clinical productivity demands and financial stressors at academic medical centers add to the existing challenges faced by CEs. This perspective seeks to provide a framework for junior CEs to consider with the goal of maximizing their chance of academic success. We discuss six action areas that we consider central to flourishing at academic medical centers: 1. Clarify what success means and define goals; 2. Seek mentorship and be a responsible mentee; 3. Develop a niche and engage in relevant professional development; 4. Network; 5. Transform educational activities into scholarship; and 6. Seek funding and other resources.

Using cognitive mapping to define key domains for successful attending rounds.

BACKGROUNDWard attending rounds are an integral part of internal medicine education. Being a good teacher is necessary, but not sufficient for successful rounds. Understanding perceptions of successful attending rounds (AR) may help define key areas of focus for enhancing learning, teaching and patient care.OBJECTIVEWe sought to expand the conceptual framework of 30 previously identified attributes contributing to successful AR by: 1) identifying the most important attributes, 2) grouping similar attributes, and 3) creating a cognitive map to define dimensions and domains contributing to successful rounds.DESIGNMulti-institutional, cross-sectional study design.PARTICIPANTSWe recruited residents and medical students from a university-based internal medicine residency program and a community-based family medicine residency program. Faculty attending a regional general medicine conference, affiliated with multiple institutions, also participated.MAIN MEASURESParticipants performed an unforced card-sorting exercise, grouping attributes based on perceived similarity, then rated the importance of attributes on a 5-point Likert scale. We translated our data into a cognitive map through multi-dimensional scaling and hierarchical cluster analysis.KEY RESULTSThirty-six faculty, 49 residents and 40 students participated. The highest rated attributes (mean rating) were “Teach by example (bedside manner)” (4.50), “Sharing of attending’s thought processes” (4.46), “Be approachable—not intimidating” (4.45), “Insist on respect for all team members” (4.43), “Conduct rounds in an organized, efficient & timely fashion” (4.39), and “State expectations for residents/students” (4.37). Attributes were plotted on a two-dimensional cognitive map, and adequate convergence was achieved. We identified five distinct domains of related attributes: 1) Learning Atmosphere, 2) Clinical Teaching, 3) Teaching Style, 4) Communicating Expectations, and 5) Team Management.CONCLUSIONSWe identified five domains of related attributes essential to the success of ward attending rounds.

Does Residency Training Improve Performance of Physical Examination Skills

Background:Few studies use objective structured clinical examinations (OSCEs) to measure physical examination skills of internal medicine residents. Little is known about performance by year of residency training. Purpose:To determine differences between postgraduate year (PGY)-1 and PGY-3 residents on performance and comfort of physical examination skills. Methods:In a cross-sectional study, we tested 16 PGY-1 (weeks 0 and 4) and 8 PGY-3 internal medicine residents with a five-station OSCE. Results:PGY-3 residents performed better than PGY-1 week 0 residents (P = 0.03) but not PGY-1 week 4 residents (P = 0.42). PGY-1 resident performance improved after 1 month of inpatient wards experience (P < 0.001). PGY-3 residents had higher comfort compared to PGY-1 week 0 residents (P = 0.003) but not PGY-1 week 4 residents (P = 0.10). Conclusions:Senior residents performed better and were more confident on physical examination skills, but the difference disappeared after 1 month of internship. This calls into question how much further learning occurs with physical examination throughout residency training.

Impact of an educational intervention on provider knowledge, attitudes, and comfort level regarding counseling women ages 40–49 about breast cancer screening

Background Mammography screening for women under the age of 50 is controversial. Groups such as the US Preventive Services Task Force recommend counseling women 40–49 years of age about mammography risks and benefits in order to incorporate the individual patient’s values in decisions regarding screening. We assessed the impact of a brief educational intervention on the knowledge and attitudes of clinicians regarding breast cancer screening. Methods The educational intervention included a review of the risks and benefits of screening, individual risk assessment, and counseling methods. Sessions were led by a physician expert in breast cancer screening. Participants were physicians and nurses in 13 US Department of Veterans Affairs primary care clinics in Alabama. Outcomes were as follows: 1) knowledge assessment of mammogram screening recommendations; 2) counseling practices on the risks and benefits of screening; and 3) comfort level with counseling about screening. Outcomes were assessed by survey before and after the intervention. Results After the intervention, significant changes in attitudes about breast cancer screening were seen. There was a decrease in the percentage of participants who reported that they would screen all women ages 40–49 years (82% before the intervention, 9% afterward). There was an increase in the percentage of participants who reported that they would wait until the patient was 50 years old before beginning to screen (12% before the intervention, 38% afterward). More participants (5% before, 53% after; P<0.001) said that they would discuss the patient’s preferences. Attitudes favoring discussion of screening benefits increased, though not significantly, from 94% to 99% (P=0.076). Attitudes favoring discussion of screening risks increased from 34% to 90% (P<0.001). The comfort level with discussing benefits increased from a mean of 3.8 to a mean of 4.5 (P<0.001); the comfort level with discussing screening risks increased from 2.7 to 4.3 (P<0.001); and the comfort level with discussing cancer risks and screening preferences with patients increased from 3.2 to 4.3 (P<0.001). (The comfort levels measurements were assessed by using a Likert scale, for which 1= not comfortable and 5= very comfortable.) Conclusion Most clinicians in the US Department of Veterans Affairs ambulatory practices in Alabama reported that they routinely discuss mammography benefits but not potential harms with patients. An educational intervention detailing recommendations and counseling methods affected the knowledge and attitudes about breast cancer screening. Participants expressed greater likelihood of discussing screening options in the future.

Evaluation of an advanced physical diagnosis course using consumer preferences methods: the nominal group technique.

Background:Current evaluation tools of medical school courses are limited by the scope of questions asked and may not fully engage the student to think on areas to improve. The authors sought to explore whether a technique to study consumer preferences would elicit specific and prioritized information for course evaluation from medical students. Methods:Using the nominal group technique (4 sessions), 12 senior medical students prioritized and weighed expectations and topics learned in a 100-hour advanced physical diagnosis course (4-week course; February 2012). Students weighted their top 3 responses (top = 3, middle = 2 and bottom = 1). Results:Before the course, 12 students identified 23 topics they expected to learn; the top 3 were review sensitivity/specificity and high-yield techniques (percentage of total weight, 18.5%), improving diagnosis (13.8%) and reinforce usual and less well-known techniques (13.8%). After the course, students generated 22 topics learned; the top 3 were practice and reinforce advanced maneuvers (25.4%), gaining confidence (22.5%) and learn the evidence (16.9%). The authors observed no differences in the priority of responses before and after the course (P = 0.07). Conclusions:In a physical diagnosis course, medical students elicited specific and prioritized information using the nominal group technique. The course met student expectations regarding education of the evidence-based physical examination, building skills and confidence on the proper techniques and maneuvers and experiential learning. The novel use for curriculum evaluation may be used to evaluate other courses—especially comprehensive and multicomponent courses.

Teaching behaviors that define highest rated attending physicians: A study of the resident perspective

Abstract Background: Better understanding teaching behaviors of highly rated clinical teachers could improve training for teaching. We examined teaching behaviors demonstrated by higher rated attending physicians. Methods: Qualitative and quantitative group consensus using the nominal group technique (NGT) among internal medicine residents and students on hospital services (2004–2005); participants voted on the three most important teaching behaviors (weight of 3 = top rated, 1 = lowest rated). Teaching behaviors were organized into domains of successful rounding characteristics. We used teaching evaluations to sort attending physicians into tertiles of overall teaching effectiveness. Results: Participants evaluated 23 faculty in 17 NGT sessions. Participants identified 66 distinct teaching behaviors (total sum of weights [sw] = 502). Nineteen items had sw ≥ 10, and these were categorized into the following domains: Teaching Process (n = 8; sw = 215, 42.8%), Learning Atmosphere (n = 5; sw = 145, 28.9%), Role Modeling (n = 3; sw = 74, 14.7%) and Team Management (n = 3; sw = 65, 12.9%). Attendings in the highest tertile received a larger number of votes for characteristics within the Teaching Process domain (56% compared to 39% in lowest tertile). Conclusions: The most effective teaching behaviors fell into two broad domains: Teaching Process and Learning Atmosphere. Highest rated attending physicians are most recognized for characteristics in the Teaching Process domain.

OSCE standards-setting procedure facilitated by digital technology.

What problems were addressed? There is a need to declare standards for high-stakes objective structured clinical examination (OSCE) assessments. A number of methods have been described, including variations of the Hofstee, borderline regression and Angoff methods. All require preparation of the evaluators (judges) and a discussion process which yields greater defensibility and credibility of pass/ fail cut points. One challenge posed in this process concerns the real-time manipulation of data such that the judges (clinicians and clinical educators) can complete the evaluation and discussion procedure efficiently. Data collection sheets and manual input of data were used in the past; however, mobile technologies are now available to increase efficiency and allow for the real-time review of cases and of agreement on judgements. What was tried? Our medical college has purchased a survey tool called Qualtrics (qualtrics.com) that is used for student and faculty questionnaires and is available on an intuitive iPad platform. Judges are invited by team leaders to join the survey via an e-mail which provides a secure survey link. The interface allows the judges to choose the likelihood that a given ‘borderline’ student will perform the assessment item successfully, using a sliding bar which serves as a visual analogue scale of infinite definition; the scale is supported by a three-digit number (000–100) such that a user sees exactly what score the mobile slider scale represents. Once all items have been addressed, each user submits the survey and the Qualtrics platform allows for the immediate export of a datasheet which can be used to review all of the judgements. We conducted a 2.5-hour standard-setting session with eight faculty members to complete a three-station OSCE Angoff procedure. The session began with a 30-minute orientation to the Angoff process, and a discussion of the definition of ‘borderline’ performance and judgement agreement protocol. Once all faculty staff were ready to progress, we presented the first case summary followed by deployment of the first Qualtrics survey. Survey results were exported into Excel and the whole group reviewed the judgements. In the event that a minimum–maximum range exceeded 20%, the judges discussed the case until they reached agreement; the process was repeated for the three stations. The impact data collected from the judges illustrate the ease of use, thoughtful and visual decision making, and immediacy of results to be obtained by converging iPad’s mobile technology with Qualtrics. What lessons were learned? The process was completed in 2.5 hours despite five of the eight judges being completely na€ıve to the Angoff method. Only one judge had been involved in a similar exercise in the past. The technology allowed for the immediate review of responses from the panel and facilitated what would have otherwise been a significant amount of manual data entry (20–25 items per case per judge). Furthermore, the technology promises to facilitate the performance of initial judgements away from campus, thus allowing the judges to convene only for the discussion of controversies.

AM last page. Using Visual Radar Graph Representation of Learner Achievement to Complement the RIME Framework.

• The RIME (Reporter, Interpreter, Manager, Educator) evaluation framework developed by Pangaro can be used to yield insight into a trainee’s development. • A previous AM Last Page presented a tabular map of the RIME framework as it relates to the Accreditation Council for Graduate Medical Education (ACGME) competencies. • Keister and colleagues have validated the use of radar graphs as promising tools for feedback and assessment of trainees. • Achievement of milestones and progression along the RIME axes can be visually represented for formative or summative evaluation. • Modification to Pangaro’s developmental markers (Introduced, Repeated practice, Proficiency, Mastery) with a minor mathematical manipulation allows representation as a visual radar graph for the RIME framework. – The mathematical conversion is necessary to give relative value to each step in the progress. – The conversion scale can be developed locally by programs to highlight the relative value given to these constructs at individual training programs. Using Visual Radar Graph Representation of Learner Achievement to Complement the RIME Framework Juan Cendán, MD, associate professor of surgery and assistant dean for simulation, Caridad Hernández, MD, associate professor of medicine, and Analia Castiglioni, MD, associate professor of medicine, University of Central Florida, College of Medicine