C. Scott Smith

Cultural consensus analysis as a tool for clinic improvements.

Some problems in clinic function recur because of unexpected value differences between patients, faculty, and residents. Cultural consensus analysis (CCA) is a method used by anthropologists to identify groups with shared values. After conducting an ethnographic study and using focus groups, we developed and validated a CCA tool for use in clinics. Using this instrument, we identified distinct groups with 6 important value differences between those groups. An analysis of these value differences suggested specific and pragmatic interventions to improve clinic functioning. The instrument has also performed well in preliminary tests at another clinic.

The impact of an ambulatory firm system on quality and continuity of care

The author assessed the effects on quality and continuity of care at a Veterans Affairs hospital as a result of its conversion to an interdisciplinary firm system. Before the firm system was implemented, ambulatory care at the hospital was provided in two medicine clinic areas and in one unscheduled “walk-in” clinic. Care for intercurrent illnesses was frequently not coordinated. The staff from eight clinical services were involved in restructuring into three, interdisciplinary firm teams. These firm teams were created without the addition of new staff. Quality was defined by patient satisfaction, staff satisfaction, re-admissions within 10 days of a hospital discharge, and length of visit. Continuity was defined by percentage of visits to the primary care team (defined as the physician or the physician paired with midlevel practitioner if applicable). Patient satisfaction increased from 4.43 to 4.84 (5- point Likert scale, P <.001). Staff satisfaction increased from 4.3 to 6.24 (7-point Likert scale, P <.001). Re-admissions within 10 days of hospital discharge decreased by 28% (P <.01). Length of visit decreased by 9.5% (P <.0001). Continuity improved from 47% to 69% of visits to the primary care team (P<.002). These results more than justified the staff time needed to convert to a firm system.

A randomized multicenter trial to improve resident teaching with written feedback

Background: Residents have an important role as teachers of medical students. Purpose: This study addressed assessment of the impact of written learner feedback on resident teaching. Methods: Senior residents (N = 30) were randomized into either an experimental group receiving feedback or a control group receiving no feedback. Team members were interviewed to identify the residents’ teaching strengths and weaknesses, and rate residents on overall teaching effectiveness and on eight specific teaching characteristics. Residents received a one‐page summary of interview comments and mean rating scores for three consecutive rotations. Results: Over a 1‐year period, there was a significant difference between the mean group ratings for the teaching characteristics Establishes Rapport (p < .01) and Provides Direction and Feedback (p < .05). Overall Teaching Effectiveness tended to improve for the experimental group over the year and remain stable for the control group (p = .07). Conclusions: Written feedback to s...

Resident perceptions of the educational value of night float rotations.

Background: Night float rotations are being increasingly used in the era of resident physician work-hour regulations, but their impact on resident education is not clear. Purpose: Our objective was to clarify resident perceptions of the educational aspects of night float rotations. Methods: An anonymous survey of internal medicine residents at a university-based residency program was completed. Results: Responses were received from 116 of 163 surveyed residents (71%). Residents attended less residents’ report (0.10 ± .43 vs. 2.70 + 0.93 sessions/week, p< .001) and fewer grand rounds sessions (0.14 ± 0.25 vs. 0.43 ± 0.28 sessions/week, p< .001) and spent less time reading, (2.63 ± 2.0 vs. 3.33 ± 1.6 hr/week, p< .001) interacting with attending physicians (0.57 ± 1.1 vs. 2.97 ± 1.5 hr/week, p< .001) and sleeping at home (6.3 ± 1.2 vs. 7.10 ± 0.9 hr/day, p< .001) on night float rotations than on non–night float rotations. Residents had strongly negative opinions about the educational value of night float, sleep cycle adjustment issues, and impact on their personal lives, which correlated with resident evaluations from the regular program evaluation process. In free responses, residents commented that they liked the autonomy and opportunity to improve triage skills on these rotations and confirmed their negative opinions about the sleep–wake cycle and interference with personal lives. Conclusions: Internal medicine residents at a university-based program have negative opinions regarding the educational value of night float rotations. Further work is necessary to determine whether problems exist across programs and specialties.

A Broader Theoretical Model for Feedback in Ambulatory Care.

Ask faculty if they provide feedback and they will likely reply “Sure, its important, I do it all the time”. Ask medical students if they receive feedback and they may say, “We hardly ever get it”. Ask most residents if they receive feedback and you get “Rarely, but its not that helpful anyway”. How is it that these perceptions are so strikingly different? Can they be talking about the same thing? If we wish to improve the educational value of feedback, we must understand these differences. One useful clue is provided by the adage “do as I say, not as I do”. This saying suggests that there are two types of feedback: that which we can measure against a standard and describe (traditional feedback), and that which comes from being immersed in real situations (situated feedback). This dichotomy is a useful construct from which to understand feedback.

A multisite, multistakeholder validation of the accreditation council for graduate medical education competencies

Purpose The Accreditation Council for Graduate Medical Education’s (ACGME’s) six-competency framework has not been validated across multiple stakeholders and sites. The objective of this study was to perform a multisite validation with five stakeholder groups. Method This was a cross-sectional, observational study carried out from October to December, 2011, in the internal medicine residency continuity clinics of eight internal medicine residency programs in the Pacific Northwest, including a VA, two academic medical centers, a military medical center, and four private hospitals. The authors performed a cultural consensus analysis (CCA) and a convergent-discriminant analysis using previously developed statements based on internal medicine milestones related to the six competencies. Ten participants were included from each of five stakeholder groups: patients, nurses, residents, faculty members, and administrators from each training site (total: 400 participants). Results Moderate to high agreement and coherence for all groups were observed (CCA eigenvalue ratios ranging from 2.16 to 3.20); however, high differences in ranking order were seen between groups in four of the CCA statements, which may suggest between-group tension in these areas. Analyses revealed excellent construct validity (Zcontrast score of 5.323, P < .0001) for the six-competency framework. Average Spearman correlation between same-node statements was 0.012, and between different-node statements it was –0.096. Conclusions The ACGME’s six-competency framework has reasonable face and construct validity across multiple stakeholders and sites. Stakeholders appear to share a single mental model of competence in this learning environment. Data patterns suggest possible improvements to the competency-milestone framework.

Developing a Cultural Consensus Analysis Based on the Internal Medicine Milestones (M-CCA)

A national task force identified domains and developmental milestones from the national competencies for resident training. Cultural Consensus Analysis (CCA) is a standard anthropological technique that can identify value conflicts. We created a CCA based on the internal medicine milestones (M-CCA) in 3 steps: converted the 38 domains into active statements; reduced the total number to 12 by summarizing and combining; and simplified the wording. This M-CCA needs further validation, after which it may be useful for assessing the 6-competency model.

Toward an ecological perspective of resident teaching clinic.

Teaching clinic managers struggle to convert performance data into meaningful behavioral change in their trainees, and quality improvement measures in medicine have had modest results. This may be due to several factors including clinical performance being based more on team function than individual action, models of best practice that are over-simplified for real patients with multiple chronic diseases, and local features that influence behavior but are not aligned with core values. Many are looking for a new conceptual structure to guide them. In this paper we briefly review several theories of action from the social and complexity sciences, and synthesize these into a coherent ‘ecological perspective’. This perspective focuses on stabilizing features and narrative, which select for behaviors in clinic much like organisms are selected for in an ecosystem. We have found this perspective to be a useful guide for design, measurement, and joint learning in the teaching clinic.

Changes in Idaho primary care physician clinical work hours, 1996-2009.

increase in related costs, calling attention to the increasingly important role these hospitals play in the US health care system. Drs Votto and Hotes and Dr Muldoon correctly state that our study cannot be used to infer whether care in a longterm acute care hospital is beneficial or harmful compared with an alternative site of care. As we stated in our discussion, there are plausible reasons why long-term acute care hospitals might either improve or worsen outcomes after critical illness. The studies quoted by Votto and Hotes, as well as others, provide important preliminary evidence but do not offer definitive conclusions. Rigorous comparative effectiveness research is needed to determine not only which patients may benefit from the services long-term acute care hospitals provide, but also the optimal site of care for these services. We disagree with the letter writers that MedPAR files lack sufficient “clinical nuances” or are “necessarily dated” and thus are unable to contribute to the policy debate. Despite their well-known limitations, for decades administrative data such as MedPAR have been an essential resource for important questions concerning US health policy. By suggesting an implausible evidentiary standard, the writers create a world in which the structures and processes that comprise the health care system cannot be critically evaluated. Such a scenario is neither practical nor tenable. For matters so consequential to patient welfare and public finances as the care of critically ill individuals, real-world effectiveness research using the best available data are urgently needed.

Interprofessional transformation of clinical education: The first six years of the Veterans Affairs Centers of Excellence in Primary Care Education

This paper describes the Centers of Excellence in Primary Care Education (CoEPCE), a seven-site collaborative project funded by the Office of Academic Affiliations (OAA) within the Veterans Health Administration of the United States Department of Veterans Affairs (VA). The CoEPCE was established to fulfill OAAs vision of large-scale transformation of the clinical learning environment within VA primary care settings. This was accomplished by funding new Centers within VA facilities to develop models of interprofessional education (IPE) to teach health professions trainees to deliver high quality interprofessional team-based primary care to Veterans. Using reports and data collected and maintained by the National Coordinating Center over the first six years of the project, we describe program inputs, the multicomponent intervention, activities undertaken to develop the intervention, and short-term outcomes. The findings have implications for lessons learned that can be considered by others seeking large-scale transformation of education within the clinical workplace and the development of interprofessional clinical learning environments. Within the VA, the CoEPCE has laid the foundation for IPE and collaborative practice, but much work remains to disseminate this work throughout the national VA system.