Julie Cerese

Organizational Factors Associated with High Performance in Quality and Safety in Academic Medical Centers

Purpose Leaders of academic medical centers (AMCs) are challenged to ensure consistent high performance in quality and safety across all clinical services. The authors sought to identify organizational factors associated with AMCs that stood out from their peers in a composite scoring system for quality and safety derived from patient-level data. Method A scoring method using measures of safety, mortality, clinical effectiveness, and equity of care was applied to discharge abstract data from 79 AMCs for 2003–2004. Six institutions (three top and three average performers) were selected for site visits; the performance status of the six institutions was withheld from the site visit team. Through interviews and document review, the team sought to identify factors that were associated with the performance status of the institution. Results The scoring system discriminated performance among the 79 AMCs in a clinically meaningful way. For example, the transition of a typical 500-bed hospital from average to top levels of performance could result in 150 fewer deaths per year. Abstraction of key findings from the interview notes revealed distinctive themes in the top versus average performers. Common qualities shared by top performers included a shared sense of purpose, a hands-on leadership style, accountability systems for quality and safety, a focus on results, and a culture of collaboration. Conclusions Distinctive leadership behaviors and organizational practices are associated with measurable differences in patient-level measures of quality and safety.

Failure to rescue: validation of an algorithm using administrative data.

Background:Failure to rescue (FTR), the rate of death in patients suffering 1 of 6 in-hospital complications, is an Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator calculated from administrative data. Objective:We sought to assess the accuracy of the AHRQ FTR algorithm. Methods:We undertook a retrospective chart review of 60 denominator cases of FTR identified by the algorithm at each of 40 University HealthSystem Consortium institutions. The primary outcome was the overall accuracy of the algorithm compared with chart review. We also assessed accuracy by complication type, patient characteristics, institution, service assignment, and mortality. Results:Of 2354 cases, 1193 (50.7%) were accurately identified by the algorithm as having had at least one of the FTR-qualifying complications during hospitalization. Of the 3073 complications identified in these patients, 1497 (48.7%) were correctly flagged by the algorithm, 907 (29.5%) were present on admission, 419 (13.6%) were not confirmed by chart review, and 250 (8.1%) met a predefined complication-specific criterion for exclusion. The case accuracy rate varied significantly by institution (mean, 50.7%; range, 18.3–100%; P < 0.001), service assignment (surgical service, 62.9% vs. nonsurgical service, 42.9%; P < 0.001), and mortality (alive, 43.9% vs. dead, 67.5%; P < 0.001) but was not affected by patients’ age, gender, race, or insurance status. Conclusions:As currently calculated from administrative data, the FTR algorithm misidentifies half of the cases on average, is least accurate for nonsurgical cases, and is widely variable across institutions. This indicator may be useful internally to flag possible cases of quality failure but has limitations for external institutional comparisons. Improvements in coding quality and consistency across institutions are needed.

Critical Success Factors for Performance Improvement Programs

BACKGROUND Most health care organizations struggle with the design and implementation of effective, systemwide improvement programs. In 2000, the University HealthSystem Consortium initiated a benchmarking project to identify the organizational elements that predict a successful perfermance improvement (PI) program and that are best suited to support change initiatives. METHODS Forty-one organizations completed a survey about the presence of critical components, processes used to improve performance, and organizational PI structures. Follow-up site visits were conducted at three organizations. CRITICAL SUCCESS FACTORS FOR A PI PROGRAM: Eight organizational success factors for an effective performance improvement program were identified: (1) Strong Administrative Fxecutive and Performance Improvement Leadership, (2) Active Involvement of the Board of Trustees, (3) Effective Oversight Structure, (4) Expert Performance Improvement Staff, (5) Physician Involvement and Accountability, (6) Active Staff Involvement, (7) Effective Use of Information Resources-Data Used for Decision Making, and (8) Effective Communication Strategy. DISCUSSION The approach offered is grounded in the belief that effective organizational structures and processes are prerequisites to improving health care delivery. Although some empirical support for the proposed model is provided, additional research will be required to determine the effectiveness of this approach.

Improved coding of postoperative deep vein thrombosis and pulmonary embolism in administrative data (AHRQ patient safety indicator 12) after introduction of new ICD-9-CM diagnosis codes

Background:Symptomatic venous thromboembolism is a common postoperative complication. The Agency for Healthcare Research and Quality (AHRQ) has developed a Patient Safety Indicator 12 to assist hospitals, payers, and other stakeholders to identify patients who experienced this complication. Objectives:To determine whether newly created and recently redefined ICD-9-CM codes improved the criterion validity of Patient Safety Indicator 12, based on new samples of records dated after October 2009. Research Design, Subjects, Measures:Two sources of data were used: (1) UHC retrospective case-control study of risk factors for acute symptomatic venous thromboembolism occurring within 90 days after total knee arthroplasty in teaching hospitals; (2) chart abstraction data by volunteer hospitals participating in the Validation Pilot Project of the AHRQ. Results:In the UHC sample, the positive predictive value (PPV) was 99% (125/126) and the negative predictive value was 99.4% (460/463). In the AHRQ sample, the overall PPV was 81% (126/156). Conclusions:The PPV based on both samples shows substantial improvement compared with the previously reported PPVs of 43%–48%, suggesting that changes in ICD-9-CM code architecture and better coding guidance can improve the usefulness of coded data.

The reliability of AHRQ Common Format Harm Scales in rating patient safety events.

Objectives A study was conducted to determine the reliability of Agency for Healthcare Research & Quality (AHRQ) Common Format Harm Scale versions 1.1 and 1.2 in rating patient safety events among users of the UHC Patient Safety Net, a Web-based incident reporting tool. Methods To test interrater agreement, UHC developed a survey tool consisting of patient event scenarios. In 2011, a survey evaluating Harm Scale v.1.1 was distributed to 921 quality, risk, and safety (QRS) managers at 89 organizations; in 2012, a second survey evaluating Harm Scale v.1.2 was sent to 13,280 managers at 102 organizations. Results Regardless of the version used, in 3 of 9 scenarios, fewer than 60% of respondents agreed on a single score. Interrater agreement increased for certain event scenarios with v.1.2 but decreased for other scenarios. Interrater reliability was moderate for both v.1.1 (k = 0.51) and v.1.2 (k = 0.47). Interrater agreement improved in v.1.2 when results were limited to more experienced raters but still remained in the moderate range (k = 0.58). Conclusions AHRQ Common Format Harm Scale v.1.1 and v.1.2 both had moderate interrater reliability. Using Harm Scale v.1.1, respondents had difficulty distinguishing “injury limited to additional treatment” from “temporary harm,” whereas, using Harm Scale v.1.2, respondents had difficulty distinguishing moderate harm from one of the adjacent levels—mild or severe harm. This study provides valuable data that can inform harm scale revision to improve the quality of aggregate safety data used to define and direct safety efforts.

The characteristics of patients frequently admitted to academic medical centers in the United States

BACKGROUND The recent intense attention to hospital readmissions and their implications for quality, safety, and reimbursement necessitates understanding specific subsets of readmitted patients. Frequently admitted patients, defined as patients who are admitted 5 or more times within 1 year, may have some distinguishing characteristics that require novel solutions. METHODS A comprehensive administrative database (University HealthSystem Consortiums Clinical Data Base/Resource Manager™) was analyzed to identify demographic, social, and clinical characteristics of frequently admitted patients in 101 US academic medical centers. RESULTS We studied 28,291 frequently admitted patients with 180,185 admissions over a 1‐year period (2011–2012). These patients comprise 1.6% of all patients, but account for 8% of all admissions and 7% of direct costs. Their admissions are driven by multiple chronic conditions; compared to other hospitalized patients, they have significantly more comorbidities (an average of 7.1 vs 2.5), and 84% of their admissions are to medical services. A minority, but significantly more than other patients, have comorbidities of psychosis or substance abuse. Moreover, although they are slightly more likely than other patients to be on Medicaid or to be uninsured (27.6% vs 21.6%), nearly three‐quarters have private or Medicare coverage. CONCLUSIONS Patients who are frequently admitted to US academic medical centers are likely to have multiple complex chronic conditions and may have behavioral comorbidities that mediate their health behaviors, resulting in acute episodes requiring hospitalization. This information can be used to identify solutions for preventing repeat hospitalization for this small group of patients who consume a highly disproportionate share of healthcare resources. Journal of Hospital Medicine 2015;10:563–568.

Governance Practices and Performance in US Academic Medical Centers

Recognition of the complex nature of modern health care delivery has led to interest in investigating the ways in which various factors, including governance structures and practices, influence health care quality. In this study, the chief executive officers (CEOs) of US academic medical centers were surveyed to elicit their perceptions of board structures, activities, and attitudes reflecting 6 widely identified governance best practices; the relationship between use of these practices and organizational performance, based on the University HealthSystem Consortium’s Quality & Accountability rankings, was assessed. High-performing hospitals showed greater use of all 6 practices, but the strongest evidence supported a focus on board member education and development, the rigorous use of performance measures to guide quality improvement, and systematic board self-assessment processes. All hospitals, even those with the highest quality ratings, had major gaps in their use of best practices for CEO and board assessments. These findings can serve as the basis for developing sound board improvement plans.

Outcomes of an Initial Set of Standardized Performance Measures for Inpatient Mental Health

BACKGROUND In January 2006, the University HealthSystem Consortium (UHC) convened a committee of experts from academic health centers to identify an initial set of important standardized performance measures for inpatient psychiatric services and to evaluate the current state of performance in these measures at eight academic health centers. METHOD The eight UHC academic medical centers completed a retrospective review of 20 inpatient psychiatric records on patients who were 18-65 years of age with a primary diagnosis of psychosis and a length of stay > or = 2 days. The performance measures, derived from practice standards and the consensus of an interdisciplinary committee of experts, focused on the processes of care, including screening, assessment, treatment, coordination, continuity, and safety. RESULTS Although there was variability in organizational performance in a number of the psychiatric measures, some organizations demonstrated high levels of performance. Performance measures indicating the greatest improvement opportunities for organizations included notification of outpatient mental health provider of the psychiatric hospitalization within two days; collaboration with the outpatient mental health provider and/or primary care physician; and scheduling a follow-up appointment within seven days of discharge. DISCUSSION This initial benchmarking project in mental health at academic health centers shows that there is a range of conformity to important processes of care in the inpatient mental health setting. The results of the notification, collaboration, and continuity measures in this study highlight national concerns regarding the lack of communication and collaboration between providers in the transition through the continuum of services. Future quality measurement projects in mental health services should integrate clinical process measures with outcome measures.

How Accurate is the AHRQ Patient Safety Indicator for Hospital‐Acquired Pressure Ulcer in a National Sample of Records?

Abstract: In 2008, we conducted a retrospective cross-sectional study to determine the test characteristics of the Agency for Healthcare Research and Quality patient safety indicator (PSI) for hospital-acquired pressure ulcer (PU). We sampled 1,995 inpatient records that met PSI 3 criteria and 4,007 records assigned to 14 DRGs with the highest empirical rates of PSI 3, which did not meet PSI 3 criteria, from 32 U.S. academic hospitals. We estimated the positive predictive value (PPV), sensitivity, and specificity of PSI 3 using both the software version contemporary to the hospitalizations (v3.1) and an approximation of the current version (v4.4). Of records that met PSI 3 version 3.1 criteria, 572 (PPV 28.3%; 95% CI 23.6–32.9%) were true positive. PU that was present on admission (POA) accounted for 76% of the false-positive records. Estimated sensitivity was 48.2% (95% CI 41.0–55.3%) and specificity 71.4% (95% CI 68.3–74.5%). Reclassifying records based on reported POA information and PU stage to approximate version 4.4 of PSI 3 improved sensitivity (78.6%; 95% CI 62.7–94.5%) and specificity (98.0; 95% CI 97.1–98.9%). In conclusion, accounting for POA information and PU staging to approximate newer versions of the PSI software (v4.3) moderately improves validity.

An exploratory project on the state of quality measures in mental health at academic health centers.

Millions of Americans suffer from mental disorders that can affect both the quality of their lives and their mortality.1 Although effective treatments exist, many Americans do not receive adequate care for their mental illnesses due to barriers that include stigma, fragmented services, cost, workforce shortages, unavailable services, and the overuse, underuse, and misuse of care.2,3 Mental illness often goes unrecognized; appropriate prevention strategies are not implemented; and treatment is inappropriate, incomplete, or fragmented.2,4 The mental health system is itself fragmented, with gaps in care for children, adults with serious mental illness, and older adults with mental illness, and there are also gaps between medical care and mental health care.2 Access to quality mental health care is poor, and there is inconsistent implementation of evidencebased treatments across systems. Variations from evidencebased practice have been shown to result in poorer patient outcomes.2,5,6 Health care organizations, professionals, administrators, consumers, and policymakers need to recognize that mental health is an essential component of overall