Joanne Cuny

Evaluation of the predictive value of ICD-9-CM coded administrative data for venous thromboembolism in the United States

OBJECTIVE To determine the positive predictive value of International Classification of Disease, 9th Revision, Clinical Modification (ICD-9-CM) discharge codes for acute deep vein thrombosis or pulmonary embolism. MATERIALS AND METHODS Retrospective review of 3456 cases hospitalized between 2005 and 2007 that had a discharge code for venous thromboembolism, using 3 sample populations: a single academic hospital, 33 University HealthSystem Consortium hospitals, and 35 community hospitals in a national Joint Commission study. Analysis was stratified by position of the code in the principal versus a secondary position. RESULTS Among 1096 cases that had a thromboembolism code in the principal position the positive predictive value for any acute venous thrombosis was 95% (95%CI:93-97), whereas among 2360 cases that had a thromboembolism code in a secondary position the predictive value was lower, 75% (95%CI:71-80). The corresponding positive predictive values for lower extremity deep-vein thrombosis or pulmonary embolism were 91% (95%CI:86-95) and 50% (95%CI:41-58), respectively. More highly defined codes had higher predictive value. Among codes in a secondary position that were false positive, 22% (95%CI:16-27) had chronic/prior venous thrombosis, 15% (95%CI:10-19) had an upper extremity thrombosis, 6% (95%CI:4-8) had a superficial vein thrombosis, and 7% (95%CI:4-13) had no mention of any thrombosis. CONCLUSIONS ICD-9-CM codes for venous thromboembolism had high predictive value when present in the principal position, and lower predictive value when in a secondary position. New thromboembolism codes that were added in 2009 that specify chronic thrombosis, upper extremity thrombosis and superficial venous thrombosis should reduce the frequency of false-positive thromboembolism codes.

Failure to rescue: validation of an algorithm using administrative data.

Background:Failure to rescue (FTR), the rate of death in patients suffering 1 of 6 in-hospital complications, is an Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicator calculated from administrative data. Objective:We sought to assess the accuracy of the AHRQ FTR algorithm. Methods:We undertook a retrospective chart review of 60 denominator cases of FTR identified by the algorithm at each of 40 University HealthSystem Consortium institutions. The primary outcome was the overall accuracy of the algorithm compared with chart review. We also assessed accuracy by complication type, patient characteristics, institution, service assignment, and mortality. Results:Of 2354 cases, 1193 (50.7%) were accurately identified by the algorithm as having had at least one of the FTR-qualifying complications during hospitalization. Of the 3073 complications identified in these patients, 1497 (48.7%) were correctly flagged by the algorithm, 907 (29.5%) were present on admission, 419 (13.6%) were not confirmed by chart review, and 250 (8.1%) met a predefined complication-specific criterion for exclusion. The case accuracy rate varied significantly by institution (mean, 50.7%; range, 18.3–100%; P < 0.001), service assignment (surgical service, 62.9% vs. nonsurgical service, 42.9%; P < 0.001), and mortality (alive, 43.9% vs. dead, 67.5%; P < 0.001) but was not affected by patients’ age, gender, race, or insurance status. Conclusions:As currently calculated from administrative data, the FTR algorithm misidentifies half of the cases on average, is least accurate for nonsurgical cases, and is widely variable across institutions. This indicator may be useful internally to flag possible cases of quality failure but has limitations for external institutional comparisons. Improvements in coding quality and consistency across institutions are needed.

Detection of postoperative respiratory failure: How predictive is the agency for healthcare research and quality's patient safety indicator?

BACKGROUND Patient Safety Indicator (PSI) 11, or postoperative respiratory failure, was developed by the US Agency for Healthcare Research and Quality to detect incident cases of respiratory failure after elective operations through use of ICD-9-CM diagnosis and procedure codes. We sought to determine the positive predictive value (PPV) of this indicator. STUDY DESIGN We conducted a retrospective cross-sectional study, sampling consecutive cases that met PSI 11 criteria from 18 geographically diverse academic medical centers on or before June 30, 2007. Trained abstractors from each center reviewed medical records using a standard instrument. We assessed the PPV of the indicator (with 95% CI adjusted for clustering within centers) and conducted descriptive analyses of the cases. RESULTS Of 609 cases that met PSI 11 criteria, 551 (90.5%; 95% CI, 86.5-94.4%) satisfied the technical criteria of the indicator and 507 (83.2%; 95% CI, 77.2-89.3%) represented true cases of postoperative respiratory failure from a clinical standpoint. The most frequent reasons for being falsely positive were nonelective hospitalization, prolonged intubation for airway protection, and insufficient evidence to support a diagnosis of acute respiratory failure. Fifty percent of true-positive cases involved substantial baseline comorbidities, and 23% resulted in death. CONCLUSIONS Although PSI 11 predicts true postoperative respiratory failure with relatively high frequency, the indicator does not limit detection to preventable cases. The PPV of PSI 11 might be increased by excluding cases with a principal diagnosis suggestive of a nonelective hospitalization and those with head or neck procedures. Removing the diagnosis code criterion from the indicator might also increase PPV, but would decrease the number of true positive cases detected by 20%.

Using the Agency for Healthcare Research and Quality patient safety indicators for targeting nursing quality improvement.

Quantifying the critical impact nurses have on the prevention and early recognition of potential complications and adverse events, such as those identified by the Agency for Healthcare Research and Quality (AHRQ) patient safety indicators (PSI), is becoming increasingly important. In this paper, we describe how the AHRQ PSI may be used to identify nursing-specific opportunities to improve care based on data from the national AHRQ PSI validation pilot project.

Designing an Abstraction Instrument: Lessons from Efforts to Validate the AHRQ Patient Safety Indicators

BACKGROUND The U.S. Agency for Healthcare Research and Quality (AHRQ) and other organizations have developed quality indicators based on hospital administrative data. Characteristics of effective abstraction instruments were identified for determining both the positive predictive value (PPV) of Patient Safety Indicators (PSIs) and the extent to which hospitals and clinicians could have prevented adverse events. METHODS Through an iterative process involving nurse abstractors, physicians, and nurses with quality improvement experience, and health services researchers, 25 abstraction instruments were designed for 12 AHRQ provider-level morbidity PSIs. Data were analyzed from 13 of these instruments, and data are being collected using several more. FINDINGS Common problems in designing the instruments included avoiding uninformative questions and premature termination of the abstraction process, anticipating misinterpretation of questions, allowing an appropriate range of response options; using clear terminology, optimizing the flow of the abstraction process, balancing the utility of data against abstractor burden, and recognizing the needs of end users, such as hospitals and quality improvement professionals and researchers, for the abstracted information. CONCLUSIONS Designing medical record abstraction instruments for quality improvement research involves several potential pitfalls. Understanding how we addressed these challenges might help both investigators and users of outcome indicators to appreciate the strengths and limitations of outcome-based quality indicators and tools designed to validate or investigate such indicators within provider organizations.

Variation in academic medical centers' coding practices for postoperative respiratory complications: implications for the AHRQ postoperative respiratory failure Patient Safety Indicator.

Background:The Agency for Healthcare Research and Quality Patient Safety Indicator (PSI) 11 uses International Classification of Disease, 9th Clinical Modification diagnosis code 518.81 (“Acute respiratory failure”)—but not the closely related alternative, 518.5 (“Pulmonary insufficiency after trauma and surgery”)—to detect cases of postoperative respiratory failure. We sought to determine whether hospitals vary in the use of 518.81 versus 518.5 and whether such variation correlates with coder beliefs. Study Design:We conducted a cross-sectional analysis of administrative data from July 2009 through June 2010 for UHC (formerly University HealthSystem Consortium)-affiliated centers to assess the use of diagnosis codes 518.81 and 518.5 in PSI 11-eligible cases. We also surveyed coders at these centers to evaluate whether variation in the use of 518.81 versus 518.5 might be linked to coder beliefs. We asked survey respondents which diagnosis they would use for 2 ambiguous cases of postoperative pulmonary complications and how much they agreed with 6 statements about the coding process. Results:UHC-affiliated centers demonstrated wide variation in the use of 518.81 and 518.5, ranging from 0 to 26 cases and 0 to 56 cases/1000 PSI 11-eligible hospitalizations, respectively. Of 56 survey respondents, 64% chose 518.81 and 30% chose 518.5 for a clinical scenario involving postoperative respiratory failure, but these responses were not associated with actual coding of 518.81 or 518.5 at the center level. Sixty-two percent of respondents agreed that they are constrained by the words that physicians use. Their self-reported likelihood of querying physicians to clarify the diagnosis was significantly associated with coding of 518.5 at the center level. Conclusions:The extent to which diagnosis code 518.81 is used relative to 518.5 varies considerably across centers, based on local coding practice, the specific wording of physician documentation, and coder–physician communication. To standardize the coding of postoperative respiratory failure, the 518.81 and 518.5 codes have recently been revised to make the available options clearer and mutually exclusive, which may improve the capacity of PSI 11 to discriminate true differences in quality of care.

Outcomes of an Initial Set of Standardized Performance Measures for Inpatient Mental Health

BACKGROUND In January 2006, the University HealthSystem Consortium (UHC) convened a committee of experts from academic health centers to identify an initial set of important standardized performance measures for inpatient psychiatric services and to evaluate the current state of performance in these measures at eight academic health centers. METHOD The eight UHC academic medical centers completed a retrospective review of 20 inpatient psychiatric records on patients who were 18-65 years of age with a primary diagnosis of psychosis and a length of stay > or = 2 days. The performance measures, derived from practice standards and the consensus of an interdisciplinary committee of experts, focused on the processes of care, including screening, assessment, treatment, coordination, continuity, and safety. RESULTS Although there was variability in organizational performance in a number of the psychiatric measures, some organizations demonstrated high levels of performance. Performance measures indicating the greatest improvement opportunities for organizations included notification of outpatient mental health provider of the psychiatric hospitalization within two days; collaboration with the outpatient mental health provider and/or primary care physician; and scheduling a follow-up appointment within seven days of discharge. DISCUSSION This initial benchmarking project in mental health at academic health centers shows that there is a range of conformity to important processes of care in the inpatient mental health setting. The results of the notification, collaboration, and continuity measures in this study highlight national concerns regarding the lack of communication and collaboration between providers in the transition through the continuum of services. Future quality measurement projects in mental health services should integrate clinical process measures with outcome measures.

How Accurate is the AHRQ Patient Safety Indicator for Hospital‐Acquired Pressure Ulcer in a National Sample of Records?

Abstract: In 2008, we conducted a retrospective cross-sectional study to determine the test characteristics of the Agency for Healthcare Research and Quality patient safety indicator (PSI) for hospital-acquired pressure ulcer (PU). We sampled 1,995 inpatient records that met PSI 3 criteria and 4,007 records assigned to 14 DRGs with the highest empirical rates of PSI 3, which did not meet PSI 3 criteria, from 32 U.S. academic hospitals. We estimated the positive predictive value (PPV), sensitivity, and specificity of PSI 3 using both the software version contemporary to the hospitalizations (v3.1) and an approximation of the current version (v4.4). Of records that met PSI 3 version 3.1 criteria, 572 (PPV 28.3%; 95% CI 23.6–32.9%) were true positive. PU that was present on admission (POA) accounted for 76% of the false-positive records. Estimated sensitivity was 48.2% (95% CI 41.0–55.3%) and specificity 71.4% (95% CI 68.3–74.5%). Reclassifying records based on reported POA information and PU stage to approximate version 4.4 of PSI 3 improved sensitivity (78.6%; 95% CI 62.7–94.5%) and specificity (98.0; 95% CI 97.1–98.9%). In conclusion, accounting for POA information and PU staging to approximate newer versions of the PSI software (v4.3) moderately improves validity.

An exploratory project on the state of quality measures in mental health at academic health centers.

Millions of Americans suffer from mental disorders that can affect both the quality of their lives and their mortality.1 Although effective treatments exist, many Americans do not receive adequate care for their mental illnesses due to barriers that include stigma, fragmented services, cost, workforce shortages, unavailable services, and the overuse, underuse, and misuse of care.2,3 Mental illness often goes unrecognized; appropriate prevention strategies are not implemented; and treatment is inappropriate, incomplete, or fragmented.2,4 The mental health system is itself fragmented, with gaps in care for children, adults with serious mental illness, and older adults with mental illness, and there are also gaps between medical care and mental health care.2 Access to quality mental health care is poor, and there is inconsistent implementation of evidencebased treatments across systems. Variations from evidencebased practice have been shown to result in poorer patient outcomes.2,5,6 Health care organizations, professionals, administrators, consumers, and policymakers need to recognize that mental health is an essential component of overall

The Relationship Between Evidence-Based Practices and Survival in Patients Requiring Prolonged Mechanical Ventilation in Academic Medical Centers

Studies suggest variable adoption of evidencebased practice guidelines. The authors hypothesized that compliance with guidelines for patients requiring mechanical ventilation would vary among academic medical centers and that this variation might be associated with survival. A total of 1463 intensive care unit cases receiving continuousmechanical ventilation for >96 hourswere reviewed. The variation in mortality based on compliance with 6 evidence-based practiceswas determined, and the effect of each intervention was estimated using a logistic regression model. Compliance varied widely across the participating centers. Astrong associationwith survival was seen for 2 of the 6practices: sedationmanagementand glycemic control (odds ratios fordeath of 0.30and0.46, respectively, eachP< .01). Spontaneous breathing trials, deep venous thrombosis prophylaxis, semirecumbent positioning, and stress ulcer prophylaxis were not associated with survival in the model. More consistent adoption of these practices represents an opportunity for academicmedical centers and was associated with enhanced survival.