Peter E. Rivard

Using patient safety indicators to estimate the impact of potential adverse events on outcomes.

The authors estimated the impact of potentially preventable patient safety events, identi- fied by Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSIs), on patient outcomes: mortality, length of stay (LOS), and cost. The PSIs were applied to all acute inpatient hospitalizations at Veterans Health Administration (VA) facil- ities in fiscal 2001. Two methods—regression analysis and multivariable case matching— were used independently to control for patient and facility characteristics while predicting the effect of the PSI on each outcome. The authors found statistically significant (p < .0001) excess mortality, LOS, and cost in all groups with PSIs. The magnitude of the excess varied considerably across the PSIs. These VA findings are similar to those from a previously published study of nonfederal hospitals, despite differences between VA and non-VA systems. This study contributes to the literature measuring outcomes of medical errors and provides evidence that AHRQ PSIs may be useful indicators for comparison across delivery systems.

Tracking rates of patient safety indicators over time: Lessons from the veterans administration

Background:The Patient Safety Indicators (PSIs), developed by the Agency for Healthcare Research and Quality, are useful screening tools for highlighting areas in which quality should be further investigated and providing useful benchmarks for tracking progress. Objectives:Our objectives were to: 1) provide a descriptive analysis of the incidence of PSI events from 2001 to 2004 in the Veterans Health Administration (VA); 2) examine trends in national PSI rates at the hospital discharge level over time; and 3) assess whether hospital characteristics (eg, teaching status, number of beds, and degree of quality improvement implementation) and baseline safety-related hospital performance predict future hospital safety-related performance. Methods:We examined changes in risk-adjusted PSI rates at the discharge level, calculated the correlation between hospitals’ risk-adjusted PSI rates in 2001 with subsequent years, and developed generalized linear models to examine predictors of hospitals’ 2004 risk-adjusted PSI rates. Results:Risk-adjusted rates of 2 of the 15 PSIs demonstrated significant trends over time. Rates of iatrogenic pneumothorax increased over time, whereas rates of failure to rescue decreased. Most PSIs demonstrated consistent rates over time. After accounting for patient and hospital characteristics, hospitals’ baseline risk-adjusted PSI rates were the most important predictors of their 2004 risk-adjusted rates for 8 PSIs. Conclusions:The PSIs are useful tools for tracking and monitoring patient safety events in the VA. Future research should investigate whether trends reflect better or worse care or increased attention to documenting patient safety events.

Testing the Association Between Patient Safety Indicators and Hospital Structural Characteristics in VA and Nonfederal Hospitals

This study tested the association between hospital structural characteristics—teaching status, bedsize, and nurse staffing—and potentially preventable adverse events. The authors calculated 14 Agency for Healthcare Research and Quality Patient Safety Indicators (PSIs) and a PSI composite, using discharge databases from VA and nonfederal hospitals. This study compared the likelihood of PSI events in hospitals, controlling for structural and other characteristics, including patients’ case-mix. Additional controls were employed to account for differences in VA versus nonfederal patients and data. The study found some associations, most notably a positive (unfavorable) association between status as a major teaching hospital and six PSIs. However, for most PSIs, the authors found no association between the structural characteristics tested and likelihood of PSI events. The study’s findings extend previous research showing a lack of consistent relationship between structural characteristics and patient safety. However, the results also suggest continued need for examination of the relationship between teaching status and potentially preventable adverse events.

Development of trigger tools for surveillance of adverse events in ambulatory surgery

Background The trigger tool methodology uses clinical algorithms applied electronically to ‘flag’ medical records where adverse events (AEs) have most likely occurred. The authors sought to create surgical triggers to detect AEs in the ambulatory care setting. Methods Four consecutive steps were used to develop ambulatory surgery triggers. First, the authors conducted a comprehensive literature review for surgical triggers. Second, a series of multidisciplinary focus groups (physicians, nurses, pharmacists and information technology specialists) provided user input on trigger selection. Third, a clinical advisory panel designed an initial set of 10 triggers. Finally, a three-phase Delphi process (surgical and trigger tool experts) evaluated and rated the suggested triggers. Results The authors designed an initial set of 10 surgical triggers including five global triggers (flagging medical records for the suspicion of any AE) and five AE-specific triggers (flagging medical records for the suspicion of specific AEs). Based on the Delphi rating of the triggers utility for system-level interventions, the final triggers were: (1) emergency room visit(s) within 21 days from surgery; (2) unscheduled readmission within 30 days from surgery; (3) unscheduled procedure (interventional radiological, urological, dental, cardiac or gastroenterological) or reoperation within 30 days from surgery; (4) unplanned initial hospital length of stay more than 24 h; and (5) lower-extremity Doppler ultrasound order entry and ICD code for deep vein thrombosis or pulmonary embolus within 30 days from surgery. Conclusion The authors therefore propose a systematic methodology to develop trigger tools that takes into consideration previously published work, end-user preferences and expert opinion.

Examining the validity of AHRQ's patient safety indicators (PSIs): is variation in PSI composite score related to hospital organizational factors?

Increasing use of Agency for Healthcare Research and Quality’s Patient Safety Indicators (PSIs) for hospital performance measurement intensifies the need to critically assess their validity. Our study examined the extent to which variation in PSI composite score is related to differences in hospital organizational structures or processes (i.e., criterion validity). In site visits to three Veterans Health Administration hospitals with high and three with low PSI composite scores (“low performers” and “high performers,” respectively), we interviewed a cross-section of hospital staff. We then coded interview transcripts for evidence in 13 safety-related domains and assessed variation across high and low performers. Evidence of leadership and coordination of work/communication (organizational process domains) was predominantly favorable for high performers only. Evidence in the other domains was either mixed, or there were insufficient data to rate the domains. While we found some evidence of criterion validity, the extent to which variation in PSI rates is related to differences in hospitals’ organizational structures/processes needs further study.

Quality improvement for patient safety: project-level versus program-level learning.

Background: Improving quality and patient safety is of increasing strategic importance to health care organizations. However, simply increasing the volume of quality improvement (QI) activity does not necessarily improve patient outcomes. There is a need for greater understanding of QI success factors. Purpose: This study looked for differences in QI implementation across hospitals with a range of performance on Patient Safety Indicators. Methods: We conducted an exploratory comparative case study of 4 Veterans Health Administration hospitals including site visits and interviews with leaders and staff. Findings: Two themes emerged. Project-level QI learning is assessing and modifying specific QI projects relative to expectations. Program-level QI learning is assessing and modifying the overall QI endeavor. The nature of project-level QI learning was similar across sites, whereas we identified qualitative differences across organizations in program-level QI learning. The highest performing organization was evaluating and refining its overall approach to QI, whereas the others were learning how to build and control QI programs. Practice Implications: Program-level QI learning may be key if a QI program is to succeed in improving patient outcomes. This type of organizational learning entails a big-picture, organization-wide view of QI. It also entails second-order organizational learning based on assessment not only of whether QI is being done correctly but also whether the right QI activities are being done, for the right reasons. The organization is “learning to learn.” In addition to gaining mastery and control of QI, leaders regularly engage with staff in rethinking QI and experimenting with new approaches. Leaders also assess how QI activity fits in the organization’s developmental journey and how it supports realization of strategy.

Partnering With VA Stakeholders to Develop a Comprehensive Patient Safety Data Display Lessons Learned From the Field

Health care systems are increasingly burdened by the large numbers of safety measures currently being reported. Within the Veterans Administration (VA), most safety reporting occurs within organizational silos, with little involvement by the frontline users of these measures. To provide a more integrated picture of patient safety, the study team partnered with multiple VA stakeholders and engaged potential frontline users at 2 hospitals to develop a Guiding Patient Safety (GPS) tool. The GPS is currently in its fourth generation; once approval is obtained from senior leadership, implementation will begin. Stakeholders were enthusiastic about the GPS’s user-friendly format, comprehensive content, and potential utility for improving safety. These findings suggest that stakeholder engagement is a critical first step in the development of tools that will more likely be used by frontline users. Policy makers and researchers may consider adopting this innovative partnered-research model in developing future national initiatives to deliver meaningful programs to frontline users.

Assessing the potential adoption and usefulness of concurrent, action-oriented, electronic adverse drug event triggers designed for the outpatient setting.

Background: Adverse drug event (ADE) detection is an important priority for patient safety research. Trigger tools have been developed to help identify ADEs. In previous work we developed seven concurrent, action-oriented, electronic trigger algorithms designed to prompt clinicians to address ADEs in outpatient care. Objectives: We assessed the potential adoption and usefulness of the seven triggers by testing the positive predictive validity and obtaining stakeholder input. Methods: We adapted ADE triggers, “bone marrow toxin—white blood cell count (BMT-WBC),” “bone marrow toxin - platelet (BMT-platelet),” “potassium raisers,” “potassium reducers,” “creatinine,” “warfarin,” and “sedative hypnotics,” with logic to suppress flagging events with evidence of clinical intervention and applied the triggers to 50,145 patients from three large health care systems. Four pharmacists assessed trigger positive predictive value (PPV) with respect to ADE detection (conservatively excluding ADEs occurring during clinically appropriate care) and clinical usefulness (i.e., whether the trigger alert could change care to prevent harm). We measured agreement between raters using the free kappa and assessed positive PPV for the trigger’s detection of harm, clinical usefulness, and both. Stakeholders from the participating health care systems rated the likelihood of trigger adoption and the perceived ease of implementation. Findings: Agreement between pharmacist raters was moderately high for each ADE trigger (kappa free > 0.60). Trigger PPVs for harm ranged from 0 (Creatinine, BMT-WBC) to 17 percent (potassium raisers), while PPV for care change ranged from 0 (WBC) to 60 percent (Creatinine). Fifteen stakeholders rated the triggers. Our assessment identified five of the seven triggers as good candidates for implementation: Creatinine, BMT-Platelet, Potassium Raisers, Potassium Reducers, and Warfarin. Conclusions: At least five outpatient ADE triggers performed well and merit further evaluation in outpatient clinical care. When used in real time, these triggers may promote care changes to ameliorate patient harm.

A study of Cook Islands nurses' attitudes towards electronic medical records technology

The Polynesian nurse prided herself on her ability to describe her patients pain in a one‐page hand written note to her doctor. Now, the electronic medical records system was asking her to choose 1 to 10 on a drop down menu scale. This paper analyzed through a series of interviews what it is like for nurses in the remote Cook Islands to use Western‐designed medical information systems. Their experiences were examined through the lens of social construction of learning and cultural diffusion. Analysis of qualitative interview data suggested cultural as well as infrastructural barriers to engaged learning. Future researchers are advised to consider Facilitating Factors and Cultural Embeddedness constructs when considering technology adoption attitudes of medical practitioners in remote archipelago.

Caveats for high reliability in healthcare

Purpose Given the pace of industry change and the rapid diffusion of high reliability organization (HRO) approaches, lags and divergences have arisen between research and practice in healthcare. The purpose of this paper is to explore several of these theory-practice gaps and propose implications for research and practice. Design/methodology/approach Classic and cutting-edge HRO literature is applied to analyze two industry trends: delivery system integration, and the confluence of patient-as-consumer and patient-centered care. Findings Highly reliable integrated delivery systems will likely function very differently from classic HRO organizations. Both practitioners and researchers should address conditions such as how a system is bounded, how reliable the system should be and how interdependencies are handled. Additionally, systems should evaluate the added uncertainty and variability introduced by enhanced agency on the part of patients/families in decision making and in processes of care. Research limitations/implications Dramatic changes in the sociotechnical environment are influencing the coupling and interactivity of system elements in healthcare. Researchers must address the maintenance of reliability across organizations and the migration of decision-making power toward patients and families. Practical implications As healthcare systems integrate, managers attempting to apply HRO principles must recognize how these systems present new and different reliability-related challenges and opportunities. Originality/value This paper provides a starting point for the advancement of research and practice in high-reliability healthcare by providing an in-depth exploration of the implications of two major industry trends.