Matthew C. Scanlon

Relevance of the agency for healthcare research and quality patient safety indicators for children's hospitals

Objectives. Patient safety indicators (PSIs) were developed by the Agency for Healthcare Research and Quality. Our objectives were (1) to apply these algorithms to the National Association of Childrens Hospitals and Related Institutions (NACHRI) Aggregate Case Mix Comparative Database for 1999–2002, (2) to establish mean rates for each of the PSI events in childrens hospitals, (3) to investigate the inadequacies of PSIs in relation to pediatric diagnoses, and (4) to express the data in such a way that childrens hospitals could use the PSIs determined to be appropriate for pediatric use for comparison with their own data. In addition, we wanted to use the data to set priorities for ongoing clinical investigations and to propose interventions if the indicators demonstrated preventable errors. Methods. The Agency for Healthcare Research and Quality PSI algorithms (version 2.1, revision 1) were applied to childrens hospital administrative data (1.92 million discharges) from the NACHRI Aggregate Case Mix Comparative Database for 1999–2002. Rates were measured for the following events: complications of anesthesia, death in low-mortality diagnosis-related groups (DRGs), decubitus ulcer, failure to rescue (ie, death resulting from a complication, rather than the primary diagnosis), foreign body left in during a procedure, iatrogenic pneumothorax, infection attributable to medical care (ie, infections related to surgery or device placement), postoperative hemorrhage or hematoma, postoperative pulmonary embolism or venous thrombosis, postoperative wound dehiscence, and accidental puncture/laceration. Results. Across the 4 years of data, the mean risk-adjusted rates of PSI events ranged from 0.01% (0.1 event per 1000 discharges) for a foreign body left in during a procedure to 14.0% (140 events per 1000 discharges) for failure to rescue. Review of International Classification of Diseases, Ninth Revision, Clinical Modification codes associated with each PSI category showed that the failure to rescue and death in low-mortality DRG indicators involved very complex cases and did not predict preventable events in the majority of cases. The PSI for infection attributable to medical care appeared to be accurate the majority of the time. Incident risk-adjusted rates of infections attributable to medical care averaged 0.35% (3.5 events per 1000 discharges) and varied up to fivefold from the lowest rate to the highest rate. The highest rates were up to 1.8 times the average. Conclusions. PSIs derived from administrative data are indicators of patient safety concerns and can be relevant as screening tools for childrens hospitals; however, cases identified by these indicators do not always represent preventable events. Some, such as a foreign body left in during a procedure, iatrogenic pneumothorax, infection attributable to medical care, decubitus ulcer, and venous thrombosis, seem to be appropriate for pediatric care and may be directly amenable to system changes. Evidence-based practices regarding those particular indicators that have been reported in the adult literature need to be investigated in the pediatric population. In their present form, 2 of the indicators, namely, failure to rescue and death in low-mortality DRGs, are inaccurate for the pediatric population, do not represent preventable errors in the majority of pediatric cases, and should not be used to estimate quality of care or preventable deaths in childrens hospitals. The PSIs can assist institutions in prioritizing chart review-based investigations; if clusters of validated events emerge in reviews, then improvement activities can be initiated. Large aggregate databases, such as the NACHRI Case Mix Database, can help establish mean rates of potential pediatric events, giving childrens hospitals a context within which to examine their own data.

Near-infrared spectroscopy in neonates before palliation of hypoplastic left heart syndrome.

BACKGROUND Neonates with hypoplastic left heart syndrome have circulatory vulnerability that results in shock and high risk of mortality without intervention. High arterial saturation (SaO(2)) is often used as a proxy for inadequate systemic oxygen delivery and triggers the use of invasive therapies to restore circulatory balance. We hypothesized that preoperative use of near-infrared spectroscopy (NIRS) would reduce the need for invasive therapies, including controlled ventilation and inspired gas manipulation. METHODS A Human Research Review Board-approved retrospective review of patients who had stage 1 palliation from January 2000 to January 2006 was conducted. Preoperative patient characteristics, cardiorespiratory support, and monitored data were collected for all patients. Cerebral and somatic tissue oxyhemoglobin saturations were recorded for patients with preoperative NIRS monitoring. RESULTS The studied cohort included 92 patients, 47 without and 45 with preoperative NIRS. Patient characteristics were similar between groups. Differences were observed in preoperative respiratory support. Controlled ventilation was less common in the NIRS group (51% versus 79%, p = 0.005) as was the use of inspired nitrogen (16% versus 70%, p = 0.001). The NIRS patients had higher mean SaO(2) (92% versus 88%, p = 0.001). Age at surgery was similar between groups (5.7 +/- 3.2 versus 6.5 +/- 5.2 days, p = 0.3). Early survival was 96% in each group. CONCLUSIONS Near-infrared spectroscopy monitoring of patients with hypoplastic left heart syndrome awaiting palliation provides noninvasive assessment of oxygen delivery and simplified management, with reduced use of controlled ventilation and inspired gas. Higher SaO(2) in the NIRS group was not associated with impaired systemic oxygen delivery, and did not lead to earlier palliation or postoperative mortality.

Evaluation of the agency for healthcare research and quality pediatric quality indicators.

OBJECTIVES. Pediatric quality indicators were developed in 2006 by the Agency for Healthcare Research and Quality to identify potentially preventable complications in hospitalized children. Our objectives for this study were to (1) apply these algorithms to an aggregate childrens hospitals discharge abstract database, (2) establish rates for each of the pediatric quality indicator events in the childrens hospitals, (3) use direct chart review to investigate the accuracy of the pediatric quality indicators, (4) calculate the number of complications that were already present on admission and, therefore, not attributable to the specific hospitalization, and (5) evaluate preventability and calculate positive predictive value for each of the indicators. In addition, we wanted to use the data to set priorities for ongoing clinical investigation. METHODS. The Agency for Healthcare Research and Quality pediatric quality indicator algorithms were applied to 76 childrens hospitals discharge abstract data (1794675 discharges) from 2003 to 2005. Rates were calculated for 11 of the pediatric quality indicators from all 3 years of discharge data: accidental puncture or laceration, decubitus ulcer, foreign body left in during a procedure, iatrogenic pneumothorax in neonates at risk, iatrogenic pneumothorax in nonneonates, postoperative hemorrhage or hematoma, postoperative respiratory failure, postoperative sepsis, postoperative wound dehiscence, selected infections caused by medical care, and transfusion reaction. Subsequently, clinicians from 28 childrens hospitals reviewed 1703 charts in which complications had been identified. They answered questions as to correctness of secondary diagnoses that were associated with the indicator, whether a complication was already present on admission, and whether that complication was preventable, nonpreventable, or uncertain. RESULTS. Across 3 years of data the rates of pediatric quality indicators ranged from a low of 0.01/1000 discharges for transfusion reaction to a high of 35/1000 for postoperative respiratory failure, with a median value of 1.85/1000 for the 11 pediatric quality indicators. Indicators were often already present on admission and ranged from 43% for infection caused by medical care to 0% for iatrogenic pneumothorax in neonates, with a median value of 16.9%. Positive predictive value for the subset of pediatric quality indicators occurring after admission was highest for decubitus ulcer (51%) and infection caused by medical care (40%). Because of the very large numbers of cases identified and its low preventability, the indicator postoperative respiratory failure is particularly problematic. The initial definition includes all children on ventilators postoperatively for >4 days with few exclusions. Being on a ventilator for 4 days would be a normal occurrence for many children with extensive surgery; therefore, the majority of the time does not indicate a complication and makes the indicator inappropriate. CONCLUSIONS. A subset of pediatric quality indicators derived from administrative data are reasonable screening tools to help hospitals prioritize chart review and subsequent improvement projects. However, in their present form, true preventability of these complications is relatively low; therefore, the indicators are not useful for public hospital comparison. Identifying which complications are present on admission versus those that occur within the hospitalization will be essential, along with adequate risk adjustment, for any valid comparison between institutions. Infection caused by medical care and decubitus ulcers are clinically important indicators once the present-on-admission status is determined. These complications cause significant morbidity in hospitalized children, and research has shown a high level of preventability. The pediatric quality indicator software can help childrens hospitals objectively review their cases and target improvement activities appropriately. The postoperative-respiratory-failure indicator does not represent a complication in the majority of cases and, therefore, should not be included for hospital screening or public comparison. Chart review should become part of the development process for quality indicators to avoid inappropriate conclusions that misdirect quality-improvement resources.

Automation and adaptation: nurses' problem-solving behavior following the implementation of bar-coded medication administration technology

The most common change facing nurses today is new technology, particularly bar-coded medication administration technology (BCMA). However, there is a dearth of knowledge on how BCMA alters nursing work. This study investigated how BCMA technology affected nursing work, particularly nurses’ operational problem-solving behavior. Cognitive systems engineering observations and interviews were conducted after the implementation of BCMA in three nursing units of a freestanding pediatric hospital. Problem-solving behavior, associated problems, and goals were specifically defined and extracted from observed episodes of care. Three broad themes regarding BCMA’s impact on problem solving were identified. First, BCMA allowed nurses to invent new problem-solving behavior to deal with pre-existing problems. Second, BCMA made it difficult or impossible to apply some problem-solving behaviors that were commonly used pre-BCMA, often requiring nurses to use potentially risky workarounds to achieve their goals. Third, BCMA created new problems that nurses were either able to solve using familiar or novel problem-solving behaviors, or unable to solve effectively. Results from this study shed light on hidden hazards and suggest three critical design needs: (1) ecologically valid design; (2) anticipatory control; and (3) basic usability. Principled studies of the actual nature of clinicians’ work, including problem solving, are necessary to uncover hidden hazards and to inform health information technology design and redesign.

Effects of mental demands during dispensing on perceived medication safety and employee well being: A study of workload in pediatric hospital pharmacies

BACKGROUND Pharmacy workload is a modifiable work system factor believed to affect both medication safety outcomes and employee outcomes, such as job satisfaction. OBJECTIVES This study sought to measure the effect of workload on safety and employee outcomes in 2 pediatric hospitals and to do so using a novel approach to pharmacy workload measurement. METHODS Rather than measuring prescription volume or other similar indicators, this study measured the type and intensity of mental demands experienced during the medication dispensing tasks. The effects of external (interruptions, divided attention, and rushing) and internal (concentration and effort) task demands on perceived medication error likelihood, adverse drug event likelihood, job dissatisfaction, and burnout were statistically estimated using multiple linear and logistic regression. RESULTS Pharmacists and pharmacy technicians reported high levels of external and internal mental demands during dispensing. The study supported the hypothesis that external demands (interruptions, divided attention, and rushing) negatively impacted medication safety and employee well-being outcomes. However, as hypothesized, increasing levels of internal demands (concentration and effort) were not associated with greater perceived likelihood of error, adverse drug events, or burnout and even had a positive effect on job satisfaction. CONCLUSIONS Replicating a prior study in nursing, this study shows that new conceptualizations and measures of workload can generate important new findings about both detrimental and beneficial effects of workload on patient safety and employee well-being. This study discusses what those findings imply for policy, management, and design concerning automation, cognition, and staffing.

Performance of the Pediatric Index of Mortality 2 for pediatric cardiac surgery patients

Objective: To evaluate the performance of the Pediatric Index of Mortality 2 (PIM-2) for pediatric cardiac surgery patients admitted to the pediatric intensive care unit (PICU). Design: Retrospective cohort analysis. Setting: Multi-institutional PICUs. Patients: Children whose PICU admission had an associated cardiac surgical procedure. Interventions: None. Measurements and Main Results: Performance of the PIM-2 was evaluated with both discrimination and calibration measures. Discrimination was assessed with a receiver operating characteristic curve and associated area under the curve measurement. Calibration was measured across defined groups based on mortality risk, using the Hosmer-Lemeshow goodness-of-fit test. Analyses were performed initially, using the entire cohort, and then based on operative status (perioperative defined as procedure occurring within 24 hrs of PICU admission and preoperative as occurring >24 hrs from the time of PICU admission). A total of 9,208 patients were identified as cardiac surgery patients with 8,391 (91%) considered as perioperative. Average age of the entire cohort was 3.3 yrs (median, 10 mos, 0–18 yrs), although preoperative children tended to be younger (median, <1 month). Preoperative patients also had longer PICU median lengths of stay than perioperative patients (12 days [1–375 days] vs. 3 days [1–369 days], respectively). For the entire cohort, the PIM-2 had fair discrimination power (area under the curve, 0.80; 95% confidence interval, 0.77–0.83) and poor calibration (p < .0001). Its predictive ability was similarly inadequate for quality assessment (standardized mortality ratio, 0.81; 95% confidence interval, 0.72–0.90) with significant overprediction in the highest-decile risk group. For the subpopulations, the model continued to perform poorly with low area under the curves for preoperative patients and poor calibration for both groups. PIM-2 tended to overpredict mortality for perioperative patients and underpredict for preoperative patients (standardized mortality ratios, 0.69 [95% confidence interval, 0.59–0.78] and 1.48 [95% confidence interval, 1.27–1.70], respectively). Conclusions: The PIM-2 demonstrated poor performance with fair discrimination, poor calibration, and predictive ability for pediatric cardiac surgery population and thus cannot be recommended in its current form as an adequate adjustment tool for quality measurement in this patient group.

Value of human factors to medication and patient safety in the intensive care unit.

Conventional wisdom suggests that the “human factor” in critical care environments is reason for inadequate medication and patient safety. “Human factors” (or human factors engineering) is also a scientific discipline and practice of improving human performance. Using decades of human factors research, this paper evaluates a range of common beliefs about patient safety through a human factors lens. This evaluation demonstrates that human factors provides a framework for understanding safety failures in critical care settings, offers insights into how to improve medication and patient safety, and reminds us that the “human factor” in critical care units is what allows these time-pressured, information-intense, mentally challenging, interruption-laden, and life-or-death environments to function so safely so much of the time.

Alterations in End-of-Life Support in the Pediatric Intensive Care Unit

OBJECTIVE: Our purpose was to examine alterations in end-of-life support in a multiinstitutional sample of PICUs. METHODS: This was a retrospective, descriptive study. Variables collected included end-of-life support category, race, length of stay, operative status, reason for admission, and Pediatric Index of Mortality 2 score, as well as the number of ICU beds and the presence of trainees. RESULTS: There were 1745 deaths at 35 institutions between January 1, 2004, and September 30, 2005. Of those, 1263 had complete data and were analyzed. The end-of-life support category distribution was as follows: brain death, 296 (23%); do not resuscitate, 205 (16%); limitation of support, 36 (3%); withdrawal of support, 579 (46%); no limitation, 124 (10%); no advance directives, 23 (2%). For further analyses, end-of-life support categories were grouped as limitation (ie, do not resuscitate, limitation of support, or withdrawal of support) versus no limitation (ie, no limitation or no advance directive). Brain death was not included in further analyses. The majority of deaths were in the limitation group (n = 820 [85%]), and 12 (40%) of 30 institutions had 100% of deaths in this group. There were significant differences between institutions (P < .001). Decisions for limitation were seen less frequently in the black race (112 [76%] of 147 deaths; P = .037) and in institutions with no trainees (56 [69%] of 81 deaths; P < .001). CONCLUSIONS: Decisions to limit support are common. Black race and an absence of trainees are associated with decreased frequency of limitation decisions.

Determining pediatric intensive care unit quality indicators for measuring pediatric intensive care unit safety

Introduction: The measurement of quality and patient safety continues to gain increasing importance, as these measures are used for both healthcare improvement and accountability. Pediatric care, particularly that provided in pediatric intensive care units, is sufficiently different from adult care that specific metrics are required. Body: Pediatric critical care requires specific measures for both quality and safety. Factors that may affect measures are identified, including data sources, risk adjustment, intended use, reliability, validity, and the usability of measures. The 18-month process to develop seven pediatric critical care measures proposed for national use is described. Specific patient safety metrics that can be applied to pediatric intensive care units include error-, injury-, and risk-based approaches. Conclusion: Measurement of pediatric critical care quality and safety will likely continue to evolve. Opportunities exist for intensivists to contribute and lead in the development and refinement of measures.

Discrepancies between medication orders and infusion pump programming in a paediatric intensive care unit

Background Errors and the incorrect use of medications are significant sources of risk and harm to children in US hospitals. The risk associated with medication infusions has led to recommendations for the adoption of technologies including computer order physician entry (CPOE) and ‘smart’ infusion pumps despite a paucity of evidence demonstrating the ability of these technologies to reduce harm to paediatric inpatients. Objective To measure discrepancies between medication orders for infusions entered into a CPOE system and the medication being infused as measured by the programmed settings of the smart infusion pump within a paediatric intensive care unit. Methods This study used a prospective, observational design in a 30-bed paediatric intensive care unit. Data were simultaneously collected from the medication orders in the CPOE system and the bedside smart infusion pumps by trained observers. Analysis consisted of a line-by-line comparison of order observation data with the pump observation data. Conclusions Of 296 observations of medication infusions and 231 observations of intravenous fluid infusions, the frequency of discrepancies between orders entered and pumps programming ranged from 24.3% for observed medications to 42.4% for observed fluids. Anti-infectives (100%), concentrated electrolytes (46.7%) and anticoagulants (46.2%) were associated with greatest discrepancy between orders and programmed doses.