Julie M. Fiskio

Improving Acceptance of Computerized Prescribing Alerts in Ambulatory Care

Computerized drug prescribing alerts can improve patient safety, but are often overridden because of poor specificity and alert overload. Our objective was to improve clinician acceptance of drug alerts by designing a selective set of drug alerts for the ambulatory care setting and minimizing workflow disruptions by designating only critical to high-severity alerts to be interruptive to clinician workflow. The alerts were presented to clinicians using computerized prescribing within an electronic medical record in 31 Boston-area practices. There were 18,115 drug alerts generated during our six-month study period. Of these, 12,933 (71%) were noninterruptive and 5,182 (29%) interruptive. Of the 5,182 interruptive alerts, 67% were accepted. Reasons for overrides varied for each drug alert category and provided potentially useful information for future alert improvement. These data suggest that it is possible to design computerized prescribing decision support with high rates of alert recommendation acceptance by clinicians.

A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests

PURPOSE To determine the impact of giving physicians computerized reminders about apparently redundant clinical laboratory tests. SUBJECTS AND METHODS We performed a prospective randomized controlled trial that included all inpatients at a large teaching hospital during a 15-week period. The intervention consisted of computerized reminders at the time a test was ordered that appeared to be redundant. Main outcome measures were the proportions of clinical laboratory orders that were canceled and the proportion of the tests that were actually performed. RESULTS During the study period, there were 939 apparently redundant laboratory tests among the 77,609 study tests that were ordered among the intervention (n = 5,700 patients) and control (n = 5,886 patients) groups. In the intervention group, 69% (300 of 437) of tests were canceled in response to reminders. Of 137 overrides, 41% appeared to be justified based on chart review. In the control group, 51% of ordered redundant tests were performed, whereas in the intervention group only 27% of ordered redundant tests were performed (P <0.001). However, the estimated annual savings in laboratory charges was only

Improving Response to Critical Laboratory Results with Automation: Results of a Randomized Controlled Trial

35,000. This occurred because only 44% of redundant tests performed had computer orders, because only half the computer orders were screened for redundancy, and because almost one-third of the reminders were overridden. CONCLUSIONS Reminders about orders for apparently redundant laboratory tests were effective when delivered. However, the overall effect was limited because many tests were performed without corresponding computer orders, and many orders were not screened for redundancy.

Reducing Vancomycin Use Utilizing a Computer Guideline: Results of a Randomized Controlled Trial

Objective: To evaluate the effect of an automatic alerting system on the time until treatment is ordered for patients with critical laboratory results. Design: Prospective randomized controlled trial. Intervention: A computer system to detect critical conditions and automatically notify the responsible physician via the hospitals paging system. Patients: Medical and surgical inpatients at a large academic medical center. One two-month study period for each service. Main outcomes: Interval from when a critical result was available for review until an appropriate treatment was ordered. Secondary outcomes were the time until the critical condition resolved and the frequency of adverse events. Methods: The alerting system looked for 12 conditions involving laboratory results and medications. For intervention patients, the covering physician was automatically notified about the presence of the results. For control patients, no automatic notification was made. Chart review was performed to determine the outcomes. Results: After exclusions, 192 alerting situations (94 interventions, 98 controls) were analyzed. The intervention group had a 38 percent shorter median time interval (1.0 hours vs. 1.6 hours, P = 0.003; mean, 4.1 vs. 4.6 hours, P = 0.003) until an appropriate treatment was ordered. The time until the alerting condition resolved was less in the intervention group (median, 8.4 hours vs. 8.9 hours, P = 0.11; mean, 14.4 hours vs. 20.2 hours, P = 0.11), although these results did not achieve statistical significance. The impact of the intervention was more pronounced for alerts that did not meet the laboratorys critical reporting criteria. There was no significant difference between the two groups in the number of adverse events. Conclusion: An automatic alerting system reduced the time until an appropriate treatment was ordered for patients who had critical laboratory results. Information technologies that facilitate the transmission of important patient data can potentially improve the quality of care.

Electronically Screening Discharge Summaries for Adverse Medical Events

BACKGROUND Vancomycin-resistant enterococci represent an increasingly important cause of nosocomial infections. Minimizing vancomycin use represents a key strategy in preventing the spread of these infections. OBJECTIVE To determine whether a structured ordering intervention using computerized physician order entry that requires use of a guideline could reduce intravenous vancomycin use. DESIGN Randomized controlled trial assessing frequency and duration of vancomycin therapy by physicians. PARTICIPANTS AND SETTING Three hundred ninety-six physicians and 1,798 patients in a tertiary-care teaching hospital. INTERVENTION Computer screen displaying, at the time of physician order entry, an adaptation of the Centers for Disease Control and Prevention guidelines for appropriate vancomycin use. MAIN OUTCOME MEASURES The frequency of initiation and renewal of vancomycin therapy as well the duration of therapy prescribed on a per prescriber basis. RESULTS Compared with the control group, intervention physicians wrote 32 percent fewer orders (11.3 versus 16.7 orders per physician; P = 0.04) and had 28 percent fewer patients for whom they either initiated or renewed an order for vancomycin (7.4 versus 10.3 orders per physician; P = 0.02). In addition, the duration of vancomycin therapy attributable to physicians in the intervention group was 36 percent lower than the duration of therapy prescribed by control physicians (26.5 versus 41.2 days; P = 0.05). Analysis of pharmacy data confirmed a decrease in the overall hospital use of intravenous vancomycin during the study period. CONCLUSION Implementation of a computerized guideline using physician order entry decreased vancomycin use. Computerized guidelines represent a promising tool for changing prescribing practices.

Overrides of medication-related clinical decision support alerts in outpatients

Objective: Detecting adverse events is pivotal for measuring and improving medical safety, yet current techniques discourage routine screening. The authors hypothesized that discharge summaries would include information on adverse events, and they developed and evaluated an electronic method for screening medical discharge summaries for adverse events. Design: A cohort study including 424 randomly selected admissions to the medical services of an academic medical center was conducted between January and July 2000. The authors developed a computerized screening tool that searched free-text discharge summaries for trigger words representing possible adverse events. Measurements: All discharge summaries with a trigger word present underwent chart review by two independent physician reviewers. The presence of adverse events was assessed using structured implicit judgment. A random sample of discharge summaries without trigger words also was reviewed. Results: Fifty-nine percent (251 of 424) of the discharge summaries contained trigger words. Based on discharge summary review, 44.8% (327 of 730) of the alerted trigger words indicated a possible adverse event. After medical record review, the tool detected 131 adverse events. The sensitivity and specificity of the screening tool were 69% and 48%, respectively. The positive predictive value of the tool was 52%. Conclusion: Medical discharge summaries contain information regarding adverse events. Electronic screening of discharge summaries for adverse events using keyword searches is feasible but thus far has poor specificity. Nonetheless, computerized clinical narrative screening methods could potentially offer researchers and quality managers a means to routinely detect adverse events.

Automated Evidence-based Critiquing of Orders for Abdominal Radiographs: Impact on Utilization and Appropriateness

BACKGROUND Electronic prescribing is increasingly used, in part because of government incentives for its use. Many of its benefits come from clinical decision support (CDS), but often too many alerts are displayed, resulting in alert fatigue. OBJECTIVE To characterize the override rates for medication-related CDS alerts in the outpatient setting, the reasons cited for overrides at the time of prescribing, and the appropriateness of overrides. METHODS We measured CDS alert override rates and the coded reasons for overrides cited by providers at the time of prescribing. Our primary outcome was the rate of CDS alert overrides; our secondary outcomes were the rate of overrides by alert type, reasons cited for overrides at the time of prescribing, and override appropriateness for a subset of 600 alert overrides. Through detailed chart reviews of alert override cases, and selective literature review, we developed appropriateness criteria for each alert type, which were modified iteratively as necessary until consensus was reached on all criteria. RESULTS We reviewed 157,483 CDS alerts (7.9% alert rate) on 2,004,069 medication orders during the study period. 82,889 (52.6%) of alerts were overridden. The most common alerts were duplicate drug (33.1%), patient allergy (16.8%), and drug-drug interactions (15.8%). The most likely alerts to be overridden were formulary substitutions (85.0%), age-based recommendations (79.0%), renal recommendations (78.0%), and patient allergies (77.4%). An average of 53% of overrides were classified as appropriate, and rates of appropriateness varied by alert type (p<0.0001) from 12% for renal recommendations to 92% for patient allergies. DISCUSSION About half of CDS alerts were overridden by providers and about half of the overrides were classified as appropriate, but the likelihood of overriding an alert varied widely by alert type. Refinement of these alerts has the potential to improve the relevance of alerts and reduce alert fatigue.

A Computer-Based Intervention for Improving the Appropriateness of Antiepileptic Drug Level Monitoring

OBJECTIVE Inappropriate utilization of diagnostic testing has been well documented. The purpose of this study was to measure the impact of presenting real time, evidence-based critiques about the appropriateness of abdominal radiograph (KUB) orders on physician decision making. DESIGN Prospective trial where evidence-based critiques were presented to ordering clinicians in two kinds of situations: (1) a KUB was likely to have a low probability of providing useful information, or (2) an alternative view(s) was more appropriate given the clinical circumstance. There were two phases of the trial: Phase 1 was a 9-week period where evidence-based critiques were presented at the time of ordering a KUB, followed by Phase 2, a 19-week period in which orderers were randomized to receive critiques either amended to include both institutional data regarding the utility of the critiques and stronger messages about the lack of utility of the study, or the same critiques as presented in Phase 1, depending upon indication. Based upon the radiologists report of their interpretation of the exams, the results of the examinations were scored as positive, equivocal, or negative using structured criteria. RESULTS 299 KUBs in Phase 1 and 385 KUBs in Phase 2 received at least one critique. Cancellation rates of low yield films were low, and were similar in Phase 1 and 2, 8/258 (3%) vs. 10/283 (4%). Compliance with the recommendation for alternative view(s) was higher: 19/104 (38%) in Phase 1 vs. 96/176 (55%) in Phase 2 (p = 0.006). The results differentiated low-yield from non-low-yield films: 5% of low-yield films vs. 20% of non-low-yield films were positive in Phase 2 (p < 0.0001). Surgical physicians were less likely to cancel (p = 0.07) or to change to the suggested view(s) (p < 0.0001) than medical physicians or nurses. CONCLUSIONS The intervention identified clinical situations in which KUBs appeared to have a low clinical yield. In response to evidence-based critiques, providers were reluctant to cancel their order, but were more willing to change to different views. To reduce the number of inappropriate radiographic films, stronger incentives or interventions may be required.

Monitoring Amiodarone's Toxicities: Recommendations, Evidence, and Clinical Practice

We designed and implemented 2 automated, computerized screens for use at the time of antiepileptic drug (AED) test order entry to improve appropriateness by reminding physicians when a potentially redundant test was ordered and providing common indications for monitoring and pharmacokinetics of the specific AED. All computerized orders for inpatient serum AED levels during two 3-month periods were included in the study. During the 3-month period after implementation of the automated intervention, 13% of all AED tests ordered were canceled following computerized reminders. For orders appearing redundant, the cancellation rate was 27%. For nonredundant orders, 4% were canceled when information on specific AED monitoring and pharmacokinetics was provided. The cancellation rate was sustained after 4 years. There has been a 19.5% decrease in total AED testing volume since implementation of this intervention, despite a 19.3% increase in overall chemistry test volume. Inappropriateness owing to repeated testing before pharmacologic steady state was reached decreased from 54% of all AED orders to 14.6%. A simple, automated, activity-based intervention targeting a specific test-ordering behavior effectively reduced inappropriate laboratory testing. The sustained benefit supports the idea that computerized interventions may durably affect physician behavior. Computerized delivery of such evidence-based boundary guidelines can help narrow the gap between evidence and practice.

Using an Electronic Medical Record to Identify Opportunities to Improve Compliance with Cholesterol Guidelines

We sought to develop an explicit evidence‐based model of medication monitoring and to evaluate monitoring practices and adverse drug events in patients taking amiodarone at one institution.