Marilyn D. Paterno

Automating Complex Guidelines for Chronic Disease: Lessons Learned

There is scant published experience with implementing complex, multistep computerized practice guidelines for the long-term management of chronic diseases. We have implemented a system for creating, maintaining, and navigating computer-based clinical algorithms integrated with our electronic medical record. This article describes our progress and reports on lessons learned that might guide future work in this field. We discuss issues and obstacles related to choosing and adapting a guideline for electronic implementation, representing and executing the guideline as a computerized algorithm, and integrating it into the clinical workflow of outpatient care. Although obstacles were encountered at each of these steps, the most difficult were related to workflow integration.

Physician Alerts to Prevent Symptomatic Venous Thromboembolism in Hospitalized Patients

Background— Venous thromboembolism (VTE) prophylaxis remains underused among hospitalized patients. We designed and carried out a large, multicenter, randomized controlled trial to test the hypothesis that an alert from a hospital staff member to the attending physician will reduce the rate of symptomatic VTE among high-risk patients not receiving prophylaxis. Methods and Results— We enrolled patients using a validated point score system to detect hospitalized patients at high risk for symptomatic VTE who were not receiving prophylaxis. We randomized 2493 patients (82% on Medical Services) from 25 study sites to the intervention group (n=1238), in which the responsible physician was alerted by another hospital staff member, or the control group (n=1255), in which no alert was issued. The primary end point was symptomatic, objectively confirmed VTE within 90 days. Patients whose physicians were alerted were more than twice as likely to receive VTE prophylaxis as control subjects (46.0% versus 20.6%; P<0.0001). The symptomatic VTE rate was lower in the intervention group (2.7% versus 3.4%; hazard ratio, 0.79; 95% CI, 0.50 to 1.25), but the difference did not achieve statistical significance. The rate of major bleeding at 30 days in the alert group was similar to that in the control group (2.1% versus 2.3%; P=0.68). Conclusions— A strategy of direct notification of the physician by a staff member increases prophylaxis use and leads to a reduction in the rate of symptomatic VTE in hospitalized patients. However, VTE prophylaxis continues to be underused even after physician notification, especially among Medical Service patients.

Factors influencing alert acceptance: a novel approach for predicting the success of clinical decision support.

BACKGROUND Clinical decision support systems can prevent knowledge-based prescription errors and improve patient outcomes. The clinical effectiveness of these systems, however, is substantially limited by poor user acceptance of presented warnings. To enhance alert acceptance it may be useful to quantify the impact of potential modulators of acceptance. METHODS We built a logistic regression model to predict alert acceptance of drug-drug interaction (DDI) alerts in three different settings. Ten variables from the clinical and human factors literature were evaluated as potential modulators of provider alert acceptance. ORs were calculated for the impact of knowledge quality, alert display, textual information, prioritization, setting, patient age, dose-dependent toxicity, alert frequency, alert level, and required acknowledgment on acceptance of the DDI alert. RESULTS 50,788 DDI alerts were analyzed. Providers accepted only 1.4% of non-interruptive alerts. For interruptive alerts, user acceptance positively correlated with frequency of the alert (OR 1.30, 95% CI 1.23 to 1.38), quality of display (4.75, 3.87 to 5.84), and alert level (1.74, 1.63 to 1.86). Alert acceptance was higher in inpatients (2.63, 2.32 to 2.97) and for drugs with dose-dependent toxicity (1.13, 1.07 to 1.21). The textual information influenced the mode of reaction and providers were more likely to modify the prescription if the message contained detailed advice on how to manage the DDI. CONCLUSION We evaluated potential modulators of alert acceptance by assessing content and human factors issues, and quantified the impact of a number of specific factors which influence alert acceptance. This information may help improve clinical decision support systems design.

A pilot study of distributed knowledge management and clinical decision support in the cloud

OBJECTIVE Implement and perform pilot testing of web-based clinical decision support services using a novel framework for creating and managing clinical knowledge in a distributed fashion using the cloud. The pilot sought to (1) develop and test connectivity to an external clinical decision support (CDS) service, (2) assess the exchange of data to and knowledge from the external CDS service, and (3) capture lessons to guide expansion to more practice sites and users. MATERIALS AND METHODS The Clinical Decision Support Consortium created a repository of shared CDS knowledge for managing hypertension, diabetes, and coronary artery disease in a community cloud hosted by Partners HealthCare. A limited data set for primary care patients at a separate health system was securely transmitted to a CDS rules engine hosted in the cloud. Preventive care reminders triggered by the limited data set were returned for display to clinician end users for review and display. During a pilot study, we (1) monitored connectivity and system performance, (2) studied the exchange of data and decision support reminders between the two health systems, and (3) captured lessons. RESULTS During the six month pilot study, there were 1339 patient encounters in which information was successfully exchanged. Preventive care reminders were displayed during 57% of patient visits, most often reminding physicians to monitor blood pressure for hypertensive patients (29%) and order eye exams for patients with diabetes (28%). Lessons learned were grouped into five themes: performance, governance, semantic interoperability, ongoing adjustments, and usability. DISCUSSION Remote, asynchronous cloud-based decision support performed reasonably well, although issues concerning governance, semantic interoperability, and usability remain key challenges for successful adoption and use of cloud-based CDS that will require collaboration between biomedical informatics and computer science disciplines. CONCLUSION Decision support in the cloud is feasible and may be a reasonable path toward achieving better support of clinical decision-making across the widest range of health care providers.

Multi-screen electronic alerts to augment venous thromboembolism prophylaxis

Venous thromboembolism (VTE) prophylaxis in high-risk patients is frequently underutilised. We previously devised a one-screen computer alert program that identified hospitalised patients at high risk for VTE who were not receiving prophylaxis and advised their physicians to prescribe prophylaxis. While this strategy reduced the 90-day incidence of symptomatic VTE by 41%, the majority of electronic alerts were ignored. We have now developed a serial three-screen alert computer program designed to educate physicians who initially declined to order prophylaxis after a single screen alert. Of a total cohort of 880, the responsible physicians for 425 patients received a single electronic alert, whereas 455 who declined prophylaxis after the first screen received the second and third screens of the novel three-screen alert. Our enhanced serial three-screen alert program generated VTE prophylaxis orders for 58.4% of the 455 patients whose physicians initially declined to order prophylaxis following the one-screen alert. There was no significant difference in symptomatic 90-day VTE rates between the two cohorts (2.8% for the one-screen vs. 2.2% for the three-screen, p=0.55). We conclude that our three-screen computer alert program can markedly increase prophylaxis among physicians who decline an initial single screen alert.

A multi-layered framework for disseminating knowledge for computer-based decision support.

BACKGROUND There are several challenges in encoding guideline knowledge in a form that is portable to different clinical sites, including the heterogeneity of clinical decision support (CDS) tools, of patient data representations, and of workflows. METHODS We have developed a multi-layered knowledge representation framework for structuring guideline recommendations for implementation in a variety of CDS contexts. In this framework, guideline recommendations are increasingly structured through four layers, successively transforming a narrative text recommendation into input for a CDS system. We have used this framework to implement rules for a CDS service based on three guidelines. We also conducted a preliminary evaluation, where we asked CDS experts at four institutions to rate the implementability of six recommendations from the three guidelines. CONCLUSION The experience in using the framework and the preliminary evaluation indicate that this approach has promise in creating structured knowledge, to implement in CDS systems, that is usable across organizations.

Patient education program for venous thromboembolism prevention in hospitalized patients.

PURPOSE Up to 15% of clinician-ordered doses of injectable pharmacological prophylaxis to prevent venous thromboembolism are not administered. Patient refusal accounts for nearly 50% of these omitted doses. We conducted a prospective cohort study to determine whether a patient education program would improve medication adherence to clinician-ordered injectable prophylactic anticoagulation. METHODS We identified 528 hospitalized patients ordered to receive injectable pharmacological venous thromboembolism prophylaxis. We evaluated the impact of pharmacist-led patient education sessions on medication adherence (defined as the ratio of doses administered to doses scheduled) compared with our historical cohort. RESULTS Individualized patient education sessions were conducted within 24 hours of the initial order for prophylactic anticoagulation in 99% of patients. Adherence to clinician-ordered pharmacological venous thromboembolism prophylaxis was higher after the patient education program than in our historical cohort (94.4% vs 89.9%, P <.0001). The proportion of patients receiving 100% of scheduled doses of injectable pharmacological venous thromboembolism prophylaxis was higher after our novel patient education program than in our historical cohort (73.7% vs 62.4%, P=.001). Patient refusal as a reason for omitted doses was less frequent after the patient education program (29.3% vs 43.7%, P=.001). CONCLUSION Pharmacist-led individualized patient education sessions were associated with higher medication adherence to clinician-ordered injectable pharmacological venous thromboembolism prophylaxis and a reduction in patient refusal as a reason for omitted doses. A randomized controlled trial to evaluate the impact of a patient education program on medication adherence to pharmacological venous thromboembolism prophylaxis is warranted.

A highly scalable, interoperable clinical decision support service.

OBJECTIVE To create a clinical decision support (CDS) system that is shareable across healthcare delivery systems and settings over large geographic regions. MATERIALS AND METHODS The enterprise clinical rules service (ECRS) realizes nine design principles through a series of enterprise java beans and leverages off-the-shelf rules management systems in order to provide consistent, maintainable, and scalable decision support in a variety of settings. RESULTS The ECRS is deployed at Partners HealthCare System (PHS) and is in use for a series of trials by members of the CDS consortium, including internally developed systems at PHS, the Regenstrief Institute, and vendor-based systems deployed at locations in Oregon and New Jersey. Performance measures indicate that the ECRS provides sub-second response time when measured apart from services required to retrieve data and assemble the continuity of care document used as input. DISCUSSION We consider related work, design decisions, comparisons with emerging national standards, and discuss uses and limitations of the ECRS. CONCLUSIONS ECRS design, implementation, and use in CDS consortium trials indicate that it provides the flexibility and modularity needed for broad use and performs adequately. Future work will investigate additional CDS patterns, alternative methods of data passing, and further optimizations in ECRS performance.

Lessons learned from implementing service-oriented clinical decision support at four sites: A qualitative study

OBJECTIVE To identify challenges, lessons learned and best practices for service-oriented clinical decision support, based on the results of the Clinical Decision Support Consortium, a multi-site study which developed, implemented and evaluated clinical decision support services in a diverse range of electronic health records. METHODS Ethnographic investigation using the rapid assessment process, a procedure for agile qualitative data collection and analysis, including clinical observation, system demonstrations and analysis and 91 interviews. RESULTS We identified challenges and lessons learned in eight dimensions: (1) hardware and software computing infrastructure, (2) clinical content, (3) human-computer interface, (4) people, (5) workflow and communication, (6) internal organizational policies, procedures, environment and culture, (7) external rules, regulations, and pressures and (8) system measurement and monitoring. Key challenges included performance issues (particularly related to data retrieval), differences in terminologies used across sites, workflow variability and the need for a legal framework. DISCUSSION Based on the challenges and lessons learned, we identified eight best practices for developers and implementers of service-oriented clinical decision support: (1) optimize performance, or make asynchronous calls, (2) be liberal in what you accept (particularly for terminology), (3) foster clinical transparency, (4) develop a legal framework, (5) support a flexible front-end, (6) dedicate human resources, (7) support peer-to-peer communication, (8) improve standards. CONCLUSION The Clinical Decision Support Consortium successfully developed a clinical decision support service and implemented it in four different electronic health records and four diverse clinical sites; however, the process was arduous. The lessons identified by the Consortium may be useful for other developers and implementers of clinical decision support services.

Use of Traumatic Brain Injury Prediction Rules With Clinical Decision Support

The investigators provide data from a multicenter trial regarding whether implementation of prediction rules safely decreases computed tomography use in children with minor head trauma. OBJECTIVES: We determined whether implementing the Pediatric Emergency Care Applied Research Network (PECARN) traumatic brain injury (TBI) prediction rules and providing risks of clinically important TBIs (ciTBIs) with computerized clinical decision support (CDS) reduces computed tomography (CT) use for children with minor head trauma. METHODS: Nonrandomized trial with concurrent controls at 5 pediatric emergency departments (PEDs) and 8 general EDs (GEDs) between November 2011 and June 2014. Patients were <18 years old with minor blunt head trauma. Intervention sites received CDS with CT recommendations and risks of ciTBI, both for patients at very low risk of ciTBI (no Pediatric Emergency Care Applied Research Network rule factors) and those not at very low risk. The primary outcome was the rate of CT, analyzed by site, controlling for time trend. RESULTS: We analyzed 16 635 intervention and 2394 control patients. Adjusted for time trends, CT rates decreased significantly (P < .05) but modestly (2.3%–3.7%) at 2 of 4 intervention PEDs for children at very low risk. The other 2 PEDs had small (0.8%–1.5%) nonsignificant decreases. CT rates did not decrease consistently at the intervention GEDs, with low baseline CT rates (2.1%–4.0%) in those at very low risk. The control PED had little change in CT use in similar children (from 1.6% to 2.9%); the control GED showed a decrease in the CT rate (from 7.1% to 2.6%). For all children with minor head trauma, intervention sites had small decreases in CT rates (1.7%–6.2%). CONCLUSIONS: The implementation of TBI prediction rules and provision of risks of ciTBIs by using CDS was associated with modest, safe, but variable decreases in CT use. However, some secular trends were also noted.