Brittany L. Melton

Applying human factors principles to alert design increases efficiency and reduces prescribing errors in a scenario-based simulation.

OBJECTIVE To apply human factors engineering principles to improve alert interface design. We hypothesized that incorporating human factors principles into alerts would improve usability, reduce workload for prescribers, and reduce prescribing errors. MATERIALS AND METHODS We performed a scenario-based simulation study using a counterbalanced, crossover design with 20 Veterans Affairs prescribers to compare original versus redesigned alerts. We redesigned drug-allergy, drug-drug interaction, and drug-disease alerts based upon human factors principles. We assessed usability (learnability of redesign, efficiency, satisfaction, and usability errors), perceived workload, and prescribing errors. RESULTS Although prescribers received no training on the design changes, prescribers were able to resolve redesigned alerts more efficiently (median (IQR): 56 (47) s) compared to the original alerts (85 (71) s; p=0.015). In addition, prescribers rated redesigned alerts significantly higher than original alerts across several dimensions of satisfaction. Redesigned alerts led to a modest but significant reduction in workload (p=0.042) and significantly reduced the number of prescribing errors per prescriber (median (range): 2 (1-5) compared to original alerts: 4 (1-7); p=0.024). DISCUSSION Aspects of the redesigned alerts that likely contributed to better prescribing include design modifications that reduced usability-related errors, providing clinical data closer to the point of decision, and displaying alert text in a tabular format. Displaying alert text in a tabular format may help prescribers extract information quickly and thereby increase responsiveness to alerts. CONCLUSIONS This simulation study provides evidence that applying human factors design principles to medication alerts can improve usability and prescribing outcomes.

A Graduate Student Mentoring Program to Develop Interest in Research

Objective. To assess the impact of a graduate student mentoring program on student interest in research and postgraduate education and on graduate student confidence in mentoring. Methods. Undergraduate and pharmacy students (mentees) and graduate students (mentors) were matched and participated in the study, which required them to engage in at least 2 discussions regarding research and careers. Mentees completed a pre- and post-assessment of their perceptions of research, postgraduate training plans, and perceptions about mentors. Mentors completed a pre- and post-assessment of their perceptions about themselves as mentors and their confidence in mentoring. Results. Although there were no significant differences among the mentees’ perceptions of research or the mentors’ confidence in mentoring, qualitative analysis indicated that the mentees’ perceptions of research improved and that the mentors believed their mentoring skills improved. Conclusions. Based on the results of the qualitative analysis, implementing a graduate student mentoring program may help improve students’ perceptions of research and graduate students’ confidence in mentoring, which could increase student interest in postgraduate education and prepare mentors for future leadership roles.

Identification of adverse drug-drug interactions through causal association rule discovery from spontaneous adverse event reports

OBJECTIVE Drug-drug interaction (DDI) is of serious concern, causing over 30% of all adverse drug reactions and resulting in significant morbidity and mortality. Early discovery of adverse DDI is critical to prevent patient harm. Spontaneous reporting systems have been a major resource for drug safety surveillance that routinely collects adverse event reports from patients and healthcare professionals. In this study, we present a novel approach to discover DDIs from the Food and Drug Administrations adverse event reporting system. METHODS Data-driven discovery of DDI is an extremely challenging task because higher-order associations require analysis of all combinations of drugs and adverse events and accurate estimate of the relationships between drug combinations and adverse event require cause-and-effect inference. To efficiently identify causal relationships, we introduce the causal concept into association rule mining by developing a method called Causal Association Rule Discovery (CARD). The properties of V-structures in Bayesian Networks are utilized in the search for causal associations. To demonstrate feasibility, CARD is compared to the traditional association rule mining (AR) method in DDI identification. RESULTS Based on physician evaluation of 100 randomly selected higher-order associations generated by CARD and AR, CARD is demonstrated to be more accurate in identifying known drug interactions compared to AR, 20% vs. 10% respectively. Moreover, CARD yielded a lower number of drug combinations that are unknown to interact, i.e., 50% for CARD and 79% for AR. CONCLUSION Evaluation analysis demonstrated that CARD is more likely to identify true causal drug variables and associations to adverse event.

Patient Satisfaction With Pharmacist-Led Chronic Disease State Management Programs:

Purpose: To assess patient satisfaction, perception of self-management, and perception of disease state knowledge with pharmacist-led diabetes and cardiovascular disease state management (DSM) programs. Methods: A self-insured chain of grocery store pharmacies in the Kansas City metropolitan area administers pharmacist-led diabetes and cardiovascular DSM programs for eligible employees and dependents. A modified version of the Diabetes Disease State Management Questionnaire was used to assess patient satisfaction with the DSM programs. Demographic information was also collected. Survey items were based on a 5-point Likert scale (1 = strongly disagree and 5 = strongly agree). Patients were eligible to complete the survey if he or she had been in at least 1 DSM program for 6 months. Data were assessed using descriptive statistics and analysis of variance. Results: Across 20 pharmacies, 281 eligible participants were identified, and 46% (n = 128) completed a survey. Means for summed items relating to overall satisfaction (8 items), self-management (5 items), and knowledge (4 items) were 36.6/40 (standard deviation [SD] = 3.9), 20.9/25 (SD = 3.4), and 17.6/20 (SD = 2.1), respectively. Participant comments further indicated that the program and pharmacists are helpful and increase motivation and accountability. Conclusions: Positive patient responses to the program support use of pharmacist-led DSM programs.

Design and Evaluation of an Electronic Override Mechanism for Medication Alerts to Facilitate Communication Between Prescribers and Pharmacists

Background: Computerized medication alerts can often be bypassed by entering an override rationale, but prescribers’ override reasons are frequently ambiguous to pharmacists who review orders. Objective: To develop and evaluate a new override mechanism for adverse reaction and drug-drug interaction alerts. We hypothesized that the new mechanism would improve usability for prescribers and increase the clinical appropriateness of override reasons. Methods: A counterbalanced, crossover study was conducted with 20 prescribers in a simulated prescribing environment. We modified the override mechanism timing, navigation, and text entry. Instead of free-text entry, the new mechanism presented prescribers with a predefined set of override reasons. We assessed usability (learnability, perceived efficiency, and usability errors) and used a priori criteria to evaluate the clinical appropriateness of override reasons entered. Results: Prescribers rated the new mechanism as more efficient (Wilcoxon signed-rank test, P = 0.032). When first using the new design, 5 prescribers had difficulty finding the new mechanism, and 3 interpreted the navigation to mean that the alert could not be overridden. The number of appropriate override reasons significantly increased with the new mechanism compared with the original mechanism (median change of 3.0; interquartile range = 3.0; P < 0.0001). Conclusions: When prescribers were given a menu-based choice for override reasons, clinical appropriateness of these reasons significantly improved. Further enhancements are necessary, but this study is an important first step toward a more standardized menu of override choices. Findings may be used to improve communication through e-prescribing systems between prescribers and pharmacists.

Association of Uninterrupted Oral Anticoagulation During Cardiac Device Implantation with Pocket Hematoma.

Purpose Implantation of permanent pacemakers (PPMs) or implantable cardiac defibrillators (ICDs) may be complicated by the development of pocket hematomas. Current practice guidelines provide little guidance to clinicians about the preferred strategy for chronic oral anticoagulation (OAC). The purpose of this study was to examine the frequency and clinical significance of pocket hematoma among patients receiving uninterrupted OAC during cardiac device implantation. Methods This was a retrospective cohort study of adult patients undergoing cardiac device implantation between January 1, 2011, and December 31, 2012, at an academic teaching hospital. Medical records were reviewed for demographics, comorbidities, and medications. The primary outcome was development of pocket hematomas within 30 days of device implantation. Clinical significance was based on the need for additional intervention. Data were assessed using descriptive statistics, logistic regression, and chi-square tests. Results The final cohort included 380 patients. The median age was 68.4 years, and 56.6% were male. Cardiovascular comorbidities were common. Among 80 patients receiving uninterrupted OAC, 71.3% were taking warfarin, 11.2% rivaroxaban, and 17.5% dabigatran. The incidence of pocket hematomas for the entire cohort was 9.7%, of which 1.3% were clinically significant. Pocket hematoma occurred in 21.4% of patients continued on OAC versus 7.7% of those not anticoagulated (P = .001). Pocket hematoma was more common among those receiving ICDs than PPMs (18.5% vs 5.7%, respectively; P < .001). Conclusions Continuing chronic OAC increased pocket hematoma formation but most were clinically insignificant. Pocket hematoma occurred irrespective of the oral anticoagulant drug used, but additional study is needed to determine comparative risks among the drugs.

CPOE Teams: A Prescription for Pharmacist Intervention.

In 2010, roughly 1 in 5 hospitals had implemented computerized provider order entry (CPOE).1 Hospitals are expected to meet the standards for Meaningful Use in order to avoid paying penalties, therefore the number of hospitals adopting CPOE is expected to increase dramatically. What this means for physicians is relatively clear; the way they order medications will change from a paper medium to an electronic one. How to improve physician acceptance of CPOE and factors influencing their use of such systems is well documented. The switch to CPOE for pharmacists, however, is more than a just an exchange of mediums; rather, it is a sweeping transformation in practice and, for some, it creates a need to justify their continued employment. Despite these far-reaching effects on the practice of pharmacy in institutions, pharmacists are not always granted a significant or early position on CPOE implementation teams. Pharmacists are uniquely positioned in the hospital system to see aspects of a CPOE system that no other member of the health care team sees and will likely encounter alerts that no other group will be required to address. In 2001, when CPOE was beginning to take hold and before residency-trained clinical pharmacists were commonplace in hospitals, the American Society of Health-System Pharmacists (ASHP) developed a list of recommendations to address pharmacists’ roles in the implementation process.2 Many things have changed since that list was initially released. CPOE systems have become more sophisticated, and the number of functions they can and should perform has multiplied. The pharmacist’s role will shift from being predominantly order entry toward primarily providing direct patient care. That means the need for pharmacists’ early involvement in CPOE implementation is even greater because pharmacists will be using all aspects of the system and not just processing orders. Clinical and dispensing pharmacists should be involved in choosing and developing the institution’s CPOE system. Both play an important part in the effective functioning of a pharmacy department and both bring unique insights into the process. Clinical pharmacists understand workflow in the context of direct patient care and must be able to enter orders into the system on behalf of the physician; they also need access to the full array of alerts and decision support tools that can impact drug-drug interactions, dosing, allergies, and more. Dispensing pharmacists may not always need the full array of clinical tools, but they frequently have a greater understanding of order verification and may often be called upon to ensure that complex or unusual physician-entered orders are processed in a manner that will be clearly understood by the nursing staff. Dispensing pharmacists and clinical pharmacists both need ready access to more than just the prescription ordering aspects of a CPOE system. To accurately assess presented alerts, pharmacists need easy access to laboratory results, chart notes, and even administration records. Given the breadth of information that pharmacists need to provide comprehensive patient care, they should be recruited from both the clinical and dispensing areas of the pharmacy department when the initial steering committee is formed. CPOE vendors will provide information about their systems and often will provide on-site demonstrations to promote their products. This is a prime time for pharmacists to test the capabilities of a system they will use on a daily basis and to see what the alerts and other clinical decision support tools look like and assess their usability. The preparation of specific scenarios prior to such demonstrations allows for consistency in testing possible CPOE systems and ensures that all functions are usable from the pharmacist’s perspective. This is where the inclusion of clinical and dispensing pharmacists on the CPOE team becomes imperative. A clinical pharmacist with little experience working in order entry/verification may miss limitations inherent in a system that a dispensing pharmacist would catch because of the differences in how they would normally interact with the CPOE system. It is easy to overlook all the components of a CPOE system to which a pharmacist may need access in order to provide appropriate patient care, and it is even easier to think that the most important group for system approval are physicians. CPOE represents a fundamental shift in how hospital pharmacists practice, and their stake in the success or failure of the system is not to be ignored. Pharmacists, both clinical and dispensing, must be involved in the selection and implementation of a suitable CPOE system from the start of the process to make sure that their needs are appropriately addressed. Attempting to implement a CPOE system without a clearly written prescription for both clinical and dispensing pharmacist involvement in the entire process, can slow adoption and acceptance of the system and can also lead to reductions in job satisfaction and increased patient safety risks.

Evaluation of Cognitive Encoding Across Different Medication Alert Designs

As part of the Meaningful Use criteria, electronic health record systems are required to include medication alerts to warn prescribers about medication allergies and drug-drug interactions before prescriptions are dispensed to patients. There is a paucity of research on medication alert design. Additionally, studies have not evaluated how well alert designs support prescribers’ cognitive encoding, even though this is a key step in human information processing for warnings. The objective of this study was to develop a methodological protocol for assessing prescribers’ encoding of medication alerts and evaluate prescribers’ cognitive encoding in response to two different alert designs. We hypothesized that a redesigned alert display that incorporates human factors principles would significantly increase the amount of information that is accurately encoded. A counterbalanced, crossover study was conducted with 20 prescribers in a human-computer interaction laboratory. We measured prescribers’ free recall as well as their ability to identify that three warning messages were displayed. The proportion of total data elements that prescribers were able to accurately recall was significantly greater for the redesigned versus original alerts (median difference in Original-Redesign = -.09, IQR = .15, Wilcoxon signed-rank test p = .006). Additionally, with the redesigned alerts, more prescribers accurately reported that three warnings were displayed (Exact McNemar’s test p = .002). Study methods may be useful for evaluating cognitive encoding in the healthcare domain and results may inform future medication alert designs.

Communicating to Improve Continuity of Care.

*Assistant Professor, School of Pharmacy, University of Kansas, 3901 Rainbow Boulevard, Wescoe 6012, Kansas City, KS 66160; phone: 913-588-5392; fax: 913-5882355; e-mail: bmelton2@kumc.edu George Bernard Shaw once said, “The single biggest problem in communication is the illusion that it has taken place.”1(p71) In the recent past, health care providers operated in silos, and communication, as well as collaboration, was limited. Despite new care models centering on collaborative care and the concept of effective and real-time communication, the application of such communication methods is limited to computerized provider order entry (CPOE) and clinical decision support (CDS). CPOE and CDS systems are meant to assist providers with their decision making and facilitate communication. Although the goal of these systems is to provide communication, it may be an illusion of communication not only for the administrators, but also for the providers and patients. There are 3 areas in which communication breakdown can occur (Figure 1). First is the intent of CDS. There is a belief by administrators and providers that CDS will eliminate their communication woes and protect providers from missing information central to safe patient care. When the CDS is both well designed and patient specific, it is an effective tool that presents the needed information in a format that can be readily understood by the provider. This gives hospitals the illusion that CDS is an effective interaction and will meet all their needs. The intent of CDS and the expectation of administrators is that it is effectively interacting with providers, however the CDS is actually creating an opportunity for more medication errors because important information is lost due to poor design. It has also been shown that changes in communication practices and patterns occur and can be an unintended adverse consequence of CPOE and CDS implementation.2 For example, a physician may chose a response from a drop-down list and unknowingly select the wrong option, something that did not occur when orders were written on paper.3 In this regard, pharmacists face the same challenges as physicians; pharmacists expect the CDS to interact with them as needed, when in fact it may not fulfill their needs. Redesigning CDS to present only necessary information in a format that is rapidly interpreted, such as presenting laboratory information in a table rather than a string of text, can cut the number of errors almost in half and improve provider satisfaction.4 Systems can reduce error by providing redundancies and limiting the amount of information presented in a drop-down list or in an alert. Second is when the CDS is actually transmitting information. In a good system, only relevant and necessary information is presented along with options for addressing the patient safety issue. In truth, CDS may not be designed well; it may provide too much or vague information or not take into account the needs of the providers who will be using the CDS. This can result in frustration or alert fatigue. It is estimated that up to 96% of CDS alerts are overridden, often due to fatigue.5 Even in a well-designed system, when the physician has a more complete picture of the patient, he/she may decide not to act upon the CDS information and will appropriately override an alert. These overrides can include a rationale to justify the physician’s action, such as indicating that the patient-reported medication allergy is non–life threatening or is not a true allergy and the medication can safely be administered. When physicians enter a rationale or reason for overriding a CDS alert, that rationale is often visible to the pharmacist who is processing the order as part of care coordination and information sharing. However, the physician may enter the most expedient rationale rather than the most clinically valid, not realizing that a pharmacist will review the override rationale and rely upon the information provided. This can result in a delay in care, as the pharmacist may have to spend time deciphering the physician’s rationale or calling the physician. In a well-designed system, the pharmacist, like the physician, can see the necessary information and make a well-informed decision; but in some systems,

Informatics: catching the next wave in pharmacy education.

In a society dominated by smartphones and the Internet, US health care is finding itself in a position where it has to race to catch the wave of technological advances sweeping through all aspects of our daily lives. The Health Information Technology for Economic and Clinical Health (HITECH) Act of 2013 mandated the implementation of electronic health records (EHRs) and computerized prescriber order entry (CPOE) systems.1 “Meaningful use” is a buzz-phrase on the minds of many hospital administrators and pharmacy managers. Compliance with the vague and confusing requirements for meaningful use is necessary to receive incentives from the Centers for Medicare & Medicaid Services (CMS), but this leaves many pharmacy managers feeling that they have been set adrift without the knowledge or experience needed to navigate these uncharted waters. Pharmacists are extremely well trained to deal with the clinical components of health care, but when it comes to evaluating and implementing EHR and CPOE systems, pharmacists find themselves unprepared. Even in the recent past, pharmacists did not foresee a time when such skills would be an integral part of the profession. The American Society of Health-System Pharmacists (ASHP) broadly defines pharmacy informatics as the “use and integration of data, information, knowledge, technology, and automation in the medication-use process for the purpose of improving health outcomes.”2(p200) Today, this includes systems such as CPOE, EHR, clinical decision support, and many others. To meet such broad expectations, pharmacists need to be able to hit the ground running upon graduation; on-the-job training is not only too late, but in many cases there is a lack of resources available to provide such training. Some pharmacists may be expected to contribute to the selection and/or implementation of such systems shortly after beginning professional practice. Even those not involved in system implementation can expect to interact with these systems as part of their daily duties regardless of their chosen practice. Attempting to implement or use any of these systems without understanding the foundations for the technology, potentials for errors, or how to address problems as they arise can lead to undue stress upon the pharmacist and the potential introduction of additional patient and medication safety hazards for the health care organization. Traditionally, pharmacists did not need an education in health information technology to be successful, but that has changed. In almost every practice setting, pharmacists are required to use computers and increasingly must rely upon electronic references and automated technology such as robots to complete their duties. The Accreditation Council for Pharmacy Education (ACPE) recognized this change in the professional landscape and included informatics in their accreditation standards in 2007.3 Informatics became a hot topic, and professional organizations released statements encouraging the integration of informatics in the practice of pharmacy and pharmacy curricula.2,4 Despite the push to include informatics in professional curricula, a 2008 study identified that only 36% of pharmacy schools had enough informatics material in their curricula to meet the ACPE guidelines.5 It is reasonable to assume that percentage has increased, as many new schools of pharmacy have opened since the publication of that study and established schools have been responding to the ACPE standards to incorporate informatics in the curriculum. Are these changes fast enough to keep pace with the needs of the profession? Additionally, long-practicing pharmacists may seek to update their skillset to ride the technology wave, and they will also need informatics resources. As pharmacist’s roles in health care have changed and expanded over time, pharmacy schools have responded by adding subject matter specialists to their faculty. Soon, adding an informatics specialist to their cardiology, pediatric, and oncologic specialists will be a necessity to ensure that their students are well prepared for the profession they are about to enter. Even though most of today’s pharmacy students have grown up using technology such as cell phones, computers, and the Internet, they still need exposure to informatics during their professional training to introduce them to the technology they will use upon graduation and to give them the skills that will allow them to critically evaluate the advantages and shortcomings of the technology they use. The inclusion of pharmacists in CPOE implementation teams is common, but many of these pharmacists have no substantive exposure to these systems and are poorly equipped to evaluate them or address the needs of other users as they arise. Because pharmacists often use components of these systems that no other health care professional sees, it is vital that pharmacists understand and can critically evaluate those components and the systems as a whole. Although several professional organizations have recognized the need for informatics to be a portion of the PharmD curriculum, this area of study has not yet caught up to the technology, and pharmacists are still entering the workforce ill prepared for the technological challenges that await them. If we do not catch the technology wave now, we may find ourselves washed away by it in the future.