Alissa L. Russ

Usability evaluation of a medication reconciliation tool: Embedding safety probes to assess users’ detection of medication discrepancies

OBJECTIVEnAn electronic medication reconciliation tool was previously developed by another research team to aid provider-patient communication for medication reconciliation. To evaluate the usability of this tool, we integrated artificial safety probes into standard usability methods. The objective of this article is to describe this method of using safety probes, which enabled us to evaluate how well the tool supports users detection of medication discrepancies.nnnMATERIALS AND METHODSnWe completed a mixed-method usability evaluation in a simulated setting with 30 participants: 20 healthcare professionals (HCPs) and 10 patients. We used factual scenarios but embedded three artificial safety probes: (1) a missing medication (i.e., omission); (2) an extraneous medication (i.e., commission); and (3) an inaccurate dose (i.e., dose discrepancy). We measured users detection of each probe to estimate the probability that a HCP or patient would detect these discrepancies. Additionally, we recorded participants detection of naturally occurring discrepancies.nnnRESULTSnEach safety probe was detected by ≤50% of HCPs. Patients detection rates were generally higher. Estimates indicate that a HCP and patient, together, would detect 44.8% of these medication discrepancies. Additionally, HCPs and patients detected 25 and 45 naturally-occurring discrepancies, respectively.nnnDISCUSSIONnOverall, detection of medication discrepancies was low. Findings indicate that more advanced interface designs are warranted. Future research is needed on how technologies can be designed to better aid HCPs and patients detection of medication discrepancies.nnnCONCLUSIONnThis is one of the first studies to evaluate the usability of a collaborative medication reconciliation tool and assess HCPs and patients detection of medication discrepancies. Results demonstrate that embedded safety probes can enhance standard usability methods by measuring additional, clinically-focused usability outcomes. The novel safety probes we used may serve as an initial, standard set for future medication reconciliation research. More prevalent use of safety probes could strengthen usability research for a variety of health information technologies.

Design concepts to support management of outpatient consultations in the Veterans Health Administration

This poster describes a project to improve understanding of the challenges associated with managing consultations in the Veterans Health Administration (VHA). We conducted interviews and observations with primary care providers and specialists at two VHA facilities. Using qualitative analysis, we identified cognitive requirements, challenges associated with each, and design seeds. During the poster session, we will present design concepts exploring interventions to support management of consultations.

Evaluation of Cognitive Encoding Across Different Medication Alert Designs

As part of the Meaningful Use criteria, electronic health record systems are required to include medication alerts to warn prescribers about medication allergies and drug-drug interactions before prescriptions are dispensed to patients. There is a paucity of research on medication alert design. Additionally, studies have not evaluated how well alert designs support prescribers’ cognitive encoding, even though this is a key step in human information processing for warnings. The objective of this study was to develop a methodological protocol for assessing prescribers’ encoding of medication alerts and evaluate prescribers’ cognitive encoding in response to two different alert designs. We hypothesized that a redesigned alert display that incorporates human factors principles would significantly increase the amount of information that is accurately encoded. A counterbalanced, crossover study was conducted with 20 prescribers in a human-computer interaction laboratory. We measured prescribers’ free recall as well as their ability to identify that three warning messages were displayed. The proportion of total data elements that prescribers were able to accurately recall was significantly greater for the redesigned versus original alerts (median difference in Original-Redesign = -.09, IQR = .15, Wilcoxon signed-rank test p = .006). Additionally, with the redesigned alerts, more prescribers accurately reported that three warnings were displayed (Exact McNemar’s test p = .002). Study methods may be useful for evaluating cognitive encoding in the healthcare domain and results may inform future medication alert designs.