Kathleen Adams

Design of Admission Medication Reconciliation Technology: A Human Factors Approach to Requirements and Prototyping

OBJECTIVE: Our objectives were to (1) develop an in-depth understanding of the workflow and information flow in medication reconciliation, and (2) design medication reconciliation support technology using a combination of rapid-cycle prototyping and human-centered design. BACKGROUND: Although medication reconciliation is a national patient safety goal, limitations both of physical environment and in workflow can make it challenging to implement durable systems. We used several human factors techniques to gather requirements and develop a new process to collect a medication history at hospital admission. METHODS: We completed an ethnography and time and motion analysis of pharmacists in order to illustrate the processes used to reconcile medications. We then used the requirements to design prototype multimedia software for collecting a bedside medication history. We observed how pharmacists incorporated the technology into their physical environment and documented usability issues. RESULTS: Admissions occurred in three phases: (1) list compilation, (2) order processing, and (3) team coordination. Current medication reconciliation processes at the hospital average 19 minutes to complete and do not include a bedside interview. Use of our technology during a bedside interview required an average of 29 minutes. The software represents a viable proof-of-concept to automate parts of history collection and enhance patient communication. However, we discovered several usability issues that require attention. CONCLUSIONS: We designed a patient-centered technology to enhance how clinicians collect a patients medication history. By using multiple human factors methods, our research team identified system themes and design constraints that influence the quality of the medication reconciliation process and implementation effectiveness of new technology.

Evaluation of Multimedia Medication Reconciliation Software: A Randomized Controlled, Single-Blind Trial to Measure Diagnostic Accuracy for Discrepancy Detection

BACKGROUND The Veterans Affairs Portland Healthcare System developed a medication history collection software that displays prescription names and medication images. OBJECTIVE This article measures the frequency of medication discrepancy reporting using the medication history collection software and compares with the frequency of reporting using a paper-based process. This article also determines the accuracy of each method by comparing both strategies to a best possible medication history. STUDY DESIGN Randomized, controlled, single-blind trial. SETTING Three community-based primary care clinics associated with the Veterans Affairs Portland Healthcare System: a 300-bed teaching facility and ambulatory care network serving Veteran soldiers in the Pacific Northwest United States. PARTICIPANTS Of 212 patients with primary care appointments, 209 patients fulfilled the study requirements. INTERVENTION Patients randomized to a software-directed medication history or a paper-based medication history. Randomization and allocation to treatment groups were performed using a computer-based random number generator. Assignments were placed in a sealed envelope and opened after participant consent. The research coordinator did not know or have access to the treatment assignment until the time of presentation. MAIN OUTCOME MEASURES The primary analysis compared the discrepancy detection rates between groups with respect to the health record and a best possible medication history. RESULTS Of 3,500 medications reviewed, we detected 1,435 discrepancies. Forty-six percent of those discrepancies were potentially high risk for causing an adverse drug event. There was no difference in detection rates between treatment arms. Software sensitivity was 83% and specificity was 91%; paper sensitivity was 81% and specificity was 94%. No participants were lost to follow-up. CONCLUSION The medication history collection software is an efficient and scalable method for gathering a medication history and detecting high-risk discrepancies. Although it included medication images, the technology did not improve accuracy over a paper list when compared with a best possible medication history. TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT02135731.

Application of the Cognitive-Socio-Technical Framework to Usability Evaluation: A Case Study of Patient Controlled Analgesia Order Sets

Simulations with representative end-users have been heralded as a strong practice for evaluating the quality of interface designs and predicting performance of human-computer interactions. Given that medical contexts are high-stakes complex adaptive systems, it is valuable to utilize a theoretical framework to organize testing and a suite of low-cost test methods that can be used modularly as time and resources dictate. We used the Cognitive-Socio-Technical Framework to organize our study and integrate usability findings of the Patient Controlled Analgesia (PCA) order sets. PCAs enable patients to self-administer intravenous opioids within defined dose and frequency thresholds. If not ordered properly, PCAs carry risk for causing delirium, hypotension, respiratory depression, and death. In 2015, our patient safety office received reports of several adverse events associated with orders that failed to correctly estimate patient needs within the context of patient comorbidities. In response, the safety office partnered with an interdisciplinary team of quality improvement specialists, informaticians, and clinical stakeholders to design new order sets with delineated care pathways and medication dosing recommendations. They completed a root cause analysis, and reviewed the evidencebased literature. The findings identified cognitive and sociotechnical barriers that included knowledge gaps, process variations, and design flaws. The order sets had to help providers estimate opiate needs and provide reasonable starting doses for continuous and demand opiate prescriptions. This information was used to develop a set of proposed design solutions that included organization of care pathways around patient “archetypes”, bundling orders based upon context, including dose and range defaults, and making a pharmacy consult available. The informaticians then generated serviceable prototypes of the order sets that interlaced functional requirements with published design standards and usability heuristics. To test the computerized decision support (CDS) and surface issues, the informatics team conducted a formative usability evaluation. The protocol consisted of a low-cost agglomeration of three techniques: 1) user simulations accompanied by a “Think-Aloud” process, 2) our Agile Task Analysis (ATA) that combines screenshots with elements of an engineer’s hierarchical task analysis embedded in an ethnographic field log, and 3) semi-structured debriefing interviews with test subjects. We recruited a convenience sample of n=9 first year surgical interns to complete a series of use cases of gradually increasing complexity. After every two participants, developers effected menu design revisions, informed by the usability findings. Major components included: 1) shifting the burden of complexity from the intended user to the development team, 2) creating automatic deployment of pharmacy and palliative care consults upon identification of complex patients, and 3) limiting ordering choices to recommended guidelines. The average task performance increased from 0.31 to 0.88. Therefore, the order set redesign was deemed safer and stable enough to deploy with the understanding that individual task comprehension and ordering performance varies and will need to be monitored.