Heidi Kramer

Non-expert interpretations of hurricane forecast uncertainty visualizations

Uncertainty represented in visualizations is often ignored or misunderstood by the non-expert user. The National Hurricane Center displays hurricane forecasts using a track forecast cone, depicting the expected track of the storm and the uncertainty in the forecast. Our goal was to test whether different graphical displays of a hurricane forecast containing uncertainty would influence a decision about storm characteristics. Participants viewed one of five different visualization types. Three varied the currently used forecast cone, one presented a track with no uncertainty, and one presented an ensemble of multiple possible hurricane tracks. Results show that individuals make different decisions using uncertainty visualizations with different visual properties, demonstrating that basic visual properties must be considered in visualization design and communication.

Checking the lists: A systematic review of electronic checklist use in health care.

OBJECTIVE We conducted a literature search to examine the effects and experiences surrounding the transition from paper to electronic checklists in healthcare settings. We explore the types of electronic checklists being used in health care, how and where they were evaluated and seek to identify the successes and failures of using electronic checklists in healthcare, including use of checklists to ensure completeness of documentation in the electronic medical record. BACKGROUND Formalized checklist use as a memory and decision aid in aviation has resulted in significant increases in safety in that domain. Checklists have also been successfully introduced to reduce errors in some areas of healthcare; however, in some contexts checklists failed to provide some of the expected benefits. Adapting and integrating checklists electronically into the healthcare workflow provides opportunities and challenges that need to be better understood to make checklist adoption a success in health care. METHOD We conducted a literature search of the English language literature in MEDLINE using PubMed for peer-reviewed literature of implementation and use of electronic or computerized checklists related to clinical or healthcare use. We reviewed the studies and included in this review those papers that discussed in depth the development process and that conducted controlled studies to assess the effectiveness of checklists and the evaluation of their acceptance in the clinical context. RESULTS The literature search using the keywords electronic checklist OR computerized checklist returned a total of 23 peer-reviewed papers. Out of these 15 were included in the review, with 8 excluded because they did not evaluate checklist use for patient care. APPLICATION More rigorous application of known principles and methods from Human Computer Interaction research and the behavioral sciences can provide a clearer, more comprehensive understanding of the conditions that affect the development and use of checklists.

High Prevalence of Medication Discrepancies Between Home Health Referrals and Centers for Medicare and Medicaid Services Home Health Certification and Plan of Care and Their Potential to Affect Safety of Vulnerable Elderly Adults

To describe the prevalence of discrepancies between medication lists that referring providers and home healthcare (HH) nurses create.

The influence of different graphical displays on nonexpert decision making under uncertainty.

Understanding how people interpret and use visually presented uncertainty data is an important yet seldom studied aspect of data visualization applications. Current approaches in visualization often display uncertainty as an additional data attribute without a well-defined context. Our goal was to test whether different graphical displays (glyphs) would influence a decision about which of 2 weather forecasts was a more accurate predictor of an uncertain temperature forecast value. We used a statistical inference task based on fictional univariate normal distributions, each characterized by a mean and standard deviation. Participants viewed 1 of 5 different glyph types representing 2 weather forecast distributions. Three of these used variations in spatial encoding to communicate the distributions and the other 2 used nonspatial encoding (brightness or color). Four distribution pairs were created with different relative standard deviations (uncertainty of the forecasts). We found that there was a difference in how decisions were made with spatial versus nonspatial glyphs, but no difference among the spatial glyphs themselves. Furthermore, the effect of different glyph types changed as a function of the variability of the distributions. The results are discussed in the context of how visualizations might improve decision making under uncertainty.

Evaluation of an Electronic Module for Reconciling Medications in Home Health Plans of Care

OBJECTIVES Transitions in patient care pose an increased risk to patient safety. One way to reduce this risk is to ensure accurate medication reconciliation during the transition. Here we present an evaluation of an electronic medication reconciliation module we developed to reduce the transition risk in patients referred for home healthcare. METHODS Nineteen physicians with experience in managing home health referrals were recruited to participate in this within-subjects experiment. Participants completed medication reconciliation for three clinical cases in each of two conditions. The first condition (paper-based) simulated current practice - reconciling medication discrepancies between a paper plan of care (CMS 485) and a simulated Electronic Health Record (EHR). For the second condition (electronic) participants used our medication reconciliation module, which we integrated into the simulated EHR. To evaluate the effectiveness of our medication reconciliation module, we employed repeated measures ANOVA to test the hypotheses that the module will: 1) Improve accuracy by reducing the number of unaddressed medication discrepancies, 2) Improve efficiency by reducing the reconciliation time, 3) have good perceived usability. RESULTS The improved accuracy hypothesis is supported. Participants left more discrepancies unaddressed in the paper-based condition than the electronic condition, F (1,1) = 22.3, p < 0.0001 (Paper Mean = 1.55, SD = 1.20; Electronic Mean = 0.45, SD = 0.65). However, contrary to our efficiency hypothesis, participants took the same amount of time to complete cases in the two conditions, F (1, 1) =0.007, p = 0.93 (Paper Mean = 258.7 seconds, SD = 124.4; Electronic Mean = 260.4 seconds, SD = 158.9). The usability hypothesis is supported by a composite mean ability and confidence score of 6.41 on a 7-point scale, 17 of 19 participants preferring the electronic system and an SUS rating of 86.5. CONCLUSION We present the evaluation of an electronic medication reconciliation module that increases detection and resolution of medication discrepancies compared to a paper-based process. Further work to integrate medication reconciliation within an electronic medical record is warranted.

The Effects of Interruption Context on Task Performance

This study examines the effects of contextual similarity or dissimilarity of interruptions during task performance. Participants engaged in a series of working memory tasks using a computer interface. While performing these tasks they were intermittently interrupted and required to perform a different task. Each participant was interrupted with four different tasks that varied in context from identically related information given during the primary task to unrelated contextual information. Performance was assessed based on the total number of errors during each task before and after an interruption. Results revealed an increase in error rate when a participant was interrupted with a contextually dissimilar task compared to the primary task. These findings suggest that a person who experiences an interruption by a task with contextually identical information to the primary task will make fewer errors following the completion of the interruption compared to a task with contextually dissimilar information.

Does the Shoe Fit? Applying Lessons Learned in Aviation to Healthcare

Aviation’s successful use of Decision Support Systems (DSS) has not been replicated in the healthcare subset of DSS referenced as Clinical Decision Support (CDS). Here the domains of healthcare and aviation are compared and contrasted providing an overview of the adaptation of lessons learned in aviation to healthcare. We propose there are differences in characteristics inherent to the contexts of aviation and healthcare that affect the data necessary for efficient, effective CDS systems. Specifically, ten context characteristics are discussed that jointly and separately affect the availability, quantity, quality and temporal relevance of the data. By providing remedies for overcoming deficiencies and supporting accurate representation of the data perhaps then CDS systems will meet their potential for improved adoption, user satisfaction and patient outcomes.

Using Salience to Guide Searches in Digital Displays

Technological innovations including software applications and the internet have increased the ability to collect and store information. However, the increasing amounts of information are used by operators who are constrained by their limited cognitive processing capacity. One common approach to improve the search for information in digital displays is the concept of increased stimulus-driven salience to guide the user. To examine search efficiency, sufficiency and decision accuracy we conducted an experiment to explore the effect of the congruence of salience and relevance and the effect of salience manipulation (position or size). Our study adapted the Wason selection task and manipulated salience by size in a tag cloud and position in a list. The results show that increased salience of relevant items does not insure an optimal search. Care should be taken to match salience with relevance; however, increased salience should not be expected to promote a sufficient search with increased decision accuracy.

Structuring the Puzzle: Protocol Use in Disease Outbreak Investigations

Disease outbreaks affect millions of Americans every year and have potentially large health and financial costs. State and local health departments in conjunction with the CDC have the responsibility for investigating and managing disease outbreaks. Individual states define dozens of diseases as reportable. To manage the scope and diversity of these diseases public health agencies rely on the use of protocols. Protocols can be effective tools for improving performance and reducing errors. However, there are limitations with the use of protocols in natural systems such as disease investigations. Our study included 41 semi-structured interviews with public health disease outbreak investigators. This paper discusses some of the benefits and limitations related to the use of protocols in disease outbreak investigations. We also suggest areas where Human Factors Engineering can support public health disease investigations.