James Schaffer

Effects of Information Availability on Command-and-Control Decision Making Performance, Trust, and Situation Awareness

Objective: We investigated how increases in task-relevant information affect human decision-making performance, situation awareness (SA), and trust in a simulated command-and-control (C2) environment. Background: Increased information is often associated with an improvement of SA and decision-making performance in networked organizations. However, previous research suggests that increasing information without considering the task relevance and the presentation can impair performance. Method: We used a simulated C2 task across two experiments. Experiment 1 varied the information volume provided to individual participants and measured the speed and accuracy of decision making for task performance. Experiment 2 varied information volume and information reliability provided to two participants acting in different roles and assessed decision-making performance, SA, and trust between the paired participants. Results: In both experiments, increased task-relevant information volume did not improve task performance. In Experiment 2, increased task-relevant information volume reduced self-reported SA and trust, and incorrect source reliability information led to poorer task performance and SA. Conclusion: These results indicate that increasing the volume of information, even when it is accurate and task relevant, is not necessarily beneficial to decision-making performance. Moreover, it may even be detrimental to SA and trust among team members. Application: Given the high volume of available and shared information and the safety-critical and time-sensitive nature of many decisions, these results have implications for training and system design in C2 domains. To avoid decrements to SA, interpersonal trust, and decision-making performance, information presentation within C2 systems must reflect human cognitive processing limits and capabilities.

Getting the Message?: A Study of Explanation Interfaces for Microblog Data Analysis

In many of todays online applications that facilitate data exploration, results from information filters such as recommender systems are displayed alongside traditional search tools. However, the effect of prediction algorithms on users who are performing open-ended data exploration tasks through a search interface is not well understood. This paper describes a study of three interface variations of a tool for analyzing commuter traffic anomalies in the San Francisco Bay Area. The system supports novel interaction between a prediction algorithm and a human analyst, and is designed to explore the boundaries, limitations and synergies of both. The degree of explanation of underlying data and algorithmic process was varied experimentally across each interface. The experiment (N=197) was performed to assess the impact of algorithm transparency/explanation on data analysis tasks in terms of search success, general insight into the underlying data set and user experience. Results show that 1) presence of recommendations in the user interface produced a significant improvement in recall of anomalies, 2) participants were able to detect anomalies in the data that were missed by the algorithm, 3) participants who used the prediction algorithm performed significantly better when estimating quantities in the data, and 4) participants in the most explanatory condition were the least biased by the algorithms predictions when estimating quantities.

User Preferences for Hybrid Explanations

Hybrid recommender systems combine several different sources of information to generate recommendations. These systems demonstrate improved accuracy compared to single-source recommendation strategies. However, hybrid recommendation strategies are inherently more complex than those that use a single source of information, and thus the process of explaining recommendations to users becomes more challenging. In this paper we describe a hybrid recommender system built on a probabilistic programming language, and discuss the benefits and challenges of explaining its recommendations to users. We perform a mixed model statistical analysis of user preferences for explanations in this system. Through an online user survey, we evaluate explanations for hybrid algorithms in a variety of text and visual, graph-based formats, that are either novel designs or derived from existing hybrid recommender systems.

A study of dynamic information display and decision-making in abstract trust games

Abstract User interfaces that display dynamic information have the ability to influence decision makers in networked settings where many individuals collaborate. To understand how varying levels of information support affects behavior (cooperation vs. defection) in a social dilemma, a user interface (UI) was developed and an online experiment (N=901) was conducted based on the iterated Diner’s Dilemma, a version of the n-player Prisoner’s Dilemma. There were 3 main findings: (1) as more UI support was given, participants became more likely to retaliate against defection than they were to initiate defection; (2) participant situation awareness (SA) increased as more UI support was given but decreased in the presence of forgiving co-actors; and (3) the need for UI support to make good decisions was diminished as co-actors became more likely to exploit. These results can inform the design of information support tools for collaborative settings.

Interactive interfaces for complex network analysis: An information credibility perspective

This paper discusses and evaluates the impact of visualization and interaction strategies for extracting quality information from data in complex networks such as microblogs. Two different approaches to interactive visual representations of data are discussed: an interactive node-link graph and a novel approach where content is separated into interactive lists based on data properties. To assess the two approaches in terms of information credibility, the TopicNets system is compared with “Fluo”, a novel system. An analysis scenario is performed through each system on a set of big data filtered from the Twitter message service. The exposure of content, trade-offs between algorithmic power and interaction complexity, methods for content filtering, and strategies for recommending new content are assessed for each system. Fluo is found to improve on TopicNets ability to efficiently find relevant content primarily by providing a more structured content view, however, TopicNets is more customizable and boasts features which are critical for an expert analyst. The paper concludes with general insights on interface design for information filtering systems to maximize perceived quality of information.

MxR Framework for Uncertainty Based Explanation for Uncovering Adversarial Behavior

Mixed Reality (MxR) technologies have previously been explored in military applications oriented towards supporting both individual situational awareness and team collaborations. Ongoing technological advances in MxR have expanded its potential usage by military analyst teams to view, digest, and evaluate information from multiple data sources for uncovering adversarial behavior. Towards facilitating improved situational awareness, MxR is a promising medium for explaining patterns in data to uncover vital information. By extension, analyst collaborations conducted using MxR may further enhance collaborative decision making techniques used in military settings. In general, explanations provide summarizations or descriptive information that supports conclusions, depending on the desired level of abstraction. However, explanations that summarize information may not always preserve the underlying uncertainty present in the data. This work proposes a fused reason-based explanation technique for MxR that may help bring clarity to data where patterns may be unexpected, potentially revealing adversarial behavior.

An analysis of student behavior in two massive open online courses

Massive open online courses (MOOCs) have high potential for improving education worldwide, but understanding of student behavior and situations is difficult to achieve in online settings. Network analytics and visualizations can assist instructors with supporting understanding of student behavior as courses unfold. In this work, we perform a visual comparative analysis of two different MOOC courses to analyze the impacts of course structure differences and demonstrate the benefits of visual network analysis in this context. We present several insights: (1) behavior features that are best for prediction of student attrition varied with course structure, (2) a large proportion (about 35%) of students never received a reply to their original post and this was correlated with an eventual dropout, and (3) students that received a reply to their original post were twice as likely to post again. We contribute several information visualizations of student network data and draw recommendations for MOOC instructors and designers of course systems.

PhysPix: instantaneous rigid body simulation of rasters

Modern physics engines process collisions by leveraging vector representations (e.g. Box2D or Open Dynamics Engine (ODE)), which means that artists who work with pixel-based 2D content must map their pixel drawings onto representations such as Delaunay triangulations [Shewchuk 1996]. Effects such as destruction then require remeshing, which can be onerous to perform at runtime. The alternative is pixel-perfect collision handling, but past games such as Worms! and Scorched Earth that use this approach have not attempted true rigid body simulations. We present PhysPix, a 2D rigid body simulation framework based on pixels. PhysPix allows 1) artist control over the exact boundaries used for objects in the simulation, 2) natural bitmap-based support for destruction, and 3) an intuitive painting interface for properties such as non-uniform weight distributions.