Michael Jenkins

Comparing Uncertainty Visualizations for a Dynamic Decision-Making Task:

Supporting complex decision making requires conveying relevant information characteristics or qualifiers. The authors tested transparency and numeric annotation for displaying uncertainty about object identity. Participants performed a “missile defense” game in which they decided whether to destroy moving objects (which were either threatening missiles or nonthreatening birds and planes) before they reached a city. Participants were provided with uncertain information about the objects’ classifica-tions. Uncertainty was represented through the transparency of icons representing the objects and/or with numeric annotations. Three display methods were created. Icons represented the most likely object classification (with solid icons), the most likely object classification (with icons whose transparency represented the level of uncertainty), or the probability that the icon was a missile (with transparency). In a fourth condition, participants could choose among the representations. Icons either were or were not annotated with numeric probability labels. Task performance was highest when participants could toggle the displays, with little effect of numeric annotation. In conditions in which probabilities were available graphically or numerically, participants chose to engage objects when they were farther from the city and had a lower probability of being a missile. Results provided continued support for the use of graphical uncertainty representations, even when numeric representations are present.

Towards context-aware hard/soft information fusion: Incorporating situationally qualified human observations into a fusion process for intelligence analysis

This paper describes a methodology for incorporating human observations into a hard+soft information fusion process for counterinsurgency intelligence analysis. The goal of incorporating human observations into the information fusion process is important as it extends the ability of the fusion algorithms to associate and merge disparate pieces of information by allowing for information collected from soft data sources (e.g., human observations) to be included in the process along with information collected from hard data sources (e.g., radar sensors). This goal is accomplished through the employment of fuzzy membership functions used in similarity scoring, for data association and situation assessment. These membership functions are based on situationally qualified error characteristics. Error characteristics represent the key to this process by allowing for accurate uncertainty alignment based on the known and/or unknown state of context dependent variables that have been empirically determined to influence the accuracy of human estimation for a given category- in this case human age estimation.

Towards context aware data fusion: Modeling and integration of situationally qualified human observations to manage uncertainty in a hard+soft fusion process

This paper presents a framework for characterizing errors associated with different categories of human observation combined with a method for integrating these into a hard+soft data fusion system. Error characteristics of human observers (often referred to as soft data sensors) have typically been artificially generated and lack contextual considerations that in a real-world application can drastically change the accuracy and precision of these characteristics. The proposed framework and method relies on error values that change based upon known and unknown states of qualifying variables empirically shown to affect observation accuracy under different contexts. This approach allows fusion systems to perform uncertainty alignment on data coming from human observers. The preprocessed data yields a more complete and reliable situation assessment when it is processed by data association and stochastic graph matching algorithms. This paper also provides an approach and results of initial validation testing of the proposed methodology. The testing approach leverages error characterization models for several exemplar categories of observation in combination with simulated synthetic data. Results have shown significant performance improvements with respect to both data association and situation assessment fusion processes with an average F-measure improvement of 0.16 and 0.20 for data association and situation assessment respectively. These F-measure improvements are representative of fewer incorrect and missed associations and fewer graph matching results, which then must be considered by human analysts. These benefits are expected to translate into a reduction of the overall cognitive workload facing human analysts in situations where they are tasked with developing and maintaining situational awareness.

Are Behavioral Measures Useful for Detecting Cognitive Workload During Human-Computer Interaction?

Commonly used techniques for measuring cognitive workload during human-computer interactions can be cumbersome or intrusive to task performance. In the current work, we examine the utility of heuristic behavior analysis, including keystroke dynamics, mouse tracking, and body positioning for measuring cognitive workload during direct interactions between humans and computers. We present a method for modeling behavioral measures as well as physiological and neurophysiological data using probabilistic, statistical, and machine learning algorithms for real-time estimation of human states. We believe this discussion will inform the capability to provide estimates of cognitive workload in real-world scenarios.

Towards a connected bicycle to communicate with vehicles and infrastructure : Multimodel alerting interface with Networked Short-Range Transmissions (MAIN-ST)

The Connected Vehicles program is a multimodal US Department of Transportation (USDOT) initiative that enables safer, smarter, and greener surface transportation using dedicated wireless communication technology. Although significant efforts are being made to bring motor vehicles and transportation infrastructure onto this connected network, bicycles have been largely overlooked. Bringing cyclists onto this network will enable other connected vehicles and infrastructure to be aware of their presence, and allow cyclists to take advantage of the safety and transportation benefits of receiving information from other connected entities. To connect bicycles, we are designing a prototype Multimodal Alerting Interface with Networked Short-Range Transmissions (MAIN-ST). MAIN-ST brings cyclists onto the vehicle-to-vehicle (V2V) and vehicle-to-infrastructure (V2I) networks to enable a suite of safe cycling capabilities. This paper describes our progress accomplished over a 6-month period, and documents the feasibility of the MAIN-ST technology approach.

Linguistic Estimations of Human Attributes

Qualitative linguistic data provides unique, valuable information that can only come from human observers. Data fusion systems find it challenging to incorporate this “soft data” as they are primarily designed to analyze quantitative, hard-sensor data with consistent formats and qualified error characteristics. This research investigates how people produce linguistic descriptions of human physical attributes. Thirty participants were asked to describe seven actors’ ages, heights, and weights in two naturalistic video scenes, using both numeric estimates and linguistic descriptors. Results showed that not only were a large number of linguistic descriptors used, but they were also used inconsistently. Only 10% of the 189 unique terms produced were used by four or more participants. Especially for height and weight, we found that linguistic terms are poor devices for transmitting estimated values due to the large and overlapping ranges of numeric estimates associated with each term. Future work should attempt to better define the boundaries of inclusion for more frequently used terms and to create a controlled language lexicon to gauge whether or not that improves the precision of natural language terms.

Perception of Meta-Information Representation: A Psychophysical Approach

Previous research has identified many effective methods to visualize different types of meta-information, or information qualifiers; however, these methods are often incorporated without understanding how the graphical codes are perceived and how the encoded information is interpreted by display users. This results in display designers selecting graphical codes to represent meta-information without empirical evidence to determine the appropriateness of these selections. To help address this lack of guidance, this paper presents a systematic study of how people perceive two graphical codes (saturation and opacity) and relate those codes to different types of meta-information. Results were generated using psychophysical scaling methods, and provide visualization designers with a means to more appropriately design meta-information representations.

Evaluating the Creation and Interpretation of Causal Influence Models

Bayesian networks (BNs) are probabilistic models frequently used to capture domain knowledge for use in computational systems that can reason about states, causes, and effects. While BNs have many advantages, their complexity can hamper the process of knowledge elicitation and encoding. First, domain experts may not have expertise in artificial reasoning or probabilistic models, and that lack of understanding may complicate the elicitation of probabilities relevant to BN model structure. In addition, BNs require the definition of a priori, conditional probabilities: for complex models, this requires eliciting large numbers of complex probabilities. Multiple “canonical modeling” approaches, such as Causal Influence Models (CIMs), have been developed to address these complexities. However, little progress has been made towards human-in-the-loop evaluation of such approaches - specifically, their accessibility and usability, their related user interfaces, and how they enable a user to correctly create and interpret variables and probabilistic relationships. In this study, we evaluated the CIM approach (implemented in a software application) to determine the effect on user task performance. Results indicate that the model complexity has an adverse effect on performance when users are interpreting an existing model; that semantics of a model may impact performance; and that users were generally successful in creating new models of different situations.

Augmented Reality and Mixed Reality Prototypes for Enhanced Mission Command/Battle Management Command and Control (BMC2) Execution

This work provides an overview of three prototype augmented reality (AR) applications developed for the Microsoft HoloLens with the intention of exploring the strengths and weaknesses of AR for supporting planning and decision making–specifically, within the domain of US Army Mission Command and Battle Management Command and Control (BMC2) execution. We present each prototype application accompanied by the target audience from whom we sought feedback, key features of the application, and technical goals we hoped to achieve, demonstrate, and evaluate. Findings with respect to AR strengths and weaknesses, framed around the technical goals of each of the AR applications, are then presented to begin to shed light on limitations and opportunities of the state-of-the-art in AR hardware, and the potential to support Mission Command and BMC2 stakeholders. This material is based upon work supported by the Communications-Electronics Research, Development and Engineering Center (CERDEC) under Contract No. W56KGU-18-C-0002. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of CERDEC.

Effects of Prolonged Use of Mixed Reality Systems in Occupational Settings Discussion Panel

Augmented or mixed reality devices overlay computer-generated sensory information that alters one’s current view of the real world. Gaming, military, and instructional applications are fairly prevalent, however, industrial applications are still in their infancy despite this dramatic increase in commercial products. Being a novel application to industry, designers of these systems have focused on function; but have only given a cursory look towards user population safety, ergonomic risk hazards, and long-term exposure concerns. How should designers design for human safety use concerns while maintaining system function? Besides form and function what, if any, safety considerations should public consumers and industries contemplate before buying commercial ready off-the-shelf systems? Ultimately, how do we use ergonomics assessments to better design, assess, and demonstrate business case use of mixed reality devices to benefit labor-intensive occupational tasks? The session will start with initial lectures and introductions from the panel, followed by an encouraged panel discussion with the audience led by the moderators. Moderators: Gregory Garrett, chair & Christopher Reid, co-chair