Maria Riveiro

Evaluation of uncertainty visualization techniques for information fusion

This paper highlights the importance of uncertainty visualization in information fusion, reviews general methods of representing uncertainty and presents perceptual and cognitive principles from Tufte, Chambers and Bertin as well as users experiments documented in the literature. Examples of uncertainty representations in information fusion are analyzed using these general theories. These principles can be used in future theoretical evaluations of existing or newly developed uncertainty visualization techniques before usability testing with actual users.

Visual Analytics for the Detection of Anomalous Maritime Behavior

The surveillance of large sea areas often generates huge amounts of multidimensional data. Exploring, analyzing and finding anomalous behavior within this data is a complex task. Confident decisions upon the abnormality of a particular vessel behavior require a certain level of situation awareness that may be difficult to achieve when the operator is overloaded by the available information. Based on a visual analytics process model, we present a novel system that supports the acquisition of situation awareness and the involvement of the user in the anomaly detection process using two layers of interactive visualizations. The system uses an interactive data mining module that supports the insertion of the users knowledge and experience in the creation, validation and continuous update of the normal model of the environment.

Evaluating the effect of visually represented geodata uncertainty on decision-making: systematic review, lessons learned, and recommendations

ABSTRACT For many years, uncertainty visualization has been a topic of research in several disparate fields, particularly in geographical visualization (geovisualization), information visualization, and scientific visualization. Multiple techniques have been proposed and implemented to visually depict uncertainty, but their evaluation has received less attention by the research community. In order to understand how uncertainty visualization influences reasoning and decision-making using spatial information in visual displays, this paper presents a comprehensive review of uncertainty visualization assessments from geovisualization and related fields. We systematically analyze characteristics of the studies under review, i.e., number of participants, tasks, evaluation metrics, etc. An extensive summary of findings with respect to the effects measured or the impact of different visualization techniques helps to identify commonalities and differences in the outcome. Based on this summary, we derive “lessons learned” and provide recommendations for carrying out evaluation of uncertainty visualizations. As a basis for systematic evaluation, we present a categorization of research foci related to evaluating the effects of uncertainty visualization on decision-making. By assigning the studies to categories, we identify gaps in the literature and suggest key research questions for the future. This paper is the second of two reviews on uncertainty visualization. It follows the first that covers the communication of uncertainty, to investigate the effects of uncertainty visualization on reasoning and decision-making.

Interactive Visualization of Normal Behavioral Models and Expert Rules for Maritime Anomaly Detection

Maritime surveillance systems analyze vast amounts of heterogeneous sensor data from a large number of objects. In order to support the operator while monitoring such systems, the identification of anomalous vessels or situations that might need further investigation may reduce the operators cognitive load. While it is worth acknowledging that many existing mining applications support identification of anomalous behavior, autonomous anomaly detection systems are rarely used in the real world, since the detection of anomalous behavior is normally not a well-defined problem and therefore, human expert knowledge is needed. This calls for the development of interaction components that can support the user in the detection process. In order to support the comprehension of the knowledge embedded in the system, we propose an interactive way of visualizing expert rules and normal behavioral models built from the data. The overall goal is to facilitate the validation and update of these models and signatures, supporting the insertion of human expert knowledge while improving confidence and trust in the system.

Effects of visualizing uncertainty on decision-making in a target identification scenario

Abstract This paper presents an empirical study that addresses the effects the visualization of uncertainty has on decision-making. We focus our investigations on an area where uncertainty plays an important role and the decision time is limited. For that, we selected an air defense scenario, where expert operators have a few minutes to make a well-informed decision based on uncertain sensor data regarding the identity of an object and where the consequences of a late or wrong decision are severe. An approach for uncertainty visualization is proposed and tested using a prototype that supports the interactive analysis of multivariate spatio-temporal sensor data. The uncertainty visualization embeds the accuracy of the sensor data values using the thickness of the lines in the graphical representation of the sensor values. Semi-transparent filled circles represent the uncertain position, while a track quality value between 0 and 1 accounts for the quality of the estimated track for each target. Twenty-two experienced air traffic operators were divided into two groups (with and without uncertainty visualization) and carried out identification and prioritization tasks using the prototype. The results show that the group aided by visualizations of uncertainty needed significantly fewer attempts to make a final identification, and a significant difference between the groups when considering the identities and priorities assigned was observed (participants with uncertainty visualization selected higher priority values and more hostile and suspect identities). These results may show that experts put themselves in the “worst-case scenario” in the presence of uncertainty when safety is an issue. Additionally, the presentation of uncertainty neither increased the participants׳ expressed workload, nor the time needed to make a classification. However, the inclusion of the uncertainty information did not have a significant effect on the performance (true positives, false negatives and false positives) or the participants׳ expressed confidence in their decisions.

Reasoning about anomalies: a study of the analytical process of detecting and identifying anomalous behavior in maritime traffic data

The goal of visual analytical tools is to support the analytical reasoning process, maximizing human perceptual, understanding and reasoning capabilities in complex and dynamic situations. Visual analytics software must be built upon an understanding of the reasoning process, since it must provide appropriate interactions that allow a true discourse with the information. In order to deepen our understanding of the human analytical process and guide developers in the creation of more efficient anomaly detection systems, this paper investigates how is the human analytical process of detecting and identifying anomalous behavior in maritime traffic data. The main focus of this work is to capture the entire analysis process that an analyst goes through, from the raw data to the detection and identification of anomalous behavior. Three different sources are used in this study: a literature survey of the science of analytical reasoning, requirements specified by experts from organizations with interest in port security and user field studies conducted in different marine surveillance control centers. Furthermore, this study elaborates on how to support the human analytical process using data mining, visualization and interaction methods. The contribution of this paper is twofold: (1) within visual analytics, contribute to the science of analytical reasoning with practical understanding of users tasks in order to develop a taxonomy of interactions that support the analytical reasoning process and (2) within anomaly detection, facilitate the design of future anomaly detector systems when fully automatic approaches are not viable and human participation is needed.

The role of visualization and interaction in maritime anomaly detection

The surveillance of large sea, air or land areas normally involves the analysis of large volumes of heterogeneous data from multiple sources. Timely detection and identification of anomalous behavior or any threat activity is an important objective for enabling homeland security. While it is worth acknowledging that many existing mining applications support identification of anomalous behavior, autonomous anomaly detection systems for area surveillance are rarely used in the real world. We argue that such capabilities and applications present two critical challenges: (1) they need to provide adequate user support and (2) they need to involve the user in the underlying detection process. In order to encourage the use of anomaly detection capabilities in surveillance systems, this paper analyzes the challenges that existing anomaly detection and behavioral analysis approaches present regarding their use and maintenance by users. We analyze input parameters, detection process, model representation and outcomes. We discuss the role of visualization and interaction in the anomaly detection process. Practical examples from our current research within the maritime domain illustrate key aspects presented.

VISAD: an interactive and visual analytical tool for the detection of behavioral anomalies in maritime traffic data

Monitoring the surveillance of large sea areas normally involves the analysis of huge quantities of heterogeneous data from multiple sources (radars, cameras, automatic identification systems, reports, etc.). The rapid identification of anomalous behavior or any threat activity in the data is an important objective for enabling homeland security. While it is worth acknowledging that many existing mining applications support identification of anomalous behavior, autonomous anomaly detection systems are rarely used in the real world. There are two main reasons: (1) the detection of anomalous behavior is normally not a well-defined and structured problem and therefore, automatic data mining approaches do not work well and (2) the difficulties that these systems have regarding the representation and employment of the prior knowledge that the users bring to their tasks. In order to overcome these limitations, we believe that human involvement in the entire discovery process is crucial. Using a visual analytics process model as a framework, we present VISAD: an interactive, visual knowledge discovery tool for supporting the detection and identification of anomalous behavior in maritime traffic data. VISAD supports the insertion of human expert knowledge in (1) the preparation of the system, (2) the establishment of the normal picture and (3) in the actual detection of rare events. For each of these three modules, VISAD implements different layers of data mining, visualization and interaction techniques. Thus, the detection procedure becomes transparent to the user, which increases his/her confidence and trust in the system and overall, in the whole discovery process.

Evaluation of Normal Model Visualization for Anomaly Detection in Maritime Traffic

Monitoring dynamic objects in surveillance applications is normally a demanding activity for operators, not only because of the complexity and high dimensionality of the data but also because of other factors like time constraints and uncertainty. Timely detection of anomalous objects or situations that need further investigation may reduce operators’ cognitive load. Surveillance applications may include anomaly detection capabilities, but their use is not widespread, as they usually generate a high number of false alarms, they do not provide appropriate cognitive support for operators, and their outcomes can be difficult to comprehend and trust. Visual analytics can bridge the gap between computational and human approaches to detecting anomalous behavior in traffic data, making this process more transparent. As a step toward this goal of transparency, this article presents an evaluation that assesses whether visualizations of normal behavioral models of vessel traffic support two of the main analytical tasks specified during our field work in maritime control centers. The evaluation combines quantitative and qualitative usability assessments. The quantitative evaluation, which was carried out with a proof-of-concept prototype, reveals that participants who used the visualization of normal behavioral models outperformed the group that did not do so. The qualitative assessment shows that domain experts have a positive attitude toward the provision of automatic support and the visualization of normal behavioral models, as these aids may reduce reaction time and increase trust in and comprehensibility of the system.

Supporting the Analytical Reasoning Process in Maritime Anomaly Detection: Evaluation and Experimental Design

Despite the growing number of systems providing visual analytic support for investigative analysis, few empirical studies include investigations on the analytical reasoning process that needs to be supported. In this paper, we present an approach to evaluate the ability of certain visual representations from an integrated visual-computational environment to support the completion of representative tasks. The problem area studied is the detection and identification of anomalous vessels and situations while monitoring maritime traffic data. This paper presents: (1) a brief review of current evaluation methodologies within information visualization and visual analytics, (2) an analysis of operators analytical reasoning process (derived from field work in maritime control centers and a literature review on analytical reasoning theories), (3) a list of representative tasks for usability evaluation and (4) an approach to evaluate the use of normal behavioral models representations during the detection process.