William A. Pike

The science of interaction

There is a growing recognition within the visual analytics community that interaction and inquiry are inextricable. It is through the interactive manipulation of a visual interface–the analytic discourse–that knowledge is constructed, tested, refined and shared. This article reflects on the interaction challenges raised in the visual analytics research and development agenda and further explores the relationship between interaction and cognition. It identifies recent exemplars of visual analytics research that have made substantive progress toward the goals of a true science of interaction, which must include theories and testable premises about the most appropriate mechanisms for human–information interaction. Seven areas for further work are highlighted as those among the highest priorities for the next 5 years of visual analytics research: ubiquitous, embodied interaction; capturing user intentionality; knowledge-based interfaces; collaboration; principles of design and perception; interoperability; and interaction evaluation. Ultimately, the goal of a science of interaction is to support the visual analytics and human–computer interaction communities through the recognition and implementation of best practices in the representation and manipulation of visual displays.

A Situated Knowledge Representation of Geographical Information

In this paper we present an approach to conceiving of, constructing and comparing the concepts developed and used by geographers, environmental scientists and other earth science researchers to help describe, analyze and ultimately understand their subject of study. Our approach is informed by the situations under which concepts are conceived and applied, captures details of their construction, use and evolution and supports their ultimate sharing along with the means for deep exploration of conceptual similarities and differences that may arise among a distributed network of researchers. The intent here is to support different perspectives onto GIS resources that researchers may legitimately take, and to capture and compute with aspects of epistemology, to complement the ontologies that are currently receiving much attention in the GIScience community.

Beyond ontologies: Toward situated representations of scientific knowledge

In information systems that support knowledge-discovery applications such as scientific exploration, reliance on highly structured ontologies as data-organization aids can be limiting. With current computational aids to science work, the human knowledge that creates meaning out of analyses is often only recorded when work reaches publication-or worse, left unrecorded altogether-for lack of an ontological model for scientific concepts that can capture knowledge as it is created and used. We argue for an approach to representing scientific concepts that reflects (1) the situated processes of science work, (2) the social construction of knowledge, and (3) the emergence and evolution of understanding over time. In this model, knowledge is the result of collaboration, negotiation, and manipulation by teams of researchers. Capturing the situations in which knowledge is created and used helps these collaborators discover areas of agreement and discord, while allowing individual inquirers to maintain different perspectives on the same information. The capture of provenance information allows historical trails of reasoning to be reconstructed, allowing end users to evaluate the utility and trustworthiness of knowledge representations. We present a proof-of-concept system, called Codex, based on this situated knowledge model. Codex supports visualization of knowledge structures through concept mapping, and enables inference across those structures. The proof-of-concept is deployed in the domain of geoscience to support distributed teams of learners and researchers.

Connecting GEON: Making sense of the myriad resources, researchers and concepts that comprise a geoscience cyberinfrastructure

Simply placing electronic geoscience resources such as datasets, methods, ontologies, workflows and articles in a digital library or cyberinfrastructure does not mean that they will be used successfully by other researchers or educators. It is also necessary to provide the means to locate potentially useful content, and to understand it. Without suitable provision for these needs, many useful resources will go undiscovered, or else will be found but used inappropriately. In this article, we describe an approach to discovering, describing and understanding e-resources based on the notion that meaning is carried in the interconnections between resources and the actors in the cyberinfrastructure (including individuals, groups, organizations), as well as by ontologies and conventional metadata. Navigation around this universe is achieved by implementing the idea of perspectives as dynamic, conceptual views (defined by SPARQL-like queries against an OWL schema) that not only act as filters, but also dynamically promote and demote concepts, relationships and properties according to their immediate relevance. We describe a means to represent a wide variety of interactions between resources using the notion of a knowledge nexus, and we illustrate its use with resources and actors from the Geosciences Network (GEON) cyberinfrastructure community. We also closely link browsing and visualizing strategies to our nexus, drawing on ideas from semiotics to move resources and connections not currently of interest from the foreground to the background, and vice versa, using a new form of adaptive perspective. We illustrate our ideas via ConceptVista, an open-source concept mapping application that provides rich, visual depictions of the resources, cyber-community and myriad connections between them. Examples are presented that show how geoscientific knowledge can be explored not only via ontological structure, but also by use cases, social networks, citation graphs and organization charts; all of which may carry some aspects of meaning for the user.

Real-time visualization of network behaviors for situational awareness

Plentiful, complex, and dynamic data make understanding the state of an enterprise network difficult. Although visualization can help analysts understand baseline behaviors in network traffic and identify off-normal events, visual analysis systems often do not scale well to operational data volumes (in the hundreds of millions to billions of transactions per day) nor to analysis of emergent trends in real-time data. We present a system that combines multiple, complementary visualization techniques coupled with in-stream analytics, behavioral modeling of network actors, and a high-throughput processing platform called MeDICi. This system provides situational understanding of real-time network activity to help analysts take proactive response steps. We have developed these techniques using requirements gathered from the government users for which the tools are being developed. By linking multiple visualization tools to a streaming analytic pipeline, and designing each tool to support a particular kind of analysis (from high-level awareness to detailed investigation), analysts can understand the behavior of a network across multiple levels of abstraction.

The Scalable Reasoning System: Lightweight visualization for distributed analytics

A central challenge in visual analytics is the creation of accessible, widely distributable analysis applications that bring the benefits of visual discovery to as broad a user base as possible. Moreover, to support the role of visualization in the knowledge creation process, it is advantageous to allow users to describe the reasoning strategies they employ while interacting with analytic environments. We introduce an application suite called the scalable reasoning system (SRS), which provides Web-based and mobile interfaces for visual analysis. The service-oriented analytic framework that underlies SRS provides a platform for deploying pervasive visual analytic environments across an enterprise. SRS represents a ldquolightweightrdquo approach to visual analytics whereby thin client analytic applications can be rapidly deployed in a platform-agnostic fashion. Client applications support multiple coordinated views while giving analysts the ability to record evidence, assumptions, hypotheses and other reasoning artifacts. We describe the capabilities of SRS in the context of a real-world deployment at a regional law enforcement organization.

Scalable visual reasoning: Supporting collaboration through distributed analysis

We present a visualization environment called the scalable reasoning system (SRS) that provides a suite of tools for the collection, analysis, and dissemination of reasoning products. This environment is designed to function across multiple platforms, bringing the display of visual information and the capture of reasoning associated with that information to both mobile and desktop clients. The service-oriented architecture of SRS facilitates collaboration and interaction between users regardless of their location or platform. Visualization services allow data processing to be centralized and analysis results to be collected from distributed clients in real time. We use the concept of ldquoreasoning artifactsrdquo to capture the analytic value attached to individual pieces of information and collections thereof, helping to fuse the foraging and sense-making loops in information analysis. Reasoning structures composed of these artifacts can be shared across platforms while maintaining references to the analytic activity (such as interactive visualization) that produced them.

Real Time Scalable Visual Analysis on Mobile Devices

Interactive visual presentation of information can help an analyst gain faster and better insight from data. When combined with situational or context information, visualization on mobile devices is invaluable to in-field responders and investigators. However, several challenges are posed by the form-factor of mobile devices in developing such systems. In this paper, we classify these challenges into two broad categories - issues in general mobile computing and issues specific to visual analysis on mobile devices. Using NetworkVis and Infostar as example systems, we illustrate some of the techniques that we employed to overcome many of the identified challenges. NetworkVis is an OpenVG-based real-time network monitoring and visualization system developed for Windows Mobile devices. Infostar is a flash-based interactive, real-time visualization application intended to provide attendees access to conference information. Linked time-synchronous visualization, stylus/button-based interactivity, vector graphics, overview-context techniques, details-on-demand and statistical information display are some of the highlights of these applications.

The scalable reasoning system: lightweight visualization for distributed analytics

A central challenge in visual analytics is the creation of accessible, widely distributable analysis applications that bring the benefits of visual discovery to as broad a user base as possible. Moreover, to support the role of visualization in the knowledge creation process, it is advantageous to allow users to describe the reasoning strategies they employ while interacting with analytic environments. We introduce an application suite called the scalable reasoning system (SRS), which provides web-based and mobile interfaces for visual analysis. The service-oriented analytic framework that underlies SRS provides a platform for deploying pervasive visual analytic environments across an enterprise. SRS represents a ‘lightweight’ approach to visual analytics whereby thin client analytic applications can be rapidly deployed in a platform-agnostic fashion. Client applications support multiple coordinated views while giving analysts the ability to record evidence, assumptions, hypotheses and other reasoning artifacts. We describe the capabilities of SRS in the context of a real-world deployment at a regional law enforcement organization.

Putting Security in Context: Visual Correlation of Network Activity with Real-World Information

To effectively identify and respond to cyber threats, computer security analysts must understand the scale, motivation, methods, source, and target of an attack. Central to developing this situational awareness is the analyst’s world knowledge that puts these attributes in context. What known exploits or new vulnerabilities might an anomalous traffic pattern suggest? What organizational, social, or geopolitical events help forecast or explain attacks and anomalies? Few visualization tools support creating, maintaining, and applying this knowledge of the threat landscape. Through a series of formative workshops with practicing security analysts, we have developed a visualization approach inspired by the human process of contextualization; this system, called NUANCE, creates evolving behavioral models of network actors at organizational and regional levels, continuously monitors external textual information sources for themes that indicate security threats, and automatically determines if behavior indicative of those threats is present on a network.