Susan S. Kirschenbaum

Turning pictures into numbers

We present a study of complex visualization usage by expert meteorological forecasters. We performed a protocol analysis and examined the types of visualizations they examined. We present evidence for how experts are able to make use of complex visualizations. Our findings suggest that users of complex visualizations create qualitative mental models from which they can then generate quantitative information. In order to build their qualitative mental models, forecasters integrated information across multiple visualizations and extracted primarily qualitative information from visualizations in a goal-directed manner. We discuss both theoretical and practical implications of this study.

Contending with complexity : Developing and using a scaled world in applied cognitive research

Scaled worlds preserve certain functional relationships of a complex task environment while paring away others. The functional relationships preserved are defined by the questions of interest to the researcher. Different scaled worlds of the same task may preserve and pare away different functional relationships. In this paper we use the example of Ned to discuss the use of scaled worlds in applied cognitive research. Ned is based on a detailed cognitive task analysis of submarine approach officers as they attempt to localize an enemy submarine hiding in deep water. For Ned we attempted to preserve the functional relationships inherent in the approach officers information environment while paring away other aspects of his task environment. Scaled worlds attempt to maintain the realism inherent in the preserved functional relationship while being tractable for the researcher and engaging to the participant.

Effects of Graphic and Verbal Probability Information on Command Decision Making

The performance effects of graphic and verbal representations of uncertainty were investigated within the context of a spatial problem., Sixteen experienced naval submarine officers acted as decision makers (i.e., submarine commanders) in eight simulated scenarios. Four scenario problems were presented with either a verbal or a graphic representation of uncertainty. The degree of uncertainty was controlled by manipulating oceanic conditions and information modeling. The graphic representation of uncertainty resulted in superior range estimates only when the oceanic noise was high and the environmental information was properly modeled. No reliable differences in confidence were observed. These results suggest that for spatial problems, a graphic/spatial representation of uncertainty may considerably improve the judgments of decision makers.

The Relationship Between Spatial Transformations and Iconic Gestures

Current theories of gesture production all suggest that spatial working memory is a critical component of iconic gesture production. However, none of the models has a selection mechanism for what aspect of spatial working memory is gestured. We explored how expert and journeyman scientists gestured while discussing their work. Participants were most likely to make iconic gestures about change over time (spatial transformations), less likely to gesture about spatial relations and locations (geometric relations), and far less likely to gesture about the magnitude of spatial entities. We also found that experts were especially likely to have a high degree of association between iconic gestures and spatial transformations. These results show that different features of spatial language are gestured about at different rates. We suggest that current gesture production models need to be expanded to include selection mechanisms to account for these differences.

Visualizing uncertainty: the impact on performance.

Objective: This work investigated the impact of uncertainty representation on performance in a complex authentic visualization task, submarine localization. Background: Because passive sonar does not provide unique course, speed, and range information on a contact, the submarine operates under significant uncertainty. There are many algorithms designed to address this problem, but all are subject to uncertainty. The extent of this solution uncertainty can be expressed in several ways, including a table of locations (course, speed, range) or a graphical area of uncertainty. Method: To test the hypothesis that the representation of uncertainty that more closely matches the experts’ preferred representation of the problem would better support performance, even for the nonexpert., performance data were collected using displays that were either stripped of the spatial or the tabular representation. Results: Performance was more accurate when uncertainty was displayed spatially. This effect was only significant for the nonexperts for whom the spatial displays supported almost expert-like performance. This effect appears to be due to reduced mental effort. Conclusion: These results suggest that when the representation of uncertainty for this spatial task better matches the expert’s preferred representation of the problem even a nonexpert can show expert-like performance. Application: These results could apply to any domain where performance requires working with highly uncertain information.

When using the tool interferes with doing the task

How much time the user spends working on a task versus fiddling with the tool is an important aspect of usability. The concept of the ratio and distribution of tool-only operations to total operations is proposed to capture this aspect.

CONTENDING WITH COMPLEXITY: THE DEVELOPMENT AND USE OF SCALED WORLDS AS RESEARCH TOOLS

Complex, real-world behavior takes place in complex, real-world environments. Efforts to study cognition in such environments can be hampered by difficulty in accurately tracking information flow. This problem may be tackled by studying task performance in the context of a scaled world—an abstracted version of the task environment designed to elucidate information flow while maintaining the critical elements ofthat environment. Scaled worlds are discussed in the context of our current research, Project NEMO.

Submariner Situation Assessment: A Cognitive Process Analysis and Modeling Approach

A cognitive process analysis and modeling approach to task analysis is described in the context of Project Nemo, a research effort aimed at explicating situation assessment behavior in commanders of nuclear powered attack submarines. The approach is structured around the rationality and problem space principles outlined in Card, Moran and Newell (1983). The process analysis phases involve characterizing the task domain as well as the subjects goals, operators, and knowledge. The modeling phases involves instantiating the elements from the process analysis phases into a runnable computational cognitive model. The behavior of this model is then judged against a standard, such as expert judgment or the commanders behavior, in order to evaluate the sufficiency of the cognitive process analysis. Unlike conventional task analysis methods, this approach enables the analyst not only to describe task behavior at a detailed cognitive process level, but to evaluate the precision of that description.

Cognitive architectures and HCI

The Cognitive Architectures and Human-Computer Interaction Workshop examined computational cognitive modeling approaches to human-computer interaction issues (HCI). The five major architectures and variations represented were briefly summarized. Participants compared approaches to a set of selected HCI problems and alternative solutions, and compared the strengths and weaknesses of the architectures. A list of additional issues was generated and discussed.

Improved Workflow, Environmental Effects Analysis and User Control for Tactical Weather Forecasting

A project called Environmental Visualization (EVIS) conducted and completed testing of a capability to improve the air strike forecasting workflow. The capability was tested in a yoked experimental paradigm that had pairs of fleet forecasters generating an air strike forecast, one forecaster using the EMES capability and one using the current web-based tools. Three pairs of forecasters were tested, and within each pair, the EMES capability supported substantially faster (∼ 40%) forecast completion time. Feedback from the fleet users and personnel was positive about the approach we are taking and the capability that was tested. The team received substantial constructive feedback on the initial capability and will incorporate this into the next development phase.