Justin G. Hollands

Guidelines for Presenting Quantitative Data in HFES Publications

This article provides guidelines for presenting quantitative data in papers for publication. The article begins with a reader-centered design philosophy that distills the maxim “know thy user” into three components: (a) know your users′ tasks, (b) know the operations supported by your displays, and (c) match users operations to the ones supported by your display. Next, factors affecting the decision to present data in text, tables, or graphs are described: the amount of data, the readers′ informational needs, and the value of visualizing the data. The remainder of the article outlines the design decisions required once an author has selected graphs as the data presentation medium. Decisions about the type of graph depend on the readers′ experience and informational needs as well as characteristics of the independent (predictor) variables and the dependent (criterion) variable. Finally, specific guidelines for the design of graphs are presented. The guidelines were derived from empirical studies, analyses of graph readers′ tasks, and practice-based design guidelines. The guidelines focus on matching the specific sensory, perceptual, and cognitive operations required to read a graph to the operations that the graph supports.

Trust and Reliance on an Automated Combat Identification System

Objective: We examined the effects of aid reliability and reliability disclosure on human trust in and reliance on a combat identification (CID) aid. We tested whether trust acts as a mediating factor between belief in and reliance on a CID aid. Background: Individual CID systems have been developed to reduce friendly fire incidents. However, these systems cannot positively identify a target that does not have a working transponder. Therefore, when the feedback is “unknown”, the target could be hostile, neutral, or friendly. Soldiers have difficulty relying on this type of imperfect automation appropriately. Method: In manual and aided conditions, 24 participants completed a simulated CID task. The reliability of the aid varied within participants, half of whom were told the aid reliability level. We used the difference in response bias values across conditions to measure automation reliance. Results: Response bias varied more appropriately with the aid reliability level when it was disclosed than when not. Trust in aid feedback correlated with belief in aid reliability and reliance on aid feedback; however, belief was not correlated with reliance. Conclusion: To engender appropriate reliance on CID systems, users should be made aware of system reliability. Application: The findings can be applied to the design of information displays for individual CID systems and soldier training.

Revisiting confidence intervals for repeated measures designs

Loftus and Masson (1994) proposed a method for computing confidence intervals (CIs) in repeated measures (RM) designs and later proposed that RM CIs for factorial designs should be based on number of observations rather than number of participants (Masson & Loftus, 2003). However, determining the correct number of observations for a particular effect can be complicated, given that its value depends on the relation between the effect and the overall design. To address this, we recently defined a general number-of-observations principle, explained why it obtains, and provided step-by-step instructions for constructing CIs for various effect types (Jarmasz & Hollands, 2009). In this note, we provide a brief summary of our approach.

Beyond Identity: Incorporating System Reliability Information Into an Automated Combat Identification System

Objective: The aim of this study was to evaluate display formats for an automated combat identification (CID) aid. Background: Verbally informing users of automation reliability improves reliance on automated CID systems. A display can provide reliability information in real time. Method: We developed and tested four visual displays that showed both target identity and system reliability information. Display type (pie, random mesh) and display proximity (integrated, separated) of identity and reliability information were manipulated. In Experiment 1, participants used the displays while engaging targets in a simulated combat environment. In Experiment 2, participants briefly viewed still scenes from the simulation. Results: Participants relied on the automation more appropriately with the integrated display than with the separated display. Participants using the random mesh display showed greater sensitivity than those using a pie chart. However, in Experiment 2, the sensitivity effects were limited to lower reliability levels. Conclusion: The integrated display format and the random mesh display were the most effective displays tested. Application: We recommend the use of the integrated format and a random mesh display to indicate identity and reliability information with an automated CID system.

Understanding bias in proportion production.

The Stevens exponent (beta) can be obtained from proportion estimation judgments using the power model. In this article, the authors extend that model to proportion production, in which the relative magnitudes of 2 stimuli are adjusted to correspond to a numeric proportion (e.g., 1/4 or .25). The model predicts that when beta < 1, small proportions are underproduced, and large proportions are overproduced, but it predicts the reverse when beta > 1, which is the opposite of the predicted patterns for estimation. Eight participants estimated and produced magnitudes and proportions with spatial volume (beta < 1; Experiment 1) and color saturation (beta > 1; Experiment 2). The models predictions were generally supported. An extension of the model using reference points can account for multicycle patterns shown by some participants.

Blue Force Tracking Effects of Spatial Error on Soldier Performance

Blue force tracking (BFT) is a military technology that provides positional awareness of friendly forces on a digital map through global positioning system (GPS) technology. For dismounted soldiers, having readily available information on the location of friendly forces can be critical for mission success. However, GPS can report positions that are spatially inaccurate. The present study required 36 military participants to lead a team through a simulated mission in a virtual environment. The mission required the participant to find and support friendly forces engaged in a firefight with enemy forces. Participants had a digital map, an unreliable BFT device, or a perfectly reliable BFT device. The results indicated that participants using BFT engaged enemy forces more quickly, used their BFT to gain a wider scope of their environment, and had lower workload. For most measures, there were no significant differences between reliable and unreliable BFT, suggesting that even an unreliable BFT can provide benefits to soldier performance.

Selecting Methods for the Analysis of Reliance on Automation

Human reliance on imperfect automation has been the subject of many laboratory experiments. Across these studies, a diverse set of indices of reliance have been used, even in studies with similar experiment settings. This inconsistency of reliance measures makes the meta-analysis of experimental findings difficult. Moreover, few researchers have rigorously defined the optimal level of automation reliance in their settings, making it difficult to judge the appropriateness of that reliance. This paper attempts to guide researchers in selecting and interpreting measures of automation reliance behavior. We review the reliance analysis methods in existing research and propose four criteria for selecting among them. It is recommended that, where possible, future studies define optimal reliance to make unequivocal judgment about the appropriateness of reliance. In addition, more reliable and insightful conclusions can be obtained through the use of multiple measures.

Viewpoint Tethering for Remotely Operated Vehicles Effects on Complex Terrain Navigation and Spatial Awareness

Objective: The effect of viewpoint on the navigation of complex terrain and on spatial awareness was examined with the use of a simulated remotely operated vehicle. Background: The ability to build terrain models in real time may soon allow remote vehicular control from any viewpoint. A virtual tether couples the viewpoint to the vehicle’s position and orientation, but shows more of the terrain than a fully immersive egocentric display. In this sense, it provides visual momentum by providing a view that incorporates egocentric and exocentric qualities. Method: For this study, 12 participants navigated a simulated vehicle across complex virtual terrain using five different display viewpoints: egocentric, dynamic tether, rigid tether, 3-D exocentric, and 2-D exocentric. While navigating, participants had to avoid being seen by simulated enemy units. After the navigation task, participants’ spatial awareness was assessed using a recognition task. Results: The tethered displays minimized the time during which the participant’s vehicle was visible to enemy positions. The egocentric display was more effective than exocentric displays (2-D or 3-D) for navigation, and the exocentric displays were more effective than egocentric for time seen during navigation and the recognition task. The tethered displays produced intermediate results for navigation and recognition. Conclusion: Viewpoint tethering produced the most effective displays for minimizing time seen, but tethered displays were less effective than egocentric and exocentric displays for navigation and recognition, respectively. Application: A tethered display is recommended for applications in which it is necessary to understand the relation of nearby locations to one’s own location.

Smooth Rotation of 2-D and 3-D Representations of Terrain: An Investigation Into the Utility of Visual Momentum

Objective: The potential advantage of visual momentum in the form of smooth rotation between two-dimensional (2-D) and three-dimensional (3-D) displays of geographic terrain was examined. Background: The relative effectiveness of 2-D and 3-D displays is task dependent, leading to the need for multiple frames of reference as users switch tasks. The use of smooth rotation to provide visual momentum has received little scrutiny in the task-switching context. A cognitive model of the processes involved in switching viewpoints on a set of spatial elements is proposed. Methods: In three experiments, participants judged the properties of two points placed on terrain depicted as 2-D or 3-D displays. Participants indicated whether Point A was higher than Point B, or whether Point B could be seen from Point A. Participants performed the two tasks in pairs of trials, switching tasks and displays within the pair. In the continuous transition condition the display dynamically rotated in depth from one display format to the other. In the discrete condition there was an instantaneous viewpoint shift that varied across experiments (Experiment 1: immediate; Experiment 2: delay; Experiment 3: preview). Results: Performance after continuous transition was superior to that after discrete transition. Conclusion: The visual momentum provided by smooth rotation helped users switch tasks. Application: The use of dynamic transition is recommended when observers examine multiple views of terrain over time. The model may serve as a useful heuristic for designers. The results are pertinent to command and control, geological engineering, urban planning, and imagery analysis domains.

Visual Momentum and Task Switching with 2D and 3D Displays of Geographic Terrain

We were interested in determining if the visual momentum provided by gradual transition between two- and three-dimensional (2D and 3D) views of geographic terrain aided task switching. Twenty-two participants made judgments about the properties of two points placed on 2D or 3D displays of terrain. Participants performed the tasks in pairs of trials, switching tasks and displays between trials. On half the trials (continuous transition), the display dynamically rotated in depth from one display format to the other. On the other half (discrete transition), participants were immediately shown the alternate display format. The results showed that response time after transition was less for the continuous condition, and that accuracy was greater for the continuous condition, especially for the 3D display. We argue that this was because the continuous transition provided improved visual momentum between consecutive displays, and recommend the use of dynamic transition when switching views on geographic terrain.