Daniel Gartenberg

Situation awareness recovery.

Objective: We describe a novel concept, situation awareness recovery (SAR), and we identify perceptual and cognitive processes that characterize SAR. Background: Situation awareness (SA) is typically described in terms of perceiving relevant elements of the environment, comprehending how those elements are integrated into a meaningful whole, and projecting that meaning into the future. Yet SA fluctuates during the time course of a task, making it important to understand the process by which SA is recovered after it is degraded. Method: We investigated SAR using different types of interruptions to degrade SA. In Experiment 1, participants watched short videos of an operator performing a supervisory control task, and then the participants were either interrupted or not interrupted, after which SA was assessed using a questionnaire. In Experiment 2, participants performed a supervisory control task in which they guided vehicles to their respective targets and either experienced an interruption, during which they performed a visual search task in a different panel, or were not interrupted. Results: The SAR processes we identified included shorter fixation durations, increased number of objects scanned, longer resumption lags, and a greater likelihood of refixating on objects that were previously looked at. Conclusions: We interpret these findings in terms of the memory-for-goals model, which suggests that SAR consists of increased scanning in order to compensate for decay, and previously viewed cues act as associative primes that reactivate memory traces of goals and plans.

Dynamic Operator Overload: A Model for Predicting Workload During Supervisory Control

Crandall and Cummings & Mitchell introduced fan-out as a measure of the maximum number of robots a single human operator can supervise in a given single-human-multiple-robot system. Fan-out is based on the time constraints imposed by limitations of the robots and of the supervisor, e.g., limitations in attention. Adapting their work, we introduced a dynamic model of operator overload that predicts failures in supervisory control in real time, based on fluctuations in time constraints and in the supervisors allocation of attention, as assessed by eye fixations. Operator overload was assessed by damage incurred by unmanned aerial vehicles when they traversed hazard areas. The model generalized well to variants of the baseline task. We then incorporated the model into the system where it predicted in real time, when an operator would fail to prevent vehicle damage and alerted the operator to the threat at those times. These model-based adaptive cues reduced the damage rate by one-half relative to a control condition with no cues.

Collecting health-related data on the smart phone: mental models, cost of collection, and perceived benefit of feedback

We describe a mobile health application that collects data relevant to the treatment of insomnia and other sleep-related problems. The application is based on the principles from neuroergonomics, which emphasizes assessment of the brain’s alertness system in everyday, naturalistic environments, and ubiquitous computing. Application benefits include the ability to incorporate both embedded data collection and retrospective manual data input—thus providing the user with a rewarding data access process. The retrospective data input feature was evaluated by comparing an older version of the retrospective editing interface with a newly developed one. The time course of user interactions was precisely measured by exporting time stamps of user interactions using the Google App Engine. We also developed models that closely fit the time course of user interactions using the Goals, Operators, Methods, and Selection rules (GOMS) modeling method. The user data and GOMS models demonstrated that the retrospective sleep tracking feature of the new interface was faster to use but that the retrospective habit tracking feature was slower. Survey results indicated that participants enjoyed using the newly developed interface more than the old interface for the assessment of both sleep and habits. These findings indicate that a mobile application should be designed not only to reduce the amount of time it takes a user to input data, but also to conform to the user’s mental models of its behavior.

Adaptive automation and cue invocation: the effect of cue timing on operator error

Adaptive automation (AA) can improve performance while addressing the problems associated with a fully automated system. The best way to invoke AA is unclear, but two ways include critical events and the operators state. A hybrid model of AA invocation, the dynamic model of operator overload (DMOO), that takes into account critical events and the operators state was recently shown to improve performance. The DMOO initiates AA using critical events and attention allocation, informed by eye movements. We compared the DMOO with an inaccurate automation invocation system and a system that invoked AA based only on critical events. Fewer errors were made with DMOO than with the inaccurate system. In the critical event condition, where automation was invoked at an earlier point in time, there were more memory and planning errors, while for the DMOO condition, which invocated automation at a later point in time, there were more perceptual errors. These findings provide a framework for reducing specific types of errors through different automation invocation.

An ACT-R Process Model of the Signal Duration Phenomenon of Vigilance

Performance on tasks that require sustained attention can be impacted by various factors that include: signal duration, the use of declarative memory in the task, the frequency of critical stimuli that require a response, and the event-rate of the stimuli. A viable model of the ability to maintain vigilance ought to account for these phenomena. In this paper, we focus on one of these critical factors: signal duration. For this we use results from Baker (1963), who manipulated signal duration in a clock task where the second hand moved in a continuous swipe motion. The critical stimuli were stoppages of the hand that lasted for 200, 300, 400, 600, or 800 ms. The results provided evidence for an interaction between condition and time-on-task, where performance declined at a faster rate as the signal duration decreased. In this paper, we describe an ACT-R model that uses fatigue mechanisms from Gunzelmann et al. (2009) that were proposed to account for the impact of sleep loss on sustained attention performance. The research demonstrates how those same mechanisms can be used to understand vigilance task performance. This illustrates an important foundation for predicting and tracking vigilance decrements in applied settings, and validates a mechanism that creates a theoretical link between the vigilance decrement and sleep loss.

Validation of sleep-2-Peak: A smartphone application that can detect fatigue-related changes in reaction times during sleep deprivation

Despite its high sensitivity and validity in the context of sleep loss, the Psychomotor Vigilance Test (PVT) could be improved. The aim of the present study was to validate a new smartphone PVT-type application called sleep-2-Peak (s2P) by determining its ability to assess fatigue-related changes in alertness in a context of extended wakefulness. Short 3-min versions of s2P and of the classic PVT were administered at every even hour during a 35-h total sleep deprivation protocol. In addition, subjective measures of sleepiness were collected. The outcomes on these tests were then compared using Pearson product-moment correlations, t tests, and repeated measures within-groups analyses of variance. The results showed that both tests significantly correlated on all outcome variables, that both significantly distinguished between the alert and sleepy states in the same individual, and that both varied similarly through the sleep deprivation protocol as sleep loss accumulated. All outcome variables on both tests also correlated significantly with the subjective measures of sleepiness. These results suggest that a 3-min version of s2P is a valid tool for differentiating alert from sleepy states and is as sensitive as the PVT for tracking fatigue-related changes during extended wakefulness and sleep loss. Unlike the PVT, s2P does not provide feedback to subjects on each trial. We discuss how this feature of s2P raises the possibility that the performance results measured by s2P could be less impacted by motivational confounds, giving this tool added value in particular clinical and/or research settings.

Improving Vigilance Analysis Methodology: Questioning the Successive Versus Simultaneous Distinction

We describe a vigilance experiment of a successive task and a simultaneous task. Successive tasks require comparing the current stimulus on the screen to a representation in memory (i.e. making a declarative memory retrieval), whereas simultaneous tasks require making a comparative judgment based on information that is available on the screen. When analyzing the data from this experiment using conventional methods, there was an effect of time-on-task (i.e. block), an effect of task type, and an interaction between block and task type. These findings were consistent with previously reported studies regarding the successive and simultaneous vigilance task distinction, which interpret such findings as evidence that the decrement is more severe for successive tasks. But different results and conclusions are made when more appropriate analyses of the data are used, such as: including block as an interval variable instead of a categorical variable and making the dependent variable detection of critical signals instead of using A’. When these analysis techniques were used, there was no effect of task type and there was no interaction with time on task. This raises questions about many of the findings in the literature, especially those regarding the successive and simultaneous distinction.

Time Pressure, Memory, and Task Knowledge Facilitate the Opportunism Heuristic in Dynamic Tasks

There are a variety of strategies that operators can utilize when performing a dynamic task, yet operator strategies are typically studied in a well-controlled environment that prevents the possibility of these strategies from interacting or competing with one another. In this study we investigated operator strategy use in a dynamic supervisory control task. We identified four possible strategies that the operator may use: scanning, opportunism, task knowledge, and memory. In order to determine the impact of time pressure on strategy use, we manipulated the speed of the vehicles. We found that as time pressure increased, operators shifted from a scanning strategy to a heuristic opportunistic strategy. We also found that when operators used task knowledge and memory they were more likely to be opportunistic.

Examining the Role of Task Requirements in the Magnitude of the Vigilance Decrement

The vigilance decrement in sustained attention tasks is a prevalent example of cognitive fatigue in the literature. A critical challenge for current theories is to account for differences in the magnitude of the vigilance decrement across tasks that involve memory (successive tasks) and those that do not (simultaneous tasks). The empirical results described in this paper examine this issue by comparing performance, including eye movement data, between successive and simultaneous tasks that require multiple fixations to encode the stimulus for each trial. The findings show that differences in the magnitude of the vigilance decrement between successive and simultaneous tasks were observed only when a response deadline was imposed in the analysis of reaction times. This suggests that memory requirements did not exacerbate the deleterious impacts of time on task on the ability to accurately identify the critical stimuli. At the same time, eye tracking data collected during the study provided evidence for disruptions in cognitive processing that manifested as increased delays between fixations on stimulus elements and between encoding the second stimulus element and responding. These delays were particularly pronounced in later stages of encoding and responding. The similarity of the findings for both tasks suggests that the vigilance decrement may arise from common mechanisms in both cases. Differences in the magnitude of the decrement arise as a function of how degraded cognitive processing interacts with differences in the information processing requirements and other task characteristics. The findings are consistent with recent accounts of the vigilance decrement, which integrate features of prior theoretical perspectives.

Optimal interface location and limits of gesture proficiency in an automobile

This paper explores the optimal location of gesture based in-vehicle technology for minimizing driver distraction, as well as the specific manipulative gestures that would accompany such gesture based in-vehicle technologies. Three different vehicle locations and ten different driver gestures were evaluated during testing. Participants in the study performed each of the gestures at all three locations, and results indicated that most individuals preferred using the steering wheel location. Participants’ responses indicated that they felt most comfortable with a repertoire of about seven gestures. Our initial findings suggest that a gesture-based interface system might be most effective when placed at the three o’clock position on the steering wheel, and when thee systems operate using a maximum of seven gestures.