Ronald L. Small

Left. No, Right! Development of the Frame of Reference Transformation Tool (FORT):

A computational model was developed to predict the spatial-cognitive difficulties imposed when the operator must transform information along up to 6 degrees of freedom between a display (viewed at different orientations), and either a cognitive understanding or a compensatory control. The model applies to pilots, robotics operators, navigators or surgeons using endoscopic procedures. Penalties (in workload, errors or time) in the frame of reference transformation (FORT) are based on psychological findings in spatial cognition, such as mental rotation, depth compression, population stereotypes and verbally mediated strategies. We present the graphical user interface for exercising the model; then show how we have applied it to an astronaut space-shuttle, Hubble rendezvous sequence. Finally we validate the model against two existing data sets, one for cognition and one for control. The model could be used as the basis for both a design analysis tool and a real-time operator aiding system.

Unusual Attitude Recoveries With a Spatial Disorientation Icon

Twenty-two participants (12 with prior flying experience and 12 without), performed a series of trials in a low fidelity flight simulator in which they attempted to recover from a series of unusual attitudes. Two display variables were examined: (1) the attitude display was either a traditional head-up display (HUD) with an inside-out motion and pitch ladder, or an Arc-Segmented Attitude Reference (ASAR) display; and (2) a command icon that pointed to the appropriate airplane rotation for recovery direction in pitch and roll was either present or absent. The results revealed that the icon speeded the initial correction and reduced the number of roll reversal errors, while the traditional HUD decreased the total time to recovery. Experienced pilots were more disrupted by the ASAR display than were novices.

Multisensory Enhancement of Command Displays for Unusual Attitude Recovery

In a low-fidelity, fixed-base F-16 flight simulator, 12 fighter pilots attempted to recover from unusual pitch-down inverted attitudes. Recovery was done in a control condition, aided by a heads-up display (HUD) only, and with 3 command display augmentations: (a) a command visual icon, pointing in the direction of appropriate control; (b) the icon augmented with a voice command; and (c) the icon augmented with a tactile command. All 3 command displays reduced the time to make the initial recovery response relative to the control condition, and decreased the frequency of initial incorrect roll responses. The tactile and voice augmentations also improved the speed of initial recovery response from the most severe inversions.

A Pilot Spatial Orientation Aiding System

Spatial disorientation (SD) is a normal human response to accelerations in flight. Its cost to the US military is over

Space shuttle flight crew spatial orientation survey results.

300 million per year, with comparable costs to US civil aviation. Due to significantly increased research over the past decade, some progress is being made in helping pilots avoid SD’s adverse consequences (i.e., mishaps, incidents, or accidents). We used a multi-sensory model-based approach to intelligently trigger SD countermeasures. Results for two simulated and one actual SD event were very promising, and indicate that the next research step should include verifying and validating our models, and evaluating our spatial orientation aiding system (SOAS) via pilot-in-the-loop simulations and flight tests. SOAS evaluates the aircraft motions, pilot workload, aircraft state, and world state to determine if the pilot is probably suffering from SD, and, if so, SOAS applies a range of countermeasures to assist the pilot in recognizing the SD and recovering. Countermeasures include visual cues, audio cues, audio recovery commands, tactile cues, and olfactory cues, as well as the more extreme measures of auto-recovery and auto-ejection (to save the pilots life, only if all other countermeasures have failed).

N-SEEV in SOAS Predicting Time to Notice for Multi-Modal Cockpit Alerting Events

BACKGROUND & METHOD A survey was distributed to 77 Space Shuttle flight crew members; 40 responded covering 71 missions. The goal was to capture historical information before Shuttle retirement and to better understand subjective experiences of illusory sensations due to the transition from 1-G to microgravity and back. RESULTS We analyzed the response data to answer four questions: 1) Do older astronauts suffer more from illusory sensations than younger astronauts? We conclude that they do not because younger flight crew had about twice the rate of illusory sensations as older flight crew. 2) Do trial head motions during re-entry in an effort to hasten readaptation to 1-G really help? Apparently not because those who made trial head motions had a 38% rate of illusory sensations whereas those who did not make trial head motions had a 15% rate of illusory sensations. 3) Do symptoms decrease as flight experience increases? Yes, as reported in other publications, although there are individual exceptions. 4) Do longer duration missions lead to more illusory sensations and readaptation difficulties than shorter duration missions? Yes, the rate of illusory sensations for longer missions was 38%, whereas it was 24% for shorter missions. DISCUSSION Based upon our results, long-duration missions may induce orientation problems that could have significant mission impacts.

Cognitive Task Analysis of Commercial Jet Aircraft Pilots during Instrument Approaches for Baseline and Synthetic Vision Displays

This paper presents the current work to extend the N-SEEV model of visual attention to both the auditory and tactile modalities in support a cockpit adaptive automation system for pilot spatial disorientation. Cockpit countermeasure systems use visual, auditory and tactile modalities to communicate problems to the pilot. The SOAS spatial orientation aiding system uses all three modalities to support a disoriented pilot. The system initiates increasingly intrusive countermeasure as it determines that the probability and severity of a detected disorientation is increasing. N-SEEV has been included within SOAS to support the change to countermeasure levels based on the prediction of the pilot noticing countermeasure onset. Previous versions of N-SEEV could predict a time-to-notice for the onset of a visual cockpit countermeasure. In this work, N-SEEV is extended to include noticing predictions for both the auditory and tactile modes.