Janet J. Turnage

Effects of graded dosages of alcohol on nine computerized repeated-measures tests

We report a controlled laboratory validation experiment to provide an empirical sequel to the general background of the development of the Automated Performance Test System (APTS) that was presented in Turnage et al. (1992). The purpose of this study was to index performance deficit against various Blood Alcohol Concentrations (BACs) of small (0.05% BAC), medium (0.10% BAC), and large (0.15% BAC) dosages as well as a placebo (0.00 BAC) condition. Blood alcohol concentrations, measured in four different ways, were highly reliable and the method with the highest intercorrelation was whole blood. When taken singly, eight of the nine tests produced significant (p < 0.001) relationships with the disparate blood alcohol levels, which were essentially monotonic. A multiple regression analysis suggested that most of the tests were behaving similarly and that two or three tests accounted for 52 to 54% of the variance attributable to alcohol that was explained by the longer battery. While all tests appear valid, some of them appeared more sensitive than others (viz., code substitution, manikin, and choice reaction time). From the standpoint of these tests, greater changes occurred in cognitive function between the placebo and 0.05 level than between the 0.05 and 0.10 levels. However, the greatest reduction in performance occurred between 0.10 and 0.15, and the relatively abrupt nature of this change implies that sharp decrements in cognitive performance occur at that point. There were individual differences in resistance to alcohol, and there is strong inference that these differences would be reliable if they were tested again. We believe that further development and study of such techniques is warranted for use in fitness-for-duty testing and development of a dose equivalency index.

Development of microcomputer-based mental acuity tests.

Recent disasters have focused attention on performance problems due to the use of alcohol and controlled substances in the workplace. Environmental stressors such as thermal extremes, mixed gases, noise, motion, and vibration also have adverse effects on human performance and operator efficiency. However, the lack of a standardized, sensitive, human performance assessment battery has probably delayed the systematic study of the deleterious effects of various toxic chemicals and drugs at home and in the workplace. The collective goal of the research reported here is the development of a menu of tests embedded in a coherent package of hardware and software that may be useful in repeated-measures studies of a broad range of agents that can degrade human performance. A menu of 40 tests from the Automated Performance Test System (APTS) is described, and the series of interlocking studies supporting its development is reviewed. The APTS tests, which run on several versions of laptop portables and desktop personal computers, have been shown to be stable, reliable, and factorially rich, and to have predictive validities with holistic measures of intelligence and simulator performances. In addition, sensitivity studies have been conducted in which performance changes due to stressors, agents, and treatments were demonstrated. We believe that tests like those described here have prospective use as an adjunct to urine testing for the screening for performance loss of individuals who are granted access to workplaces and stations that impact public safety.

Communality of Videogame Performances with Tracking Tasks

Performances on three videogames were compared with two different tracking tasks and a third dual tracking task, after stability of performance was achieved by 40 male subjects. The six tasks were performed in succession for three weeks each, and one year separated administration of the tracking tasks from the videogames. Convergence of the cross-correlations among these tasks occurred such that higher correlations were attained for scores obtained late in practice (late/late) of two tasks (even when they were performed a year apart) rather than for scores obtained earlier in training (viz., early/early). High communality between these two types of tasks was observed as indicated by virtual overlap between the videogames and the tracking tasks, even though different control media (joysticks, knobs), control orders (zero/first), and control inputs (displacement/force transducer) were required. Because these tasks apparently index similar abilities, there may be advantages to measuring tracking skills with videogames, which include low cost, wide availability, inherent interest to subjects and motivation, rapid approach to stable levels, and, in many cases, greater apparent face validity relative to real world performance requirements.

RELIABILITY GENERALIZATION: A VIABLE KEY FOR ESTABLISHING VALIDITY GENERALIZATION '

Even with radical restriction of range, reliability coefficients from 10 studies gave an average interstudy value of .74, suggesting constancy of reliability over diverse experiments. A value from a new test can help index reliability of tests not previously studied.

Individual Differences in Visual, Motor and Cognitive Performances: Correlations with a Simulated Shuttle Landing Task

Numerous physiological changes which occur during and following space flight are well-documented. It seems likely that the frequency and severity of these physiological changes would be accompanied by changes in cognition, sensory system integrity, skilled motor control, and complex decision making. However, changes in performance in these areas are not well documented nor understood. Documenting the extent and nature of such potential performance decrements, to the extent that they occur, becomes increasingly important as NASA prepares for longer duration missions on space station and contemplates more complex missions in future exploration of space. We explored the application of a methodology we are developing called surrogate testing. The method requires the linkage of complex real world tasks to more elemental tasks. This report shows, as first steps, predictive validities of such tests for a simulated shuttle landing and provides a series of surrogate scores which can be used to evaluate stresses and drug effects.

Diagnosis of Alcohol Intoxication: Effectiveness of Cognitive and Neurovestibular Field Sobriety Tests

Tests from an automated performance test battery of cognitive tests and the standardized field sobriety tests (FST) used nationwide by law enforcement officers were administered in three experiments involving graded dosages of alcohol. In the first experiment, subjects were raised to one of four levels of alcohol dosage in four different sessions. In the second experiment, the descending branch of the blood alcohol level (BAL) curve was monitored from .15 BAL, and cognitive and motor performances were assessed by the Automated Performance Test System (APTS) and FST. In the third experiment, the ascending and descending limbs of the alcohol dosage curve were followed. Dose-response relationships were conducted and were statistically significant (p < .001) in all three studies for all but one test when evaluated singly. Using either test battery, composite scores could be employed to index degraded performance from elevated blood alcohol levels. The best single test was gaze nystagmus from the FST battery and the next best was code substitution from the cognitive battery. Taken singly, the individual tests ranged from 66% to 81% in terms of correctly detecting the dichotomous criterion of < 0.10 versus 0.10 BAC and above. These results are discussed in the context of standards setting for driving under the influence (DUI) and the use of behavioral tests to evaluate over-the-counter and prescription drugs.

The use of Surrogate Techniques for the Measurement of Team Performance

This paper describes shortcomings in current team performance measurement methodologies, discusses emerging observational and automated measurement techniques, and describes surrogate measurement concepts in the context of team performance. Research using surrogate approaches is suggested to improve the reliability of team assessment and to increase the sensitivity of team measures to conditions that are likely to improve or degrade team performance.

Training Transfer in a Tank Gunnery Training System

Three tank gunnery trainers were studied to determine learning transfer over repeated trails. Devices included the TOPGUN trainer, a part-task, reduced-fidelity tank gunnery trainer; the Videodisk Gunnery Trainer (VIGS), another part-task, limited-fidelity trainer; and the Conduct-of-Fire Trainer (COFT), a full-fidelity trainer. The objective was to determine the degree of gunnery skills transfer between the part-task gunnery trainers and the full-fidelity simulator. COFT criterion performances were examined for two pretraining groups (either TOPGUN first, then VIGS, or VIGS first, then TOPGUN) and a control group in order to determine which pretraining sequence leads to better performance. Each training group, composed of 20 subjects, received two multiple-mission engagement trials on four consecutive days (2 VIGS-2 TOPGUN, or vice versa) before COFT transfer. Results showed significant Group and Trial effects for transfer between TOPGUN and VIGS and significant transfer to COFT performance regardless of the prior sequence of training.

Application of Multiple Cut-Offs for Fitness for Duty Testing

“Sensitivity,” (percentage correct detections of the number treated) is the most often used index of psychological tests or test battery adequacy, although other factors (stability, reliability, other forms of validity, usability, etc.) are also reported as positive features. This paper reports on a method to improve “specificity” [ratio of persons correctly identified as not treated to total number not treated]. Specificity issues are a great concern, where behavioral testing may be used for managerial or regulatory decisions about workers. In such cases the percentage of workers with appreciable dosages may be < 10% and false positive percentages thereby become important. In two alcohol experiments, we empirically validated different multiple cut-offs seeking good false positive rates. In two additional alcohol studies we cross validated our optimum findings and found false positive rates of 3.6% (96.4% specificity) can be achieved with a combination of 6% decrement on three of four tests while retaining adequate sensitivity.

Readiness for Duty: Tuning False Positives by Simulation from Empirical Data

For applications such as the assessment of environmental stress or toxic agents, the metric requirements of performance test batteries include stability, reliability and sensitivity. However, fitness-for-duty applications present additional organizational and management laboratory conditions where the sensitivities of these test batteries are evaluated, the ratio of “treated” to “untreated” subjects is usually 50/50. However, in the workplace, the percentage of persons who are expected NOT to be impaired may be <5% and unless the accuracy of the psychological tests exceeds one minus the percentage NOT impaired (1−.05 = 95%) the percentage of false positives will exceed the percentage of impaired persons identified - thus false positives become a primary management focus. Data from four different empirical data sets (N>100), with multiple repeated measures (15−40 sessions) and a battery of six computerized tests, were implemented into an interactive computerized algorithm. By varying: a) multiple cut-offs; b) number trials in baseline; and c) select decrement criteria, we were able to tune false positive rates to levels lower than three percent.