Leslie A. Whitaker

Tank Crew Performance: Effects of Speech Intelligibility on Target Acquisition and Subjective Workload Assessment

Thirty tank crews were tested in the Ft. Knox COFT tank simulator. The COFT simulator is a gunnery training facility. The crews task was to shoot specified enemy targets. Each crew consisted of a tank commander and a gunner. The commander told the gunner, via an intercom system, which enemy object was the next target. Performance and subjective workload were measured as a function of the speech intelligibility transmitted by the intercom system. Five levels of intelligibility were tested. The measures of operational effectiveness were the number of targets correctly fired upon and the gunners latency. Subjective workload was measured using the Subjective Workload Assessment Technique (SWAT). Gunner performance and subjective workload covaried across intelligibility levels. Performance was not significantly affected until intelligibility levels fell to 50%. However, SWAT ratings increased linearly with decreasing intelligibility level.

A Usability Evaluation for Telemedicine Medical Equipment

This paper describes a conceptual framework and human factors evaluation for usability assessment of commercially available telemedicine technology. Four criteria of human factors evaluation are utilized: (1) technical acceptability; (2) operational effectiveness; (3) clinical appropriateness; and (4) equipment selection. These criteria were applied to commercially available video-otoscope systems for otological evaluations.

Communication between Crews: The Effects of Speech Intelligibility on Team Performance:

Speech is a critical method of communication among group members while they are trying to accomplish a task. The present research program is designed to determine the impact of speech communication on performing a variety of communication-intensive tasks. A model describing performance as a function of auditory workload has guided this research. This model states that transmission, linguistic, and individual factors each contribute to auditory workload and hence influence task performance. The current study focused on two transmission factors: speech intelligibility and communication structure. Previous work in this program has reported the performance of two- or three-person crews operating alone to accomplish various tasks. The present study examined the team performance of two crews operating interactively to accomplish more complex tasks. Speech intelligibility was varied from 100% to 25% and was measured using the Modified Rhymes Test. Twelve crews were tested at the Closed Combat Test Bed using M1A2 tank simulators. The results of this study are consistent with those predicted by the auditory workload model; decrements in task performance occur at higher levels of intelligibility for more complex tasks than for less complex tasks. In addition to the task performance measured in this study, verbal protocols taken from recorded transcripts were coded as evidence of the changes in communication structure when speech intelligibility is varied. The implications of these findings for communication using cellular telephones and radio communication are discussed.

Assessing Cognitive Workload Metrics for Evaluating Telecommunication Tasks

Telecommunication technology provides the potential for teamwork across distances. In this way, geographically separated team members can communicate electronically to solve problems. An experiment was conducted to test the sensitivity of three cognitive workload metrics to tasks performed under conditions of telecommunication vs. face-to-face communication. Twenty-three teams participated as subjects. A block puzzle pattern task was tested at four levels of puzzle difficulty under the two communication conditions. Task performance (time to complete and errors), as well as cognitive workload ratings were measured. Three workload metrics [Subjective Workload Assessment Technique (SWAT), NASA-TLX, and Modified Cooper Harper (MCH)] were compared. Each persons spatial ability was assessed using the Cognitive Laterality Battery. Both performance and workload varied as a function of puzzle difficulty. While all workload metrics were able to discriminate amongst some of the puzzle difficulty levels, SWAT had the two advantages of providing the most sensitive measure of difficulty plus a wider range of workload ratings.

Effects of Speech Intelligibility, Morphological Confusions, and Redundancy on Task Performance

Speech is a vital means of communication for completing many tasks. The speech intelligibility needed for successful communication may be degraded by ambient noise levels, poor communication equipment, or hearing impairments. The present research tested the impact of speech message content on task performance under conditions of degraded speech intelligibility. Sixteen subjects participated in a laboratory experiment using on a board game. Message redundancy, morphological confusions, and speech intelligibility were varied. Task performance and subjective workload were measured. Morphological confusions adversely affected performance (both speed and accuracy) and subjective workload (SWAT) to an increasingly greater extent as speech intelligibility decreased. High redundancy improved the accuracy of performance and subjective workload when speech intelligibility decreased; however, high redundant messages were longer and required more time to process than low redundant messages. These results extend earlier work which measured the impact of morphological confusions and redundancy on speech intelligibility itself to the measurement of their impact on performance. Implications for the development of message content guidelines and their impact on performance are discussed.

Bidder's Associate: a case-based reasoning system to improve cost estimates of manufactured products

Klein Associates is building a software system that will aid the bidder in writing a successful bid for a new solicitation. The software is called Bidders Associate and is a prototype for a Dayton manufacturing firm which has a database containing approximately 1000 previously built products. Approximately 200 of these products have been made more than once and are likely to be solicited again. Although the data for previously built products are stored on a mainframe computer, the access by the bidders current system is cumbersome and inefficient. Bidders Associate aids the bidder in three ways: (1) accessing an appropriate product from the firms data base; (2) adjusting bid cost estimates on the basis of the retrieved products cost data; (3) producing a final bid for the current solicitation.<<ETX>>

Usability Testing for Telemedicine Systems: A Methodology and Case Study

This paper presents the conceptual framework and methodology that has been developed to perform usability evaluations of commercially available equipment for use in telemedicine applications. Specifically, the three components of the evaluation methodology, namely, technical acceptability, operational effectiveness, and clinical appropriateness, are described. This methodology was used to evaluate commercially available video-otoscope systems for use in a store-and-forward teleconsultation project.

Effects of speech intelligibility upon performance

The assessment of the ability of a talker to communicate with other persons, under given conditions, is normally accomplished by means of speech intelligibility testing. The resultant scores, however, do not provide any quantification of how well the communicating individuals performed the task at hand. An experiment was conducted in which professional military crews operated a tank simulator at five levels of speech intelligibility ranging from very good to extremely poor. Measures such as time to drive to an engagement location, number of targets hit, time to perform the entire mission, etc., were obtained. The results of this study will serve as a first step for establishing more realistic acoustical limits for military systems, guide the design of improved communication systems, and assist operations analysts in better defining war gaming parameters.

Telemedicine Applications: Evaluation Methodologies for Clinical Intervention, Consultation, and Home Health Care

Telemedicine is a method that uses current telecommunications and information technologies to deliver health care to individuals at a distance. This encompasses the diagnosis, treatment, monitoring, and education of patients using systems that allow access to expert advice and patient information regardless of the patient, provider or information location. Telemedicine systems have been implemented for the past 25 years with marginal success. The integration of their technological and human components has seldom been satisfactorily evaluated. The purpose of this panel is to encourage a forum for the discussion of the panels and the audiences experiences in telemedicine. The goal is to establish a dialogue that can continue during the development of telemedicine evaluation methodologies. Toward this end, the five panel speakers will briefly discuss their experiences in the field, followed by audience discussion of their own experiences, and closing with a summary of this discussion by the panel chair.

Modeling Operational Criteria for Evaluating Speech Communication

Thesis: Evaluation of systems can be conducted best under controlled circumstances which approximate operational conditions. In the present paper, we have examined this thesis as it applies to the test and evaluation of multi-person systems. Our specific focus has been the development of a conceptual model of speech communication requirements and the study of the impact of degraded speech intelligibility on performance of these multi-person systems. To obtain the control necessary to evaluate performance using speech intelligibility, an electronic circuit was developed and employed in a series of simulated tests of operational tasks. The application of these research findings to the task of system test and evaluation is described in the present paper.