A. William Evans

TPL-KATS—concept map: a computerized knowledge assessment tool

Abstract A wide variety of instruments for the assessment of structural knowledge are used in psychology, education, and engineering. A software tool, TPL-KATS—Concept Map, was developed to examine the organization of mental representations via the use of a computer. This software tool provides a more efficient and user-friendly method than the traditional manual approaches to concept mapping used in the past. TPL-KATS—Concept Map can automate the entire concept map task, from instructions, to administration, to scoring. Although originally developed as a data collection tool for psychological studies, this software can be a valuable resource in training, interface design, and education as well.

Team Size, Team Role, Communication Modality, and Team Coordination in the Distributed Operation of Multiple Heterogeneous Unmanned Vehicles

The aim of this study was to investigate the effects of team size, communication modality, and team role on measures of team process over time. As the use of semiautonomous, unmanned vehicles increases, it is pertinent to investigate issues associated with the human teams that will control them, including consideration of team roles and the design of systems to support these roles. Using a 1:35 scale facility for military operations in urban terrain, distributed teams of two or three operators used a simulated, unmanned aerial vehicle and a scaled, unmanned ground vehicle to complete two distinct phases of a mission. The teams used either audio or instant messaging for communication, and the analysis of communication assessed their discussion of target identification, target classification, object localization, and vehicle navigation. Findings indicate that the addition of a third teammate resulted in greater role specificity, and the use of audio increased communication of task-relevant content. The proper assignment of team roles can improve the ability of a team to both acquire and synthesize information from remote environments. Furthermore, proper system design can improve the flow of information between teammates over time.

The Influence of Team Size and Communication Modality on Team Effectiveness with Unmanned Systems

This study examines the effects of team size (2 versus 3 operators) and communication modality (audio versus text) on team performance. Performance and workload measures from 112 undergraduate students from the University of Central Florida were used in this analysis. Results indicated that performance was optimal for teams of three operators using audio systems for distributed communication. Results with the NASA TLX showed patterns where workload was lower in the audio condition. Results with the Multiple Resources Questionnaire (MRQ) showed a reversed trend with a higher score in the audio condition, which was attributed to increases in items associated with audio processing.

Interactive Effects of Backup Behavior and Spatial Abilities in the Prediction of Teammate Workload Using Multiple Unmanned Vehicles

This study examined the interactive effects of spatial ability and team process on operator workload, while using multiple unmanned vehicles. The hypotheses also focused on how these effects might change when using different measures of spatial ability. In order to examine this, the Guilford-Zimmerman Spatial Visualization and Spatial Orientation scores of an unmanned aerial vehicle (UAV) operator and navigation support provided by this UAV operator to an unmanned ground vehicle (UGV) operator were used as variables predicting UGV operator workload while performing a reconnaissance task. Results indicated that the interaction of the “guiders” spatial visualization and navigation support and the interaction between spatial orientation and navigation support not only accounted for unique variances in the prediction of his/her teammates workload, but they also produced qualitatively different patterns of results. In identifying these unique contributions, the importance of using multiple spatial ability measures in (unmanned vehicle) research is highlighted.

Mental Model Assessments: Is There Convergence Among Different Methods?

Knowledge elicitation and mental model assessment methods are becoming increasingly popular in applied psychology. However, there continue to be questions about the psychometrics of knowledge elicitation methods. Specifically, more needs to be known regarding the stability and consistency of the results over time (i.e., whether the methods are reliable) and regarding the degree to which the results correctly represent the underlying knowledge structures (i.e., whether the methods are valid). This paper focuses on the convergence among three different assessment methods: (a) pairwise relatedness ratings using Pathfinder, (b) concept mapping, and (c) card sorting. Thirty-six participants completed all three assessments using the same set of twenty driving-related terms. Assessment sequences were counterbalanced, and participants were randomly assigned to one of the six assessment sequences. It was found that the three assessment methods showed very low convergence as measured by the average correlation across the three methods within the same person. Indeed, convergence was lower than the sharedness across participants (as measured by the average correlation across participants within the same assessment method). Additionally, there were order effects among the different assessment sequences. Implications for research and practice are discussed.

Demonstration: Advancing Robotics Research Through the Use of a Scale Mout Facility

This demonstration serves as an introduction to the CARAT scale MOUT (Military Operation in Urban Terrain) facility developed at the Team Performance Laboratory (TPL) at the University of Central Florida (UCF). Advances in automated military vehicles require research to understand how best to allocate control of these vehicles. Whether, discussing uninhabited ground vehicles (UGVs) or air vehicles (UAVs), many questions still exist as to the optimum level of performance with respect to the ratio of human controls to vehicles. The scale MOUT facility at UCF allows researchers to investigate these issues without sacrificing large costly equipment and without requiring vast physical areas, within which to test such equipment. This demonstration provides an introduction to the scale MOUT facility, describes the basic need for this tool, presents its advantages over full size counterparts, as well as several other possible uses for the facility.

Effects of Sharing Control of Unmanned Vehicles on Backup Behavior and Workload in Distributed Operator Teams

This study examined the effects of sharing vehicle control on workload and support behaviors with teams using multiple unmanned systems. The design of systems control was manipulated such that teammates could only control their own vehicle or share control of their vehicle with a teammate. Process variables focused on requests for navigational support from an unmanned ground vehicle (UGV) operator and the support that was provided by the unmanned aerial vehicle (UAV) operator. The primary outcome of interest was workload, as measured by the NASA TLX. Results indicated that there was an interaction between the design manipulation and UGV requests for support in the prediction of support that was provided by an UAV operator, where support was only provided when teams did not share control of their teammates vehicle. The sharing of control and provision of navigation support both increased workload for the UAV operator.

Is a Picture Worth a Thousand Words? Investigating Structural Knowledge with Textual and Pictorial Stimuli

The differences between the processing of textual and pictorial information have been a topic of research for some time now. Previous research concerning the modality of information has often concentrated on the speed of processing rather than the organizational differences that may exist. This experiment utilized card sorting to evaluate the changes in knowledge organization that occur when information is presented in text and picture formats. In addition to this, the structure of the elicitation task was manipulated to evaluate its effects on sharedness. It was found that textual stimuli produced a greater sharedness among participants in a free sorting task. However, for a structured sorting task, results reversed, and pictorial stimuli created a greater level of sharedness. Overall, structured sorting tasks produced a greater level of sharedness than the free sorting condition, regardless of modality.

Investigating the Effects of Modality and Instruction on Structural Knowledge

Previous research has shown that individuals process stimuli differently based on the modality in which the stimuli are presented. In an effort to further understand how stimuli are processed and organized, the current study utilized card sorting in one of three media modalities (text only, picture only, and text and picture combined) and in one of two instructional styles (free sort or structured sort, via function) to assess the degree of sharedness between the resulting mental models. It was believed that structure sorts would produces greater levels of sharedness and that the semantic nature of the structured sort would yield the greatest levels of sharedness in the text only condition. Structured sorts did in fact produce better results than unstructured sorts as expected; however, in the structured sort, the picture only condition provided the best results. In addition, comparisons were made to a previous study suggesting that the proper instruction by modality combination should be evaluated based on domain and desired outcomes. The implications of these findings as well as the need for further research are discussed.

Structural Knowledge Assessment with the Team Performance Lab's Knowledge Analysis Test Suite (TPL-KATS)

Card sorting and concept mapping are two popular techniques used for assessing structural knowledge. However, manual administration of these tasks has been cumbersome in the past. This demonstration serves as an introduction to the TPL-KATS – Card Sort and Concept Map software tool developed at the Team Performance Lab at the University of Central Florida. TPL-KATS is presented as a possible solution to the problems of manual administration. The software contains computerized versions of both techniques, with automatic administration and scoring capabilities. This demonstration provides an introduction to the card sort and concept map tasks, and describes the basic need for this software tool. A full description of TPL-KATS – Card Sort and Concept Map software is presented, along with a demonstration of its advantages over current techniques.