Raegan M. Hoeft

TPL-KATS—concept map: a computerized knowledge assessment tool

Abstract A wide variety of instruments for the assessment of structural knowledge are used in psychology, education, and engineering. A software tool, TPL-KATS—Concept Map, was developed to examine the organization of mental representations via the use of a computer. This software tool provides a more efficient and user-friendly method than the traditional manual approaches to concept mapping used in the past. TPL-KATS—Concept Map can automate the entire concept map task, from instructions, to administration, to scoring. Although originally developed as a data collection tool for psychological studies, this software can be a valuable resource in training, interface design, and education as well.

Scoring Concepts Maps: Can a Practical Method of Scoring Concept Maps be Used to Assess Trainee's Knowledge Structures?

Previous research has indicated that the structure of an individuals knowledge may be just as important as the quantity of knowledge. Given this, using assessment methods that elicit trainees knowledge structures seems imperative for predicting performance. Unfortunately, incorporation of these methods has been hindered, in part, due to the complexity of methods used to derive a score and the belief that simpler methods of scoring will not provide accurate information about an individuals knowledge. Presented here is a study that investigated whether this claim was true for the structural knowledge elicitation method, concept mapping. Twenty-six participants were run through a same-day training and assessment session. Following, concept maps were scored using a simple method and a complex method. Results indicated that both scoring methods produced significantly higher scores for the trained group and significantly lower scores for the untrained group. In addition, there was a very strong, positive relationship between the two scoring methods. Finally, both methods produced a moderately high correlation with the paper-pencil assessment. Results and implications are discussed further within the paper.

11. “A FRENCHMAN, A GERMAN, AND AN ENGLISHMAN …”: THE IMPACT OF CULTURAL HETEROGENEITY ON TEAMS

Most traditional research on work groups has studied groups and teams that are homogeneous with respect to culture. To alleviate the dearth of material on culturally heterogeneous teams, this chapter provides an overview of the impact of cultural diversity on groups and teams in today’s workforce. First, we focus on the problems involved in defining the constructs of “teams” and “culture.” Second, we provide a brief review of the cultural factors that have been identified as affecting human performance. This review serves as the basis for the third section of this chapter, which investigates if – and how – cultural heterogeneity affects team performance. Finally, we conclude with how culturally diverse workplaces can be managed and how to improve performance when faced with cultural diversity.

Demonstration: Advancing Robotics Research Through the Use of a Scale Mout Facility

This demonstration serves as an introduction to the CARAT scale MOUT (Military Operation in Urban Terrain) facility developed at the Team Performance Laboratory (TPL) at the University of Central Florida (UCF). Advances in automated military vehicles require research to understand how best to allocate control of these vehicles. Whether, discussing uninhabited ground vehicles (UGVs) or air vehicles (UAVs), many questions still exist as to the optimum level of performance with respect to the ratio of human controls to vehicles. The scale MOUT facility at UCF allows researchers to investigate these issues without sacrificing large costly equipment and without requiring vast physical areas, within which to test such equipment. This demonstration provides an introduction to the scale MOUT facility, describes the basic need for this tool, presents its advantages over full size counterparts, as well as several other possible uses for the facility.

Exploring the Role of Shared Mental Models for Implicit Coordination in Teams

Previous research has suggested that when high-performing teams are experiencing increased workload, they will adaptively shift from explicit to more implicit forms of coordination. This is thought to occur because the team members have shared mental models (SMMs) which allow them to anticipate one anothers needs. However, it is currently not known how SMMs are related to implicit coordination. Much of the research on SMMs had focused on the actual level of sharedness and, to some degree, on the accuracy of each team members model. However, to our knowledge, none has investigated the relationship between SMMs and implicit coordination. Furthermore, one line of research that has received very little attention is the notion of perceptions of sharedness. Must team members have an accurate perception of how well they share mental models in order to exploit them via implicit coordination? The purpose of this paper is to explore these fundamental questions that drive the process of implicit coordination.

Is a Picture Worth a Thousand Words? Investigating Structural Knowledge with Textual and Pictorial Stimuli

The differences between the processing of textual and pictorial information have been a topic of research for some time now. Previous research concerning the modality of information has often concentrated on the speed of processing rather than the organizational differences that may exist. This experiment utilized card sorting to evaluate the changes in knowledge organization that occur when information is presented in text and picture formats. In addition to this, the structure of the elicitation task was manipulated to evaluate its effects on sharedness. It was found that textual stimuli produced a greater sharedness among participants in a free sorting task. However, for a structured sorting task, results reversed, and pictorial stimuli created a greater level of sharedness. Overall, structured sorting tasks produced a greater level of sharedness than the free sorting condition, regardless of modality.

A HUMAN FACTORS ANALYSIS OF THE CURRENT U.S. NOTICES TO AIRMEN (NOTAM) SYSTEM

We performed a human factors analysis of the format and dissemination of the Notices to Airmen (NOTAM; Federal Aviation Administration [FAA], 2002a) system to determine possible causes for dissatisfaction with the system. In the analysis, we utilized design principles developed for the FAA in the FAA Human Factors Design Guide (Wagner, Birt, Snyder, & Duncanson, 1996). The document provides guidance and information to those involved in all areas of the aviation industry for the design and evaluation of systems and equipment. This evaluation of the NOTAM system demonstrated that NOTAMs do not follow many of the basic human factors principles contained in the referenced design guide. Furthermore, analyses of NOTAMs based on the FAA design principles augmented findings from a survey (Hoeft, Kochan, & Jentsch, 2003) that suggested performance and satisfaction with the system might be greatly increased if human factors-based changes to the NOTAM system were implemented.

Kids Get Sick Too: A Proposed Child Simulator Sickness Questionnaire

The increasing use of video games and virtual reality for education, as well as entertainment, warrants a method of assessing the degree to which these modalities of presentation may adversely affect the viewers. For adults, this tool currently exists in the Simulator Sickness Questionnaire (SSQ; Kennedy, Lane, Berbaum, & Lilienthal, 1993). However, for children, who are more exposed to these technologies and who are more prone to these adverse effects, there is no appropriate method of attaining this important information. Often, researchers will attempt to adapt some of the questions on the SSQ to fit their purposes; however this creates a completely unstandardized method of assessing the amount of simulator sickness experienced. This paper introduces a Child Simulator Sickness Questionnaire as a simple, short, standardized method of collecting simulator sickness symptom data for children.

Investigating the Effects of Modality and Instruction on Structural Knowledge

Previous research has shown that individuals process stimuli differently based on the modality in which the stimuli are presented. In an effort to further understand how stimuli are processed and organized, the current study utilized card sorting in one of three media modalities (text only, picture only, and text and picture combined) and in one of two instructional styles (free sort or structured sort, via function) to assess the degree of sharedness between the resulting mental models. It was believed that structure sorts would produces greater levels of sharedness and that the semantic nature of the structured sort would yield the greatest levels of sharedness in the text only condition. Structured sorts did in fact produce better results than unstructured sorts as expected; however, in the structured sort, the picture only condition provided the best results. In addition, comparisons were made to a previous study suggesting that the proper instruction by modality combination should be evaluated based on domain and desired outcomes. The implications of these findings as well as the need for further research are discussed.

Structural Knowledge Assessment with the Team Performance Lab's Knowledge Analysis Test Suite (TPL-KATS)

Card sorting and concept mapping are two popular techniques used for assessing structural knowledge. However, manual administration of these tasks has been cumbersome in the past. This demonstration serves as an introduction to the TPL-KATS – Card Sort and Concept Map software tool developed at the Team Performance Lab at the University of Central Florida. TPL-KATS is presented as a possible solution to the problems of manual administration. The software contains computerized versions of both techniques, with automatic administration and scoring capabilities. This demonstration provides an introduction to the card sort and concept map tasks, and describes the basic need for this software tool. A full description of TPL-KATS – Card Sort and Concept Map software is presented, along with a demonstration of its advantages over current techniques.