Tristan E. Johnson

Measuring Sharedness of Team-Related Knowledge: Design and Validation of a Shared Mental Model Instrument

Abstract Many researchers have explored how people share and construct similar mental models in teams. They have claimed that successful team performance depends on a shared mental model of team members about task, team, equipment and situation. Most of the literature has illustrated simplified relationships between a teams mental model and their performance without a valid instrument addressing the confined and relevant constructs of a shared mental model. This paper describes the instrument development steps and the conceptual framework for factors associated with shared mental models. After development and refinement, the instrument was finalized for use to measure team-related knowledge. The final instrument consists of 42 items that are linked to the five emergent factors of shared mental models including general task and team knowledge, general task and communication skills, attitude toward teammates and task, team dynamics and interactions, and team resources and working environment.

Intrateam Communication and Performance in Doubles Tennis

Verbal and nonverbal communication is a critical mediator of performance in team sports and yet there is little extant research in sports that involves direct measures of communication. Our study explored communication within NCAA Division I female tennis doubles teams. Video and audio recordings of players during doubles tennis matches captured the communications that took place between and during points. These recordings were coded and sequential analysis computed using the Discussion Analysis Tool software (Jeong, 2003). Results indicated that most communications were emotional (i.e., > 50%) or action statements (i.e., > 25%). Winning teams exhibited significantly different communication sequences than losing teams. In particular, winning teams had a more homogeneous model of communication, which perhaps makes message interpretation more reliable. Finally, winning teams exchanged twice as many messages as losing teams.

Model-Based Methods for Assessment, Learning, and Instruction: Innovative Educational Technology at Florida State University

In this chapter, we describe our research and development efforts relating to eliciting, representing, and analyzing how individuals and small groups conceptualize complex problems. The methods described herein have all been developed and are in various states of being validated. In addition, the methods we describe have been automated and most have been integrated in an online model-based set of tools called HIMATT (Highly Interactive Model-based Assessment Tools and Technologies; available for research purposes at http://himatt.ezw.uni-freiburg.de/cgi-bin/hrun/himatt.pl and soon to be available on a server at Florida State University). HIMATT continues to expand in terms of the tools and technologies included. Our methods and tools represent an approach to learning and instruction that is now embedded in many of the graduate courses at Florida State University and also at the University of Freiburg. We call our approach model-based because it integrates representations of mental models and internal cognitive processes with tools that are used to (a) assess progress of learning, and (b) provide the basis for informative and reflective feedback during instruction.

Effects of a collaborative annotation method on students' learning and learning-related motivation and affect

Two studies tested the effectiveness of a web-based collaborative annotation system (Hy-Lighter) for learning comprehension, and learning-related affect and motivation. In an undergraduate course setting, students (N=27) in study 1, (1) highlighted and annotated selected articles, and (2) highlighted and annotated selected articles and reviewed peer highlights and annotations. In a graduate course setting, students (N=40) in study 2, (1) highlighted and annotated selected articles, and (2) highlighted and annotated selected articles and reviewed peer highlights and annotations. Control groups in both studies read a hard copy of the articles -without using HyLighter and engaging in its associated annotation practices. The main dependent variables included: (a) performance on quizzes, and (b) a number of affective and motivational variables related to reading assignments and academic success. Although not statistically significant, summative assessment scores were higher for students using HyLigther relative to the ones exposed to conventional instruction. HyLighter use also seemed to be associated with more positive affect in undergraduate students relative to their graduate counterparts. Somewhat equivocal findings between the two studies were attributed to the differential implementation of the software in and outside of the classroom. Recommendations for optimal use and desired outcomes were advanced.

A Design Framework for an Online English Writing Course

This chapter proposes a design framework that applies Merrill’s first principles of instruction to an online college English writing course. The framework consists of five interrelated principles grounded in learning and instructional theories and research; it emphasizes task-centered instructional design. In addition, as a way of learners’ practice and evaluation of writing within a task-centered approach, the use of peer review is articulated in the framework. Moreover, the measurement of learners’ mental models is also described with its benefits on the provision of feedback on individual learning progression. The framework provides solid directions for research and development for the improvement of English writing.

Selection of Team Interventions Based on Mental Model Sharedness Levels Measured by the Team Assessment and Diagnostic Instrument (TADI)

Researchers have claimed that successful team performance depends on shared mental models. While there are a number of techniques that have been employed to measure shared knowledge, Johnson and colleagues (2007) developed and validated an instrument for measuring team-related knowledge. This chapter focuses on the application of the Team Assessment and Diagnostic Instrument (TADI). Using the results of this five-factor model (including general task and team knowledge, general task and communication skills, attitude toward teammates and task, team dynamics and interactions, and team resources and working environment), TADI is used to assess the current state of team alignment with respect to the five team-related knowledge factors. Based on the alignment and degree of response, this measure can be used to assess the level of team synergy as well as determine misalignment in specific areas of teammates’ mental models. With this information, team members, leaders, and coaches can better anticipate team problems thereby guiding the selection of team performance interventions ultimately mitigating team problems and improving team learning and performance.

Effects of an instructional gaming characteristic on learning effectiveness, efficiency, and engagement: using a storyline for teaching basic statistical skills

The study explored instructional benefits of a storyline gaming characteristic (GC) on learning effectiveness, efficiency, and engagement with the use of an online instructional simulation for graduate students in an introductory statistics course. A storyline is a game-design element that connects scenes with the educational content. In order to examine the interactions between the storyline GC and human performance, a storyline was embedded in a simulation. The goal of the simulation was to engage students in problem-solving and data analysis in the context of basic statistics by using real-world examples. The authors developed two different versions of the simulation: (1) Simulation+No GC, and (2) Simulation+Storyline GC. Both versions shared the same instructional content but differed in the presence or absence of a storyline GC. The results indicated that adding a storyline to a simulation did not result in significant improvements in learning effectiveness, efficiency, or engagement. However, both instructional methods (simulation and simulation with a storyline) showed significant learning gains from pre- to post-test. The findings of this study offer future directions for embedding a storyline GC into learning content.

Assessment of Student’s Emotions in Game-Based Learning

Research has shown that emotions are directly linked to cognition and there is a strong correlation between affect and learning. This notion along with recent technological advancements has prompted researchers from many disciplines to turn their attention toward adding an affective component to human-computer dialog. This chapter discusses emotion assessment methods, recent empirical research related to examining students’ affective states in entertainment and educational games, and conceptual, methodological, and technological issues associated with developing emotion recognition models. An overview of emotion recognition research suggests that there is little consensus on what emotions should be measured and how to do it. Moreover, it is still not clear how emotions affect human learning and performance.

Implementation of an Online Social Annotation Tool in a College English Course

An online social annotation tool was implemented in the context of utilizing question-answering tasks with reading documents. The tool and tasks were used in order to foster students’ cognitive development with higher-order thinking, critical analysis, and development of sophisticated arguments in English writing. The effects of the tool on students’ mental models as well as their motivation for and achievement in a college argument and persuasion course were investigated. The findings are discussed along with implications and possibilities for future studies.

Delving into alumni perceptions about the impact and effectiveness of two certificate programs: meeting their mission?

The purpose of the study was to analyze two certificate programs in regard to the impacts on alumni professional career and strengths and weaknesses of certificate programs in the views of their alumni. The sample consisted of 58 participants who completed one of the certificate programs. The results showed that alumni rated self-improvement as the biggest benefit, career advancement benefit as average, and career change benefit as low from the certificate programs. Also, alumni thought that all program components were of strong quality, but the majority of alumni still wanted to see an increased emphasis on teaching, interaction with other students, support, and assessment feedback focus of the program.