Anne M. Sinatra

Designing Adaptive Instruction for Teams: a Meta-Analysis

The goal of this research was the development of a practical architecture for the computer-based tutoring of teams. This article examines the relationship of team behaviors as antecedents to successful team performance and learning during adaptive instruction guided by Intelligent Tutoring Systems (ITSs). Adaptive instruction is a training or educational experience tailored by artificially-intelligent, computer-based tutors with the goal of optimizing learner outcomes (e.g., knowledge and skill acquisition, performance, enhanced retention, accelerated learning, or transfer of skills from instructional environments to work environments). The core contribution of this research was the identification of behavioral markers associated with the antecedents of team performance and learning thus enabling the development and refinement of teamwork models in ITS architectures. Teamwork focuses on the coordination, cooperation, and communication among individuals to achieve a shared goal. For ITSs to optimally tailor team instruction, tutors must have key insights about both the team and the learners on that team. To aid the modeling of teams, we examined the literature to evaluate the relationship of teamwork behaviors (e.g., communication, cooperation, coordination, cognition, leadership/coaching, and conflict) with team outcomes (learning, performance, satisfaction, and viability) as part of a large-scale meta-analysis of the ITS, team training, and team performance literature. While ITSs have been used infrequently to instruct teams, the goal of this meta-analysis make team tutoring more ubiquitous by: identifying significant relationships between team behaviors and effective performance and learning outcomes; developing instructional guidelines for team tutoring based on these relationships; and applying these team tutoring guidelines to the Generalized Intelligent Framework for Tutoring (GIFT), an open source architecture for authoring, delivering, managing, and evaluating adaptive instructional tools and methods. In doing this, we have designed a domain-independent framework for the adaptive instruction of teams.

Creating a Team Tutor Using GIFT

With the movement in education towards collaborative learning, it is becoming more important that learners be able to work together in groups and teams. Intelligent tutoring systems (ITSs) have been used successfully to teach individuals, but so far only a few ITSs have been used for the purpose of training teams. This is due to the difficulty of creating such systems. An ITS for teams must be able to assess complex interactions between team members (team skills) as well as the way they interact with the system itself (task skills). Assessing team skills can be difficult because they contain social components such as communication and coordination that are not readily quantifiable. This article addresses these difficulties by developing a framework to guide the authoring process for team tutors. The framework is demonstrated using a case study about a particular team tutor that was developed using a military surveillance scenario for teams of two. The Generalized Intelligent Framework for Tutoring (GIFT) software provided the team tutoring infrastructure for this task. A new software architecture required to support the team tutor is described. This theoretical framework and the lessons learned from its implementation offer conceptual scaffolding for future authors of ITSs.

The Challenges of Building Intelligent Tutoring Systems for Teams

Intelligent Tutoring Systems have been useful for individual instruction and training, but have not been widely created for teams, despite the widespread use of team training and learning in groups. This paper reviews two projects that developed team tutors: the Team Multiple Errands Task (TMET) and the Recon Task developed using the Generalized Intelligent Framework for Tutoring (GIFT). Specifically, this paper 1) analyzes why team tasks have significantly more complexity than an individual task, 2) describes the two team-based platforms for team research, and 3) explores the complexities of team tutor authoring. Results include a recommended process for authoring a team intelligent tutoring system based on our lessons learned that highlights the differences between tutors for individuals and team tutors.

A Personalized GIFT: Recommendations for Authoring Personalization in the Generalized Intelligent Framework for Tutoring

Personalization of learning content can have a positive impact on learning in a computer based environment. Personalization can occur in a number of different ways, such as including an individual’s name or entered content throughout the learning materials, or selecting examples based on self-reported preferences. The Generalized Intelligent Framework for Tutoring (GIFT) is an open-source, domain independent intelligent tutoring system framework. GIFT includes a number of different authoring tools (e.g., GIFT Authoring Tool, Survey Authoring System) that can be used to generate adaptive courses. In its current form, GIFT does not have specific mechanisms to support personalization of materials to the individual user based on pre-entered preferences. The current paper describes ways that personalization research has previously been conducted with GIFT. The paper additionally provides recommendations on new features that could be added to GIFT’s authoring tools in order to support personalizing learning materials, guidance, and surveys that are provided to the learner.

Operationalizing the C’s of Teamwork in an Intelligent Tutoring System

One of the difficulties in creating a team-focused intelligent tutoring system (ITS) is defining the measures used to assess the team’s performance. While the team research literature offers nine C’s of teamwork to consider, e.g., cooperation, communication, etc., it can also be difficult to implement these in real-world practice. This paper reviews the approach used in three team ITSs in which the C’s were used, offering guidance for future implementation of team tutors.

The Generalized Intelligent Framework for Tutoring (GIFT) as a Tool for Human Factors Professionals

The Generalized Intelligent Framework for Tutoring (GIFT) is an open-source domain-independent intelligent tutoring system (ITS) framework. It provides the tools and capabilities to design a complete ITS, manage student instruction, and serve as a testbed for experimentation and analysis. GIFT was developed to provide a cost-effective and efficient means for developing ITSs that have interchangeable parts and can be reused. Additionally, GIFT provides the ability for human factors professionals to develop and run complete experiments in a computer-based environment. This demonstration will provide an introduction to GIFT, along with the motivation behind its design. Further, a demonstration of the tools that are useful to human factors practitioners and psychologists will be provided, and examples of courses that have been developed using GIFT will be shown.

Leveraging Cognitive Psychology Principles to Enhance Adaptive Instruction

Intelligent Tutoring Systems (ITSs) can be used for computer-based adaptive instruction that can be utilized in many ways including both in the classroom and on a student’s own time. ITSs can be particularly useful for remediation and confirming that a student fully understands a topic that is important in an educational course. As many individuals will be using ITSs on their own time, and it is a unique opportunity to customize to an individual, it is helpful to design the material that is being delivered to the student to be as memorable as possible. There are numerous strategies and theories within cognitive psychology that have been heavily researched, and lead to improved memory, and retention when put into place. The current paper discusses how these cognitive psychology strategies can be leveraged and utilized within ITSs in order to lead to improved outcomes. Additionally, there are suggestions on how to incorporate these strategies within ITSs.

Me and My VE, Part 5: Applications in Human Factors Research and Practice

Recent advances in technology have made virtual environments, virtual reality, augmented reality, and simulations more affordable and accessible to researchers, companies, and the general public, which has led to many novel use cases and applications. A key objective of human factors research and practice is determining how these technology-rich applications can be designed and applied to improve human performance across a variety of contexts. This session will demonstrate some of the distinct and diverse uses of virtual environments and mixed reality environments in an alternative format. The session will begin with each demonstrator providing a brief overview of their virtual environment (VE) and a description of how it has been used to address a particular problem or research need. Following the description portion of the session, each VE will be set-up at a demonstration station in the room, and session attendees will be encouraged to directly interact with the virtual environment and ask demonstrators questions about their research and inquire about the effectiveness of using VE for research, training, and evaluation purposes. The overall objective of this alternative session is to increase the awareness of how human factors professionals use VE technologies and increase the awareness of the capabilities and limitations of VE in supporting the work of HF professionals.

Feedback Design Considerations for Intelligent Team Tutoring Systems

Challenges arise when developing a computer-based Intelligent Team Tutoring System (ITTS) that attempts to deliver feedback to teams as effectively as a human tutor. The purpose of this current work is to outline elements of feedback that should be considered when designing feedback for an ITTS. The authors present the results of a study that consisted of 32 participants grouped into 16 teams of two. Each team conducted a surveillance task where they received individual or team feedback. Feedback content was written using either the bald (direct feedback; no need for interpretation) or off-record (general feedback; interpretation needed) etiquette strategy. The results showed that feedback delivered using the bald etiquette strategy positively correlated with improved performance. The results also showed that team level feedback positively correlated with more accurate self-assessment among participants. This suggests that in an ITTS, direct feedback can lead to better performance, and that feedback provided at the team level can help to align self-interpretation of performance with actual task performance.

Recommendations for Use of Adaptive Tutoring Systems in the Classroom and in Educational Research

The current paper and presentation provide background on the different uses of intelligent tutoring systems (ITSs) in context of course instruction, discusses specific instructor considerations that are associated with their use, and ways to use ITSs for educational research. Instructor considerations include the time necessary to plan prior to constructing an ITS, the process of constructing ITS lessons for use by students, the method in which students will interact with the ITS, approaches to incorporating ITS use into classes, and the information that instructors would find useful to be output from the ITS. Specifically, the Generalized Intelligent Framework for Tutoring (GIFT), an open-source, domain independent ITS framework will be discussed as an approach to creating adaptive tutoring content for classroom use. GIFT includes straightforward authoring tools for instructors and Subject Matter Experts (SMEs). These authoring tools are powerful, do not require a background in computer science to use, and result in fully adaptive computer-based lessons. Additionally, GIFT provides the flexibility for instructors to bring their pre-generated and already existing instructional material to the system and use it to create ITS lessons. The authoring tools allow the instructor to determine the path of their lesson and the components that their students will experience (i.e. surveys, quizzes, lesson materials, videos). The paper includes details about the development of an instructor dashboard in GIFT, ways for an instructor to use GIFT for educational research, and a discussion of general output information from ITSs that would be relevant to instructors.