Travis J. Wiltshire

Shifting the paradigm of music instruction: implications of embodiment stemming from an augmented reality guitar learning system.

Musical instruction often includes materials that can act as a barrier to learning. New technologies using augmented reality may aid in reducing the initial difficulties involved in learning music by lowering these barriers characteristic of traditional instructional materials. Therefore, this set of studies examined a novel augmented reality guitar learning system (i.e., the Fretlight® guitar) in regards to current theories of embodied music cognition. Specifically, we examined the effects of using this system in comparison to a standard instructional material (i.e., diagrams). First, we review major theories related to musical embodiment and specify a niche within this research space we call embodied music technology for learning. Following, we explicate two parallel experiments that were conducted to address the learning effects of this system. Experiment 1 examined short-term learning effects within one experimental session, while Experiment 2 examined both short-term and long-term effects across two sessions spaced at a 2-week interval. Analyses demonstrated that, for many of our dependent variables, all participants increased in performance across time. Further, the Fretlight® condition consistently led to significantly better outcomes via interactive effects, including significantly better long term retention for the learned information across a 2 week time interval. These results are discussed in the context of embodied cognition theory as it relates to music. Potential limitations and avenues for future research are described.

Toward understanding social cues and signals in human–robot interaction: effects of robot gaze and proxemic behavior

As robots are increasingly deployed in settings requiring social interaction, research is needed to examine the social signals perceived by humans when robots display certain social cues. In this paper, we report a study designed to examine how humans interpret social cues exhibited by robots. We first provide a brief overview of perspectives from social cognition in humans and how these processes are applicable to human–robot interaction (HRI). We then discuss the need to examine the relationship between social cues and signals as a function of the degree to which a robot is perceived as a socially present agent. We describe an experiment in which social cues were manipulated on an iRobot AvaTM mobile robotics platform in a hallway navigation scenario. Cues associated with the robot’s proxemic behavior were found to significantly affect participant perceptions of the robot’s social presence and emotional state while cues associated with the robot’s gaze behavior were not found to be significant. Further, regardless of the proxemic behavior, participants attributed more social presence and emotional states to the robot over repeated interactions than when they first interacted with it. Generally, these results indicate the importance for HRI research to consider how social cues expressed by a robot can differentially affect perceptions of the robot’s mental states and intentions. The discussion focuses on implications for the design of robotic systems and future directions for research on the relationship between social cues and signals.

Prospects for direct social perception: a multi-theoretical integration to further the science of social cognition

In this paper we suggest that differing approaches to the science of social cognition mirror the arguments between radical embodied and traditional approaches to cognition. We contrast the use in social cognition of theoretical inference and mental simulation mechanisms with approaches emphasizing a direct perception of others’ mental states. We build from a recent integrative framework unifying these divergent perspectives through the use of dual-process theory and supporting social neuroscience research. Our elaboration considers two complementary notions of direct perception: one primarily stemming from ecological psychology and the other from enactive cognition theory. We use this as the foundation from which to offer an account of the informational basis for social information and assert a set of research propositions to further the science of social cognition. In doing so, we point out how perception of the minds of others can be supported in some cases by lawful information, supporting direct perception of social affordances and perhaps, mental states, and in other cases by cues that support indirect perceptual inference. Our goal is to extend accounts of social cognition by integrating advances across disciplines to provide a multi-level and multi-theoretic description that can advance this field and offer a means through which to reconcile radical embodied and traditional approaches to cognitive neuroscience.

Social Cognitive and Affective Neuroscience in Human–Machine Systems: A Roadmap for Improving Training, Human–Robot Interaction, and Team Performance

This paper augments recent advances in social cognitive and affective neuroscience (SCAN) and illustrates their relevance to the development of novel human-machine systems. Advances in this area are crucial for understanding and exploring the social, cognitive, and neural processes that arise during human interactions with complex sociotechnological systems. Overviews of the major areas of SCAN research, including emotion, theory of mind, and joint action, are provided as the basis for describing three applications of SCAN to human-machine systems research and development. Specifically, this paper provides three examples to demonstrate the broad interdisciplinary applicability of SCAN and the ways it can contribute to improving a number of human-machine systems with the pursuit of further research in this vein. These include applying SCAN to learning and training, informing the field of human-robot interaction (HRI), and, finally, for enhancing team performance. The goal is to draw attention to the insights that can be gained by integrating SCAN with ongoing human-machine system research and to provide guidance to foster collaborations of this nature. Toward this end, we provide a systematic set of notional research questions for each detailed application within the context of the three major emphases of SCAN research. In turn, this study serves as a roadmap for preliminary investigations that integrate SCAN and human-machine system research.

Towards Modeling Social-Cognitive Mechanisms in Robots to Facilitate Human-Robot Teaming

For effective human-robot teaming, robots must gain the appropriate social-cognitive mechanisms that allow them to function naturally and intuitively in social interactions with humans. However, there is a lack of consensus on social cognition broadly, and how to design such mechanisms for embodied robotic systems. To this end, recommendations are advanced that are drawn from HRI, psychology, robotics, neuroscience and philosophy as well as theories of embodied cognition, dual process theory, ecological psychology, and dynamical systems. These interdisciplinary and multi-theoretic recommendations are meant to serve as integrative and foundational guidelines for the design of robots with effective social-cognitive mechanisms.

Technology as Teammate: Examining the Role of External Cognition in Support of Team Cognitive Processes

In this paper we advance team theory by describing how cognition occurs across the distribution of members and the artifacts and technology that support their efforts. We draw from complementary theorizing coming out of cognitive engineering and cognitive science that views forms of cognition as external and extended and integrate this with theorizing on macrocognition in teams. Two frameworks are described that provide the groundwork for advancing theory and aid in the development of more precise measures for understanding team cognition via focus on artifacts and the technologies supporting their development and use. This includes distinctions between teamwork and taskwork and the notion of general and specific competencies from the organizational sciences along with the concepts of offloading and scaffolding from the cognitive sciences. This paper contributes to the team cognition literature along multiple lines. First, it aids theory development by synthesizing a broad set of perspectives on the varied forms of cognition emerging in complex collaborative contexts. Second, it supports research by providing diagnostic guidelines to study how artifacts are related to team cognition. Finally, it supports information systems designers by more precisely describing how to conceptualize team-supporting technology and artifacts. As such, it provides a means to more richly understand process and performance as it occurs within sociotechnical systems. Our overarching objective is to show how team cognition can both be more clearly conceptualized and more precisely measured by integrating theory from cognitive engineering and the cognitive and organizational sciences.

Leveraging Social Judgment Theory to Examine the Relationship between Social Cues and Signals in Human-Robot Interactions

Human-robot interaction (HRI) research requires new techniques for understanding the social dynamics that occur at the interface between humans and robots. Prior work has focused on incorporating the social cues and social signals distinction from the field of social signal processing and complementing this with recent advances in understanding human social cognition that specify two primary types of cognitive processes. A related account, stemming from Social Judgment Theory (SJT), specifies a Lens Model for which cues can be interpreted as well as the task conditions that would induce either of the types of cognitive processes. Surprisingly, SJT-based research has not yet examined the social cue and signal relationship. We argue it provides an ideal path forward for such research and we integrate these related disciplines of study to provide a theoretically derived account that can be useful for both the design of humanhuman and HRI experiments focused on social interaction dynamics.

Complex collaborative problem-solving processes in mission control.

INTRODUCTION NASAs Mission Control Center (MCC) is responsible for control of the International Space Station (ISS), which includes responding to problems that obstruct the functioning of the ISS and that may pose a threat to the health and well-being of the flight crew. These problems are often complex, requiring individuals, teams, and multiteam systems, to work collaboratively. Research is warranted to examine individual and collaborative problem-solving processes in this context. Specifically, focus is placed on how Mission Control personnel-each with their own skills and responsibilities-exchange information to gain a shared understanding of the problem. The Macrocognition in Teams Model describes the processes that individuals and teams undertake in order to solve problems and may be applicable to Mission Control teams. METHOD Semistructured interviews centering on a recent complex problem were conducted with seven MCC professionals. In order to assess collaborative problem-solving processes in MCC with those predicted by the Macrocognition in Teams Model, a coding scheme was developed to analyze the interview transcriptions. RESULTS Findings are supported with excerpts from participant transcriptions and suggest that team knowledge-building processes accounted for approximately 50% of all coded data and are essential for successful collaborative problem solving in mission control. Support for the internalized and externalized team knowledge was also found (19% and 20%, respectively). DISCUSSION The Macrocognition in Teams Model was shown to be a useful depiction of collaborative problem solving in mission control and further research with this as a guiding framework is warranted.

Effects of Robot Gaze and Proxemic Behavior on Perceived Social Presence during a Hallway Navigation Scenario

Robots are increasingly being introduced into task environments that require the ability to exhibit appropriate social functionality. The present study is an examination of how social cues conveyed by a robot, during a brief interaction, affect the perception of the robot as a socially present agent. Participants were exposed to one of three gaze conditions and two proxemic behavioral programs during a number of experimental trials involving path-crossing in a hallway setting. Results indicated that participants perceived the robot as more socially present when it exhibited a passive proxemic behavior and more socially present over time; though, findings varied at the sub-scale level. Design recommendations are presented for roboticists.

Applications of Cognitive Transformation Theory: Examining the Role of Sensemaking in the Instruction of Air Traffic Control Students

Complex domains require cognitive work for which current approaches to training may be ill-suited. To improve training for cognitive work, Klein and Baxter have proposed Cognitive Transformation Theory (CTT), a learning theory that characterizes sensemaking processes as essential to the development of expertise. The objectives of this research were to compare CTT with the instructional strategies of two expert air traffic control instructors to evaluate the relevance of CTT’s four teaching practices, propose refinements to CTT, and identify potential instructional strategies to serve as guidance for the application of CTT. Data were collected using cognitive task analysis methods, including course observation, artifact examination, and knowledge elicitation with two instructors and seven of their students. Data were coded using categories derived from theory and patterns emergent within the data. Results suggest that many of the instructional strategies used were consistent with the teaching practices of CTT and that learning was aligned with the active sensemaking claims of CTT. An integrated set of instructional strategies and a few refinements to CTT are advanced to further its application to training in complex domains. Although this set of strategies may benefit current training practices, further research is needed to evaluate their effectiveness.