Stefanos Nikolaidis

Human-robot cross-training: computational formulation, modeling and evaluation of a human team training strategy

We design and evaluate human-robot cross-training, a strategy widely used and validated for effective human team training. Cross-training is an interactive planning method in which a human and a robot iteratively switch roles to learn a shared plan for a collaborative task. We first present a computational formulation of the robots interrole knowledge and show that it is quantitatively comparable to the human mental model. Based on this encoding, we formulate human-robot cross-training and evaluate it in human subject experiments (n = 36). We compare human-robot cross-training to standard reinforcement learning techniques, and show that cross-training provides statistically significant improvements in quantitative team performance measures. Additionally, significant differences emerge in the perceived robot performance and human trust. These results support the hypothesis that effective and fluent human-robot teaming may be best achieved by modeling effective practices for human teamwork.

Optimization of Temporal Dynamics for Adaptive Human-Robot Interaction in Assembly Manufacturing.

Human-robot collaboration presents an opportunity to improve the efficiency of manufacturing and assembly processes, particularly for aerospace manufacturing where tight integration and variability in the build process make physical isolation of robotic-only work challenging. In this paper, we develop a robotic scheduling and control capability that adapts to the changing preferences of a human co-worker or supervisor while providing strong guarantees for synchronization and timing of activities. This innovation is realized through dynamic execution of a flexible optimal scheduling policy that accommodates temporal disturbance. We describe the Adaptive Preferences Algorithm that computes the flexible scheduling policy and show empirically that execution is fast, robust, and adaptable to changing preferences for workflow. We achieve satisfactory computation times, on the order of seconds for moderatelysized problems, and demonstrate the capability for human-robot teaming using a small industrial robot.

Efficient Model Learning from Joint-Action Demonstrations for Human-Robot Collaborative Tasks

We present a framework for learning human user models from joint-action demonstrations that enables the robot to compute a robust policy for a collaborative task with a human. The learning takes place completely automatically, without any human intervention. First, we describe the clustering of demonstrated action sequences into different human types using an unsupervised learning algorithm. These demonstrated sequences are also used by the robot to learn a reward function that is representative for each type, through the employment of an inverse reinforcement learning algorithm. The learned model is then used as part of a Mixed Observability Markov Decision Process formulation, wherein the human type is a partially observable variable. With this framework, we can infer, either offline or online, the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this new user and will be robust to deviations of the human actions from prior demonstrations. Finally we validate the approach using data collected in human subject experiments, and conduct proof-of-concept demonstrations in which a person performs a collaborative task with a small industrial robot.We present a framework for automatically learning human user models from joint-action demonstrations that enables a robot to compute a robust policy for a collaborative task with a human. First, the demonstrated action sequences are clustered into different human types using an unsupervised learning algorithm. A reward function is then learned for each type through the employment of an inverse reinforcement learning algorithm. The learned model is then incorporated into a mixed-observability Markov decision process (MOMDP) formulation, wherein the human type is a partially observable variable. With this framework, we can infer online the human type of a new user that was not included in the training set, and can compute a policy for the robot that will be aligned to the preference of this user. In a human subject experiment (n=30), participants agreed more strongly that the robot anticipated their actions when working with a robot incorporating the proposed framework (p<0.01), compared to manually annotating robot actions. In trials where participants faced difficulty annotating the robot actions to complete the task, the proposed framework significantly improved team efficiency (p <0.01). The robot incorporating the framework was also found to be more responsive to human actions compared to policies computed using a hand-coded reward function by a domain expert (p<0.01). These results indicate that learning human user models from joint-action demonstrations and encoding them in a MOMDP formalism can support effective teaming in human-robot collaborative tasks.

Human-robot mutual adaptation in collaborative tasks: Models and experiments

Adaptation is critical for effective team collaboration. This paper introduces a computational formalism for mutual adaptation between a robot and a human in collaborative tasks. We propose the Bounded-Memory Adaptation Model, which is a probabilistic finite-state controller that captures human adaptive behaviors under a bounded-memory assumption. We integrate the Bounded-Memory Adaptation Model into a probabilistic decision process, enabling the robot to guide adaptable participants towards a better way of completing the task. Human subject experiments suggest that the proposed formalism improves the effectiveness of human-robot teams in collaborative tasks, when compared with one-way adaptations of the robot to the human, while maintaining the human’s trust in the robot.

Improved human-robot team performance through cross-training, an approach inspired by human team training practices

We design and evaluate a method of human–robot cross-training, a validated and widely used strategy for the effective training of human teams. Cross-training is an interactive planning method in which team members iteratively switch roles with one another to learn a shared plan for the performance of a collaborative task. We first present a computational formulation of the robot mental model, which encodes the sequence of robot actions necessary for task completion and the expectations of the robot for preferred human actions, and show that the robot model is quantitatively comparable to the mental model that captures the inter-role knowledge held by the human. Additionally, we propose a quantitative measure of robot mental model convergence and an objective metric of model similarity. Based on this encoding, we formulate a human–robot cross-training method and evaluate its efficacy through experiments involving human subjects ( n = 60 ) . We compare human–robot cross-training to standard reinforcement learning techniques, and show that cross-training yields statistically significant improvements in quantitative team performance measures, as well as significant differences in perceived robot performance and human trust. Finally, we discuss the objective measure of robot mental model convergence as a method to dynamically assess human errors. This study supports the hypothesis that the effective and fluent teaming of a human and a robot may best be achieved by modeling known, effective human teamwork practices.

Game-Theoretic Modeling of Human Adaptation in Human-Robot Collaboration

In human-robot teams, humans often start with an inaccurate model of the robot capabilities. As they interact with the robot, they infer the robots capabilities and partially adapt to the robot, i.e., they might change their actions based on the observed outcomes and the robots actions, without replicating the robots policy. We present a game-theoretic model of human partial adaptation to the robot, where the human responds to the robots actions by maximizing a reward function that changes stochastically over time, capturing the evolution of their expectations of the robots capabilities. The robot can then use this model to decide optimally between taking actions that reveal its capabilities to the human and taking the best action given the information that the human currently has. We prove that under certain observability assumptions, the optimal policy can be computed efficiently. We demonstrate through a human subject experiment that the proposed model significantly improves human-robot team performance, compared to policies that assume complete adaptation of the human to the robot.

Formalizing Human-Robot Mutual Adaptation: A Bounded Memory Model

Mutual adaptation is critical for effective team collaboration. This paper presents a formalism for human-robot mutual adaptation in collaborative tasks. We propose the bounded-memory adaptation model (BAM), which captures human adaptive behaviors based on a bounded memory assumption. We integrate BAM into a partially observable stochastic model, which enables robot adaptation to the human. When the human is adaptive, the robot will guide the human towards a new, optimal collaborative strategy unknown to the human in advance. When the human is not willing to change their strategy, the robot adapts to the human in order to retain human trust. Human subject experiments indicate that the proposed formalism can significantly improve the effectiveness of human-robot teams, while human subject ratings on the robot performance and trust are comparable to those achieved by cross training, a state-of-the-art human-robot team training practice.

Human-Robot Interactive Planning using Cross-Training: A Human Team Training Approach

Robots are increasingly introduced to work in concert with people in high-intensity domains, such as manufacturing, space exploration and hazardous environments. Although there are numerous studies on human teamwork and coordination in these settings, very little prior work exists on applying these models to human-robot interaction. In this paper we propose a novel framework for applying prior art in Shared Mental Models (SMMs) to promote eective human-robot teaming. We present a computational teaming model to encode joint action in a human-robot team. We present results from human subject experiments that evaluate human-robot teaming in a virtual environment. We show that cross-training, a common practice used for improving human team shared mental models, yields statistically signicant improvements in convergence of the computational teaming model (p=0.02) and in the human participants’ perception that the robot performed according to their preferences (p=0.01), as compared to robot training using a standard interactive reinforcement learning approach.

Developing an Adaptive Robotic Assistant for Close Proximity Human-Robot Collaboration in Space

In this paper, we present a framework for an adaptive and risk-aware robot motion planning and control, and discuss how such a framework could handle uncertainty in human workers’ actions and robot localization. We build on our prior investigation, where we describe how uncertainty in human actions can be modeled using the entropy rate in a Markov Decision Process. We then describe how we can incorporate this model of uncertainty into simulations of a simple collaborative system, involving one human worker and one robotic assistant, to produce risk-aware robot motions. Next, we highlight the difficulties associated with localization uncertainty in a space environment and describe how we can incorporate this uncertainty into an adaptive system as well. Expected advantages of an adaptive system are described, including increases in overall efficiency due to reductions in idle time, increases in concurrent motion, faster task execution, as well as subjective improvements in the worker’s satisfaction with the assistant and reduced worker stress and fatigue. A pilot experiment designed to evaluate the benefits of introducing risk-aware motion planning is described. It is found that human-robot teams in which the robot utilizes risk-aware motion planning show on average 24% more concurrent motion and execute the task 13% faster, while simultaneously improving safety by having a 19.9% larger mean separation distance between the human and robot workers. Finally, possible future system developments and user studies are discussed.

Planning with Trust for Human-Robot Collaboration

Trust is essential for human-robot collaboration and user adoption of autonomous systems, such as robot assistants. This paper introduces a computational model which integrates trust into robot decision-making. Specifically, we learn from data a partially observable Markov decision process (POMDP) with human trust as a latent variable. The trust-POMDP model provides a principled approach for the robot to (i) infer the trust of a human teammate through interaction, (ii) reason about the effect of its own actions on human behaviors, and (iii) choose actions that maximize team performance over the long term. We validated the model through human subject experiments on a table-clearing task in simulation (201 participants) and with a real robot (20 participants). The results show that the trust-POMDP improves human-robot team performance in this task. They further suggest that maximizing trust in itself may not improve team performance.