Andrew S. Clare

The Role of Human-Automation Consensus in Multiple Unmanned Vehicle Scheduling

Objective: This study examined the impact of increasing automation replanning rates on operator performance and workload when supervising a decentralized network of heterogeneous unmanned vehicles. Background: Futuristic unmanned vehicles systems will invert the operator-to-vehicle ratio so that one operator can control multiple dissimilar vehicles connected through a decentralized network. Significant human-automation collaboration will be needed because of automation brittleness, but such collaboration could cause high workload. Method: Three increasing levels of replanning were tested on an existing multiple unmanned vehicle simulation environment that leverages decentralized algorithms for vehicle routing and task allocation in conjunction with human supervision. Results: Rapid replanning can cause high operator workload, ultimately resulting in poorer overall system performance. Poor performance was associated with a lack of operator consensus for when to accept the automation’s suggested prompts for new plan consideration as well as negative attitudes toward unmanned aerial vehicles in general. Participants with video game experience tended to collaborate more with the automation, which resulted in better performance. Conclusion: In decentralized unmanned vehicle networks, operators who ignore the automation’s requests for new plan consideration and impose rapid replans both increase their own workload and reduce the ability of the vehicle network to operate at its maximum capacity. Application: These findings have implications for personnel selection and training for futuristic systems involving human collaboration with decentralized algorithms embedded in networks of autonomous systems.

Operator Objective Function Guidance for a Real-Time Unmanned Vehicle Scheduling Algorithm

Abstract : Advances in autonomy have made it possible to invert the typical operator-to-unmanned-vehicle ratio so that a single operator can now control multiple heterogeneous unmanned vehicles. Algorithms used in unmanned-vehicle path planning and task allocation typically have an objective function that only takes into account variables initially identified by designers with set weightings. This can make the algorithm seemingly opaque to an operator and brittle under changing mission priorities. To address these issues, it is proposed that allowing operators to dynamically modify objective function weightings of an automated planner during a mission can have performance benefits. A multiple-unmanned-vehicle simulation test bed was modified so that operators could either choose one variable or choose any combination of equally weighted variables for the automated planner to use in evaluating mission plans. Results from a human-participant experiment showed that operators rated their performance and confidence highest when using the dynamic objective function with multiple objectives. Allowing operators to adjust multiple objectives resulted in enhanced situational awareness, increased spare mental capacity, fewer interventions to modify the objective function, and no significant differences in mission performance. Adding this form of flexibility and transparency to automation in future unmanned vehicle systems could improve performance, engender operator trust, and reduce errors.

Mixed-initiative strategies for real-time scheduling of multiple unmanned vehicles

Advances in autonomy have made it possible to invert the typical operator-to-unmanned vehicle ratio so that a single operator can now control multiple heterogeneous Unmanned Vehicles (UVs). Real-time scheduling and task assignment for multiple UVs in uncertain environments will require the computational ability of optimization algorithms combined with the judgment and adaptability of human supervisors through mixed-initiative systems. The goal of this paper is to analyze the interactions between operators and scheduling algorithms in two human- in-the-loop multiple UV control experiments. The impact of real-time operator modifications to the objective function of an optimization algorithm for multi-UV scheduling is described. Results from outdoor multiple UV flight tests using a human-computer collaborative scheduling system are presented, which provide valuable insight into the impact of environmental uncertainty and vehicle failures on system effectiveness.

Flight Testing a Heterogeneous Multi-UAV System with Human Supervision

This paper presents the outdoor ight test results of a decentralized multi-UAV system supervised by a human operator. The system balances the roles of the human operator and the UAV autonomous behaviors with the objective of maximizing the execution performance. The operator manages the mission by inputting and modifying tasks instead of controlling individual UAVs. The Consensus-Based Bundle Algorithm (CBBA) is used as a real-time, scalable, dynamic multi-agent multi-task planning algorithm to allocate tasks approved by the operator to UAVs. A team of three quadrotors and one xed wing UAV collaborated in an operationally relevant scenario supporting a cargo UAV resupply mission. Thirteen of fourteen multi-UAV outdoor ight test trials successfully accomplished the mission objectives. The framework was shown to be robust to system failures and degradations commonly encountered during eld testing primarily because of health monitoring and management tools that were incorporated in the design. Instances of task allocation and path planning churning were observed which are linked to uncertainties of operating outdoors. Lessons learned during ight test operations are highlighted as they are relevant to other similar types of systems and missions.

Assessing Operator Workload and Performance in Expeditionary Multiple Unmanned Vehicle Control

A future concept of operations for co ntrolling unmanned vehicles has a single, forwarddeployed soldier supervising multiple, heterogeneous unmanned vehicles. This operator would collaborate with an automated planner that presents the operator with a choice of potential plans for how the unmanned vehicles could efficiently search the dynamic environment for new targets, track identified targets, and destroy hostile targets. A key step towards this future concept is to determine how often the automated planner should provide the operator with potential plans without causing a detrimentally high workload for the operator, which could have negative performance consequences. This research describes the development of the Onboard Planning System for Unmanned Vehicles Supporting Expeditionary Reconnaissance and Surveillance (OPS-USERS) testbed and an experiment to measure operator workload and performance in a set of scenarios with different replanning intervals. The results show that the replan interval has a significant effect on workload and performance with significantly lower performance at the lowest replan interval . Replanning too frequently in a human-on-the-loop collaboration can cause higher levels of workload and lower performance. Further research is required to determine the impact o f low workload in a highly autonomous system on overall performance. Also, operator strategy and trust in automation with the OPS-USERS system deserves further research.

Holistic Modeling for Human-Autonomous System Interaction

For complex systems that embed automation, but also rely on human interaction for guidance and contingency management, holistic models are needed that provide for an understanding of the individual human and computer elements, and address the critical interactions of such complex systems. Discrete event simulation (DES) models and system dynamics (SD) models are two different approaches that can be used to address these requirements. Both modelling approaches can support the designers of future autonomous vehicle (AV) systems by simulating the impact of alternate designs on vehicle, operator, and system performance. However, the DES modelling approach is likely best suited for using probabilistic distributions to accurately model an operator who is a serial processor of discrete tasks, as well as an environment with randomly occurring events. The SD modelling approach is better suited for modelling continuous performance feedback that is temporally dependent and is affected by qualitative variables such as trust.

Innovative Systems for Human Supervisory Control of Unmanned Vehicles

The development of Unmanned Vehicles (UVs) with increasing autonomy has enabled a transition from teleoperation to Human Supervisory Control (HSC). In this demonstration, participants can test three innovative operator interfaces for HSC of UVs. The first system allows users to control a Micro Air Vehicle (MAV) via a hand-held device, such as an iPhone®, through high-level waypoint commands and fine-grained nudge controls. The second system enables a single operator to collaborate with an automated planner to control multiple heterogeneous UVs from a laptop-sized display for searching for, tracking and engaging moving ground targets. The third system is designed to aid in planning on naval aircraft carrier decks, serving as a decision support tool for a supervisor overseeing and scheduling the activity of people, vehicles and unmanned vehicles working in this complex and uncertain environment.

Identifying Suitable Algorithms for Human-Computer Collaborative Scheduling of Multiple Unmanned Vehicles

Real-time scheduling and task assignment for multiple Unmanned Vehicles (UVs) in uncertain environments will require the computational ability of optimization algorithms combined with the judgment and adaptability of human supervisors. Identifying the characteristics that make a scheduling algorithm suitable for human-computer collaboration is essential for the development of an effective scheduling system. This high-level systems analysis paper begins the process of deriving requirements for collaborative scheduling algorithms by conducting a survey of 117 publications within the past five years in academia and industry on multiple UV scheduling algorithms. The goal of the survey is to determine the types and frequency of scheduling algorithms that are currently in use and to classify the characteristics and capabilities of these algorithms. Results show that academia has settled on meta-heuristic and auction-based algorithms as providing the best balance of performance and computational speed. In industry, however, the most widely used solution methods are “iterative” approaches that monotonically improve the schedule with further iterations. Industry-developed algorithms are more likely to be capable of scheduling heterogeneous UVs, but university researchers have developed more algorithms that can account for uncertainty and provide estimates of robustness. The different objectives of industry practitioners and academic researchers may be driving these disparities. Addressing this gap will be essential to the development and adoption of future humancomputer collaborative scheduling systems.

Influencing Trust for Human–Automation Collaborative Scheduling of Multiple Unmanned Vehicles

Objective: We examined the impact of priming on operator trust and system performance when supervising a decentralized network of heterogeneous unmanned vehicles (UVs). Background: Advances in autonomy have enabled a future vision of single-operator control of multiple heterogeneous UVs. Real-time scheduling for multiple UVs in uncertain environments requires the computational ability of optimization algorithms combined with the judgment and adaptability of human supervisors. Because of system and environmental uncertainty, appropriate operator trust will be instrumental to maintain high system performance and prevent cognitive overload. Method: Three groups of operators experienced different levels of trust priming prior to conducting simulated missions in an existing, multiple-UV simulation environment. Results: Participants who play computer and video games frequently were found to have a higher propensity to overtrust automation. By priming gamers to lower their initial trust to a more appropriate level, system performance was improved by 10% as compared to gamers who were primed to have higher trust in the automation. Conclusion: Priming was successful at adjusting the operator’s initial and dynamic trust in the automated scheduling algorithm, which had a substantial impact on system performance. Application: These results have important implications for personnel selection and training for futuristic multi-UV systems under human supervision. Although gamers may bring valuable skills, they may also be potentially prone to automation bias. Priming during training and regular priming throughout missions may be one potential method for overcoming this propensity to overtrust automation.

Assessing Operator Strategies for Adjusting Replan Alerts in Controlling Multiple Unmanned Vehicles

Abstract This study examined the impact of allowing an operator to adjust the rate of prompts to view automation-generated plans on operator performance and workload when supervising a decentralized network of heterogeneous unmanned vehicles. Background: Future unmanned vehicles systems will invert the operator-to-vehicle ratio so that one operator can control multiple vehicles with different capabilities, connected through a decentralized network. A previous experiment showed that higher rates of replan prompting led to higher workload and lower system performance. Poor performance was associated with a lack of operator consensus for when to accept the automations suggested prompts for new plan consideration. Method: Three initial rates of replanning were tested on an existing, multiple unmanned vehicle simulation environment that leverages decentralized algorithms for vehicle routing and task allocation, in conjunction with human supervision. Operators were provided with the ability to adjust the rate of replanning. Results: The majority of the operators chose to adjust the rate at which they were prompted to replan. Operators favored particular replan intervals, no matter which initial replan interval they started at. It was found that different initial replan intervals produced differences in mission performance. In addition, increasing amounts of replanning caused the system to destroy more targets but do a poorer job at tracking targets. Conclusion: Operators have preferences for the rate at which they prefer to view automation-generated plans. Allowing operators to institute these preferences influenced the overall mission performance. Further research is necessary to determine the full impact of the operators’ strategies for changing the replan intervals on net mission performance. Application: Future unmanned vehicles systems designs should incorporate the flexibility to allow operators to adjust the frequency at which the automation generates new plans for approval.