Carl E. Nehme

Predicting Operator Capacity for Supervisory Control of Multiple UAVs

With reduced radar signatures, increased endurance and the removal of humans from immediate threat, uninhabited (also known as unmanned) aerial vehicles (UAVs) have become indispensable assets to militarized forces. UAVs require human guidance to varying degrees and often through several operators. However, with current military focus on streamlining operations, increasing automation, and reducing manning, there has been an increasing effort to design systems such that the current many-toone ratio of operators to vehicles can be inverted. An increasing body of literature has examined the effectiveness of a single operator controlling multiple uninhabited aerial vehicles. While there have been numerous experimental studies that have examined contextually how many UAVs a single operator could control, there is a distinct gap in developing predictive models for operator capacity. In this chapter, we will discuss previous experimental research for multiple UAV control, as well as previous attempts to develop predictive models for operator capacity based on temporal measures. We extend this previous research by explicitly considering a cost-performance model that relates operator performance to mission costs and complexity. We conclude with a meta-analysis of the temporal methods outlined and provide recommendation for future applications.

Modeling Workload Impact in Multiple Unmanned Vehicle Supervisory Control

Discrete-event simulations for futuristic unmanned vehicle (UV) systems enable a cost- and time-effective methodology for evaluating various autonomy and human-automation design parameters. Operator mental workload is an important factor to consider in such models. We suggest that the effects of operator workload on system performance can be modeled in such a simulation environment through a quantitative relation between operator attention and utilization, i.e., operator busy time used as a surrogate real-time workload measure. To validate our model, a heterogeneous UV simulation experiment was conducted with 74 participants. Performance-based measures of attention switching delays were incorporated in the discrete-event simulation model by UV wait times due to operator attention inefficiencies (WTAIs). Experimental results showed that WTAI is significantly associated with operator utilization (UT) such that high UT levels correspond to higher wait times. The inclusion of this empirical UT-WTAI relation in the discrete-event simulation model of multiple UV supervisory control resulted in more accurate replications of data, as well as more accurate predictions for alternative UV team structures. These results have implications for the design of future human-UV systems, as well as more general multiple task supervisory control models.

On Efficient Cooperative Strategies between UAVs and Humans in a Dynamic Environment

In this paper, we consider an automation support system aiming at facilitating ecient cooperation between unmanned vehicles (UAVs) and remotely located human operators, in a dynamically-changing environment. In particular, we consider the following problem. A number of UAVs must visit and service (e.g., classify) targets that are generated according to a spatio-temporal Poisson process, uniformly in a convex, compact region of the plane. The vehicles can plan their motion autonomously. However, once a vehicle is in the proximity of a target location, it requires assistance from a human to nish the service at that location. The system objective is to minimize the expected waiting time between the appearance of a target, and the time of completion of its service. We model the performance of the human operators to vary with its utilization factor and propose several coordination strategies for joint target assignments for the humans and the UAVs.

Mission Planning and Monitoring for Heterogeneous Unmanned Vehicle Teams: A Human-Centered Perspective

Future unmanned systems in the military will be highly heterogeneous in nature, with vehicles from multiple domains—aerial, underwater, and land—working in collaborative teams to complete a variety of missions. The complexity of supervising these teams will be enormous and will rely on human creativity, judgment, and experience. Therefore, the design and development of mission planning and monitoring technologies must be rooted in a deep understanding of the human operators role as mission manager, and must effectively address the reasoning skills and limitations of both the human and autonomous intelligent system. In this paper we present our work to approach these supervisory issues from a human-centered perspective. We first review the findings of a cognitive task analysis, through which we defined critical informational requirements and developed display interfaces for human operators developing and executing mission plans for a small team of underwater and aerial unmanned vehicles. These findings raise several operations issues for unmanned vehicle management, namely (1) vehicle and task heterogeneity and (2) the coordination of command and control across a vehicle team. We discuss the impact of both of these design requirements on the human-centered development of mission planning tools for unmanned systems. Finally, we introduce an investigative approach to support the rapid evaluation of interfaces that flexibly accommodate alternative command and control philosophies for heterogeneous automated systems using a combination of modeling and human-in-the-loop evaluation processes

Predictive Model for Human-Unmanned Vehicle Systems

Advances in automation are making it possible for a single operator to control multiple unmanned vehicles. However, the complex nature of these teams presents a difficult and exciting challenge for designers of human–unmanned vehicle systems. To build such systems effectively, models must be developed that describe the behavior of the human–unmanned vehicle team and that predict how alterations in team composition and system design will affect the system’s overall performance. In this paper, we present a method for modeling human–unmanned vehicle systems consisting of a single operator and multiple independent unmanned vehicles. Via a case study, we demonstrate that the resulting models provide an accurate description of observed human-unmanned vehicle systems. Additionally, we demonstrate that the models can be used to predict how changes in the human-unmanned vehicle interface and the unmanned vehicles’ autonomy alter the system’s performance.

Predicting the Impact of Heterogeneity on Unmanned-Vehicle Team Performance

Several recent studies have addressed the possible impact of using highly autonomous platforms to invert todays multiple-operators-per-single-unmanned-vehicle control paradigm. These studies, however, have generally focused on homogeneous vehicle teams and have not addressed the potential effects of vehicle, capability, or mission type heterogeneity on operator control capacity. Important implications of heterogeneous unmanned teams include increases in the diversity of potential team configurations, as well as the diversity of possible attention allocation strategies that may be utilized by operators in managing a given vehicle team. This paper presents preliminary findings from a modeling and simulation effort exploring the impact of heterogeneity on the supervisory control of unmanned vehicle teams. Results from a discrete event simulation study suggest that performance costs of team heterogeneity are highly dependent on resultant changes in operator utilization. Heterogeneous teams that result in lower overall operator utilization may lead to improved performance under certain operator control strategies.

Generating Requirements for Futuristic Hetrogenous Unmanned Systems

A cognitive task analysis (CTA) is an effective analysis technique for deriving design requirements for many task domains. However, traditional CTA approaches have limited applicability to futuristic systems because CTA approaches generally require access to subject matter experts, documentation, and previous implementations from which to draw assumptions and expertise. In this paper, we introduce a hybrid CTA framework that allows the generation of information and display requirements for futuristic systems for which no current implementations exist. This analysis technique involves a four-step process including: 1) generating a scenario task overview, 2) generating an event flow diagram, 3) generating situation awareness requirements, and 4) creating decision ladders for critical decisions. We demonstrate the effectiveness of this process through a case study in which functional and interface requirements are generated for the supervisory control of multiple, heterogeneous unmanned vehicles.