Stefan Bieniawski

Vehicle Swarm Rapid Prototyping Testbed

Increased levels of vehicle collaboration and auton omy are seen as a means to reduce overall mission completion costs while expanding mi ssion capabilities and increasing mission assurance for complex coupled system of systems. Systems health management technologies have made rapid advances that enable systems to know their own condition and capabilities, thus creating the opportunity for unprecedented lev els of adaptive control, real-time reconfiguration, and mission contingency management. Multi-agent task allocation and mission managements systems must account for vehicle- and system-level health-related issues to ensure that these systems are reliable an d cost effective to operate. Boeing’s Vehicle Swarm Technology Lab (VSTL), established in 2004, includes a 100’x50’x20’ testbed equipped with a vision-based motion capture indoor localization system. The testbed provides a cost-effective rapid prototyping capabil ity for integrating health-based adaptive control of subsystems, vehicle, mission, and swarms to guarantee top-level system-of-systems performance metrics. The lab’s heterogeneous fleet includes over 20 heterogeneous air vehicles, including VTOL and fixed wing, along with their ground stations and communication links in addition to heterogeneous ground vehicles and wall climbing robots. This paper discusses the Boeing VSTL design and capabilities, including the indoor localization system, multi-vehicle command and control (C2) and operator interface, realtime virtual environment, and health-based adaptive behaviors. The lab supports rapid prototyping and exploration of various multi-vehicl e operational concept of operations and missions including persistent surveillance, area se arch and tracking, and high density air traffic management. Additionally, the lab supports experimentation tasks for many other platform configuration and collaborative air, groun d, space, and maritime autonomous system of systems concepts.

Flutter Suppression Using Micro-Trailing Edge Effectors

Recent developments in actuator technology have resulted in small, simple flow control devices capable of affecting the flow field over flight vehicles sufficiently to generate control forces. One of the devices which has been under investigation is the Micro-Trailing Edge Effector (MiTE), which consists of a small, 1-5% chord, vertically sliding flap mounted at the trailing edge. The high bandwidth and good control authority with little required power makes the device an ideal candidate for active control of flutter in high aspect ratio wings. Unfortunately traditional control techniques do not address the non-linear nature of the device or the competing performance goals arising from large numbers of distributed devices. Novel approaches to control design, such as reinforcement learning, are therefore required. To demonstrate the aeroelastic control capability of the MiTEs and to explore reinforcement learning techniques, an experimental model has been designed, fabricated, and tested. This paper details the experimental model and the accompanying analytical model. Design, manufacturing and open loop testing of the experimental model and comparisons with the analytical predictions are presented. This paper also covers the controller design for flutter suppression using reinforcement learning policy search techniques. The results of closed loop testing, resulting in successful flutter suppression with the MiTEs, is presented.

A comparative study of probability collectives based multi-agent systems and genetic algorithms

We compare Genetic Algorithms (GAs) with Probability Collectives (PC), a new framework for distributed optimization and control. In contrast to GAs, PC-based methods do not update populations of solutions. Instead they update an explicitly parameterized probability distribution p over the space of solutions. That updating of p arises as the optimization of a functional of p. The functional is chosen so that any p that optimizes it should be p peaked about good solutions. The PC approach has deep connections with both game theory and statistical physics. We review the PC approach using its motivation as the information theoretic formulation of bounded rationality for multi-agent systems (MAS). It is then compared with GAs on a diverse set of problems. To handle high dimensional surfaces, in the PC method investigated here p is restricted to a product distribution. Each distribution in that product is controlled by a separate agent. The test functions were selected for their difficulty using either traditional gradient descent or genetic algorithms. On those functions the PC-based approach significantly outperforms traditional GAs in both rate of descent, trapping in false minima, and long term optimization.

Adaptive, Distributed Control of Constrained Multi-Agent Systems

Product Distribution (PD) theory was recently developed as a framework for analyzing and optimizing distributed systems. In this paper we demonstrate its use for adaptive distributed control of Multi-Agent Systems (MASýs), i.e., for distributed stochastic optimization using MASýs. One common way to perform the optimization is to have each agent run a Reinforcement Learning (RL) algorithm. PD theory provides an alternative based upon using a variant of Newtonýs method operating on the agentýs probability distributions. We compare this alternative to RL-based search in three sets of computer experiments. The PD-theory-based approach outperforms the RL-based scheme in all three domains.

Flight Control with Distributed Efiectors

Recent developments in actuator technology have resulted in small, simple devices capable of afiecting the ∞ow fleld over ∞ight vehicles su‐ciently to generate control forces. One of the devices which has been under investigation is the Miniature-Trailing Edge Efiector (MiTE), which consists of a small, 1-5% chord, moveable surface mounted at the wing trailing edge. The high bandwidth and good control authority make the device an ideal candidate for control of both the rigid body and ∞exible modes of a ∞ight vehicle. Unfortunately traditional control techniques do not address the non-linear nature of the device or the competing performance goals arising from large numbers of distributed devices. Novel approaches to control design are therefore required. To demonstrate the potential of this type of ∞ight control architecture and to explore suitable control synthesis techniques, a remotely piloted ∞ight vehicle has been developed. This paper details the ∞ight vehicle including a distributed ∞ight control system based upon MiTEs. The latter system includes distributed sensing, logic, and actuation. This paper also describes an applicable novel control synthesis technique based upon the theory of collectives. The theory and its application to the design of distributed ∞ight control systems is presented. Results of ∞ight tests with conventional control surfaces and with a MiTE based control system are provided.

Exploring Health-Enabled Mission Concepts in the Vehicle Swarm Technology Laboratory

The use of health based information in mission plan ning offers the opportunity to significantly enhance overall mission assurance. D eveloping mission concepts, even at a simple level, requires coordination of multiple ass ets and determination of common interfaces suitable for heterogeneous fleets. For systems that are subject to real failures, simulation offers the challenges of developing real istic scenarios and realistic health emulation. An alternative explored here is the use of indoor, rapid prototyping labs for exploring larger scale, heterogeneous mission conce pts. Of particular interest are persistent missions were faults are a key driver in the aggreg ate mission performance. Results of flight tests with several different sample missions will b e presented. These missions range from non-cooperative to cooperative and include a range of tasks.

Chapter 4 - Probability Collectives in Optimization

Abstract This article concerns “blackbox optimization” algorithms in which one iterates the following procedure: Choose a value x ∈ X , getting statistical information about an associated value G ( x ) , then use the set of all pairs { ( x , G ( x ) ) } found so far to choose a next x value at which to sample G , the goal being to find x s with as small G ( x ) as possible, and to do so as fast as possible. Examples of conventional blackbox optimization algorithms are genetic algorithms, simulated annealing, etc. These conventional algorithms work directly with values x , stochastically mapping the set { ( x , G ( x ) ) } to the next x . The distribution over new x s that gets sampled is never explicitly optimized. In contrast, in the Probability Collectives (PC) approach, one explicitly uses the set { ( x , G ( x ) ) } to optimize the probability distribution over x that will be sampled. This article reviews some of the work that has been done on Probability Collectives, in particular presenting some of the many experiments that have demonstrated its power.

Distributed Adaptive Control: Beyond Single-Instant, Discrete Control Variables

In extensive form noncooperative game theory, at each instant t, each agent i sets its state x i independently of the other agents, by sampling an associated distribution, q i(x i). The coupling between the agents arises in the joint evolution of those distributions. Distributed control problems can be cast the same way. In those problems the system designer sets aspects of the joint evolution of the distributions to try to optimize the goal for the overall system. Now information theory tells us what the separate q i of the agents are most likely to be if the system were to have a particular expected value of the objective function G(x 1, x 2, ...). So one can view the job of the system designer as speeding an iterative process. Each step of that process starts with a specified value of E(G), and the convergence of the q i to the most likely set of distributions consistent with that value. After this the target value for E q(G) is lowered, and then the process repeats. Previous work has elaborated many schemes for implementing this process when the underlying variables x i all have a finite number of possible values and G does not extend to multiple instants in time. That work also is based on a fixed mapping from agents to control devices, so that the the statistical independence of the agents’ moves means independence of the device states. This paper also extends that work to relax all of these restrictions. This extends the applicability of that work to include continuous spaces and Reinforcement Learning. This paper also elaborates how some of that earlier work can be viewed as a first-principles justification of evolution-based search algorithms.