Matthew A. Vavrina

Automated Battery Swap and Recharge to Enable Persistent UAV Missions

This paper introduces a hardware platform for automated battery changing and charging for multiple UAV agents. The automated station holds a buffer of 8 batteries in a novel dual-drum structure that enables a “hot” battery swap, thus allowing the vehicle to remain powered on throughout the battery changing process. Each drum consists of four battery bays, each of which is connected to a smartcharger for proper battery maintenance and charging. The hot-swap capability in combination with local recharging and a large 8-battery capacity allow this platform to refuel multiple UAVs for long-duration and persistent missions with minimal delays and no vehicle shutdowns. Experimental results from the RAVEN indoor flight test facility are presented that demonstrate the capability and robustness of the battery change/charge station in the context of a multi-agent, persistent mission where surveillance is continuously required over a specified region.

An automated battery management system to enable persistent missions with multiple aerial vehicles

This paper presents the development and hardware implementation of an autonomous battery maintenance mechatronic system that significantly extends the operational time of battery powered small-scaled unmanned aerial vehicles (UAVs). A simultaneous change and charge approach is used to overcome the significant downtime experienced by existing charge-only approaches. The automated system quickly swaps a depleted battery of a UAV with a replenished one while simultaneously recharging several other batteries. This results in a battery maintenance system with low UAV downtime, arbitrarily extensible operation time, and a compact footprint. Hence, the system can enable multi-agent UAV missions that require persistent presence. This capability is illustrated by developing and testing in flight a centralized autonomous planning and learning algorithm that incorporates a probabilistic health model dependent on vehicle battery health that is updated during the mission, and replans to improve the performance based on the improved model. Flight test results are presented for a 3-h-long persistent mission with three UAVs that each has an endurance of 8-10 min on a single battery charge (more than 100 battery swaps).

Agent capability in persistent mission planning using approximate dynamic programming

This paper presents an extension of our previous work on the persistent surveillance problem. An extended problem formulation incorporates real-time changes in agent capabilities as estimated by an onboard health monitoring system in addition to the existing communication constraints, stochastic sensor failure and fuel flow models, and the basic constraints of providing surveillance coverage using a team of autonomous agents. An approximate policy for the persistent surveillance problem is computed using a parallel, distributed implementation of the approximate dynamic programming algorithm known as Bellman Residual Elimination. This paper also presents flight test results which demonstrate that this approximate policy correctly coordinates the team to simultaneously provide reliable surveillance coverage and a communications link for the duration of the mission and appropriately retasks agents to maintain these services in the event of agent capability degradation.

Proactive planning for persistent missions using composite model-reference adaptive control and approximate dynamic programming

This paper extends prior work on the persistent mission problem where real-time changes in agent capability are included in the problem formulation. Here, we couple the mission planner with a low-level adaptive controller in real time to: (1) Provide robustness against actuator degradations and (2) Use parameters internal to the adaptive controller to provide valuable insight into the physical capabilities of the agent. These parameters, in conjunction with sensor health information, form a more complete measure of agent capability, which is used online and in forward planning to enable both reactive and proactive behavior. Flight results are presented for a persistent mission scenario where actuator degradations are induced to demonstrate: (1) The robustness of the composite adaptive controller and its successful integration with the agent level health-monitoring and mission-level planning systems and (2) The reactive and proactive qualities of the planning system in persistently re-tasking agents under actuator and sensor health degradations.

Scalable, MDP-based planning for multiple, cooperating agents

This paper introduces an approximation algorithm for stochastic multi-agent planning based on Markov decision processes (MDPs). Specifically, we focus on a decentralized approach for planning the actions of a team of cooperating agents with uncertainties in fuel consumption and health-related models. The core idea behind the algorithm presented in this paper is to allow each agent to approximate the representation of its teammates. Each agent therefore maintains its own planner that fully enumerates its local states and actions while approximating those of its teammates. In prior work, the authors approximated each teammate individually, which resulted in a large reduction of the planning space, but remained exponential (in n - 1 rather than in n, where n is the number of agents) in computational scalability. This paper extends the approach and presents a new approximation that aggregates all teammates into a single, abstracted entity. Under the persistent search & track mission scenario with 3 agents, we show that while resulting performance is decreased nearly 20% compared with the centralized optimal solution, the problem size becomes linear in n, a very attractive feature when planning online for large multi-agent teams.

Multi-Agent Planning for Persistent Missions with Automated Battery Management

This paper introduces and demonstrates a full hardware testbed for research in long-duration missions for multiple, autonomous agents. Speci cally, we describe an automated battery management platform designed to service multiple quadrotor agents in the MIT RAVEN and Boeing VSTL ight environments. The changing/charging station allows the quadrotor’s spent battery to be quickly swapped for a fresh one without requiring it to power down or wait for recharge a signi cant bene t in persistent and/or time-critical missions. We focus on a multi-agent persistent search and track scenario and construct both centralized and decentralized MDP-based mission planners. We further show that for the three agent case, decentralized planners (one for each agent) o er a 99% reduction in computation time and only a relatively small (10%) degradation in overall mission performance when compared to the centralized approach over a long-term simulated mission.

Experimental Demonstration of Multi-Agent Learning and Planning under Uncertainty for Persistent Missions with Automated Battery Management

planning problems in presence of state-correlated uncertainty.An online learning and planning framework is used to address the problem of improving planner performance for missions with state-dependent uncertain agent health dynamics. The framework includes a previously introduced Decentralized Multi-agent Markov decision process (Dec-MMDP) as an online planning algorithm that is scalable in number of agents, and Incremental Feature Discovery (iFDD) which is a compact and fast learning algorithm for estimating parameters of a state-correlated uncertainty model. In combination, this architecture yield an integrated learning-planning algorithm where the planning performance improves as uncertainty is reduced through learning. The presented algorithms are validated in a persistent search and track scenario with a novel automated battery swapping/recharging system that enables the UAVs to collaboratively track targets over durations that are signicantly larger than individual vehicle endurance with a single battery. The results indicate that the architecture can be used as an computationally ecient solution to multi-agent uncertain cooperative planning problems.