Fast Adaptation via Meta Learning in Multi-agent Cooperative Tasks

Abstract

Multi-agent deep reinforcement learning (MADRL) has been used for disaster rescue and other robotic applications. In most MADRL cases, agents are trained only to deal with specific tasks, so once there are unpredictable changes in target tasks, the earlier trained-models might no longer work, and agents have to get trained again from scratch in limited time, which means traditional MADRL is unsuited to unpredictable post-disaster environments. In order to promote the scalability and flexibility of the multi-agent system, meta-learning, which has achieved few shot learning successfully in supervised deep learning field, could reuse the earlier models as the prior knowledge to guide and speed up the new task learning process. In this work, we propose a framework to apply meta learning methods to multi-agent deep deterministic policy gradient (MADDPG), and achieve efficient adaption to new tasks with less time and fewer samples. Some common experiences in the previous tasks learning will be refined as the meta knowledge. Once the tasks get changed with new scenarios, agents will get retrained with good initial network parameters based on this meta knowledge. Although the new scenarios might be different from the previous scenarios, agents could quickly adjust their policy and get better performance within only few episodes. In two experiments, our method performs better learning efficiency than others, getting higher rewards in the first several episodes of new tasks learning.