Role‐based attention in deep reinforcement learning for games

Abstract

Reinforcement learning method that learns while interacting with the environment, relies heavily on the concept of state as the input to the policy and value function. In the task, the view of agent contains a lot of information, and it is difficult for the agent to learn to ignore the irrelevant information and focus on the key information. Inspired by recent work in attention models for computer vision, we present a role‐based attention model for reinforcement learning. The proposed model uses convolutional neural networks to generate soft attention maps, adding crucial role information in the task, forcing the agent to focus on important features and distinguish task‐related information. To validate the performance in complex problems, the proposed approach is evaluated in a challenging scenario, Football Academy in Google Research Football Environment, a newly released reinforcement learning environment with physics‐based three‐dimensional simulator. The experimental results demonstrate that agents using role‐based attention mechanism can perform better in football games.