Daniel Szer

An optimal best-first search algorithm for solving infinite horizon DEC-POMDPs

In the domain of decentralized Markov decision processes, we develop the first complete and optimal algorithm that is able to extract deterministic policy vectors based on finite state controllers for a cooperative team of agents. Our algorithm applies to the discounted infinite horizon case and extends best-first search methods to the domain of decentralized control theory. We prove the optimality of our approach and give some first experimental results for two small test problems. We believe this to be an important step forward in learning and planning in stochastic multi-agent systems.

Improving coordination with communication in multi-agent reinforcement learning

We present a new algorithm for cooperative reinforcement learning in multiagent systems. We consider autonomous and independently learning agents, and we seek to obtain an optimal solution for the team as a whole while keeping the learning as much decentralized as possible. Coordination between agents occurs through communication, namely the mutual notification algorithm. We define the learning problem as a decentralized process using the MDP formalism. We then give an optimality criterion and prove the convergence of the algorithm for deterministic environments. We introduce variable and hierarchical communication strategies which considerably reduce the number of communications. Finally we study the convergence properties and communication overhead on a small example.

Coordination through Mutual Notification in Cooperative Multiagent Reinforcement Learning

We present a new algorithm for cooperative reinforcement learning in multiagent systems. Our main concern is the correct coordination between the members of the team: We seek to obtain an optimal solution for the team as a whole while keeping the learning as much decentralized as possible. We consider autonomous and independently learning agents that do not store any explicit information about their teammatesý behavior, as well as possibly different reward functions for each agent. Coordination between agents occurs through communication, namely the mutual notification algorithm.