Olivier Buffet

Active learning of MDP models

We consider the active learning problem of inferring the transition model of a Markov Decision Process by acting and observing transitions. This is particularly useful when no reward function is a priori defined. Our proposal is to cast the active learning task as a utility maximization problem using Bayesian reinforcement learning with belief-dependent rewards. After presenting three possible performance criteria, we derive from them the belief-dependent rewards to be used in the decision-making process. As computing the optimal Bayesian value function is intractable for large horizons, we use a simple algorithm to approximately solve this optimization problem. Despite the sub-optimality of this technique, we show experimentally that our proposal is efficient in a number of domains.

Intersections intelligentes pour le contrôle de véhicules sans pilote. Coordination locale et optimisation globale

Dans cet article, nous nous interessons a la gestion du trafic au sein d’un reseau routier pour vehicules sans pilote, fondee sur des prises de decision au niveau des intersections. Les intersections percoivent et controlent les vehicules a l’approche, au travers de communications infrastructure-a-vehicule, afin d’assurer un passage coordonne et sans arret. Nous proposons une approche originale a deux egards : d’une part, elle explore un principe de passage en alternance des flux aux intersections et, d’autre part, elle propose des algorithmes distribues permettant l’optimisation du trafic global du reseau. Nous presentons successivement les choix de modelisation, les algorithmes et l’etude en simulation de leurs performances comparees a des approches existantes.

Formalizing and Solving Information Collection Problems with Autonomous Sensor Systems

This paper addresses the problem of planning data collection missions for a set of Information Collection Systems (ICS) to respond to a set of information requests. This problem goes from the formalization of information needs to the optimization of ICS actions. After having formalized requests and decomposed them into elementary requests, the problem can be modeled with a graph characterizing the various aspects: coordination and assignment of ICSs, request satisfaction and ICS use optimization. Based on this graph, the problem can be solved with an A*-like search algorithm.