Hugo Gilbert

Reducing the Number of Queries in Interactive Value Iteration

To tackle the potentially hard task of defining the reward function in a Markov Decision Process MDPs, a new approach, called Interactive Value Iteration IVI has recently been proposed by Weng and Zanuttini 2013. This solving method, which interweaves elicitation and optimization phases, computes a near optimal policy without knowing the precise reward values. The procedure as originally presented can be improved in order to reduce the number of queries needed to determine an optimal policy. The key insights are that 1 asking queries should be delayed as much as possible, avoiding asking queries that might not be necessary to determine the best policy, 2 queries should be asked by following a priority order because the answers to some queries can enable to resolve some other queries, 3 queries can be avoided by using heuristic information to guide the process. Following these ideas, a modified IVI algorithm is presented and experimental results show a significant decrease in the number of queries issued.

Incremental Decision Making Under Risk with the Weighted Expected Utility Model

This paper deals with decision making under risk with the Weighted Expected Utility (WEU) model, which is a model generalizing expected utility and providing stronger descriptive possibilities. We address the problem of identifying, within a given set of lotteries, a (near-)optimal solution for a given decision maker consistent with the WEU theory. The WEU model is parameterized by two real-valued functions. We propose here a new incremental elicitation procedure to progressively reduce the imprecision about these functions until a robust decision can be made. We also give experimental results showing the practical efficiency of our method.

Fair Proportional Representation Problems with Mixture Operators

This paper deals with proportional representation problems in which a set of winning candidates must be selected according to the ballots of the voters. We investigate the use of a new class of optimization criteria to determine the set of winning candidates, namely mixture operators. In a nutshell, mixture operators are similar to weighted means where the numerical weights are replaced by weighting functions. In this paper: (1) we give the mathematical condition for which a mixture operator is fair and provide several instances of this operator satisfying this condition; (2) we show that when using a mixture operator as optimization criterion, one recovers the same complexity results as in the utilitarian case (i.e., maximizing the sum of agent’s utilities) under a light condition; (3) we present solution methods to find an optimal set of winners w.r.t. a mixture operator under both Monroe and Chamberlin-Courant multi-winner voting rules and test their computational efficiency.

Sequential Decision Making Under Uncertainty Using Ordinal Preferential Information

The research work undertaken in my thesis aims at facilitating the conception of autonomous agents able to solve complex problems in sequential decision problems e.g., planning problems in robotics.