Felipe W. Trevizan

Unifying nondeterministic and probabilistic planning through imprecise markov decision processes

This paper proposes an unifying formulation for nondeterministic and probabilistic planning. These two strands of AI planning have followed different strategies: while nondeterministic planning usually looks for minimax (or worst-case) policies, probabilistic planning attempts to maximize expected reward. In this paper we show that both problems are special cases of a more general approach, and we demonstrate that the resulting structures are Markov Decision Processes with Imprecise Probabilities (MDPIPs). We also show how existing algorithms for MDPIPs can be adapted to planning under uncertainty.

Reachability-based model reduction for Markov decision process

This paper presents how to improve model reduction for Markov decision process (MDP), a technique that generates equivalent MDPs that can be smaller than the original MDP. In order to improve the current state-of-the-art, we take advantage of the information about the initial state of the environment. Given this initial state information, we perform a reachability analysis and then employ model reduction techniques to the reachable space of the original problem. Further, we also eliminate redundancies in the original MDP in order to speed up the model reduction phase. We also contribute by empirically comparing our technique against state-of-the-art model reduction techniques and MDP solvers that do not perform model reduction. The results show that our approach dominates the current model reduction algorithms and outperforms general MDP solvers in dense problems, i.e., problems in which actions have many probabilistic outcomes.

Operator Counting Heuristics for Probabilistic Planning

For the past 25 years, heuristic search has been used to solve domain-independent probabilistic planning problems, but with heuristics that determinise the problem and ignore precious probabilistic information. In this paper, we present a generalization of the operator-counting family of heuristics to Stochastic Shortest Path problems (SSPs) that is able to represent the probability of the actions outcomes. Our experiments show that the equivalent of the net change heuristic in this generalized framework obtains significant run time and coverage improvements over other state-of-the-art heuristics in different planners.

Tableaux for Policy Synthesis for MDPs with PCTL* Constraints

Markov decision processes (MDPs) are the standard formalism for modelling sequential decision making in stochastic environments. Policy synthesis addresses the problem of how to control or limit the decisions an agent makes so that a given specification is met. In this paper we consider PCTL*, the probabilistic counterpart of CTL*, as the specification language. Because in general the policy synthesis problem for PCTL* is undecidable, we restrict to policies whose execution history memory is finitely bounded a priori. Surprisingly, no algorithm for policy synthesis for this natural and expressive framework has been developed so far. We close this gap and describe a tableau-based algorithm that, given an MDP and a PCTL* specification, derives in a non-deterministic way a system of (possibly nonlinear) equalities and inequalities. The solutions of this system, if any, describe the desired (stochastic) policies. Our main result in this paper is the correctness of our method, i.e., soundness, completeness and termination.

I-dual: Solving Constrained SSPs via Heuristic Search in the Dual Space

We consider the problem of generating optimal stochastic policies for Constrained Stochastic Shortest Path problems, which are a natural model for planning under uncertainty for resourcebounded agents with multiple competing objectives. While unconstrained SSPs enjoy a multitude of efficient heuristic search solution methods with the ability to focus on promising areas reachable from the initial state, the state of the art for constrained SSPs revolves around linear and dynamic programming algorithms which explore the entire state space. In this paper, we present i-dual, the first heuristic search algorithm for constrained SSPs. To concisely represent constraints and efficiently decide their violation, i-dual operates in the space of dual variables describing the policy occupation measures. It does so while retaining the ability to use standard value function heuristics computed by well-known methods. Our experiments show that these features enable i-dual to achieve up to two orders of magnitude improvement in run-time and memory over linear programming algorithms.

Robótica cognitiva: programação baseada em lógica para controle de robôs

The goal of the Cognitive Robotics research area is to develop robotic agents capable of high-level functions by using a programming language, based on logics, to describe the robot control program. Besides, such a language can be used to prove properties of the world and to simulate the robot behavior by running its program. This paper shows how a Lego® MindStorms™ robot can be used to implement a software agent capable of performing high level functions specified in IndiGolog - a logical language to write robot control programs, based on Situation Calculus. The application domain example is the classical problem of the Wumpus World for which the construction of a complete intelligent agent requires the integration of several Artificial Intelligence techniques, such as: reactive planning; hierarquical and goal achievement planning; plan execution; reasoning with incomplete information; generation and discrimination of hypotheses about the world state; and belief changes.