Leslie Pack Kaelbling

Planning and acting in partially observable stochastic domains

In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains. We begin by introducing the theory of Markov decision processes (mdps) and partially observable MDPs (pomdps). We then outline a novel algorithm for solving pomdps off line and show how, in some cases, a finite-memory controller can be extracted from the solution to a POMDP. We conclude with a discussion of how our approach relates to previous work, the complexity of finding exact solutions to pomdps, and of some possibilities for finding approximate solutions.

Acting under uncertainty: discrete Bayesian models for mobile-robot navigation

Discrete Bayesian models have been used to model uncertainty for mobile-robot navigation, but the question of how actions should be chosen remains largely unexplored. This paper presents the optimal solution to the problem, formulated as a partially observable Markov decision process. Since solving for the optimal control policy is intractable, in general, it goes on to explore a variety of heuristic control strategies. The control strategies are compared experimentally, both in simulation and in runs on a robot.

The synthesis of digital machines with provable epistemic properties

Researchers using epistemic logic as a formal framework for studying knowledge properties of AI systems often interpret the knowledge formula K(x, φ) to mean that machine x encodes φ in its state as a syntactic formula or can derive it inferentially. By defining K(x, φ), instead, in terms of the correlation between the state of the machine and that of its environment, the formal properties of modal system S5 can be satisfied without having to store representations of formulas as data structures. In this paper, we apply the correlational definition of knowledge to machines with composite structure. In particular, we describe how epistemic properties of synchronous digital machines can be analyzed, starting at the level of gates and delays, by modeling the machines components as agents in a multi-agent system and reasoning about the flow of information among them. We also introduce Rex, a language for recursively computing machine descriptions, and illustrate how it can be used to construct machines with provable knowledge properties.

Action and planning in embedded agents

Embedded agents are computer systems that sense and act on their environments, monitoring complex dynamic conditions and affecting the environment in goal-directed ways. This paper briefly reviews the situated automata approach to agent design and explores issues of planning and action in the situated-automata framework.

Belief space planning assuming maximum likelihood observations

We cast the partially observable control problem as a fully observable underactuated stochastic control problem in belief space and apply standard planning and control techniques. One of the difficulties of belief space planning is modeling the stochastic dynamics resulting from unknown future observations. The core of our proposal is to define deterministic beliefsystem dynamics based on an assumption that the maximum likelihood observation (calculated just prior to the observation) is always obtained. The stochastic effects of future observations are modeled as Gaussian noise. Given this model of the dynamics, two planning and control methods are applied. In the first, linear quadratic regulation (LQR) is applied to generate policies in the belief space. This approach is shown to be optimal for linearGaussian systems. In the second, a planner is used to find locally optimal plans in the belief space. We propose a replanning approach that is shown to converge to the belief space goal in a finite number of replanning steps. These approaches are characterized in the context of a simple nonlinear manipulation problem where a planar robot simultaneously locates and grasps an object.

A situated view of representation and control

Intelligent agents are systems that have a complex, ongoing interaction with an environment that is dynamic and imperfectly predictable. Agents are typically difficult to program because the correctness of a program depends on the details of how the agent is situated in its environment. In this paper, we present a methodology for the design of situated agents that is based on situated-automata theory. This approach allows designers to describe the informational content of an agents computational states in a semantically rigorous way without requiring a commitment to conventional run-time symbolic processing. We start by outlining this situated view of representation, then show how it contributes to design methodologies for building systems that track perceptual conditions and take purposeful actions in their environments.

Learning symbolic models of stochastic domains

In this article, we work towards the goal of developing agents that can learn to act in complex worlds. We develop a probabilistic, relational planning rule representation that compactly models noisy, nondeterministic action effects, and show how such rules can be effectively learned. Through experiments in simple planning domains and a 3D simulated blocks world with realistic physics, we demonstrate that this learning algorithm allows agents to effectively model world dynamics.

Integrated task and motion planning in belief space

We describe an integrated strategy for planning, perception, state estimation and action in complex mobile manipulation domains based on planning in the belief space of probability distributions over states using hierarchical goal regression (pre-image back-chaining). We develop a vocabulary of logical expressions that describe sets of belief states, which are goals and subgoals in the planning process. We show that a relatively small set of symbolic operators can give rise to task-oriented perception in support of the manipulation goals. An implementation of this method is demonstrated in simulation and on a real PR2 robot, showing robust, flexible solution of mobile manipulation problems with multiple objects and substantial uncertainty.

Ecological robotics

There are striking parallels between ecological psychology and the new trends in robotics and computer vision, particularly regarding how agents interact with the environment. We present some ideas from ecological psychology, including control laws using optic flow, affordances, and action modes, and describe our implementation of these concepts in two mobile robots that can avoid obstacles and chase or flee moving targets solely by using optic flow. The properties of these methods were explored further in simulation. This work ties in with that of others who argue for a methodological approach in robotics that forgoes a central model or planner. Not only might ecological psychology contribute to robotics, but robotic implementations might, in turn, provide a test bed for ecological principles and a source of ideas that could be tested in animals and humans.

A Dynamical Model of Visually-Guided Steering, Obstacle Avoidance, and Route Selection

Using a biologically-inspired model, we show how successful route selection through a cluttered environment can emerge from on-line steering dynamics, without explicit path planning. The model is derived from experiments on human walking performed in the Virtual Environment Navigation Lab (VENLab) at Brown. We find that goals and obstacles behave as attractors and repellors of heading, the direction of locomotion, for an observer moving at a constant speed. The influence of a goal on turning rate increases with its angle from the heading and decreases exponentially with its distance; the influence of an obstacle decreases exponentially with angle and distance. Linearly combining goal and obstacle terms allows us to simulate paths through arbitrarily complex scenes, based on information about obstacles in view near the heading direction and a few meters ahead. We simulated the model on a variety of scene configurations and observed generally efficient routes, and verified this behavior on a mobile robot. Discussion focuses on comparisons between dynamical models and other approaches, including potential field models and explicit path planning. Effective route selection can thus be performed on-line, in simple environments as a consequence of elementary behaviors for steering and obstacle avoidance.