Steve Hanks

An algorithm for probabilistic planning

Abstract We define the probabilistic planning problem in terms of a probability distribution over initial world states, a boolean combination of propositions representing the goal, a probability threshold, and actions whose effects depend on the execution-time state of the world and on random chance. Adopting a probabilistic model complicates the definition of plan success: instead of demanding a plan that provably achieves the goal, we seek plans whose probability of success exceeds the threshold. In this paper, we present buridan , an implemented least-commitment planner that solves problems of this form. We prove that the algorithm is both sound and complete. We then explore buridans efficiency by contrasting four algorithms for plan evaluation, using a combination of analytic methods and empirical experiments. We also describe the interplay between generating plans and evaluating them, and discuss the role of search control in probabilistic planning.

Interactive Assessment of User Preference Models: The Automated Travel Assistant

This paper presents the candidate/critique model of interactive problem solving, in which an automated problem solver communicates candidate solutions to the user and the user critiques those solutions. The system starts with minimal information about the user’s preferences, and preferences are elicited and inferred incrementally by analyzing the critiques. The system’s goal is to present “good” candidates to the user, but to do so it must learn as much as possible about his preferences in order to improve its choice of candidates in subsequent iterations. This system contrasts with traditional decision-analytic and planning frameworks in which a complete model is elicited beforehand or is constructed by a human expert. The paper presents the Automated Travel Assistant, an implemented prototype of the model that interactively builds flight itineraries using realtime airline information. The ATA is available on the World Wide Web and has had over 4000 users between May and October 1996.

Decision-theoretic planning: structural assumptions and computational leverage

Planning under uncertainty is a central problem in the study of automated sequential decision making, and has been addressed by researchers in many different fields, including AI planning, decision analysis, operations research, control theory and economics. While the assumptions and perspectives adopted in these areas often differ in substantial ways, many planning problems of interest to researchers in these fields can be modeled as Markov decision processes (MDPs) and analyzed using the techniques of decision theory. This paper presents an overview and synthesis of MDP-related methods, showing how they provide a unifying framework for modeling many classes of planning problems studied in AI. It also describes structural properties of MDPs that, when exhibited by particular classes of problems, can be exploited in the construction of optimal or approximately optimal policies or plans. Planning problems commonly possess structure in the reward and value functions used to describe performance criteria, in the functions used to describe state transitions and observations, and in the relationships among features used to describe states, actions, rewards, and observations. Specialized representations, and algorithms employing these representations, can achieve computational leverage by exploiting these various forms of structure. Certain AI techniques-- in particular those based on the use of structured, intensional representations--can be viewed in this way. This paper surveys several types of representations for both classical and decision-theoretic planning problems, and planning algorithms that exploit these representations in a number of different ways to ease the computational burden of constructing policies or plans. It focuses primarily on abstraction, aggregation and decomposition techniques based on AI-style representations.

A domain-independent algorithm for plan adaptation

The paradigms of transformational planning, case-based planning, and plan debugging all involve a process known as plan adaptation -- modifying or repairing an old plan so it solves a new problem. In this paper we provide a domain-independent algorithm for plan adaptation, demonstrate that it is sound, complete, and systematic, and compare it to other adaptation algorithms in the literature. Our approach is based on a view of planning as searching a graph of partial plans. Generative planning starts at the graphs root and moves from node to node using plan-refinement operators. In planning by adaptation, a library plan--an arbitrary node in the plan graph--is the starting point for the search, and the plan-adaptation algorithm can apply both the same refinement operators available to a generative planner and can also retract constraints and steps from the plan. Our algorithms completeness ensures that the adaptation algorithm will eventually search the entire graph and its systematicity ensures that it will do so without redundantly searching any parts of the graph.

Utility Models for Goal‐Directed, Decision‐Theoretic Planners

AI planning agents are goal‐directed: success is measured in terms of whether an input goal is satisfied. The goal gives structure to the planning problem, and planning representations and algorithms have been designed to exploit that structure. Strict goal satisfaction may be an unacceptably restrictive measure of good behavior, however.

Modeling a dynamic and uncertain world I: symbolic and probabilistic reasoning about change

Abstract Intelligent agency requires some ability to predict the future. An agent must ask itself what is presently its best course of action given what it now knows about what the world will be like when it intends to act. This paper presents a system that uses a probabilistic model to reason about the effects of an agents proposed actions on a dynamic and uncertain world, computing the probability that relevant propositions will hold at a specified point in time. The model allows for incomplete information about the world, the occurrence of exogenous (unplanned) events, unreliable sensors, and the possibility of an imperfect causal theory. The system provides an application program with answers to questions of the form “is the probability that ϕ will hold in the world at time t greater than τ?” It is unique among algorithms for probabilistic temporal reasoning in that it tries to limit its inference according to the proposition, time, and probability threshold provided by the application. The system will also notify the application if subsequent evidence invalidates its answer to a query.

Efficient information gathering on the Internet

The Internet offers unprecedented access to information. At present most of this information is free, but information providers ore likely to start charging for their services in the near future. With that in mind this paper introduces the following information access problem: given a collection of n information sources, each of which has a known time delay, dollar cost and probability of providing the needed information, find an optimal schedule for querying the information sources. We study several variants of the problem which differ in the definition of an optimal schedule. We first consider a cost model in which the problem is to minimize the expected total cost (monetary and time) of the schedule, subject to the requirement that the schedule may terminate only when the query has been answered or all sources have been queried unsuccessfully. We develop an approximation algorithm for this problem and for an extension of the problem in which more than a single item of information is being sought. We then develop approximation algorithms for a reward model in which a constant reward is earned if the information is successfully provided, and we seek the schedule with the maximum expected difference between the reward and a measure of cost. The monetary and time costs may either appear in the cost measure or be constrained not to exceed a fixed upper bound; these options give rise to four different variants of the reward model.

A probabilistic model of action for least-commitment planning with information gathering

AI planning algorithms have addressed the problem of generating sequences of operators that achieve some input goal, usually assuming that the planning agent has perfect control over and information about the world. Relaxing these assumptions requires an extension to the action representation that allows reasoning both about the changes an action makes and the information it provides. This paper presents an action representation that extends the deterministic STRIPS model, allowing actions to have both causal and informational effects, both of which can be context dependent and noisy. We also demonstrate how a standard least-commitment planning algorithm can be extended to include informational actions and contingent execution.

Chapter 10 – Probabilistic Temporal Reasoning

Research in probabilistic temporal reasoning is devoted to building models of systems that change stochastically over time. Probabilistic dynamical systems have been studied in Statistics, Operations Research, and the Decision Sciences, though usually not with the emphasis on computational inference models and structured representations that characterizes much work in AI. At the same time, a related body of work in the AI literature has developed probabilistic extensions to the deterministic temporal reasoning representations and algorithms that have been studied actively in AI from the field’s inception. This chapter develops a unifying view of probabilistic temporal reasoning as it has been studied in the optimization, statistical, and AI literatures. It discusses two main bodies of work, which differ on their fundamental views of the problem:

Efficient temporal reasoning for plan projection

Efficient and expressive temporal reasoning can only be achieved by taking advantage of regularities within the structure of the domain. We present a novel temporal reasoning system which demonstrates this approach, applied to the task of temporal projection.