Daniel Bryce

Planning graph heuristics for belief space search

Some recent works in conditional planning have proposed reachability heuristics to improve planner scalability, but many lack a formal description of the properties of their distance estimates. To place previous work in context and extend work on heuristics for conditional planning, we provide a formal basis for distance estimates between belief states. We give a definition for the distance between belief states that relies on aggregating underlying state distance measures. We give several techniques to aggregate state distances and their associated properties. Many existing heuristics exhibit a subset of the properties, but in order to provide a standardized comparison we present several generalizations of planning graph heuristics that are used in a single planner. We compliment our belief state distance estimate framework by also investigating efficient planning graph data structures that incorporate BDDs to compute the most effective heuristics. We developed two planners to serve as test-beds for our investigation. The first, CAltAlt, is a conformant regression planner that uses A* search. The second, POND, is a conditional progression planner that uses AO* search. We show the relative effectiveness of our heuristic techniques within these planners. We also compare the performance of these planners with several state of the art approaches in conditional planning.

A Tutorial on Planning Graph Based Reachability Heuristics

The primary revolution in automated planning in the last decade has been the very impressive scale-up in planner performance. A large part of the credit for this can be attributed squarely to the invention and deployment of powerful reachability heuristics. Most, if not all, modern reachability heuristics are based on a remarkably extensible data structure called the planning graph, which made its debut as a bit player in the success of GraphPlan, but quickly grew in prominence to occupy the center stage. Planning graphs are a cheap means to obtain informative look-ahead heuristics for search and have become ubiquitous in state-of-the-art heuristic search planners. We present the foundations of planning graph heuristics in classical planning and explain how their flexibility lets them adapt to more expressive scenarios that consider action costs, goal utility, numeric resources, time, and uncertainty.

Planning interventions in biological networks

Modeling the dynamics of biological processes has recently become an important research topic in computational biology and systems engineering. One of the most important reasons to model a biological process is to enable high-throughput in-silico experiments that attempt to predict or intervene in the process. These experiments can help accelerate the design of therapies through their rapid and inexpensive replication and alteration. While some techniques exist for reasoning with biological processes, few take advantage of the flexible and scalable algorithms popular in AI research. In reasoning about interventions in biological processes, where scalability is crucial for feasible application, we apply AI planning-based search techniques and demonstrate their advantage over existing enumerative methods. We also present a novel formulation of intervention planning that relies on models that characterize and attempt to change the phenotype of a system. We study three biological systems: the yeast cell cycle, a model of the human aging process, and the Wnt5a network governing the metastasis of melanoma in humans. The contribution of our investigation is in demonstrating that: (i) prior approaches, based on dynamic programming, cannot scale as well as heuristic search, and (ii) the newly found scalability enables us to plan previously unknown sequences of interventions that reveal novel and biologically significant responses in the systems which are consistent with biological knowledge in the literature.

Sequential Monte Carlo in reachability heuristics for probabilistic planning

Some of the current best conformant probabilistic planners focus on finding a fixed length plan with maximal probability. While these approaches can find optimal solutions, they often do not scale for large problems or plan lengths. As has been shown in classical planning, heuristic search outperforms bounded length search (especially when an appropriate plan length is not given a priori). The problem with applying heuristic search in probabilistic planning is that effective heuristics are as yet lacking. In this work, we apply heuristic search to conformant probabilistic planning by adapting planning graph heuristics developed for non-deterministic planning. We evaluate a straight-forward application of these planning graph techniques, which amounts to exactly computing a distribution over many relaxed planning graphs (one planning graph for each joint outcome of uncertain actions at each time step). Computing this distribution is costly, so we apply Sequential Monte Carlo (SMC) to approximate it. One important issue that we explore in this work is how to automatically determine the number of samples required for effective heuristic computation. We empirically demonstrate on several domains how our efficient, but sometimes suboptimal, approach enables our planner to solve much larger problems than an existing optimal bounded length probabilistic planner and still find reasonable quality solutions.

Planning for Gene Regulatory Network Intervention

Modeling the dynamics of cellular processes has recently become a important research area of many disciplines. One of the most important reasons to model a cellular process is to enable high-throughput in-silico experiments that attempt to predict or intervene in the process. These experiments can help accelerate the design of therapies through their cheap replication and alteration. While some techniques exist for reasoning with cellular processes, few take advantage of the flexible and scalable algorithms popularized in AI research. We apply AI planning based search techniques to a well-studied gene regulatory network model and demonstrate its clear advantage over existing methods based on enumeration

State agnostic planning graphs: deterministic, non-deterministic, and probabilistic planning

Planning graphs have been shown to be a rich source of heuristic information for many kinds of planners. In many cases, planners must compute a planning graph for each element of a set of states, and the naive technique enumerates the graphs individually. This is equivalent to solving a multiple-source shortest path problem by iterating a single-source algorithm over each source. We introduce a data-structure, the state agnostic planning graph, that directly solves the multiple-source problem for the relaxation introduced by planning graphs. The technique can also be characterized as exploiting the overlap present in sets of planning graphs. For the purpose of exposition, we first present the technique in deterministic (classical) planning to capture a set of planning graphs used in forward chaining search. A more prominent application of this technique is in conformant and conditional planning (i.e., search in belief state space), where each search node utilizes a set of planning graphs; an optimization to exploit state overlap between belief states collapses the set of sets of planning graphs to a single set. We describe another extension in conformant probabilistic planning that reuses planning graph samples of probabilistic action outcomes across search nodes to otherwise curb the inherent prediction cost associated with handling probabilistic actions. Finally, we show how to extract a state agnostic relaxed plan that implicitly solves the relaxed planning problem in each of the planning graphs represented by the state agnostic planning graph and reduces each heuristic evaluation to counting the relevant actions in the state agnostic relaxed plan. Our experimental evaluation (using many existing International Planning Competition problems from classical and non-deterministic conformant tracks) quantifies each of these performance boosts, and demonstrates that heuristic belief state space progression planning using our technique is competitive with the state of the art.

A Hybrid Architecture for Correct-by-Construction Hybrid Planning and Control

This paper describes Hy-CIRCA, an architecture for verified, correct-by-construction planning and execution for hybrid systems, including nonlinear continuous dynamics. Hy-CIRCA addresses the high computational complexity of such systems by first planning at an abstract level, and then progressively refining the original plan. Hy-CIRCA integrates the dReal nonlinear SMT solver with enhanced versions of the SHOP2 HTN planner and the CIRCA Controller Synthesis Module CSM. SHOP2 computes a high level nominal mission plan, the CIRCA CSM develops reactive controllers for the mission steps, accounting for disturbances, and dReal verifies that the plans are correct with respect to continuous dynamics. In this way, Hy-CIRCA decomposes reasoning about the plan and judiciously applies the different solvers to the problems they are best at.

Planning for Multiple Preferences versus Planning with No Preference

Many planning applications must address conflicting plan objectives, such as cost, duration, and resource consumption, and decision makers want to know the possible tradeoffs. Traditionally, such problems are solved by invoking a single-objective algorithm (such as A*) on multiple, alternative preferences of the objectives to identify nondominated plans. The less-popular alternative is to delay such reasoning and directly optimize multiple plan objectives with a search algorithm like multiobjective A* (MOA*). The relative performance of these two approaches hinges upon the number of 𝑓-values computed for individual search nodes. A* may revisit a node several times and compute a different 𝑓-value each time. MOA* visits each node once and may compute some number of 𝑓-values (each estimating the value of a different nondominated solution constructed from the node). While A* does not share 𝑓-values between searches for different solutions, MOA* can sometimes find multiple solutions while computing a single 𝑓-value per node. The results of extensive empirical comparison show that (i) the performance of multiple invocations of a single-objective A* versus a single invocation of MOA* is often worse in time and quality and (ii) that techniques for balancing per node cost and exploration are promising.