Loosely Coupled Formulations for Automated Planning: An Integer Programming Perspective
Menkes Hector Louis van den Briel, Thomas Vossen, Subbarao Kambhampati
aa r X i v : . [ c s . A I] O c t Journal of Artificial Intelligence Research 31 (2008) 217-257 Submitted 09/07; published 02/08
Loosely Coupled Formulations for Automated Planning: AnInteger Programming Perspective
Menkes H.L. van den Briel [email protected]
Department of Industrial EngineeringArizona State University, Tempe, AZ 85281 USA
Thomas Vossen [email protected]
Leeds School of BusinessUniversity of Colorado at Boulder, Boulder CO, 80309 USA
Subbarao Kambhampati [email protected]
Department of Computer Science and EngineeringArizona State University, Tempe, AZ 85281 USA
Abstract
We represent planning as a set of loosely coupled network flow problems, where eachnetwork corresponds to one of the state variables in the planning domain. The networknodes correspond to the state variable values and the network arcs correspond to the valuetransitions. The planning problem is to find a path (a sequence of actions) in each networksuch that, when merged, they constitute a feasible plan. In this paper we present a num-ber of integer programming formulations that model these loosely coupled networks withvarying degrees of flexibility. Since merging may introduce exponentially many orderingconstraints we implement a so-called branch-and-cut algorithm, in which these constraintsare dynamically generated and added to the formulation when needed. Our results are verypromising, they improve upon previous planning as integer programming approaches andlay the foundation for integer programming approaches for cost optimal planning.
1. Introduction
While integer programming approaches for automated planning have not been able toscale well against other compilation approaches (i.e. satisfiability and constraint satisfac-tion), they have been extremely successful in the solution of many real-world large scaleoptimization problems. Given that the integer programming framework has the potentialto incorporate several important aspects of real-world automated planning problems (forexample, numeric quantities and objective functions involving costs and utilities), there issignificant motivation to investigate more effective integer programming formulations forclassical planning as they could lay the groundwork for large scale optimization (in termsof cost and resources) in automated planning. In this paper, we study a novel decomposi-tion based approach for automated planning that yields very effective integer programmingformulations.
1. We use the term integer programming to refer to integer linear programming unless stated otherwise. c (cid:13) an den Briel, Vossen & Kambhampati Decomposition is a general approach to solving problems more efficiently. It involvesbreaking a problem up into several smaller subproblems and solving each of the subprob-lems separately. In this paper we use decomposition to break up a planning problem intoseveral interacting (i.e. loosely coupled) components. In such a decomposition, the planningproblem involves both finding solutions to the individual components and trying to mergethem into a feasible plan. This general approach, however, prompts the following questions:(1) what are the components, (2) what are the component solutions, and (3) how hard is itto merge the individual component solutions into a feasible plan?
We let the components represent the state variables of the planning problem. Figure 1illustrates this idea using a small logistics example, with one truck and a package thatneeds to be moved from location 1 to location 2. There are a total of five components inthis example, one for each state variable. We represent the components by an appropriatelydefined network, where the network nodes correspond to the values of the state variable(for atoms this is T = true and F = false), and the network arcs correspond to the valuetransitions. The source node in each network, represented by a small in-arc, correspondsto the initial value of the state variable. The sink node(s), represented by double circles,correspond to the goal value(s) of the state variable. Note that the effects of an actioncan trigger value transitions in the state variables. For example, loading the package atlocation 1 makes the atom pack - in - truck true and pack - at - loc truck - at - loc at - loc
1, 2 = at - loc
2, and t = in - truck ), and the arcs correspond to thevalue transitions. We let the component solutions represent a path of value transitions in the state variables.In the networks, nodes and arcs appear in layers. Each layer represents a plan periodin which, depending on the structure of the network, one or more value transitions canoccur. The networks in Figures 1 and 2 each have three layers (i.e. plan periods) and theirstructure allows values to persist or change exactly once per period. The layers are used tosolve the planning problem incrementally. That is, we start with one layer in each networkand try to solve the planning problem. If no plan is found, all networks are extended byone extra layer and a new attempt is made to solve the planning problem. This process isrepeated until a plan is found or a time limit is reached. In Figures 1 and 2, a path (i.e. oosely Coupled Formulations for Automated Planning
Unload at loc2 loc1 loc2truck-at-loc1truck-at-loc2pack-at-loc1pack-at-loc2pack-in-truck FT FT FT FTFT FT FT FTFT FT FT FTFT FT FT FTFT FT FT FT
Load at loc1Load at loc1 Drive loc1 → loc2Drive loc1 → loc2 Unload at loc2Unload at loc2 --- - -- - Load at loc1
Figure 1: Logistics example broken up into five components (binary-valued state variables)that are represented by network flow problems.
21 21 21 2121 21 21 21 t t t t
Drive loc1 → loc2Load at loc1 Unload at loc2 - Load at loc1 Unload at loc2 truck-locationpack-location
Figure 2: Logistics example broken up into two components (multi-valued state variables)that are represented by network flow problems.a solution) from the source node to one of the sink nodes is highlighted in each network.Since the execution of an action triggers value transitions in the state variables, each pathin a network corresponds to a sequence of actions. Consequently, the planning problemcan be thought of as a collection of network flow problems where the problem is to find apath (i.e. a sequence of actions) in each of the networks . However, interactions between an den Briel, Vossen & Kambhampati the networks impose side constraints on the network flow problems, which complicate thesolution process.
We solve these loosely coupled networks using integer programming formulations. Onedesign choice we make is that we expand all networks (i.e. components) together, so thecost of finding solutions for the individual networks as well as merging them depends on thedifficulty of solving the integer programming formulation. This, in turn, typically dependson the size of the integer programming formulation, which is partly determined by thenumber of layers in each of the networks. The simplest idea is to have the number oflayers of the networks equal the length of the plan, just as in sequential planning where theplan length equals the number of actions in the plan. In this case, there will be as manytransitions in the networks as there are actions in the plan, with the only difference that asequence of actions corresponding to a path in a network could contain no-op actions.An idea to reduce the required number of layers is by allowing multiple actions to beexecuted in the same plan period. This is exactly what is done in Graphplan (Blum & Furst,1995) and in other planners that have adopted the Graphplan-style definition of parallelism.That is, two actions can be executed in parallel (i.e. in the same plan period) as long asthey are non-interfering. In our formulations we adopt more general notions of parallelism.In particular, we relax the strict relation between the number of layers in the networksand the length of the plan by changing the network representation of the state variables.For example, by allowing multiple transitions in each network per plan period we permitinterfering actions to be executed in the same plan period. This, however, raises issuesabout how solutions to the individual networks are searched and how they are combined.When the network representations for the state variables allow multiple transitions in eachnetwork per plan period, and thus become more flexible, it becomes harder to merge thesolutions into a feasible plan. Therefore, to evaluate the tradeoffs in allowing such flexiblerepresentations, we look at a variety of integer programming formulations.We refer to the integer programming formulation that uses the network representationshown in Figures 1 and 2 as the one state change model, because it allows at most onetransition (i.e. state change) per plan period in each state variable. Note that in this networkrepresentation a plan period mimics the Graphplan-style parallelism. That is, two actionscan be executed in the same plan period if one action does not delete the precondition oradd-effect of the other action. A more flexible representation in which values can change atmost once and persist before and after each change we refer to as the generalized one statechange model. Clearly, we can increase the number of changes that we allow in each planperiod. The representations in which values can change at most twice or k times, we referto as the generalized two state change and the generalized k state change model respectively.One disadvantage with the generalized k state change model is that it creates one variablefor each way to do k value changes, and thus introduces exponentially many variables perplan period. Therefore, another network representation that we consider allows a path ofvalue transitions in which each value can be visited at most once per plan period. Thisway, we can limit the number of variables, but may introduce cycles in our networks. The oosely Coupled Formulations for Automated Planning integer programming formulation that uses this representation is referred to as the statechange path model.In general, by allowing multiple transitions in each network per plan period (i.e. layer),the more complex the merging process becomes. In particular, the merging process checkswhether the actions in the solutions of the individual networks can be linearized into afeasible plan. In our integer programming formulations, ordering constraints ensure feasiblelinearizations. There may, however, be exponentially many ordering constraints when wegeneralize the Graphplan-style parallelism. Rather than inserting all these constraints in theinteger programming formulation up front, we add them as needed using a branch-and-cutalgorithm. A branch-and-cut algorithm is a branch-and-bound algorithm in which certainconstraints are generated dynamically throughout the branch-and-bound tree.We show that the performance of our integer programming (IP) formulations show newpotential and are competitive with SATPLAN04 (Kautz, 2004). This is a significant resultbecause it forms a basis for other more sophisticated IP-based planning systems capable ofhandling numeric constraints and non-uniform action costs. In particular, the new potentialof our IP formulations has led to their successful use in solving partial satisfaction planningproblems (Do, Benton, van den Briel, & Kambhampati, 2007). Moreover, it has initiated anew line of work in which integer and linear programming are used in heuristic state-spacesearch for automated planning (Benton, van den Briel, & Kambhampati, 2007; van denBriel, Benton, Kambhampati, & Vossen, 2007).The remainder of this paper is organized as follows. In Section 2 we provide a briefbackground on integer programming and discuss some approaches that have used integerprogramming to solve planning problems. In Section 3 we present a series of integer pro-gramming formulations that each adopt a different network representation. We describe howwe set up these loosely coupled networks, provide the corresponding integer programmingformulation, and discuss the different variables and constraints. In Section 4 we describethe branch-and-cut algorithm that is used for solving these formulations. We provide ageneral background on the branch-and-cut concept and show how we apply it to our for-mulations by means of an example. Section 5 provides experimental results to determinewhich characteristics in our approach have the greatest impact on performance. Relatedwork is discussed in Section 6 and some conclusions are given in Section 7.
2. Background
Since our formulations are based on integer programming, we briefly review this techniqueand discuss its use in planning. A mixed integer program is represented by a linear objectivefunction and a set of linear inequalities:min { cx : Ax ≥ b, x , ..., x p ≥ , x p +1 , ..., x n ≥ } , where A is an ( m × n ) matrix, c is an n -dimensional row vector, b is an m -dimensionalcolumn vector, and x an n -dimensional column vector of variables. If all variables arecontinuous ( p = 0) we have a linear program , if all variables are integer ( p = n ) we havean integer program , and if x , ..., x p ∈ { , } we have a mixed 0-1 program . The set S = { x , ..., x p ≥ , x p +1 , ..., x n ≥ Ax ≥ b } is called the feasible region , and an n -dimensional column vector x is called a feasible solution if x ∈ S . Moreover, the function an den Briel, Vossen & Kambhampati cx is called the objective function , and the feasible solution x ∗ is called an optimal solution if the objective function is as small as possible, that is, cx ∗ = min { cx : x ∈ S } Mixed integer programming provides a rich modeling formalism that is more general thanpropositional logic. Any propositional clause can be represented by one linear inequality in0-1 variables, but a single linear inequality in 0-1 variables may require exponentially manyclauses (Hooker, 1988).The most widely used method for solving (mixed) integer programs is by applying abranch-and-bound algorithm to the linear programming relaxation , which is much easierto solve . The linear programming (LP) relaxation is a linear program obtained from theoriginal (mixed) integer program by relaxing the integrality constraints:min { cx : Ax ≥ b, x , ..., x n ≥ } Generally, the LP relaxation is solved at every node in the branch-and-bound tree, until(1) the LP relaxation gives an integer solution, (2) the LP relaxation value is inferior to thecurrent best feasible solution, or (3) the LP relaxation is infeasible, which implies that thecorresponding (mixed) integer program is infeasible.An ideal formulation of an integer program is one for which the solution of the lin-ear programming relaxation is integral. Even though every integer program has an idealformulation (Wolsey, 1998), in practice it is very hard to characterize the ideal formula-tion as it may require an exponential number of inequalities. In problems where the idealformulation cannot be determined, it is often desirable to find a strong formulation ofthe integer program. Suppose that the feasible regions P = { x ∈ R n : A x ≥ b } and P = { x ∈ R n : A x ≥ b } describe the linear programming relaxations of two IP formula-tions of a problem. Then we say that formulation for P is stronger than formulation for P if P ⊂ P . That is, the feasible region P is subsumed by the feasible region P . In otherwords P improves the quality of the linear relaxation of P by removing fractional extremepoints.There exist numerous powerful software packages that solve mixed integer programs. Inour experiments we make use of the commercial solver CPLEX 10.0 (Inc., 2002), which iscurrently one of the best LP/IP solvers.The use of integer programming techniques to solve artificial intelligence planning prob-lems has an intuitive appeal, especially given the success IP has had in solving similar typesof problems. For example, IP has been used extensively for solving problems in transporta-tion, logistics, and manufacturing. Examples include crew scheduling, vehicle routing, andproduction planning problems (Johnson, Nemhauser, & Savelsbergh, 2000). One potentialadvantage is that IP techniques can provide a natural way to incorporate several importantaspects of real-world planning problems, such as numeric constraints and objective functionsinvolving costs and utilities.Planning as integer programming has, nevertheless, received only limited attention. Oneof the first approaches is described by Bylander (1997), who proposes an LP heuristic forpartial order planning algorithms. While the LP heuristic helps to reduce the number ofexpanded nodes, the evaluation is rather time-consuming. In general, the performance of
2. While the integer programming problem is NP -complete (Garey & Johnson, 1979) the linear program-ming problem is polynomially solvable (Karmarkar, 1984). oosely Coupled Formulations for Automated Planning IP often depends on the structure of the problem and on how the problem is formulated.The importance of developing strong IP formulations is discussed by Vossen et al. (1999),who compare two formulations for classical planning: (1) a straightforward formulationbased on the conversion of the propositional representation by SATPLAN which yieldsonly mediocre results, and (2) a less intuitive formulation based on the representation ofstate transitions which leads to considerable performance improvements. Several ideas thatfurther improve formulation based on the representation of state transitions are described byDimopoulos (2001). Some of these ideas are implemented in the IP-based planner Optiplan(van den Briel & Kambhampati, 2005). Approaches that rely on domain-specific knowledgeare proposed by Bockmayr and Dimopoulos (1998, 1999). By exploiting the structure ofthe planning problem these IP formulations often provide encouraging results. The use ofLP and IP has also been explored for non-classical planning. Dimopoulos and Gerevini(2002) describe an IP formulation for temporal planning and Wolfman and Weld (1999)use LP formulations in combination with a satisfiability-based planner to solve resourceplanning problems. Kautz and Walser (1999) also solve resource planning problems, butuse domain-specific IP formulations.
3. Formulations
This section describes four IP formulations that model the planning problem as a collectionof loosely coupled network flow problems. Each network represents a state variable, in whichthe nodes correspond to the state variable values, and the arcs correspond to the valuetransitions. The state variables are based on the SAS+ planning formalism (B¨ackstr¨om &Nebel, 1995), which is a planning formalism that uses multi-valued state variables insteadof binary-valued atoms. An action in SAS+ is modeled by its pre-, post- and prevail-conditions. The pre- and post-conditions express which state variables are changed andwhat values they must have before and after the execution of the action, and the prevail-conditions specify which of the unchanged variables must have some specific value beforeand during the execution of an action. A SAS+ planning problem is described by a tupleΠ = h C, A, s , s ∗ i where: • C = { c , ..., c n } is a finite set of state variables, where each state variable c ∈ C has anassociated domain V c and an implicitly defined extended domain V + c = V c ∪{ u } , where u denotes the undefined value . For each state variable c ∈ C , s [ c ] denotes the value of c in state s . The value of c is said to be defined in state s if and only if s [ c ] = u . Thetotal state space S = V c × ... × V c n and the partial state space S + = V + c × ... × V + c n are implicitly defined. • A is a finite set of actions of the form h pre, post, prev i , where pre denotes the pre-conditions, post denotes the post-conditions, and prev denotes the prevail-conditions.For each action a ∈ A , pre [ c ] , post [ c ] and prev [ c ] denotes the respective conditions onstate variable c . The following two restrictions are imposed on all actions: (1) Oncethe value of a state variable is defined, it can never become undefined. Hence, for all c ∈ C , if pre [ c ] = u then pre [ c ] = post [ c ] = u ; (2) A prevail- and post-condition ofan action can never define a value on the same state variable. Hence, for all c ∈ C ,either post [ c ] = u or prev [ c ] = u or both. an den Briel, Vossen & Kambhampati • s ∈ S denotes the initial state and s ∗ ∈ S + denotes the goal state. While SAS+planning allows the initial state and goal state to be both partial states, we assumethat s is a total state and s ∗ is a partial state. We say that state s is satisfied bystate t if and only if for all c ∈ C we have s [ c ] = u or s [ c ] = t [ c ]. This implies that if s ∗ [ c ] = u for state variable c , then any defined value f ∈ V c satisfies the goal for c .To obtain a SAS+ description of the planning problem we use the translator componentof the Fast Downward planner (Helmert, 2006). The translator is a stand-alone componentthat contains a general purpose algorithm which transforms a propositional descriptionof the planning problem into a SAS+ description. The algorithm provides an efficientgrounding that minimizes the state description length and is based on the preprocessingalgorithm of the MIPS planner (Edelkamp & Helmert, 1999).In the remainder of this section we introduce some notation and describe our IP formu-lations. The formulations are presented in such a way that they progressively generalize theGraphplan-style parallelism through the incorporation of more flexible network representa-tions. For each formulation we will describe the underlying network, and define the variablesand constraints. We will not concentrate on the objective function as much because theconstraints will tolerate only feasible plans. For the formulations that are described in this paper we assume that the following infor-mation is given: • C : a set of state variables; • V c : a set of possible values (i.e. domain) for each state variable c ∈ C ; • E c : a set of possible value transitions for each state variable c ∈ C ; • G c = ( V c , E c ) : a directed domain transition graph for every c ∈ C ;State variables can be represented by a domain transition graph, where the nodes correspondto the possible values, and the arcs correspond to the possible value transitions. An exampleof the domain transition graph of a variable is given in Figure 3. While the example depictsa complete graph, a domain transition graph does not need to be a complete graph.Furthermore, we assume as given: • E ac ⊆ E c represents the effect of action a in c ; • V ac ⊆ V c represents the prevail condition of action a in c ; • A Ec := { a ∈ A : | E ac | > } represents the actions that have an effect in c , and A Ec ( e )represents the actions that have the effect e in c ; • A Vc := { a ∈ A : | V ac | > } represents the actions that have a prevail condition in c ,and A Vc ( f ) represents the actions that have the prevail condition f in c ; • C a := { c ∈ C : a ∈ A Ec ∪ A Vc } represents the state variables on which action a has aneffect or a prevail condition. oosely Coupled Formulations for Automated Planning hgf Figure 3: An example of a domain transition graph, where V c = { f, g, h } are the possiblevalues (states) of c and E c = { ( f, g ) , ( f, h ) , ( g, f ) , ( g, h ) , ( h, f ) , ( h, g ) } are thepossible value transitions in c .Hence, each action is defined by its effects (i.e. pre- and post-conditions) and its prevailconditions. In SAS+ planning, actions can have at most one effect or prevail condition ineach state variable. In other words, for each a ∈ A and c ∈ C , we have that E ac and V ac areempty or | E ac | + | V ac | ≤
1. An example of how the effects and prevail conditions affect oneor more domain transition graphs is given in Figure 4. hgfhgf hgf
Figure 4: An example of how action effects and prevail conditions are represented in adomain transition graph. Action a has implications on three state variables C a = { c , c , c } . The effects of a are represented by E ac = { ( f, g ) } and E ac = { ( h, f ) } ,and the prevail condition of a is represented by V ac = { h } .In addition, we use the following notation: • V + c ( f ): to denote the in-arcs of node f in the domain transition graph G c ; • V − c ( f ): to denote the out-arcs of node f in the domain transition graph G c ; • P + c,k ( f ): to denote paths of length k in the domain transition graph G c that end atnode f . Note that P + c, ( f ) = V + c ( f ). • P − c,k ( f ): to denote paths of length k in the domain transition graph G c that start atnode f . Note that P − c, ( f ) = V − c ( f ). an den Briel, Vossen & Kambhampati • P ∼ c,k ( f ): to denote paths of length k in the domain transition graph G c that visit node f , but that do not start or end at f . Our first IP formulation incorporates the network representation that we have seen inFigures 1 and 2. The name one state change relates to the number of transitions that weallow in each state variable per plan period. The restriction of allowing only one valuetransition in each network also restricts which actions we can execute in the same planperiod. It happens to be the case that the network representation of the 1SC formulationincorporates the standard notion of action parallelism which is used in Graphplan (Blum& Furst, 1995). The idea is that actions can be executed in the same plan period as longas they do not delete the precondition or add-effect of another action. In terms of valuetransitions in state variables, this is saying that actions can be executed in the same planperiod as long as they do not change the same state variable (i.e. there is only one valuechange or value persistence in each state variable).
Figure 5 shows a single layer (i.e. period) of the network which underlies the 1SC formula-tion. If we set up the IP formulation with T plan periods, then there will be T + 1 layersof nodes and T layers of arcs in the network (the zeroth layer of nodes is for the initialstate and the remaining T layers of nodes and arcs are for the successive plan periods). Foreach possible state transition there is an arc in the state change network. The horizontalarcs correspond to the persistence of a value, and the diagonal arcs correspond to the valuechanges. A solution path to an individual network follows the arcs whose transitions aresupported by the action effect and prevail conditions that appear in the solution plan. h gf h gf t Figure 5: One state change (1SC) network.
We have two types of variables in this formulation: action variables to represent the execu-tion of an action, and arc flow variables to represent the state transitions in each network. oosely Coupled Formulations for Automated Planning
We use separate variables for changes in a state variable (the diagonal arcs in the 1SCnetwork) and for the persistence of a value in a state variable (the horizontal arcs in the1SC network). The variables are defined as follows: • x at ∈ { , } , for a ∈ A, ≤ t ≤ T ; x at is equal to 1 if action a is executed at plan period t , and 0 otherwise. • ¯ y c,f,t ∈ { , } , for c ∈ C , f ∈ V c , 1 ≤ t ≤ T ; ¯ y c,f,t is equal to 1 if the value f of statevariable c persists at period t , and 0 otherwise. • y c,e,t ∈ { , } , for c ∈ C, e ∈ E c , ≤ t ≤ T ; y c,e,t is equal to 1 if the transition e ∈ E c in state variable c is executed at period t , and 0 otherwise. There are two classes of constraints. We have constraints for the network flows in eachstate variable network and constraints for the action effects that determine the interactionsbetween these networks. The 1SC integer programming formulation is: • State change flows for all c ∈ C , f ∈ V c X e ∈ V − c ( f ) y c,e, + ¯ y c,f, = (cid:26) f = s [ c ]0 otherwise . (1) X e ∈ V − c ( f ) y c,e,t +1 + ¯ y c,f,t +1 = X e ∈ V + c ( f ) y c,e,t + ¯ y c,f,t for 1 ≤ t ≤ T − X e ∈ V + c ( f ) y c,e,T + ¯ y c,f,T = 1 if f = s ∗ [ c ] (3) • Action implications for all c ∈ C , 1 ≤ t ≤ T X a ∈ A : e ∈ E ac x at = y c,e,t for e ∈ E c (4) x at ≤ ¯ y c,f,t for a ∈ A, f ∈ V ac (5)Constraints (1), (2), and (3) are the network flow constraints for state variable c ∈ C .Constraint (1) ensures that the path of state transitions begins in the initial state of thestate variable and constraint (3) ensures that, if a goal exists, the path ends in the goalstate of the state variable. Note that, if the goal value for state variable c is undefined(i.e. s ∗ [ c ] = u ) then the path of state transitions may end in any of the values f ∈ V c .Hence, we do not need a goal constraint for the state variables whose goal states s ∗ [ c ] areundefined. Constraint (2) is the flow conservation equation and enforces the continuity ofthe constructed path.Actions may introduce interactions between the state variables. For instance, the effectsof the load action in our logistics example affect two different state variables. Actions linkstate variables to each other and these interactions are represented by the action implication an den Briel, Vossen & Kambhampati constraints. For each transition e ∈ E c , constraints (4) link the action execution variablesthat have e as an effect (i.e. e ∈ E ac ) to the arc flow variables. For example, if an action x at with effect e ∈ E ac is executed, then the path in state variable c must follow the arcrepresented by y c,e,t . Likewise, if we choose to follow the arc represented by y c,e,t , thenexactly one action x at with e ∈ E ac must be executed. The summation on the left hand sideprevents two or more actions from interfering with each other, hence only one action maycause the state change e in state variable c at period t .Prevail conditions of an action link state variables in a similar way as the action effectsdo. Specifically, constraint (5) states that if action a is executed at period t ( x at = 1), then the prevail condition f ∈ V ac is required in state variable c at period t (¯ y c,f,t = 1). In our second formulation we incorporate the same network representation as in the 1SCformulation, but adopt a more general interpretation of the value transitions, which leadsto an unconventional notion of action parallelism. For the G1SC formulation we relax thecondition that parallel actions can be arranged in any order by requiring a weaker condition.We allow actions to be executed in the same plan period as long as there exists some orderingthat is feasible. More specifically, within a plan period a set of actions is feasible if (1) thereexists an ordering of the actions such that all preconditions are satisfied, and (2) there isat most one state change in each of the state variables. This generalization of conditionsis similar to what Rintanen, Heljanko and Niemel¨a (2006) refer to as the ∃ -step semanticssemantics.To illustrate the basic concept, let us again examine our small logistics example intro-duced in Figure 1. The solution to this problem is to load the package at location 1, drive thetruck from location 1 to location 2, and unload the package at location 2. Clearly, this planwould require three plan periods under Graphplan-style parallelism as these three actionsinterfere with each other. If, however, we allow the load at loc drive loc → loc some ordering of the actions that is viable. The question is, of course, howto incorporate this idea into an IP formulation.
21 21 2121 21 21 t t t
Load at loc1Drive loc1 → loc2Load at loc1 Unload at loc2Unload at loc2 truck-locationpack-location Figure 6: Logistics example represented by network flow problems with generalized arcs. oosely Coupled Formulations for Automated Planning
This example illustrates that we are looking for a set of constraints that allow sets ofactions for which: (1) all action preconditions are met, (2) there exists an ordering of theactions at each plan period that is feasible, and (3) within each state variable, the value ischanged at most once. The incorporation of these ideas only requires minor modificationsto the 1SC formulation. Specifically, we need to change the action implication constraintsfor the prevail conditions and add a new set of constraints which we call the orderingimplication constraints.
The minor modifications are revealed in the G1SC network. While the network itself isidentical to the 1SC network, the interpretation of the transition arcs is somewhat different.To incorporate the new set of conditions, we implicitly allow values to persist (the dashedhorizontal arcs in the G1SC network) at the tail and head of each transition arc. Theinterpretation of these implicit arcs is that in each plan period a value may be required asa prevail condition, then the value may change, and the new value may also be required asa prevail condition as shown in Figure 7. hg hf
Generalized state change arcPeriod tf ghgf hgf
G1SC networkPeriod t Figure 7: Generalized one state change (G1SC) network.
Since the G1SC network is similar to the 1SC network the same variables are used, thus,action variables to represent the execution of an action, and arc flow variables to representthe flow through each network. The difference in the interpretation of the state change arcsis dealt with in the constraints of the G1SC formulation, and therefore does not introduceany new variables. For the variable definitions, we refer to Section 3.2.2.
We now have three classes of constraints, that is, constraints for the network flows in eachstate variable network, constraints for linking the flows with the action effects and prevailconditions, and ordering constraints to ensure that the actions in the plan can be linearizedinto a feasible ordering. an den Briel, Vossen & Kambhampati
The network flow constraints for the G1SC formulation are identical to those in the 1SCformulation given by (1)-(3). Moreover, the constraints that link the flows with the actioneffects are equal to the action effect constraints in the 1SC formulation given by (4). TheG1SC formulation differs from the 1SC formulation in that it relaxes the condition thatparallel actions can be arranged in any order by requiring a weaker condition. This weakercondition affects the constraints that link the flows with the action prevail conditions, andintroduces a new set of ordering constraints. These constraints of the G1SC formulationare given as follows: • Action implications for all c ∈ C , 1 ≤ t ≤ Tx at ≤ ¯ y c,f,t + X e ∈ V + c ( f ) y c,e,t + X e ∈ V − c ( f ) y c,e,t for a ∈ A, f ∈ V ac (6) • Ordering implications X a ∈ V (∆) x at ≤ | V (∆) | − ∈ G prec (7)Constraint (6) incorporates this new set of conditions for which actions can be executedin the same plan period. In particular, we need to ensure that for each state variable c , thevalue f ∈ V c holds if it is required by the prevail condition of action a at plan period t . Thereare three possibilities: (1) The value f holds for c throughout the period. (2) The value f holds initially for c , but the value is changed to a value other than f by another action. (3)The value f does not hold initially for c , but the value is changed to f by another action.In either of the three cases the value f holds at some point in period t so that the prevailcondition for action a can be satisfied. In words, the value f may prevail implicitly as longas there is a state change that includes f . As before, the prevail implication constraintslink the action prevail conditions to the corresponding network arcs.The action implication constraints ensure that the preconditions of the actions in theplan are satisfied. This, however, does not guarantee that the actions can be linearizedinto a feasible order. Figure 7 indicates that there are implied orderings between actions.Actions that require the value f as a prevail condition must be executed before the actionthat changes f into g . Likewise, an action that changes f into g must be executed beforeactions that require the value g as a prevail condition. The state change flow and actionimplication constraints outlined above indicate that there is an ordering between the actions,but this ordering could be cyclic and therefore infeasible. To make sure that an orderingis acyclic we start by creating a directed implied precedence graph G prec = ( V prec , E prec ).In this graph the nodes a ∈ V prec correspond to the actions, that is, V prec = A , and wecreate a directed arc (i.e. an ordering) between two nodes ( a, b ) ∈ E prec if action a has to beexecuted before action b in time period t , or if b has to be executed after a . In particular,we have E prec = [ ( a,b ) ∈ A × A,c ∈ C,f ∈ V ac ,e ∈ E bc : e ∈ V − c,f ( a, b ) ∪ [ ( a,b ) ∈ A × A,c ∈ C,g ∈ V bc ,e ∈ E ac : e ∈ V + c,g ( a, b ) oosely Coupled Formulations for Automated Planning The implied orderings become immediately clear from Figure 8. The figure on the leftdepicts the first set of orderings in the expression of E prec . It says that the ordering betweentwo actions a and b that are executed in the same plan period is implied if action a requiresa value to prevail that action b deletes. Similarly, the figure on the right depicts second setof orderings in the expression of E prec . That is, an ordering is implied if action a adds theprevail condition of b . hg hff g hg hff g a b a b Figure 8: Implied orderings for the G1SC formulation.The ordering implication constraints ensure that the actions in the final solution can belinearized. They basically involve putting an n -ary mutex relation between the actions thatare involved in each cycle. Unfortunately, the number of ordering implication constraintsgrows exponentially in the number of actions. As a result, it will be impossible to solvethe resulting formulation using standard approaches. We address this complication byimplementing a branch-and-cut approach in which the ordering implication constraints areadded dynamically to the formulation. This approach is discussed in Section 4. k State Change (G k SC) Formulation
In the G1SC formulation actions can be executed in the same plan period if (1) there existsan ordering of the actions such that all preconditions are satisfied, and (2) there occurs atmost one value change in each of the state variables. One obvious generalization of thiswould be to relax the second condition and allow at most k c value changes in each statevariable c , where k c ≤ | V c | −
1. By allowing multiple value changes in a state variable perplan period we, in fact, permit a series of value changes. Specifically, the G k SC modelallows series of value changes.Obviously, there is a tradeoff between loosening the networks versus the amount ofwork it takes to merge the individual plans. While we have not implemented the G k SCformulation, we provide some insight in this tradeoff by describing and evaluating the G k SCformulation with k c = 2 for all c ∈ C We will refer to this special case as the generalizedtwo state change (G2SC) formulation. One reason we restrict ourselves to this special caseis that the general case of k state changes would introduce exponentially many variablesin the formulation. There are IP techniques, however, that deal with exponentially manyvariables (Desaulniers, Desrosiers, & Solomon, 2005), but we will not discuss them here. The network that underlies the G2SC formulation is equivalent to G1SC, but spans an extralayer of nodes and arcs. This extra layer allows us to have a series of two transitions per plan an den Briel, Vossen & Kambhampati period. All transitions are generalized and implicitly allow values to persist just as in theG1SC network. Figure 9 displays the network corresponding to the G2SC formulation. Inthe G2SC network there are generalized one and two state change arcs. For example, thereis a generalized one state change arc for the transition ( f, g ), and there is a generalized twostate changes arc for the transitions { ( f, g ) , ( g, h ) } . Since all arcs are generalized, each valuethat is visited can also be persisted. We also allow cyclic transitions, such as, { ( f, g ) , ( g, f ) } if f is not the prevail condition of some action. If we were to allow cyclic transitions inwhich f is a prevail condition of an action, then the action ordering in a plan period can notbe implied anymore (i.e. the prevail condition on f would either have to occur before thevalue transitions to g , or after it transitions back to f ). Thus if there is no prevail conditionon f then we can safely allow the cyclic transition { ( f, g ) , ( g, f ) } . hg f hgf hg f Period t hg f hg f Period t fgh Figure 9: Generalized two state change (G2SC) network. On the left the subnetwork thatconsists of generalized one state change arcs and no-op arcs, on the right the sub-network that consists of the generalized two state change arcs. The subnetworkfor the two state change arcs may include cyclic transitions, such as, { ( f, g ) , ( g, f ) } as long as f is not the prevail condition of some action. As before we have variables representing the execution of an action, and variables repre-senting the flows over one state change (diagonal arcs) or persistence (horizontal arcs). Inaddition, we have variables representing paths over two consecutive state changes. Hence,we have variables for each pair of state changes ( f, g, h ) such that ( f, g ) ∈ E c and ( g, h ) ∈ E c .We will restrict these paths to visit unique values only, that is, f = g , g = h , and h = f ,or if f is not a prevail condition of any action then we also allow paths where f = h . Thevariables from the G1SC formulation are also used in G2SC formulation. There is, however,an additional variable to represent the arcs that allow for two state changes: • y c,e ,e ,t ∈ { , } , for c ∈ C , ( e , e ) ∈ P c, , 1 ≤ t ≤ T ; y c,e ,e ,t is equal to 1 ifthere exists a value f ∈ V c and transitions e , e ∈ E c , such that e ∈ V + c ( f ) and e ∈ V − c ( f ), in state variable c are executed at period t , and 0 otherwise. oosely Coupled Formulations for Automated Planning We again have our three classes of constraints, which are given as follows: • State change flows for all c ∈ C , f ∈ V c X ( e ,e ) ∈ P − c, ( f ) y c,e ,e , + X e ∈ V − c ( f ) y c,e, + ¯ y c,f, = (cid:26) f = s [ c ]0 otherwise . (8) X ( e ,e ) ∈ P − c, ( f ) y c,e ,e ,t +1 + X e ∈ V − c ( f ) y c,e,t +1 + ¯ y c,f,t +1 = X ( e ,e ) ∈ P + c, ( f ) y c,e ,e ,t + X e ∈ V + c ( f ) y c,e,t + ¯ y c,f,t for 1 ≤ t ≤ T − X ( e ,e ) ∈ P + c, ( f ) y c,e ,e ,T + X e ∈ V + c ( f ) y c,e,T + ¯ y c,f,T = 1 if { f ∈ s ∗ [ c ] } (10) • Action implications for all c ∈ C , 1 ≤ t ≤ T X a ∈ A : e ∈ E ac x at = y c,e,t + X ( e ,e ) ∈ P c, : e = e ∨ e = e y c,e ,e ,t for e ∈ E c (11) x at ≤ ¯ y c,f,t + X e ∈ V + c ( f ) y c,e,t + X e ∈ V − c ( f ) y c,e,t + X ( e ,e ) ∈ P ∼ c, ( f ) y c,e ,e ,t + X ( e ,e ) ∈ P + c, ( f ) y c,e ,e ,t + X ( e ,e ) ∈ P − c, ( f ) y c,e ,e ,t for a ∈ A, f ∈ V ac (12) • Ordering implications X a ∈ V (∆) x at ≤ | V (∆) | − ∈ G prec (13)Constraints (8), (9), and (10) represent the flow constraints for the G2SC network. Con-straints (11) and (12) link the action effects and prevail conditions with the correspondingflows, and constraint 13 ensures that the actions can be linearized into some feasible order-ing. There are several ways to generalize the network representation of the G1SC formulationand loosen the interaction between the networks. The G k SC formulation presented onegeneralization that allows up to k transitions in each state variable per plan period. Sinceit uses exponentially many variables another way to generalize the network representationof the G1SC formulation is by requiring that each value can be true at most once per planperiod. To illustrate this idea we consider our logistics example again, but we now use an den Briel, Vossen & Kambhampati
21 2121 21 t t
Load at loc1Drive loc1 → loc2Unload at loc2-Load at loc1-Unload at loc2- truck-locationpack-location Figure 10: Logistics example represented by network flow problems that allow a path ofvalue transitions per plan period such that each value can be true at most once.a network representation that allows a path of transitions per plan period as depicted inFigure 10.Recall that the solution to the logistics example consists of three actions: first load thepackage at location 1, then drive the truck from location 1 to location 2, and last unloadthe package at location 2. Clearly, this solution would not be allowed within a single planperiod under Graphplan-style parallelism. Moreover, it would also not be allowed withina single period in the G1SC formulation. The reason for this is that the number of valuechanges in the package - location state variable is two. First, it changes from pack - at - loc pack - in - truck , and then it changes from pack - in - truck to pack - at - loc
2. As before, however,there does exists an ordering of the three actions that is feasible. The key idea behind thisexample is to show that we can allow multiple value changes in a single period. If we limitthe value changes in a state variable to simple paths, that is, in one period each value isvisited at most once, then we can still use implied precedences to determine the orderingrestrictions.
In this formulation each value can be true at most once in each plan period, hence thenumber of value transitions for each plan period is limited to k c where k c = | V c | − c ∈ C . In the PathSC network, nodes appear in layers and correspond to the values of thestate variable. However, each layer now consists of twice as many nodes. If we set up anIP encoding with a maximum number of plan periods T then there will be T layers. Arcswithin a layer correspond to transitions or to value persistence, and arcs between layersensure that all plan periods are connected to each other.Figure 11 displays a network corresponding to the state variable c with domain V c = { f, g, h } that allows multiple transitions per plan period. The arcs pointing rightwardscorrespond to the persistence of a value, while the arcs pointing leftwards correspond to thevalue changes. If more than one plan period is needed the curved arcs pointing rightwards oosely Coupled Formulations for Automated Planning link the layers between two consecutive plan periods. Note that with unit capacity on thearcs, any path in the network can visit each node at most once. hgf hgf PathSC networkPeriod t Figure 11: Path state change (PathSC) network.
We now have action execution variables and arc flow variables (as defined in the previousformulations), and linking variables that connect the networks between two consecutivetime periods. These variables are defined as follows: • z c,f,t ∈ { , } , for c ∈ C, f ∈ V c , ≤ t ≤ T ; z c,f,t is equal to 1 if the value f of statevariable c is the end value at period t , and 0 otherwise. As in the previous formulations, we have state change flow constraints, action implicationconstraints, and ordering implication constraints. The main difference is the underlyingnetwork. The PathSC integer programming formulation is given as follows: • State change flows for all c ∈ C , f ∈ V c z c,f, = (cid:26) f = s [ c ]0 otherwise . (14) X e ∈ V + c ( f ) y c,e,t + z c,f,t − = ¯ y c,f,t (15)¯ y c,f,t = X e ∈ V − c ( f ) y c,e,t + z c,f,t for 1 ≤ t ≤ T − z c,f,T = 1 if f ∈ s ∗ [ c ] (17) • Action implications for all c ∈ C , 1 ≤ t ≤ T X a ∈ A : e ∈ E ac x at = y c,e,t for e ∈ E c (18) x at ≤ ¯ y c,f,t for f ∈ V ac (19) an den Briel, Vossen & Kambhampati • Ordering implications X a ∈ V (∆) x at ≤ | V (∆) | − ∈ G prec ′ (20)Constraints (14)-(17) are the network flow constraints. For each node, except for the initialand goal state nodes, they ensure a balance of flow (i.e. flow-in must equal flow-out). Theinitial state node has a supply of one unit of flow and the goal state node has a demand ofone unit of flow, which are given by constraints (14) and (17) respectively. The interactionsthat actions impose upon different state variables are represented by the action implicationconstraints (18) and (19), which have been discussed earlier.The implied precedence graph for this formulation is given by G prec ′ = ( V prec ′ , E prec ′ ).It has an extra set of arcs to incorporate the implied precedences that are introduced whentwo actions imply a state change in the same class c ∈ C . The nodes a ∈ V prec ′ againcorrespond to actions, and there is an arc ( a, b ) ∈ E prec ′ if action a has to be executedbefore action b in the same time period, or if b has to be executed after a . More specifically,we have E prec ′ = E prec ∪ [ ( a,b ) ∈ A × A,c ∈ C,f ∈ V c ,e ∈ E ac ,e ′ ∈ E bc : e ∈ V + c ( f ) ∧ e ′ ∈ V − c ( f ) ( a, b )As before, the ordering implication constraints (20) ensure that the actions in the solu-tion plan can be linearized into a feasible ordering.
4. Branch-and-Cut Algorithm
IP problems are usually solved with an LP-based branch-and-bound algorithm. The basicstructure of this technique involves a binary enumeration tree in which branches are prunedaccording to bounds provided by the LP relaxation. The root node in the enumeration treerepresents the LP relaxation of the original IP problem and each other node represents asubproblem that has the same objective function and constraints as the root node exceptfor some additional bound constraints. Most IP solvers use an LP-based branch-and-boundalgorithm in combination with various preprocessing and probing techniques. In the lastfew years there has been significant improvement in the performance of these solvers (Bixby,2002).In an LP-based branch-and-bound algorithm, the LP relaxation of the original IP prob-lem (the solution to the root node) will rarely be integer. When some integer variable x has a fractional solution v we branch to create two new subproblems, such that the boundconstraint x ≤ ⌊ v ⌋ is added to the left-child node, and x ≥ ⌈ v ⌉ is added to the right-childnode. This branching process is carried out recursively to expand those subproblems whosesolution remains fractional. Eventually, after enough bounds are placed on the variables, aninteger solution is found. The value of the best integer solution found so far, Z ∗ , is referredto as the incumbent and is used for pruning. oosely Coupled Formulations for Automated Planning In a minimization problem, branches emanating from nodes whose solution value Z LP isgreater than the current incumbent, Z ∗ , can never give rise to a better integer solution aseach child node has a smaller feasible region than its parent. Hence, we can safely eliminatesuch nodes from further consideration and prune them. Nodes whose feasible region havebeen reduced to the empty set, because too many bounds are placed on the variables, canbe pruned as well.When solving an IP problem with an LP-based branch-and-bound algorithm we mustconsider the following two decisions. If several integer variables have a fractional solution,which variable should we branch on next, and if the branch we are currently working onis pruned, which subproblem should we solve next? Basic rules include use the “mostfractional variable” rule for branching variable selection and the “best objective value” rulefor node selection.For our formulations a standard LP-based branch-and-bound algorithm approach is veryineffective due to the large number (potentially exponentially many) ordering implicationconstraints in the G1SC, G2SC, and PathSC formulations. While it is possible to reducethe number of constraints by introducing additional variables (Martin, 1991), the resultingformulations would still be intractable for all but the smallest problem instances. Therefore,we solve the IP formulations with a so-called branch-and-cut algorithm, which considers theordering implication constraints implicitly. A branch-and-cut algorithm is a branch-and-bound algorithm in which certain constraints are generated dynamically throughout thebranch-and-bound tree. A flowchart of our branch-and-cut algorithm is given in Figure 12.If, after solving the LP relaxation, we are unable to prune the node on the basis of theLP solution, the branch-and-cut algorithm tries to find a violated cut, that is, a constraintthat is valid but not satisfied by the current solution. This is also known as the separationproblem . If one or more violated cuts are found, the constraints are added to the formulationand the LP is solved again. If none are found, the algorithm creates a branch in theenumeration tree (if the solution to the current subproblem is fractional) or generates afeasible solution (if the solution to the current subproblem is integral).The basic idea of branch-and-cut is to leave out constraints from the LP relaxationof which there are too many to handle efficiently, and add them to the formulation onlywhen they become binding at the solution to the current LP. Branch-and-cut algorithmshave successfully been applied in solving hard large-scale optimization problems in a widevariety of applications including scheduling, routing, graph partitioning, network design,and facility location problems (Caprara & Fischetti, 1997).In our branch-and-cut algorithm we can stop as soon as we find the first feasible solution,or we can implicitly enumerate all nodes (through pruning) and find the optimal solutionfor a given objective function. Note that our formulations can only be used to find boundedlength optimal plans. That is, find the optimal plan given a plan period (i.e. a boundedlength). In our experimental results, however, we focus on finding feasible solutions. At any point during runtime that the cut generator is called we have a solution to thecurrent LP problem, which consists of the LP relaxation of the original IP problem plusany added bound constraints and added cuts. In our implementation of the branch-and-cut an den Briel, Vossen & Kambhampati
Initialize LP Node selectionLP solver BranchingCut generatorSTART Nodes found?Cuts found? STOPFeasible? Prune Z LP > Z * ? noyes yesnoyes no yes noInteger?noyesOptimize? yesno Figure 12: Flowchart of our branch-and-cut algorithm. For finding any feasible solution (i.e.optimize = no) the algorithm stops as soon as the first feasible integer solutionis found. When searching for the optimal solution (i.e. optimize = yes) for thegiven formulation we continue until no open nodes are left. oosely Coupled Formulations for Automated Planning algorithm, we start with an LP relaxation in which the ordering implication constraints areomitted. So given a solution to the current LP relaxation, which could be fractional, theseparation problem is to determine whether the solution violates one of the omitted orderingimplication constraints. If so, we identify the violated ordering implication constraints, addthem to the formulation, and resolve the new problem.
In the G1SC, G2SC, and PathSC formulations an ordering implication constraint is vio-lated if there is a cycle in the implied precedence graph. Separation problems involvingcycles occur in numerous applications. Probably the best known of its kind is the travel-ing salesman problem in which subtours (i.e. cycles) are identified and subtour eliminationconstraints are added to the current LP. Our algorithm for separating cycles is based onthe one described by Padberg and Rinaldi (1991). We are interested in finding the shortestcycle in the implied precedence graph, as the shortest cycle cuts off more fractional extremepoints. The general idea behind this approach is as follows:1. Given a solution to the LP relaxation, determine the subgraph G t for plan period t consisting of all the nodes a for which x at > a, b ) ∈ G t , define the weights w a,b := x at + x bt − d a,b for all pairs (( a, b ) ∈ G t ) based on arcweights ¯ w a,b := 1 − w a,b (for example, using the Floyd-Warshall all-pairs shortest pathalgorithm).4. If d a,b − w b,a < a, b ) ∈ G t , there exists a violated cycle constraint.While the general principles behind branch-and-cut algorithms are rather straightfor-ward, there are a number of algorithmic and implementation issues that may have a signif-icant impact on overall performance. At the heart of these issues is the trade-off betweencomputation time spent at each node in the enumeration tree and the number of nodesthat are explored. One issue, for example, is to decide when to generate violated cuts.Another issue is which of the generated cuts (if any) should be added to the LP relaxation,and whether and when to delete constraints that were added to the LP before. In ourimplementation, we have only addressed these issues in a straightforward manner: cuts aregenerated at every node in the enumeration tree, the first cut found by the algorithm isadded, and constraints are never deleted from the LP relaxation. However, given the po-tential of more advanced strategies that has been observed in other applications, we believethere still may be considerable room for improvement. In this section we will show the workings of our branch-and-cut algorithm on the G1SCformulation using a small hypothetical example involving two state variables c and c , fiveactions A A A A
4, and A
5, and one plan period. In particular we will show how thecycle detection procedure works and how an ordering implication constraint is generated. an den Briel, Vossen & Kambhampati
Figure 13 depicts a solution to the current LP of the planning problem. For state variable c we have that actions A A g , A h , and action A g into h . Likewise, for state variable c wehave that action A g into f , action A g into h , andaction A f . Note that the given solution is fractional. Thereforesome of the action variables have fractional values. In particular, we have x A = x A = 0 . x A = 0 .
2, and x A = x A = 1. In other words, actions A A A A A A A c . While this would generally be infeasible,the actions are executed only fractionally, so this is actually a feasible solution to the LPrelaxation of the IP formulation. hg hff gA A A A hg hff gA A A A A A A A A A Figure 13: Solution to a small hypothetical planning example. The solution to the currentLP has flows over the indicated paths and executes actions A A A A
4, and A G prec = ( V prec , E prec ), where we have V prec = { A , A , A , A , A } and E prec = { ( A , A A , A A , A A , A } . The or-dering ( A , A c . A g in c while A g to h in c , which implies that A A
3. The other orderings are established in a similar way. Thecomplete implied precedence graph for this example is given in Figure 14.The cycle detection algorithm gets the implied precedence graph and the solution to thecurrent LP as input. Weights for each arc ( a, b ) ∈ E prec are determined by the values of theaction variables in the current solution. We have the LP solution that is given in Figure13, so in this example we have w A ,A = w A ,A = 0 . w A ,A = 1, and w A ,A = 0 .
6. Thelength of the shortest path from A A w a,b is equal to 0 . . . d A ,A = 0 . w A ,A = 0 .
6. Since d A ,A − w A ,A <
0, we have a violated cycle(i.e. violated ordering implication) that includes all actions that are on the shortest pathfrom A A A A
3, and A
4, which can be retrieved by the shortest path algorithm). oosely Coupled Formulations for Automated Planning
This generates the following ordering implication constraint x A + x A + x A ≤
2, which willbe added to the current LP. Note that this ordering constraint is violated by the currentLP solution, as x A + x A + x A = 0 . . .
6. Once the constraint is added to theLP, the next solution will select a set of actions that does not violate the newly added cut.This procedure continues until no cuts are violated and the solution is integer. A2 A3 A4A1
Implied precedence graph(0.6,0.4) (0.8,0.2)(0.8,0.2)(1,0) A5 Figure 14: Implied precedence graph for this example, where the labels show ( w a,b , ¯ w a,b ).
5. Experimental Results
The described formulations are based on two key ideas. The first idea is to decompose theplanning problem into several loosely coupled components and represent these componentsby an appropriately defined network. The second idea is to reduce the number of planperiods by adopting different notions of parallelism and use a branch-and-cut algorithmto dynamically add constraints to the formulation in order to deal with the exponentiallymany action ordering constraints in an efficient manner.To evaluate the tradeoffs of allowing more flexible network representations we comparethe performance of the one state change (1SC) formulation, the generalized one state changeformulation (G1SC), the generalized two state change (G2SC) formulation, and the statechange path (PathSC) formulation. For easy reference, an overview of these formulationsis given in Figure 15.In our experiments we focus on finding feasible solutions. Note, however, that ourformulations can be used to do bounded length optimal planning. That is, given a planperiod (i.e. a bounded length), find the optimal solution.
To compare and analyze our formulations we use the STRIPS domains from the secondand third international planning competitions (IPC2 and IPC3 respectively). That is,Blocksworld, Logistics, Miconic, Freecell from IPC2 and Depots, Driverlog, Zenotravel,Rovers, Satellite, and Freecell from IPC3. We do not compare our formulations on theSTRIPS domains from IPC4 and IPC5 mainly because of a peripheral limitation of thecurrent implementation of the G2SC and PathSC formulations. In particular, the G2SCformulation cannot handle operators that change a state variable from an undefined valueto a defined value, and the PathSC formulation cannot handle such operators if the domain an den Briel, Vossen & Kambhampati
Each state variable can change or Each state variable can change (andprevail a value at most once per prevail a value before and after eachplan period. change) at most once per plan period. hgf hgf g hff gh
G2SC PathSC
Each state variable can change (and The state variable can change anyprevail a value before and after each number of times, but each valuechange) at most twice per plan can be true at most once per planperiod. Cyclic changes ( f, g, f ) are period.allowed only if f is not theprevail condition of some action hgf hgf hgffgh hgf hgf Figure 15: Overview of the 1SC, G1SC, G2SC, and PathSC formulations.size of the state variable is larger than two. Because of these limitations we could not testthe G2SC formulation on the Miconic, Satellite and Rovers domains, and we could not testthe PathSC formulation on the Satellite domain.In order to setup our formulations we translate a STRIPS planning problem into amulti-valued state description using the translator of the Fast Downward planner (Helmert,2006). Each formulation uses its own network representation and starts by setting thenumber of plan periods T equal to one. We try to solve this initial formulation and if noplan is found, T is increased by one, and then try to solve this new formulation. Hence,the IP formulation is solved repeatedly until the first feasible plan is found or a 30 minutetime limit (the same time limit that is used in the international planning competitions) isreached. We use CPLEX 10.0 (ILOG Inc., 2002), a commercial LP/IP solver, for solvingthe IP formulations on a 2.67GHz Linux machine with 1GB of memory.We set up our experiments as follows. First, in Section 5.2 we provide a brief overview ofour main results by looking at aggregated results from IPC2 and IPC3. Second, in Section5.3, we give a more detailed analysis on our loosely coupled encodings for planning and oosely Coupled Formulations for Automated Planning focus on the tradeoffs of reducing the number of plan periods to solve a planning problemversus the increased difficulty in merging the solutions to the different components. Third,in Section 5.4 we briefly touch upon how different state variable representations of the sameplanning problem can influence performance. In this general overview we compare our formulations to the following planning systems:Optiplan (van den Briel & Kambhampati, 2005), SATPLAN04 (Kautz, 2004), SATPLAN06(Kautz & Selman, 2006), and Satplanner (Rintanen et al., 2006) .Optiplan is an integer programming based planner that participated in the optimal trackof the fourth international planning competition . Like our formulations, Optiplan modelsstate transitions but it does not use a factored representation of the planning domain.In particular, Optiplan represents state transitions in the atoms of the planning domain,whereas our formulations use multi-valued state variables. Apart from this, Optiplan isvery similar to the 1SC formulation as they both adopt the Graphplan-style parallelism.SATPLAN04, SATPLAN06, and Satplanner are satisfiability based planners. SAT-PLAN04 and SATPLAN06 are versions of the well known system SATPLAN (Kautz &Selman, 1992), which has a long track record in the international planning competitions.Satplanner has not received that much attention, but is among the state-of-the-art in plan-ning as satisfiability. Like our formulations Satplanner generalizes the Graphplan-styleparallelism to improve planning efficiency.The main results are summarized by Figure 16. It displays aggregate results from IPC2and IPC3, where the number of instances solved (y-axis) is drawn as a function of log time(x-axis). We must note that the graph with the IPC2 results favors the PathSC formulationover all other planners. However, as we will see in Section 5.3, this is mainly a reflectionof its exceptional performance in the Miconic domain rather than its overall performancein IPC2. Morever, the graph with the IPC3 results does not include the Satellite domain.We decided to remove this domain, because we could not run it on the public versions ofSATPLAN04 and SATPLAN06 nor the G2SC and PathSC formulations. While the resultsin Figure 16 provide a rather coarse overview, they sum up the following main findings. • Factored planning using loosely coupled formulations helps improve performance . Notethat all integer programming formulations that use factored representations, that is1SC, G1SC, G2SC, and PathSC (except the G2SC formulation which could not be
3. We note that that SATPLAN04, SATPLAN06, Optiplan, and the 1SC formulation are “step-optimal”while the G1SC, G2SC, and PathSC formulations are not. There is, however, considerable controversyin the planning community as to whether the step-optimality guaranteed by Graphplan-style plannershas any connection to plan quality metrics that users would be interested in. We refer the reader toKambhampati (2006) for a longer discussion of this issue both by us and several prominent researchersin the planning community. Given this background, we believe it is quite reasonable to compare our for-mulations to step-optimal approaches, especially since our main aim here is to show that IP formulationshave come a long way and that they can be made competitive with respect to SAT-based encodings.This in turn makes it worthwhile to consider exploiting other features of IP formulations, such as theiramenability to a variety of optimization objectives as we have done in our recent work (van den Brielet al., 2007).4. A list of participating planners and their results is available at http://ipc04.icaps-conference.org/ an den Briel, Vossen & Kambhampati tested on all domains), are able to solve more problem instances in a given amountof time than Optiplan, which does not use a factored representation. Especially, thedifference between 1SC and Optiplan is remarkable as they both adopt the Graphplan-style parallelism. In Section 5.3, however, we will see that Optiplan does perform wellin domains that are either serial by design or have a significant serial component. • Decreasing the encoding size by relaxing the Graphplan-style parallelism helps improveperformance . This is not too surprising, Dimopoulos et al. (1997) already note that areduction in the number of plan periods helps improve planning performance. How-ever, this does not always hold because of the tradeoff between reducing the numberplan periods versus the increased difficulty in merging the solutions to the differentcomponents. In Section 5.3 we will see that different relaxations of Graphplan-styleparallelism lead to different results. For example, the PathSC formulation shows supe-rior performance in Miconic and Driverlog, but does poorly in Blocksworld, Freecell,and Zenotravel. Likewise, the G2SC formulation does well in Freecell, but it does notseem to excel in any other domain. • Planning as integer programming shows new potential . The conventional wisdom inthe planning community has been that planning as integer programming cannot com-pete with planning as satisfiability or constraint satisfaction. In Figure 16, however,we see that the 1SC, G1SC and PathSC formulation can compete quite well withSATPLAN04. While SATPLAN04 is not state-of-the-art in planning as satisfiabil-ity anymore, it does show that planning as integer programming has come a longway. The fact that IP is competitive allows us to exploit its other virtues such asoptimization (Do et al., 2007; Benton et al., 2007; van den Briel et al., 2007).
Solution time (sec) S o l ve d i n t s t a n ces ( I P C ) Satplanner SAT06SAT04 1SCG1SC G2SCPathSC Optiplan
Solution time (sec) S o l ve d i n t s t a n ces ( I P C ) Figure 16: Aggregate results of the second and third international planning competitions.
In this section we compare our IP formulations and try to evaluate the benefits of allowingmore flexible network representations. Specifically, we are interested in the effects of reduc-ing the number of plan periods required to solve the planning problem versus dealing with oosely Coupled Formulations for Automated Planning merging solutions to the different components. Reducing the number of plan periods canlead to smaller encodings, which can lead to improved performance. However, it also makesthe merging of the loosely coupled components harder, which could worsen performance.In order to compare our formulations we will analyze the following two things. First, weexamine the performance of our formulations by comparing their solution times on probleminstances from IPC2 and IPC3. In this comparison we will include results from Optiplanas it gives us an idea of the differences between a formulation based on Graphplan andformulations based on loosely coupled components. Moreover, it will also show us theimprovements in IP based approaches for planning. Second, we examine the number ofplan periods that each formulation needs to solve each problem instance. Also, we will lookat the tradeoffs between reducing the number of plan periods and the increased difficultyin merging the solutions of the loosely coupled components. In this comparison we willinclude results from Satplanner because, just like our formulations, it adopts a generalizednotion of the Graphplan-style parallelism.We use the following figures and table. Figure 17 shows the total solution time (y-axis)needed to solve the problem instances (x-axis), Figure 18 shows the number of plan periods(y-axis) to solve the problem instances (x-axis), and Table 1 shows the number of orderingconstraints that were added during the solution process, which can be seen as an indicatorof the merging effort. The selected problem instances in Table 1 represent the five largestinstances that could be solved by all of our formulations (in some domains, however, notall formulations could solve at least five problem instances).The label
GP steps in Figure 18 represents the number of plan steps that SATPLAN06,a state-of-the-art Graphplan-based planner, would use. In the Satellite domain, however,we use the results from the 1SC formulation as we were unable to run the public versionof SATPLAN06 in this domain. We like to point out that Figure 18 is not intended tofavor one formulation over the other, it simply shows that it is possible to generate encod-ings for automated planning that use drastically fewer plan periods than Graphplan-basedencodings.
Blocksworld is the only domain in which Optiplan solves more problems than our formu-lations. In Zenotravel and Satellite, Optiplan is generally outperformed with respect tosolution time, and in Rovers and Freecell, Optiplan is generally outperformed with respectto the number of problems solved. As for the other IP formulations, the G1SC providesthe overall best performance and the performance of the PathSC formulation is somewhatirregular. For example, in Miconic, Driverlog and Rovers the PathSC formulation does verywell, but in Depots and Freecell it does rather poorly.In the Logistics domain all formulations that generalize the Graphplan-style parallelism(i.e. G1SC, G2SC, and PathSC) scale better than the 1SC formulation and Optiplan, whichadopt the Graphplan-style parallelism. Among G1SC, G2SC, and PathSC formulationsthere is no clear best performer, but in the larger Logistics problems the G1SC formulationseems to do slightly better. The Logistics domain provides a great example of the tradeoffbetween flexibility and merging. By allowing more actions to be executed in each planperiod, generally shorter plans (in terms of number of plan periods) are needed to solve an den Briel, Vossen & Kambhampati the planning problem (see Figure 18), but at the same time merging the solutions to theindividual components will be harder as one has to respect more ordering constraints (seeTable 1).
Optiplan versus 1SC.
If we compare the 1SC formulation with Optiplan, we notethat Optiplan fares well in domains that are either serial by design (Blocksworld) or indomains that have a significant serial aspect (Depots). We think that Optiplan’s advantageover the 1SC formulation in these domains is due to the following two possibilities. First,our intuition is that in serial domains the reachability and relevance analysis in Graphplan isstronger in detecting infeasible action choices (due to mutex propagation) than the networkflow restrictions in the 1SC formulation. Second, it appears that the state variables in thesedomains are more tightly coupled (i.e. the actions have more effects, thus transitions in onestate variable are coupled with several transitions in other state variables) than in mostother domains, which may negatively affect the performance of the 1SC formulation.
When comparing the 1SC formulation with the G1SC formulationwe can see that in all domains, except in Blocksworld and Miconic, the G1SC formulationsolves at least as many problems as the 1SC formulation. The results in Blocksworld arenot too surprising and can be attributed to semantics of this domain. Each operator inBlocksworld requires one state change in the state variable of the arm ( stack and putdown change the status of the arm to arm − empty , and unstack and pickup change the statusof the arm to holding − x where x is the block being lifted). Since, the 1SC and theG1SC formulations both allow at most one state change in each state variable, there is nopossibility for the G1SC formulation to allow more than one action to be executed in thesame plan period. Given this, one may think that the 1SC and G1SC formulations shouldsolve at least the same number of problems, but in this case the prevail constraints (5) ofthe 1SC formulation are stronger than the prevail constraints (6) of the G1SC formulation.That is, the right-hand side of (6) subsumes (i.e. allows for a larger feasible region in the LPrelaxation) than the right-hand side of (5). In Figure 17 we can see this slight advantageof 1SC over G1SC in the Blocksworld domain.The results in the Miconic domain are, on the other hand, not very intuitive. Wewould have expected the G1SC formulation to solve at least as many problems as the 1SCformulation, but this did not turn out to be the case. One thing we noticed is that in thisdomain the G1SC formulation required a lot more time to determine that there is no planfor a given number of plan periods. G1SC versus G2SC and PathSC.
Table 1 only shows the five largest problems ineach domain that were solved by the formulations, yet it is representative for the whole setof problems. The table indicates that when Graphplan-style parallelism is generalized, moreordering constraints are needed to ensure a feasible plan. On average, the G2SC formulationincludes more ordering constraints than the G1SC formulation, and the PathSC formulationin its turn includes more ordering constraints than the G2SC formulation. The performanceof these formulations as shown by Figure 17 varies per planning domain. The PathSCformulation does well in Miconic and Driverlog, the G2SC formulation does well in Freecell,and the G1SC does well in Zenotravel. Because of these performance differences, we believethat the ideal amount of flexibility in the generalization of Graphplan-style parallelism isdifferent for each planning domain. oosely Coupled Formulations for Automated Planning
Blocksworld S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCG2SCPathSC 0.010.1110100100010000 1 3 5 7 9 11 13 15 17 19 21 23 25 27
Logistics S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCG2SCPathSC
Freecell (IPC2) S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCG2SCPathSC
Miconic S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCG2SCPathSC
Depots S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCG2SCPathSC 0.010.1110100100010000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Driverlog S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCG2SCPathSC0.010.1110100100010000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Zenotravel S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCG2SCPathSC 0.010.1110100100010000 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Rovers S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCPathSC0.010.1110100100010000 1 2 3 4 5 6 7 8 9 10 11
Satellite S o l u t i on t i m e ( sec . ) Optiplan1SCG1SC 0.010.1110100100010000 1 2 3 4 5
Freecell (IPC3) S o l u t i on t i m e ( sec . ) Optiplan1SCG1SCG2SCPathSC
Figure 17: Solution times in the planning domains of the second and third internationalplanning competition. an den Briel, Vossen & Kambhampati
Blocksworld P l a n p e r i od s GPstepsSatplannerG1SCG2SC PathSC
Logistics P l a n p e r i od s GPstepsSatplannerG1SCG2SC PathSC
Freecell (IPC2) P l a n p e r i od s GPstepsSatplannerG1SCG2SC PathSC
Miconic P l a n p e r i od s GPstepsSatplannerG1SCPathSC051015202530 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Depots P l a n p e r i od s GPstepsSatplannerG1SCG2SC PathSC 024681012141618 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Driverlog P l a n p e r i od s GPstepsSatplannerG1SCG2SC PathSC0246810 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Zenotravel P l a n p e r i od s GPstepsSatplannerG1SCG2SC PathSC 0246810121416 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Rovers P l a n p e r i od s GPstepsSatplannerG1SCPathSC02468101214 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Satellite P l a n p e r i od s GPstepsSatplannerG1SC 024681012141618 1 2 3 4 5 6
Freecell (IPC3) P l a n p e r i od s GPstepsSatplannerG1SCG2SC PathSC
Figure 18: Number of plan periods required for each formulation to solve the planningproblems in the second and third international planning competition. oosely Coupled Formulations for Automated Planning
Problem G1SC G2SC PathSCBlocksworld5-1 0 0 165-2 0 0 626-0 0 0 56-1 0 0 58-2 0 0 62Logistics10-1 0 7 1111-0 0 0 1011-1 0 0 4912-0 0 16 914-0 0 7 110Freecell2-1 0 0 02-2 0 0 02-3 0 0 842-4 0 2 05-5 0 3 *Miconic6-4 0 - 07-0 0 - 07-2 0 - 27-3 0 - 49-4 0 - 5 Problem G1SC G2SC PathSCDepots1 0 0 72 0 0 210 0 2 *13 0 0 3017 0 0 *Driverlog7 1 16 28 32 62 1089 58 80 3210 5 5 6911 6 30 11Zenotravel5 0 4 4856 0 5 610 0 0 21411 0 0 58612 0 60 6259Rovers14 0 - 1315 12 - 1116 1 - 9217 1 - 418 1 - 192Satellite5 2 - -6 6 - -7 8 - -9 14 - -11 0 - -Freecell1 0 0 32 0 0 24803 0 1 19894 * 1 *5 * 0 *
Table 1: Number of ordering constraints, or cuts, that were added dynamically through thesolution process to problems in IPC2 (left) and IPC3 (right). A dash ‘-’ indicatesthat the IP formulation could not be tested on the domain and a star ‘*’ indicatesthat the formulation could not solve the problem instance within 30 minutes. an den Briel, Vossen & Kambhampati
In Figure 18, we see that in all domains the flexible network representation of the G1SCformulation is slightly more general than the 1-linearization semantics that is used by Sat-planner. That is, the number of plan periods required by the G1SC formulation is alwaysless than or equal to the number of plan periods used by Satplanner. Moreover, the flexiblenetwork representation of the G2SC and PathSC formulations are both more general thanthe one used by the G1SC formulation. One may think that the network representationof the PathSC formulation should provide the most general interpretation of action paral-lelism, but since the G2SC network representations allows some values to change back totheir original value in the same plan period this is not always the case.In the domains of Logistics, Freecell, Miconic, and Driverlog, the PathSC never requiredmore than two plan periods to solve the problem instances. For the Miconic domain thisis very easy to understand. In Miconic there is an elevator that needs to bring travelersfrom one floor to another. The state variables representation of this domain has one statevariable for the elevator and two for each traveler (one to represent whether the traveler hasboarded the elevator and one to represent whether the traveler has been serviced). Clearly,one can devise a plan such that each value of the state variable is visited at most twice. Theelevator simply could visit all floors and pickup all the travelers, and then visit all floorsagain to bring them to their destination floor.
An interesting question is to find out whether different state variable representations lead todifferent performance benefits. In our loosely coupled formulations we have components thatrepresent multi-valued state variables. However, the idea of modeling value transitions asflows in an appropriately defined network can be applied to any binary or multi-valued statevariable representation. In this section we concentrate on the efficiency tradeoffs betweenbinary and multi-valued state descriptions. As there are generally fewer multi-valued statevariables than binary atoms needed to describe a planning problem, we can expect ourformulations to be more compact when they use a multi-valued state description. For thiscomparison we only concentrate on the G1SC formulation as it showed the overall bestperformance among our formulations. In our recent work (van den Briel, Kambhampati, &Vossen, 2007) we analyze different state variable representations in more detail.Table 2 compares the encoding size for the G1SC formulation on a set of problemsusing either a binary or multi-valued state description. The table clearly shows that theencoding size becomes significantly smaller (both before and after CPLEX presolve) whena multi-valued state description is used. The encoding size before presolve gives an idea ofthe impact of using a more compact multi-valued state description, whereas the encodingsize after presolve shows how much preprocessing can be done by removing redundanciesand substituting out variables.Figure 19 shows the total solution time (y-axis) needed to solve the problem instances(x-axis). Since we did not make any changes to the G1SC formulation, the performancedifferences are the result of using different state descriptions. In several domains the multi-valued state description shows a clear advantage over the binary state description whenusing the G1SC formulation, but there are also domains in which the multi-valued state oosely Coupled Formulations for Automated Planning description does not provide too much of an advantage. In general, however, the G1SCformulation using a multi-valued state description leads to the same or better performancethan using a binary state description. In all our tests, we encountered only one probleminstance (Rovers pfile10) in which the binary state description notably outperformed themulti-valued state description.
6. Related Work
There are only few integer programming-based planning systems. Bylander (1997) considersan IP formulation based on converting the propositional representation given by Satplan(Kautz & Selman, 1992) to an IP formulation with variables that take the value 1 if acertain proposition is true, and 0 otherwise. The LP relaxation of this formulation isused as a heuristic for partial order planning, but tends to be rather time-consuming. Adifferent IP formulation is given by Vossen et al. (1999). They consider an IP formulationin which the original propositional variables are replaced by state change variables. Statechange variables take the value 1 if a certain proposition is added, deleted, or persisted,and 0 otherwise. Vossen et al. show that the formulation based on state change variablesoutperforms a formulation based on converting the propositional representation. Van denBriel and Kambhampati (2005) extend the work by Vossen et al. by incorporating some ofthe improvements described by Dimopoulos (2001). Other integer programming approachesfor planning rely on domain-specific knowledge (Bockmayr & Dimopoulos, 1998, 1999) orexplore non-classical planning problems (Dimopoulos & Gerevini, 2002; Kautz & Walser,1999).In our formulations we model the transitions of each state variable as a separate flowproblem, with the individual problems being connected through action constraints. TheGraphplan planner introduced the idea of viewing planning as a network flow problem, butit did not decompose the domain into several loosely coupled components. The encodingsthat we described are related to the loosely-coupled modular planning architecture by Sri-vastava, Kambhampati, and Do (2001), as well as factored planning approaches by Amirand Engelhardt (2003), and Brafman and Domshlak (2006). The work by Brafman andDomshlak, for example, proposes setting up a separate CSP problem for handling each fac-tor. These individual factor plans are then combined through a global CSP. In this way, ithas some similarities to our work (with our individual state variable flows corresponding toencodings for the factor plans). Although Brafman and Domshlak do not provide empiricalevaluation of their factored planning framework, they do provide some analysis on whenfactored planning is expected to do best. It would be interesting to adapt their minimaltree-width based analysis to our scenario.The branch-and-cut concept was introduced by Gr¨otschel, R¨unger, and Reinelt (1984)and Padberg and Rinaldi (1991), and has been successfully applied in the solution ofmany hard large-scale optimization problems (Caprara & Fischetti, 1997). In planning,approaches that use dynamic constraint generation during search include RealPlan (Srivas-tava et al., 2001) and LPSAT (Wolfman & Weld, 1999).Relaxed definitions for Graphplan-style parallelism have been investigated by severalother researchers. Dimopoulos et al. (1997) were the first to point out that it is not necessaryto require two actions to be independent in order to execute them in the same plan period. an den Briel, Vossen & Kambhampati
Binary MultiBefore presolve After presolve Before presolve After presolveProblem
Table 2: Formulation size for binary and multi-valued state description of problem instancesfrom the IPC2 and IPC3 in number of variables ( oosely Coupled Formulations for Automated Planning
Blocksworld S o l u t i on t i m e ( sec . ) binarymulti Logistics S o l u t i on t i m e ( sec . ) binarymulti Freecell (IPC2) S o l u t i on t i m e ( sec . ) binarymulti Miconic S o l u t i on t i m e ( sec . ) binarymulti Depots S o l u t i on t i m e ( sec . ) binarymulti 0.010.1110100100010000 1 2 3 4 5 6 7 8 9 10 11 Driverlog S o l u t i on t i m e ( sec . ) binarymulti Zenotravel S o l u t i on t i m e ( sec . ) binarymulti Rovers S o l u t i on t i m e ( sec . ) binarymulti Satellite S o l u t i on t i m e ( sec . ) binarymulti Freecell (IPC3) S o l u t i on t i m e ( sec . ) binarymulti Figure 19: Comparing binary state descriptions with multi-valued state descriptions usingthe G1SC formulation. an den Briel, Vossen & Kambhampati
In their work they introduce the property of post-serializability of a set of actions. A setof actions A ′ ⊆ A is said to be post-serializable if (1) the union of their preconditionsis consistent, (2) the union of their effects is consistent, and (3) the preconditions-effectsgraph is acyclic. Where the preconditions-effects graph is a directed graph that containsa node for each action in A ′ , and an arc ( a, b ) for a, b ∈ A ′ if the preconditions of a areinconsistent with the effects of a . For certain planning problems Dimopoulos et al. (1997)show that their modifications reduce the number of plan periods and improve performance.Rintanen (1998) provides a constraint-based implementation of their idea and shows thatthe improvements hold over a broad range of planning domains.Cayrol et al. (2001) introduce the notion of authorized linearizations, which implies anorder for the execution of two actions. In particular, an action a ∈ A authorizes an action b ∈ A implies that if a is executed before b , then the preconditions of b will not be deletedby a and the effects of a will not be deleted by b . The notion of authorized linearizationsis very similar to the property of post-serializability. If we were to adopt these ideas inour network-based representations it would compare to the G1SC network in which thegeneralized state change arcs (see Figure 7) only allows values to prevail after, but notbefore, each of the transition arcs.A more recent discussion on the definitions of parallel plans is given by Rintanen, Hel-janko and Niemel¨a (2006). Their work introduces the idea of ∃ -step semantics, which saysthat it is not necessary that all parallel actions are non-interfering as long as they can beexecuted in at least one order. ∃ -step semantics provide a more general interpretation ofparallel plans than the notion of authorized linearizations used by LCGP (Cayrol et al.,2001). The current implementation of ∃ -step semantics in Satplanner is, however, some-what restricted. While the semantics allow actions to have conflicting effects, the currentimplementation of Satplanner does not.
7. Conclusions
This work makes two contributions: (1) it improves state of the art in modeling planning asinteger programming, and (2) it develops novel decomposition methods for solving boundedlength (in terms of number of plan periods) planning problems.We presented a series of IP formulations that represent the planning problem as a setof loosely coupled network flow problems, where each network flow problem correspondsto one of the state variables in the planning domain. We incorporated different notionsof action parallelism in order to reduce the number of plan periods needed to find a planand to improve planning efficiency. The IP formulations described in this paper have ledto their successful use in solving partial satisfaction planning problems (Do et al., 2007).Moreover, they have initiated a new line of work in which integer and linear programmingare used in heuristic state-space search for automated planning (Benton et al., 2007; van denBriel et al., 2007). It would be interesting to see how our approach in the context of IPformulations applies to formulations based on satisfiability and constraint satisfaction. oosely Coupled Formulations for Automated Planning
Acknowledgments
This research is supported in part by the NSF grant IIS308139, the ONR grant N000140610058,and by the Lockheed Martin subcontract TT0687680 to Arizona State University as partof the DARPA integrated learning program. We thank Derek Long for his valuable input,and we especially thank the anonymous reviewers whose attentive comments and helpfulsuggestions have greatly improved this paper.
References
Amir, E., & Engelhardt, B. (2003). Factored planning. In
Proceedings of the 18th Interna-tional Joint Conference on Artificial Intelligence (IJCAI-2003) , pp. 929–935.B¨ackstr¨om, C., & Nebel, B. (1995). Complexity results for SAS + planning. ComputationalIntelligence , (4), 625–655.Benton, J., van den Briel, M., & Kambhampati, S. (2007). A hybrid linear programmingand relaxed plan heuristic for partial satisfaction planning problems. In Proceedings ofthe International Conference on Automated Planning and Scheduling (ICAPS-2007) ,pp. 34–41.Bixby, R. E. (2002). Solving real-world linear programs: a decade and more of progress.
Operations Research , (1), 3–15.Blum, A., & Furst, M. (1995). Fast planning through planning graph analysis. In Proceedingsof the 14th International Joint Conference on Artificial Intelligence (IJCAI-1995) , pp.1636–1642.Bockmayr, A., & Dimopoulos, Y. (1998). Mixed integer programming models for plan-ning problems. In
Working notes of the CP-98 Constraint Problem ReformulationWorkshop .Bockmayr, A., & Dimopoulos, Y. (1999). Integer programs and valid inequalities for plan-ning problems. In
Proceedings of the European Conference on Planning (ECP-1999) ,pp. 239–251.Brafman, R., & Domshlak, C. (2006). Factored planning: How, when, and when not. In
Proceedings of the 21st National Conference on Artificial Intelligence (AAAI-2006) ,pp. 809–814.Bylander, T. (1997). A linear programming heuristic for optimal planning. In
Proceedingsof the 14th National Conference on Artificial Intelligence (AAAI-1997) , pp. 694–699.Caprara, A., & Fischetti, M. (1997).
Annotated Bibliographies in Combinatorial Optimiza-tion , chap. Branch and Cut Algorithms, pp. 45–63. John Wiley and Sons.Cayrol, M., R´egnier, P., & Vidal, V. (2001). Least commitment in graphplan.
ArtificialIntelligence , (1), 85–118.Desaulniers, G., Desrosiers, J., & Solomon, M. (Eds.). (2005). Column Generation . Springer.Dimopoulos, Y. (2001). Improved integer programming models and heuristic search for AIplanning. In
Proceedings of the European Conference on Planning (ECP-2001) , pp.301–313. an den Briel, Vossen & Kambhampati
Dimopoulos, Y., & Gerevini, A. (2002). Temporal planning through mixed integer program-ming. In
Proceedings of the Workshop on Planning for Temporal Domains (AIPS-2002) , pp. 2–8.Dimopoulos, Y., Nebel, B., & Koehler, J. (1997). Encoding planning problems in non-monotic logic programs. In
Proceedings of the 4th European Conference on Planning(ECP-1997) , pp. 167–181.Do, M., Benton, J., van den Briel, M., & Kambhampati, S. (2007). Planning with goalutility dependencies. In
Proceedings of the 20th International Joint Conference onArtificial Intelligence (IJCAI-2007) , pp. 1872–1878.Edelkamp, S., & Helmert, M. (1999). Exhibiting knowledge in planning problems to mini-mize state encoding length. In
Proceedings of the European Conference on Planning(ECP-1999) , pp. 135–147.Garey, M., & Johnson, D. (1979).
Computers and Intractability: a Guide to the Theory ofNP-Completeness . Freeman and Company, N.Y.Gr¨otschel, M., J¨unger, M., & Reinelt, G. (1984). A cutting plane algorithm for the linearordering problem.
Operations Research , , 1195–1220.Helmert, M. (2006). The Fast Downward planning system. Journal of Artifical IntelligenceResearch , , 191–246.Hooker, J. (1988). A quantitative approach to logical inference. Decision Support Systems , , 45–69.ILOG Inc., Mountain View, CA (2002). ILOG CPLEX 8.0 user’s manual .Johnson, E., Nemhauser, G., & Savelsbergh, M. (2000). Progress in linear programmingbased branch-and-bound algorithms: An exposition.
INFORMS Journal on Comput-ing , , 2–23.Kambhampati, S., & commentary from other planning researchers (2006). Onthe suboptimality of optimal planning track at ipc 2006. http://raos-ruminations.blogspot.com/2006/07/on-suboptimality-of-optimal-planning.html.Karmarkar, N. (1984). A new polynomial time algorithm for linear programming. Combi-natorica , (4), 373–395.Kautz, H. (2004). SATPLAN04: Planning as satisfiability. In Working Notes on the FourthInternational Planning Competition (IPC-2004) , pp. 44–45.Kautz, H., & Selman, B. (1992). Planning as satisfiability. In
Proceedings of the EuropeanConference on Artificial Intelligence (ECAI-1992) .Kautz, H., & Selman, B. (2006). SATPLAN04: Planning as satisfiability. In
Working Noteson the Fifth International Planning Competition (IPC-2006) , pp. 45–46.Kautz, H., & Walser, J. (1999). State-space planning by integer optimization. In
Proceedingsof the 16th National Conference on Artificial Intelligence (AAAI-1999) , pp. 526–533.Martin, K. (1991). Using separation algorithms to generate mixed integer model reformu-lations.
Operations Research Letters , , 119–128. oosely Coupled Formulations for Automated Planning Padberg, M., & Rinaldi, G. (1991). A branch-and-cut algorithm for the resolution of large-scale symmetric traveling salesman problems.
SIAM Review , , 60–100.Rintanen, J. (1998). A planning algorithm not based on directional search. In Proceedingsof the Sixth International Conference on Principles of Knowledge Representation andReasoning (KR-1998) , pp. 617–624.Rintanen, J., Heljanko, K., & Niemel¨a, I. (2006). Planning as satisfiability: parallel plansand algorithms for plan search.
Artificial Intelligence , (12), 1031–1080.Srivastava, B., Kambhampati, S., & Do, M. (2001). Planning the project management way:Efficient planning by effective integration of causal and resource reasoning in realplan. Artificial Intelligence , (1-2), 73–134.van den Briel, M., Benton, J., Kambhampati, S., & Vossen, T. (2007). An LP-based heuristicfor optimal planning. In Proceedings of the International Conference of Principles andPractice of Constraint Programming (CP-2007) , pp. 651–665.van den Briel, M., & Kambhampati, S. (2005). Optiplan: Unifying IP-based and graph-basedplanning.
Journal of Artificial Intelligence Research , , 623–635.van den Briel, M., Kambhampati, S., & Vossen, T. (2007). Fluent merging: A general tech-nique to improve reachability heuristics and factored planning. In Proceedings of theWorkshop Heuristics for Domain-Independent Planning: Progress, Ideas, Limitations,Challenges (ICAPS-2007) .Vossen, T., Ball, M., Lotem, A., & Nau, D. (1999). On the use of integer programmingmodels in AI planning. In
Proceedings of the 18th International Joint Conference onArtificial Intelligence (IJCAI-1999) , pp. 304–309.Wolfman, S., & Weld, D. (1999). The LPSAT engine and its application to resource planning.In
Proceedings of the 18th International Joint Conference on Artificial Intelligence(IJCAI-1999) , pp. 310–317.Wolsey, L. (1998).
Integer Programming . John Wiley and Sons.. John Wiley and Sons.