Exploring Instance Generation for Automated Planning
Özgür Akgün, Nguyen Dang, Joan Espasa, Ian Miguel, András Z. Salamon, Christopher Stone
EExploring Instance Generationfor Automated Planning ¨Ozg¨ur Akg¨un, Nguyen Dang, Joan Espasa, Ian Miguel, Andr´as Z. Salamon,and Christopher Stone
School of Computer Science, University of St Andrews, UK { ozgur.akgun,nttd,jea20,ijm,Andras.Salamon,cls29 } @st-andrews.ac.uk Abstract.
Many of the core disciplines of artificial intelligence have setsof standard benchmark problems well known and widely used by the com-munity when developing new algorithms. Constraint programming andautomated planning are examples of these areas, where the behaviourof a new algorithm is measured by how it performs on these instances.Typically the efficiency of each solving method varies not only betweenproblems, but also between instances of the same problem. Therefore,having a diverse set of instances is crucial to be able to effectively eval-uate a new solving method. Current methods for automatic generationof instances for Constraint Programming problems start with a declar-ative model and search for instances with some desired attributes, suchas hardness or size. We first explore the difficulties of adapting this ap-proach to generate instances starting from problem specifications writ-ten in PDDL, the de-facto standard language of the automated planningcommunity. We then propose a new approach where the whole planningproblem description is modelled using essence , an abstract modellinglanguage that allows expressing high-level structures without committingto a particular low level representation in PDDL.
The planning task consists of selecting a sequence of actions in order to achievea specified goal from specified starting conditions. This type of problem arises inmany contexts. Consider, for example, the delivery of a set of packages by vehiclefrom a depot to a set of destinations. The allocation of packages and drivers tovehicles must be planned, as well as the route for each vehicle, while respectingpackage delivery deadlines, vehicle capacities and driver shift restrictions.Given their importance, the automated solution of planning problems is acentral discipline of Artificial Intelligence. The difficulty of solving planning prob-lems grows rapidly with their size in terms of the number of objects and possibleactions under consideration. Over many years, a great deal of effort by differentresearch groups has resulted in the development of highly efficient AI planningsystems [27]. Testing algorithms across a wide range of problem instances iscrucial to ensure the validity of any claim about one algorithm being betterthan another. However, when it comes to evaluations, typically limited sets of a r X i v : . [ c s . A I] S e p Akg¨un et al. problems are used and thus the full picture is rarely seen. Finding and encodinginteresting instances is a time-consuming task, and due to the nature of someproblems, is sometimes out of reach from the researcher perspective.In the International Planning Competitions (IPC) state of the art planningsystems are empirically evaluated on a set of benchmark problems. The competi-tions have, amongst others, a track that focuses on planners that can learn fromprevious runs. This track uses manually coded problem generators to providethe planners with a varied set of problems. This variety of problems is crucial toensure that planners can learn and generalise well on new and unseen situations.The problem of automated instance generation is to make the process ofcreating benchmark problems more accessible and efficient. Instance generationhas been a focus of the SAT community for several decades [22]. Generatingrandom SAT instances while considering different parameters such as the lengthof clauses, the number of variables, or their connectivity, has helped to illuminatehow algorithms behave and how they perform in different circumstances. Havingautomated tools to generate interesting instances allows algorithm developers toevaluate and compare the algorithms across a wide range of instances, providinga detailed picture of their comparative strengths and weaknesses.In this work we explore the adaptation of a successful tool [1] that auto-matically generates instances with desirable properties from a single problemspecification in essence , a high-level modelling language for Constraint Pro-gramming (CP) [4]. We discuss two approaches (Section 5 and 6) to make theinstance generation process in [1] work with PDDL [18], the de-facto standardlanguage in the automated planning community. Both approaches have theirown limitations regarding flexibility , efficiency and automation . We then makea proposal for a new planning modelling language using essence (Section 7).Thanks to the rich expressiveness of the language, all discussed limitations ofthe PDDL-based instance generation approaches are overcome. Fuentetaja et al. [7] approach the generation of satisfiable planning instancesas a planning problem, where users manually write some declarative semantics-related information to describe the generation of different instances. They havepointed out the limitation of PDDL as a representation for describing the in-stance generation problem, as the task of generating valid initial states is complexand requires information about the meaning of predicates not expressed in thedomain definition. PDDL representation is only effective for representing validstates and the transition functions between them. Augmenting PDDL to improveits expressiveness has been proposed several times [14,5,9]. This allows addingextra information into the domain definition, and could potentially lead to moregeneral applications of planning [23].The usefulness of automated generation of benchmark instances has beendemonstrated in various fields, such as combinatorial optimisation [25], SAT [12]and model fitting [20] especially in the context of instance space analysis [24]. In xploring Instance Generation for Automated Planning 3
CP, benchmark instances created using problem-specific generators have beenproposed for various problem classes, such as binary Constraint SatisfactionProblem [11,19] and Random Constraint Networks [29]. In Operations Research,several instance generators and benchmarks for well-known combinatorial opti-misation problems, such as the Knapsack problem [13] and the Nurse Rosteringproblem [28], have been provided and widely used. All of those approaches aresemi-automated, as the instance generators were manually created.In discrete optimisation Ullrich [26] developed a tool for the generation ofinstances in the TSP, Max-SAT and Load Allocation problems. Even in this casegenerating instances for new problem classes requires some user input, howeverthe formal language they developed simplifies the process of producing instancesfor new domains.Recently, a new approach for fully automated instance generation has beenproposed for CP [1,2]. The approach allows users to declaratively describe aCP problem and properties of valid instances in a single CP model using thehigh-level constraint modelling language essence [4]. An instance generator isthen created by the automated modelling tool
Conjure [3]. Finally, instanceswith desirable properties are generated through a combination of the automatedalgorithm configuration tool irace [17] and the essence constraint modellingtoolchain developed by the CP group at St Andrews , which includes Conjure ,Savile Row [21] (a constraint modelling assistant) and minion [8] (a CP solver).All steps are done in a completely automated fashion. The whole process of thesystem is shown in Figure 1. (1) CP problem specification in Essence● Problem parameters● Validity constraints● Decision variables and problem constraints conjure (2) Instance generation problem in Essence (3) Valid Instances with desirable properties irace andEssence CP-toolchain
Fig. 1: The automated instance generation approach for CP problems proposedin [1,2]
A classical planning problem is defined as a tuple Π = (cid:104) V, A, I, G (cid:105) where V isa set of propositions (or Boolean variables), A is a set of actions, I is the initialstate and G is a formula over V that any goal state must satisfy.A state is a total assignment to the variables. Actions are formalized as pairs (cid:104) p, e (cid:105) , where p is a set of preconditions and e a set of effects. More formally, p isa set of Boolean expressions over V , while e is a set of assignments. An action a = (cid:104) p, e (cid:105) is executable in a given state s if s | = p .The state resulting from https://constraintmodelling.org/ Akg¨un et al. executing action a on state s is denoted by apply ( a, s ) = s (cid:48) . The new state s (cid:48) isdefined by assigning new values to the variables according to the active effects,and retaining the values of the variables that are not assigned values by any ofthe active effects. A plan of length n for a planning problem Π is a sequence ofactions a ; a ; . . . ; a n such that apply ( a n , . . . , apply ( a , apply ( a , I )) . . . ) | = G .An example planning problem is the floor-tile problem used in the In-ternational Planning Competition 2014 (IPC-14) [27]. The problem includes aset of robots sharing the task of painting a pattern on floor tiles. The robotsmove around in four directions (up, down, left and right) and can paint with onecolour at a time. They are also able to change the current colour to any otherone. However, due to their design robots can only paint the tiles that are in frontor behind them. Finally, once a tile has been painted, no robot can stand on it.Automated planning models are typically expressed in the Planning DomainDefinition Language (PDDL) [18]. The user describes the problem in a domain file, in terms of predicates and actions with parameters. In turn, these parameters(or free variables) can be instantiated with a set of defined objects. A concreteinstance is expressed in a problem file, where the initial state, problem objectsand goal are defined. Figure 2 shows how predicates and actions are declared ina domain file for the floor-tile problem (a problem description model), andFigure 3 depicts an example problem file (an instance). ( : predicates ; s t a t e v a r i a b l e s are defined in the p r e d i c a t e s s e c t i o n ( robot − at ? r − robot ?x − t i l e ) ; at what t i l e a robot i s ( up ?x − t i l e ?y − t i l e )(down ?x − t i l e ?y − t i l e )( r i g h t ?x − t i l e ?y − t i l e )( l e f t ?x − t i l e ?y − t i l e )( c l e a r ?x − t i l e )( painted ?x − t i l e ? c − c o l o r )( robot − has ? r − robot ? c − c o l o r )( a v a i l a b l e − c o l o r ? c − c o l o r ) )( : action move up: parameters (? r − robot ? from − t i l e ? to − t i l e ): precondition ( and ( robot − at ? r ? from ) ( up ? to ? from ) ( c l e a r ? to ) ): e f f e c t ( and ( robot − at ? r ? to ) ( not ( robot − at ? r ? from ) )( c l e a r ? from ) ( not ( c l e a r ? to ) ) ) ). . . Fig. 2: Snippets of the floor-tile
PDDL problem description.
A planning model is typically an abstraction of a real-world problem, where themodeller has some predefined assumptions on how the real-world works. Thecorrectness of this abstraction derives from a valid initial state, and that the xploring Instance Generation for Automated Planning 5 ( define ( problem toy )( : domain f l o o r − t i l e )( : objects t i l e 0 − −
1t i l e 1 − − − t i l erobot1 robot2 − robotwhite black − c o l o r )( : i n i t ( robot − at robot1 t i l e 0 −
1) ( robot − has robot1 white )( robot − at robot2 t i l e 1 −
1) ( robot − has robot2 black )( a v a i l a b l e − c o l o r white ) ( a v a i l a b l e − c o l o r black )( c l e a r t i l e 0 −
0) ( c l e a r t i l e 1 − − −
1) ( up t i l e 0 − − − −
1) (down t i l e 1 − − − −
0) ( r i g h t t i l e 1 − − − −
1) ( l e f t t i l e 1 − − goal ( and ( painted t i l e 0 − − Fig. 3: An example floor-tile instance.transitions of the state variables between steps respect the implicit constraints.When a model and an instance respect all the implicit assumptions by the mod-eller we say that it is a valid problem. As an example, some implicit assumptionsby the modeller in the floor-tile domain (Figure 2) are that in the initial stateeach robot must be at exactly one tile, or that any given cell can only have oneunique cell on top of it. Moreover, in the IPC-14 published instances, the tiles’structure (represented by up , down , right and left ) always forms a square grid.In the automated planning community, sometimes when a problem is re-leased, a Python or Java program is included to generate instances automati-cally. The validity properties of the problem are normally hard-coded in thoseprograms, and therefore appear implicitly in the generated problem instances.An alternative approach is to allow modellers to express the validity prop-erties of a planning problem declaratively. An instance generator with thosevalidity constraints integrated is then automatically created. Specifying thoseproperties using a declarative modelling language provides flexibility, as userscan easily add or update the validity specification without having to modify thegenerator software’s source code directly. This is the approach of the essence -based automated instance generation system that we propose to build upon.There is a large variety of validity properties arising in classical planningdomains. In this section, we discuss six arbitrarily selected IPC-14 problems (inthe Sequential track) and the validity properties of their published benchmarkinstances. A brief description of these problems follows. We refer to the com-petition website and the accompanying paper [27] for further details of thesebenchmarks. – city-car : This problem simulates the impact of road building and demolitionon traffic flows. A city is represented as a grid, in which each node is a https://helios.hud.ac.uk/scommv/IPC-14/ Akg¨un et al. junction and edges are potential roads. Cars start from different positionsand have to reach their final destinations. A finite number of roads can bebuilt to connect two junctions and allowing a car to move between them. – floor-tile : A set of robots use colours to paint patterns in floor tiles. Therobots can move around and paint with one colour at a time and can alsochange their colours. Once a tile has been painted, no robot can stand on it. – hiking : This problem simulates a walking trip that lasts several days, whereeach day one walk is done with a partner. The walks are over a long circularroute, without ever walking backwards. Tents and items of luggage can becarried in a car between the start and end of the walking routes if necessary. – cave-diving : There are divers with different skills and confidence, and eachcan carry tanks of air. These divers must be hired to go into an underwatercave system and either take photos or prepare the way for other divers bydropping full tanks of air. The cave is too narrow for more than one diver toenter at a time. Divers must exit the cave and decompress at the end. Theycan therefore only make a single trip into the cave. Divers have hiring costsinversely proportional to how hard they are to work with. – child-snack : This involves making and serving sandwiches with various in-gredients for a group of children in which some are allergic to gluten. – barman : A robot barman manipulates drink dispensers, glasses, and a shaker.The goal is to find a plan of the robot’s actions that serves a desired set ofdrinks. Robot hands can only grasp one object at a time and glasses needto be empty and clean to be filled.The properties in these benchmark instances can be roughly divided into twogroups. The first group involves constraints between the variables emerging froma single predicate. They typically include three types of constraints: exactly-k , at-most-k and at-least-k , where k is a parameter of the constraints. For example,from the floor-tile model an instances illustrated in Figure 2 and 3, we caninfer that each robot begins at exactly one tile. This could be represented by, fora given robot, an exactly-1 constraint on the variables that result from groundingthe robot-at predicate.The second group of validity constraints involves structural constraints be-tween predicates in a planning problem. The published instances of both city-car and floor-tile have square-grid underlying maps . The maps of floor-tile re-quire rectangular grids with up, down, left and right connections, while themaps in city-car instances allow cells to be connected horizontally as well asdiagonally. In cave-diving a cave forms a tree-shaped structure. Sequences arerepresented in the hiking and barman domains, while other problems from thecompetition use stacks or weighted undirected graphs.If we want to generate valid instances randomly, it is necessary to take thesevalidity constraints into account during the modelling stage. This is becauseit would be vanishingly unlikely that an instance randomly sampled from theunconstrained instance space would be valid. For example, consider the floor-tile domain from Figure 2 and focus on the predicates up and down , which state ifa tile is on top (or bottom) of another. If the up predicate states that a tile xploring Instance Generation for Automated Planning 7 is on top of another, the down predicate should be coherent and correspondwith the inverse assignment. If we generate instances randomly, only a smallfraction would satisfy this property for any given pair of tiles, and this fractionshrinks exponentially as the instance size grows. Another example involves the robot-at predicate, which specifies the tile where a robot is located. It would beunlikely for randomly generated instances to respect the fact that a robot canonly be in one place at one time. Almost surely we would end up with instanceswhere a robot is located at various places simultaneously. Combining all theimplied constraints used by a modeller, it is clear that the chance of randomlygenerating valid instances is negligibly small. It is possible to make this claimmathematically precise but we leave this for future work. (1.1) Planning problem specification in PDDL● Objects and predicates● Validity constraints via special keywords ● Operators and conditions conjure (2) Instance generation problem in Essence (3) Valid Instances with desirable properties (in Essence) (1.2) (Partial) problem specification in Essence● Objects and predicates● Validity constraints rantanplan (4) PDDL instances irace andEssence CP-toolchain rantanplan Fig. 4: Automated instance generation for planning using Augmented PDDLThe essence language allows expressing validity constraints for a ConstraintProgramming (CP) problem as where statements. As shown in Figure 1, the sys-tem we build upon starts by receiving the problem description in essence withvalidity constraints as input (step 1), and
Conjure will automatically createan instance generator in essence (step 2), which will then be tuned by irace in combination with the CP solver minion to generate instances with desirableproperties (step 3). In order to employ the same automated methodology forplanning problems, we need to be able to create an instance generator automat-ically from a planning problem description. Ideally we would want to use theplanning modelling language PDDL to express those validity constraints insidethe problem description in step 1 of Figure 1.Unfortunately, due to the low-level nature of standard PDDL, the task be-comes extremely tedious and error-prone for a human. Firstly, as classical plan-ning only deals with Boolean variables, many-valued variables cannot be directlyexpressed. This type of variables is essential for modelling common validity con-straints such as that a robot can only start at exactly one place. Secondly, many
Akg¨un et al. structural constraints (such as a graph being connected) cannot be expressedin a purely first-order language like PDDL [16, Corollary 3.19]. Verifying that agrid specified by adjacency predicates is well-formed, as in our example model,appears also to be impossible to express in pure PDDL, with even restrictedversions of this problem still under active investigation [15].In this section, we will therefore discuss a solution approach (Figure 4) wherewe augment PDDL with a new declarative section and extra keywords to allowthe modeller to express validity constraints in a PDDL problem description (step1.1 in Figure 4). The real modelling of those constraints is then generated au-tomatically by rantanplan [6], a parser specially developed to translate an aug-mented PDDL model to the high-level language essence (step 1.2 in Figure 4). essence is more expressive than PDDL and allows modelling of structural va-lidity constraints directly. ( : instance − c o n s t r a i n t s( i n i t ( f o r a l l (? r − robot )( and ( exactly − k ( robot − at ? r ) 1 True ) ; a robot s t a r t s in a t i l e ( exactly − k ( robot − has ? r ) 1 True ) ) ) ) ; and has one colour; nothing s t a r t s painted ( i n i t ( f o r a l l (? t − t i l e ) ( exactly − k ( painted ? t ) 0 True ) ) ) ; we are not i n t e r e s t e d in the c l e a r p r e d i c a t e s in the goal s t a t e ( goal ( f o r a l l (? t − t i l e ) ( not ( appear ( c l e a r ? t ) ) ) ) ) Fig. 5: Snippets of the instance-constraints section from the floor-tile domain.Validity constraints on the initial and goal states are defined in a new sec-tion starting with the new keyword instance-constraints . The following newoperands are added for single-predicate validity constraints: init , goal , xor , min , max , exactly-k , atleast-k , atmost-k and appear . By default, when considering nu-meric functions, the range of values generated goes from 0 to INT MAX (a defaultupper bound for integer variables, which can be specified by users). The modaloperators min and max accept the name of a function and an integer. This furtherrestricts the range of possible values generated. Figure 5 shows an example ofthe instance-constraints section for floor-tile using those new keywords. init and goal are modal operators that accept a constraint. This constraintwill be then applied to the initial or goal state, respectively. xor implements thexor logical operation, and has been added for convenience. exactly-k , atleast-k and atmost-k are a family of terms that accept a schematic fluent, a numberand a value. As their name imply, they restrict the values taken by the subset ofgrounded variables generated by that schematic fluent. When specifying theseterms, typically we will be interested in one or two parameters of the constrained xploring Instance Generation for Automated Planning 9 schematic fluent. For conciseness, the underscore character ( ) can be used toindentify parameters that we do not care, acting as a placeholder in a patternmatching style. Finally, the appear predicate is used to describe the goal state.The initial state is always a total assignment, as per the closed world assumption,but the goal can be a partial assignment. appear can be combined with the not operator to force a state variable to not appear in the goal state. It is useful toavoid considering non-interesting goals to the modeller.Not all the new operands are translated directly to essence . min and max determine the size of the integer domains of the related PDDL fluents, while init and goal control what is constrained. The cardinality constraints follow a pattern k { <,>,= } sum([toInt(x = value) | fluent ]) , where value is what we aresearching for, and the fluent placeholder iterates over the tuples belonging tothe fluent, which is represented as a function. Following the floor-tile domain,Figure 6 shows how a constraint expressing that a given robot can only be atone place in the initial state is translated to essence . ; a robot s t a r t s in one t i l e ( : instance − c o n s t r a i n t s( i n i t ( f o r a l l (? r − robot )( exactly − k( robot − at ? r ) 1 True ) ) ) ) (a) Constraint expressed in PDDL ; a robot starts in one tile forAll var_r : robot .1 = sum ([ toInt (value = true)| ((p0 ,_),value) <-init[robot_at],var_r = p0 ] (b) Constraint translated to essence Fig. 6: The translation of exactly-k constraint from PDDL to essence
As described in Section 4, another group of validity constraints involves implicitrequirements on structures of the underlying map in a planning problem, such asthe connections between tiles in the floor-tile problem. It is possible to expressgrids quite simply if the relations used to express the grid structure make useof the geometry of the plane. However, automated instance generation shouldnot restrict the choices made in modelling problems. In the IPC-14 benchmarkdataset, all instances of floor-tile have the tiles forming a square-grid structureand tiles’ connections are represented using adjacency relations left, right, up and down . Validity constraints to ensure that these adjacency relations express asquare-grid map cannot be efficiently modelled using PDDL due to the limitedexpressiveness of first-order logic. Checking that the adjacency relations form avalid grid requires reconstructing geometric information about placement of tileson the plane. Although it is possible to express the property that the adjacencyrelations form a grid for special cases, even this requires solving a challengingtiling problem, and the general case is currently open [15]. ( : instance − c o n s t r a i n t s i n i t ( isLRUDSquareGrid ( t i l e , up , down , l e f t , r i g h t ) ). . . ) ) $ ----- Objects and Domains -------- given n_tile: int (1..) letting tile be domain int (1 .. n_tile)$ ---- Auxiliaries ------ given tile_size: int (1..) where n_tile = tile_size * tile_size$ ----- Initial State -------- given init: record {up : function ( total ) (tile ,tile) --> bool ,down : function ( total ) (tile ,tile) --> bool ,right : function ( total ) (tile ,tile) --> bool ,left : function ( total ) (tile ,tile) --> bool } whereforAll u,v : tile .init[up]((u,v)) <-> u = v + tile_size /\init[down ]((u,v)) <-> u = v - tile_size /\init[left ]((u,v)) <-> (u = v + 1) /\((u %init[right ]((u,v)) <-> (u = v - 1) /\((u % Fig. 7: Expression for the isLRUDSquareGrid keyword in essence
We introduce new PDDL keywords to express those structural constraints,and provide automated translation of those keywords to a CP model in essence through rantanplan. An example on expressing a square grid using { left, right,up, down } predicates is shown in Figure 7. The newly introduced PDDL key-word isLRUDquareGrid is used and an auxiliary variable indicating the size of thesquare grid (variable tile size in the essence specification) is generated andwill be tuned by irace during the instance generation process. However, there aretwo limitations of this automated approach, as we will explain below.The first limitation is on the flexibility to express structural validity con-straints. Consider the square-grid structure as an example. There are variousways to define the local connections of cells in a square grid. The choice is nor-mally made based on the specification of planning actions for a specific planningproblem. For floor-tile , the locality relations { up, down, left, right } areused as the moving actions of a robot at each tile can only follow those directions.However, for the city-car problem, a car can either move horizontally or diag-onally. The underlying square-grid map in city-car is then represented usingtwo predicates: same line and diagonal for every ordered pair of horizontallyand diagonally adjacent cells, respectively.The second limitation is about scalability . As PDDL predicates are basicallyboolean functions, the size of validity constraints expressed directly using thosepredicates can grow very quickly. For example, the square-grid structure ex-pressed in Figure 7 needs to use a nested for loop for u,v: tile . For a grid sizeof 20 ×
20, the total number of constraints is 4 × . On a computer with Intel xploring Instance Generation for Automated Planning 11 Xeon E5-2640 2.4Ghz CPUs, generating a single instance with such grid sizeusing the CP solver minion takes about 20 minutes. The time increase also growsfast, as it takes more than 30 minutes to generate a 22 ×
22 grid.
In this section, we discuss an alternative approach to the augmented-PDDLinstance generation proposed in the previous section. This is a hybrid approachwhere users model validity constraints directly in essence . A summary of thisapproach is shown in Figure 8. Compared to the previous approach in Figure 4,step 1.2 is now done manually by users. It offers solutions to the two limitationsof the augmented-PDDL approach. (1.1) Planning problem specification in PDDL● Objects and predicates● Operators and conditions(1.2) Validity constraints in Essence (2) Instance generation problem in Essence (3) Valid Instances with desirable properties irace andEssence CP-toolchainconjure (4) PDDL instances
An instance converter
Fig. 8: Automated instance generation for planning problems using a hybrid ap-proach, with problem specification in PDDL and validity constraints in essence .Firstly, the new approach allows users to make full use of the high-levelmodelling language essence to freely express any validity constraints using con-straint modelling, instead of relying on a predefined set of keywords supportedby rantanplan. This overcomes the first limitation on flexibility.Secondly, by modelling validity constraints directly in essence , users areno longer tied to the relatively low-level boolean representations of PDDL. Itis well-known that recovering structure from low-level representation is a dif-ficult and expensive process, as illustrated in the work of Helmert [10] wheremany-valued variables were detected from PDDL representations with Booleanvariables. By expressing the constraints directly in essence , knowledge aboutimplicit structures present in the problem can be easily expressed, which resultsin more efficient representations. For example, in the square-grid structure of floor-tile , each of the up, down, left, right relations can be expressed asa function mapping from each tile in the grid to at most one other tile. Com-pared to the low-level representations using boolean functions as in Figure 4,this new representation significantly reduce the number of validity constraints(from 4 × n to 4 × n for any n × n square grid). Figure 9 shows a compar-ison of the time required by our system to generate square-grid structures forthe floor-tile problem using the augmented-PDDL approach and the new one described in this section. The results clearly indicate that the high-level repre-sentation (the orange line) significantly improve the efficiency of the instancegeneration process.
20 40 60 80 100 grid size t i m e ( s ) pddlessence Fig. 9: Time to generate floor-tile square-grid instances using our automatedinstance generation system. The blue line is the approach described in Section 5,and the orange line is the one described in Section 6.Another advantage of using high-level representations is that some validityconstraints are automatically encoded inside the abstract essence types them-selves without the need of any explicit constraints to express them. An exampleof the full description of all validity constraints for the floor-tile problem isillustrated in Figure 10. As we can see, the first two validity constraints, whichrequire that each robot can be in only one tile and has one colour at a time, areautomatically satisfied thanks to their representations as total functions from robot to tile ( robot at ) and from robot to color ( robot has ).Despite all the advantages explained above, this approach has a major limi-tation in terms of automation . Instead of writing only one specification for eachplanning problem of interest, users now have to provide two extra separatedinputs to the system. The first one is an essence specification expressing thevalidity constraints (step 1.2 in Figure 8), with variable names matched withthe predicates in the original PDDL problem description (step 1.1 in Figure 8).As there are several possible PDDL representations from a high-level abstractdescription of a problem, the second input is a manually written program toconvert the instances in essence back to the PDDL representation specifiedin Step 1.1 (from step 3 to step 4 in Figure 8). It is extremely difficult to au-tomate the generation of such converter as the system has to recognise whichPDDL representation is the right match. In the next section, we propose anelegant approach that can overcome all limitations on flexibility and efficiencydiscussed so far without any trade-offs on automation. As we will explain, thewhole instance generation process can be fully automated and various encodings,including PDDL, could be supported in a completely transparent manner. xploring Instance Generation for Automated Planning 13 given n_robot : int (1..) given tile_size: int (1..) letting n_tile be tile_size*tile_size given n_color : int (1..) letting robot be domain int (1 .. n_robot) letting tile be domain int (1. n_tile) letting color be domain int (1.. n_color)$ ----- Initial State -------- given init: record {robot_at : function ( total ) robot --> tile ,robot_has : function ( total ) robot --> color ,up : function tile --> tile , down : function tile --> tile ,left : function tile --> tile , right : function tile --> tile ,clear : set of tile , available_color : set of color} whereforAll c: color . c in init[ available_color ], $ all colors are available forAll u : tile .u in init[clear], $ all titles are clear$ square -grid constraintsu in defined (init[up]) <-> init[up](u) = u - tile_size ,u in defined (init[down ]) <-> init[down ](u) = u + tile_size ,u in defined (init[left ]) <-> (init[left ](u) = u - 1)/\ (u %u in defined (init[right ]) <-> init[right ](u) = u + 1/\ (u % given goal: record {painted : function ( minSize
1) tile --> color}
Fig. 10: Validity constraints for the whole floor-tile problem expressed directlyin essence
This section discusses abandoning a given PDDL description of a planning do-main as the starting point for generating instances, and instead extending the essence language to support the abstract specification of planning problems.The key feature of the essence language is the provision of high-level typeconstructors, such as set , relation and function , which allow a problem to bespecified directly in terms of the combinatorial structure to be found. Specifyinga planning problem in essence would remove the considerable difficulty of tryingto recover this structure from a PDDL description as discussed in the precedingsections. This would simplify the process of instance generation for a planningproblem. By providing a refinement of a planning problem specification to aPDDL model, we can also automate much of the work of producing a PDDLencoding of a problem and ensure that the instances generated are synchronisedwith the model chosen. These advantages result in a straightforward adaptationof the CP automated instance generation system, as presented in Figure 11.A simple approach to enabling the specification of planning problems in essence is to introduce a plan type constructor. In much the same way asPDDL, this would need to support a representation of the objects in the do-main, initial and goal states, and plan operators. However, we would gain thefar more expressive types available in essence in order to specify each of these (1) Planning problem specification in Essence● Objects and states● Validity constraints● Operators and conditions conjure (2) Instance generation problem in Essence (3) Valid Instances with desirable properties irace andEssence CP-toolchain Fig. 11: Automated instance generation for planning problems in essence . given n_robot : int (1..) given n_colour : int (1..) given tile_size : int (1..) letting CLEAR be letting GRID be domain matrix indexed by [ int (1.. tile_size), int (1.. tile_size)] of int (CLEAR ..) letting COLOUR be new type of size n_colour letting STATE be domain record {robots : sequence ( size n_robot) of record{row : int , column: int ,colour: COLOUR},grid : GRID} given init : STATE where $ all tiles are clear at the initial state forAll i, j : int (1.. tile_size) . init[grid ][i,j] = CLEAR given goal : GRID find p : plan with state STATEwith initialState initwith goalState state[grid] = goalwith actions [goUp , goDown , goLeft , goRight , paintUp , paintDown ,changeColour]
Fig. 12: A possible essence specification of the floor-tile problem.elements. PDDL adopts an implicit frame condition in which all parts of thestate not explicitly referenced by an action are assumed to be unchanged, whichit also seems sensible to employ in an essence plan type constructor.We will illustrate with a hypothetical essence specification of the floor-tile problem. Figure 12 presents the specification of the abstract plan decision vari-able p via a new plan type constructor, which expects four arguments. The firstis the state of the planning domain, which is specified using the existing record type, here capturing the position and colour of each of the robots as well as thecurrent grid state. The initial state is given as a parameter of the same type. Thegoal, which may only concern a part of the problem state, is flexibly expressedas a set of constraints. For the floor-tile domain, the goal is a particular gridconfiguration, expressed as an equality on the grid part of the plan state.The final argument is the list of available actions. Figure 13 presents a hypo-thetical action representing movement upwards on the grid. The goUp action isparameterised on the element of the state that must be selected by the planner,in this case which of the robots to move. Preconditions and effects are expressedas constraints on the parts of the state affected, hence benefiting from the ab-stract types and operators in essence . As in PDDL, there is an implied framecondition that any parts of the state not mentioned are unchanged. xploring Instance Generation for Automated Planning 15 letting goUp(r in robots) be domain action { precondition: grid[r[row]-1, r[column ]] = CLEAR ,effects: r[row]’ = r[row]-1} Fig. 13: A hypothetical essence plan action compatible with the plan statedefined in Fig. 12. The parameter r is to be chosen by the planner from amongthose robots in the plan state. The operator ’ denotes the state at the subsequentstep in the plan. An action such as this could be further annotated with a costvalue, if required.An essence specification of a planning problem such as the one describedin this section could be used for both instance generation and automated mod-elling. The parameters of the plan domain are apparent in the specification ofFigure 12 and, via the types available in essence , their structure is apparentrather than having to be recovered from a lower level description. Most of thevalidity constraints for the floor-tile problem are implicitly implied in thehigh-level representations. This simplifies instance generation considerably, andwould allow a similar approach to that used for essence specifications of con-straint satisfaction/optimisation problems [1].Automated modelling could also follow the current practice of refining essence specifications into constraint models using Conjure . With PDDL as the tar-get language, refinement rules would have to be written to encode the essence types and operators describing the plan state and actions. As is the case forconstraint models, multiple refinement rules could be written for the same typeto enable alternative PDDL encodings to be generated automatically. A fur-ther opportunity would be to exploit the existing
Conjure infrastructure torefine the specification to a constraint model of the planning problem, providingalternative solution options via CP, SAT, or SMT through Savile Row.
We have discussed various approaches for adapting a CP automated instancegeneration system for planning where the problem descriptions are written inPDDL, the standard modelling language for planning problems. The limitationsof those approaches are explained and a new language for describing planningproblems using essence , a high-level constraint modelling language, is proposed.Automated instance generation for planning based on essence offers greaterflexibility, efficiency and expressivity compared with its PDDL-based counter-part. In future work, an implementation of the proposal will be provided and athorough evaluation of such an instance generation system will be done.
Acknowledgements
This work is supported by EPSRC grant EP/P015638/1.Nguyen Dang is a Leverhulme Early Career Fellow.
References (1-2), 123–191 (2000)6. Bofill, M., Espasa, J., Villaret, M.: The RANTANPLAN planner: system descrip-tion. Knowledge Eng. Review (5), 452–464 (2016)7. Fuentetaja, R., De la Rosa, T.: A planning-based approach for generating planningproblems. In: Workshops at AAAI (2012)8. Gent, I.P., Jefferson, C., Miguel, I.: Minion: A fast scalable constraint solver. In:ECAI. pp. 98–102. IOS Press (2006), http://ebooks.iospress.nl/volumearticle/26589. Haslum, P., Scholz, U.: Domain knowledge in planning: Representation and use.In: Workshop on PDDL at ICAPS (2003)10. Helmert, M.: Concise finite-domain representations for PDDL planning tasks. Ar-tificial Intelligence (5-6), 503–535 (2009)11. van Hemert, J.I.: Evolving binary constraint satisfaction problem instances thatare difficult to solve. In: The 2003 Congress on Evolutionary Computation, 2003.CEC’03. vol. 2, pp. 1267–1273. IEEE (2003)12. Horie, S., Watanabe, O.: Hard instance generation for SAT. In: International Sym-posium on Algorithms and Computation. pp. 22–31. Springer (1997)13. Julstrom, B.A.: Evolving heuristically difficult instances of combi-natorial problems. In: GECCO 2009. pp. 279–286. ACM (2009).https://doi.org/10.1145/1569901.156994114. Kautz, H.A., Selman, B.: The Role of Domain-Specific Knowledge in the Planningas Satisfiability Framework. In: Proceedings of the Fourth International Conferenceon Artificial Intelligence Planning Systems, Pittsburgh, Pennsylvania, USA. pp.181–189 (1998)15. Kopczy´nski, E.: Axiomatizing rectangular grids with no extra non-unary relations.arXiv:1912.09797v1 (2019), https://arxiv.org/abs/1912.09797v116. Libkin, L.: Elements of Finite Model Theory. Springer (2004).https://doi.org/10.1007/978-3-662-07003-117. L´opez-Ib´a˜nez, M., Dubois-Lacoste, J., C´aceres, L.P., Birattari, M., St¨utzle, T.: Theirace package: Iterated racing for automatic algorithm configuration. OperationsResearch Perspectives , 43–58 (2016). https://doi.org/10.1016/j.orp.2016.09.002,http://iridia.ulb.ac.be/irace/18. McDermott, D., Ghallab, M., Howe, A., Knoblock, C., Ram, A., Veloso, M., Weld,D., Wilkins, D.: PDDL – the planning domain definition language (1998)xploring Instance Generation for Automated Planning 1719. Moreno-Scott, J.H., Ort´ız-Bayliss, J.C., Terashima-Mar´ın, H., Conant-Pablos,S.E.: Challenging heuristics: evolving binary constraint satisfaction problems. In:Proceedings of the 14th annual conference on Genetic and evolutionary computa-tion. pp. 409–416 (2012)20. Mu˜noz, M.A., Villanova, L., Baatar, D., Smith-Miles, K.: Instance spacesfor machine learning classification. Machine Learning (1), 109–147 (2018).https://doi.org/10.1007/s10994-017-5629-521. Nightingale, P., Akg¨un, ¨O., Gent, I.P., Jefferson, C., Miguel, I., Spracklen, P.:Automatically improving constraint models in Savile Row. Artificial Intelligence , 35–61 (2017). https://doi.org/10.1016/j.artint.2017.07.00122. Selman, B., Mitchell, D.G., Levesque, H.J.: Generating hard satisfiability problems.Artificial Intelligence (1-2), 17–29 (1996)23. Shleyfman, A., Karpas, E.: Position paper: Reasoning about domains with PDDL.In: 2018 AAAI Spring Symposia, Stanford University, Palo Alto, California, USA.AAAI Press (2018)24. Smith-Miles, K., Baatar, D., Wreford, B., Lewis, R.: Towards objective measures ofalgorithm performance across instance space. Computers & Operations Research , 12–24 (2014)25. Smith-Miles, K., Lopes, L.: Measuring instance difficulty for combinatorial opti-mization problems. Computers & Operations Research (5), 875–889 (2012)26. Ullrich, M., Weise, T., Awasthi, A., L¨assig, J.: A generic problem instance generatorfor discrete optimization problems. In: GECCO 2018. pp. 1761–1768. ACM (2018).https://doi.org/10.1145/3205651.320828427. Vallati, M., Chrpa, L., Grze´s, M., McCluskey, T.L., Roberts, M., Sanner, S., et al.:The 2014 international planning competition: Progress and trends. AI Magazine (3), 90–98 (2015)28. Vanhoucke, M., Maenhout, B.: On the characterization and generation of nursescheduling problem instances. European Journal of Operational Research (2),457–467 (2009). https://doi.org/10.1016/j.ejor.2008.03.04429. Xu, K., Boussemart, F., Hemery, F., Lecoutre, C.: Random constraint satisfaction:Easy generation of hard (satisfiable) instances. Artificial Intelligence171