Controller Synthesis with Inductive Proofs for Piecewise Linear Systems: an SMT-based Algorithm
Zhenqi Huang, Yu Wang, Sayan Mitra, Geir E. Dullerud, Swarat Chaudhuri
CController Synthesis with Inductive Proofs for Piecewise LinearSystems: an SMT-based Algorithm
Zhenqi Huang, Yu Wang, Sayan Mitra, Geir E. Dullerud and Swarat Chaudhuri
Abstract — We present a controller synthesis algorithm forreach-avoid problems for piecewise linear discrete-time systems.Our algorithm relies on SMT solvers and in this paper wefocus on piecewise constant control strategies. Our algorithmgenerates feedback control laws together with inductive proofsof unbounded time safety and progress properties with respectto the reach-avoid sets. Under a reasonable robustness assump-tion, the algorithm is shown to be complete. That is, it eithergenerates a controller of the above type along with a proof ofcorrectness, or it establishes the impossibility of the existenceof such controllers. To achieve this, the algorithm iterativelyattempts to solve a weakened and strengthened versions ofthe SMT encoding of the reach-avoid problem. We presentpreliminary experimental results on applying this algorithmbased on a prototype implementation.
I. I
NTRODUCTION
A Satisfiability Modulo Theory (SMT) problem is a clas-sical decision problem in computer science [6]. It takes asinput a logical formula in first-order logic that can involvecombinations of background theories, and requires one todecide whether or not the formula has a satisfying solution.For a bounded time horizon k , a simplest SMT problemin Equation (1), for instance, is an encoding of a searchfor a sequence of control inputs vectors u , . . . , u k thatdrives a discrete time linear open-loop control system fromevery initial state in the hypercube r , . s n to the hypercube r . , s n in k steps, while always keeping the state inside thehypercube r , s n . D u , . . . u k , @ x P r , . s n , @ t P t , . . . , k ´ u , let x t ` “ Ax t ` Bu t such that x t P r , s n and x k P r . , s n . (1)This example has several constraints that are defined interms of the quantified variables u i and x i , numericalconstants (including those in the matrices A and B ), andthe background theory of linear real arithmetic. An SMTsolver is a software tool that solves SMT problems byeither giving an assignment to the variables that satisfyall the constraints or by saying that none exists. ModernSMT solvers routinely handle linear problems with thousandsof variables and millions of constraints, so much as theyhave become the engines for innovation in verification andsynthesis for computer software and hardware [2], [10], [13].Although many control systems can only be modeled bymeans of nonlinear arithmetic over the real numbers in-volving transcendental functions that make the correspondingSMT problems undecidable, the solvers are evolving rapidlyand several incorporate approximate decision procedures fornonlinear arithmetic [15]. These technological developments motivated us (and others [20], [21]) to explore SMT-basedcontroller synthesis.In this paper, we present an algorithm that uses SMTsolvers for synthesizing controllers for discrete time systems.The dynamics of the system is given as a piecewise linearfeedback control system. The control requirements are thestandard reach-avoid specification [7], [8]: a set of states Goal that has to be reached while always staying inside a
Safe set.A key difficulty in using SMT for synthesis, is that theresulting SMT problem has to encode the unrolled dynamicsof the system with the unknown controller inputs. In theabove simple example, this gave rise to k control input vari-ables and the intermediate states. For more general nonlinearmodels, the intermediate states cannot be written down inclosed form and one has to unroll the over-approximationsof the dynamics. This can then lead to overtly conservativeanswers from the solver. We present a technique that avoidsthis problem by synthesizing the control law together withan inductive proof of its correctness. The proof has twoparts: (a) an inductive invariant that implies safety and (b) aranking function that implies progress. A positive side-effectof this is that it can not only synthesize controllers withunderstandable correctness proofs, but it can also establishthe nonexistence of provably correct controllers (of a certaintemplate).In Section III we define the system model, the reach-avoidsynthesis problem and a particular notion of robustness ofmodels. In Section IV we first present a basic SMT encodingof the synthesis problem and then a strengthened and a weak-ened version this encoding. Using these two encodings, inSection V we present the synthesis algorithm, its soundnessand relative completeness. In Section VI we illustrate anapplication of the algorithm in a vehicle navigation problemand conclude in Section VII.II. R ELATED W ORK
Researchers have recently used SMT solvers for synthesiz-ing programs and strategies in games. The approach in [21]uses SMT to find controllers for general linear temporal logic(LTL) specifications by stitching together motion primitivesfrom a library. Unlike our encoding with inductive proofs, theapproach of [21] involves bounded unrolling of the dynamics.In [20], the authors used SMT solvers to synthesizeintegrated task and motion plans by constructing a placementgraph. In [4], a constraint-based approach was developedto solve games on infinite graphs between the system and a r X i v : . [ c s . S Y ] S e p he adversary. Our work can be seen as introducing controltheoretic constraints to the SMT formulation.The authors of [11], [26] proposed a game theoreticalapproach to synthesize controller for the reach-avoid prob-lem, first for continuous and later for switched systems. Inthese approaches, the reach set of the system is computedby solving a non-linear Hamilton-Jacobi-Isaacs PDE. Ourmethodology, instead of formulating a general optimizationproblem for which the solution may not be easily com-putable, solves a special case exactly and efficiently. Withthis building block, we are able to solve more generalproblems through abstraction and refinement.Model predictive control (MPC) can as well be used tosolve the reach-avoid problem [3], [24]. In each cycle of anMPC, the optimal input for reaching the goal while avoidingthe obstacle, is computed for a fixed prediction horizon.Then, the first part of the optimal control input is applied,and a new input is computed from the new state, and soon. As the prediction horizon increases, the applied inputconverges to the optimal reach-avoid input. In the contrast,our approach can be used to synthesis controls for possiblyunbounded horizon with safety and progress guarantees andcan establish nonexistence of controller of certain type.There is a large body of results on automata theoreticapproaches for controller synthesis [1], [16], [18], [23], [25].The approach here is to construct a finite abstraction ofthe dynamical system and then invoke the LTL synthesisalgorithms such as the one in [5]. This approach has beenapplied to several systems and several software tools forsynthesis have been implemented [9], [19].The authors of [17], [22] build Markov decision treesto synthesize control policies with maximum probability ofsatisfying the specifications. Our method is very differentsince we consider deterministic systems and try to synthesizecontroller that are guaranteed to satisfy the specifications.III. PRELIMINARIES AND B ACKGROUND
Sets and Functions:
For a natural number N , r N s denotes the set t , , . . . , N ´ u . Given two functions f, g : A Ñ R n , we use d p f, g q “ | f p a q ´ g p a q| to denote the (cid:96) distance between f and g , where | ¨ | is the standard -norm.We will use finite collections of sets to approximatearbitrary compact subsets in R n . For a finite collection P of subsets of R n and a subset S Ď R n , we say that P preserves S if there exists a subset P Ď P such that (i) Ť P P P P “ S , and (ii) @ P P P z P , P X S “ H . In otherwords, P completely and exactly represents S .A finite partition P of a compact subset S Ď R n is a finitedisjoint collection of sets that exactly cover S . The resolution of a partition P is the maximum diameter of the sets in P .For two partitions P , P of a compact set S , we say that P subsumes P , if for any I P P , there exists I P P such that I Ď I . Piecewise Linear Systems, Feedback, and Robustness:
A piecewise linear system M is a tuple p X , U , loc , I , F q where (a) X Ď R n is a compact set called the state space ,(b) U Ď R m is a compact set called the input space , (c) loc is a finite set called the set of locations , (d) I “ t P l u l P loc isa partition of X and each element of I is called a locationinvariant , and (e) F “ t f l u l P loc is a collection of linear dynamic functions f l : X ˆ U Ñ X .The evolution of the continuous state of the system isgoverned by the dynamic function of the location invariantit is currently in. For any time t P N , a state x t P P l andan input u t P U the next state of the system is given by thediscrete-time dynamics: x t ` “ f l p x t , u t q . (2)A general static state-feedback control law can be thoughtof as a function u : X Ñ U that maps each state to an input.In many systems, sensors and controller hardware have afinite resolution, and therefore, such a general law cannot beimplemented. In this paper, we assume that M is associatedwith a controller table C which is a partition of the statespace X and the u : C Ñ U maps each partition in C to aninput. Essentially u is a look-up table, which assign an inputfor every equivalence class defined by C .For a feedback control policy u and a system (2), thenext state is just a function of the current state. We denote post M p x, u q “ f l p x, u p x qq if x P P l . The subscript M isdropped if it is clear in the context. We denote by post t p x, u q the state reached from x after the t th step. For a compactset of states S Ď X , we define post p S, u q “ t x : D x P S such that x “ post p x, u qu . The t step post operation post t p S, u q is defined similarly.Our synthesis algorithm will be complete for systemmodels upto some imprecision in the model. For a system M “ p X , U , loc , I , F q and (cid:15) ą , another system M is an (cid:15) -perturbation of M if it is identical to M except that theset of dynamic functions for M is F “ t f l u l P loc , such thatfor each l P loc , d p f l , f l q ď (cid:15) . We denote by B (cid:15) p M q the setof all models that are (cid:15) -perturbations of M . Reach-Avoid Control Problem (
RAC ): A reach-avoidcontrol ( RAC ) problem is parameterized by the systemmodel M , the controller table C , and three sets of states Init , Safe , Goal Ď X called the initial, safe and goal states.We will assume that these sets have some finite represen-tation (for example, hyperrectangles, polytopes). We definewhat it means to solve a RAC problem with a feedbackcontrol policy u . Definition 1. A solution to a RAC is a feedback controlpolicy u : C Ñ U such that for any initial state x P Init , thestates visited by the system satisfies the condition: ‚ (Safety) for all t P N , x t P Safe and ‚ (Progress) there exists T P N such that x T P Goal . Throughout the paper a
RAC is uniquely specified by amodel M as the rest parameters are fixed.IV. C ONSTRAINT - BASED S YNTHESIS
A major barrier in encoding
RAC as an SMT problem isthat the safety and progress requirements are over unboundedime. Moreover, these requirements are stated in terms ofthe future reachable states of the system and computingthat in and on itself is a hard problem. Instead of workingwith unbounded time reach sets, we address this problemby encoding a set of rules that inductively prove safety andprogress of the control system.
A. Inductive Synthesis Rules
In addition to searching for the feedback control law u : C Ñ U , the SMT problem will encode the search for (a) aninductive invariant Inv Ď X that proves safety with u , and(b) a ranking function V that proves progress with u .In order to constrain the search, we will fix a template forthe ranking function. For this paper, we will use the template C Ñ N , that is, any function that is piecewise constant onthe partition of the state space C . This choice has an easyinterpretation: each entry in the controller table gives therank of the controller along with the feedback law. Let V denote the countable set of all such functions. Each rankingfunction V P V maps every state in X to a natural number.For any C P C , V p C q is the natural number that all the states x P C map to. Now we are ready to present the basic rulesencoding inductive synthesis of RAC :Find u : C Ñ U , V P V , Inv Ď X such that:R1: Init Ď Inv
R2: post p Inv, u q Ď Inv
R3:
Inv Ď Safe
R4: C Ď Goal ðñ V p C q “ R5: C Ď Inv ^ post p C, u q X C ‰ Hñ V p C q ě V p C q R6: C Ď Inv z Goal ^ post k p C, u q X C ‰ Hùñ V p C q ą V p C q .Fig. 1: Basic rules Π p M, C , V q for synthesis for RAC .Rules R1-R3 imply that
Inv is a fixed-point of post withcontrol u that contains Init and is contained in
Safe , andtherefore, is adequate for proving safety. Rule R4 states thethe rank of any region C vanishes iff it is in Goal . RuleR5 encodes the (Lyapunov-like) property that the rank ofany region C is nonincreasing along trajectories. Finally,rule R6 states that for any non- Goal region C , the rankdecreases with u within k steps, where k is an inductionparameter of this encoding. For RAC specified by model M with controller table C and a template V , we denote theSMT problem (Figure 1) as Π p M, C , V q For some control u , ranking function V P V and Inv Ď X , we write u , V, Inv |ù Π p M, C , V q if the Rules R1-R6 are satisfied. Theorem 2 (Soundness) . If u , V, Inv |ù Π p M, C , V q , then u solves RAC specified by M .Proof. Let u , V, Inv satisfy rules in Figure (1). Fixing any x P Init , we prove safety and progress conditions separately. For the sake of clarity, we supress the dependence on k . From R1, x P Inv . Combined with R2, we have for any t P N , post u p x, t q P Inv . Since
Inv Ď Safe (R3) we have post u p x, t q P Safe for any t . Thus the safety condition holds.We assume x P C such that C X Goal “ H ; otherwise theprogress condition holds trivially. From R4 we have V p C q ą . From R5 and R6, in at most kV p C q steps, V decreasesto 0. By R this implies that x reaches the goal. a) Robustness Modulo Templates: In Section III wedefined perturbations of system models, here we lift thedefinition to the corresponding synthesis rules: Π p M , C , V q is an (cid:15) -perturbation of Π p M, C , V q if (i) the controller table and the ranking templates are identical C “ C , V “ V , and(ii) the model M P B (cid:15) p M q is an (cid:15) -perturbation of M . Definition 3.
For a controller table C and a template ofranking functions V , a RAC specified by M is robust modulo p C , V q if there exists (cid:15) ą such that either of the followingholds:(i) there exists a control u and a ranking function V P V such that for any M P B (cid:15) p M q , u , V, Inv |ù Π p M , C , V q with some Inv Ď X , or(ii) for none of M P B (cid:15) p M q , the synthesis problem Π p M , C , V q is satisfiable. In Theorem 7 we will show that our synthesis algorithmis also relatively complete with respect to this notion ofrobustness.
B. Weakened and Strengthened Rules
The main challenge in solving Π p M, C , V q is the post operator in Rules R2, R5, and R6. We need a reasonablerepresentation of post for this computation to be effective.In this work, we use a finite partition P of the statespace (which preserves C , Init , Safe , Goal ) for computingthe post . This choices is somewhat independent of the restof the methodology and any other template (for example,linear functions, support functions, zonotopes) could be usedinstead of the fixed partitions.The key idea to solve it is to create over and under approximations of the post operator with respect to therepresentation of choice—in this case representation usingthe fixed partition P . These operators are then used to createweakened and strengthened versions of the basic inductiverules that can be effectively solved as SMT problems.We define an over-approximation ( P - post ) and an under-approximation ( P - post ) of the post operator with respect toa partition P as follows: for any compact S Ď X , P - post p S, u q “ ď P P P ^ P X post p S, u q‰H P, (3) P - post p S, u q “ ď P P P ^ P Ď post p S, u q P. (4)Roughly, the over-approximation P - post p S, u q computes theminimum superset of S which is preserved by P andthe under-approximation P - post p S, u q computes the max-imum subset of S which is preserved by P . We define - post t p S, u q and P - post t p S, u q as the t step over andunder-approximations in the usual way. Proposition 4.
For any measurable S Ď X , a post operatorand any partition P , the following properties hold:(i) P - post p S, u q Ď post p S, u q Ď P - post p S, u q ,(ii) If P subsumes P , then P - post p S, u q Ě P - post p S, u q and P - post p S, u q Ď P - post p S, u q , and(iii) For any (cid:15) ą , D δ ą such that for any P with reso-lution less than δ , d p P - post p S, u q , P - post p S, u qq ă (cid:15) . Instead of searching for an exact inductive invariant
Inv , the weakened and strengthened versions of the syn-thesis rules presented below try to find under (
M ust )and over-approximations (
M ay ) of the invariant using the P - post p S, u q and P - post p S, u q operators.Find u : C Ñ U , V P V , Must Ď X such thatW1: Init Ď Must
W2: P - post p Must , u q Ď Must
W3:
Must Ď Safe
W4: C Ď Goal ðñ V p C q “ W5: C Ď Must ^ C Ď P - post p C, u qùñ V p C q ě V p C q W6: C Ď Must z Goal ^ C Ď P - post k p C, u qùñ V p C q ą V p C q Fig. 2: Weakened rules Π w P p M, C , V q for synthesis.Find u : C Ñ U , V P V , May Ď X s.t.:S1: Init Ď May
S2: P - post p May , u q Ď May
S3:
May Ď Safe
S4: C Ď Goal ðñ V p C q “ S5: C Ď May ^ C Ď P - post p C, u qùñ V p C q ě V p C q S6: C Ď May z Goal ^ C Ď P - post k p C, u qùñ V p C q ą V p C q Fig. 3: Strengthened rules Π s P p M, C , V q for synthesis. Lemma 5.
For any P , the following hold:(i) if u , V, Inv |ù Π p M, C , V q , then there exist Must Ď X such that u , V, Must |ù Π w P p M, C , V q ; and(ii) if u , V, May |ù Π s P p M, C , V q , then exists Inv Ď X such that u , V, Inv |ù Π p M, C , V q .Proof. Suppose u , V, Inv |ù Π p M, C , V q . We will show thatthere exists a Must Ď X satisfying the weakened rules (W1-W6). Fix a u . From Proposition 4, the operator P - post p¨ , u q is upper bounded by post p¨ , u q . Since post p¨ , u q has afixed point Inv , the fixed point of P - post p S, u q exists.Let Must be the fixed point defined by W1-W2 and wehave
Must Ď Inv . It follows that
Must Ď Inv Ď Safe and W3 holds. W4 is inherited from R4. For any C Ď Must ^ C Ď P - post p S, u qp C q , we have May Ď Inv and P - post p S, u qp C q Ď post u p C q . From R5, therefore, V p C q ě V p C q and W5 holds. Similarly, P - post p S, u qp C, k q Ď post u p C, k q , thus W6 also holds. Therefore, u , V, Must |ù Π w P p M, C , V q .The proof of second part is similar.The above lemma states that the weakening and strength-ening of the synthesis rules are sound. With the additionalrobustness condition, we can show that either the former isunsatisfiable or the latter is satisfiable. Lemma 6.
If a
RAC specified by M is robust modulo C , V ,then there exists a sufficiently fine partition P such thateither (i) Π w P p M, C , V q is unsatisfiable or (ii) Π s P p M, C , V q is satisfiable.Proof. We discuss the two cases in Definition 3. In thisprove post, P ´ post, P ´ post without a subscript denotethe operator with respect to model M .Suppose there exists (cid:15) ą such that some controller u and ranking function V P V solves all (cid:15) -perturbations of Π p M, C , V q . That is, for each M P B (cid:15) p M q , there exists a Inv M , such that u , V, Inv M |ù Π p M q . We define Inv (cid:15) as the union of all such
Inv M ’s. Roughly, Inv (cid:15) is the setof states that can be visited for some (cid:15) -perturbation of M with controller u . Since every Inv M satisfies R3, R5, R6in Figure 1, it can be shown that (i) Inv (cid:15) Ď Safe , (ii) C Ď Inv (cid:15) ^ post p C, u q X C ‰ H ùñ V p C q ě V p C q ,and (iii) C Ď Inv (cid:15) z Goal ^ post k p C, u q X C ‰ H ùñ V p C q ą V p C q . Also, any subset of Inv (cid:15) also satisfy theabove three formula. From Proposition 4, for sufficiently finepartition P , for any S Ď X , d p P - post p S, u q , post p S, u qq ď (cid:15) . We will inductively prove that the May set with re-spect to this partition P is a subset of Inv (cid:15) . (i) Initially,
Init Ď Inv (cid:15) . (ii) For any set S Ď Inv (cid:15) , and any state x P P - post p S, u q , it suffice to prove x P Inv (cid:15) . First,since d p P - post p S, u q , post p S, u qq ď (cid:15) . We can find a state x P post p S, u q Ď Inv (cid:15) such that || x ´ x || ď (cid:15) . Since x is in Inv (cid:15) , it is reached by some model M P B (cid:15) p M q forthe first time. We construct a model M that is identicalto M elsewhere except that at state x the dynamics is x “ post M p x, u q . It is easy to show that M is a (cid:15) -perturbation of M which visits x with controller u . Thus x P Inv (cid:15) . By (i) and (ii) above, we derive
May Ď Inv (cid:15) .It follows that the strengthened rules S3, S5 and S6 aresatisfied. In addition, S1-S2 is satisfied by the definitionof
Must and S4 is just inherited from R4. Therefore, thestrengthened rules are satisfiable.Otherwise suppose exists (cid:15) ą such that none of the (cid:15) -perturbation of Π p M, C , V q is satisfiable. Again from Propo-sition 4, for sufficiently fine partition P , for any S Ď X , d p P - post p S, u q , post p S, u qq ď (cid:15) . We on the contrary assumethere exists some controller u , V, Must |ù Π w P p M, C , V q .We define a model M such that for any cell C P C and each state x P C , the dynamics of M is capturedby post M p x, u q “ P roj p post p x, u q , P - post p C, u qq . Theoperator P roj p x, A q is a projection that maps x to astate in A that is closest to x . It can be shown that is an (cid:15) -perturbation of M . Moreover, for any cell C Ď X , post M C “ P - post p C, u q . Thus, the problem of Π p M , C , V q and Π w P p M, C , V q are identical. It follows that u , V, Must |ù Π p M , C , V q , which contradicts the fact noneof (cid:15) -perturbation of Π is satisfiable modulo C , V .V. SMT- BASED S YNTHESIS A LGORITHM
We introduce an algorithm for controller synthesis for
RAC using the strengthened and weakened inductive SMTencodings of the previous section. The algorithm takes asinput the model M , the controller table/partition C , the tem-plate for the ranking function V and the three sets Init , Safe and
Goal that define
RAC problem. It iteratively refines thepartition P for representing invariants and makes subroutinecalls to the SMT solver with the strengthened and weakenedrules until it either finds a controller law u or outputs K .Specifically, in each iteration, (a) if the strengthened problem Π s P is satisfiable then it returns the satisfying u . (b) if theweakened problem Π w P is unsatisfiable then it returns K .Otherwise, (c) it refines the partition P (using the M ust setcomputed from Π w P ). The Refine p P , Must q function creates Algorithm 1:
SMT-based Synthesis Algorithm input: M, C , V , Init , Safe , Goal ; P Ð initP artition ; while True do p val s , u q Ð Solve p Π s P p M , C , V qq ; p val w , Must q Ð
Solve p Π w P p M , C , V qq ; if val s “ SAT then return u else if val w “ UNSAT then return K else P Ð Refine p P , Must q ; end end a finer partition of P . For the completeness result, we requirethat for any P , by iteratively applying Refine , the resolutionof the resulting partition can be made arbitrarily fine. InSection V-B, we discuss several heuristics for refinement thatpotentially improve the performance of the algorithm.
A. Soundness and Relative Completeness
We will next sketch the arguments for the correctnessof the algorithm. Soundness of the algorithm implies thatwhenever it outputs u , (i) that u is a control law that solvesthe RAC problem, (ii) the
May set obtained from solving Π s P in the final iteration is an inductive proof certificates forsafety with u , and (iii) the V is a k -step inductive proofcertificate for progress with u . And, whenever the algorithmoutputs K then there does not exists a controller u , a rankingfunction V P V and an invariant Inv Ď such that the above(i)-(iii) holds. In addition, we show that the algorithm is relative com-plete. That is, if RAC is robust modulo C , V , then thealgorithm terminates with one of the above answers. Theorem 7.
The algorithm is sound and relatively complete.Proof. Soundness.
If the algorithm terminates and return u ,then for some partition P , the SMT solver returns a satisfyingsolution u with Π s P p M q . From Lemma 5, u solves the RAC .Otherwise if the algorithm terminates and returns K , thenfor some partition P , the SMT solver on Π w P p M q returnsUNSAT. From Lemma 5, there is no control that solves the RAC problem modulo V . Relative Completeness.
Since Π p M q is a robust RAC modulo V , from Lemma 6, we know that for a sufficientlyfine partition, either Π w P p M q is unsatisfiable or Π s P p M q isare satisfiable. Thus the while-loop will terminate as thealgorithm creates fine enough partitions. B. Guided Refinement
There are different ways in which the refinement of thepartition P can be implemented without compromising thesoundness and the relative completeness guarantees. Thenaive strategy of subdividing every equivalence class in P , increases the size of the SMT problems quickly. Asour algorithm solves both the weakened and strengthenedversions of the problem simultaneously, we can marshallextra information in performing refinement. For example,when the weakened rules return a possible control u alongwith its proof V, Must , even though this controller u cannotbe proven (to be safe and progress making) with the strength-ened rules, it can provide useful information for guiding therefinement. Definition 8.
For a partition P and a set S that is preservedby P , P is a S -guided refinement of P if P is derived byrefining the cells of P that are in S . One key observation is that, a X z Must -guided refinementhelps in generating safety proofs (S3 and W3), while a
Must -guided refinement can improve the precesion of progressproofs (S5-S6 and W5-W6). The following proposition for-malizes part of this intuition and states that for given acontroller u , refining the cells in Must does not improve theprecision of the fixed-point
Must , May computed by rulesS1-S2 and W1-W2.
Proposition 9.
For any control u , any set Init and anypartitions P , let Must , May be the fixed point of operator P - post p¨ , u q and P - post p¨ , u q containing Init . Let P be a Must -guided refinement of P and Must , May be the fixedpoint of P - post p¨ , u q and P - post p¨ , u q containing Init .Then,
Must “ Must and May “ May . By above proposition, a
Must -guided refinement providesno help in generating better safety proofs. However, fromProposition 4, a finer partition P increase the precision of P - post p C, u q and P - post p C, u q . Since the rules S5-S6 andW5-W6 involve computing P - post p C, u q and P - post p C, u q or cells in May and
Must respectively, a
Must -guidedrefinement possibly increases the precision of these rules.Based on the above observations, we can adopt to the follow-ing heuristics for refinement: If the
Must set is close to theunsafe set, perform X z Must -guided refinement, otherwiseperform
Must -guided refinement.VI. P
ROTOTYPE I MPLEMENTATION AND E XPERIMENTS
We implemented the synthesis algorithm in Python usingthe the CVC4 SMT solver [2]. In this section, we brieflyreport preliminary results on applying it to a simple classof navigation problems. With this implementation, we wereable to automatically synthesize correct controls (and theirinductive proofs) for some configurations and proved impos-sibility for others.
Vehicle Navigation Problem:
We consider a reach-avoidproblem for a vehicle that follows piecewise linear approxi-mation of Dubin’s dynamics. The system model has statevariables r x, y, v, θ s T : position, velocity and heading angle ofthe vehicle. It has input variables r α, β s T : the accelerationand the turning rate. From the continuous Dubin’s vehiclemodel: x “ v cos θ, y “ v sin θ, v “ α, θ “ vβ. weconstruct a switched linear model by partitioning the domainof θ and v into locations, and for each location wecompute an approximate linear dynamics. The result is aswitched-linear model: x ` “ x ` av ` b, y ` “ y ` cv ` d, v ` “ v ` α, θ ` “ θ ` eβ, (5)where a, b, c, d, e have different values in different locations.The piecewise linearized model preserves some properties ofthe original system. For example, the linearized model cannotturn in place: if the velocity is close to 0, the heading θ cannot change. Moreover, the velocity is non-negative, whichfurther restricts its maneuverability. These properties giverise to interesting RAC problem instances where the systemhas no satisfying control law.We allow finitely many discrete input values and compute P - post p C, u q and P - post p C, u q offline as follows: For agiven partition P , and a cell C P P , we first identify a setof cells N p C q such that C P N p C q can visit some state in C in one step. Then, for each possible input u , we computethe one step reach set of post p N p C q , u q with the help fromreachability tools such as [12], [14]. Thus we just need toidentify the input combinations such that C is covered by orintersected with post p N p C q , u q . Experimental Results:
We performed several experi-ments for the above class of problems using our prototypeimplementation. We search for a control policy as a look-up table, specified by a controller table . We utilize a controller table C with 768 cells in total. In Figure 5 and 4,the grids illustrate the projection of controller table to x, y coordinates.We create a partition P by further partitioning each cell in C into 4 pieces, with which we construct both the weakenedand the strengthened rules. For some cases, we provedthe impossibility of synthesis. We visualize such a case in Fig. 4: A RAC instance that is impossible to solve. The gridillustrates the controller table, the green block at the bottomleft corner is
Init , the blue rectangle at the top right is
Goal ,the smaller red blocks are unsafe.Fig. 5: A
RAC instance that has a satisfying control law.The lighter connected region is the
Must set and the darkerregion together with the lighter region is the
May set.Figure 4. While for other cases, we successfully synthesizeda control policy. An example is illustrated in Figure 5. Thesatisfying control policy is synthesized with an inductiveproof, namely the
May set and the ranking function V .In the constraints of this synthesis problem, there are768 real-valued variables for control input in each cell,3072 integer variables for values of the ranking functionfor each partition and 3072 boolean variables indicateswhether a partition is reached. The weakened or strengthenedinductive rules are encoded in roughly 7000 constraints. Theconstraints are solved by CVC4 [2] in 10 minutes.VII. C ONCLUSION
In this work, we studied the controller synthesis problemof discrete-time systems with possibly unbounded time safetyand progress specifications. Leveraging the growing strengthof modern SMT tools, we propose an algorithm that findscontrollers as well as inductive proofs of their correctness.Specifically, the algorithm creates a weaker and a strongerversion of the synthesis problem and encodes them as SMTproblems. By solving the controller synthesis problems forthese two bounding systems automatically with SMT solvers,we can solve the synthesis problem for the original system.We prove that this algorithm is sound and relatively completeand show that the solution given by the strengthened systemrovide a guidance for refining the bounding system. Ourexperimental results based on a prototype implementationsuggest that this can be a promising direction of investigationfor controller synthesis research.Since the core problem of computing over-approximationsof post are decoupled from synthesis in this formulation,one future direction of research that this work opens up isto extend this framework to nonlinear system models. Theperformance of the algorithm depends on the templates ofthe control, ranking function and invariants. Thus, to exploredifferent classes of templates and study their performance inour synthesis framework is also a natural next step.R
EFERENCES[1] E. Aydin Gol, Xuchu Ding, M. Lazar, and C. Belta. Finite Bisimula-tions for Switched Linear Systems.
IEEE Transactions on AutomaticControl , 59(12):3122–3134, December 2014.[2] Clark Barrett, Christopher L Conway, Morgan Deters, Liana Hadarean,Dejan Jovanovi´c, Tim King, Andrew Reynolds, and Cesare Tinelli.Cvc4. In
Computer aided verification , pages 171–177. Springer, 2011.[3] Alberto Bemporad, Francesco Borrelli, Manfred Morari, et al. Modelpredictive control based on linear programming˜ the explicit solution.
IEEE Transactions on Automatic Control , 47(12):1974–1985, 2002.[4] Tewodros Beyene, Swarat Chaudhuri, Corneliu Popeea, and AndreyRybalchenko. A constraint-based approach to solving games oninfinite graphs. In
Proceedings of the 41st annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages , pages221–234. ACM, 2014.[5] Roderick Bloem, Barbara Jobstmann, Nir Piterman, Amir Pnueli, andYaniv Sa’ar. Synthesis of reactive(1) designs.
Journal of Computerand System Sciences , 78(3):911 – 938, 2012. In Commemoration ofAmir Pnueli.[6] Egon B¨orger, Erich Gr¨adel, and Yuri Gurevich.
The classical deci-sion problem . Perspectives in mathematical logic. Springer, Berlin,Heidelberg, New York, 1997.[7] Alvaro Cardenas, Saurabh Amin, Bruno Sinopoli, Annarita Giani,Adrian Perrig, and Shankar Sastry. Challenges for securing cyberphysical systems. In
Workshop on future directions in cyber-physicalsystems security , 2009.[8] Alvaro A C´ardenas, Saurabh Amin, and Shankar Sastry. Researchchallenges for the security of control systems. In
HotSec , 2008.[9] Feng Chen and Grigore Rosu. Mop: Reliable software developmentusing abstract aspects.
Technical Report UIUCDCS-R-2006- 2776,Department of Computer Science, University of Illinois at Urbana-Champaign , October 2006.[10] Leonardo De Moura and Nikolaj Bjørner. Z3: An efficient smt solver.In
Tools and Algorithms for the Construction and Analysis of Systems ,pages 337–340. Springer, 2008.[11] Jerry Ding, Eugene Li, Haomiao Huang, and Claire J Tomlin.Reachability-based synthesis of feedback policies for motion planningunder bounded disturbances. In
Robotics and Automation (ICRA),2011 IEEE International Conference on , pages 2160–2165. IEEE,2011.[12] Parasara Sridhar Duggirala, Sayan Mitra, Mahesh Viswanathan, andMatthew Potok. C2e2: A verification tool for stateflow models.[13] Bruno Dutertre and Leonardo De Moura. The yices smt solver.
Toolpaper at http://yices. csl. sri. com/tool-paper. pdf , 2(2), 2006.[14] Goran Frehse, Colas Le Guernic, Alexandre Donz´e, Scott Cotton,Rajarshi Ray, Olivier Lebeltel, Rodolfo Ripado, Antoine Girard, ThaoDang, and Oded Maler. Spaceex: Scalable verification of hybridsystems. In
Computer Aided Verification , pages 379–395. Springer,2011.[15] Sicun Gao, Soonho Kong, and Edmund M. Clarke. dReal: AnSMT Solver for Nonlinear Theories over the Reals. In Maria PaolaBonacina, editor,
Automated Deduction CADE-24 , number 7898 inLecture Notes in Computer Science, pages 208–214. Springer BerlinHeidelberg, 2013.[16] H. Kress-Gazit, G.E. Fainekos, and G.J. Pappas. Temporal-Logic-Based Reactive Mission and Motion Planning.
IEEE Transactions onRobotics , 25(6):1370–1381, December 2009. [17] M. Lahijanian, S.B. Andersson, and C. Belta. Temporal Logic MotionPlanning and Control With Probabilistic Satisfaction Guarantees.
IEEETransactions on Robotics , 28(2):396–409, April 2012.[18] Jun Liu and Necmiye Ozay. Abstraction, Discretization, and Robust-ness in Temporal Logic Control of Dynamical Systems. In
Proceedingsof the 17th International Conference on Hybrid Systems: Computationand Control , HSCC ’14, pages 293–302, New York, NY, USA, 2014.ACM.[19] Manuel Mazo Jr, Anna Davitian, and Paulo Tabuada. Pessoa: A toolfor embedded controller synthesis. In
Computer Aided Verification ,pages 566–569. Springer, 2010.[20] S. Nedunuri, S. Prabhu, M. Moll, S. Chaudhuri, and L.E. Kavraki.SMT-based synthesis of integrated task and motion plans from planoutlines. In , pages 655–662, May 2014.[21] I. Saha, R. Ramaithitima, V. Kumar, G.J. Pappas, and S.A. Seshia.Automated composition of motion primitives for multi-robot systemsfrom safe LTL specifications. In , pages 1525–1532, September 2014.[22] M´aria Svoreˇnov´a, Martin Chmel´ık, Kevin Leahy, Hasan Ferit Eniser,Krishnendu Chatterjee, Ivana ˇCern´a, Calin Belta, et al. Temporal logicmotion planning using pomdps with parity objectives. 2015.[23] Maria Svorenova, Jan Kretinsky, Martin Chmelik, Krishnendu Chat-terjee, Ivana Cerna, and Calin Belta. Temporal Logic Control forStochastic Linear Systems using Abstraction Refinement of Probabilis-tic Games. arXiv:1410.5387 [cs] , October 2014. arXiv: 1410.5387.[24] Valerio Turri, Adriano Carvalho, H Eric Tseng, Karl H Johansson, andFrancesco Borrelli. Linear model predictive control for lane keepingand obstacle avoidance on low curvature roads. In
Intelligent Trans-portation Systems-(ITSC), 2013 16th International IEEE Conferenceon , pages 378–383. IEEE, 2013.[25] T. Wongpiromsarn, U. Topcu, and R.M. Murray. Receding HorizonTemporal Logic Planning.
IEEE Transactions on Automatic Control ,57(11):2817–2830, November 2012.[26] Zhengyuan Zhou, Ryo Takei, Haomiao Huang, and Claire J Tomlin.A general, open-loop formulation for reach-avoid games. In