Automated Termination Analysis of Polynomial Probabilistic Programs
Marcel Moosbrugger, Ezio Bartocci, Joost-Pieter Katoen, Laura Kovács
aa r X i v : . [ c s . P L ] O c t Automated TerminationAnalysis of Polynomial Probabilistic Programs ⋆ Marcel Moosbrugger , Ezio Bartocci , Joost-Pieter Katoen , and Laura Kovács TU Wien, Austria RWTH Aachen University, Germany
Abstract.
The termination behavior of probabilistic programs depends on the out-comes of random assignments. Almost sure termination (AST) is concerned withthe question whether a program terminates with probability one on all possibleinputs. Positive almost sure termination (PAST) focuses on termination in a fi-nite expected number of steps. This paper presents a fully automated approachto the termination analysis of probabilistic while-programs whose guards and ex-pressions are polynomial expressions. As proving (positive) AST is undecidable ingeneral, existing proof rules typically provide sufficient conditions. These condi-tions mostly involve constraints on supermartingales. We consider four proof rulesfrom the literature and extend these with generalizations of existing proof rulesfor (P)AST. We automate the resulting set of proof rules by effectively computingasymptotic bounds on polynomials over the program variables. These bounds areused to decide the sufficient conditions – including the constraints on supermartin-gales – of a proof rule. Our software tool A
MBER can thus check AST, PAST,as well as their negations for a large class of polynomial probabilistic programs,while carrying out the termination reasoning fully with polynomial witnesses. Ex-perimental results show the merits of our generalized proof rules and demonstratethat A
MBER can handle probabilistic programs that are out of reach for other state-of-the-art tools.
Keywords:
Probabilistic Programming · Almost sure Termination · Martingales ·Asymptotic Bounds · C-finite Recurrences
Classical program termination.
Termination is a key property in program analysis [15].The question whether a program terminates on all possible inputs – the universal haltingproblem – is undecidable. Proof rules based on ranking functions have been developedthat impose sufficient conditions implying (non-)termination. Automated terminationchecking has given rise to powerful software tools such as AProVE [20] and NaTT [42](using term rewriting), and UltimateAutomizer [25] (using automata theory). These toolshave shown to be able to determine the termination of several intricate programs. The ⋆ This research was supported by the WWTF ICT19-018 grant ProbInG, the ERC Starting GrantSYMCAR 639270, the ERC Advanced Grant FRAPPANT 787914, and the Austrian FWFproject W1255-N23. Joost-Pieter Katoen is financially supported by the ERC AdG GrantFRAPPANT (787914). M. Moosbrugger et al. x := 10 while x > do x := x +1 [ / ] x − end (a) x := 10 while x > do x := x − / ] x +2 end (b) x := 0, y := 0 while x + y < do x := x +1 [ / ] x − y := y + x [ / ] y − x end (c) x := 10, y := 0 while x > do y := y +1 x := x +4 y [ / ] x − y end (d) Fig. 1: Examples of probabilistic programs in our probabilistic language. Program 1a is asymmetric 1D random walk. The program is almost surely terminating (AST) but not positivelyalmost surely terminating (PAST). Program 1b is not AST. Programs 1c and 1d contain dependentvariable updates with polynomial guards and both programs are PAST. industrial tool Terminator [14] has taken termination proving into practice and is ableto prove termination – or even more general liveness properties – of e.g., device driversoftware. Rather than seeking a single ranking function, it takes a disjunctive terminationargument using sets of ranking functions. Other results include termination proving meth-ods for specific program classes such as linear and polynomial programs, see, e.g., [8,23].
Termination of probabilistic program.
Probabilistic programs extend sequential pro-grams with the ability to draw samples from probability distributions. They are usede.g. for, encoding randomized algorithms, planning in AI, security mechanisms, and incognitive science. In this paper, we consider probabilistic while-programs with discreteprobabilistic choices, in the vein of the seminal works [33] and [36]. Termination ofprobabilistic programs differs from the classical halting problem in several respects, e.g.,probabilistic programs may exhibit diverging runs that have probability mass zero in total.Such programs do not always terminate, but terminate with probability one – they almostsurely terminate. An example of such a program is given in Figure 1a where variable x isincremented by with probability / , and otherwise decremented with this amount. Thisprogram encodes a one-dimensional (1D) left-bounded random walk starting at position . Another important difference to classical termination is that the expected number ofprogram steps until termination may be infinite, even if the program almost surely termi-nates. Thus, almost sure termination (AST) does not imply that the expected number ofsteps until termination is finite. Programs that have a finite expected runtime are referredto as positively almost surely terminating (PAST). Figure 1c is a sample program that isPAST. While PAST implies AST, the converse does not hold, as evidenced by Figure 1a:the program of Figure 1a terminates with probability one but needs infinitely many stepson average to reach x =0 , hence is not PAST. (The terminology AST and PAST wascoined in [7] and has its roots in the theory of Markov processes.) Proof rules for AST and PAST.
Proving termination of probabilistic programs is hard:AST for a single input is as hard as the universal halting problem, whereas PAST is evenharder [29]. Termination analysis of probabilistic programs is currently attracting quitesome attention. It is not just of theoretical interest. For instance, a popular way to analyzeprobabilistic programs in machine learning is by using some advanced form of simulation.If, however, a program is not PAST, the simulation may take forever. In addition, the useof probabilistic programs in safety-critical environments [2,6,19] necessitates providingformal guarantees on termination. Different techniques are considered for probabilisticprogram termination ranging from probabilistic term rewriting [3], sized types [16], and utomated Termination Analysis of Polynomial Probabilistic Programs 3
Büchi automata theory [13], to weakest pre-condition calculi for checking PAST [30]. Alarge body of works considers proof rules that provide sufficient conditions for provingAST, PAST, or their negations. These rules are based on martingale theory, in particu-lar supermartingales. They are stochastic processes that can be (phrased in a simplifiedmanner) viewed as the probabilistic analog of ranking functions: the value of a randomvariable represents the “value” of the function at the beginning of a loop iteration. Succes-sive random variables model the evolution of the program loop. Being a supermartingalemeans that the expected value of the random variables at the end of a loop does not ex-ceed its value at the start of the loop. Constraints on supermartingales form the essentialpart of proof rules. For example, the AST proof rule in [37] requires the existence of asupermartingale whose value decreases at least with a certain amount by at least a certainprobability on each loop iteration. Intuitively speaking, the closer the supermartingalescomes to zero – indicating termination – the more probable it is that it increases more.The AST proof rule in [37] is applicable to prove AST for the program in Figure 1a; yet,it cannot be used to prove PAST of Figures 1c-1d. On the other hand, the PAST proofrule in [9,18] requires that the expected decrease of the supermartingale on each loopiteration is at least some positive constant ǫ and on loop termination needs to be at mostzero – very similar to the usual constraint on ranking functions. While [9,18] can be usedto prove the program in Figure 1c to be PAST, these works cannot be used for Figure 1a.They cannot be used for proving Figure 1d to be PAST either. The rule for showing non-AST [12] requires the supermartingale to be repulsing. This intuitively means that thesupermartingale decreases on average with at least ε and is positive on termination. Fig-uratively speaking, it repulses terminating states. It can be used to prove the program inFigure 1b to be not AST. In summary, while existing works for proving AST, PAST, andtheir negations are generic in nature, they are also restricted for classes of probabilisticprograms. In this paper, we propose relaxed versions of existing proof rules for probabilis-tic termination that turn out to treat quite a number of programs that could not be provenotherwise (Section 4).
In particular, (non-)termination of all four programs of Figure 1can be proven using our proof rules.
Automated termination checking of AST and PAST.
Whereas there is a large body oftechniques and proof rules, software tool support to automate checking termination ofprobabilistic programs is still in its infancy.
This paper presents novel algorithms to au-tomate various proof rules for probabilistic programs: the three aforementioned proofrules [9,18,37,12] and a variant of the non-AST proof rule to prove non-PAST [12] . Wealso present relaxed versions of each of the proof rules, going beyond the state-of-the-art in the termination analysis of probabilistic programs. We focus on so-called Prob-solvable loops, extending [4]. Namely, we define Prob-solvable loops as probabilisticwhile-programs whose guards compare two polynomials (over program variables) andwhose body is a sequence of random assignments with polynomials as right-hand sidesuch that a variable x , say, only depends on variables preceding x in the loop body. Whilerestrictive, Prob-solvable loops cover a vast set of interesting probabilistic programs (seeRemark 1). An essential property of our programs is that the statistical moments of pro- For automation, the proof rule of [37] is considered for constant decrease and probabilityfunctions. M. Moosbrugger et al. gram variables can be obtained as closed-form formulas [4].
The key of our algorithmicapproach is a procedure for computing asymptotic lower, upper and absolute boundson polynomial expressions over program variables in our programs (Section 5).
This en-ables a novel method for automating probabilistic termination and non-termination proofrules based on (super)martingales, going beyond the state-of-the-art in probabilistic ter-mination. Our relaxed proof rules allow us to fully automate (P)AST analysis by usingonly polynomial witnesses. Our experiments provide practical evidence that polynomialwitnesses within Prob-solvable loops are sufficient to certify most examples from theliterature and even beyond (Section 6). Providing necessary conditions for (P)AST withProb-solvable loops is an interesting line for further work.
Our probabilistic termination framework A MBER . We have implemented our algorith-mic approach in the publicly available tool A
MBER . It exploits asymptotic bounds overpolynomial martingales and uses the tool
MORA [4] for computing the first-order mo-ments of program variables and the computer algebra system package diofant . It em-ploys over- as well as under-approximations realized by a simple static analysis. A
MBER establishes probabilistic termination in a fully automated manner and has the followingunique characteristics: – it includes the first implementation of the AST proof rule of [37], – it is the first tool capable of certifying AST for programs that are not PAST, and – it is the first tool that brings the various proof rules under a single umbrella: AST,PAST, non-AST and non-PAST.An experimental evaluation on various benchmarks shows that: (1) A MBER is superior toexisting tools for automating PAST [40] and AST [9], (2) the relaxed proof rules enableproving substantially more programs, and (3) A
MBER is able to automate the terminationchecking of intricate probabilistic programs (within the class of programs considered)that could not be automatically handled so far (Section 6). For example, A
MBER solves23 termination benchmarks that no other automated approach could so far handle.Main contributions.
To summarize, the main contributions of this paper are:1. Relaxed proof rules for (non-)termination, enabling treating a wider class of programs(Section 4).2. Efficient algorithms to compute asymptotic bounds on polynomial expressions of pro-gram variables (Section 5).3. Automation: a realisation of our algorithms in the tool A
MBER (Section 6).4. Experiments showing the superiority of A
MBER over existing tools for proving (P)AST(Section 6).
We denote by N and R the set of natural and real numbers, respectively. Further, let R denote R ∪{ + ∞ , −∞} , R +0 the non-negative reals and R [ x ,...,x m ] the polynomial ringin x ,...,x m over R . We write x := E (1) [ p ] E (2) [ p ] ... [ p m − ] E ( m ) for the probabilisticupdate of program variable x , denoting the execution of x := E ( j ) with probability p j ,for j = 1 ,...,m − , and the execution of x := E ( m ) with probability − P m − j =1 p j , where utomated Termination Analysis of Polynomial Probabilistic Programs 5 m ∈ N . We write indices of expressions over program variables in round brackets and use E i for the stochastic process induced by a single expression E . This section introducesour programming language extending the class of so-called Prob-solvable loops [4] anddefines the probability space introduced by such programs. We assume the reader to befamiliar with probability theory [32].
Prob-solvable loops [4] are syntactically restricted probabilistic programs with polyno-mial expressions over program variables. The statistical higher-order moments of pro-gram variables, like expectation and variance of such loops, can always be computed asfunctions of the loop counter. In this paper, we extend Prob-solvable loops with polyno-mial loop guards in order to study their termination behavior, as follows.
Definition 1 (Prob-solvable loop L ). A Prob-solvable loop L with real-valued variables x (1) ,...,x ( m ) , where m ∈ N , is a program of the form: I L while G L do U L end , with – (Init) I L is a sequence x (1) := r (1) ,...,x ( m ) := r ( m ) of m assignments, with r ( j ) ∈ R – (Guard) G L is a strict inequality P > Q , where
P, Q ∈ R [ x (1) ,...,x ( m ) ] – (Update) U L is a sequence of m probabilistic updates of the form x ( j ) := a ( j x ( j ) + P ( j [ p j ] a ( j x ( j ) + P ( j [ p j ] ... [ p j ( l j − ] a ( jl j ) x ( j ) + P ( jl j ) , where a ( jk ) ∈ R +0 are constants, P ( jk ) ∈ R [ x (1) ,...,x ( j − ] are polynomials, p ( jk ) ∈ [0 , and P k p jk < . If L is clear from the context, the subscript L is omitted from I L , G L , and U L . Figure 1gives four example Prob-solvable loops. Remark 1 (Prob-solvable expressiveness).
The enforced order of assignments in the loopbody of Prob-solvable loops seems restrictive. Notwithstanding these syntactic restric-tions, many non-trivial probabilistic programs can be naturally modeled as succinct Prob-solvable loops. These include complex stochastic processes such as 2D random walksand dynamic Bayesian networks [5]. Almost all existing benchmarks on automated prob-abilistic termination analysis fall within the scope of Prob-solvable loops (cf. Section 6).In the sequel, we consider an arbitrary Prob-solvable loop L and provide all defini-tions relative to L . The semantics of L is defined next, by associating L with a probabilityspace. A probabilistic program, and thus a Prob-solvable loop, can be semantically described asa probabilistic transition system [9] or as a probabilistic control flow graph [12], whichin turn induce an infinite Markov chain (MC) . An MC is associated with a sequencespace [32], a special probability space. In the sequel, we associate L with the sequencespace of its corresponding MC, similarly as in [24]. To this end, we first define the notions state and run for a Prob-solvable loop. In fact, [12] consider Markov decision processes, but in absence of non-determinism inProb-solvable loops, Markov chains suffice for our purpose. M. Moosbrugger et al.
Definition 2 (State, Run of L ). The state of a Prob-solvable loop is a vector s ∈ R m ,where m is the number of variables in L . Let s [ j ] or s [ x ( j ) ] denote the j -th componentof the vector s representing the value of the variable x ( j ) in state s . A run ϑ of L is aninfinite sequence of states. Note that any infinite sequence of states is a run. Infeasible runs will however be assignedmeasure . We write s (cid:15) B to denote that the logical formula B holds in state s . A probabil-ity space ( Ω,Σ, P ) consists of a measurable space ( Ω,Σ ) and a probability measure P forthis space. First, we define a measurable space for L and later equip it with a probabilitymeasure. Definition 3 (Loop Space of L ). The Prob-solvable loop L induces a canonical measur-able space ( Ω L ,Σ L ) , called loop space , where – the sample space Ω L := ( R m ) ω is the set of all program runs, – the σ -algebra Σ L is the smallest σ -algebra containing all cylinder sets Cyl ( π ) := { πϑ | ϑ ∈ ( R m ) ω } for all finite prefixes π ∈ ( R m ) + , that is Σ L := h{ Cyl ( π ) | π ∈ ( R m ) + }i σ . To turn the loop space of L into a proper probability space, we introduce a probabilitymeasure. To this end, we define the probability p ( π ) of a finite non-empty prefix π ofa program run. Let µ I ( s ) denote the probability that, after initialization I L , the loop L is in state s . Because probabilistic constructs are not allowed in I L , µ I ( s ) is a Dirac-distribution, such that µ I ( s ) = 1 for the unique state s defined by I L and µ I ( s ′ ) = 0 for s ′ = s . Moreover, µ U ( s,s ′ ) denotes the probability that, after one loop iteration startingin state s , the resulting program state is s ′ . Note that µ I ( s ) and µ U ( s,s ′ ) are solely deter-mined by I L and U L . The probability p ( π ) of a finite non-empty prefix π of a programrun is then defined as p ( s ) := µ I ( s ) , p ( πss ′ ) := ( p ( πs ) · [ s ′ = s ] , if s (cid:15) ¬G L p ( πs ) · µ U ( s,s ′ ) , if s (cid:15) G L where [ ... ] denote the Iverson brackets, i.e. [ s ′ = s ] is iff s ′ = s . Intuitively, p ( π ) is theprobability that prefix π is the sequence of the first | π | program states when executing L .We note that the effect of the loop body U is considered as atomic. Definition 4 (Loop Measure of L ). The loop measure of a Prob-solvable loop L is acanonical probability measure P L : Σ L → [0 , on the loop space of L , with P L ( Cyl ( π )) := p ( π ) . The loop space and the loop measure of L form the probability space ( Ω L ,Σ L , P L ) . In order to formalize termination properties of a Prob-solvable loop L , we define the looping time of L to be a random variable in L ’s loop space. A random variable X ina probability space ( Ω, Σ, P ) is a ( Σ -)measurable function X : Ω → R , i.e. for everyopen interval U ⊆ R it holds that X − ( U ) ∈ Σ . The expected value of a random vari-able X , denoted by E ( X ) , is defined as the Lebesgue integral of X over the probability utomated Termination Analysis of Polynomial Probabilistic Programs 7 space, i.e. E ( X ) := R Ω Xd P . In the special case that X takes only countably many values,we have E ( X ) = R Ω Xd P = P r ∈ X ( Ω ) P ( X = r ) · r . We now define the looping time of aProb-solvable loop L , as follows. Definition 5 (Looping Time of L ). The looping time of L is the random variable T ¬G : Ω → N ∪{∞} , where T ¬G ( ϑ ) := inf { i ∈ N | ϑ i (cid:15) ¬G} . Intuitively, the looping time T ¬G maps a program run of L to the index of the first statefalsifying the loop guard G of L or to ∞ if no such state exists. We now formalize termi-nation properties of L using the looping time T ¬G . Definition 6 (Termination of L ). The Prob-solvable loop L is AST if P ( T ¬G < ∞ ) = 1 . L is PAST if E ( T ¬G ) < ∞ . For a thorough analysis of the hardness of deciding AST and PAST we refer to [29].While for arbitrary probabilistic programs, answering P ( T ¬G < ∞ ) and E ( T ¬G < ∞ ) is undecidable, sufficient conditions for AST, PAST and their negations have been de-veloped [9,18,37,12]. These works use (super)martingales which are special stochasticprocesses. In this section, we adopt the general setting of martingale theory to a Prob-solvable loop L and then formalize sufficient termination conditions for L in Section 3. Definition 7 (Stochastic Process of L ). A stochastic process ( X i ) i ∈ N is a sequence ofrandom variables. Every arithmetic expression E over the program variables of L in-duces the stochastic process ( E i ) i ∈ N , E i : Ω → R with E i ( ϑ ) := E ( ϑ i ) . For a run ϑ of L , E i ( ϑ ) is the evaluation of E in the i -th state of ϑ . In the sequel, for a boolean condition B over program variables x of L , we write B i torefer to the result of substituting x by x i in B . In Figure 1a, the stochastic process ( x i ) i ∈ N is such that every x i maps a given program run ϑ to the value of the variable x in the i -th state of ϑ . Note that the σ -algebra Σ L contains the cylinder sets for finite programrun prefixes of arbitrary length. This does not capture the gradual information gain whenexecuting L iteration by iteration. In probability theory, filtrations are a standard notionto formalize the information available at a specific point in time. Definition 8 (Filtration [32]).
For a probability space ( Ω, Σ, P ) , a filtration is a se-quence ( F i ) i ∈ N such that (1) every F i is a sub- σ -algebra and (2) F i ⊆ F i +1 . Further, ( Ω,Σ, ( F i ) i ∈ N , P ) is called a filtered probability space . We adopt filtrations to Prob-solvable loops and enrich the loop space of L to a filteredprobability space, as follows. Definition 9 (Loop Filtration of L ). The loop filtration ( F L i ) i ∈ N of Σ L is defined by F L i = h{ Cyl ( π ) | π ∈ ( R m ) + , | π | = i +1 }i σ . ( Ω L , Σ L , ( F L ) i ∈ N , P L ) is a filteredprobability space of L . M. Moosbrugger et al.
Based on Definition 9, note that F L is the smallest σ -algebra containing the cylindersets of finite prefixes of program runs of length . That is, the cylinder sets of finite pre-fixes of program runs of length greater than or equal to are not present in F L . Hence, F L captures exactly the information available about the program run after executing justthe initialization I L . Similarly, F L i captures the information about the program run af-ter the loop body U L has been executed i times. In Figure 1a, for example, the event { ϑ ∈ Ω | x i ( ϑ ) = r } denoted by { x i = r } is F L i -measurable for every i ∈ N and every r ∈ R , as the value of x i depends only on information available up to the i -th iteration ofthe loop body of Figure 1a. The following definition formalizes this observation. Definition 10 (Adapted Process [32]).
A stochastic process ( X i ) i ∈ N is said to be adapted to a filtration ( F i ) i ∈ N if X i is F i -measurable for every i ∈ N . It is not hard to argue that, for any arithmetic expression E over the variables of L , theinduced stochastic process ( E i ) i ∈ N is adapted to the loop filtration F L i of L : the value of E i only depends on the information available up to the i -th loop iteration of L .The concept of (super)martingales builds upon the notion of conditional expected val-ues which is defined as follows. Definition 11 (Conditional Expected Value [32]).
For a probability space ( Ω, Σ, P ) ,an integrable random variable X and a sub- σ -algebra ∆ ⊆ Σ , the expected value of X conditioned on ∆ , E ( X | ∆ ) , is any ∆ -measurable function such that for every D ∈ ∆ wehave R D E ( X | ∆ ) d P = R D Xd P . The random variable E ( X | ∆ ) is almost surely unique. We now introduce (super)martingales as special stochastic processes. In Section 3 thesenotions are used to define sufficient conditions for PAST, AST and their negations.
Definition 12 (Martingales).
Let ( Ω,Σ, ( F i ) i ∈ N , P ) be a filtered probability space and ( M i ) i ∈ N be an integrable stochastic process adapted to ( F i ) i ∈ N . Then ( M i ) i ∈ N is a martingale if E ( M i +1 | F i ) = M i (or equivalently E ( M i +1 − M i | F i ) = 0 ). More-over, ( M i ) i ∈ N is called a supermartingale (SM) if E ( M i +1 | F i ) ≤ M i (or equivalently E ( M i +1 − M i | F i ) ≤ ). For an arithmetic expression E over the program variables of L ,the conditional expected value E ( E i +1 − E i | F i ) is called the martingale expression of E . While AST and PAST are undecidable in general [29], sufficient conditions, called proofrules , for AST and PAST have been introduced, see e.g. [9,18,37,12]. In this section, wesurvey four proof rules, adapted to Prob-solvable loops. In the sequel, a pure invariant isa loop invariant in the classical deterministic sense [26]. Based on the probability spacecorresponding to L , a pure invariant holds before and after every iteration of L . The proof rule for PAST introduced in [9] relies on the notion of ranking supermartin-gales (RSMs), which is a SM that decreases by a fixed positive ǫ on average at every loopiteration. Intuitively, RSMs resemble ranking functions for deterministic programs, yetfor probabilistic programs. utomated Termination Analysis of Polynomial Probabilistic Programs 9 Theorem 1 (Ranking-Supermartingale-Rule (RSM-Rule) [9], [18]).
Let M : R m → R be an expression over the program variables of L and I a pure invariant of L . Assumethe following conditions hold for all i ∈ N :1. (Termination) G ∧ I = ⇒ M >
2. (RSM Condition) G i ∧ I i = ⇒ E ( M i +1 − M i | F i ) ≤ − ǫ , for some ǫ > .Then, L is PAST. Further, M is called an ǫ -ranking supermartingale .Example 1. Consider Figure 1c, set M := 100 − x − y and ǫ := 2 and let I be true . Con-dition (1) of Theorem 1 trivially holds. Further, M is also an ǫ -ranking supermartingale,as E ( M i +1 − M i | F i ) = 100 − E ( x i +1 | F i ) − E ( y i +1 | F i ) − x i + y i = − − x i ≤− . That is because E ( x i +1 | F i ) = x i + 1 and E ( y i +1 | F i ) = y i + x i + 1 . Figure 1c isthus proved PAST using the RSM-Rule. Recall that Figure 1a is AST but not PAST, and hence the RSM-rule cannot be used forFigure 1a. By relaxing the ranking conditions, the proof rule in [37] uses general super-martingales to prove AST of programs that are not necessarily PAST.
Theorem 2 (Supermartingale-Rule (SM-Rule) [37]).
Let M : R m → R ≥ be an ex-pression over the program variables of L and I a pure invariant of L . Let p : R ≥ → (0 , (for probability ) and d : R ≥ → R > (for decrease ) be antitone (i.e. monotonically de-creasing) functions. Assume the following conditions hold for all i ∈ N :1. (Termination) G ∧ I = ⇒ M >
2. (Decrease) G i ∧ I i = ⇒ P ( M i +1 − M i ≤ − d ( M i ) | F i ) ≥ p ( M i )
3. (SM Condition) G i ∧ I i = ⇒ E ( M i +1 − M i | F i ) ≤ .Then, L is AST. Intuitively, the requirement of d and p being antitone forbids that the “execution progress”of L towards termination becomes infinitely small while still being positive. Example 2.
The SM-Rule can be used to prove AST for Figure 1a. Consider M := x , p := / , d := 1 and I := true . Clearly, p and d are antitone. The remaining conditions ofTheorem 2 also hold as (1) x > ⇒ x > ; (2) x decreases by d with probability p inevery iteration; and (3) E ( M i +1 − M i | F i ) = x i − x i ≤ . While Theorems 1 and 2 can be used for proving AST and PAST, respectively, they arenot applicable to the analysis of non-terminating Prob-solvable loops. Two sufficient con-ditions for certifying the negations of AST and PAST have been introduced in [12] usingso-called repulsing-supermartingales . Intuitively, a repulsing-supermartingale M on av-erage decreases in every iteration of L and on termination is non-negative. Figuratively, M repulses terminating states. Theorem 3 (Repulsing-AST-Rule (R-AST-Rule) [12]).
Let M : R m → R be an expres-sion over the program variables of L and I a pure invariant of L . Assume the followingconditions hold for all i ∈ N :
1. (Negative) M <
2. (Non-Termination) ¬G ∧ I = ⇒ M ≥
3. (RSM Condition) G i ∧ I i = ⇒ E ( M i +1 − M i | F i ) ≤ − ǫ , for some ǫ >
4. ( c -Bounded Differences) | M i +1 − M i | < c , for some c > .Then, L is not AST. M is called an ǫ -repulsing supermartingale with c -bounded differ-ences .Example 3. Consider Figure 1b and let M := − x , c := 3 , ǫ := / and I := true . All fourabove conditions hold: (1) − x = − < ; (2) x ≤ ⇒ − x ≥ ; (3) E ( M i +1 − M i |F i ) = − x i − / + x i = − / ≤ − ǫ ; and (4) | x i − x i +1 | < . Thus, Figure 1b is not AST.While Theorem 3 can prove programs not to be AST, and thus also not PAST, it cannotbe used to prove programs not to be PAST when they are AST. For example, Theorem 3cannot be used to prove that Figure 1a is not PAST. To address such cases, a variation ofthe R-AST-Rule [12] for certifying programs not to be PAST arises by relaxing the condi-tion ǫ > of the R-AST-Rule to ǫ ≥ . We refer to this variation by Repulsing-PAST-Rule(R-PAST-Rule) . Example 4.
Consider Figure 1a. We set M := − x , c := 1 and ǫ := 0 . Note that E ( M i +1 − M i | F i ) = − x i + x i ≤ and it is easy to see that all four conditions of Theorem 3 hold(with ǫ ≥ ). Thus, the R-PAST-Rule proves that Figure 1a is not PAST. While Theorems 1-3 provide sufficient conditions proving PAST, AST and their nega-tions, the applicability to Prob-solvable loops is somewhat restricted. For example, theRSM-Rule cannot be used to prove Figure 1d to be PAST using the simple expression M := x , as explained in detail with Example 5, but may require more complex witnessesfor certifying PAST, complicating automation. In this section, we relax the conditions ofTheorems 1-3 by requiring these conditions to only hold “eventually”. A property P ( i ) parameterized by a natural number i ∈ N holds eventually if there is an i ∈ N such that P ( i ) holds for all i ≥ i . Our relaxations of probabilistic termination proof rules can intu-itively be described as follows: If L , after a fixed number of steps, almost surely reaches astate from which the program is PAST or AST, then the program is PAST or AST, respec-tively. Let us first illustrate the benefits of reasoning with “eventually” holding propertiesfor probabilistic termination in the following example. x := x , y := 0 while x > do y := y +1 x := x +( y −
5) [ / ] x − ( y − end (a) x := , y := 2 while x > do y := / · yx := x +1 − y [ / ] x − y end (b) Fig. 2: Prob-solvable loops which require our relaxed proof rules for termination analysis.utomated Termination Analysis of Polynomial Probabilistic Programs 11
Example 5 (Limits of the RSM-Rule and SM-Rule).
Consider Figure 1d. Setting M := x ,we have the martingale expression E ( M i +1 − M i | F i ) = − y i / + y i + / = − i / + i + / .Since E ( x i +1 − x i | F i ) is non-negative for i ∈ { , , , } , we conclude that M is not anRSM. However, Figure 1d either terminates within the first three iterations or, after threeloop iterations, is in a state such that the RSM-Rule is applicable. Therefore, Figure 1d isPAST but the RSM-Rule cannot directly prove using M := x . A similar restriction of theSM-Rule can be observed for Figure 2a. By considering M := x , we derive the martingaleexpression E ( x i +1 − x i | F i ) = 0 , implying that M is a martingale for Figure 2a. However,the decrease function d for the SM-Rule cannot be defined because, for example, in thefifth loop iteration of Figure 2a, there is no progress as x is almost surely updated with itsprevious value. However, after the fifth iteration of Figure 2a, x always decreases by atleast with probability / and all conditions of the SM-Rule are satisfied after the fifthloop iteration. Thus, Figure 2a either terminates within the first five iterations or reachesa state from which it terminates almost surely. Consequently, Figure 2a is AST but theSM-Rule cannot directly prove it using M := x .We relax the RSM-Rule and SM-Rule of Theorems 1 and 2 as follows. Theorem 4 (Relaxed Termination Proof Rules).
For the RSM-Rule to certify PAST of L , it is sufficient that conditions (1) - (2) of Theorem 1 hold eventually (instead of for all i ∈ N ). Similarly, for the SM-Rule to certify AST of L , it is sufficient that conditions (1) - (3) of Theorem 2 hold eventually.Proof. We prove the relaxation of the RSM-Rule. The proof of the relaxed SM-Rule isanalogous. Let L := I while G do U end be as in Definition 1. Assume L satisfies the con-ditions (1)-(2) of Theorem 1 after some i ∈ N . We construct the following probabilisticprogram P , where i is a new variable not appearing in L : I ; i := 0 while i < i do U ; i := i +1 endwhile G do U end (1)We first argue that if P is PAST, then so is L . Assume P to be PAST. Then, the loopingtime of L is either bounded by i or it is PAST, by the definition of P . In both cases, L is PAST. Finally, observe that P is PAST if and only if its second while-loop is PAST.However, the second while-loop of P can be certified to be PAST using the RSM-Ruleand additionally using i ≥ i as an invariant. ⊓⊔ Remark 2.
The central point of our proof rule relaxations is that they allow for simplerwitnesses. While for Example 5 it can be checked that M := x + 2 y +5 is an RSM, theexample illustrates that the relaxed proof rule allows for a much simpler PAST witness(linear instead of exponential). This simplicity is key for automation.Similar to Theorem 4, we relax the R-AST-Rule and the R-PAST-Rule. However, com-pared to Theorem 4, it is not enough for a non-termination proof rule to certify non-ASTfrom some state onward, because L may never reach this state as it might terminate ear-lier. Therefore, a necessary assumption when relaxing non-termination proof rules comeswith ensuring that L has a positive probability of reaching the state after which a proofrule witnesses non-termination. This is illustrated in the following example . Example 6 (Limits of the R-AST-Rule).
Consider Figure 2b and set M := − x . As a result,we get E ( M i +1 − M i | F i ) = y i / − / = − i / − / . Thus, E ( M i +1 − M i | F i ) = 0 for i = 0 , implying that M cannot be an ǫ -repulsing supermartingale with ǫ > for all i ∈ N .However, after the first iteration of L , M satisfies all requirements of the R-AST-Rule.Moreover, L always reaches the second iteration because in the first iteration x almostsurely does not change. From this follows that Figure 2b is not AST.The following theorem formalizes the observation of Example 6 relaxing the R-AST-Rule and R-PAST-Rule of Theorem 3. Theorem 5 (Relaxed Non-Termination Proof Rules for).
For the R-AST-Rule to cer-tify non-AST for L (Theorem 3), as well as for the R-PAST-Rule to certify non-PAST for L (Theorem 3), if P ( M i < > for some i ≥ , it suffices that conditions (2) - (4) holdfor all i ≥ i (instead of for all i ∈ N ).Proof. We prove the relaxation of the R-AST-Rule. The proof for the R-PAST-Rule isanalogous. Let L := I while G do U end be as in Definition 1. Assume L satisfies con-ditions (2)-(4) of the R-AST-Rule for all i ≥ i for some fixed i ∈ N . Moreover, assume P ( M i < > .We construct again a probabilistic program P as in (1). Observe that for the secondwhile-loop of P , we have i ≥ i . By assumption, the second while-loop of P satisfiesconditions (2)-(4) of the R-AST-Rule. By the R-AST-Rule, we conclude P being not AST,if there is a Cyl ( π ) ∈ F P i , such that P P ( Cyl ( π )) > and M i ( ϑ ) < for all ϑ ∈ Cyl ( π ) .By the definition of P , it then follows for L that if there is a Cyl ( π ) ∈ F L i , suchthat P L ( Cyl ( π )) > and M i ( ϑ ) < for all ϑ ∈ Cyl ( π ) , then L is not AST. As P L ( M i < > , we conclude that such a Cyl ( π ) exists and derive that L is not AST. ⊓⊔ Note that for a repulsing supermartingale M , the condition P ( M i < > implies thatthere is a positive probability of reaching iteration i , because M would have to be almostsurely non-negative upon termination.In what follows, whenever we write RSM-Rule, SM-Rule, R-AST-Rule or R-PAST-Rule we refer to our relaxed versions of the proof rules. The two major challenges when automating reasoning with the proof rules of Sections 3and 4 are (i) constructing expressions M over the program variables and (ii) proving in-equalities involving E ( M i +1 − M i | F i ) . In this section, we address these two challengesfor Prob-solvable loops. For the loop guard G L = P > Q , let G L denote the polynomial P − Q . As before, if L is clear from the context, we omit the subscript L . It holds that G > is equivalent to G . (i) Constructing (super)martingales M : For a Prob-solvable loop L , the polynomial G is a natural candidate for the expression M in termination proof rules (RSM-Rule, SM-Rule) and − G for the expression M in the non-termination proof rules (R-AST-Rule,R-PAST-Rule). Hence, we construct potential (super)martingales M by setting M := G utomated Termination Analysis of Polynomial Probabilistic Programs 13 for the RSM-Rule and the SM-Rule, and M := − G for the R-AST-Rule and the R-PAST-Rule. The property G = ⇒ G > , a condition of the RSM-Rule and the SM-Rule, triviallyholds. Moreover, for the R-AST-Rule and R-PAST-Rule the condition ¬G = ⇒ − G ≥ is satisfied. The remaining conditions of the proof rules are: – RSM-Rule: (a) G i = ⇒ E ( G i +1 − G i | F i ) ≤ − ǫ for some ǫ > – SM-Rule: (a) G i = ⇒ E ( G i +1 − G i | F i ) ≤ and (b) G i = ⇒ P ( G i +1 − G i ≤ − d | F i ) ≥ p for some p ∈ (0 , and d ∈ R + (for the purpose of efficient automation, we restrictthe functions d ( r ) and p ( r ) to be constant) – R-AST-Rule: (a) G i = ⇒ E ( − G i +1 + G i | F i ) ≤ − ǫ for some ǫ > and (b) | G i +1 − G i | ≤ c , for some c > .All non-trivial conditions above express bounds over G i . Choosing G as the potentialwitness may seem simplistic. However, Example 5 already illustrated how our relaxedproof rules can mitigate the need for more complex witnesses (even exponential ones).The computational effort in our approach does not lie in synthesizing a complex witnessbut in constructing asymptotic bounds for the loop guard. Our approach can therefore beseen as complementary to approaches synthesizing more complex witnesses [9,10,12].The martingale expression E ( G i +1 − G i | F i ) is an expression over program variables,whereas G i +1 − G i cannot be interpreted as a single expression but through a distributionof expressions. Definition 13 (One-step Distribution).
For expression H over the program vari-ables of Prob-solvable loop L , let the one-step distribution U H L be the distribution E P ( H i +1 = E | F i ) with support set supp ( U H L ) := { B | U H L ( B ) > } . We refer toexpressions B ∈ supp ( U H L ) by branches of H . The notation U H L is chosen to suggest that the loop body U L is “applied” to the expres-sion H , leading to a distribution of expressions. Intuitively, the support supp ( U H L ) of anexpression H contains all possible updates of H after executing a single iteration. Example 7 (One-step Distribution).
Consider the following Prob-solvable loop: x := 1, y := 1 while x > do y := y +1 [ / ] y +2 x := x + y [ / ] x − y end For the expression H := x , the one-step distribution U H L is as follows: ExpressionE U H L ( E ) x i +2 xiyi +2 xi + y i +2 yi +1 1 / x i +2 xiyi +4 xi + y i +4 yi +4 1 / x i − xiyi − xi + y i +2 yi +1 1 / x i − xiyi − xi + y i +4 yi +4 1 / Anyother E The first entry in the table can be derived like: x i +1 = ( x i + y i +1 ) = x i +2 x i y i +1 + y i +1 (with probability / ) = x i +2 x i ( y i +1)+( y i +1) (with probability / · / ) = x i +2 x i y i +2 x i + y i +2 y i +1 (with probability / ) (ii) Proving inequalities involving E ( M i +1 − M i | F i ) : To automate the terminationanalysis of L with the proof rules from Section 3, we need to compute bounds for the ex-pression E ( G i +1 − G i | F i ) as well as for the branches of G . In addition, our relaxed proofrules from Section 4 only need asymptotic bounds, i.e. bounds which hold eventually. InSection 5.2, we propose Algorithm 1 for computing asymptotic lower and upper bounds for any polynomial expression over program variables of L . Our procedure allows usto derive bounds for E ( G i +1 − G i | F i ) and the branches of G . Before formalizing ourmethod, let us first illustrate how reasoning with asymptotic bounds helps to apply termi-nation proof rules to L . Example 8 (Asymptotic Bounds for the RSM-Rule).
Consider the following program: x := 1, y := 0 while x < do y := y +1 x := x + y [ / ] / · x end Observe y i = i . The martingale expression for G = 100 − x is E ( G i +1 − G i | F i ) = / (100 − x i − ( i + 1) ) + / (100 − x i / ) − (100 − x i ) = − x i / − i / − i − / . Notethat if the term − x i / would not be present in E ( G i +1 − G i | F i ) , we could certify theprogram to be PAST using the RSM-Rule because − i / − i − / ≤ − / for all i ≥ .However, by taking a closer look at the variable x , we observe that it is eventually andalmost surely lower bounded by the function α · − i for some α ∈ R + . Therefore, eventu-ally − x i / ≤ − β · − i for some β ∈ R + . Thus, eventually E ( G i +1 − G i | F i ) ≤ − γ · i forsome γ ∈ R + . By our RSM-Rule, the program is PAST. Remark 3.
For Example 8, it would in fact be enough to impose a lower bound of on x .However, for more complex examples, tight asymptotic bounds are essential.We next present our method for computing asymptotic bounds over martingale expres-sions in Sections 5.1-5.2. Based on these asymptotic bounds, in Section 5.3 we introducealgorithmic approaches for our proof rules from Section 4, solving our aforementionedchallenges (i)-(ii) in a fully automated manner (Section 5.4). Algorithm 1 computes asymptotic bounds on monomials over program variables. To en-sure termination of Algorithm 1, it is important that there are no circular dependenciesamong monomials. By the definition of Prob-solvable loops, this indeed holds for pro-gram variables (monomials of order 1). Every Prob-solvable loop L comes with an order-ing on its variables and every variable is restricted to only depend linearly on itself andpolynomially on previous variables. Acyclic dependencies naturally extend from singlevariables to monomials. Definition 14 (Monomial Ordering).
Let L be a Prob-solvable loop with variables x (1) ,...,x ( m ) . Let y = Q mj =1 x p j ( j ) and y = Q mj =1 x q j ( j ) , where p j ,q j ∈ N , be two monomi-als over the program variables. The order (cid:22) on monomials over the program variablesof L is defined by y (cid:22) y ⇐⇒ ( p m ,...,p ) ≤ lex ( q m ,...,q ) , where ≤ lex is the lexico-graphic order on N m . The order (cid:22) is total because ≤ lex is total. With y ≺ y we denote y (cid:22) y ∧ y = y . utomated Termination Analysis of Polynomial Probabilistic Programs 15 Example 9 (Monomials).
Let L be a Prob-solvable loop with variables x (1) ,...,x ( m ) . Thefollowing statements hold for the monomial order (cid:22) : ≺ x (1) ≺ x (2) ≺ ... ≺ x ( m − ≺ x ( m ) , x k (1) ≺ x (2) for any k ∈ N x ≺ x and x x x ≺ x x x .To prove acyclic dependencies for monomials we exploit the following fact. Lemma 1.
Let y ,y ,z ,z be monomials. If y (cid:22) z and y (cid:22) z then y · y (cid:22) z · z . By structural induction over monomials and Lemma 1, Lemma 2 can be established.
Lemma 2 (Monomial Acyclic Dependency).
Let x be a monomial over the programvariables of L . For every branch B ∈ supp ( U x L ) and monomial y in B , y (cid:22) x holds.Proof. We use structural induction over monomials. The base case for which x is a singlevariable holds by the definition of L being a Prob-solvable loop. Let x := s · t where s and t are monomials over the variables of L and – for every B s ∈ supp ( U s L ) and every monomial u in B s it holds that u (cid:22) s , – for every B t ∈ supp ( U t L ) and every monomial w in B t it holds that w (cid:22) t ,Let B ∈ supp ( U x L ) be an arbitrary branch of x . By definition of U x L , we get B = B s · B t ,where B s is a branch of s and B t is a branch of t . Note that B s and B t are polynomialsover program variables or equivalently linear combinations of monomials. Therefore, forevery monomial y in B we have y = u · w where u is a monomial in B s and w a monomialin B t . By the induction hypothesis, u (cid:22) s and w (cid:22) t . Using Lemma 1, we get u · w (cid:22) s · t which means y (cid:22) x . ⊓⊔ Lemma 2 states that the value of a monomial x over the program variables of L only de-pends on the value of monomials y which precede x in the monomial ordering (cid:22) . This en-sures the dependencies among monomials over the program variables of L to be acyclic. The structural result on monomial dependencies from Lemma 2 allows for recursive pro-cedures over monomials. This is exploited in Algorithm 1 for computing asymptoticbounds for monomials. The standard Big-O notation does not differentiate between pos-itive and negative functions, as it considers the absolute value of functions. We, however,need to differentiate between functions like i and − i . Therefore, we introduce the no-tions of Domination and
Bounding Functions . Definition 15 (Domination).
Let F be a finite set of functions from N to R . A function g : N → R is dominating F if eventually α · g ( i ) ≥ f ( i ) for all f ∈ F and some α ∈ R + . Afunction g : N → R is dominated by F if all f ∈ F dominate { g } . Intuitively, a function f dominates a function g if f eventually surpasses g modulo apositive constant factor. Exponential polynomials are sums of products of polynomialswith exponential functions, i.e. P j p j ( x ) · c xj , where c j ∈ R +0 . All functions arising inAlgorithms 1-5 are exponential polynomials. For a finite set F of exponential polyno-mials, a function dominating F and a function dominated by F are easily computable with standard techniques, by analyzing the terms of the functions in the finite set F .With dominating ( F ) we denote an algorithm computing an exponential polynomialdominating F . With dominated ( F ) we denote an algorithm computing an exponen-tial polynomial dominated by F . We assume the functions returned by the algorithms dominating ( F ) and dominated ( F ) to be monotone and either non-negative or non-positive. Example 10 (Domination).
The following statements are true: dominates {− i + i +5 } , i dominates { i } , i · i dominates { i · i + i , i + i , − i } , i is dominated by { i − i +1 , i − } and − i is dominated by { i − i , − · − i } . Definition 16 (Bounding Function for L ). Let E be an arithmetic expression over theprogram variables of L . Let l,u : N → R be monotone and non-negative or non-positive.1. l is a lower bounding function for E if eventually P ( α · l ( i ) ≤ E i | T ¬G > i ) = 1 forsome α ∈ R + .2. u is an upper bounding function for E if eventually P ( E i ≤ α · u ( i ) | T ¬G > i ) = 1 forsome α ∈ R + .3. An absolute bounding function for E is an upper bounding function for | E | . A bounding function imposes a bound on an expression E over the program variablesholding eventually, almost surely, and modulo a positive constant factor. Moreover, boundson E only need to hold as long as the program has not yet terminated.Given a Prob-solvable loop L and a monomial x over the program variables of L , Algo-rithm 1 computes a lower and upper bounding function for x . Because every polynomialexpression is a linear combination of monomials, the procedure can be used to computelower and upper bounding functions for any polynomial expression over L ’s programvariables by substituting every monomial with its lower or upper bounding function de-pending on the sign of the monomial’s coefficient. Once a lower bounding function l and an upper bounding function u are computed, an absolute bounding function can becomputed by dominating ( { u, − l } ) .In Algorithm 1 candidates for bounding functions are modeled using recurrence rela-tions. Solutions s ( i ) of these recurrences are closed-form candidates for bounding func-tions parameterized by loop iteration i . Algorithm 1relies on the existence of closed-formsolutions of recurrences. While closed-forms of general recurrences do not always exist, aproperty of C-finite recurrences , that is of linear recurrences with constant coefficients, isthat their closed-forms always exist and are computable [31]. In all occurring recurrences,we consider a monomial over program variables as a single function. Therefore, through-out this section, all recurrences arising from a Prob-solvable loop L in Algorithm 1 areC-finite or can be turned into C-finite recurrences. Moreover, closed-forms s ( i ) of C-finite recurrences are given by exponential polynomials. Therefore, for any solution s ( i ) to a C-finite recurrence and any constant r ∈ R , the following holds: ∃ α,β ∈ R + , ∃ i ∈ N : ∀ i ≥ i : α · s ( i ) ≤ s ( i + r ) ≤ β · s ( i ) . (2)Intuitively, the property states that constant shifts do not change the asymptotic behaviorof s . We use this property at various proof steps in this section. Moreover, we recall thatlimits of exponential polynomials are computable [22]. utomated Termination Analysis of Polynomial Probabilistic Programs 17 For every monomial x , every branch B ∈ supp ( U x L ) is a polynomial over the programvariables. Let Rec ( x ) := { coefficient of x in B | B ∈ supp ( U x L ) } denote the set of co-efficients of the monomial x in all branches of L . Let Inhom ( x ) := { B − c · x | B ∈ supp ( U x L ) and c = coefficient of x in B } denote all the branches of the monomial x with-out x and its coefficient. The symbolic constants c and c in Algorithm 1 represent arbi-trary initial values of the monomial x for which bounding functions are computed. Thefact that they are symbolic ensures that all potential initial values are accounted for. c represents positive initial values and − c negative initial values. The symbolic constant d is used in the recurrences to account for the fact that the bounding functions only holdmodulo a constant. Intuitively, if we use the bounding function in a recurrence we need torestore the lost constant. Sign ( x ) is an over-approximation of the sign of the monomial x , i.e., if ∃ i : P ( x i > > , then + ∈ Sign ( x ) and if ∃ i : P ( x i < > , then − ∈ Sign ( x ) . Algorithm 1:
Computing bounding functions for monomials
Input:
A Prob-solvable loop L and a monomial x over L ’s variables Output:
Lower and upper bounding functions l ( i ) , u ( i ) for x inhomBoundsUpper := { upper bounding function of P | P ∈ Inhom ( x ) } inhomBoundsLower := { lower bounding function of P | P ∈ Inhom ( x ) } U ( i ) := dominating ( inhomBoundsUpper ) L ( i ) := dominated ( inhomBoundsLower ) maxRec := max Rec ( x ) minRec := min Rec ( x ) I := ∅ if + ∈ Sign ( x ) then I := I ∪{ c } ; if − ∈ Sign ( x ) then I := I ∪{− c } ; uCand := closed-forms of { y i +1 = r · y i + d · U ( i ) | r ∈ { minRec , maxRec } ,y ∈ I } lCand := closed-forms of { y i +1 = r · y i + d · L ( i ) | r ∈ { minRec , maxRec } ,y ∈ I } u ( i ) := dominating ( uCand ) l ( i ) := dominated ( lCand ) return l ( i ) ,u ( i ) Lemma 2, the computability of closed-forms of C-finite recurrences and the fact thatwithin a Prob-solvable loop only finitely many monomials can occur, implies the termi-nation of Algorithm 1. Its correctness is stated in the next theorem.
Theorem 6 (Correctness of Algorithm 1).
The functions l ( i ) , u ( i ) returned by Algo-rithm 1 on input L and x are a lower bounding function and an upper bounding functionfor x , respectively.Proof. Intuitively, it has to be shown that regardless of the paths through the loop bodytaken by any program run, the value of x is always eventually upper bounded by somefunction in uCand and eventually lower bounded by some function in lCand (almostsurely and modulo positive constant factors). We show that x is always eventually up-per bounded by some function in uCand . The proof for the lower bounding function isanalogous. Let ϑ ∈ Σ be a possible program run, i.e. P ( Cyl ( π )) > for all finite prefixes π of ϑ .Then, for every i ∈ N , if T ¬G ( ϑ ) > i , the following holds: x i +1 ( ϑ ) = a (1) · x i ( ϑ )+ P (1) i ( ϑ ) or x i +1 ( ϑ ) = a (2) · x i ( ϑ )+ P (2) i ( ϑ ) or ... or x i +1 ( ϑ ) = a ( k ) · x i ( ϑ )+ P ( k ) i ( ϑ ) , where a ( j ) ∈ Rec ( x ) and P ( j ) ∈ Inhom ( x ) are polynomials over program variables. Let u ( i ) , ..., u k ( i ) be upper bounding functions of P (1) , ..., P ( k ) , which are computed re-cursively at line 10. Moreover, let U ( i ) := dominating ( { u ( i ) ,...,u k ( i ) } ) , minRec =min Rec ( x ) and maxRec = max Rec ( x ) . Let l ∈ N be the smallest number such that forall j ∈ { ,...,k } and i ≥ l : P ( P ( j ) i ≤ α j · u j ( i ) | T ¬G > i ) = 1 for some α j ∈ R + , and (3) u j ( i ) ≤ β · U ( i ) for some β ∈ R + (4)Thus, all inequalities from the bounding functions u j and the dominating function U hold from l onward. Because U is a dominating function, it is by definition either non-negative or non-positive. Assume U ( i ) to be non-negative, the case for which U ( i ) isnon-positive is symmetric. Using the facts (3) and (4), we establish: For the constant γ := β · max j =1 ..k α j , it holds that P ( P ( j ) i ≤ γ · U ( i ) | T ¬G > i ) = 1 for all j ∈ { ,...,k } and all i ≥ l . Let l be the smallest number such that l ≥ l and U ( i + l ) ≤ δ · U ( i ) forall i ≥ l and some δ ∈ R + . Case 1, x i is almost surely negative for all i ≥ l : Consider the recurrence relation y = m , y i +1 = minRec · y i + η · U ( i ) , where η := max( γ,δ ) and m is the maximumvalue of x l ( ϑ ) among all possible program runs ϑ . Note that m exists because thereare only finitely many values x l ( ϑ ) for possible program runs ϑ . Moreover, m is nega-tive by our case assumption. By induction, we get P ( x i ≤ y i − l | T ¬G > i ) = 1 for all i ≥ l . Therefore, for a closed-form solution s ( i ) of the recurrence relation y i , we get P ( x i ≤ s ( i − l ) | T ¬G > i ) = 1 for all i ≥ l . We emphasize that s exists and can effec-tively be computed because y i is C-finite. Moreover, s ( i − l ) ≤ θ · s ( i ) for all i ≥ l forsome l ≥ l and some θ ∈ R + . Therefore, s satisfies the bound condition of an upperbounding function. Also, s is present in uCand by choosing the symbolic constants c and d to represent − m and η respectively. The function u ( i ) := dominating ( uCand ) ,at line 12, is dominating uCand (hence also s ), is monotone and either non-positive ornon-negative. Therefore, u ( i ) is an upper bounding function for x . Case 2, x i is not almost surely negative for all i ≥ l : Thus, there is a possible pro-gram run ϑ ′ such that x i ( ϑ ′ ) ≥ for some i ≥ l . Let l ≥ l be the smallest numbersuch that x l ( ˆ ϑ ) ≥ for some possible program run ˆ ϑ . This number certainly exists, as x i ( ϑ ′ ) is non-negative for some i ≥ l . Consider the recurrence relation y = m , y i +1 = maxRec · y i + η · U ( i ) , where η := max( γ, δ ) and m is the maximum value of x l ( ϑ ) among all possible program runs ϑ . Note that m exists because there are only finitelymany values x l ( ϑ ) for possible program runs ϑ . Moreover, m is non-negative because m ≥ x l ( ˆ ϑ ) ≥ . By induction, we get P ( x i ≤ y i − l | T ¬G > i ) = 1 for all i ≥ l . Therefore,for a solution s ( i ) of the recurrence relation y i , we get P ( x i ≤ s ( i − l ) | T ¬G > i ) = 1 utomated Termination Analysis of Polynomial Probabilistic Programs 19 for all i ≥ l . As above, s exists and can effectively be computed because y i is C-finite.Moreover, s ( i − l ) ≤ θ · s ( i ) for all i ≥ l for some l ≥ l and some θ ∈ R + . Therefore, s satisfies the bound condition of an upper bounding function Also, s is present in uCand by choosing the symbolic constants c and d to represent m and η respectively. The func-tion u ( i ) := dominating ( uCand ) , at line 12, is dominating uCand (hence also s ), ismonotone and either non-positive or non-negative. Therefore, u ( i ) is an upper boundingfunction for x . ⊓⊔ Example 11 (Bounding functions).
We illustrate Algorithm 1 by computing boundingfunctions for x and the Prob-solvable loop from Example 8: We have Rec ( x ) := { , } and Inhom ( x ) = { y , } . Computing bounding functions recursively for P ∈ Inhom ( x ) = { y , } is simple, as we can give exact bounds leading to inhomBoundsUpper = { i , } and inhomBoundsLower = { i , } . Consequently, we get U ( i ) = i , L ( i ) = 0 , maxRec =2 and minRec = . With a rudimentary static analysis of the loop, we determine the (ex-act) over-approximation Sign ( x ) := { + } by observing that x > and all P ∈ Inhom ( x ) are strictly positive. Therefore, uCand is the set of closed-form solutions of the recur-rences y := c , y i +1 := 2 y i + d · i and y := c , y i +1 := y i + d · i . Similarly, lCand is the set of closed-form solutions of the recurrences y := c , y i +1 := 2 y i and y := c , y i +1 := y i . Using any algorithm for computing closed-forms of C-finite recurrences,we obtain uCand = { c i − di − di + 3 d i − d, c − i + 2 di − di − d − i + 12 d } and lCand = { c i , c − i } . This leads to the upper bounding function u ( i ) = 2 i and thelower bounding function l ( i ) = 2 − i . The bounding functions l ( i ) and u ( i ) can be usedto compute bounding functions for expressions containing x linearly by replacing x by l ( i ) or u ( i ) depending on the sign of the coefficient of x . For instance, eventually andalmost surely the following inequality holds: − x i − i − i − ≤ − · α · − i − i − i − for some α ∈ R + . The inequality results from replacing x i by l ( i ) . Therefore, eventuallyand almost surely − x i − i − i − ≤ − β · i for some β ∈ R + . Thus, − i is an upperbounding function for the expression − x i − i − i − . Remark 4.
Note that Algorithm 1 describes a general procedure computing boundingfunctions for special sequences. Figuratively, that is for sequences s such that s i +1 = f ( s i ,i ) but in every step the function f is chosen non-deterministically among a fixedset of special functions (corresponding to branches in our case). We reserve the inves-tigation of potential applications of bounding functions for such sequences beyond theprobabilistic setting for future work. Using Algorithm 1 to compute bounding functions for polynomial expressions over pro-gram variables at hand, we are now able to formalize our algorithmic approaches automat-ing the termination analysis of Prob-solvable loops using the proof rules from Section 4.Given a Prob-solvable loop L and a polynomial expression E over L ’s variables, we de-note with lbf ( E ) , ubf ( E ) and abf ( E ) functions computing a lower, upper and absolutebounding function for E respectively. Our algorithmic approach for proving PAST usingthe RSM-Rule is given in Algorithm 2. Algorithm 2:
Ranking-Supermartingale-Rule for proving PAST
Input:
Prob-solvable loop L Output: If true then L with G satisfies the RSM-Rule; hence L is PAST E := E ( G i +1 − G i | F i ) u ( i ) := ubf ( E ) limit := lim i →∞ u ( i ) return limit < Example 12 (Algorithm 2).
Let us illustrate Algorithm 2 with the Prob-solvable loopfrom Examples 8 and 11. Applying Algorithm 2 on L leads to E = − x i − i − i − .We obtain the upper bounding function u ( i ) := − i for E . Because lim i →∞ u ( i ) < ,Algorithm 2 returns true. This is valid because u ( i ) having a negative limit witnesses that E is eventually bounded by a negative constant and therefore is eventually an RSM.We recall that all functions arising from L are exponential polynomials (see Sec-tion 5.2) and that limits of exponential polynomials are computable [22]. Therefore, thetermination of Algorithm 2 is guaranteed and its correctness is stated next. Theorem 7 (Correctness of Algorithm 2).
If Algorithm 2 returns true on input L , then L with G L satisfies the RSM-Rule.Proof. When returning true at line 4 we have P ( E i ≤ α · u ( i ) | T ¬G > i ) = 1 forall i ≥ i and some i ∈ N , α ∈ R + . Moreover, u ( i ) < − ǫ for all i ≥ i for some i ∈ N , by the definition of lim . From this follows that ∀ i ≥ max( i ,i ) almost surely G i = ⇒ E ( G i +1 − G i | F i ) ≤ − α · ǫ , which means G is eventually an RSM. ⊓⊔ Our approach proving AST using the SM-Rule is captured with Algorithm 3.
Algorithm 3:
Supermartingale-Rule for proving AST
Input:
Prob-solvable loop L Output: If true , L with G satisfies the SM-Rule with constant d and p ; hence L is AST E := E ( G i +1 − G i | F i ) u ( i ) := ubf ( E ) if not eventually u ( i ) ≤ then return false ; for B ∈ supp ( U G L ) do d ( i ) := ubf ( B − G ) limit := lim i →∞ d ( i ) if limit < then return true ; end return false Example 13 (Algorithm 3).
Let us illustrate Algorithm 3 for the Prob-solvable loop L from Figure 2a: Applying Algorithm 3 on L yields E ≡ and u ( i ) = 0 . The expression G ( = x ) has two branches. One of them is x i − y i + 4 , which occurs with probability / . When the for-loop of Algorithm 3 reaches this branch B = x i − y i + 4 on line 4, itcomputes the difference B − G = − y i + 4 . An upper bounding function for B − G is utomated Termination Analysis of Polynomial Probabilistic Programs 21 given by d ( i ) = − i . Because lim i →∞ d ( i ) < , Algorithm 3 returns true. This is validbecause of the branch B witnessing that G eventually decreases by at least a constantwith probability / . Therefore, all conditions of the SM-Rule are satisfied and L is AST. Theorem 8 (Correctness of Algorithm 3).
If Algorithm 3 returns true on input L , then L with G L satisfies the SM-Rule with constant d and p .Proof. Similarly as for the correctness of Algorithm 2, G is a supermartingale if Algo-rithm 3 returns true . Moreover, there is a branch B ∈ supp ( U G L ) such that G changeseventually and almost surely by at most α · d ( i ) , for some α ∈ R + . In addition, because lim i →∞ d ( i ) < , it follows that d ( i ) ≤ − ǫ for all i ≥ i for some i ∈ N , ǫ ∈ R + . Therefore,eventually G decreases by at least α · ǫ with probability at least U G L ( B ) > . Hence, allconditions of the SM-Rule are satisfied. ⊓⊔ As established in Section 4, the relaxation of the R-AST-Rule requires that there is apositive probability of reaching the iteration i after which the conditions of the proof rulehold. Regarding automation, we strengthen this condition by ensuring that there is a pos-itive probability of reaching any iteration, i.e. ∀ i ∈ N : P ( G i ) > . Obviously, this implies P ( G i ) > . Furthermore, with CanReachAnyIteration ( L ) we denote a computableunder-approximation of ∀ i ∈ N : P ( G i ) > . That means, CanReachAnyIteration ( L ) im-plies ∀ i ∈ N : P ( G i ) > . Our approach proving non-AST is summarized in Algorithm 4. Algorithm 4:
Repulsing-AST-Rule for proving non-AST
Input:
Prob-solvable loop L Output: if true , L with − G satisfies the R-AST-Rule; hence L is not AST E := E ( − G i +1 + G i | F i ) u ( i ) := ubf ( E ) if not eventually u ( i ) ≤ then return false ; if ¬ CanReachAnyIteration ( L ) then return false ; ǫ ( i ) := − u ( i ) if ǫ ( i ) Ω (1) then return false ; differences := { B + G | B ∈ supp ( U − G L ) } diffBounds := { abf ( d ) | d ∈ differences } c ( i ) := dominating ( diffBounds ) return c ( i ) ∈ O (1) Example 14 (Algorithm 4).
Let us illustrate Algorithm 4 for the Prob-solvable loop L from Figure 2a: Applying Algorithm 4 on L leads to E = y i − = − i − and to theupper bounding function u ( i ) = − for E on line 2. Therefore, the if-statement on line 3is not executed, which means − G is eventually a ǫ -repulsing supermartingale. Moreover,with a simple static analysis of the loop, we establish CanReachAnyIteration ( L ) tobe true, as there is a positive probability that the loop guard does not decrease. Thus,the if-statement on line 4 is not executed. Also, the if-statement on line 6 is not executed,because ǫ ( i ) = − u ( i ) = 1 is constant and therefore in Ω (1) . E eventually decreases by ǫ =1 (modulo a positive constant factor), because u ( i ) = − is an upper bounding functionfor E . We have differences = { − y i , y i } . Both expressions in differences have an absolute bounding function of . Therefore, diffBounds = { } . As a result on line 9 wehave c ( i ) = 1 , which eventually and almost surely is an upper bound on | − G i +1 + G i | (modulo a positive constant factor). Therefore, the algorithm returns true. This is correct,as all the preconditions of the R-AST-Rule are satisfied (and therefore L is not AST). Theorem 9 (Correctness of Algorithm 4).
If Algorithm 4 returns true on input L , then L with − G L satisfies the R-AST-Rule.Proof. With the same reasoning as for the correctness of Algorithm 3, − G is a super-martingale if Algorithm 4 returns true . Moreover, the condition P ( − G i < > of theR-AST-Rule is satisfied, due to the under-approximation CanReachAnyIteration ( L ) and the if-statement on line 4. The function u ( i ) is an up-per bounding function for E ( − G i +1 + G i | F i ) . Hence, eventually and almost surely E ( − G i +1 + G i | F i ) ≤ − α · ǫ ( i ) for ǫ ( i ) := − u ( i ) and some α ∈ R + . The if-statement atline 6 ensures that ǫ ( i ) is lower bounded by a constant. Therefore, − G eventually is an ( α · ǫ ) -repulsing supermartingale. The function c ( i ) , assigned to dominating ( diffBounds ) ,is a function dominating absolute bounding functions of all branches of − G i +1 + G i .Consequently, c ( i ) is a bound on the differences of G , i.e. eventually and almost surely | − G i +1 + G i | ≤ β · c ( i ) for some β ∈ R + . Algorithm 4 returns true only if c ( i ) can bebounded by a constant which in turn means G has ( β · c ) -bounded differences. Thus, ifAlgorithm 4 returns true , all preconditions of the R-AST-Rule are satisfied. ⊓⊔ We finally provide Algorithm 5 for the R-PAST-Rule. The algorithm is a variation ofAlgorithm 4 (for the R-AST-Rule). The if-statement on line 2 forces − G to be a martin-gale. Therefore, after the if-statement − G is an ǫ -repulsing supermartingale with ǫ = 0 . Algorithm 5:
Repulsing-PAST-Rule for proving non-PAST
Input:
Prob-solvable loop L Output: If true , L with − G satisfies the R-PAST-Rule; hence L is not PAST E := E ( − G i +1 + G i | F i ) if E then return false ; if ¬ CanReachAnyIteration ( L ) then return false ; differences := { B + G | B ∈ supp ( U − G L ) } diffBounds := { abf ( d ) | d ∈ differences } c ( i ) := dominating ( diffBounds ) return c ( i ) ∈ O (1) A question arising when combining our algorithmic approaches from Section 5.3 into aunifying framework is that, given a Prob-solvable loop L , what algorithm to apply firstfor determining L ’s termination behavior? In [4] the authors provide an algorithm forcomputing an algebraically closed-form of E ( M i ) , where M is a polynomial over L ’svariables. The following lemma explains how the expression E ( M i +1 − M i ) relates tothe expression E ( M i +1 − M i | F i ) . utomated Termination Analysis of Polynomial Probabilistic Programs 23 Lemma 3 (Rule out Rules for L ). Let ( M i ) i ∈ N be a stochastic process. If E ( M i +1 − M i | F i ) ≤ − ǫ then E ( M i +1 − M i ) ≤ − ǫ , for any ǫ ∈ R + .Proof. E ( M i +1 − M i | F i ) ≤ − ǫ = ⇒ (Monotonicity of E ) E ( E ( M i +1 − M i | F i )) ≤ E ( − ǫ ) ⇐⇒ (Property of E ( · | F i ) ) E ( M i +1 − M i ) ≤ E ( − ǫ ) ⇐⇒ ( − ǫ is constant) E ( M i +1 − M i ) ≤ − ǫ ⊓⊔ The contrapositive of Lemma 3 provides a criterion to rule out the viability of a givenproof rule. For a Prob-solvable loop L , if E ( G i +1 − G i ) then E ( G i +1 − G i | F i ) ,meaning G is not a supermartingale. The expression E ( G i +1 − G i ) depends only on i and can be computed by E ( G i +1 − G i ) = E ( G i +1 ) − E ( G i ) , where the expected value E ( G i ) is computed as in [4]. Therefore, in some cases, proof rules can automatically bedeemed nonviable, without the need to compute bounding functions. We implemented and combined our algorithmic approaches from Section 5 in the newsoftware tool A
MBER to stand for
Asymptotic Martingale Bounds . A
MBER and all bench-marks are available at https://github.com/probing-lab/amber . A
MBER uses M
ORA [4] for computing the first-order moments of program variables and the
DIO - FANT package as its computer algebra system. Computing dominating and dominated
The dominating and dominated proceduresused in Algorithms 1 and 4 are implemented by combining standard algorithms for Big-Oanalysis and bookkeeping of the asymptotic polarity of the input functions. Consider thefollowing two input-output-pairs which our implementation of dominating would pro-duce: (a) dominating ( { i + 10 , · i − i } ) = i and (b) dominating ( {− i + 50 , − i + i − · i } ) = − i . For (a) i is eventually greater than all functions in the input setmodulo a constant factor because all functions in the input set are O ( i ) . Therefore, i dominates the input set. For (b), the first function is O ( i ) and the second is O ( i ) . In thiscase, however, both functions are eventually negative. Therefore, − i is a function dom-inating the input set. The two examples illustrate the idea of implementing dominating (and similarly dominated ) by leveraging standard Big-O algorithms and tracking theasymptotic polarity of the input functions. Important is the fact that an exponential poly-nomial P j p j ( i ) · c ij , where c j ∈ R +0 will always be eventually either only positive or onlynegative (or if identical to ). https://github.com/diofant/diofant Sign Over-Approximation
The over-approximation
Sign ( x ) of the signs of a monomial x used in Algorithm 1 is implemented by a simple static analysis: For a monomial x con-sisting solely of even powers, Sign ( x ) = { + } . For a general monomial x , if x ≥ andall monomials on which x depends, together with their associated coefficients are alwayspositive, then − 6∈ Sign ( x ) . For example, if supp ( U x L ) = { x i + 2 y i − z i ,x i + u i } , then − 6∈ Sign ( x ) if x ≥ as well as − 6∈ Sign ( y ) , + Sign ( z ) and − 6∈ Sign ( u ) . Otherwise, − ∈ Sign ( x ) . The over-approximation for + Sign ( x ) is analogous. Reachability Under-Approximation
CanReachAnyIteration ( L ) , used in Algorithm 4,needs to satisfy the property that if CanReachAnyIteration ( L ) is true, then there isa positive probability that the loop L reaches any iteration. In A MBER , we implementthis under-approximation in the following way:
CanReachAnyIteration ( L ) is true ifthere is a branch B of the loop guard polynomial G L such that B − G L i is non-negativefor all i ∈ N . Otherwise, CanReachAnyIteration ( L ) is false. In other words, if CanReachAnyIteration ( L ) is true, then in any iteration there is a positive probability of G L not decreasing. Bound Computation Improvements
In addition to Algorithm 1 computing boundingfunctions for monomials of program variables, A
MBER implements the following re-finements:1. A monomial x is deterministic, which means it is independent of probabilistic choices,if x has a single branch and only depends on monomials having single branches. Inthis case, the exact value of x in any iteration is given by its first-order moments andbounding functions can be obtained by using these exact representations.2. Bounding functions for an odd power p of a monomial x can be computed by u ( i ) p and l ( i ) p , where u ( i ) is an upper- and l ( i ) a lower bounding function for x .Whenever the above enhancements are applicable, A MBER prefers them over Algorithm 1.
Experimental Setting and Comparisons
Regarding programs which are PAST, we com-pare A
MBER against the tool A
BSYNTH [40] and the tool in [9] which we refer to asM
GEN . A
BSYNTH uses a system of inference rules over the syntax of probabilistic pro-grams to derive bounds on the expected resource consumption of a program and can,therefore, be used to certify PAST. In comparison to A
MBER , A
BSYNTH requires thedegree of the bound to be provided upfront. Moreover, A
BSYNTH cannot refute the exis-tence of a bound and therefore cannot handle programs that are not PAST. M
GEN uses lin-ear programming to synthesize linear martingales and supermartingales for probabilistictransition systems with linear variable updates. To certify PAST, we extended the M
GEN toolchain of [9], as follows: We combined M
GEN with the SMT solver Z3 [39] in orderto find or refute the existence of conic combinations of the (super)martingales derived byM
GEN which yield RSMs.With A
MBER -L IGHT we refer to a variant of A
MBER without the relaxations of theproof rules introduced in Section 4. That is, with A
MBER -L IGHT the conditions of theproof rules need to hold for all i ∈ N , whereas with A MBER the conditions are allowed toonly hold eventually. For all benchmarks, we compare A
MBER against A
MBER -L IGHT utomated Termination Analysis of Polynomial Probabilistic Programs 25 to show the effectiveness of the respective relaxations. For each experimental table (Ta-bles 1-3), ✓ symbolizes that the respective tool successfully certified PAST/AST/non-AST for the given program; ✗ means it failed to certify PAST/AST/non-AST. Further, NA indicates the respective tool failed to certify PAST/AST/non-AST because the givenprogram is out-of-scope of the tool’s capabilities. Every benchmark has been run on a ma-chine with a . GHz Intel i7 (Gen 6) processor and 16 GB of RAM and finished withina timeout of seconds, where most benchmarks terminated within a few seconds. Benchmarks
We evaluated A
MBER against 38 probabilistic programs. We present our ex-perimental results by separating our benchmarks within three categories: (i) 21 programswhich are PAST (Table 1), (ii) 11 programs which are AST (Table 2) but not necessar-ily PAST, and (iii) 6 programs which are not AST (Table 3). The benchmarks have eitherbeen introduced in the literature on probabilistic programming [40,9,4,21,37], are adapta-tions of well-known stochastic processes or have been designed specifically to test uniquefeatures of A
MBER , like the ability to handle polynomial real arithmetic.The 21 PAST benchmarks consist of 10 programs representing the original bench-marks of M
GEN [9] and A
BSYNTH [40] augmented with 11 additional probabilistic pro-grams. Not all benchmarks of M
GEN and A
BSYNTH could be used for our comparison asM
GEN and A
BSYNTH target related but different computation tasks than certifying PAST.Namely, M
GEN aims to synthesize (super)martingales, but not ranking ones, whereas A B - SYNTH focuses on computing bounds on the expected runtime. Therefore, we adopted all (50) benchmarks from [9] (11) and [40] (39) for which the termination behavior is non-trivial. A benchmark is trivial regarding PAST if either (i) there is no loop, (ii) the loop isbounded by a constant, or (iii) the program is meant to run forever. Moreover, we cleansedthe benchmarks of programs for which the witness for PAST is just a trivial combinationof witnesses for already included programs. For instance, the benchmarks of [40] con-tain multiple programs that are concatenated constant biased-random-walks. These arerelevant benchmarks when evaluating A
BSYNTH for discovering bounds, but would blurthe picture when comparing against A
MBER for PAST certification. With these criteria,out of the 50 original benchmarks of [9] and [40] 10 remain. We augment these 10 pro-grams with 11 additional benchmarks which have either been introduced in the literatureon probabilistic programming [4,21,37], are adaptations of well-known stochastic pro-cesses or have been designed specifically to test unique features of A
MBER . Notably, outof the 50 original benchmarks from [40] and [9], only 2 remain which are included in ourbenchmarks and which A
MBER cannot prove PAST (because they are not Prob-solvable).All benchmarks for PAST, AST and non-AST are available at at https://github.com/probing-lab/amber . Experiments with PAST – Table 1:
Out of the 21 PAST benchmarks, A
MBER certifies18 programs. A
MBER cannot handle the benchmarks nested_loops and sequential_loops ,as these examples use nested or sequential loops and thus are not expressible as Prob-solvable loops. The benchmarks exponential_past_1 and exponential_past_2 are out ofscope of A
BSYNTH because they require real numbers, while A
BSYNTH can only han-dle integers. All benchmarks which are out of scope of M
GEN +Z3 are so because theycontain non-linear variable updates or non-linear guards. Our experiments from Table 1shows that A
MBER outperforms both A
BSYNTH and M
GEN +Z3 for Prob-solvable loops,even when our relaxed proof rules from Section 4 are not used. Yet, our experiments show
Program A M B E R A M B E R - L I GH T A B S YN T H M G E N + Z ✓ ✓ ✗ NA biased_random_walk_constant ✓ ✓ ✓ ✓ biased_random_walk_exp ✓ ✓ ✗ ✓ biased_random_walk_poly ✓ ✗ ✗ ✗ binomial_past ✓ ✓ ✓ ✓ complex_past ✓ ✗ ✗ NA consecutive_bernoulli_trails ✓ ✓ ✓ ✓ coupon_collector_4 ✓ ✗ ✗ ✓ coupon_collector_5 ✓ ✗ ✗ ✓ dueling_cowboys ✓ ✓ ✓ ✓ exponential_past_1 ✓ ✓ NA NA
Program A M B E R A M B E R - L I GH T A B S YN T H M G E N + Z exponential_past_2 ✓ ✓ NA NA geometric ✓ ✓ ✓ ✓ geometric_exponential ✗ ✗ ✗ ✗ linear_past_1 ✓ ✓ ✗ ✗ linear_past_2 ✓ ✓ ✗ NA nested_loops NA NA ✓ ✗ polynomial_past_1 ✓ ✗ ✗ NA polynomial_past_2 ✓ ✗ ✗ NA sequential_loops NA NA ✓ ✗ tortoise_hare_race ✓ ✓ ✓ ✓
Total ✓
18 12 8 9
Table 1: 21 programs which are PAST. that our relaxed proof rules enable A
MBER to certify 6 examples to be PAST, which couldnot be proved without these relaxations by A
MBER -L IGHT . Experiments with AST – Table 2:
We compare A
MBER against A
MBER -L IGHT on 11benchmarks which are AST but not necessarily PAST. Therefore, the SM-Rule is neededto certify AST. To the best of our knowledge, A
MBER is the first tool able to certify ASTfor such programs, a reason why we compared A
MBER only against A
MBER -L IGHT onthis set of examples. The benchmark symmetric_2d_random_walk , which A
MBER failsto certify as AST, models the symmetric random walk in R and is still out of reach of cur-rent automation techniques. In [37] the authors mention that a closed-form expression M and functions p and d satisfying the conditions of the SM-Rule have not been discoveredyet. The benchmark fair_in_limit_random_walk involves non-constant probabilities andcan therefore not be modeled as a Prob-solvable loop. Experiments with non-AST – Table 3:
We compare A
MBER against A
MBER -L IGHT on6 benchmarks which are not AST. To the best of our knowledge, A
MBER is the firsttool able to certify non-AST for such programs, and thus we compared A
MBER onlyagainst A
MBER -L IGHT . In [12], where the notion of repulsing supermartingales and theR-AST-Rule are introduced, the authors also propose automation techniques. However,the authors of [12] claim that their “experimental results are basic“ and their compu-tational methods are evaluated on only 3 examples, without having any available toolsupport. For the benchmarks in Table 3, the outcomes of A
MBER and A
MBER -L IGHT coincide. The reason for this is R-AST-Rule’s condition that the martingale expressionhas to have c -bounded differences. This condition forces a suitable martingale expressionto be bounded by a linear function, which is also the reason why A MBER cannot certifythe benchmark polynomial_nast . utomated Termination Analysis of Polynomial Probabilistic Programs 27 Program A
MBER A MBER -L IGHT fair_in_limit_random_walk
NA NA gambling ✓ ✓ symmetric_2d_random_walk ✗ ✗ symmetric_random_walk_constant_1 ✓ ✓ symmetric_random_walk_constant_2 ✓ ✓ symmetric_random_walk_exp_1 ✓ ✗ symmetric_random_walk_exp_2 ✓ ✗ symmetric_random_walk_linear_1 ✓ ✗ symmetric_random_walk_linear_2 ✓ ✓ symmetric_random_walk_poly_1 ✓ ✗ symmetric_random_walk_poly_2 ✓ ✗
Total ✓ Table 2: 11 programs which are AST andnot necessarily PAST.
Program A
MBER A MBER -L IGHT biased_random_walk_nast_1 ✓ ✓ biased_random_walk_nast_2 ✓ ✓ biased_random_walk_nast_3 ✓ ✓ biased_random_walk_nast_4 ✓ ✓ binomial_nast ✓ ✓ polynomial_nast ✗ ✗
Total ✓ Table 3: 6 programs which are not AST.
Experimental Summary
Our results from Tables 1-3 demonstrate that: – A MBER outperforms the state-of-the-art in automating PAST certification for Prob-solvable loops (Table 1). – Complex probabilistic programs which are AST and not PAST as well as programswhich are not AST can automatically be certified as such by A
MBER (Tables 2, 3). – The relaxations of the proof rules introduced in Section 4 are helpful in automatingthe termination analysis of probabilistic programs, as evidenced by the performanceof A
MBER against A
MBER -L IGHT (Tables 1 - 3).
Proof Rules for Probabilistic Termination
Several proof rules have been proposed inthe literature to provide sufficient conditions for the termination behavior of probabilis-tic programs. The work of [9] uses martingale theory to characterize positive almost suretermination (PAST) . In particular, the notion of a ranking supermartingale (RSM) is intro-duced together with a proof rule (RSM-Rule) to certify PAST, as discussed in Section 3.1.The approach of [18] extended this method to include (demonic) non-determinism andcontinuous probability distributions, showing the completeness of the RSM-Rule for thisprogram class. The compositional approach proposed in [18] was further strengthenedin [28] to a sound approach using the notion of descent supermartingale map . In [1], theauthors introduced lexicographic
RSMs.The SM-Rule discussed in Section 3.2 was introduced in [37]. It is worth mentioningthat this proof rule is also applicable to non-deterministic probabilistic programs. Thework of [27] presented an independent proof rule based on supermartingales with lowerbounds on conditional absolute differences. Both proof rules are based on supermartin-gales and can certify AST for programs that are not necessarily PAST. The approach of [41] examined martingale-based techniques for obtaining bounds on reachability prob-abilities — and thus termination probabilities— from an order-theoretic viewpoint. Thenotions of nonnegative repulsing supermartingales and γ -scaled submartingales , accom-panied by sound and complete proof rules, have also been introduced. The R-AST-Rulefrom Section 3.3 was proposed in [12] mainly for obtaining bounds on the probability ofstochastic invariants.An alternative approach is to exploit weakest precondition techniques for probabilisticprograms, as presented in the seminal works [33,34] that can be used to certify AST. Thework of [36] extended this approach to programs with non-determinism and providedseveral proof rules for termination. These techniques are purely syntax-based. In [30] aweakest precondition calculus for obtaining bounds on the expected termination time ofa program was proposed. This calculus comes with proof rules to reason about loops andcertify PAST. Automation of Martingale Techniques
The work of [9] proposed an automated procedure— by using Farkas’ lemma — to synthesize linear (super)martingales for probabilisticprograms with linear variable updates. This technique was considered in our experimen-tal evaluation, cf. Section 6. The algorithmic construction of supermartingales was ex-tended to treat (demonic) non-determinism in [11] and to polynomial supermartingalesin [10] using semi-definite programming. The recent work of [13] uses ω -regular de-composition to certify AST. They exploit so-called localized ranking supermartingales,which can be synthesized efficiently but must be linear. This approach however cannotprove AST for programs that are not PAST. Other Approaches
Abstract interpretation is used in [38] to prove the probabilistic termi-nation of programs for which the probability of taking a loop k times decreases at leastexponentially with k . In [17], a sound and complete procedure deciding AST is givenfor probabilistic programs with a finite number of reachable states from any initial state.The work of [40] gave an algorithmic approach based on potential functions for com-puting bounds on the expected resource consumption of probabilistic programs. In [35],model checking is exploited to automatically verify whether a parameterized family ofprobabilistic concurrent systems is AST.Finally, the class of Prob-solvable loops considered in this paper extends [4] to a widerclass of loops. While [4] focused on computing statistical higher-order moments, ourwork addresses the termination behavior of probabilistic programs. The related approachof [21] computes exact expected runtimes of constant probability programs and providesa decision procedure for AST and PAST for such programs. Our programming modelstrictly generalizes the constant probability programs of [21], by supporting polynomialloop guards, updates and martingale expressions. This paper reported on the automation of termination analysis of probabilistic while-programs whose guards and expressions are polynomial expressions. To this end, weintroduced mild relaxations of existing proof rules for AST, PAST, and their negations, utomated Termination Analysis of Polynomial Probabilistic Programs 29 by requiring their sufficient conditions to hold only eventually. The key to our approach isthat the structural constraints of Prob-solvable loops allow for automatically computingalmost sure asymptotic bounds on polynomials over program variables. Prob-solvableloops cover a vast set of complex and relevant probabilistic processes including randomwalks and dynamic Bayesian networks [5]. Only two out of 50 benchmarks in [9,40]are outside the scope of Prob-solvable loops regarding PAST certification. The almostsure asymptotic bounds were used to formalize algorithmic approaches for proving AST,PAST, and their negations. Moreover, for Prob-solvable loops four different proof rulesfrom the literature uniformly come together in our work.Our approach is implemented in the software tool A
MBER ( https://github.com/probing-lab/amber ),offering a fully automated approach to probabilistic termination. Our experimental re-sults show that our relaxed proof rules enable proving probabilistic (non-)terminationof more programs than could be treated before. A comparison to the state-of-art in au-tomated analysis of probabilistic termination reveals that A MBER significantly outper-forms related approaches. To the best of our knowledge, A
MBER is the first tool to auto-mate AST, PAST, non-AST and non-PAST in a single tool-chain.There are several directions for future work. These include extensions to Prob-solvableloops such as symbolic distributions, more complex control flow, and non-determinism.We will also consider program transformations that translate programs into our format.Extensions of the SM-Rule algorithm with non-constant probability and decrease func-tions are also in our interest.
References
1. Agrawal, S., Chatterjee, K., Novotný, P.: Lexicographic ranking supermartingales: anefficient approach to termination of probabilistic programs. Proceedings of the ACM on Pro-gramming Languages (POPL), 34:1–34:32 (Dec 2017). https://doi.org/10.1145/3158122, https://doi.org/10.1145/3158122
2. Arora, N.S., Russell, S.J., Sudderth, E.B.: NET-VISA: Network Processing VerticallyIntegrated Seismic Analysis. Bulletin of the Seismological Society of America (2A), 709(2013). https://doi.org/10.1785/01201201073. Avanzini, M., Lago, U.D., Yamada, A.: On probabilistic term rewriting. Sci.Comput. Program. (2020). https://doi.org/10.1016/j.scico.2019.102338, https://doi.org/10.1016/j.scico.2019.102338
4. Bartocci, E., Kovács, L., Stankovic, M.: Automatic generation of moment-based invariantsfor prob-solvable loops. In: Proc. of ATVA 2019: the 17th International Symposium onAutomated Technology for Verification and Analysis. LNCS, vol. 11781, pp. 255–276.Springer, Berlin, Heidelberg (2019). https://doi.org/10.1007/978-3-030-31784-3_155. Bartocci, E., Kovács, L., Stankoviˇc, M.: Analysis of bayesian networks via prob-solvableloops (2020)6. Bistline, J.E., Blum, D.M., Rinaldi, C., Shields-Estrada, G., Hecker, S.S., Paté-Cornell, M.E.:A Bayesian Model to Assess the Size of North Korea’s Uranium Enrichment Program. Science& Global Security (2), 71–100 (2015). https://doi.org/10.1080/08929882.2015.10394317. Bournez, O., Garnier, F.: Proving positive almost-sure termination. In: Giesl,J. (ed.) Proc. of RTA 2005: the 16th International Conference on Rewrit-ing Techniques and Applications. LNCS, vol. 3467, pp. 323–337. Springer,Berlin, Heidelberg (2005). https://doi.org/10.1007/978-3-540-32033-3_24, https://doi.org/10.1007/978-3-540-32033-3_24 https://doi.org/10.1007/b105073
9. Chakarov, A., Sankaranarayanan, S.: Probabilistic Program Analysis with Mar-tingales. In: Proc. of CAV 2013: the 25th International Conference on Com-puter Aided Verification. LNCS, vol. 8044, pp. 511–526. Springer-Verlag,Berlin, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39799-8_34, http://dx.doi.org/10.1007/978-3-642-39799-8_34
10. Chatterjee, K., Fu, H., Goharshady, A.K.: Termination Analysis of Probabilistic ProgramsThrough Positivstellensatz’s. In: Proc. of CAV 2016: the 28th International Conference onComputer Aided Verification. LNCS, vol. 9779, pp. 3–22. Springer International Publishing,Cham (2016). https://doi.org/10.1007/978-3-319-41528-4_111. Chatterjee, K., Fu, H., Novotný, P., Hasheminezhad, R.: Algorithmic Analysis of Qualitativeand Quantitative Termination Problems for Affine Probabilistic Programs. ACM Transac-tions on Programming Languages and Systems (TOPLAS) (2), 7:1–7:45 (May 2018).https://doi.org/10.1145/3174800, https://doi.org/10.1145/3174800
12. Chatterjee, K., Novotný, P., Zikelic, D.: Stochastic Invariants for Probabilis-tic Termination. In: Proc. of POPL 2017: the 44th ACM SIGPLAN Sympo-sium on Principles of Programming Languages. POPL 2017, vol. 52, pp. 145–160. ACM, New York, NY, USA (2017). https://doi.org/10.1145/3009837.3009873, http://doi.acm.org/10.1145/3009837.3009873
13. Chen, J., He, F.: Proving almost-sure termination by omega-regular decomposition.In: Donaldson, A.F., Torlak, E. (eds.) Proc. of PLDI 2020: the 41st ACM SIGPLANInternational Conference on Programming Language Design and Implementation. pp.869–882. ACM, New York, NY, USA (2020). https://doi.org/10.1145/3385412.3386002, https://doi.org/10.1145/3385412.3386002
14. Cook, B., Podelski, A., Rybalchenko, A.: Terminator: Beyond Safety. In: Ball, T.,Jones, R.B. (eds.) Proc. of CAV 2006: the 18th International Conference on ComputerAided Verification. LNCS, vol. 4144, pp. 415–418. Springer, Berlin, Heidelberg (2006).https://doi.org/10.1007/11817963_3715. Cook, B., Podelski, A., Rybalchenko, A.: Proving program termination. Com-mun. ACM (5), 88–98 (2011). https://doi.org/10.1145/1941487.1941509, https://doi.org/10.1145/1941487.1941509
16. Dal Lago, U., Grellois, C.: Probabilistic termination by monadic affine sized typing. ACMTrans. Program. Lang. Syst. (2), 10:1–10:65 (2019). https://doi.org/10.1145/3293605, https://doi.org/10.1145/3293605
17. Esparza, J., Gaiser, A., Kiefer, S.: Proving Termination of Probabilistic Programs UsingPatterns. In: Madhusudan, P., Seshia, S.A. (eds.) Proc. of CAV 2012: the 24th InternationalConference on Computer Aided Verification. LNCS, vol. 7358, pp. 123–138. Springer, Berlin,Heidelberg (2012). https://doi.org/10.1007/978-3-642-31424-7_1418. Ferrer Fioriti, L.L.M., Hermanns, H.: Probabilistic Termination: Soundness, Com-pleteness, and Compositionality. In: Proc. of POPL ’15: the 42Nd Annual ACMSIGPLAN-SIGACT Symposium on Principles of Programming Languages. pp. 489–501. ACM, New York, NY, USA (2015). https://doi.org/10.1145/2676726.2677001, http://doi.acm.org/10.1145/2676726.2677001
19. Fremont, D.J., Dreossi, T., Ghosh, S., Yue, X., Sangiovanni-Vincentelli, A.L., Seshia, S.A.:Scenic: a language for scenario specification and scene generation. In: Proc. of PLDI: the40th ACM SIGPLAN Conference on Programming Language Design and Implementation.pp. 63–78. ACM, Phoenix, AZ, USA (Jun 2019). https://doi.org/10.1145/3314221.3314633, https://doi.org/10.1145/3314221.3314633 utomated Termination Analysis of Polynomial Probabilistic Programs 3120. Giesl, J., Aschermann, C., Brockschmidt, M., Emmes, F., Frohn, F., Fuhs, C., Hensel,J., Otto, C., Plücker, M., Schneider-Kamp, P., Ströder, T., Swiderski, S., Thiemann,R.: Analyzing program termination and complexity automatically with aprove. J.Autom. Reasoning (1), 3–31 (2017). https://doi.org/10.1007/s10817-016-9388-y, https://doi.org/10.1007/s10817-016-9388-y
21. Giesl, J., Giesl, P., Hark, M.: Computing expected runtimes for constant probability programs.In: Proc. of CADE 27: the 7th International Conference on Automated Deduction. LNCS, vol.11716, pp. 269–286. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-29436-6_1622. Gruntz, D.: On computing limits in a symbolic manipulation system. Ph.D. thesis, ETHZürich (1996)23. Hark, M., Frohn, F., Giesl, J.: Polynomial loops: Beyond termination. In: Proc. of LPAR2020: the 23rd International Conference on Logic for Programming, Artificial Intelligenceand Reasoning. EPiC Series in Computing, vol. 73, pp. 279–297. EasyChair, Manchester, UK(2020). https://doi.org/10.29007/nxv124. Hark, M., Kaminski, B.L., Giesl, J., Katoen, J.P.: Aiming Low Is Harder – In-duction for Lower Bounds in Probabilistic Program Verification. Proc. ACMProgram. Lang. (POPL), 37:1–37:28 (2020). https://doi.org/10.1145/3371105, https://doi.org/10.1145/3371105
25. Heizmann, M., Chen, Y., Dietsch, D., Greitschus, M., Hoenicke, J., Li, Y., Nutz, A.,Musa, B., Schilling, C., Schindler, T., Podelski, A.: Ultimate automizer and the searchfor perfect interpolants - (competition contribution). In: Beyer, D., Huisman, M. (eds.)Proc. of TACAS 2018: the 24th International Conference on Tools and Algorithms for theConstruction and Analysis of Systems. LNCS, vol. 10806, pp. 447–451. Springer, Cham(2018). https://doi.org/10.1007/978-3-319-89963-3_3026. Hoare, C.A.R.: An Axiomatic Basis for Computer Programming. Commun. ACM (10),576–580 (1969). https://doi.org/10.1145/363235.36325927. Huang, M., Fu, H., Chatterjee, K.: New Approaches for Almost-Sure Termina-tion of Probabilistic Programs. In: Ryu, S. (ed.) Proc. of APLAS 2018: the AsianSymposium on Programming Languages and Systems on Programming Languagesand Systems. pp. 181–201. LNCS, Springer International Publishing, Cham (2018).https://doi.org/10.1007/978-3-030-02768-1_1128. Huang, M., Fu, H., Chatterjee, K., Goharshady, A.K.: Modular verification for almost-suretermination of probabilistic programs. Proc. ACM Program. Lang. (OOPSLA) (Oct 2019).https://doi.org/10.1145/3360555, https://doi.org/10.1145/3360555
29. Kaminski, B.L., Katoen, J.P.: On the hardness of almost-sure termination. In: Italiano, G.F.,Pighizzini, G., Sannella, D.T. (eds.) Proc. of MFCS 2015: the 40th International Symposiumon Mathematical Foundations of Computer Science 2015. LNCS, vol. 9234, pp. 307–318.Springer, Berlin, Heidelberg (2015). https://doi.org/10.1007/978-3-662-48057-1_2430. Kaminski, B.L., Katoen, J., Matheja, C., Olmedo, F.: Weakest precondition reasoningfor expected runtimes of randomized algorithms. J. ACM (5), 30:1–30:68 (2018).https://doi.org/10.1145/3208102, https://doi.org/10.1145/3208102
31. Kauers, M., Paule, P.: The Concrete Tetrahedron: Symbolic Sums, Recurrence Equa-tions, Generating Functions, Asymptotic Estimates. Texts & Monographs in SymbolicComputation, Springer-Verlag, Wien (2011). https://doi.org/10.1007/978-3-7091-0445-3,
32. Kemeny, J.G., Snell, J.L., Knapp, A.W.: Denumerable Markov Chains: with a chap-ter of Markov Random Fields by David Griffeath. Graduate Texts in Mathematics,Springer-Verlag, New York, 2 edn. (1976). https://doi.org/10.1007/978-1-4684-9455-6, (3), 328–350 (1981). https://doi.org/10.1016/0022-0000(81)90036-2, https://doi.org/10.1016/0022-0000(81)90036-2
34. Kozen, D.: A probabilistic PDL. J. Comput. Syst. Sci. (2), 162–178 (1985).https://doi.org/10.1016/0022-0000(85)90012-135. Lengál, O., Lin, A.W., Majumdar, R., Rümmer, P.: Fair termination for parameter-ized probabilistic concurrent systems. In: Proc. of TACAS 2017: the 23rd Inter-national Conference on Tools and Algorithms for the Construction and Analysisof Systems. LNCS, vol. 10205, pp. 499–517. Springer, Berlin, Heidelberg (2017).https://doi.org/10.1007/978-3-662-54577-5_2936. McIver, A., Morgan, C.: Abstraction, Refinement and Proof for Probabilistic Sys-tems. Monographs in Computer Science, Springer, Springer, New York, NY (2005).https://doi.org/10.1007/b138392, https://doi.org/10.1007/b138392
37. McIver, A., Morgan, C., Kaminski, B.L., Katoen, J.P.: A New Proof Rule for Almost-sure Termination. Proc. ACM Program. Lang. (POPL), 33:1–33:28 (Jan 2018).https://doi.org/10.1145/3158121, http://doi.acm.org/10.1145/3158121
38. Monniaux, D.: An abstract analysis of the probabilistic termination of programs. In: Proc.of SAS 2001: the 8th International Symposium on Static Analysis. LNCS, vol. 2126, pp.111–126. Springer-Verlag, Berlin Heidelberg (2001). https://doi.org/10.1007/3-540-47764-039. de Moura, L.M., Bjørner, N.: Z3: an efficient SMT solver. In: Proc. of TACAS2008: the 14th International Conference on Tools and Algorithms for the Con-struction and Analysis of Systems. LNCS, vol. 4963, pp. 337–340. Springer-Verlag, Berlin Heidelberg (2008). https://doi.org/10.1007/978-3-540-78800-3, https://doi.org/10.1007/978-3-540-78800-3_24
40. Ngo, V.C., Carbonneaux, Q., Hoffmann, J.: Bounded expectations: resource analysis for prob-abilistic programs. In: Proceedings of the 39th ACM SIGPLAN Conference on ProgrammingLanguage Design and Implementation. pp. 496–512. PLDI 2018, Association for ComputingMachinery, Philadelphia, PA, USA (Jun 2018). https://doi.org/10.1145/3192366.3192394, https://doi.org/10.1145/3192366.3192394https://doi.org/10.1145/3192366.3192394