An Efficient Model Inference Algorithm for Learning-based Testing of Reactive Systems
AAn Efficient Model Inference Algorithmfor Learning-based Testing of Reactive Systems
Muddassar A. Sindhu
Computer Science Department,Quaid i Azam University, Islamabad 45320, Pakistan,[email protected]
Abstract.
Learning-based testing (LBT) is an emerging methodologyto automate iterative black-box requirements testing of software systems.The methodology involves combining model inference with model check-ing techniques. However, a variety of optimisations on model inferenceare necessary in order to achieve scalable testing for large systems.In this paper we describe the IKL learning algorithm which is an activeincremental learning algorithm for deterministic Kripke structures. Weformally prove the correctness of IKL. We discuss the optimisations itincorporates to achieve scalability of testing. We also evaluate a blackbox heuristic for test termination based on convergence of IKL learning.
A heuristic approach to automated test case generation (ATCG) from formalrequirements specifications known as learning-based testing (LBT) was intro-duced in [13], [14] and [17]. Learning-based testing is an iterative approach toautomate specification-based black-box testing. It encompasses both test casegeneration, execution and evaluation (the oracle step). The aim of LBT is toautomatically generate a large number of high-quality test cases by combininga model checking algorithm with an optimised model inference algorithm (aka.learning algorithm). For both procedural ([14]) and reactive systems ([15], [17])it has been shown that LBT can significantly outperform random testing in thespeed with which it finds errors in a system under test (SUT). This is becauserandom test suites generally contain a large degree of redundancy, which canbe reduced by using learning algorithms and model checkers to execute a moredirected search for software errors.An efficient and practical implementation of learning-based testing for reac-tive systems has been developed in the LBTest tool [18]. In this paper we describethe IKL (Incremental Kripke Learning) algorithm implemented in LBTest. IKLis an algorithm for active incremental learning of deterministic Kripke structures .The reliability of LBTest for producing correct test results depends crucially onthe correctness of this learning algorithm. So we give a formal definition of IKLand prove its correctness. The IKL algorithm involves a number of optimisationsnecessary to achieve scalability of testing for large software systems. We discussthese optimisations from the perspective of learning and testing. a r X i v : . [ c s . F L ] A ug The problems of coverage, and termination criteria for black-box testing,are complex and different solutions have been proposed. In LBT, convergenceof learning can sometimes be used as a criterion to terminate testing. However,heuristics are needed to estimate convergence in the context of black box testing.We will empirically evaluate the reliability of a simple heuristic for IKL.In the remainder of Section 1, we discuss the general paradigm of LBT,and specific requirements on learning for efficient testing of reactive systems. InSection 2, we review some essential mathematical preliminaries. In Section 3, wepresent the architecture of the IKL learning algorithm and its main components.These three main components are defined and analysed in detail in Sections 4, 5and 6. In Section 4, we consider a learning algorithm for families of DFA whichsupports incremental learning and projection (to be discussed in Section 1.2). InSection 5, we consider integrating a family of DFA into a single Kripke structureusing a subdirect product construction. In Section 6, we consider an efficientminimisation algorithm for deterministic Kripke structures based on Hopcroft’sDFA minimisation algorithm [12]. This is needed by the IKL algorithm to pro-duce hypothesis models that can be efficiently model checked. In Section 7, weempirically evaluate a black box heuristic to detect convergence of IKL, thatcan be used as a test termination criterion. Finally, in Section 8 we draw someconclusions and suggest prospects for further research on learning and testing.
The basic LBT paradigm requires three components:(1) a (black-box) system under test (SUT) S ,(2) a formal requirements specification Req for S , and(3) a learned model M of S .Now (1) and (2) are common to all specification-based testing, and it is really(3) that is distinctive. Learning-based testing is a heuristic iterative method toautomatically generate a sequence of test cases. The heuristic concept is to learna black-box system using tests as queries .In general, an LBT algorithm iterates the following four steps:(Step 1) Suppose that n test case inputs i , . . . , i n have been executed on S yielding the system outputs o , . . . , o n . The n input/output observations ( i , o ) , . . . , ( i n , o n ) can be synthesized into a learned model M n of S usingan incremental learning algorithm (see Section 1.2). This step involves general-ization from the observed behaviour, (which represents an incomplete descriptionof S ) to all possible behaviour. This generalisation step gives the possibility topredict previously unseen errors in S during Step 2.(Step 2) The system requirements Req are checked against the learned model M n derived in Step 1 (aka. model checking ). This process searches for a coun-terexample i n +1 to the requirements.(Step 3) The counterexample i n +1 is executed as the next test case on S , and if S terminates then the output o n +1 is obtained. If S fails this test case (i.e. the observation ( i n +1 , o n +1 ) does not satisfy Req ) then i n +1 was a true negative andwe proceed to Step 4. Otherwise S passes the test case i n +1 so the model M n was inaccurate, and i n +1 was a false negative . In this latter case, the effort ofexecuting S on i n +1 is not wasted. We return to Step 1 and apply the learningalgorithm once again to n + 1 pairs ( i , o ) , . . . , ( i n +1 , o n +1 ) to infer a refinedmodel M n +1 of S .(Step 4) We terminate with a true negative test case ( i n +1 , o n +1 ) for S .Thus an LBT algorithm iterates Steps . . . M , M , M , . . . , of S . (We usually take M tobe a null hypothesis about S .) So, with increasing values of n , it becomes moreand more likely that model checking in Step 2 will produce a true negative ifone exists.Notice, if Step 2 does not produce any counterexamples at all then to proceedwith the next iteration, we must construct the next test case i n +1 by someother method. Now active learning algorithms can be devised to generate queriesthat efficiently learn an unknown system in polynomial time. So for LBT thereis clearly an advantage to combine model checking with active learning andgenerate both types of test cases. More generally, it is useful to have accessto as wide a variety of query generation techniques as possible. So in practice,model checker and active learning queries are augmented with random querieswhen necessary. However, these different types of queries need to be combinedcarefully to achieve efficient and scalable testing. As has already been suggested in Section 1.1, for LBT to be effective at findingerrors, it is important to use the right kind of learning algorithm. As well asactive learning, several other principles for efficient testing can be found. Tomotivate the design of the IKL algorithm we will discuss two of them. For thispurpose, we focus specifically on automata learning for testing reactive systems .(LBT has also been successfully applied to testing other types of systems, seee.g. [14]). Learning algorithms for automata are also known as regular inferencealgorithms in the literature (e.g. [8]).
Incremental Learning
One efficiency principle is that a good learning algo-rithm should maximise the opportunity of the model checker in Step 2 aboveto find a true counterexample i n +1 to the requirements Req as soon as possible.
An automata learning algorithm L is said to be incremental if it can producea sequence of hypothesis automata A , A , . . . which are approximations to anunknown automata A , based on a sequence of observations of the input/outputbehaviour of A . The sequence A , A , . . . must finitely converge to A , at leastup to behavioural equivalence. In addition, the computation of each new ap-proximation A i +1 by L should reuse as much information as possible aboutthe previous approximation A i (e.g. equivalences between states). Incrementallearning algorithms are necessary for two reasons.(1) Real world systems are often too big to be completely learned and testedwithin a feasible timescale. This is mainly due to: (i) the time complexity oflearning and model checking algorithms, and (ii) the time needed to execute theindividual test cases on a large SUT.(2) Testing of specific requirements such as use cases may not require learningand analysing the entire SUT S , but only the relevant fragment of S whichimplements the requirement Req .For these two reasons, the IKL learning algorithm used in LBTest is basedon incremental learning.This concept of a relevant fragment of an SUT for testing a requirement
Req raises the question of the relative efficiency of different types of queries (testcases). We have already seen that in LBT, test cases can be generated by modelchecking, by active learning, or by some other process entirely such as randomquerying.As indicated in (1) above, the overhead of SUT execution time to answer anindividual query can be large compared with the execution time of learning andmodel checking. There are examples of industrial systems where this executiontime is of the order of minutes. So realistically, queries should be seen as “ex-pensive”. From the viewpoint of relevance therefore, as many queries as possibleshould be derived from model checking the hypothesis automaton, since thesequeries are all based on checking the requirements
Req . Conversely as few queriesas possible should be derived from the active learning algorithm. Active learningqueries have no way to reference the requirement
Req , and therefore can onlyuncover an SUT error by accident. Furthermore, active learning queries mayexplore parts of the SUT which are irrelevant to checking
Req , thereby leadingthe search for errors in a fruitless direction. Ideally, every query would representa relevant and interesting requirements-based test case.However, there is conflicting issue involved here, which is the computationaleffort needed to generate different types of queries. Model checker generatedqueries are generally computationally expensive relative to active learner gen-erated queries, often by several orders of magnitude. Therefore, if too many(perhaps even all) queries are generated by model checking, then the LBT pro-cess may slow so much that random testing is simply faster. In a practical LBTtool, the ratio between the number of model checker generated queries, and thenumber of active learning queries must be controlled to achieve a balance be-tween relevance and speed. The IKL algorithm implements a pragmatic balance between these two types of queries that we have found to be reasonably efficientin practise.Interestingly, when the balance of active learning queries becomes very high,and model checking queries are almost eliminated, we might think that LBTbecomes similar to random testing. However [26] shows that this is not thecase. Thus using active learner queries alone, LBT can achieve better functionalcoverage than random testing.
Projection
When we consider the output variables of the SUT S that appearin a specific formal black box requirement Req , we often see just a small subsetof the set of all output variables of S . This observation points to a powerful ab-straction technique for learning that can be termed bit-slicing (for propositionalvariables) or more generally projection .Like incremental learning, projection is another abstraction method that con-centrates on learning only the relevant SUT behavior needed to test the require-ment Req . Essentially, projection involves learning a quotient model of the SUTby observing just the output variables appearing in
Req . Since quotient modelsof S may be dramatically smaller than S itself, the time needed for learningand testing may be considerably reduced. Therefore, projection seems to be anessential component of a scalable LBT system. Indeed, the combination of in-cremental learning and projection seems to be particularly powerful. The IKLalgorithm incorporates both these features, and they will be discussed in furtherdetail in Sections 3 and 4. Several previous works, (for example Peled et al. [22], Groce et al. [11] and Raf-felt et al. [23]) have considered a combination of learning and model checking toachieve testing and/or formal verification of reactive systems. Within the modelchecking community the verification approach known as counterexample guidedabstraction refinement (CEGAR) also combines learning and model checking,(see e.g. Clarke et al. [7] and Chauhan et al. [6]). The LBT approach can bedistinguished from these other approaches by: (i) an emphasis on testing ratherthan verification, and (ii) the use of incremental learning and other abstrac-tion techniques specifically chosen to achieve scalable testing and faster errordiscovery (c.f. Section 1.2).In practise, most of the well-known classical regular inference algorithms suchas L* (Angluin [2]) or ID (Angluin [1]) are designed for complete rather thanincremental learning. Among the much smaller number of known incrementallearning algorithms, we can mention the RPNII algorithm (Dupont [9]) and theIID algorithm (Parekh et al. [21]) which learn Moore automata, and the ICGEalgorithm (Meinke and Fei [16]) which learns Mealy automata over abstractdata types. No algorithm which combines incremental learning and projectionhas been published in the literature. The problem of integrating active learningqueries with model checker generated queries (which in some sense take over the role of Angluin’s equivalence checker [2]) has also not been considered. Thus:(i) the design of the IKL algorithm , (ii) its formal proof of correctness , and(iii) its motivation by efficient test case generation represent the main novelcontributions of our paper.The use of minimisation algorithms in automata learning also seems notto have been considered. This is mainly because most DFA learning algorithmsnaturally infer the canonical minimal automaton. However, our use of projectionas an abstraction method for learning large Kripke structures does not leadimmediately to minimal structures. In fact, inferring non-minimal automata caneven lead to efficiency gains as we have shown elsewhere in [16].For different automata models and different notions of equivalence, the com-plexity of the minimisation problem can vary considerably. The survey [3] con-siders minimisation algorithms for DFA up to language equivalence, with timecomplexities varying between O ( n ) and O ( n log n ) . Kripke structures representa generalisation of DFA to allow non-determinism and multiple outputs. Theyhave been widely used to model concurrent and embedded systems. An algo-rithm for mimimizing Kripke structures has been given in [5]. In the presenceof non-determinism, the complexity of minimisation is quite high. Minimisationup to language equivalence requires exponential time, while minimisation up toa weaker simulation equivalence can be carried out polynomial time (see [5]).By contrast, we will show that deterministic Kripke structures can be efficientlyminimized even up to language equivalence with a worst case time complexity of O ( kn log n ) . Our generalisation of Hopcroft’s DFA minimisation algorithm todeterministic Kripke structures in Section 6 is fairly simple and straightforward.Nevertheless, this algorithm has not been previously published in the literature,and represents another novel contribution. In this section we introduce some basic concepts and notations needed to defineand prove the correctness of the IKL learning algorithm. Let Σ be any set ofsymbols then Σ ∗ denotes the set of all finite strings over Σ including the emptystring ε . The length of a string α ∈ Σ ∗ is denoted by | α | and | ε | = 0 . For strings α, β ∈ Σ ∗ , α . β denotes their concatenation.For α, β, γ ∈ Σ ∗ , if α = βγ then β is termed a prefix of α and γ is termeda suffix of α . We let Pref ( α ) denote the prefix closure of α , i.e. the set of allprefixes of α . We can also apply prefix closure pointwise to any set of strings.The set difference operation between two sets U, V , denoted by U − V , is theset of all elements of U which are not members of V . The symmetric difference operation on pairs of sets is defined by U ⊕ V = ( U − V ) ∪ ( V − U ) .A deterministic finite automaton (DFA) is a five-tuple A = ( Σ, Q, F, q , δ ) where: Σ is the input alphabet, Q is the state set, F ⊆ Q is the acceptingstate set and q ∈ Q is the starting state. The state transition function of A is amapping δ : Q × Σ → Q with the usual meaning, and can be inductively extended to a mapping δ ∗ : Q × Σ ∗ → Q where δ ∗ ( q, ε ) = q and δ ∗ ( q, σ , . . . σ n +1 ) = δ ( δ ∗ ( q, σ , . . . σ n ) , σ n +1 ) .A dead state is a state from which no accepting state can be reached, anda state which is not dead is termed live . Since input strings can be used toname states, given any distinguished dead state d we define string concatenationmodulo the dead state d , f : Σ ∗ ∪ { d } × Σ → Σ ∗ ∪ { d } , by f ( d , σ ) = d and f ( α, σ ) = α . σ for α ∈ Σ ∗ . This function is used for automaton learningin Section 4.The language L ( A ) accepted by A is the set of all strings α ∈ Σ ∗ such that δ ∗ ( q , α ) ∈ F . A language L ⊆ Σ ∗ is accepted by a DFA if and only if, L is regular , i.e. L can be defined by a regular grammar.A generalisation of DFA to allow multi-bit outputs on states is given bydeterministic Kripke structures. Let Σ = { σ , . . . , σ n } be a finite input alphabet. By a k -bit deterministic Kripke structure A we mean a five-tuple A = ( Q A , Σ, δ A : Q A × Σ → Q A , q A , λ A : Q A → B k ) where Q A is a state set, δ A is the state transition function, q A is the initial stateand λ A is the output function.As before we let δ ∗A : Q A × Σ ∗ → Q A denote the iterated state transition func-tion, where δ ∗A ( q, ε ) = q and δ ∗A ( q, σ , . . . , σ i +1 ) = δ A ( δ ∗A ( q, σ , . . . , σ i ) , σ i +1 ) .Also we let λ ∗A : Σ ∗ → B k denote the iterated output function λ ∗A ( σ , . . . , σ i ) = λ A ( δ ∗A ( q A , σ , . . . , σ i )) . More generally for any q ∈ Q define λ ∗ q ( σ , ..., σ i ) = λ A ( δ ∗A ( q, σ , . . . , σ i )) . Given any R ⊆ Q we write λ ( R ) = ∪ r ∈ R λ ( r ) . We let q.σ denote δ ( q, σ ) and R.σ denotes { r.σ | r ∈ R } for R ⊆ Q .Note that a -bit deterministic Kripke structure A is isomorphic to a DFA A (cid:48) = ( Q A (cid:48) , Σ, δ A (cid:48) : Q A (cid:48) × Σ → Q A (cid:48) , q A (cid:48) , F A (cid:48) ) , where F A (cid:48) ⊆ Q A (cid:48) and λ A (cid:48) ( q ) = true if, and only if q ∈ F A (cid:48) .In the context of Boolean valued output variables, the concept of projectionon a set of output variables will also be termed bit slicing . Let us make precisethe concept of a bit-slice or projection of a Kripke structure. Let A be a k -bit Kripke structure over a finite input alphabet Σ , A = ( Q A , Σ, δ A : Q A × Σ → Q A , q A , λ A : Q A → B k ) . For each ≤ i ≤ k define the i -th projection A i of A to be the -bit Kripkestructure where A i = ( Q A , Σ, δ A : Q A × Σ → Q A , q A , λ A i : Q A → B ) , and λ A i ( q ) = λ A ( q ) i , i.e. λ A i ( q ) is the i -th bit of λ A ( q ) .A family of k individual -bit Kripke structures can be combined into asingle k -bit Kripke structure using a subdirect product construction. This willbe discussed in Section 5. A Kripke structure A is minimal if it has no proper subalgebra. This isequivalent to all states of A being reachable from the initial state by meansof some input string. If A is a Kripke structure then A always has a minimalsubalgebra which we denote by M in ( A ) . As discussed in Section 1, IKL is an algorithm for incrementally inferring a de-terministic k -bit Kripke structure from observational data. For efficient testing,it also implements projection on output variables. An architectural view of theIKL algorithm is given in Figure 1. The basic idea of the algorithm is to learn a k -bit Kripke structure as a family of k -bit Kripke structures (i.e. DFA) usingan incremental DFA learning algorithm for each of the k individual DFA. State Space Minimisa-on Subdirect Product L a z y i n c r e m e n t a l D F A l e a r n i n g L a z y i n c r e m e n t a l D F A l e a r n i n g L a z y i n c r e m e n t a l D F A l e a r n i n g k -‐bit vector from teacher bit 1 bit 2 bit k minimised k -‐bit Kripke structure DFA 1 DFA 2 DFA k k -‐bit Kripke structure Fig. 1.
Architecture of the IKL algorithm.
For DFA learning, we use an incremental refinement of Angluin’s ID algo-rithm [1]. The dead state d used in the ID algorithm can be used in incrementallearning as an abstraction for all currently unknown information about the sys-tem to be learned. Our refinement of ID differs from the IID learning algorithmdescribed in [21] in ways which have been discussed in [24]. Note that the IKLarchitecture is modular, in the sense that other DFA learning algorithms could be used instead of ID. These might alter the overall performance of IKL from atesting perspective (see Section 8).As Figure 1 indicates, these k incremental DFA learning algorithms mustco-operate in order to jointly learn an entire family of DFA. This co-operationbetween the DFA learners is termed lazy learning . It is used to support more fre-quent model checking during testing, which is desirable for the reasons explainedin Section 1.2. The goal of lazy learning then is to learn the DFA family in away that can produce new k -bit hypothesis Kripke structures with maximumfrequency.The k individual DFA are assembled into a single k -bit Kripke structureusing a generalisation of the direct product construction known as a subdirectproduct . Without minimisation, the state space of the product automaton wouldbe very large. However, the state space can be reduced on the fly, resultingin a subalgebra of the direct product. This removes all states which are notreachable from the initial state, using some input string. The state space ofthis subdirect product is typically still very large. In order to minimise thestate space even further, we finally apply a minimisation algorithm for Kripkestructures. For this we adapt Hopcroft’s minimisation algorithm for DFA [12],and generalise it to Kripke structures. We will discuss the state space sizesachieved by the intermediate Kripke structures during the incremental learningprocess in Section 7.In the next three sections we define and prove correct the three major com-ponents of the IKL algorithm:(i) the DFA family learning algorithm F ID (Section 4),(ii) the subdirect product construction (Section 5), and(iii) a Kripke structure minimisation algorithm (Section 6)
In this section we define and prove correct an algorithm
F ID for incrementallazy learning of a family of DFA that share a common input. This approachsupports bit-sliced learning of a large Kripke structure by projection of specificoutput variables (c.f. Section 1.2). Our algorithm is derived from the ID learningalgorithm for DFA described in [1], and our correctness proof makes use of thecorrectness property of ID . Therefore, we begin by reviewing the ID algorithmitself, before turning our attention to DFA family learning. ID Algorithm
The ID algorithm and its correctness have been discussed at length in [1]. There-fore our own presentation can be brief. A finite set P ⊆ Σ ∗ of input strings issaid to be live complete for a DFA A if for every live state q ∈ Q there exists astring α ∈ P such that δ ∗ ( q , α ) = q . Given a live complete set P for a targetautomaton A , the essential idea of the ID algorithm is to first construct the set T (cid:48) = P ∪ { f ( α, b ) | ( α, b ) ∈ P × Σ } ∪ { d } of all one element extensions of stringsin P as a set of state names for the hypothesis automaton.A symbol d is added as a name for the canonical dead state. Now this deadstate can be used for incremental learning of a DFA, since parts of the DFAwhich have not yet been learned can be "hidden" inside the dead state. This isa key idea in the F ID incremental learning algorithm described in Section 4.2The set of state names is then iteratively partitioned into sets E i ( α ) ⊆ T (cid:48) for i = 0 , , . . . such that elements α , β of T (cid:48) that denote the same state in A willoccur in the same partition set, i.e. E i ( α ) = E i ( β ) . This partition refinement canbe proven to terminate and the resulting collection of sets forms a congruenceon T (cid:48) . Finally the ID algorithm constructs the hypothesis DFA as the result-ing quotient DFA. The method used to refine the partition set is to iterativelyconstruct a set V of distinguishing strings , such that no two distinct states of A have the same behaviour on all of V .In Section 4.2, we present the DFA family learning algorithm F ID so thatsimilar variables in the
F ID and ID algorithms share similar names. This ped-agogic device emphasises some similarity in the behaviour of both algorithms.However, there are also important differences of behaviour. Thus, when analysingthe behavioural properties of similar program variables, we will try to distinguishtheir context as v ID n , E IDn ( α ) , . . . etc, (for the ID algorithm) and correspondingly v cn , E cn ( α ) , . . . etc, (for the F ID algorithm). Our basic argument in the proofof correctness of
F ID is to show how the learning behaviour of
F ID on a se-quence of input strings s , . . . s n ∈ Σ ∗ can be simulated by the behaviour of ID on the prefix closure Pref ( { s , . . . s n } ) of the corresponding set of inputs { s , . . . s n } . Once this is established one can apply the correctness of ID toestablish the correctness of F ID . The correctness of the ID algorithm can bestated as follows. (i) Let P ⊆ Σ ∗ be a live complete set for a DFA A containing λ . Then given P and A as input, the ID algorithm terminates and the automaton M returned isthe canonical minimum state automaton for L ( A ) .(ii) Let l ∈ N be the maximum value of program variable i ID given P and A .For all ≤ n ≤ l and for all α ∈ T , E IDn ( α ) = { v ID j ∈ V ID | ≤ j ≤ n, αv ID j ∈ L ( A ) } . Proof. (i) See [1] Theorem 3.(ii) By induction on n . Basis.
Suppose n = 0 . Then v ID = λ . For any α ∈ T , if αv ID ∈ L ( A ) then α ∈ L ( A ) so E ID ( α ) = { v ID } . If αv ID (cid:54)∈ L ( A ) then α (cid:54)∈ L ( A ) so E ID ( α ) = ∅ .Thus E ID ( α ) = { v ID j | ≤ j ≤ , αv IDj ∈ L ( A ) } . Induction Step.
Suppose l ≥ n > . Consider any α, β ∈ P (cid:48) and b ∈ Σ suchthat E IDn − ( α ) = E IDn − ( β ) but E IDn − ( f ( α, b )) (cid:54) = E IDn − ( f ( β, b )) . Since n − < l Algorithm 1 ID Learning AlgorithmInput : A live complete set P ⊆ Σ ∗ and a DFA A to act as a teacher answeringmembership queries α ∈ L ( A )? Output: A DF A M language equivalent to the target DFA A .1. begin / /Perform Initialization3. i = 0 , v i = λ, V = { v i } , T = P ∪ { f ( α, b ) | ( α, b ) ∈ P × Σ } , T (cid:48) = T ∪ { d }
5. Construct function E for v = λ, E ( d ) = ∅ ∀ α ∈ T
8. { pose the membership query “ α ∈ L ( A )?” if the teacher’s response is yes then E ( α ) = { λ } else E ( α ) = ∅ end if
13. }14. //Refine the partition of the set T (cid:48) while ( ∃ α, β ∈ P (cid:48) and b ∈ Σ such that E i ( α ) = E i ( β ) but E i ( f ( α, b )) (cid:54)(cid:54) = E i ( f ( β, b ))) do
17. Let γ ∈ E i ( f ( α, b )) ⊕ E i ( f ( β, b )) v i +1 = bγ V = V ∪ { v i +1 } , i = i + 1 ∀ α ∈ T k pose the membership query ” αv i ∈ L ( A )?”
21. {22. if the teacher’s response is yes then E i ( α ) = E i − ( α ) ∪ { v i } else E i ( α ) = E i − ( α ) end if
26. }27. end while
28. //Construct the representation M of the target DFA A .29. The states of M are the sets E i ( α ) , where α ∈ T
30. The initial state q is the set E i ( λ )
31. The accepting states are the sets E i ( α ) where α ∈ T and λ ∈ E i ( α )
32. The transitions of M are defined as follows:33. ∀ α ∈ P (cid:48) if E i ( α ) = ∅ then add self loops on the state E i ( α ) for all b ∈ Σ else ∀ b ∈ Σ set the transition δ ( E i ( α ) , b ) = E i ( f ( α, b )) end if end. then α , β and b exist. Then E IDn − ( f ( α, b )) ⊕ E IDn − ( f ( β, b )) (cid:54) = ∅ . Consider any γ ∈ E IDn − ( f ( α, b )) ⊕ E IDn − ( f ( β, b )) and let v ID n = bγ . For any α ∈ T , if αv ID n ∈ L ( A ) then E IDn ( α ) = E IDn − ( α ) ∪ { v ID n } and if αv ID n (cid:54)∈ L ( A ) then E IDn ( α ) = E IDn − ( α ) . So by the induction hypothesis E IDn ( α ) = { v ID j ∈ V ID | ≤ j ≤ n, αv ID j ∈ L ( A ) } . F ID
Algorithm
We can now present the
F ID algorithm for incremental lazy learning of a familyof DFA. We give a rigorous proof that
F ID correctly learns in the limit in thesense of [10] (Correctness Theorem 4.2.6).Algorithm 2 is the main component of the
F ID algorithm. It learns a se-quence F , . . . , F l of families F i = ( M i , . . . , M ni ) of n DFA driven by a se-quence of input strings (queries) s , . . . , s l . The teacher is a single n -bit Kripkestructure A . Then F is a null hypothesis about the projections A , . . . , A n of A . We claim that the sequence F , F , . . . finitely converges to the projections ( A , . . . , A n ) given enough information about A , i.e when s , . . . , s l containsa live complete set of queries for each projection A i .The basic idea of Algorithm 2 is to construct in parallel a family ( E i , . . . , E ni n ) of n individual equivalence relations on the same set T k of state names. For eachequivalence relation E ji j , a set V j of distinguishing strings is incrementally gen-erated to split pairs of equivalence classes in E ji j until a congruence is achieved.Then a quotient DFA M j can be constructed from the partition of T k by thecongruence E ji j . The congruences are constructed so that E ji ⊆ E ji +1 and thusthe F ID algorithm is incremental, and fully reuses information about previousapproximations, which is efficient.Each DFA family F i = ( M i , . . . , M ni ) is constructed from the partitionfamily ( E i , . . . , E ni n ) using Synthesis Algorithm 4. When the F ID algorithmis applied to the problem of LBT, the input strings s i ∈ Σ ∗ to F ID are generatedas counterexamples to correctness (i.e. test cases). For this we execute a modelchecker on a Kripke structure A i − which is a minimised subdirect product of ( M i − , . . . , M ni − ) using a requirements specification φ expressed in temporallogic. (The construction of A i − will be detailed in Sections 5 and 6.) In the casethat no counterexamples to φ can be found in A i − then s i is randomly chosen,taking care to avoid all previously used input strings.Algorithm 3 implements lazy partition refinement, to extend E i , . . . , E ni n from being equivalence relations on states to being a family of congruences withrespect to the state transition functions δ , . . . , δ n for the synthesized DFA M , . . . , M n .Thus line 1 in Algorithm 3 searches for congruence failure in any one ofthe equivalence relations E i , . . . , E ni n . In lines 6-14 of Algorithm 3 we apply Algorithm 2 FID: a DFA Family Learning AlgorithmInput : A file S = s , . . . , s l of input strings s i ∈ Σ ∗ and an n -bit Kripkestructure A as teacher to answer queries λ ∗A ( s i ) = ? Output:
A sequence of families F t = ( M t , . . . , M nt ) of DFA for t = 0 , . . . , l .1. begin / /Perform Initialization3. for c = 1 to n do { i c = 0 , v ci c = ε , V c = { v ci c } } k = 0 , t = 0 , P = { ε } , P (cid:48) = P ∪ { d } , T = P ∪ Σ
6. //Build equivalence classes for the dead state d for c = 1 to n do { E c ( d ) = ∅ }
8. //Build equivalence classes for input strings of length zero and one9. ∀ α ∈ T { ( b , . . . , b n ) = λ ∗A ( α ) for c = 1 to n do if b c then E ci c ( α ) = { v ci c } else E ci c ( α ) = ∅
13. }14. //Refine the initial equivalence relations E , . . . , E n
15. //into congruences using Algorithm 316.17. //Synthesize an initial family F approximating A
18. //using Algorithm 4.19.20. //Process the file of examples.21. while S (cid:54) = empty do {
22. read( S, α )23. k = k+1, t = t+124. P k = P k − ∪ P ref ( α ) //prefix closure25. P (cid:48) k = P k ∪ { d } T k = P k ∪ { f ( α, b ) | α ∈ P k − P k − , b ∈ Σ } //for prefix closure27. T (cid:48) k = T k ∪ { d } ∀ α ∈ T k − T k − { for c = 1 to n do E c ( α ) = ∅ //initialise the new equivalence class E c ( α ) for j = 0 to i c do {
31. // Consider adding previous distinguishing string v cj ∈ V c
32. // to the new equivalence class E cj ( α ) ( b , . . . , b n ) = λ ∗A ( α . v cj ) if b c then E cj ( α ) = E cj ( α ) ∪ { v cj }
35. }36. }37. //Refine the current equivalence relations E i , . . . , E ni n
38. // into congruences using Algorithm 339.40. if α is consistent with F t − then F t = F t − else synthesize the family F t using Algorithm 4.43. } end. Algorithm 3
Lazy Partition Refinement1. while ( ∃ ≤ c ≤ n, ∃ α, β ∈ P (cid:48) k and ∃ σ ∈ Σ such that E ci c ( α ) = E ci c ( β ) but E ci c ( f ( α, σ )) (cid:54)(cid:54) = E ci c ( f ( β, σ )) do {
2. //Equivalence relation E ci c is not a congruence w.r.t. δ c
3. //so add a new distinguishing sequence.4.
Choose γ ∈ E ci c ( f ( α, σ )) ⊕ E ci c ( f ( β, σ )) v = σ . γ ∀ α ∈ T k { ( b , . . . , b n ) = λ ∗ A ( α . v ) for c = 1 to n do { if E ci c ( α ) = E ci c ( β ) and E ci c ( f ( α, σ )) (cid:54)(cid:54) = E ci c ( f ( β, σ )) then {
10. // Lazy refinement of equivalence relation E ci c i c = i c + 1 , v i c = v , V c = V c ∪ { v i c } if b c then E ci c ( α ) = E ci c − ( α ) ∪ { v i c } else E ci c ( α ) = E ci c − ( α ) }
14. }15. } lazy partition refinement. This technique implies reusing the new distinguishingstring v wherever possible to refine each equivalence relation E ji j that is not yeta congruence. On the other hand, any equivalence relation E ji j that is already acongruence is not refined, even though the result b j of the new query α . v mightadd some new information to M j . This brings the set of relations E i , . . . , E ni n to a simultaneous fixed point of n congruence constructions as soon as possible.It therefore helps to reduce the number of active learner queries and raise thenumber of model checker queries used during learning based testing (cf. Section1.2).We begin an analysis of the correctness of the F ID algorithm by confirmingthat the construction of hypothesis DFA carried out by Algorithm 4 is welldefined.
For each t ≥ the hypothesis DFA M , . . . , M n con-structed by the DFA Family Synthesis Algorithm 4 after t input strings havebeen applied to A are all well defined DFA. Proof.
The main task is to show δ to be well defined function and uniquelydefined for every state E i ( α ) , where α ∈ T k .Proposition 4.2.1 establishes that Algorithm 4 will generate families of welldefined DFA. However, to show that the F ID algorithm learns correctly in thelimit, we must prove that this sequence of DFA families finitely converges to the n individual projections A i of the target Kripke structure A . It will suffice toshow that the behaviour of F ID can be simulated by the behaviour of ID , since ID is known to learn correctly given a live complete set of input strings (c.f.Theorem 4.4.1.(i)). The first step in this proof is to show that the sequences of Algorithm 4
DFA Family Synthesis1. for c = 1 to n do {
2. // Synthesize the quotient DFA M c
3. The states of M c are the sets E ci c ( α ) , where α ∈ T k
4. Let q c = E ci c ( ε )
5. The accepting states are the sets E ci c ( α ) where α ∈ T k and ε ∈ E ci c ( α )
6. The transition function δ c of M c is defined as follows:7. ∀ α ∈ P (cid:48) k { if E ci c ( α ) = ∅ then ∀ b ∈ Σ { let δ c ( E ci c ( α ) , b ) = E ci c ( α ) } else ∀ b ∈ Σ { δ c ( E ci c ( α ) , b ) = E ci c ( α . b ) } } ∀ β ∈ T k − P (cid:48) k { if ∀ α ∈ P (cid:48) k { E ci c ( β ) (cid:54) = E ci c ( α ) } and E ci c ( β ) (cid:54) = ∅ then ∀ b ∈ Σ { δ c ( E ci c ( β ) , b ) = ∅ } } } return F = ( M , . . . , M n ) sets of state names P k and T k generated by F ID converge to the sets P ID and T ID of ID . Let S = s , . . . , s l be any non-empty sequence of inputstrings s i ∈ Σ ∗ for F ID and let P ID = Pref ( { λ, s , . . . , s l } ) be the prefixclosure of the corresponding input set for ID .(i) For all ≤ k ≤ l , P k = Pref ( { λ, s , . . . , s k } ) ⊆ P ID .(ii) For all ≤ k ≤ l , T k = P k ∪ { f ( α, b ) | α ∈ P k , b ∈ Σ } ⊆ T ID .(iii) P l = P ID and T l = T ID . Proof.
Clearly (iii) follows from (i) and (ii). Then (i) and (ii) are easily provedby induction on k .Observe that unlike F ID , the ID algorithm does not compute any prefixclosure of input strings. Therefore, prefix closure must be added explicitly inProposition 4.2.2, to make a correspondence between the behaviour of F ID and ID .Next we turn our attention to proving some fundamental loop invariants forAlgorithm 2. Since this algorithm in turn calls the Lazy Partition RefinementAlgorithm 3 then we have in effect a doubly nested loop structure to analyse.Clearly the outer loop counter k in Algorithm 2 and the family of inner loopcounters i c (for ≤ c ≤ n ) in Algorithm 3 both increase on each iteration.However, the relationships between these counter variables are not easily defined.Nevertheless, since all variables increase from an initial value of zero, we canassume the existence of some family of n monotone re-indexing functions thatcapture their relationships. Let S = s , . . . , s l be any non-empty sequence of strings s i ∈ Σ ∗ . The re-indexing function K Sc : N → N for F ID on input S (for each ≤ c ≤ n ) is the unique monotonically increasing function such that for each n ∈ N , K Sc ( n ) is the least integer m such that program variable k has value m while the program variable i c has value n . Thus, for example, K Sc (0) = 0 for all ≤ c ≤ n . When S is clear from the context, we may simply write K c for K Sc .With the help of these re-indexing functions we can express important in-variant properties of the distinguishing sequence variables v cj and partition setvariables E cn ( α ) . Using Proposition 4.2.2 their relationship to the correspond-ing variables v ID j and E IDn ( α ) of ID can be established. Since Algorithm 2 hasa doubly nested loop structure, the proof of Simulation Theorem 4.2.4 belowmakes use of a doubly nested induction argument. Let S = s , . . . , s l be any non-empty sequenceof strings s i ∈ Σ ∗ . For any execution of F ID on S and the n -bit Kripke struc-ture A there exists an execution of ID on Pref ( { λ, s , . . . , s l } ) and the c -thprojection A c (for each ≤ c ≤ n ) such that for all m ≥ :(i) For all n ≥ , if K c ( n ) = m then:(a) for all ≤ j ≤ n , v cj = v ID j ,(b) for all ≤ j < n , v cn (cid:54) = v cj ,(c) for all α ∈ T m , E cn ( α ) = { v cj ∈ V c | ≤ j ≤ n, αv cj ∈ L ( A c ) } .(ii) If m > then let p ∈ N be the greatest integer such that K c ( p ) = m − .Then for all α ∈ T m , E cp ( α ) = { v cj ∈ V c | ≤ j ≤ p, αv cj ∈ L ( A c ) } .(iii) The m th partition refinement of F ID terminates.
Proof.
By induction on m using Proposition 4.2.2.(i).Part (i.a) above asserts that the same distinguishing sequences are producedin the same order by F ID and ID . Part (i.b) asserts that a distinguishing se-quence is never produced twice by F ID . Part (i.c) and (ii) characterise thepartition sets E cn ( α ) as sets of all distinguishing sequences v cj that lead to anaccepting state of A c from α .Note that both ID and F ID are non-deterministic algorithms (due to thenon-deterministic choice on line 17 of Algorithm 1 and line 4 of Algorithm 3).Therefore in the statement of Theorem 4.2.4 above, we can only talk aboutthe existence of some correct simulation. Clearly there are also simulations of
F ID by ID which are not correct, but this does not affect the basic correctnessargument. Let S = s , . . . , s l be any non-empty sequence of strings s i ∈ Σ ∗ . Any execution of F ID on S and an n -bit Kripke structure A terminateswith the program variable k having value l . Proof.
Follows from Simulation Theorem 4.2.4.(iii) since clearly the while loopof Algorithm 2 terminates when the input sequence S is empty.Using the detailed analysis of the invariant properties of the program vari-ables P k and T k in Proposition 4.2.2 and v cj and E cn ( α ) in Simulation Theorem F ID
Algorithm.
Let S = s , . . . , s l be any non-empty sequenceof strings s i ∈ Σ ∗ such that { λ, s , . . . , s l } contains a live complete set foreach projection A i of A . Then F ID terminates on S . Also for each ≤ i ≤ n the hypothesis DFA M il is a canonical representation of A i . Proof.
By Corollary 4.2.5,
F ID terminates on S with the variable k havingvalue l . By Simulation Theorem 4.2.4.(i) and Theorem 4.1.1.(ii), there exists anexecution of ID on Pref ( { λ, s , . . . , s l } ) such that E in ( α ) = E IDn ( α ) for all α ∈ T l and any n such that K ( n ) = l . By Proposition 4.2.2.(iii), T l = T ID and P (cid:48) l = P (cid:48) . So letting M ID i be the canonical representation of A i constructed by ID using Pref ( { λ, s , . . . , s l } ) then M ID i and M il have the same state sets,initial states, accepting states and transitions.Our next result confirms that each hypothesis DFA M it generated after t input strings have been applied to A is consistent with all currently knownobservations about the i th projection A i . This is quite straightforward in thelight of Simulation Theorem 4.2.4. Let S = s , . . . , s l be any non-empty se-quence of strings s i ∈ Σ ∗ . For each ≤ t ≤ l and each string s ∈ { λ, s , . . . , s t } ,the hypothesis automaton M it accepts s if, and only if the i th projection A i of A does. Proof.
By definition, M it is compatible with A i on { λ, s , . . . , s t } if, andonly if, for each ≤ j ≤ t , s j ∈ L ( A i ) ⇔ λ ∈ E ii t ( s j ) , where i t is thegreatest integer such that K ( i t ) = t and the sets E ii t ( α ) for α ∈ T t are thestates of M it . Now v = λ . So by Simulation Theorem 4.2.4.(i).(c), if s j ∈ L ( A i ) then s j v ∈ L ( A i ) so v ∈ E i t ( s j ) , i.e. λ ∈ E ii t ( s j ) , and if s j (cid:54)∈ L ( A i ) then s j v (cid:54)∈ L ( A i ) so v (cid:54)∈ E ii t ( s j ) , i.e. λ (cid:54)∈ E ii t ( s j ) .We have now established a reliable method for decomposing the problem oflearning a k -bit Kripke structure A into the problem of learning a family of k individual DFA. This approach supports projection, as defined in Section 1.2and Definition 2.2. We next turn our attention to problem of efficiently recombining a family of k individual DFA (the projections) into a single k -bit deterministic Kripke struc-ture. For this we use a well known algebraic construction known as a subdirectproduct. Informally, a subdirect product of a family F = (cid:104) A i | i ∈ I (cid:105) of alge-braic structures, is any subalgebra of the direct product ΠF = Π i ∈ I A i whichprojects onto (surjectively) each of its co-ordinate algebras A i . The subdirectproduct construction was introduced in [4] as a universal decomposition methodapplicable to any algebraic structures. The reader may consult [19] for basic facts about subdirect products and their universal properties. A specific definition fordeterministic Kripke structures is given below.To begin with, we observe that for black-box testing it suffices to learn aKripke structure up to behavioural equivalence. Let A and B be k -bit Kripke structures over a finite inputalphabet Σ . We say that A and B are behaviourally equivalent , and write A ≡ B if, and only if, for every finite input sequence σ , . . . , σ i ∈ Σ ∗ we have λ ∗A ( σ , . . . , σ i ) = λ ∗B ( σ , . . . , σ i ) . Clearly, by the isomorphism identified in Section 2 between -bit Kripke struc-tures and DFA, for such structures we have A ≡ B if, and only if, L ( A (cid:48) ) = L ( B (cid:48) ) .Furthermore, if M in ( A ) is the minimal subalgebra of A then M in ( A ) ≡ A .A family of k individual -bit Kripke structures (DFA) can be combinedinto a single k -bit Kripke structure using the following instance of the subdirectproduct construction. Let A , . . . , A k be a family of 1-bit Kripke structures, A i = ( Q i , Σ, δ i : Q i × Σ → Q i , q i , λ i : Q → B ) for i = 1 , . . . , k . Define the direct product Kripke structure k (cid:89) i =1 A i = ( Q, Σ, δ : Q × Σ → Q, q , λ : Q → B k ) , where Q = (cid:81) ki =1 Q i = Q × . . . × Q k and q = ( q , . . . , q k ) . Also δ ( q , . . . , q k , σ ) = ( δ ( q , σ ) , . . . , δ k ( q k , σ ) ) ,λ ( q , . . . , q k ) = ( λ ( q ) , . . . , λ k ( q k ) ) . Associated with the direct product (cid:81) ki =1 A i we have i -th projection mapping proj i : Q × . . . × Q k → Q i , proj i ( q , . . . , q k ) = q i , ≤ i ≤ k Define the subdirect product
Min ( (cid:81) ki =1 A i ) be the minimal subalgebra of (cid:81) ki =1 A i .The reason for taking the subdirect product of the A i as the minimal sub-algebra of the direct product (cid:81) ki =1 A i is to avoid the state space explosion dueto a large number of unreachable states in the direct product itself. The statespace size of (cid:81) ki =1 A i grows exponentially with k . On the other hand, since mostof these states are unreachable from the initial state, then from the point ofview of requirements testing they are irrelevant. This subdirect product can becomputed from its components A i in time O ( k.m. | Σ | ) where m is the number ofstates in the resulting subdirect product and | Σ | is the size of the input alpha-bet. A naive algorithm based on systematic path exploration starting from the initial state can be used. We leave the definition of this algorithm as an exercisefor the reader.As is well known from universal algebra, the i -th projection mapping proj i is a homomorphism. Let A , . . . , A k be any minimal 1-bit Kripke structures. (i) For each ≤ i ≤ k , the projection mapping proj i : Min ( (cid:81) ki =1 A i ) → A i isan epimorphism. Hence Min ( (cid:81) ki =1 A i ) is a subdirect product of the A i . (ii) Min ( (cid:81) ki =1 A i ) ≡ (cid:81) ki =1 A i . Proof. (i) Immediate since the A i are minimal. (ii) Follows from the fact that M in ( A ) ≡ A .The following theorem justifies bit-sliced learning of k -bit Kripke structuresusing conventional regular inference methods for a family of DFA. It consti-tutes the correctness argument for the subdirect product component of the IKLarchitecture, as presented in Section 3. Let A be a k -bit Kripke structure over a finite input alphabet Σ . Let A , . . . , A k be the k individual 1-bit projections of A . For any 1-bitKripke structures B , . . . , B k , if, A ≡ B & . . . & A k ≡ B k then A ≡
Min ( k (cid:89) i =1 B i ) . Proof.
Use Proposition 5.3.By Correctness Theorem 4.2.6, the assumptions of Theorem 5.4 on the 1-bit Kripke structures B , . . . , B k are fulfilled by the IKL architecture, sincethese are the canonical representations of A , . . . , A k . So by Theorem 5.4, theoutput of the IKL algorithm, after DFA family learning has converged and thesubdirect product construction has been applied is a k -bit Kripke structure B that is behaviourally equivalent with the input Kripke structure A .Despite the canonical DFA B , . . . , B k being minimal, the reduced product B may still be much larger in state space size than A . This can slow down theprocess of model checking the output of IKL considerably. So it is important toreduce the state space size of B even further. This last step of the IKL algorithmwill be discussed in the next section. In this section we introduce an efficient algorithm for the minimisation of deter-ministic Kripke structures with O ( | Σ | .n log n ) time complexity. Here n is thestate space size of the Kripke structure A and | Σ | is the size of its input alpha-bet. This algorithm is applied on the back end of the IKL learning algorithmin order to speed up model checking of the learned hypothesis automata duringtesting. To define a minimisation algorithm, we need to generalise the concepts of right language and
Nerode congruence from DFA to deterministic Kripke struc-tures. We then show how Hopcroft’s DFA minimisation algorithm of [12] canbe generalised to compute the Nerode congruence ≡ of a deterministic Kripkestructure A . The quotient Kripke structure A / ≡ is minimal and language equiv-alent to A . This fact is the final result needed to prove the correctness of theIKL architecture. We will prove the correctness and complexity properties of ourminimisation algorithm from first principles. Let us consider a DFA A = ( Q, Σ, δ, q , F ) . For each state q ∈ Q of A therecorresponds a subautomaton of A rooted at q which accepts the regular language L q ( A ) ⊆ Σ ∗ , consisting of just those words accepted by the subautomaton with q as initial state. Thus L q ( A ) is the language accepted by A . The language L q ( A ) is called either the future of state q or the right language of q. A is minimal (i.e. state minimal as opposed to algebraically minimal ) if for each pair of distinctstates p, q ∈ Q , we have, L p ( A ) (cid:54) = L q ( A ) . For any regular language L ⊆ Σ ∗ there is a smallest DFA (in terms of the number of states) accepting L . ThisDFA is minimal, and is unique up to isomorphism.An equivalence relation ≡ can be defined on the states of a DFA by p ≡ q if and only if L p ( A ) = L q ( A ) . This relation is a congruence, i.e. if p ≡ q then p.σ ≡ q.σ for all σ ∈ Σ ∗ . It is known as the Nerode congruence . Consider thequotient DFA A / ≡ . This is the unique smallest DFA which accepts the regularlanguage L q ( A ) . The problem of minimizing a DFA A is therefore to computeits Nerode congruence, which will be the identity relation if, and only if A is aminimal automaton.The problem of computing a minimal Kripke structure A is an analogous butmore general problem. In this case, the right language L q ( A ) associated with astate q of A can be defined by L q ( A ) = { ( σ , ..., σ n , a ) ∈ Σ ∗ × B k | λ ∗ q ( σ , ..., σ n ) = a } . As before, A is minimal if for each pair of distinct states p, q ∈ Q we have, L p ( A ) (cid:54) = L q ( A ) . There is again a smallest Kripke structure associated with aright language L ⊆ Σ ∗ × B k . This Kripke structure is also minimal, and uniqueup to isomorphism. The Nerode congruence for a Kripke structure A is nowdefined by: p ≡ q if and only if λ ∗ p ( σ , ..., σ n ) = λ ∗ q ( σ , ..., σ n ) for all ( σ , ..., σ n ) ∈ Σ ∗ .and A / ≡ is the unique smallest Kripke structure associated with the rightlanguage L q ( A ) . So the problem of minimising A is to compute this congruence. Algorithm 5 presents an efficient algorithm to compute the Nerode congruence ≡ of a deterministic Kripke structure A , which is the same as the state set Algorithm 5
Kripke Structure Minimisation
Input:
A deterministic Kripke structure A with no unreachable states and k output bits. Output:
The Nerode congruence ≡ for A , i.e. equivalence classes of states forthe minimized structure A min behaviourally equivalent to A . Create an initial state partition P = { B q = { q (cid:48) ∈ Q | λ ( q ) = λ ( q (cid:48) ) } | q ∈ Q } .Let n = | P | . Let B , ..., B n be an enumeration of P. if n = | Q | then go to line 30. foreach σ ∈ Σ do for i ← to n do B ( σ, i ) = { q ∈ B i | ∃ r ∈ Q s.t δ ( r, σ ) = q } . /*This constitutes the subsetof states in block B i which have predecessors through input σ . */ count = n + 1 ; foreach σ ∈ Σ do choose all the subsets B ( σ, i ) (excluding any empty subsets) and put theirblock numbers i on a waiting list (i.e. an unordered set) W ( σ ) to be pro-cessed. Boolean splittable = true; while splittable do foreach σ ∈ Σ do foreach i ∈ W ( σ ) do Delete i from W ( σ ) for j ← to count − s.t. ∃ t ∈ B j with δ ( t, σ ) ∈ B ( σ, i ) do Create B (cid:48) j = { t ∈ B j | δ ( t, σ ) ∈ B ( σ, i ) } if B (cid:48) j ⊂ B j then B count = B j − B (cid:48) j ; B j = B (cid:48) j foreach σ ∈ Σ do B ( σ, count ) = { q ∈ B ( σ, j ) | q ∈ B count } ; B ( σ, j ) = { q ∈ B ( σ, j ) | q ∈ B j } if j / ∈ W ( σ ) and < | B ( σ, j ) | ≤ | B ( σ, count ) | then W ( σ ) = W ( σ ) ∪ { j } else W ( σ ) = W ( σ ) ∪ { count } count = count + 1 ; splittable = false; foreach σ ∈ Σ do if W ( σ ) (cid:54) = ∅ then splittable=true; Return partition blocks B , ..., B count . of the associated quotient Kripke structure A / ≡ . We will give a rigorous butsimple proof of the correctness of this algorithm. By means of a new inductionargument, we have simplified the correctness argument compared with [3] and[12]. First let us establish termination of the algorithm by using an appropriatewell-founded ordering for the main loop variant. Consider any pair of finite sets of finite sets A = { A , ..., A m } and B = { B , ..., B n } . We define an ordering relation ≤ on A and B by A ≤ B iff ∀ ≤ i ≤ m , ∃ ≤ j ≤ n such that A i ⊆ B j . Define A < B ⇐⇒ A ≤ B & A (cid:54) = B . Clearly ≤ is a reflexive, transitive relation. Furthermore ≤ is well-founded, i.e. there are no infinite descending chains A > A > A ... , since ∅ isthe smallest element under ≤ . Algorithm 5 always terminates.
Proof.
We have two cases for the termination of the algorithm as a result ofthe partition formed on line 1 of the algorithm: (1) when n = | Q | , and (2) when n < | Q | .Consider the case when n = | Q | then each block in the partition correspondsto a state of the given Kripke structure with a unique bit-label and hence inthis case the algorithm will terminate on line 30 by providing the description ofthese blocks.Now consider the case when n < | Q | . Then the waiting sets W ( σ ) for all σ ∈ Σ will be initialized on lines 7, 8 and the termination of the algorithm depends onproving the termination of the loop on line 10. Now W ( σ ) is intialized by loadingthe block numbers of the split sets on line 8. There are only two possiblities afterany execution of the loop. Let W m ( σ ) and W m +1 ( σ ) represent the state of thevariable W ( σ ) before and after one execution of the loop respectively at anygiven time. Then either W m ( σ ) = W m +1 ( σ ) ∪ { i } and no splitting has takenplace and i is the deleted block number, or W m ( σ ) ∪ { j } = W m +1 ( σ ) ∪ { i } or W m ( σ ) ∪ { k } = W m +1 ( σ ) ∪ { i } where j and k represent the split blocks andone of them goes into W m ( σ ) if it has fewer incoming transitions. In eithercase W m ( σ ) > W m +1 ( σ ) by Definition 30. Therefore W ( σ ) strictly decreaseswith each iteration of the loop on line 10. Since the ordering ≤ is well-founded,Algorithm 5 must terminate.Now we only need to show that when Algorithm 5 has terminated, it returnsthe Nerode congruence ≡ on states. Let P i be the partition (block set) on the ith iteration ofAlgorithm 5. For any blocks B j , B k ∈ P i and any states p ∈ B j , q ∈ B k if j (cid:54) = k then p (cid:54)≡ q . Proof.
By induction on the number i of times the loop on line 10 is executed. Basis:
Suppose i = 0 then clearly the result holds because each block createdat line 1 is distinguishable by the empty string (cid:15) . Induction Step:
Suppose i = m > . Let us assume that the proposition holdsafter m executions of the loop. Consider any B j , B k ∈ P m . During the m + 1 th execution of the loop on line10 either block B j is split into B (cid:48) j and B (cid:48)(cid:48) j or B k is split into B (cid:48) k and B (cid:48)(cid:48) k but notboth during one execution of the loop (due to line 17).Consider the case when B j is split then for any p ∈ B j , either p ∈ B (cid:48) j or p ∈ B (cid:48)(cid:48) j . But for any p ∈ B j and q ∈ B k , p (cid:54)≡ q by the induction hypothesis.Therefore, for p ∈ B (cid:48) j or p ∈ B (cid:48)(cid:48) j p (cid:54)≡ q . Hence the proposition is true for m + 1 thexecution of the loop in this case.By symmetry the same argument holds when B k is split.The following Lemma gives a simple, but very effective way to understandAlgorithm 5. Note that this analysis is more like a temporal logic argument thana loop invariant approach. This approach reflects the non-determinism inherentin the algorithm. For any states p, q ∈ Q , if p (cid:54)≡ q and initially p and q are inthe same block p, q ∈ B i then eventually p and q are split into different blocks, p ∈ B j and q ∈ B k for j (cid:54) = k . Proof.
Suppose that p (cid:54)≡ q and that initially p, q ∈ B i for some block B i . Since p (cid:54)≡ q then for some n ≥ , and σ , . . . , σ n ∈ Σ , λ ∗ ( p, σ , . . . , σ n ) (cid:54) = λ ∗ ( q, σ , . . . , σ n ) . We prove the result by induction on n . Basis
Suppose n = 0 , so that λ ( p ) (cid:54) = λ ( q ) . By line 1, p ∈ B p and q ∈ B q and B p (cid:54) = B q . So the implication holds vacuously. Induction Step
Suppose n > and for some σ , . . . , σ n ∈ Σ , λ ∗ ( p, σ , . . . , σ n ) (cid:54) = λ ∗ ( q, σ , . . . , σ n ) . (a) Suppose initially δ ( p, σ ) ∈ B ( σ , α ) and δ ( q, σ ) ∈ B ( σ , β ) for α (cid:54) = β .Consider when σ = σ on the first iteration of the loop on line 10. Clearly, B ( σ , α ) , B ( σ , β ) ∈ W ( σ ) at this point. Choosing i = α and j = i on thisiteration then since δ ( p, σ ) ∈ B ( σ , α ) we have B (cid:48) i = { t ∈ B i | δ ( t, σ ) ∈ B ( σ , α ) } ⊂ B i This holds because q ∈ B i but δ ( q, σ ) ∈ B ( σ , β ) and B ( σ , α ) (cid:54) = B ( σ , β ) so B ( σ , α ) ∩ B ( σ , β ) = ∅ and hence q (cid:54)∈ B (cid:48) i . Therefore p and q are split intodifferent blocks on the first iteration so that p ∈ B (cid:48) i and q ∈ B i − B (cid:48) i .By symmetry, choosing i = β and j = i then p and q are split on the firstloop iteration with q ∈ B (cid:48) i and p ∈ B i − B (cid:48) i . (b) Suppose initially δ ( p, σ ) , δ ( q, σ ) ∈ B ( σ , α ) for some α . Now λ ∗ ( δ ( p, σ ) , σ , . . . , σ n ) (cid:54) = λ ∗ ( δ ( q, σ ) , σ , . . . , σ n ) . So by the induction hypothesis, eventually δ ( p, σ ) and δ ( q, σ ) are split intodifferent blocks, δ ( p, σ ) ∈ B α and δ ( p, σ ) ∈ B β . At that time one of B α or B β is placed in a waiting set W ( σ ) . Then either on the same iteration of the loop on line 10 or on the next iteration, we can apply the argument of part (a) againto show that p and q are split into different blocks.Observe that only one split block is loaded into W ( σ ) on lines 21-24. Fromthe proof of Lemma 30 we can see that it does not matter logically which ofthese two blocks we insert into W ( σ ) . However, by choosing the subset withfewest incoming transitions we can obtain a worst case time complexity of order O ( | Σ | .n log n ) , as we will show. For any states p, q ∈ Q , if p (cid:54)≡ q then p and q are in differentblocks when the algorithm terminates. Proof.
Assume that p (cid:54)≡ q . (a) Suppose at line 3 that n = | Q | . Then initially, all blocks B i are singleton setsand so trivially p and q are in different blocks when the algorithm terminates. (b) Suppose at line 3 that n < | Q | . (b.i) Suppose that p and q are in different blocks initially. Since blocks are nevermerged then the result holds. (b.ii) Suppose that p and q are in the same block initially. Since p (cid:54)≡ q then theresult follows by Lemma 30.We conclude this section by verifying that our generalisation of Hopcroft’sminimisation algorithm does not actually change its time complexity. If A has n states then Algorithm 5 has worst case timecomplexity O ( | Σ | .n log n ) . Proof.
Creating the initial block partition on line 1 requires at most O ( n ) as-signments. The block subpartitioning in the loop on line 3 requires at most O ( kn ) moves of states. Also the the initialisation of the waiting lists W ( σ ) in the loopon line 7 requires at most O ( kn ) assignments.Consider one execution of the body of the loop starting on line 10, i.e. lines13 - 29. Consider any states p, q ∈ Q and suppose that δ ( p, σ ) = q for some σ ∈ Σ . Then the state p can be: (i) moved into B (cid:48) j (line 15), (ii) removed from B j (line 17), or (iii) moved into B ( σ, i ) or B ( σ, count ) (lines 19, 20) if, and onlyif, a block i is being removed from W ( σ ) such that q ∈ B ( σ, i ) at that time.(Such a block sub-partition B ( σ, i ) can be termed a splitter of q .)Now each time a block i containing q is removed from W ( σ ) its size is lessthan half of the size when it was originally entered into W ( σ ) , by lines 21-24. So i can be removed from W ( σ ) at most O ( log n ) times. Since there are at most | Σ | values of σ and n values of p , then the total number of state moves betweenblocks and block sub-partitions is at most O ( | Σ | .n log n ) . When the IKL learning algorithm is applied to the problem of learning basedtesting of software, the question naturally arises, when should we stop testing?When the system under test (SUT) is sufficiently small, exhaustive testing can be achieved if we continue until the IKL algorithm converges. But how can wedetect convergence?Traditionally, in automata learning theory, this question is answered by exe-cuting an equivalence oracle on the SUT and the hypothesis automaton such as[20]. For a DFA learning algorithm such as L* [2], learning is continued if theequivalence oracle can return a string that is incorrectly learned by the hypoth-esis DFA, otherwise learning is terminated. However, in the context of black-boxtesting a glass box equivalence oracle , based on direct comparison of the SUTand the hypothesis automaton, is not acceptable for two reasons:(1) the principles of black-box testing do not allow us to expose the SUT forglass box equivalence checking, and(2) even if we ignore (1), in practise there are no glass box equivalence checkersthat can compare an arbitrary piece of software (the SUT implementation) withthe hypothesis automaton for equivalence.Of course, a glass box equivalence oracle can be stochastically approximatedby a black-box equivalence oracle based on random queries. Random queries areeven necessary during LBT when no counterexamples can be found by modelchecking. However, a purely stochastic solution to equivalence checking is notpossible, as we will discuss below. Therefore problems (1) and (2) force us toconsider other black-box heuristics for estimating convergence of the IKL algo-rithm. Hypotheses ( H i ) S t a t e S p a c e S i z e Fig. 2.
Graph for True and Estimated Convergence for Elevator6
Figure 2 depicts the state space size of successive hypothesis automata H i ( i = 1 , ... ) generated by the IKL algorithm while learning and testing a smallreactive system against a simple temporal logic specification. In this controlledexperiment the SUT was a simplified model of an elevator, with a state spacesize of 38 states and an input alphabet of 4 symbols. This model is well withinthe scope of complete learning using IKL, which converges quickly.It is natural to consider whether any features of a graph such as Figure 2 canbe used to estimate the point of convergence. This graph is comparable in itsstructure for all similar experiments that were conducted. It shows a successionof peaks, each one well above the state space size of the underlying SUT. Howeverat some point these peaks die out and a steady state space size is reached. Eachpeak and trough seem to indicate a distinct new phase in learning, and thereforethey do shed some light on the learning activity. However, they clearly do notindicate convergence, which first appears in hypothesis automaton H . (In con-trolled experiments we can apply a glass box equivalence checker to accuratelydetermine convergence.)Although we cannot apply glass box equivalence checking between the SUTand hypothesis automata H i , we can apply it to pairs of successive hypothesis au-tomata H i and H i − , since the representations of these are known and visible. Wecan even iterate this test across n successive hypothesis automata H i , ..., H i − n (by conjunction of the outcomes) which we term n -equivalence checking . Afterconvergence has been achieved, n -equivalence checking will be positive for everyvalue of n . This gives a heuristic for black box equivalence checking that is morecomplex than stochastic equivalence checking, since the queries used to generatesuccessive hypothesis automata are not always random. Many arise from modelchecking counterexamples. It is difficult to say that queries generated by modelchecking are randomised, since they are always counterexamples to a specifictemporal logic formula, which can strongly bias their structure.We therefore decided to empirically evaluate the reliability of n -equivalencechecking as a heuristic indicator of convergence. For this evaluation we considereddifferent SUTs with different state space sizes, different temporal logic formulas,and different values of n .We chose two different SUTs, which were models of a simple cruise controllerand a simple elevator. The cruise controller model was an 8-state 5-bit Kripkestructure with an input alphabet size of 5. The elevator model was a 38-state 8-bit Kripke structure with an input alphabet size of 4. We considered four differenttemporal logic test requirements for the cruise controller and six for the elevator.These gave a total of ten convergence experiments for the two SUTs.Each of these ten experiments was then used to evaluate the n -equivalenceheuristic for n = 1 , , , . For n = 1 , the heuristic completely failed toidentify convergence (i.e. the indicator always triggered too early) for all tenexperiments. For n = 10 , just two experiments with the cruise controller (thesmaller case study) correctly identified convergence , while eight still failed. Using n = 50 all experiments correctly identified convergence. However, note thatfor increasingly large values of n we tended to overestimate the convergence point by an increasing margin. Table 1 summarises the relationship betweentrue convergence and estimated convergence for n = 50 .Requirement True Conver-gence EstimatedConvergence1 H H H H H H H H H H H H Table 1.
True and Estimated Convergence
These simple experiments suggest that for sufficiently large n , n -equivalence,can be used as a reliable heuristic indicator for convergence. However, furtherempirical and theoretical analysis still seems necessary to predict the smallestreliable value of n which minimises the problem of overestimation. We have defined and analysed a learning algorithm IKL for deterministic Kripkestructures which is efficient for applications in software testing. This algorithmextends active incremental learning with new features such as lazy learning andprojection. We have formally proved the correctness of the IKL algorithm andits main components. We have also empirically evaluated a black box heuristicfor detecting convergence of learning, which can be used to terminate testing forsmall systems under test.Incremental learning and projection combine to make IKL scalable to largersystems under test. Also, incremental and lazy learning combine to supportfrequent generation of hypothesis automata with which we can discover SUTerrors much faster than random testing by model checking. These claims havebeen empirically evaluated and supported in [17] and [28]. The IKL algorithm hasbeen implemented in the LBTest tool [18] for learning based testing of reactivesystems.We believe that the efficiency of learning-based testing can be even furtherimproved by more research on model inference. For example, the modular ar-chitecture of the IKL algorithm can support experiment with other incrementalDFA learning algorithms instead of the ID learning algorithm of Section 4, (e.g.RPNI2 [9]). The impact of the frequency of hypothesis automata generation ontesting efficiency could then be further investigated. When hypothesis genera-tion is very frequent the overhead of model checking is high, and this overheadcan slow down the entire LBT process. However, if generation is very infrequent, then little use is made of the model checker to conduct a directed search forSUT errors using queries that can falsify the user requirements. This is also inef-ficient. (Recall the discussion of Section 1.2.) More generally, we could consideran optimal tuning of the rate of hypothesis automata generation, e.g. based onthe estimated density of SUT errors.The relationship between computational learning and software testing hasbeen a fruitful line of research ever since Weyuker’s thesis [27]. Many fundamen-tal questions remain within the context of learning-based testing. For example,the execution of any automata learning algorithm can always be associated witha prefix tree construction (see e.g. [8]) based on the query set used. How can weinfluence the choice between breadth-first and depth-first search for SUT errorsusing this prefix tree? Another important question is whether we can find othertechniques to generate active learner queries besides congruence construction?Such techniques should be aimed at reducing the need for random queries, whichcan be very inefficient in practise.We gratefully acknowledge financial support for this research from the SwedishResearch Council (VR), the Higher Education Commission (HEC) of Pakistan,and the European Union under project HATS FP7-231620. References
1. D. Angluin. A note on the number of queries needed to identify regular languages.
Information and Control , 51(1):76–87, October 1981.2. D. Angluin. Learning regular sets from queries and counterexamples.
Informationand Computation , 75(1):87–106, November 1987.3. J. Berstel, L. Boasson, O. Carton, and I. Fagnot. Minimization of Automata.
ArXive-prints , October 2010.4. G. Birkhoff. Subdirect unions in universal algebra.
Bull. Amer. Math. Soc , 50:764–768, 1944.5. D. Bustan and O. Grumberg. Simulation-based minimization.
ACM Trans. Com-put. Logic , 4(2):181–206, April 2003.6. P. Chauhan, E. M. Clarke, J. H. Kukula, S. Sapra, H. Veith, and D. Wang. Auto-mated abstraction refinement for model checking large state spaces using sat basedconflict analysis. In
Proc. 14th International Conference On Formal Methods inComputer-Aided Design (FMCAD02) , 2002.7. E. Clarke, A. Gupta, J. Kukula, and O. Strichman. Sat-based abstraction refine-ment using ilp and machine learning. In
Proc. 21st International Conference OnComputer Aided Verification (CAV’02) , 2002.8. C. de la Higuera.
Grammatical Inference . Cambridge University Press, 2010.9. P. Dupont. Incremental regular inference. In
Proceedings of the Third ICGI-96 ,number 1147 in LNAI, 1996.10. E.M. Gold. Language identification in the limit.
Information and Control ,10(5):447–474, 1967.11. A. Groce, D. Peled, and M. Yannakakis. Adaptive model checking.
Logic Journalof the IGPL , 14(5):729–744, 2006.12. J. E Hopcroft. An n log n algorithm for minimizing states in a finite automaton.In Z. Kohavi and A. Paz, editors,
Theory of Machines and Computations , pages189–196. Academic Press, 1971.913. K. Meinke. Automated black-box testing of functional correctness using functionapproximation. In
ISSTA ’04: Proceedings of the 2004 ACM SIGSOFT interna-tional symposium on Software testing and analysis , pages 143–153, New York, NY,USA, 2004. ACM.14. K. Meinke and F. Niu. A learning-based approach to unit testing of numericalsoftware. In
Proc. Twenty Second IFIP Int. Conf. on Testing Software and Systems(ICTSS 2010) , number 6435 in LNCS, pages 221–235. Springer, 2010.15. K. Meinke and F. Niu. Learning-based testing for reactive systems using termrewriting technology. In
Proc. 23rd IFIP Int. Conf. on Testing Software and Sys-tems (ICTSS 2011) , number 7019 in LNCS, pages 97–114. Springer, 2011.16. K. Meinke and F. Niu. An incremental learning algorithm for extended mealyautomata. In
Proc. 5th International Symposium On Leveraging Applications ofFormal Methods, Verification and Validation , number 7610 in LNCS. Springer,2012.17. K. Meinke and M. Sindhu. Incremental learning-based testing for reactive systems.In
Proc Fifth Int. Conf. on Tests and Proofs (TAP2011) , number 6706 in LNCS,pages 134–151. Springer, 2011.18. K. Meinke and M.A. Sindhu. Lbtest: A learning-based testing tool for reactivesystems. In
Software Testing, Verification and Validation (ICST), 2013 IEEESixth International Conference on , pages 447–454, March 2013.19. K. Meinke and J.V. Tucker. Universal algebra. In
Handbook of Logic in ComputerScience: Volume 1 , pages 189–411. Oxford University Press, 1993.20. D.A. Norton. Algorithms for testing equivalence of finite state automata, with agrading tool for jflap. Technical report, Rochester Institute of Technology, Depart-ment of Computer Science, 2009.21. R.G. Parekh, C. Nichitiu, and V.G. Honavar. A polynomial time incremental algo-rithm for regular grammar inference. In
Proc. Fourth Int. Colloq. on GrammaticalInference (ICGI 98) , LNAI. Springer, 1998.22. D. Peled, M.Y. Vardi, and M. Yannakakis. Black-box checking. In
Formal Methodsfor Protocol Engineering and Distributed Systems FORTE/PSTV , pages 225–240.Kluwer, 1999.23. H. Raffelt, B. Steffen, and T. Margaria. Dynamic testing via automata learning.In
Hardware and Software: Verification and Testing , number 4899 in LNCS, pages136–152. Springer, 2008.24. M. A. Sindhu and K. Meinke. IDS: An Incremental Learning Algorithm for FiniteAutomata.
ArXiv e-prints , June 2012.25. N. Walkinshaw. Assessing test adequacy for black-box systems without specifica-tions. In
Proc. Twenty Third IFIP Int. Conf. on Testing Software and Systems(ICTSS 2011) , number 7019 in LNCS, pages 209–224. Springer, 2011.26. N. Walkinshaw, K. Bogdanov, J. Derrick, and J. Paris. Increasing functional cov-erage by inductive testing: a case study. In
Proc. Twenty Second IFIP Int. Conf.on Testing Software and Systems (ICTSS 2010) , number 6435 in LNCS, pages126–141. Springer, 2010.27. E. Weyuker. Assessing test data adequacy through program inference.
ACM Trans.Program. Lang. Syst , 5(4):641–655, 1983.28. Peter Y. Wong, Richard Bubel, Frank S. Boer, Miguel Gómez-Zamalloa, StijnGouw, Reiner Hähnle, Karl Meinke, and Muddassar Azam Sindhu. Testing abstractbehavioral specifications.