Formal Verification of Control Systems Properties with Theorem Proving
FFormal Verification of Control Systems’ Properties with TheoremProving
Dejanira Araiza-Illan and Kerstin EderDept. Computer ScienceUniversity of BristolBristol, UK { Dejanira.Araizaillan, Kerstin.Eder } @bristol.ac.ukArthur RichardsDept. Aerospace EngineeringUniversity of BristolBristol, [email protected] Abstract
This paper presents the deductive formal verification of high-level properties of control systemswith theorem proving, using the Why3 tool. Properties that can be verified with this approach includestability, feedback gain, and robustness, among others. For the systems, modelled in Simulink, wepropose three main steps to achieve the verification: specifying the properties of interest over thesignals within the model using Simulink blocks, an automatic translation of the model into Why3,and the automatic verification of the properties with theorem provers in Why3. We present amethodology to specify the properties in the model and a library of relevant assertion blocks (logicexpressions), currently in development. The functionality of the blocks in the Simulink models areautomatically translated to Why3 as ‘theories’ and verification goals by our tool implemented inMATLAB. A library of theories in Why3 corresponding to each supported block has been developedto facilitate the process of translation. The goals are automatically verified in Why3 with relevanttheorem provers. A simple first-order discrete system is used to exemplify the specification of theSimulink model, the translation process from Simulink to the Why3 formal logic language, and theverification of Lyapunov stability.
The use of formal methods allows the production of reliable and certified trustworthy autonomous sys-tems [12, 25] in a more automatic manner, facilitating the processes of verification and validation, andintroducing methodologies for systems design towards verification. Verification has become essential forsafety-critical autonomous systems. Autonomous systems have different compositional levels [12]: agent(high-level planning and decision making), control (neural networks, controllers or control systems) andhardware implementation. Many proposed modelling formalisms and formal verification examples dealwith the first layer only [9, 26, 30], or the first and second combined in a high-level abstraction (e.g.,hybrid systems). For the latter, some approaches use theorem proving [19]. Other approaches translatethe systems to decidable automata models and apply model checking [10,18,20], but these are not easilyapplicable to real-valued operations due to problems with scalability. Additionally, optimisation theoryhas been used to verify functionality and robustness of controllers for autonomous systems [14, 29].The design of control systems typically begins with formal analysis followed by numerical implemen-tation in a simulation tool like Simulink. Numerical simulations then test for correct behaviour beforethe implementation is deployed. In some cases, automatic code generation [24] is used to derive codedirectly from the simulation model. In this paper, we propose the application of deductive formal veri-fication methods on the implementation of the controller in Simulink. This allows greater confidence inthe correctness of the model with respect to its requirements. In particular, the paper provides a way totranslate Simulink models into the Why3 logic language to automatically prove results such as decrease1 a r X i v : . [ c s . S Y ] J un f a Lyapunov function. One of the future aims of proposed approach is to verify the correctness ofcontrollers based on an optimiser; for example, predictive controllers for UAV guidance [21, 22].Why3 is a free and open source tool that interfaces with different theorem provers, particularly theirSatisfiability Modulo Theory (SMT) solvers. SMT solvers are extensions of SATisfiability (SAT) solversthat can accommodate real numbers, integers, and other domains (theories), and are thus well-suited toprove control systems properties [8]. Automatic theorem provers currently supported by Why3 includeAlt-Ergo, CVC3, CVC4, E-prover, Gappa, Simplify, SPASS, Vampire, veriT, Yices and Z3, along withthe interactive provers Coq and PVS [11]. An SMT solver automatically computes the satisfiability of alogic formula based on a range of provided axioms and definitions.The closest prior work to this paper in implementation is Simcheck [23], which uses SMT solvers toperform type checking on a Simulink model with custom annotations. Stability based on Nichols plotshas been verified for a system modelled in Simulink using the MetiTarski theorem prover [8]. Otherapproaches generate annotated C code from the model, then apply theorem proving to verify stabilityproperties [15, 17]. Functional correctness of auto-generated code from Simulink models has also beenformally verified through theorem provers [1, 2, 6, 28] and model checking [27].The main novel contributions in this paper are: • A methodology to specify high-level properties for control systems in Simulink, supplemented byour custom blocks. • An automatic translation tool from Simulink to Why3, for formal verification of the properties.Our aim in pursuing this approach is to ease the inclusion of verification at design time, by incorporatingit at the block diagram level where system interconnections and insight are best expressed.In our proposed approach, higher-order logic requirements are incorporated and verified as Simulinksignals [5]. Note that this enables the same requirements to be tested by numerical simulation. Thisapproach is partly inspired by the Open Verification Library (OVL) [16], a library of modules that act asproperty checkers and can be placed into a hardware design where they are connected to regular modulesusing the circuit signals. For example, our new custom ‘Require’ block, containing the built-in ‘Assert’block, incorporates pre- and post-conditions into the Simulink model. Then the automatic translationprocess to Why3 identifies the ‘Require’ blocks as goals for theorem proving. Meanwhile, each otherblock in the model is translated into the axioms it asserts relating its input and output signals. Finally,Why3 is invoked to run a chosen SMT solver on the translated ‘theory’ and prove the goals, or otherwise.The rest of the paper presents relevant aspects of the Why3 language, and our proposed approach.Section 2 introduces our developed assertion blocks, particularly the ‘Require’ block. Section 3 presents areview of the Why3 logic language components. Section 4 describes the translation process from Simulinkto Why3, based on our developed theories in Why3 for different blocks. The translation is exemplifiedby a discrete system. Section 5 explains the verification of Lyapunov stability for the same discretesystem. The presented approach is discussed in the same section. Finally, the conclusion is presented inSection 6.
We are developing a set of assertion blocks analogous to OVL for hardware verification [16], to add toSimulink models to test and verify high-level requirements through specifications (as assertions) overthe signals in the models. The OVL was developed to facilitate the addition of verification conditions(predefined logic expressions that can act as assertions, e.g. ‘event X always happens’, assumptions orcoverage points) to any hardware design, for assertion-based verification [13] or formal verification. TheOVL checkers, written in Hardware Description Languages, receive names of the signals of interest asinputs, and are being monitored at simulation time when assertion violations are being recorded. Thesame assertions can serve as goals for formal verification.The new blocks represent a range of assertions (logic expressions) to describe properties of interestrelated to control systems over signals, and others refer to structure for the assertions. Available Simulinkassertion and logic blocks like ‘Compare To Zero’, ‘Compare To Constant’ and ‘Assertion’ are also usedas part or in conjunction with the new assertion blocks. The ‘Require’ block denotes a requirement in aHoare triple form [3] { preconditions } → [model] { postconditions } , (1)where the logic expressions for the preconditions and postconditions are translated from the assertionblocks connected to the ‘Require’ block. The translator automatically produces the verification goals2 a) The ‘Require’ Block. (b) ‘Enabled Subsystem’ InnerBlocks. (c) The ‘Require’ Block and InnerComponents. Figure 1: A First-Order System Specified with Assertion Blocks in Simulink.for the theorem prover from these blocks. The ‘Require’ block is an enabled subsystem with an ‘Assert’block inside (Fig. ?? ). They use native Simulink and MATLAB formats, thus allowing numerical testingin simulation and coverage assessment.In the first-order system of Fig. 1, we have implemented a Lyapunov function V ( x ) = x for the anal-ysis of stability. Three ‘Require’ blocks (‘require’, ‘require1’ and ‘require2’) describe stability propertiesparametrised by signals in the model: • ‘require’: x (cid:54) = 0 → V ( x ( k )) − V ( x ( k − <
0. If the signal x is different from 0, the signal leavingthe ‘Difference’ block should be negative. • ‘require1’: x (cid:54) = 0 → V ( x ) >
0. If the signal x is different from 0, then the signal V ( x ) should bepositive. • ‘require2’: x = 0 → V ( x ) = 0. If the signal x is 0, then the signal V ( x ) should be zero.More complex assertions are built-up from multiple basic blocks, which guarantees their translatabilityinto Why3 (using the predefined theories for basic blocks) and preserves the modularity in Simulink.Properties can be verified in independent sections of blocks, or even within blocks inside subsystems.Individual proofs can be assembled into a hierarchical proof structure, to provide conclusions at a higherlevel from the sequence and relations of the lower-level proofs. The Why3 logic language is based on polymorphic types and first-order logic, although higher-order logicsyntax is allowed [4, 11]. The WhyML language targets the formal verification of programs, by includingloops and other control structures. The Why3 logic language is sufficient to express and verify blockdiagrams from Simulink due to their declarative nature. The translation from Simulink to Why3 logicexpressions is based on the following Why3 components: • Theories: blocks of logic expressions with types, functions, predicates, axioms, lemmas and verifi-cation goals that can be used or cloned in a modular manner. More complex theories can be formedfrom basic theories, inheriting the axioms, lemmas, functions and declarations. They contain thecomponents for a proof. 3 heory Product_intuse import int.Intuse import real.RealInfixfunction in1 int: realfunction in2 int: realfunction out1 int: realaxiom v: forall k:int. out1 k = in1 k *. in2 kaxiom c1: forall k:int. in1 k >. 0.0 /\ in2 k >. 0.0 -> out1 k >. 0.0axiom c2: forall k:int. in1 k <. 0.0 /\ in2 k <. 0.0 -> out1 k >. 0.0end
Figure 2: Theory for the ‘Product’ Block. theory
We have developed a library of theories that provide axioms applicable to the input-output behaviourcorresponding to popular Simulink blocks, including arithmetic and logic, to facilitate the translationinto Why3. The theories are cloned following the components of a block diagram, and the axioms withineach theory are parametrised according to signal names and specific values (constants, input ranges orinitial conditions). The theory developed for the ‘Product’ block is shown in Fig. 2. In the future,additional linear algebra and control systems related theories will be developed as needed in order toverify more complex systems and properties, as in [15].In the
Product_block theory, the theories of the integer and real numbers are imported first. Then,two functions are used for the inputs and another for the output. They receive an integer (time sample)and produce a real (the signal value at that time). Three axioms describe the multiplication and somesign properties: for all sampled times, forall k:int. , the output signal is equal to the multiplication ofthe inputs ( axiom v ); the multiplication of two positive numbers produces a positive number ( axiom c1 );and the multiplication of two negative numbers produces a positive number ( axiom c2 ). The algebraicand logic symbols are used with a dot when a theory mixes integer and real numbers (i.e., both theoriesare imported in the same theory), whereas if a theory only uses integer or real numbers, the symbols areused without the dot [4]. 4 heory CompareToZero_neq_intuse import int.Intuse import real.RealInfixuse import bool.Boolfunction in1 int: realfunction out1 int: boolaxiom v1: forall k:int. out1 k = True -> in1 k <>. 0.0axiom v2: forall k:int. out1 k = False -> in1 k = 0.0end
Figure 3: Theory for the ‘Compare To Zero’ Block, for the ∼ = 0 Case.The theory developed for the ‘Compare To Zero’ block with condition case ∼ = 0 (not equal to zero)is shown in Fig. 3.In the CompareToZero_block_ne0 theory, the theories of the integer, real and Boolean numbersare imported first. Then, a function is defined for the input and another for the output. The inputreceives an integer (time sample) and produces a real value (the signal value at that time), whereas theoutput produces a Boolean value (
True or False ). Two axioms describe the conversion from a realvalued signal (input) to a Boolean (output), according to the comparison criterion: if the input value isdifferent to zero ( axiom v1 ), the output is
True ; and if the input value is zero the output is
False . The automatic translation process from Simulink to Why3 converts the signals and blocks into higher-order logic predicates [5]. For example, the functionality of a ‘Product’ block over two inputs and anoutput signal in discrete intervals of time ( k ) is expressed in higher-order logic [5] as P roduct ( in , in , out ) ≡ ∀ k. out ( k ) = in ( k ) ∗ in ( k ) . (2)The proposed and implemented procedure for the translation of systems as block diagrams in Simulinkinto Why3 language consists of the following steps, executed in automatically in MATLAB when callingthe translator function:1. Identification of the specification blocks from the rest of the model. For example, ‘Require’ blocksand their associated preconditions and postconditions, where the latter two are directly connectedto the respective labelled inputs of the ‘Require’ blocks.2. The Abstract Syntax Tree (AST) [7,23] of the model is computed automatically. We implemented aprogram that identifies the predecessors and successors of each block, using the block manipulationfunctions available in MATLAB. Each AST is translated into a theory in Why3, named after themodel to be verified. Relevant numerical theories including real numbers, integers and Booleanexpressions are imported.3. Then, every signal that leaves a block is identified, named and translated to real or Booleanfunctions. Functions that receive an integer represent discrete signals (i.e., signals of the form y ( k )where k is the time step): function
If the Simulink model has nested generic subsystems (i.e., grouping blocks with the purpose ofproviding modularity), each subsystem is translated into a theory from the computation of its internalAST and verification goals, using the previously described procedure. These theories are then clonedinto other theories, following the flow diagrams of upper levels.Consider the first-order discrete system in Simulink shown in Fig. 4, a simplification of Fig. 1. Thetranslator produces the theory in Fig. 5 automatically.The name of the theory, in the first line, corresponds to the name of the model file in Simulink.Then, the theories of the integer, real and Boolean numbers are imported. Functions corresponding tothe signals that come out of the blocks are added to the theory, specifying if they correspond to real orBoolean signals, and following the convention for the naming described before.Subsequently, the theories of the different blocks in the Simulink model are cloned and parametrisedfrom our library (contained in a file named ‘simulink’) mentioned in Section 3. The order of the blocksis given by the Simulink automatic identification numbering: the ‘Sum’ block, the ‘Product’ block (theLyapunov function), the ‘Unit Delay’ block of the Lyapunov function, the ‘Gain’ block to achieve the6ubtraction of the values of the Lyapunov function combined with the ‘Sum’ block, the ‘Gain’ blockand the ‘Unit Delay’ of the system. When a theory from the library is cloned, its name is changed tothe name of the block. The input and output functions of the new theory are renamed according tothe signals connected to the respective block. Examples of value parametrisation in the cloning are theaxioms corresponding to the gains
Neg_gain and x_gain , for the ‘Gain’ blocks in the model.The theories corresponding to the comparison blocks (the two ‘Compare To Zero’ blocks) are clonedconsidering the parameters assigned in the model, i.e., calling the respective theory that correspondsto a comparison of any of the possible types { == , ∼ = , >, <, > = , < = } . Finally, the verification goalis added by connecting the precondition and postcondition following the structure of the Hoare triple, not_zero_x_op1 k = True -> desc_grad_op1 k = True . Consider the same first-order linear discrete system as in Fig. 4, for a single dimension (scalar), x ( k + 1) = ax ( k ) , (3)and a metric for stability given by a quadratic Lyapunov function, that is always positive regardless ofits input, V ( x ) = x . (4)A system like the one in our example is considered stable if its trajectory in time converges to aregion or point in the state space, starting from different initial conditions. A way to prove stabilityof the system in Fig. 4 is the existence of a Lyapunov function that preserves its expected propertiesof positivity, and that will decrease (if the system is stable), stay in the same value (if the system ismarginally stable), or increase (if the system is unstable). Stability is proved in our examples for differentvalues of the gain a computing the discrete gradient descent of the Lyapunov function (the differencebetween the value of the function in the current time interval and in the previous interval).The conditions to prove stability through the signals has been expressed in two assertions added asblocks to the Simulink model: when the signal x is not zero (precondition), if the system is stable thegradient of the Lyapunov function should be negative (postcondition), x ( k ) (cid:54) = 0 → V ( x ( k )) − V ( x ( k − < goal G1 whenthe gain is > = 1 or < = −
1, as the implication would lead to True → False, which is in turn False. Thevalidity of the proof is expected when the gain is < > − → True,and the later to False → False, all implications resulting in True. These conclusions are derived fromthe truth table for the logical implication operation.
The stability assertions specified in the Simulink model have been checked in simulation, for differentgain values in the system loop. If the system does not comply with the specified property, the ‘Assert’block inside the ‘Require’ block will fire up a flag when running the Simulink model. The results fordifferent gains are shown in Table 1, conforming to our predictions: failure when the gain is > = 1 and < = − < > − x ( k ) is 0 and thus the precondition is false. The goal has been verified for different gains using the CVC3 SMT solver, compatible for theories withreal and integer numbers, linear arithmetic, equalities and inequalities. The results of the verification areshown in Table 2. The highlighted results indicate proved stability (not including marginal), from thevalidity of both the precondition and postcondition, and the ‘Unknown’ results show that stability couldnot be proved for those gains. These results match the assertion checks presented in Table 1. CVC3automatically computed the validity of the verification goals in less than 150 seconds, our specified timelimit in Why3. 7able 1: Checks with Different GainsGAIN CHECK RESULT-1.1 Fail-1.0 Fail -0.5 Pass
X.gain ) x ( k ) (cid:54) = 0 → V ( x ( k )) − V ( x ( k − < -0.5 Valid True True This paper presented an approach to formally verify high level properties of interest of control systems(e.g., stability) as Simulink models in a more automatic manner in Why3 (using theorem proving tools).The Simulink model is specified or annotated with assertions in blocks from our proposed referencelibrary. The Simulink models are translated automatically into the Why3 logic syntax, from the signalsin time and the block connections (computing an abstract syntax tree). The translation process is helpedby our library of theories in Why3 that correspond to the functionality of a set of Simulink blocks. Theassertion blocks are translated into verification goals, and then proved by calling different theorem proversin Why3 according to their capabilities (theories).An example of the translation and verification was provided in the form of a first-order discrete8ystem, coupled with a Lyapunov function as a metric for stability. The verification goals were proved orotherwise, according to the expected stability results for different parameters (gain and initial values),using the CVC3 theorem prover (or its embedded SMT solver). Assertion checks were performed overthe Simulink model in simulation, to show the useful duality of our proposed specification approach.Immediate future work includes extending our approach to multidimensional problems by incorpo-rating linear algebra theories in Why3. Other future work includes the expansion of our libraries ofassertion blocks and theories for supported Simulink blocks, for other high-level properties of interest(e.g., robustness) and stability methods. We are intending to apply our methodology and tools to adiverse range of systems.
Acknowledgment
The work presented in this paper was supported by the EPSRC grant EP/J01205X/1 RIVERAS: RobustIntegrated Verification of Autonomous Systems.
References [1] A. Anta, R. Majumdar, I. Saha, and P. Tabuada. Automatic verification of control system imple-mentations. In
Proc. EMSOFT , pages 9–18, Scottsdale, AZ, USA, October 2010.[2] R. Arthan, P. Caseley, C. O’Halloran, and A. Smith. ClawZ: control laws in Z. In
Proc. ICFEM ,pages 169–176, York, UK, September 2000.[3] R. Arthan, U. Martin, and P. Oliva. A Hoare logic for linear systems.
Formal Aspects of Computing ,25:345–363, 2011.[4] F. Bobot, J.C. Filliˆatre, C. March´e, G. Melquiond, and A. Paskevich.
The Why3 Platform . UniversityParis-Sud, CNRS, Inria, March 2013.[5] A. Camilleri, M. Gordon, and T. Melham. Hardware verification using higher-order logic. TechnicalReport 91, University of Cambridge, Computer Laboratory, September 1986.[6] A. Cavalcanti and P. Clayton. Verification of control systems using Circus. In
Proc. ICECCS ,Stanford, CA, USA, 2006.[7] C. Chen, J.S. Dong, and J. Sun. A formal framework for modeling and validating Simulink diagrams.
Formal Aspects of Computing , 21(5):451–483, 2009.[8] W. Denman, M.H. Zaki, S. Tahar, and L. Rodrigues. Towards flight control verification usingautomated theorem proving. In
NASA Formal Methods , volume 6617 of
Lecture Notes in ComputerScience , pages 89–100. Springer Berlin Heidelberg, 2011.[9] C. Dixon, A. Winfield, and M. Fisher. Towards temporal verification of emergent behaviours inswarm robotic systems. In
Towards Autonomous Robotic Systems , volume 6856 of
Lecture Notes inComputer Science , pages 336–347, 2011.[10] J. Ezekiel, A. Lomuscio, L. Molnar, S. Veres, and M. Prebody. Verifying fault tolerance and self-diagnosability of an autonomous underwater vehicle. In
Proc. IJCAI , pages 1659–1664, Barcelona,Spain, July 2011.[11] J.C. Filliˆatre. One logic to use them all. In
Automated Deduction , volume 7898 of
Lecture Notes inComputer Science , pages 1–20, 2013.[12] M. Fisher, L. Dennis, and M. Webster. Verifying autonomous systems.
Communications of theACM , 56(9):84–93, September 2013.[13] H.D. Foster, A.C. Krolnik, and D.J. Lacey.
Assertion-Based Design . Kluwer Academic Publishers,2004.[14] D. Henrion, M. Ganet-Schoeller, and S. Bennani. Measures and LMI for space launcher robustcontrol validation. In
Proc. Robust Control Design , volume 7, pages 236–241, Denmark, 2012.915] H. Herencia-Zapana, R. Jobredeaux, S. Owre, P.L. Garoche, E. Feron, G. Perez, and P. Ascariz.PVS linear algebra libraries for verification of control software algorithms in C/ACSL. In
NASAFormal Methods , volume 7226 of
Lecture Notes in Computer Science , pages 147–161, 2012.[16] Accellera Systems Initiative.
Accellera Standard OVL V2 Library Reference Manual . AccelleraSystems Initiative, 2013.[17] R. Jobredeaux, T. Wang, and E. Feron. Autocoding software with proofs I: Annotation translation.In
Proc. DASC , pages 7C1–1–7C1–13, 2011.[18] M.E. Johnson. Model checking safety properties of servo-loop control systems. In
Proc. DSN , pages45–50, 2002.[19] S. Mitsch, K. Ghorbal, and A. Platzer. On provable safe obstacle avoidance for autonomous roboticground vehicles. In
Proc. RSS , Berlin, Germany, June 2013.[20] R. Muradone, D. Bresolin, L. Geretti, P. Fiorini, and T. Villa. Robotic surgery.
IEEE Robotics &Automation Magazine , 18(3):24–32, September 2011.[21] A. Richards and J.P. How. Model predictive control of vehicle maneuvers with guaranteed completiontime and robust feasibility. In
Proc. ACC , volume 5, pages 4034–4040, June 2003.[22] A. Richards and J.P. How. Decentralized model predictive control of cooperating UAVs. In
Proc.CDC , volume 4, pages 4286–4291, December 2004.[23] P. Roy and N. Shankar. SimCheck: a contract type system for Simulink.
Innovations in Systemsand Software Engineering , 7:73–83, 2011.[24] A.E. Rugina and J.C. Dalbin. Experiences with the GENE-AUTO code generator in the aerospaceindustry. In
Proc. ERTS , 2010.[25] R. Simmons, C. Pecheur, and G. Srinivasan. Towards automatic verification of autonomous systems.In
Proc. IROS , volume 2, pages 1410–1415, Takamatsu, Japan, November 2000.[26] G. Sirigineedi, A. Tsourdos, B.A. White, and R. Zbikowski. Kripke modelling and verification oftemporal specifications of a multiple UAV system.
Annals of Mathematics and Artificial Intelligence ,63(1):31–52, September 2011.[27] M. Staats and M.P.E. Heimdahl. Partial translation verification for untrusted code-generators. In
Formal Methods and Software Engineering , volume 5256 of
Lecture Notes in Computer Science ,pages 226–237, 2008.[28] A. Toom, N. Izerrouken, T. Naks, M. Pantel, and O. Ssi Yan Kai. Towards reliable code generationwith an open tool: Evolutions of the Gene-Auto toolset. In
Proc. ERTS , Toulouse, France, May2010.[29] W. Wang, P. Menon, D. Bates, S. Ciabuschi, N.M. Gomes Paulino, E. Di Sotto, A. Bidaux, A. Garus,A. Kron, S. Salehi, and S. Bennani. Verification and validation of autonomous rendezvous systemsin the terminal phase. In
Proc. AAIA Guidance, Navigation, and Control Conference , pages 1–11,Minneapolis, MN, USA, August 2012.[30] M. Webster, M. Fisher, N. Cameron, and M. Jump. Formal methods for the certification of au-tonomous unmanned aircraft systems. In
Formal Methdos for the Certification of AutonomousUnmanned Aircraft Systems , volume 6894 of