Formal Consistency Checking over Specifications in Natural Languages
FFormal Consistency Checking over Specifications inNatural Languages
Rongjie Yan
State Key Laboratory of Computer Science,Institute of Software,Beijing, China
Chih-Hong Cheng
Industrial Software TechnologiesABB Corporate Research,Ladenburg, Germany
Guangquan Zhang and Yesheng Chai
School of Computer Science & Technology,Soochow University,Suzhou, China
Abstract —Early stages of system development involve outlin-ing desired features such as functionality, availability, or usability.Specifications are derived from these features that concretizevague ideas presented in natural languages. The challenge for theverification and validation of specifications arises from the syntaxand semantic gap between different representations and the needof automatic tools. In this paper, we present a requirement-consistency maintenance framework to produce consistent rep-resentations. The first part is the automatic translation fromnatural languages describing functionalities to formal logic withan abstraction of time. It extends pure syntactic parsing byadding semantic reasoning and the support of partitioning inputand output variables. The second part is the use of synthesistechniques to examine if the requirements are consistent in termsof realizability. When the process fails, the formulas that causethe inconsistency are reported to locate the problem.
I. I
NTRODUCTION
Early stages of system development involve outlining de-sired features such as functionality, availability, or usability.The importance of early verification of high-level requirementscan never be underestimated: Precise specifications can avoidoverly frequent corrections in late developing phases, and theyserve as a reference model or a test-case generator later insystem and architecture design. Nevertheless, the challengefor the early verification process arises from the syntax andsemantic gap between different representations. We considera specification to have three views. Initial specifications aregenerated from these features that concretize vague ideaspresented in natural languages. Precise specifications can berepresented under the assist of logic. But for reference modelsor even test case generators, very often they are represented asprograms [13], [18]. In many cases, one struggles to maintainthe following two types of consistency. • The semantic consistency between the intuitive meaningof a textual specification and its logic presentation. • The realizability consistency between the intuitive mean-ing of a logic specification and a model or a test casegenerator - a logic specification should guarantee theexistence of an implementation .In this paper, we present a framework that endeavors tobring consistency to different representations of the high-level specification, via a synthesis-based transformation . The E.g., consider the specification “ the output should always be the same asthe input 3 time ticks from now ”. Although it can be rewritten formally usingLTL ( G ( output ↔ XXX input ) ), it is unrealizable, as any implementationrequires to have clairvoyance over the future events. Specifica(on+in+natural+languageStanford+NLP++parser Dic(onaryLTL+formulasTransla(onInputs/Outputs Heuris(c+refinementG4LTL+Realizable?No YesController Time+abstrac(on+Seman(c+reasoning No Fig. 1. The overall workflow of SpecCC. goal is to generate, from requirements in natural languages,to two other representations automatically, thereby ensuringconsistency. We also call it
LTL-oriented , as the intermediateformat makes the use of linear temporal logic (LTL) [16].The workflow of our framework (see Figure 1 for overview)involves a loop of three stages. The first part is a translationframework that can automatically turn requirements written innatural language into formulas formalized in LTL. This portionincludes a heuristic partition of input and output variables,together with an appropriate abstraction of time. Importantly,apart from pure lexical parsing, we also introduce the use ofdictionaries to infer the meaning. This allows the solver toreason status scenarios such as on(light) ≡ ¬ off(light) , therebyavoiding the creation of two propositions. The second part isthe use of LTL synthesis [9], [2], [3] to detect the realizabilityof specifications. The last part is a semi-automatic procedurethat changes the physical interpretation of time and examinesif the partition of input-output variables is reasonable. Whenchanges from any of the two items arise, the specification isrefined and is re-analyzed by the LTL synthesis engine.The paper is structured as follows. After a brief summaryof related work, we describe the CARA system [1] as amotivating example. We present the translation frameworkfrom natural to formal language in Section IV. Then, we sum-marize the consistency checking between formal language andimplementation, together with the following refinement stepsin Section V. Section VI explains the implementation of theprototype tool SpecCC (
Spec ification C onsistency C hecking)and presents the results of applying the tool on a set of non-trivial examples. We conclude the paper in Section VII. a r X i v : . [ c s . S E ] O c t I. R
ELATED W ORK
Managing consistencies for requirements of different rep-resentations is essential in system modeling and subsequentimplementation / refinement steps.Translating requirements in natural languages to LTL hasbeen investigated in many works, which is always based on asubset of natural language. There are approaches using prop-erty patterns and scopes to assist the specification of formalproperties specified in LTL as well as other languages [6],[12]. Machine learning and data mining are also used toclassify natural language into temporal requirements [14]. Thetranslator implemented in [21] is one of the available tools,which translates specifications on hardware systems into LTLfor model checking. In the research of consistency checkingof requirements, one of the pioneering work in this domainis [8], which analyzes requirements expressed in the SCR(Software Cost Reduction) tabular notation. Li, Liu and Hehave generated pre and post conditions for system operationsof UML requirements model to check requirement consistencybased on the semantics [11]. Overall, all above results do notsupport the consistency checking between formal requirementsand implementability.In the community of formal methods, Uchitel, Brunetand Chechik have proposed a method to synthesize a set ofsafety properties in Fluent Linear Temporal Logic (FLTL) togenerate partial behavior models to reduce the effort of modelconstruction [20]. One of the closest work from ours is anapplication inside robotic domains [10], where the techniqueparses templates to generate LTL formulas and synthesizes acontroller. However, the translated specification only supportsa strict language subset (GR-1 [15]) that disallows the useof
Until and strictly limits the use of
Next to at most once.Contrarily, we allow the user to still present their require-ments in English, while the synthesis engine supports fullLTL and allows simple semantic reasoning. Another similarwork [7] translates specifications in stylized natural languageinto various formalisms from an intermediate representation,and checks the realizability with synthesis analysis with GR-1synthesis tool. Although the translation distinguishes variablesand valuations, time is not considered and valuations have tobe converted to the bit-level for synthesis.III. E
XAMPLE : CARA S
YSTEM
We use the CARA (Computer-Aided Resuscitation Algo-rithm) infusion pump control system [1], [17] as a runningexample. It is a software system developed to drive a highoutput infusion pump used for fluid resuscitation of patientssuffering from conditions that lead to hypotension, based onthe collected data on blood pressure. The main functionalityis to monitor and control the operations of the infusion pump,to drive resuscitating fluids into a patient’s blood stream.
System Description . We focus on the three main modes ( wait , manual and auto-control ) of operation in CARA, and check theconsistency of the specification on the three working modes.The system is in the wait mode when the pump is off.It does not monitor the blood pressure or control the pump.The system enters the manual mode when the pump is initiallyturned on. In this level, the software only performs monitoringfunctions. If the power supply to the pump is lost, the controlgoes to a backup battery and triggers an alarm. To leave this mode, either the pump is turned off, or a button for the auto-control mode is pressed by the care-giver. That button is onlyenabled when the pump is in the normal operative mode. Inthe auto-control mode, CARA is responsible for monitoring thestatus lines from the pump and controlling the infusion rate.When the system is in the auto-control mode, it can use threesources of blood pressure data. Among the sources such as anarterial line, a pulse-wave and a cuff, the arterial line has thehighest priority. That is, if the three sources are both available,the arterial line is used as the input. If the arterial line sourceis lost, the pulse-wave has a higher priority than the cuff.The corresponding requirements from [1] are organized asthe input of our framework. The following list enumeratessome requirements as illustrating examples. Req-08
If Air Ok signal remains low, auto-control mode is termi-nated in 3 seconds.
Req-17
When auto-control mode is entered, eventually the cuff willbe inflated.
Req-28
If a valid pressure is unavailable in 180 seconds, manual-mode should be triggered.
Req-32
If pulse wave or arterial line is available, and cuff isselected, corroboration is triggered.
Req-42
When auto-control mode is running, and the arterial line orpulse wave or cuff is lost, an alarm should sound in 60 seconds.
Req-44
If pulse wave and arterial line are unavailable, and cuff isselected, and blood pressure is not valid, next manual mode isstarted.
IV. M
AINTAINING C ONSISTENCIES BETWEEN N ATURAL L ANGUAGE AND F ORMAL L ANGUAGE
Maintaining consistencies between natural language andformal logic is obtained by automatic translation from naturallanguage to LTL. In the following subsections, we review thedefinition of LTL, propose a restricted English grammar forsyntactic parsing, and give the underlying algorithm for anextended reasoning on the semantic level and the techniquesfor abstracting time. Afterwards, we provide heuristics onpartitioning input and output variables to apply the synthesistechniques in the second level.
A. LTL Syntax and Semantics
Linear temporal logic [16] is a modal temporal logicwhose modalities refer to time. It can express properties ofpaths in a computation tree. The formulas are built up froma set of atomic propositions (AP), logical operations andtemporal modal logics. The set of supported LTL formulasare constructed from atomic propositions p ∈ AP accordingto the following grammar. φ ::= p | ¬ φ | φ ∨ φ | X φ | ♦ φ | (cid:3) φ | φ U φ , where X is the Next time operator, ♦ is the Eventually operator, (cid:3) is the
Always operator, and U is the Until operator. Given negation ( ¬ ) and disjunction ( ∨ ), we can defineconjunction ( ∧ ), implication ( → ) and equivalence ( ↔ ). Thesemantics of an LTL formula is defined on an infinite sequenceof truth assignments to the atomic propositions. For example,a temporal logic formula p is satisfiable if there is a sequenceof states such that p is true from the start of the sequence. B. Grammar of the Structured English
Natural languages allow a rich diversity of sentence struc-tures and easily cause semantic ambiguities. Hence, we do A detailed list of requirements is listed in the appendix. ot provide a translator for the full set of natural languages.Instead, we select a more structured subset, whose constitutinggrammar can encompass the needs of functional requirementsin most cases. In this subset, we only support present, futureand passive tenses with correct syntax according to the Englishgrammar. To be concise, we only present positive form of thegrammar. The negative form can be inferred accordingly. Wefirst present the grammar of the structured English. sentence ::= ( subclause, ) ∗ . ( clauses ) . ( , subclause ) ∗ subclause ::= ( subordinator ) . ( clauses ) clauses ::= ( clause ) . [ , ( conjunction ) . ( clause )] clause ::= [ modifier ] . ( subject ) . ( predicate ) . [ constraint ] subject ::= substantivessubstantives ::= ( substantive ) . [( conjunction ) . ( substantives )] substantive ::= nounpredicates ::= [ modality ] .predicatepredicate ::= verb | ( be ) .participle | ( be ) . ( complement ) participle ::= ( verb ) . ( ed ) | ( verb ) . ( ing ) complement ::= adjective | adverbmodality ::= shall | should | will | would | can | could | must subordinator ::= if | after | once | when | whenever | while | before | until | next modifier ::= globally | always | sometimes | eventually conjunction ::= and | or constraint ::= in t where t is a time constant, * (star) means the presence ofzero or more subcomponent, . (dot) means the composition ofdifferent components, and [ ] means the optional components.For noun , verb , participle , adjective and adverb , we do notdecompose them any more. The proposed grammar allowsextracting temporal operators according to the elements in subordinator and modifier .According to the definition of the grammar, a sentenceconsists of at least one clause. A clause consists of at leastone main phrase for the core meaning of the sentence,and optionally conditional or time clauses that extend themeaning. In addition to the skeleton shown in the abovegrammar, a main phrase may also contain filter constructionssuch as the and a that will be ignored in the translation.For example, for Requirement Req-17 : “
When auto-controlmode is entered, eventually the cuff will be inflated. ”, the mainclause is “ eventually the cuff will be inflated ”, and “
When auto-control mode is entered ” is a time clause. In this requirement,“ eventually ” is a modifier for the temporal operator
Eventually and “ the ” is a filter construction in the main phrase “ eventuallythe cuff ”. C. Lexical and Syntactic Parsing
A specification here is a set of sentences (requirements) inthe structured English. For every sentence, first it is parsed by anatural language parser to extract all grammatical ingredients.Then according to the dependency relation extracted by theparser, the translator decomposes the sentence into clausesrecursively to isolate the independent temporal units. After thedecomposition, we construct a syntax tree for the ingredientsof the sentence, to extract atomic propositions, and to inferthe temporal relationship of individual temporal elements ac-cording to the subordinators and modifiers. Usually an atomicproposition comes from a subject and its predicate extractedby the dependency relation from the parser, i.e., in the formof predicate subject , to combine a variable and its valuation.For multiple subjects connected with conjunctions, they are sentencesubclause clausesubordinator clause subject predicatemodifiersubject predicatewhen the cuff will be inflatedauto-control mode is enteredeventually
Fig. 2. The syntax tree of Requirement
Req-17 decomposed to generate different atomic propositions and thenconnected by the corresponding logic operators.Reconsider the example used in the last subsection. Re-quirement
Req-17 consists of two clauses. The correspondingsyntax tree is illustrated in Figure 2. By removing tenseinformation in the predicates, two atomic propositions, en-ter auto-control mode and inflate cuff are extracted from thetwo clauses. Note that only one word will be taken as thesubject in a clause in the translation process. To keep everyproposition meaningful, we need to add “ ” to contact relativewords together for a complete subject in a requirement.According to the existing study on the scopes and patterns,and their templates on LTL formula generation [19], we haveselected some frequently used patterns (Universality and Ex-istence), and the corresponding templates in our translator, totransform the elements in the syntax tree to the correspondingformulas. For example, according to the syntax tree for
Req-17 and its keywords, modifier “eventually” is translated intounary temporal operator “ ♦ ”, which will be joined with atomicproposition inflate cuff . Subordinator “when” is translated intothe logic operator “ → ”, to connect the two formulas from thetwo clauses. Accordingly, the temporal formula for Req-17 is (cid:3) ( enter auto-control mode → ♦ inflate cuff ) . D. Moving from Lexical Parsing towards Semantic Reasoning
To preserve semantic consistency and obtain succinct tem-poral formulas, additional to pure lexical parsing, we alsoadopt semantic reasoning over specifications in natural lan-guage. The semantic reasoning discussed here is to extract therelation between adjectives and adverbs (we call them antonymcandidates in the rest of the paper) respectively according totheir meaning in a specification, instead of semantic roles la-belled by a parser. More precisely, after extracting the antonymcandidates used in a specification, we group them in pairsof semantically contrasting words by looking up an antonymdictionary specified by users. With these semantically relatedwords, we can reduce the number of atomic propositions usedin the generated formulas, and avoid adding the assumptionson the mutual exclusive propositions.The most direct way is to look for the antonyms onlinefor every given word, which is easy to be implemented buttime consuming. We propose a two-step antonym extractionprocess to compute the pairs of antonyms in a specification.Intuitively, pairs of antonyms always come from the sentenceswith same subjects. Therefore, we first organize the antonymcandidates related to the same subjects according to theirdependency relation (cid:104) subject, dependent (cid:105) extracted by thenatural language parser, where every word in the dependentset is initialized with color green . If the number of words inthe dependent set for a subject is larger than one, we use an lgorithm 1:
Semantic reasoning input : Specification S in natural language output : pairs of antonyms in the specification begin wordset = ∅ // extract subject dependent words, and storeantonym candidates and the dependency relationsin wordset and subject , respectively subject = extract ( S, wordset ) for ∀ s ∈ subject do if | s.dep | > thenfor ∀ w ∈ s.dep doif w.color == green then if wordset ( w ) .antonym == ∅ then wordset ( w ) .antonym = online ( w ) antonmy = s.dep ∩ wordset ( w ) .antonym if antonmy (cid:54) = ∅ then w.color = blue for w (cid:48) ∈ antonym do w (cid:48) .color = blue wordset ( w (cid:48) ) .antonym = wordset ( w (cid:48) ) .antonym ∪ { w } return ( subject, wordset ) antonym dictionary to check whether semantically contrastingwords exist in the same set. Otherwise, we continue to dealwith other groups. The reason being that we cannot use thederived antonyms for the corresponding proposition reduction.The details on the reasoning process are depicted in Algorithm1, where subject is to group elements depending on the samesubjects, and wordset is to store the set of antonym candidateswith the extracted antonyms. To mark the status of dependentwords in a subject during semantic reasoning, we use twocolors, where blue for having found the words in the dependentset of the subject with contrasting meaning by consulting thedictionary, and green stands for the non-existence of antonymsin the set. These colors are indicators for our propositionreduction in the formula transformation process. That is, aword marked with green will be directly converted into atomicpropositions with the corresponding subject. Otherwise, forpairs of antonyms related to the same subject, we first decidethe positive and negative properties. Then we replace the wordsin negative meaning with the negative form of their antonyms.In Algorithm 1, we assume that every antonym candidatecan find its antonyms from an antonym dictionary. We firstinitialize wordset (Line 1) and extract the dependency relationfrom the specification (Line 2). In the extract function, theextracted antonym candidates are stored in wordset , whereinitially the sets of their antonyms are empty. The items in thedependent sets of subjects are initialized with green. Then forevery extracted word w in subject s , if it is not analyzed before(Line 4), we look for its antonyms from the antonym dictionaryand add the result to the antonym set of w in wordset (Line5). If it has been processed, we check whether the intersectionbetween its antonyms and the set of words in s.dep is empty(Line 6). If the interaction is not empty, we say that there existsa pair of antonyms in s.dep , and mark the corresponding wordsin s.dep with blue (Lines 7 and 8). Finally, we will completethe information for the antonyms of w (Line 9).Consider Requirements 32 and 44 presented in Section IIIas an example. The subject pulse wave has dependency rela-tion with available , and unavailable . Without semantic reason- ing, we will create two propositions as available pulse wave ,and unavailable pulse wave . In our semantic reasoning pro-cess, we need to look up the antonyms for available and unavailable in the antonym dictionary. As the intersectionbetween available and the set of antonyms for unavailable isnot empty, we say that the two words constitute a pair ofantonyms. Therefore, we can replace the proposition unavail-able pulse wave by ¬ available pulse wave . Note that wecannot reason antonyms for verbs. The reason being that differ-ent actions capture different semantics and we cannot simplyuse their negation to replace the corresponding antonyms. E. Time Counting and Abstraction
Requirements in natural language may contain timing con-straints. In LTL, one can use
Next operator to emulate discretetime. That is, decide unit time (e.g., 1 second), and definethe elapse of unit time via a
Next operator. For example,for Requirement
Req-08 :“ If Air Ok signal remains low, auto-control mode is terminated in 3 seconds ”, its LTL formulais (cid:3) ( ¬ Air ok signal → XXX terminate auto-control mode ) ,where one X is for one second.Explicit enumerating time units brings a clearer mappingfor requirement analysis. However, the larger the numberof time units in a requirement, the longer of the translatedformula with a higher complexity. To alleviate this problem,we present a rewriting technique that can abstract time with theconsideration of all the requirements inside the specification.Intuitively, we can reduce the numbers of Next operators bydividing them with the greatest common divisor (GCD) forthe lengths of successive
Next operators in the requirements.The method is sound, i.e., a specification with the greatestcommon divisor reduction is realizable, if and only if theoriginal specification is realizable.For example, in Requirements
Req-08 , Req-28 and
Req-42 in Section III, the lengths for successive
Next operatorsare 3, 180 and 60, respectively. The greatest common divisoramong the three numbers is 3. Therefore, after reduction,the lengths of the successive
Next operators for the threerequirements are 1, 60 and 20, respectively. So the corre-sponding formula for
Req-08 becomes (cid:3) ( ¬ Air ok signal → X terminate auto-control mode ) . From this example, one canobserve that the reduction is quite conservative and stillproduces formula with huge amount of Next , thereby hinderingthe later synthesis process.The complete algorithm overcomes the limitation via theintroduction of arrival errors , borrowed from the concept ofjitter . Intuitively, an arrival error specifies, for an action,whether it is allowed for arriving later or earlier. This allowsus to search for a better re-encoding of consecutive Next operators. With arrival errors, one can formulate the probleminto an optimization problem (nonlinear with degree at most 2).Formally, let
Θ = { θ , . . . , θ n } be a set of numbers ofsuccessive Next operators in the transformed formulas froma specification where θ i (cid:54) = θ j for any i (cid:54) = j . Let d be thedevisor, θ (cid:48) i be the length of successive Next operators afterthe abstraction from θ i , and ∆ i be the error introduced in Jitter is the undesired deviation from true periodicity of an assumedperiodic signal in electronics and telecommunications. he abstraction. With these notations, we obtain the followingconstraint system, for all i ∈ { , . . . , n } , θ i = θ (cid:48) i × d + ∆ i , − d < ∆ i < d, d and θ (cid:48) i ∈ Z + , and ∆ i ∈ Z , (1)where Z is the set of integers, and Z + is the set of positiveintegers including zero. Intuitively, θ i = θ (cid:48) i × d + ∆ i meansthat the algorithm rewrites θ i consecutive Next operators into θ (cid:48) i Next operators, via a factor of d . This generates an error of ∆ i . If a ∆ i is positive, it means that the corresponding eventhappens earlier in the rewritten specification. Otherwise, theevent arrives later than expected. In the above GCD case, ∆ i should always be .Based on the above constraint system, we derive a two-objective optimization problem: Reduce the use of Next (i.e.,reduce θ (cid:48) i ), and minimize the controlled error (i.e., reduce | ∆ i | ,where | ∆ i | stands for the absolute value of ∆ i ).minimize n (cid:88) i =0 θ (cid:48) i , minimize n (cid:88) i =0 | ∆ i | (2)As the first objective is more important, we let the userspecify an allowed upper-bound B for (cid:80) ni =0 | ∆ i | , and restrictthe domain for every ∆ i such that either ∆ i ∈ [0 , d ] or ∆ i ∈ [ − d, (i.e., arrival error can be either early or late, but notboth). Then the second objective can be rewritten as a singleinteger linear constraint (as one can remove the absolute sign).This reduces the two-objective optimization problem into asingle-objective optimization problem, which can be efficientlysolved by modern SMT solvers (e.g., Yices 2 [5]) via bit-blasting.Back to the example and consider again Requirements Req-08 , Req-28 and
Req-42 in Section III. The set for thelengths of
Next operators in these requirements is { , , } ,that is, θ = 3 , θ = 180 and θ = 60 . If we specify ∆ i ≥ for ≤ i ≤ , and set B to , we could obtain an optimalresult with d = 60 , and θ (cid:48) = 0 , θ (cid:48) = 3 , θ (cid:48) = 1 , ∆ = 3 , ∆ = 0 , ∆ = 0 . F. Input and Output Variable Partition
To check the consistencies of a specification, the translatoralso needs to distinguish the propositions for input and outputvariables in an LTL formula. Very commonly, we find thatevery line of a specification discusses how the system shouldresponse, when encountering a certain scenario. This can beunderstood as having an implication between an occurringevent and the corresponding response. Sometimes, the responsemight need to endure until another environment event appears.Generally speaking, for left-hand parts in an implication, orfor right-hand parts of the
Until operator, we assume that theconstituting variables are input variables. If a proposition inpositive form appears in the both sides of such operators, it isassumed as an output.We use this idea to analyze every requirement and main-tain, for each requirement, two disjoint sets for input andoutput variables. Then when all requirements are analyzed,a unification is performed by checking consistencies amongvariable sets. Whenever there exists a conflict (i.e., a variableis considered to be an input variable in one requirement but an output variable in another requirement), the translator regardsit as an output. If no input variable exists after the partition, weselect one randomly from the set of output variables as the in-put. After the partition, the translator also asks the user for theassistance. For example, for the generated formula from
Req-32 : (cid:3) ((available pulse wave || available arterial line) &&select cuff → trigger corroboration), we can conclude that available pulse wave , available arterial line and select cuff are the input variables, and the output variable is trig-ger corroboration .V. M AINTAINING C ONSISTENCIES BETWEEN F ORMAL L ANGUAGE AND I MPLEMENTABILITY
A. Synthesis
In real world applications, there are always a large amountof requirements describing functionality of the desired system.The inconsistency between various requirements may arisebetween different sequences of events (actions) defined bythe semantics of the corresponding LTL formulas. As thesynthesis technique is capable of generating a controller overa set of LTL formulas, the inconsistency between differentLTL formulas will prevent the synthesis tool from creating acontroller. Therefore, we can borrow the idea of LTL-basedsynthesis techniques to check the inconsistency among therequirements represented in LTL formulas.
B. Heuristic refinement
We use
G4LTL [3] as our underlying LTL synthesis engine,which automatically checks consistency between LTL formulastransformed from the requirements in a specification, andreports the inconsistency between neighbored requirements.However, it cannot always provide reasons causing the in-consistency, especially when the pair of requirements causinginconsistencies are not neighbored. We suggest the followingstrategy to isolate the problem. • Locate the pair of inconsistent requirements. The processstarts from a subset of consistent formulas. We can addmore formulas continuously to the subset to check whichone is not consistent with the subset. Once we have lo-cated the problem, we could filter out other formulas thatdo not contain any propositions of the located formulas. • Adjust the existing input and output variable partition. Asthe input and output partition method is heuristic, we mayhave to adjust the partition if
G4LTL reports the incon-sistency. The propositions belonging to the intermediatedvariables in the located formulas are targets to be adjusted. • Modify the requirements. When we cannot guarantee theconsistency by adjusting the partition on input and outputvariables, the specification is not consistent indeed. There-fore, we analyze the semantics of the formulas to extractthe reason of inconsistency, and modify the requirementsto ensure the consistency of the specification.VI. I
MPLEMENTATION AND E VALUATION
We have implemented a research prototype for the abovementioned framework. For the language translation part, theimplementation extends the Stanford NLP parser [4] to ana-lyze the requirements in the structured English to extract itsingredients for LTL generation. In the consistency checkingpart, we integrate the synthesis library
G4LTL [3].e have selected three case studies coming from differentapplication areas, to maintain the consistencies in the twolevels, and check the specification scalability that we canhandle in the second level.
CARA
We first have considered Requirements 1, 7, 8, 13, 16,17, 20, 28, 32, 34, 42, 44, 48, 49 and 54 among seventyrequirements that are related to the switches betweenworking modes and the corresponding responsibilitiesfor the CARA system as the input of our framework.We also adopted the detailed specifications for the threecomponents (Pump Monitor, Blood Pressure Monitor(BPM), Polling Algorithms (PA)) in the CARA systemfor evaluation.
TELEPROMISE
We have also considered the functionalspecification for the TELEPROMISE project demonstra-tor as a case study , which covers five generic applica-tions: Shopping application, Article processing applica-tion, On-line reservation application, Information appli-cation and Local bulletin board application. Rescue robot
We have modified the rescue robot scenarioused in [10] as the third case study. The responsibility ofthe robots in this scenario is to look for the injured peopleand take them to a medic who is in some room. Differentnumbers of rooms and robots have been considered here,with the constraint that two robots cannot be in the sameroom at the same time.Table I lists the results for the corresponding componentspecifications and the scale of every specification (number oflines, and number of I/Os), as well as the time consumption(in seconds) for realizability checking of requirements inLTL formulas. The performance of
G4LTL are sensitive tothe number of subformulas, the number of input and outputvariables, and the length of a formula. Therefore, the timeconsumption for realizability checking of various specificationsare quite different. For the consistency maintenance betweennatural language and formal language, the time consumptionis linear to the number of requirements, which is ignored here.In the CARA case study, the specifications for the trans-formation between three working modes and the three com-ponents are consistent. During consistency checking for theTELECOMPRISE case study,
G4LTL failed to generate con-trollers for the last two specifications. The failure was causedby the classification of input and output variables. Afterlocating the problem and modifying the input/ output variablepartition, the specifications are consistent.VII. C
ONCLUSION
We have proposed a framework for ensuring consistenciesbetween different representations of specifications. The frame-work maintains semantic consistencies between oral and for-mal specifications, while ensuring the implementability usingsynthesis. With the framework, we propose a time extractionmechanism, input-output partition heuristics, and an extensionfrom pure syntactic parsing of sentences to reason on thesemantics level. The approach is evaluated under a rich setof examples with positive results.For future work, we will increase the maturity level ofour implementation by capturing the semantics in natural XPERIMENTAL RESULTS
Name No. Specification formulas in out time(s)CARA 0 Working mode and switching 30 22 28 341 Pump Monitor 20 9 14 22.1.1 BPM: cuff detector 14 13 12 12.1.2 BPM: AL detector 15 11 14 22.1.3 BPM: pulse wave detector 14 9 12 12.2.1 BPM: initial auto control 16 14 15 12.2.2 BPM: first corroboration 19 11 16 292.2.3 BPM: valid ctrl blood pressure 13 11 10 22.2.4 BPM: cuff source handler 11 9 10 22.2.5 BPM: arterial line blood pressure 16 9 13 12.2.6 BPM: arterial line corroboration 12 8 13 12.2.7 BPM: pulse wave handler 20 10 21 233.1 (PA) Model ctrl algorithm 9 15 11 33.2 (PA) Polling algorithm 56 12 20 11TELE 1 Shopping 29 11 24 82 Article processing 17 3 13 13 On-line reservation 6 3 4 14 Information 15 8 14 15 Local bulletin board 17 7 16 1Robot 1 A robot with 4 rooms 9 2 5 12 A robot with 9 rooms 14 2 10 13 Two robots with 5 rooms 25 2 11 7 language to better understand the meaning of sentences thusavoiding manual matching between different words, and byusing learning techniques for more generic natural languages.R
EFERENCES[1] CARA increment 3 (CARA-Tagged-Requirements), version 1.2, March21, 2001.[2] A. Bohy, V. Bruyre, E. Filiot, N. Jin, and J.-F. Raskin. Acacia+, a toolfor LTL synthesis. In
CAV , pages 652–657. Springer, 2012.[3] C.-H. Cheng, C.-H. Huang, H. Ruess, and S. Stattlemann. G4LTL-ST: Automatic generation of PLC programs. In
CAV , pages 541–549.Springer, 2014.[4] M.-C. de Marneffe, B. MacCartney, and C. D. Manning. Generatingtyped dependency parses from phrase structure parses. In
SLT , 2006.[5] B. Dutertre. Yices 2.2. In
CAV , pages 737–744. Springer, 2014.[6] M. B. Dwyer, G. S. Avrunin, and J. C. Corbett. Property specificationpatterns for finite-state verification. In
FMSP , pages 7–15, 1998.[7] S. Ghosh, D. Elenius, W. Li, P. Lincoln, N. Shankar, and W. Steiner.ARSENAL: Automatically extracting requirements specifications fromnatural language.
CoRR , abs/1403.3142, 2014.[8] C. L. Heitmeyer, R. D. Jeffords, and B. G. Labaw. Automatedconsistency checking of requirements specifications.
ACM Trans. Softw.Eng. Methodol. , 5(3):231–261, July 1996.[9] B. Jobstmann and R. Bloem. Optimizations for LTL synthesis. In
FMCAD , pages 117–124, 2006.[10] H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas. Translating structuredenglish to robot controllers.
Advanced Robotics , 22(12):1343–1359,2008.[11] X. Li, Z. Liu, and J. He. Consistency checking of UML requirements.In
ICECCS , pages 411–420, 2005.[12] O. Mondragon, A. Q. Gates, and S. Roach. Prospec: Support forelicitation and formal specification of software properties.
Electron.Notes Theor. Comput. Sci. , 89(2):67–88, 2003.[13] C. D. Nguyen, A. Marchetto, and P. Tonella. Combining model-basedand combinatorial testing for effective test case generation. In
ISSTA ,pages 100–110, 2012.[14] A. P. Nikora and G. Balcom. Automated identification of LTL patternsin natural language requirements. In
ISSRE , pages 185–194, 2009.[15] N. Piterman, A. Pnueli, and Y. Saar. Synthesis of reactive (1) designs.In
VMCAI , pages 364–380, 2006.[16] A. Pnueli. The temporal logic of programs. In
FOCS , pages 46–57,1977.[17] A. Ray and R. Cleaveland. Unit verification: the CARA experience.
Int. J. Softw. Tools Technol. Transf. , 5(4):351–369, May 2004.18] A. Riesco. Test-case generation for maude functional modules. In
WADT , pages 287–301, 2012.[19] S. Salamah, A. Gates, S. Roach, and O. Mondragon. Verifying pattern-generated LTL formulas: a case study. In
SPIN , pages 200–220, 2005.[20] S. Uchitel, G. Brunet, and M. Chechik. Synthesis of partial behaviormodels from properties and scenarios.
IEEE Trans. Software Eng. ,35(3):384–406, 2009.[21] L. Zilka. Temporal logic for man. Bachelor’s thesis. Brno, But FIT,2010. A PPENDIX
A. A full list of checked requirements for the CARA system,together with the automatically generated LTL formulas
Req-01
The CARA will be operational whenever the LSTAT ispowered on.
Req-07
If an occlusion is detected, and auto control mode is run-ning, auto control mode will be terminated.
Req-08
If Air Ok signal remains low, auto control mode is termi-nated in 3 seconds.
Req-13.1
If arterial line and pulse wave are corroborated, and cuffis available, next arterial line is selected.
Req-13.2
If pulse wave is corroborated, and cuff is available, andarterial line is not corroborated, next pulse wave is selected.
R-13.3
If arterial line is not corroborated, and pulse wave is notcorroborated, and cuff is available, then cuff is selected.
Req-16
If a pump is plugged in, and an infusate is ready, and theocclusion line is clear, auto control mode can be started.
Req-17.1
When auto control mode is running, eventually the cuffwill be inflated.
Req-17.2
If start auto control button is pressed, and cuff is notavailable, an alarm is issued and override selection is provided.
Req-17.3
If alarm reset button is pressed, the alarm is disabled.
Req-17.4
If override selection is provided, if override yes ispressed, and arterial line is not corroborated, next arterial lineis selected.
Req-17.5
If override selection is provided, if override yes ispressed, and arterial line is corroborated, and pulse wave is notcorroborated, next pulse wave is selected.
Req-17.6
If override selection is provided, if override no ispressed, next manual mode is started.
Req-17.7
If cuff and arterial line and pulse wave are not available,next manual mode is started.
Req-20
If manual mode is running and start auto control button ispressed, next corroboration is triggered.
Req-28
If a valid blood pressure is unavailable in 180 seconds,manual mode should be triggered.
Req-32.1
If pulse wave or arterial line is available, and cuff isselected, corroboration is triggered.
Req-32.2
If pulse wave is selected, and arterial line is available,corroboration is triggered.
Req-34
When auto control mode is running, termi-nate auto control button should be available.
Req-42
When auto control mode is running, and the arterial line,or pulse wave or cuff is lost, an alarm should sound in 60seconds.
Req-44
If pulse wave and arterial line are unavailable, and cuff isselected, and blood pressure is not valid, next manual mode isstarted.
Req-48.1
Whenever termiante auto control button is selected, aconfirmation button is available.
Req-48.2
If a confirmation button is available, and confirma-tion yes is pressed, manual mode is started.
Req-48.3
If a confirmation button is available, and confirmation nois pressed, auto control mode is running.
Req-48.4
If a confirmation button is available, and confirma-tion yes is pressed, next confirmation yes is disabled.
Req-48.5
If a confirmation button is available, and confirmation nois pressed, next confirmation no is disabled.
Req-48.6
If a confirmation button is available, andterminate auto control button is pressed, nextterminate auto control button is disabled.
Req-49
When a start auto control button is enabled, thestart auto control button is enabled until it is pressed.
Req-54
If auto control mode is running, and impedance reading isunavailable, next auto control model is terminated.
We have applied semantic reasoning and the abstractionechnique on time constants. As there are no explicit positiveand negative meaning for a pair of antonyms, the selection forthe positive form is randomly. When there is only one pair ofadjective or adverb antonyms for a subject, we abbreviate thepropositions by just using the subject and its negative form.For example, available pulse wave is rewritten as pulse wave,and ¬ available pulse wave is rewritten as ¬ pulse wave.However, other propositions with the same subject will notbe modified.The translated LTL formulas are as follows: Req-01 (cid:3) (power lstat → ( ♦ (operational cara))) Req-07 (cid:3) (detect occlusion && run auto control mode → ( ♦ (terminate auto control))) Req-08 (cid:3) ( ¬ air ok signal → terminate auto control mode) Req-13.1 (cid:3) (corroborate arterial line && corroborate pulse wave&& cuff → select arterial line) Req-13.2 (cid:3) (corroborate pulse wave && cuff && ¬ corroborate arterial line → select pulse wave) Req-13.3 (cid:3) ( ¬ corroborate arterial line && ¬ corroborate pulsewave && cuff → select cuff) Req-16 (cid:3) (plug pump && ready infusate && clear occlusion line → start auto control mode) Req-17.1 (cid:3) (run auto control mode → ( ♦ (inflate cuff))) Req-17.2 (cid:3) (press start auto control button && ¬ cuff → is-sue alarm && provide override selection) Req-17.3 (cid:3) (press alarm reset button → ¬ alarm)
Req-17.4 (cid:3) (provide override selection → ( (cid:3) (press override yes&& ¬ corroborate arterial line → select arterial line))) Req-17.5 (cid:3) (provide override selection → ( (cid:3) (press override yes&& corroborate arterial line && ¬ corroborate pulse wave → select pulse wave))) Req-17.6 (cid:3) (provide override selection → ( (cid:3) (press override no → start manual mode))) Req-17.7 (cid:3) ( ¬ cuff&& ¬ arterial line&& ¬ pulse wave → startmanual mode) Req-20 (cid:3) ((run manual mode && press start auto control button) → trigger corroboration) Req-28 (cid:3) (XXX( ¬ blood pressure) → trigger manual mode) Req-32.1 (cid:3) ((pulse wave || arterial line) && select cuff → triggercorroboration) Req-32.2 (cid:3) (select pulse wave && arterial line → triggercorroboration) Req-34 (cid:3) (run auto control mode → terminate auto controlbutton) Req-42 (cid:3) (run mode && ( ¬ arterial line || ¬ pulse wave || ¬ cuff) → (X(sound alarm))) Req-44 (cid:3) ( ¬ pulse wave&& ¬ arterial line && select cuff && ¬ blood pressure → start manual mode) Req-48.1 (cid:3) (select termiante auto control button → confirmationbutton) Req-48.2 (cid:3) (confirmation button && press confirmation yes → start manual mode) Req-48.3 (cid:3) (confirmation button && press confirmation no → run auto control mode) Req-48.4 (cid:3) (confirmation button && press confirmation yes →¬ confirmation yes) Req-48.5 (cid:3) (confirmation button && press confirmation no →¬ confirmation no) Req-48.6 (cid:3) (confirmation button && press terminating autocontrol button → ¬ terminating auto control button)
Req-49 (cid:3) (start auto control button → ( (cid:3) ( ¬ press start autocontrol button → (start auto control button W press startauto control button)))) Req-54 (cid:3) (run auto control mode && ¬ impedance reading →→