Runtime Verification For Timed Event Streams With Partial Information
Martin Leucker, César Sánchez, Torben Scheffel, Malte Schmitz, Daniel Thoma
RRuntime Verification For Timed Event StreamsWith Partial Information ∗ Martin Leucker , César Sánchez , Torben Scheffel , Malte Schmitz , andDaniel Thoma University of Lübeck, Germany {leucker,scheffel,schmitz,thoma}@isp.uni-luebeck.de IMDEA Software Institute, Spain [email protected]
Abstract.
Runtime Verification (RV) studies how to analyze executiontraces of a system under observation. Stream Runtime Verification (SRV)applies stream transformations to obtain information from observedtraces. Incomplete traces with information missing in gaps pose a commonchallenge when applying RV and SRV techniques to real-world systemsas RV approaches typically require the complete trace without missingparts. This paper presents a solution to perform SRV on incompletetraces based on abstraction. We use TeSSLa as specification languagefor non-synchronized timed event streams and define abstract eventstreams representing the set of all possible traces that could have occurredduring gaps in the input trace. We show how to translate a TeSSLaspecification to its abstract counterpart that can propagate gaps throughthe transformation of the input streams and thus generate sound outputseven if the input streams contain gaps and events with imprecise values.The solution has been implemented as a set of macros for the originalTeSSLa and an empirical evaluation shows the feasibility of the approach.
Runtime verification (RV) is a dynamic formal method for software systemreliability. RV studies how to analyze and evaluate traces against formal specifi-cations and how to obtain program traces from the system under observation,e.g., through software instrumentation or utilization of processors’ embeddedtrace units. Since RV only inspects one execution trace of the system, it is oftenregarded to be a readily applicable but incomplete approach, that combinesformal verification with testing and debugging.Most early RV languages were based on logics common in static verification,like LTL [21], past LTL adapted for finite paths [19,12,4], regular expressions [22] ∗ This work was funded in part by the Madrid Regional Government under project
S2018/TCS-4339 (BLOQUES-CM) , by EU H2020 projects 731535
Elastest and 732016
COEMS , by Spanish National Project
BOSCO (PGC2018-102210-B-100) and by theBMBF project
ARAMiS II with funding ID 01 IS 16025. a r X i v : . [ c s . L O ] J u l M. Leucker, C. Sánchez, T. Scheffel, M. Schmitz, and D. Thoma values resetscond tt ff ff ff ff tt ff ff sum values resetscond tt ff ff tt ff ff sum
Fig. 1.
Example trace for a typical SRV specification (left) with two input streams values (with numeric values) and resets (with no internal value). The intention of thespecification is to accumulate in the output stream sum all values since the last reset.The intermediate stream cond is derived from the input streams indicating if reset hascurrently the most recent event, and thus the sum should be reset to 0. If the inputstreams contain gaps (dotted regions on the right) some information can no longer becomputed, but after a reset event the computation recovers from the data loss duringthe gap. (cid:62) denotes events with unknown data. or timed regular expressions [2]. For these logics, the monitoring problem con-sists on computing a Boolean verdict indicating whether the trace fulfills thespecification. In contrast to static analysis, however, considering only a singleconcrete trace enables the application of more complex analyses: Stream Run-time Verification (SRV) [11,6,7] uses stream transformations to derive additionalstreams as verdicts from the input streams. Using SRV one can still check if theinput stream is conformant with a specification, but additionally verify streamsin terms of their events’ data: streams in SRV can store data from richer domainsthan Booleans, including numerical values or user defined data-types, so SRVlanguages can extract quantitative values and express quantitative propertieslike “ compute the average retransmission time ” or “ compute the longest durationof a function ”. SRV cleanly separates the temporal dependencies that the streamtransformation algorithms follow from the concrete operations to be performedon the data, which are specific to each data-type. As an example for SRV considerthe trace diagram on the left of Fig. 1. We consider non-synchronized eventstreams, i.e., sequences of events with increasing timestamps and values from adata domain. Using non-synchronized event streams one can represent eventsarriving on different streams with different frequencies in a compact way withlittle computation overhead because there is no need to process additional syn-chronization events in the stream-transformation process. In this paper we usethe TeSSLa specification language [7], an SRV language for non-synchronized,timed event streams. TeSSLa has been defined to be general enough to allow fora natural translation from other common SRV formalisms, e.g., Lola [11] andStriver [17]. Therefore, our results carry over to these languages as well.Since RV is performed on traces obtained from the system under test in thedeployed environment, it is a common practical problem for RV techniques thatthe traces do not cover the entire run of the system. However, most of the previousRV approaches require the trace to be available without any interruptions in orderto obtain a verdict, because this knowledge is assumed in the semantics of thespecification logics. Especially in the case of interrupted traces with some data untime Verification For Timed Event Streams With Partial Information 3 losses applying previous RV techniques can be very challenging. Unfortunatelythose traces occur very often in practical testing and debugging scenarios, e.g.,due to interrupted experiments, buffer overflows, network errors or any othertemporary problem with the trace retrieval.In this paper we present a solution to the problem of evaluating traces withimprecise values and even interrupted traces. Our only assumption is that wehave exact knowledge of the imprecision of the trace in the following sense: (1) forevents with imprecise values we know the range of values and (2) for data losseswe know when we stop getting information and when the trace becomes reliableagain. We call such a sequence of uncertainty a gap in the trace. Our solutionautomatically propagates gaps and imprecisions, and allows to obtain soundverdicts even in the case of missing information in the input trace. The assumptionis reasonable in our target application: online non-intrusive monitoring low-levelembedded software.Fig. 1 on the right displays a case where the input stream values has a longgap in the middle. It is not possible to determine the events in the output stream sum during that gap, because we do not even know if and how many events mighthave happened during that gap. Thus, the intermediate stream cond and theoutput stream sum simply copy that gap representing any possible combinationof events that might occur. The first event after the gap is the one with the value3 on values . Because no reset happened after the end of the gap, we would add 3to the latest event’s value on sum , but the gap is the latest on sum . Thus, we onlyknow that this input event on values causes an event on sum independently ofwhat might have happened during the gap, but the value of that event completelydepends on possible events occurring during the gap. After the next event on reset the values of the following events on sum are independent of any previousevents. The monitor can fully recover from the missing information during thegap and can again produce events with precise values.In order to realize this propagation of gaps through all the steps of the stream-transformation we need to represent all potentially infinitely many concrete traces(time is dense and values are for arbitrary domains) that might have happenedduring gaps and imprecise events. An intuitive approach would be a symbolicrepresentation in terms of constraint formulas to describe the set of all possiblestreams. These formulas would then be updated while evaluating the input trace.While such a symbolic execution might work for shorter traces, the representationcan grow quickly with each input event. Consequently the computational costcould grow prohibitively with the trace length for many input traces. Instead,in this paper we introduce a framework based on abstraction [9,10]. We useabstraction in two ways:(1) Streams are lifted from concrete domains of data to abstract domains tomodel possible sets of values. For example, in our solution a stream can storeintervals as abstract numerical values.(2) We define the notion of abstract traces, which extend timed streams withthe capabilities of representing gaps. Intuitively, an abstract trace over-
M. Leucker, C. Sánchez, T. Scheffel, M. Schmitz, and D. Thoma approximates the sets of concrete traces that can be obtained by filling thegaps with all possible concrete events.Our approach allows for both gaps in the input streams as well as events carryingimprecise values. Such imprecise values can be modelled by abstract domains,e.g., intervals of real numbers. Since we rely on abstraction, we can avoid falsenegatives and false positives in the usual sense: concrete verdicts are guaranteedto hold and imprecise verdicts are clearly distinguished from concrete verdicts.The achievable precision depends on the specification and the input trace.After reproducing the semantics of the basic TeSSLa operators in Section 2, weintroduce abstract semantics of the existing basic operators of TeSSLa in Section 3.Using these abstract TeSSLa operators, we can take a TeSSLa specification onstreams and replace every TeSSLa operator with its abstract counterpart andderive an abstraction of the specification on abstract event streams. We showthat the abstract specification is a sound abstraction of the concrete specification,i.e., every concrete verdict generated by the original specification on a set S of possible input traces is represented by the abstract verdict applied to anabstraction of S . We further show that the abstract TeSSLa operators are aperfect abstraction of their concrete counterparts, i.e., that applying the concreteoperator on all individual elements of S doesn’t get you more accurate results.Finally, we show that an abstract TeSSLa specification can be implemented usingthe existing TeSSLa basic operators by representing an abstract event stream asmultiple concrete event streams carrying information about the events and thegaps. Since the perfect accuracy of the individual abstract TeSSLa operators doesnot guarantee perfect accuracy of their compositions, we discuss the accuracyof composed abstract TeSSLa specifications in Section 4. Next we present inSection 5 an advanced use-case where we apply abstract TeSSLa to streams overa complex data domain of unbounded queues, which are used to compute theaverage of all events that happened in the sliding window of the last five timeunits. In the final Section 6 we evaluate the overhead and the accuracy of theabstractions presented in this paper on representative example specifications andcorresponding input traces with gaps. Related Work.
SRV was pioneered by LOLA [11,14,15]. TeSSLa [7] generalises toasynchronous streams the original idea of LOLA of recursive equations over streamtransformations. Its design is influenced by formalisms like stream programminglanguages [18,5,16] and functional reactive programming [13]. Other approachesto handle data and time constraints include Quantitative Regular ExpressionsQRE [1] and Signal Temporal Logic [20].While ubiquitous in practice, the problem of gaps in an observation tracehas not been studied extensively. To the best of our knowledge, abstractiontechniques have not been applied to the evaluation of stream-based specifications.However, approaches to handle the absence of events or ordering information havebeen presented for MTL [3] and past-time LTL [24]. State estimation based onMarkov models has been applied to replace absent information by a probabilisticestimation [23]. The concept of abstract interpretation used throughout thispaper has been introduced in [8]. untime Verification For Timed Event Streams With Partial Information 5 A time domain is a totally ordered semi-ring ( T , , , + , · , ≤ ) that is positive, i.e., ∀ t ∈ T ≤ t . We extend the order on time domains to the set T ∞ = T ∪ {∞} with ∀ t ∈ T t < ∞ . Given a time domain T , an event stream over a data domain D is afinite or infinite sequence s = t d t . . . ∈ S D = ( T · D ) ω ∪ ( T · D ) ∗ · ( T ∞ ∪ T · D ⊥ ) where D ⊥ := D ∪ {⊥} and t i < t i +1 for all i with < i + 1 < | s | ( | s | is ∞ forinfinite sequences). An infinite event stream is an infinite sequence of timestampsand data values representing the stream’s events. A finite event stream is a finitesequence of timestamped events up to a certain timestamp that indicates theprogress of the stream. A stream can end with:– a timestamp without a data value that denotes progress up to but not includingthat timestamp,– a timestamp followed by ⊥ (or a data value) which denotes progress up to andincluding that timestamp (and an event at that timestamp),– ∞ , which indicates that no additional events will ever arrive on this stream.We refer to these cases as exclusive , inclusive and infinite progress , resp.Streams s ∈ S D can be seen as functions s : T → D ∪ {⊥ , ? } such that s ( t ) is a value d if s has an event with value d at time t or ⊥ if there is no eventat time t . For timestamps after the progress of the stream s ( t ) is ?. Formally, s ( t ) = d if s contains td , s ( t ) = ⊥ if s does not contain t , but contains a t (cid:48) > t or s ends in t ⊥ , and s ( t ) = ? otherwise. We use ticks ( s ) for the set { t ∈ T | s ( t ) ∈ D } of timestamps where s has events. A stream s is a prefix of stream r if ∀ t ∈ T s ( t ) ∈ { r ( t ) , ? } . We use the unit type U = { (cid:31) } for streamscarrying only the single value (cid:31) .A TeSSLa specification consists of a collection of stream variables and possiblyrecursive equations over these variables using the operators nil , unit , time , lift , last and delay . The semantics of recursive equations is given as the least fixed-point of the equations seen as a function of the stream variables and fixed inputstreams. See [7] for more details and Appendix A for an elaborated example. nil = ∞ ∈ S ∅ is the stream without any events and infinite progress. unit = 0 (cid:31) ∞ ∈ S U is the stream with a single unit event at timestamp zero andinfinite progress. time : S D → S T , time ( s ) := z maps the event’s values to their timestamps: z ( t ) = t if t ∈ ticks ( s ) and z ( t ) = s ( t ) otherwise. lift : ( D ⊥ × . . . × D n ⊥ → D ⊥ ) → ( S D × . . . × S D n → S D ) , lift ( f )( s , . . . , s n ) := z lifts a function f on values to a function on streams by applying f to the stream’svalues for every timestamp. The function f must not generate new events, i.e.,must fulfill f ( ⊥ , . . . , ⊥ ) = ⊥ . z ( t ) = (cid:40) f ( s ( t ) , . . . , s n ( t )) if s ( t ) (cid:54) = ? , . . . , s n ( t ) (cid:54) = ?? otherwise M. Leucker, C. Sánchez, T. Scheffel, M. Schmitz, and D. Thoma last : S D × S D (cid:48) → S D , last ( v, r ) := z takes a stream v of values and a stream r of triggers. It outputs an event with the previous value on v for every event on r . z ( t ) = d t ∈ ticks ( r ) and ∃ t (cid:48) We can now specify the stream transformations shown on the left inFigure 1 in TeSSLa. Let resets ∈ S U and values ∈ S Z be two external input eventstreams. We then derive cond ∈ S B and lst , sum ∈ S Z as follows: cond = slift ( ≤ )( time ( resets ) , time ( values )) lst = merge ( last ( sum , values ) , zero ) sum = slift ( f )( cond , lst , values ) f : B × Z × Z → Z with f ( c, l, v ) = (cid:40) if c = true l + v otherwiseUsing the operators described above one can only derive streams with times-tamps that are already present in the input streams. To derive streams with eventsat computed timestamps one can use the delay operator, which is described inAppendix B. Preliminaries. Given two partial orders ( A, (cid:22) ) and ( B, (cid:22) ) , a Galois Connection is a pair of monotone functions α : A → B and γ : B → A such that, for all a ∈ A and b ∈ B , α ( a ) (cid:22) b if and only if a (cid:22) γ ( b ) . Let ( A, (cid:22) ) be a partialorder, f : A → A a monotone function and γ : B → A a function. The function f : B → B is an abstraction of f whenever, for all b ∈ B , f ( γ ( b )) (cid:22) γ ( f ( b )) . untime Verification For Timed Event Streams With Partial Information 7 If ( α, γ ) is a Galois Connection between A and B , the function f : B → B suchthat f ( b ) := α ( f ( γ ( b )) is a perfect abstraction of f .In this section we define the abstract counterparts of the TeSSLa operators,listed in Section 2. A data abstraction of a data domain D is an abstract domain D with an element (cid:62) ∈ D and an associated concretisation function γ : D → D with γ ( (cid:62) ) = D . The abstract value (cid:62) represents any possible value from thedata domain and can be used to model an event with known timestamp butunknown value. A gap is a segment of an abstract event stream that representsall combinations of events that could possibly occur in that segment (both interms of timestamps and values). Hence an abstract event stream consists of anevent stream over a data abstraction and an associated set of known timestamps: Definition 1 (Abstract Event Stream). Given a time domain T , an abstractevent stream over a data domain D is a pair ( s, ∆ ) with s ∈ S D and ∆ ⊆ T suchthat ∆ can be represented as union of intervals whose (inclusive or exclusive)boundaries are indicated by events in an event stream. Further, we require s ( t ) (cid:54) = ⊥ ⇒ t ∈ ∆ . The set of all abstract event streams over D is denoted as P D . Theconcretisation function γ : P D → S D is defined as γ (( s, ∆ )) = { s (cid:48) | ∀ t ∈ ticks ( s ) s ( t ) ∈ γ ( s (cid:48) ( t )) ∧ ∀ t ∈ ∆ \ ticks ( s ) s ( t ) = s (cid:48) ( t ) } If the data abstraction is defined in terms of a Galois Connection a refinementordering and abstraction function can be obtained. The refinement ordering ( P D , (cid:22) ) is defined as ( s , ∆ ) (cid:22) ( s , ∆ ) iff ∆ ⊇ ∆ and ∀ t ∈ ticks ( s ) s ( t ) (cid:22) s ( t ) ∧ ∀ t ∈ ∆ \ ticks ( s ) s ( t ) = s ( t ) . The abstraction function α : 2 S D → P D isdefined as α ( S ) = sup { ( s, T ) | s ∈ S } . Note, if the data abstraction is defined interms of a Galois Connection, ( α, γ ) is a Galois Connection between S D and P D .An abstract event stream s = ( s (cid:48) , ∆ ) ∈ P D can also be seen as a function s : T → D ∪ { ? , ⊥ , (cid:96) } with s ( t ) = s (cid:48) ( t ) if t ∈ ∆ and s ( t ) = (cid:96) otherwise. Aparticular point t of an abstract event stream s can be either (a) directly at anevent ( s ( t ) ∈ D ), (b) in a gap ( s ( t ) = (cid:96) ), (c) in a gapless segment without anevent at t ( s ( t ) = ⊥ ), or (d) after the known end of the stream ( s ( t ) = ? ).We denote D ⊥ def = D ∪ {⊥ , (cid:96) } . If D is a data abstraction of a data domain D with an associated concretisation function γ , then D ⊥ is a data abstraction of D ⊥ with an associated concretisation function γ ⊥ : D ⊥ → D ∪{⊥} with γ ⊥ ( d ) = ⊥ if d = ⊥ D ∪ {⊥} if d = (cid:96) γ ( d ) if d ∈ D tt (cid:62) ff (cid:96) ⊥ tt ff ⊥ γ γ ⊥ B ⊥ B B ⊥ B The above diagram shows a possible data abstraction B of B and the correspond-ing data abstraction B ⊥ . Using the functional representation of an abstract eventstream we can now define the abstract counterparts of the TeSSLa operators: nil = ( ∞ , T ) ∈ P ∅ is the empty abstract stream without any gaps. unit = (0 (cid:31) ∞ , T ) ∈ P U is the abstract stream without any gaps and a singleevent at timestamp 0. M. Leucker, C. Sánchez, T. Scheffel, M. Schmitz, and D. Thoma time : P D → P T , time ( s ) := z is equivalent to its concrete counterpart; onlythe data domain is extended: z ( t ) = t if t ∈ ticks ( s ) and z ( t ) = s ( t ) otherwise. lift : ( D ⊥ ×· · ·× D n ⊥ → D ⊥ ) → ( P D ×· · ·×P D n → P D ) , lift ( f )( s , . . . , s n ) := z can be defined similarly to its concrete counterpart, because the abstract func-tion f takes care of the gaps: z ( t ) = (cid:40) f ( s ( t ) , . . . , s n ( t )) if s (cid:54) = ? , . . . , s n (cid:54) = ?? otherwiseThe operator lift is restricted to those functions f that are an abstraction offunctions f that can be used in lift , that is, f ( ⊥ , . . . , ⊥ ) = ⊥ . Using the abstractlift we can derive the abstract counterparts of const and merge : const ( c )( a ) := lift ( f c )( a ) with f c ( d ) := c if d (cid:54) = (cid:96) and f c ( (cid:96) ) := (cid:96) otherwisemaps all events’ values to a constant while preserving the gaps. Using const wecan define constant signals without any gaps, e.g., true := const ( true )( unit ) or zero := const (0)( unit ) . merge ( x, y ) := lift ( f )( x, y ) with f ( a (cid:54)∈ { (cid:96) , ⊥} , b ) = a , f ( ⊥ , b ) = b , f ( (cid:96) , b ∈{ (cid:96) , ⊥} ) = (cid:96) , and f ( (cid:96) , b (cid:54)∈ { (cid:96) , ⊥} ) = (cid:62) . xyz The diagram on the right shows an example tracemerging the events of the streams x and y . Thesymbol ◦ indicates a point-wise gap. Note how anevent on the first stream takes precedence over a gap on the second stream,but not the other way round, similarly to how events from the first stream areprioritized if both streams have an event at the same timestamp. last : P D × P D → P D , last ( v, r ) := z has three major extensions over itsconcrete counterpart:(1) (cid:62) is added as an output in case an event on r occurs and there were eventson the stream v of values but all followed by a gap.(2) (cid:96) is outputted for all gaps on the stream r of trigger events if there havebeen events on the stream v of values.(3) (cid:96) can also be output if an event occurs on r and no event occurred on v before except for a gap.The parts similar to the concrete operator are typeset in gray: z ( t ) = d t ∈ ticks ( r ) ∧ ∃ t (cid:48) By replacing every TeSSLaoperator in Example 1 with their ab-stract counterparts and applying it tothe abstract input streams values ∈ P Z and resets ∈ P U , we derive the abstractstream cond ∈ P B and the recursivelyderived abstract stream sum ∈ P Z : After the large gap on values , the sum streameventually recovers completely. The first reset after the point-wise gap does notlead to full recovery, because at that point the last event on values cannot beaccessed, because of the prior gap. The next reset falls into the gap, so again cond cannot be evaluated. In a similar fashion one can define an abstract delay operator as counterpart of the concrete delay . See Appendix B for details.Following from the definitions of the abstract TeSSLa operators we get: Theorem 1. Every abstract TeSSLa operator is an abstraction of its concretecounterpart. Theorem 1 implies that abstract TeSSLa operators are sound in the followingway. Let o be a concrete TeSSLa operator with the abstract counterpart o andlet s ∈ P D be an abstract event stream with a concretization function γ . Then, o ( γ ( s )) (cid:22) γ ( o ( s )) . Since abstract interpretation is compositional we can directlyfollow from the above theorem: Corollary 1. If a concrete TeSSLa specification ϕ is transformed into a specifi-cation ψ by replacing every concrete operator in ϕ with its abstract counterpart,then ψ is an abstraction of ϕ . Theorem 1 guarantees that applying abstract TeSSLa operators to the abstractevent stream is still sound regarding the underlying set of possible concrete eventstreams. However, we have established no result so far about the accuracy ofthe abstract TeSSLa operators. The abstraction returning only the completelyunknown stream ( ∆ = ∅ ) is sound but useless. The following theorem states,that our abstract TeSSLa operators are optimal in terms of accuracy. Using aperfect abstraction guarantees the abstract TeSSLa operators preserve as muchinformation as can possibly be encoded in the resulting abstract event streams. Theorem 2. Every abstract TeSSLa operator is a perfect abstraction of itsconcrete counterpart. Given a concrete TeSSLa operator o and its abstract counterpart o , andany abstract event stream s ∈ P D with the Galois Connection ( α, γ ) between S D and P D one can show that o ( s ) = α ( o ( γ ( s )) . Applying the abstract operator on the abstract event stream is as good as applying the concrete operator on everypossible event stream represented by the abstract event stream. Thus o is aperfect abstraction of o . (The detailed proof can be found in Appendix C.) Notethat we assume that f is a perfect abstraction of f to conclude that lift ( f ) is a perfect abstraction of lift ( f ) .In Corollary 1 we have shown that a specification ψ (generated by replacing theconcrete TeSSLa operator in ϕ with their abstract counterparts) is an abstractionof ϕ . Note that ψ is in general not a perfect abstraction of ϕ . We study somespecial cases of perfect abstractions of compositional specifications in Section 4.The next result states that the abstract operators can be defined in terms ofconcrete TeSSLa operators. Realizing the abstract operators in TeSSLa does notrequire an enhancement in the expressivity of TeSSLa. Theorem 3. The semantics of the abstract TeSSLa operators can be encoded inTeSSLa using only the concrete operators.Proof. One can observe that the abstract TeSSLa operators are monotone andfuture independent (the output stream up to t only depends on the input streamsup to t .) As shown in [7], TeSSLa can express every such function. (cid:117)(cid:116) A concrete TeSSLa specification consists of stream variables and possibly recursiveequations applying concrete TeSSLa operators to the stream variables. Theorem 1and Corollary 1 guarantee that a concrete TeSSLa specification can be transformedinto an abstract TeSSLa specification, which is able to handle gaps in the inputstreams. Additionally, Theorem 3 states that the abstract TeSSLa operatorscan be implemented using concrete TeSSLa operators. Combining these tworesults, one can transform a given concrete specification ϕ into a correspondingspecification ψ , which realizes the abstract TeSSLa semantics of the operators in ϕ , but only uses concrete TeSSLa operators.However, using the realization of the abstract TeSSLa operators in TeSSLaadds additional cyclic dependencies in ψ between the stream variables. A TeSSLaspecification is well-formed if every cycle of its dependency graph contains at leastone edge guarded by a last (or a delay) operator, which is required to guaranteethe existence of a unique fixed-point and hence computability (see [7]). vr last ( v, r ) Consider the trace diagram on the right showing last ( v, r ) . If v is used in a recursive manner, i.e., v is defined in terms of last ( v, r ) , then the first eventon v could start a gap on last ( v, r ) that could start a gap on v at the sametimestamp. As a result v has an unguarded cyclic dependency and hence thespecification is not well-formed. To overcome this issue one can split up the valueand gap calculation sequentially, reintroducing guards in the cyclic dependency: Definition 2 (Unrolled Abstract Last). We define two variants of the ab-stract last, last ⊥ and last (cid:96) as follows. Let z = last ( v, r ) , then last ⊥ ( v, r ) := z ⊥ untime Verification For Timed Event Streams With Partial Information 11 and last (cid:96) ( v, r, d ) := z (cid:96) . z ⊥ ( t ) = (cid:40) z ( t ) if z ( t ) (cid:54) = (cid:96) ⊥ otherwise z (cid:96) ( t ) = d ( t ) if t ∈ ticks ( d ) (cid:96) if t / ∈ ticks ( d ) ∧ z ( t ) = (cid:96) ⊥ otherwise Function last ⊥ executes a normal calculation of the events, in the same way anabstract last would do, but neglecting gaps and outputting ⊥ as long as thereis no event. Function last (cid:96) takes a third input stream and outputs its eventsdirectly, but calculates gaps correctly as last would do.Since the trigger input of a last operator cannot be recursive in a well-formedspecification, a recursive equation using one last has the form x = last ( v, r ) and v = f ( x, c ) , where c is a vector of streams not involved in the recursion and f does not introduce further last (or delay) operators. Now, this equation systemcan be rewritten in the following equivalent form: x (cid:48) = last ⊥ ( v, r ) v (cid:48) = f ( x (cid:48) , c ) x = last (cid:96) ( v (cid:48) , r, x (cid:48) ) v = f ( x, c ) This pattern can be repeated if multiple recursive abstract lasts are used andcan also be applied in a similar fashion to mutually recursive equations and thedelay operator. A concrete TeSSLa specification ϕ can be transformed into an abstract TeSSLaspecification ψ by replacing the concrete operators with their abstract coun-terparts. For two functions f and g with corresponding abstractions f and g the function composition f ◦ g is an abstraction of f ◦ g . Unfortunately,even if f and g are perfect abstractions, f ◦ g is not necessarily a perfectabstraction. Hence, ψ needs not be a perfect abstraction of ϕ . In this sectionwe discuss the perfection of two common compositional TeSSLa operators: (1)the slift defined in Section 3 is a composition of last in lift , which realizessignal semantics; (2) last ( time ( v ) , r ) , which is a common pattern used whencomparing timestamps.The slift is defined as the lift applied to the synchronized versions x (cid:48) and y (cid:48) of the input streams x and y . The input stream x is synchronized with y bykeeping the original events of x and reproducing the last known value of x forevery timestamp with an event on y , but not on x . Theorem 4. If f is a perfect abstraction of f then slift ( f ) is a perfectabstraction of slift ( f ) .Proof. Since slift is defined on abstract event streams we need to consider gaps.The stream x (cid:48) does not have any gap or event until the first gap or event on x .After the first gap or event on x the synchronized stream x (cid:48) contains a gap or event at every timestamp where x or y contain a gap or event. Because slift is symmetric in terms of the event pattern the same holds for y (cid:48) . By definition, slift ( f )( x, y ) = z contains an event or gap iff x (cid:48) and y (cid:48) contain an event orgap, because f is a total function. The output stream z contains an event iff x (cid:48) and y (cid:48) contain events. The events values are ensured to be as precise as possible,because f is a perfect abstraction of f . (cid:117)(cid:116) a bvr last ( time ( v ) , r ) lastTime ( v, r ) [ a, b ] TeSSLa allows arbitrary computa-tions on the timestamps of events us-ing the time operator. The specification z = time ( v ) derives a stream z from v byreplacing all event’s values in v with theevent’s timestamps. The stream variable z can now be used in any computation expressible in TeSSLa. Hence, TeSSLadoes not distinguish between timestamps and other values, and consequently ab-stract TeSSLa specifications cannot make use of the monotonicity of time. As anexample consider the trace diagram on the right. The stream last ( time ( v ) , r ) is derived from v by composing time and last . Since time changes theevents values with their timestamps, the last does not know any longer thatwe are interested in the last timestamp of v and can only produce an eventwith the value (cid:62) representing all possible values. To overcome this issue we de-fine lastTime ( v, r ) := last ( time ( v ) , r ) and provide a direct abstraction, whichallows a special treatment of timestamps. Definition 3 (Time Aware Abstract Last). Let y = last ( time ( v ) , r ) ,we define lastTime : P D × P D (cid:48) → P T , lastTime ( v, r ) := z as z ( t ) = [ a, b ] if y ( t ) = (cid:62) with a = inf { t (cid:48) < t | ∀ t (cid:48) Theorem 5. lastTime is a perfect abstraction of lastTime . A similar problem occurs if slift is used to compare event’s timestamps. InExample 2 the stream cond derived by comparing the timestamps of values and resets has two events with the unknown data value (cid:62) because of priorgaps on values . Since the slift is defined in terms of lift and last we candefine the function sliftTime ( f )( x, y ) as an abstraction for the special case sliftTime ( f )( x, y ) = slift ( f )( time ( x ) , time ( y )) by using lastTime instead of last and ensuring that f uses interval arithmetics to abstract f . Note that sliftTime ( f ) is a perfect abstraction of sliftTime ( f ) . values resetscond tt ff ff tt ff tt ff tt ff sum Example 3. To illustrate the perfect ab-straction sliftTime we update the def-inition of cond in Example 2 as follows: cond = sliftTime ( ≤ )( resets , values ) . Theevents drawn in red now have concretevalues instead of (cid:62) as in Example 2. untime Verification For Timed Event Streams With Partial Information 13 load 1 .3 .5 .7 .4 .3 .8 (cid:55)→ . (cid:55)→ . , (cid:55)→ . (cid:55)→ . , (cid:55)→ . , (cid:55)→ . (cid:55)→ . < (cid:62) (cid:55)→ . , (cid:55)→ . < (cid:62) (cid:55)→ . , (cid:55)→ . , (cid:55)→ . avg [0 , 1] [ . , . .34 Fig. 2. Example trace of the abstract queue specification. In this section we demonstrate how to apply the techniques presented in thispaper to specifications with richer data domains. In particular, we show now aTeSSLa specification that uses a queue to compute the average load of a processorin the last five time units. The moving window is realized using a queue storingall events that happened in the time window. The stream load ∈ S R contains anevent every time the input load changes: stripped = slift ( remOlder )( time ( load ) , merge ( last ( queue , load ) , (cid:104)(cid:105) ))) queue = lift ( enq )( time ( load ) , load , stripped ) avg = lift ( int )( queue , time ( load )) int ( q, u ) = fold ( f, q, , u ) f ( a, b, v, acc ) = acc + v · ( b − a ) / The queue operation enq adds elements to the queue, while remOlder removeselements with a timestamp older than five time units. The function int accumu-lates all values in the queue weighted by the length of the corresponding signalpiece. The queue operation fold is used to fold the function f over all elementsfrom the queue with the initial accumulator 0 until the timestamp u . Hence f iscalled for every element in the queue with the timestamps a and b , the element’svalue v and the accumulator. Consequently, the specification adds elements tothe queue, removes the expired elements and accumulates the remaining values.Using our approach we replace every operator with its abstract counterpartand represent abstract queues appropriately such that also queues with partlyunknown entries can be modeled. By doing this we obtain a specification that isable to handle gaps in the input stream, as illustrated in Fig. 2.We can extend the example such that the queue only holds a predefinedmaximum number of events (to guarantee a finite state implementation). Whenremoving events we represent these as unknown entries in the abstract queues.The abstract fold is capable of computing the interval of possible average loadsfor queues with unknown elements anyhow.Note that the average load is only updated for every event on the inputstream. Using a delay operator, we can set a timeout whenever an element leaves the sliding window in the abstract setting. The element is removed from thequeue at that timeout and the new value of the queue is updated with theremaining elements. Formal definitions of the queue functions as well as thecomplete specifications are available online . As discussed in Section 3.1 the abstract TeSSLa operators can be implementedusing only the existing concrete TeSSLa operators. We implemented the abstractTeSSLa operators as macros specified in the TeSSLa language itself such that theexisting TeSSLa engine presented in [7] can handle abstract TeSSLa specifications.An abstract event stream ( s, ∆ ) ∈ P D can be represented as two TeSSLa streams s ∈ S D and s d ∈ S X , where X contains the following six possible changes of ∆ :inclusive start, exclusive start, inclusive end, exclusive end, point-wise gap andpoint-wise event in a gap. Using this encoding it is sufficient to look up the latest s d ( t (cid:48) ) with t (cid:48) ≤ t to decide whether t ∈ ∆ . While this encoding already allows adecent implementation of abstract TeSSLa we go one step further and assume afinite time domain with a limited precision, e.g., 64 bit integers or floats. Underthis assumption there is always a known smallest relative timestamp ε . Hence,we can use the encoding s d ∈ S B where an event s d ( t ) = true encodes a startinclusive and s d ( t ) = false an end exclusive. This encoding captures the mostcommon cases and simplifies the implementation of union and intersection on ∆ enormously since they can now be realized as slift ( ∨ ) and slift ( ∧ ) , resp. The otherpossible switches at timestamp t can be represented as follows: s d ( t + ε ) = true encodes an exclusive start, s d ( t + ε ) = false encodes an inclusive end, s d ( t ) = true and s d ( t + ε ) = false encodes a point-wise event in a gap, and s d ( t ) = false and s d ( t + ε ) = true encodes a point-wise gap. Using this encoding the abstractTeSSLa operators do not need to handle these additional cases explicitly.Furthermore, assuming the smallest relative timestamp ε , we can avoid theneed to perform the unrolling defined in Defs. 2 by delaying the second part ofthe computation to the next possible timestamp t + ε .As a final efficiency improvement we simplified last before the first event onthe stream of values, which are not relevant in practice. The abstract operatorand hence abstract specifications are of course still a sound abstraction of theirconcrete counterparts, but due to over-abstractions no longer a perfect one duringthis initial event-less phase of the stream of values.The implementation in form of a macro library for the existing TeSSLa engineis available together with all the examples and scripts used in the followingempirical evaluation and can be experimented with in a web IDE .In the following empirical evaluation we measure the accuracy of the abstrac-tions presented in this paper. An abstract event stream represents input datawith some sequences of data loss, where we do not know if any events mighthave been occurred or what their values have been. Applying an abstract TeSSLa http://tessla-a.isp.uni-luebeck.de/ http://tessla-a.isp.uni-luebeck.de/ untime Verification For Timed Event Streams With Partial Information 15 specification to such an input stream takes these gaps into account and providesoutput streams that in turn contain sequences of gaps and sequences containingconcrete events. To evaluate the accuracy of this procedure we compare theoutput of an abstract TeSSLa specification with the best possible output. P D P D S D S D S D II ϕ γ γϕ ιι Let r ∈ P D be an abstract event stream. We ob-tain the set R of all possible input streams containingall possible variants that might have happened dur-ing gaps in r by applying the concretization function γ on the abstract input stream. Now we can applythe concrete TeSSLa specification ϕ to all streamsin R and get the set S of concrete output streams.On the other hand we apply the abstract TeSSLa specification ϕ directly to r and get the abstract output stream s . Now S is the set of all possible outputstreams and γ ( s ) is the set of output streams defined by the abstract TeSSLaspecification. The diagram on the right depicts this comparison process. a b c To compare γ ( s ) and S in a quantitative waywe define the ignorance measure ι : 2 S D → I =[0 , scoring the ambiguity of such a set of streams,i.e., how similar the different streams in the set are.Events in non-synchronized streams might not havecorresponding events at the same timestamp on theother streams. Hence we refer to the signal semantics of event streams wherethe events represent the changes of a piece-wise constant signal. As depicted onthe right with three event streams over the finite data domain { , , } , we scoretimestamps based on how many event streams have the same value with respectto the signal semantics at that timestamp. These scores are then integratedand normalized throughout the length of the streams. See Appendix E for thetechnical details. Using this ignorance measure we can now compute the optimalignorance i := ι ( S ) ∈ I and the ignorance k := ι ( γ ( s )) ∈ I of the streamsproduced by the abstract TeSSLa specification.For the evaluation we took several example specifications and correspondinginput traces representing different use-cases of TeSSLa and compared the optimalignorance with the ignorance of abstract TeSSLa. Note that computing theoptimal ignorance requires to derive all possible variants of events that mighthave happened during gaps, which are in general infinitely many and in thespecial case of only point-wise gaps still exponentially many. Hence this can onlybe done on rather short traces with only a few point-wise gaps. As a measure forthe overhead imposed by using the abstraction compared to the concrete TeSSLaspecification we use the computation depth, i.e., the depth of the dependencygraph of the computation nodes of the specifications. While runtimes are highlydepending on implementation details of the used TeSSLa engines, the computationdepth is a good indicator for the computational overhead in terms of how manyconcrete TeSSLa operators are needed to realize the abstract TeSSLa specification.See Fig. 3 for a graphical representation and Appendix F for numerical results. r e s e t - c o u n t overhead010203040 00.10.20.30.4ignorance r e s e t - s u m overhead010203040 00.10.20.30.4ignorance fi l t e r - e x a m p l e overhead010203040 00.10.20.30.4ignorance v a r i a b l e - p e r i o d overhead010203040 00.10.20.30.4ignorance b u r s t s overhead010203040 00.10.20.30.4ignorance q u e u e overhead010203040 00.10.20.30.4ignorance fi n i t e - q u e u e overhead010203040 00.10.20.30.4ignorance s e l f - u p d a t i n g - q u e u e overhead010203040 00.10.20.30.4ignorance computationdepth overheadoptimalignoranceignorance ofabstract TeSSLa Fig. 3. Empirical results. The first three examples represent the class of common, simple TeSSLaspecifications without complex interdependencies and no generation of additionalevents with delay : Reset-count counts between reset events; reset-sum sumsup events between reset events; and filter-example filters events occurring in acertain timing-pattern. For these common specifications the overhead is small andthe abstraction is perfectly accurate. The burst example checks if events appearaccording to a complex pattern. In the abstraction we loose accuracy because thestarting point of a burst is not accessible by last after a gap. A similar problemoccurs in the queue example where we use a complex data domain to developa queue along an event stream. If last produces (cid:62) after a gap all informationabout the queue before the gap is lost. For variable-period the abstraction isnot perfectly accurate, because the delay is used to generate events periodicallydepending on an external input. This gets even worse for the self-updating queuewhere complex computations are performed depending on events generated by a delay . Surprisingly, the finite-queue is again perfectly accurate, because the sizeof the queue is limited in a way that eliminates the inaccuracy of the abstractionin this particular example. By replacing the basic operators of TeSSLa with abstract counterparts, weobtained a framework where properties and analyses can be specified with respectto complete traces and automatically evaluated for partially known traces. Wehave shown that these abstract operators can be encoded in TeSSLa, allowingexisting evaluation engines to be reused. This is particularly useful as TeSSLacomprises a very small core language suitable for implementation in soft- as well ashardware. Using the example of sliding windows, we demonstrated how complexdata structures like queues can be abstracted. Using finite abstractions, ourapproach even facilitates using complex data structures when only limited memoryis available. Evaluating the abstract specification typically only increases thecomputational cost by a constant factor. In particular, if a concrete specificationcan be monitored in linear time (in the size of the trace) its abstract counterpart untime Verification For Timed Event Streams With Partial Information 17 can be as well. Finally, we illustrated the practical feasibility of our approach byan empirical evaluation using the freely available TeSSLa engine. References 1. Alur, R., Fisman, D., Raghothaman, M.: Regular programming for quantitativeproperties of data streams. In: ESOP. pp. 15–40. Springer (2016)2. Asarin, E., Caspi, P., Maler, O.: Timed regular expressions. J. ACM (2), 172–206(2002)3. Basin, D.A., Klaedtke, F., Zalinescu, E.: Failure-aware runtime verification ofdistributed systems. In: FSTTCS. LIPIcs, vol. 45, pp. 590–603. Schloss Dagstuhl -Leibniz-Zentrum fuer Informatik (2015)4. Bauer, A., Leucker, M., Schallhart, C.: Runtime verification for LTL and TLTL.ACM T. Softw. Eng. Meth. (4), 14 (2011)5. Berry, G.: Proof, language, and interaction: essays in honour of Robin Milner, chap.The foundations of Esterel, pp. 425–454. MIT Press (2000)6. Bozelli, L., Sánchez, C.: Foundations of Boolean stream runtime verification. In:RV. LNCS, vol. 8734, pp. 64–79. Springer (2014)7. Convent, L., Hungerecker, S., Leucker, M., Scheffel, T., Schmitz, M., Thoma, D.:Tessla: Temporal stream-based specification language. In: SBMF. LNCS, Springer(2018)8. Cousot, P., Cousot, R.: Abstract interpretation: A unified lattice model for staticanalysis of programs by construction or approximation of fixpoints. In: POPL. pp.238–252. ACM (1977)9. Cousot, P., Cousot, R.: Abstract interpretation: A unified lattice model for staticanalysis of programs by construction or approximation of fixpoints. In: POPL. pp.238–252. ACM Press (1977)10. Cousot, P., Cousot, R.: Abstract interpretation frameworks. Journal of Logic andComputation (4), 511–547 (1992)11. D’Angelo, B., Sankaranarayanan, S., Sánchez, C., Robinson, W., Finkbeiner, B.,Sipma, H.B., Mehrotra, S., Manna, Z.: LOLA: runtime monitoring of synchronoussystems. In: TIME. pp. 166–174. IEEE (2005)12. Eisner, C., Fisman, D., Havlicek, J., Lustig, Y., McIsaac, A., Campenhout, D.V.:Reasoning with temporal logic on truncated paths. In: Proc. of CAV’03. pp. 27–39.LNCS 2725, Springer (2003)13. Eliot, C., Hudak, P.: Functional reactive animation. In: Proc. of ICFP’07. pp.163–173. ACM (1997)14. Faymonville, P., Finkbeiner, B., Schirmer, S., Torfah, H.: A stream-based speci-fication language for network monitoring. In: RV. pp. 152–168. LNCS, Springer(2016)15. Faymonville, P., Finkbeiner, B., Schwenger, M., Torfah, H.: Real-time Stream-basedMonitoring. arXiv:1711.03829 (Nov 2017), http://arxiv.org/abs/1711.03829 16. Gautier, T., Le Guernic, P., Besnard, L.: SIGNAL : A declarative language forsynchronous programming of real-time systems. In: Proc. of FPCA’87. pp. 257–277.LNCS 274, Springer (1987)17. Gorostiaga, F., Sánchez, C.: Striver: Stream runtime verification for real-timeevent-streams. In: RV. pp. 282–298. LNCS, Springer (2018)18. Halbwachs, N., Caspi, P., Pilaud, D., Plaice, J.: Lustre: a declarative language forprogramming synchronous systems. In: Proc. of POPL’87. pp. 178–188. ACM Press(1987)8 M. Leucker, C. Sánchez, T. Scheffel, M. Schmitz, and D. Thoma19. Havelund, K., Roşu, G.: Synthesizing monitors for safety properties. In: TACAS.pp. 342–356. LNCS, Springer (2002)20. Maler, O., Nickovic, D.: Monitoring temporal properties of continuous signals. In:FTRTFT. pp. 152–166 (2004)21. Manna, Z., Pnueli, A.: Temporal Verification of Reactive Systems: Safety. Springer,New York (1995)22. Sen, K., Roşu, G.: Generating optimal monitors for extended regular expressions.ENTCS (2), 226–245 (2003)23. Stoller, S.D., Bartocci, E., Seyster, J., Grosu, R., Havelund, K., Smolka, S.A.,Zadok, E.: Runtime verification with state estimation. In: RV. Lecture Notes inComputer Science, vol. 7186, pp. 193–207. Springer (2011)24. Wang, S., Ayoub, A., Sokolsky, O., Lee, I.: Runtime verification of traces underrecording uncertainty. In: RV. pp. 442–456. LNCS, Springer (2011)untime Verification For Timed Event Streams With Partial Information 19 A TeSSLa Evaluation Example Consider the equation y = merge (cid:0) lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) , (cid:1) where zero = const (0 , unit ) = const (0 , (cid:31) ∞ ) = 0 ∞ .The equation can be seen as function of y with fixed input stream x = 2 (cid:31) (cid:31) ∞ .To compute the least fixed-point of this function we start with the empty stream y = 0 . We than have last ( y, x ) = 0 ⊥ lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) = 0 ⊥ merge (cid:0) lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) , (cid:1) = 00 For the next iteration we start with y = 00 : last ( y, x ) = 2 lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) = 2 merge (cid:0) lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) , (cid:1) = 002 For the next iteration we start with y = 002 : last ( y, x ) = 20 lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) = 21 merge (cid:0) lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) , (cid:1) = 0021 For the next iteration we start with y = 0021 : last ( y, x ) = 204 lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) = 214 merge (cid:0) lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) , (cid:1) = 00214 For the next iteration we start with y = 00214 : last ( y, x ) = 2041 ∞ lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) = 2142 ∞ merge (cid:0) lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) , (cid:1) = 002142 ∞ For the next iteration we start with y = 002142 ∞ : last ( y, x ) = 2041 ∞ lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) = 2142 ∞ merge (cid:0) lift (cid:0) + 1 (cid:1)(cid:0) last ( y, x ) (cid:1) , (cid:1) = 002142 ∞ So we have reached a fixed-point.Note that y (cid:118) y (cid:118) y (cid:118) y (cid:118) y (cid:118) y regarding the prefix relation. B Delay In Section 2 TeSSLa was introduced with the operators nil , unit , time , lift and last . As discussed in [7] this set of operators is sufficient to express times-tamp conservative functions, i.e., functions which do not introduce additionaltimestamps. In order to express arbitrary functions which can add events atarbitrary timestamps, TeSSLa was introduced in [7] with the delay operator. The delay operator was removed from the main content of this paper due to spacelimitations and the amount of technical details needed to define the concreteand abstract operators. Since these two operators nevertheless fit very seamlessinto the abstraction framework presented in this paper they are included in theimplementation and the empirical evaluation and their definitions are given inthis appendix. B.1 Concrete Delay delay : S T \{ } × S D → S U , delay ( d, r ) := z takes a delay stream d and a resetstream r . It emits a unit event in the resulting stream after the delay passes.Every event on the reset stream resets any delay. New delays can only be settogether with a reset event or an emitted output event. Formally, z ( t ) = (cid:31) ∃ t (cid:48) B.2 Abstract Delay In Section 3 of this paper we presented the abstract operators lift and last for their concrete counterparts lift and last . In a similar way we now define theabstract delay operator as counterpart of the concrete delay . delay : P T ∞ \{ } × P D → P U , delay ( d, r ) := z needs additional logic to handle (cid:62) events on the delay stream and gaps on the input streams. Since the outputstream is of type P U no additional logic to produce (cid:62) events is needed. Weintroduce the following additional auxiliary functions: untime Verification For Timed Event Streams With Partial Information 21 (1) maybeEvent , which captures whether there may be an event that would causethe delay to hold: maybeSetable ( d, r, t ) def = z ( t ) ∈ { (cid:31) , (cid:96) } ∨ r ( t ) ∈ D ∪ { (cid:96) } maybeNoReset ( r, t, t (cid:48) ) def = ∀ t (cid:48)(cid:48) | t Consider the example for z = delay ( d, r ) shown onthe right. If the delay produces an event after or duringa gap on the reset stream, a potential reset event mightcancel the delay earlier, creating point-wise gaps. The concrete delay has only oneactive delay at any given point in time. However, to implement the abstract delayinfinite memory may be needed to keep track of all potential delays, because it may not be known whether a delay has been canceled or started earlier by areset event. By keeping track only of the minimal and maximal potential delayswe get a finite-memory implementation. However, this implementation may bean imperfect abstraction of the delay operator which produces larger gaps. Thosegaps marked in red on the right are replaced with one large gap. Definition 4 (Finite Memory Abstract Delay). We define delay fin ( d, r ) = z to be the same as delay ( d, r ) except that z ( t ) := (cid:96) (instead of ⊥ ) if ∃ t (cid:48) In Section 3.1 we discussed that using the abstract TeSSLa operators mightintroduce additional cyclic dependencies between the stream variables. In orderto overcome this issue we presented translation in Definition 2 which reintroducesguards in the cyclic dependency by unrolling the value and gap calculationsequentially. The same approach can be applied to unroll the abstract delay operator: Definition 5 (Unrolled Abstract Delay). We define two variants of theabstract delay, delay ⊥ and delay (cid:96) as follows: Let z = delay ( d, r ) , then delay ⊥ ( d, r ) := z ⊥ and delay (cid:96) ( d, r, p ) := z (cid:96) . z ⊥ ( t ) = (cid:40) z ( t ) if z ( t ) (cid:54) = (cid:96) ⊥ otherwise z (cid:96) ( t ) = p ( t ) if t ∈ ticks ( p ) (cid:96) if t / ∈ ticks ( p ) ∧ z ( t ) = (cid:96) ⊥ otherwise Since the trigger input of a delay operator cannot be recursive in a well-formedspecification, a recursive equation using one last has the form x = delay ( d, r ) and d = f ( x, c ) , where c is a vector of streams not involved in the recursion and f does not introduce further lasts or delays. Now, this equation system can berewritten in the following equivalent form: x (cid:48) = delay ⊥ ( d, r ) d (cid:48) = f ( x (cid:48) , c ) x = delay (cid:96) ( d (cid:48) , r, x (cid:48) ) d = f ( x, c ) By doing this, the calculation of the value and the domain of the definition issplit in two parts preserving the semantics. The same pattern can be repeated ifmultiple recursive abstract lasts or delays are used.There is a remaining case to be considered, when f contains other abstractlasts or delays which lead to mutual recursive specifications. In this case, theseother abstract lasts and delays have to use x , d or r as their input instead ofthe primed counterparts because the computation requires the complete lastor delay stream, respectively. Furthermore, there may be more unrolling stepsneeded than just one to unroll the whole specification but every unrolling stepstill follows the previously described pattern for last and delay. untime Verification For Timed Event Streams With Partial Information 23 C Missing Proofs Theorem 2. Every abstract TeSSLa operator is a perfect abstraction of itsconcrete counterpart.Proof. We need to show that for all partial streams s , . . . , s n with s i ∈ P D i i and every TeSSLa operator f and its abstract counterpart f it holds that f ( s , . . . , s n ) = α ( f ( γ ( s ) , . . . , γ ( s n ))) . We use the Galois Connection ( α, γ ) aspresented in Definition 1. For nil and unit this holds trivially because they areequal to their abstract counterparts. So it remains to show this for time , lift , last and delay . Note that because the Galois Connection maps from the abstractlattice to the powerset of the (concrete) lattice, we assume all operators in thefollowing are naturally extended to sets. time ( s ) = α ( time ( γ ( s ))) holds because time changes nothing regardingthe abstract elements: gaps are copied and the values of the events do not matterbecause they are replaced with the timestamps. lift ( g )( s , . . . , s n ) = α ( lift ( g )( γ ( s ) , . . . , γ ( s n ))) holds if g is a perfectabstraction of g , because the values and gaps are only passed to g in the abstractcase which calculates the values and gaps. last ( s , s ) = α ( last ( γ ( s ) , γ ( s ))) holds because – the only special case regarding the data abstraction is when a trigger occursand there is a gap on the stream of values after the last event. Then (cid:62) isthe output which is fine because in the concretization the gap is replacedwith arbitrary values. Other then that, the data abstraction is not importantbecause in last the data values are only copied to the new events on theoutput stream. – the only special case regarding the gaps is a trigger event before any event onthe stream of values but after a gap on said stream, which results in a gap.This is fine because the gap on the stream of values is later replaced with anarbitrary or no value which is then the output when the trigger event occurs.Thus using α later on this set of streams, we get a gap where the triggerevent was. After the first event on the stream of values, the gaps from thestream of trigger events are just copied. delay ( s , s ) = α ( delay ( γ ( s ) , γ ( s ))) holds because data abstraction wise,the output can always only be unit. If we look at the gaps, there are only a fewcases where delay produces gaps: Point-gaps are created when a delay timesout which (1) is set via a gap or (2) is set normally but a gap occurs on the resetstream before the timeout. Big gaps which last until the next event on the resetstream are created when a delay can be set but on the delay stream is (3) anevent with (cid:62) as value or (4) a gap. This perfectly fits to how α ( delay ( γ ( d ) , γ ( r ))) works because in(1) the concretization function creates sets of streams where the delay is setnot at all or with a value. The delay then outputs a set of streams wheresometimes there is an output event and sometimes not which results in apoint-gap again when the abstraction function is used. (2) the concretization function creates sets of streams where where a reset andno reset happens before the timeout. This has the same implications as for(1).(3) there is a delay started with (cid:62) as value. Then the concretization functioncreates a set of streams having all possible values as delay at that point.Hence, delay results in the set of streams that have an arbitrary amountof events between the point where the delay is set and the next reset event.Using the abstraction function again this results in a big gap.(4) it is the same case as (3) but the set of streams created by the concretizationfunction contains additionally those streams with no value as delay. This hasthe same implications as in (3). (cid:117)(cid:116) Lemma 1. lastTime is a perfect abstraction of lastTime .Proof. We need to show that for all partial streams v, r the following equationholds: lastTime ( v, r ) = α ( lastTime ( γ ( v ) , γ ( r ))) . We use the Galois Connec-tion ( α, γ ) as presented in Definition 1. Note that because the Galois Connectionmaps from the abstract lattice to the powerset of the (concrete) lattice, we assumeall operators in the following are naturally extended to sets.Because as shown in Theorem 2 last and time are perfects abstractionsof last and time , respectively, to show that lastTime is a perfect abstractionof lastTime , we just need to show that for the cases where the composition of last and time is not a perfect abstraction of lastTime , lastTime is nowperfect. More precisely, the only case which is not perfect in the compositionis the (cid:62) case, which means a gap occurred after the last event on the streamof values. Then lastTime returns an interval from the timestamp of the lastevent on v to the timestamp at the end of the gap. This is the same which wouldbe done when the concretization function first creates a set of streams containingan arbitrary amount of events in the gap which results in different timestampsas output. Either the latest from an event in the gap, if one exists or from thelast event on v . The abstraction function would then creat said interval. In everyother case, lastTime outputs the same value as the composition of the perfectoperators does. (cid:117)(cid:116) D Data Abstraction of a Queue for Sliding WindowAverage Computation D.1 Concrete Queue TeSSLa can handle complex data structures and corresponding functions. Anordered queue is a sequence of timestamps and values Q D = ( T × D ) ∗ such that ∀(cid:104) t , d , t , d , . . . , t n , d n (cid:105) ∈ Q D : t i − < t i where ≤ i ≤ n untime Verification For Timed Event Streams With Partial Information 25 We use the following functions on queues: enq : T × D × Q D → Q D enq ( t, d, q ) = q (cid:104) t, d (cid:105) remOlder k : T × Q D → Q D remOlder k ( t, (cid:104) t , d, t (cid:105) q ) = (cid:40) remOlder k ( t, (cid:104) t (cid:105) q ) if t ≤ t − k (cid:104) max ( t − k, t ) , d, t (cid:105) q else remOlder k ( t, (cid:104) t , d (cid:105) ) = (cid:40) (cid:104) t − k, d (cid:105) if t < t − k (cid:104) t , d (cid:105) else remOlder k ( t, (cid:104)(cid:105) ) = (cid:104)(cid:105) fold : (( T × D ) × D → D ) × Q D × D × T → D fold ( f, (cid:104) t , d, t (cid:105) q, acc, until ) = fold ( f, (cid:104) t (cid:105) q, f ( t , t , d, acc ) , until ) fold ( f, (cid:104) t , d (cid:105) , acc, until ) = f ( t , until, d, acc ) fold ( f, (cid:104)(cid:105) , acc, until ) = acc D.2 Abstract Queue The set of our abstract version of queues we use here is defined as Q D = T ∞ × Q D . The concretisation function is given as follows γ (( t, q )) = {(cid:104) ( t , d ) . . . ( t n , d n )( t n +1 , d n +1 ) . . . ( t m , d m ) (cid:105) |(cid:104) ( t , d ) . . . ( t n , d n ) (cid:105) ∈ Q D ∧ t n < t ∧(cid:104) ( t n +1 , d n +1 ) . . . ( t m , d m ) (cid:105) ⊆ q ∧ t n +1 ≥ t } and the abstraction functions is given as α ( Q ) = sup { (0 , q ) | q ∈ Q } .To improve the accuracy of the abstraction we insert the following limitingfunction as annotations on the possible ranges of variables into the concretespecification: limit : D × D × D → D limit ( a, b, d ) = a if d < ab if d > bd elseIn a similar fashion we add the a function removing newer elements from thequeue which serves as annotation that a queue will never contain elements witha future timestamp: remNewer : T × Q D → Q D remNewer ( t, q (cid:104) t , d (cid:105) ) = (cid:40) remNewer ( t, q ) if t > tq (cid:104) t , d (cid:105) else remNewer ( t, (cid:104)(cid:105) ) = (cid:104)(cid:105) Now we define abstract versions for the previously defined functions: remOlder k : T × Q I D → Q I D remOlder k ( t, ( u, q )) = (cid:40) ( u, remOlder k ( t , q )) if t − k < u (0 , remOlder k ( t , q )) else enq : T × I D × Q I D → Q I D enq ( t, d, ( u, q )) = ( u, enq ( t, d, q )) fold : (( T × I D ) × I D → I D ) × Q I D × I D → I D fold ( f, ( u, q ) , acc, until ) = (cid:40) fold ( f, q, acc, until ) if u = 0 fold ( f, q, (cid:62) , until ) else remNewer : T × Q I D → Q I D remNewer ( t, ( u, q )) = ( min ( u, t ) , remNewer ( t, q )) limit : D × D × I D → I D limit ( a, b, [ d, d (cid:48) ]) = [ limit ( a, b, d ) , limit ( a, b, d (cid:48) )] Note, that the abstract functions use intervals I D to represent ranges ofpossible values.In order to use the data domains we have to name the following elements: (cid:62) = ( ∞ , (cid:104)(cid:105) ) ∈ Q I D and (cid:62) = [ −∞ , ∞ ] ∈ I D . Now we can provide an abstraction of the TeSSLa specification for the inputstream load ∈ P R : stripped = slift ( rem )( time ( load ) , merge ( last ( queue , load ) , (cid:104)(cid:105) ))) rem ( t, q ) = remOlder ( t, remNewer ( t, q )) queue = lift ( enq )( time ( load ) , load , stripped ) avg = lift ( int )( queue , time ( load )) int ( q, u ) = fold ( f u , q, , u ) f u ( a, b, v, acc ) = limit (cid:18) , a − u + 55 , acc (cid:19) + v · ( b − a ) / untime Verification For Timed Event Streams With Partial Information 27 D.3 Finite Abstract Queue If we want to limit the queue to a certain number of events, we just need tochange the enqueue function as follows: enq c : T × I D × Q I D × N > → Q I D enq c ( t, d, ( u, q ) , n ) = (cid:40) enq ( t, d, ( u, q )) if | q | < n enq c ( t, d, ( t , (cid:104) t (cid:105) q (cid:48) ) , n ) otherwise,with q = (cid:104) t , d , t (cid:105) q (cid:48) For q ∈ Q I D we denote with | q | the number of data values in q . D.4 Concrete Self-Updating Queue We now update the specification to be self-updating, i.e., update automaticallyevery time an event leaves the moving window completely. For this, we can usethe operator delay to create additional events. We add the following equationsto the original specification and use updated instead of queue and load in thecomputation of avg : timeout = lift ( t, q (cid:55)→ dataTimeout ( q ) − t + 5)( time ( updated ) , updated ) merged = merge ( queue , last ( updated , delay ( timeout , load ))) updated = lift ( remOlder )( time ( merged ) , merged ) where dataTimeout : Q D → T is defined as follows: dataTimeout ( (cid:104)(cid:105) ) = ∞ dataTimeout ( (cid:104) t, d (cid:105) ) = ∞ dataTimeout ( (cid:104) t, d, t (cid:48) (cid:105) q ) = t (cid:48) D.5 Abstract Self-Updating Queue The abstract version of the specification for the self-updating queue can again bederived by replacing all TeSSLa operators with their abstract counterparts: timeout = lift ( t, q (cid:55)→ dataTimeout ( q ) − t + 5)( time ( updated ) , updated ) merged = merge ( queue , last ( updated , delay ( timeout , load ))) updated = lift ( remOlder )( time ( merged )) , merged ) where dataTimeout : Q I D → T is defined as follows: dataTimeout ( u, q ) = (cid:40) dataTimeout ( q ) if u = 0 (cid:62) elseThe rest of the specification is adjusted to the abstract setting as shown before.The unrolling of last and delay as defined in Definition 2 and Definition 5can be applied in order to realize this abstract TeSSLa specification in TeSSLa. E Technical Details of the Ignorance Measure The function v : S D × T → D provides the current value of a stream at a giventimestamp according to the signal semantics: v s ( t ) := s (max { t (cid:48) ∈ ticks ( s ) | t (cid:48) < t } ) For the data domain D we assume a measure space ( D , A , µ ) with the σ -algebra A ⊆ D on D and the normalized measure µ : A → [0 , on D , s.t. ∀ D ∈ A : 0 ≤ µ ( D ) ≤ and µ ( D ) = 1 .An ignorance representation is a piece-wise constant function ig : T → A s.t. ig can be represented as a finite number of intervals. The progress of a stream s ∈ S D is a timestamp t such that ∀ t (cid:48) >t s ( t (cid:48) ) = ? and ∀ t (cid:48)