Logical Signal Processing: a Fourier Analysis of Temporal Logic
LLogical Signal Processing: a Fourier Analysis of Temporal Logic
Niraj Basnet and Houssam Abbas
Oregon State University, Corvallis OR 97330, USA { basnetn,abbasho } @oregonstate.edu Abstract.
What is the frequency content of temporal logic formulas? That is, when we monitor a signal againsta formula, which frequency bands of the signal are relevant to the logic and should be preserved, and which canbe safely discarded? This question is relevant whenever signals are filtered or compressed before being monitored,which is almost always the case for analog signals. To answer this question, we focus on monitors that measure therobustness of a signal relative to a specification in Signal Temporal Logic. We prove that robustness monitors can bemodeled using Volterra series. We then study the Fourier transforms of these Volterra representations, and providea method to derive the Fourier transforms of entire formulas. We also make explicit the measurement process intemporal logic and re-define it on the basis of distributions to make it compatible with measurements in signalprocessing. Experiments illustrate these results. Beyond compression, this work enables the integration of temporallogic monitoring into common signal processing tool chains as just another signal processing operation, and enablesa common formalism to study both logical and non-logical operations in the frequency domain, which we refer toas Logical Signal Processing.
Keywords:
Robustness monitoring · Temporal logic · Volterra series · Fourier transform
Runtime monitors in Cyber-Physical Systems (CPS) process analog signals: that is, continuous-time, continuous-valued signals generated by the physics of the system, rather than digital signals generated by computations on valuesstored in memory. These analog signals are never pristine: to begin with, they are measured, and so incur somemeasurement distortion; they are noisy, and therefore are usually filtered to reduce noise; and if they get transmitted,they are compressed and de-compressed, which introduces further distortions. All of these operations are very commonin signal processing toolchains - indeed, the act of measurement is inevitable. And all of these operations a ff ect, a priori,the verdict of the runtime monitor. Yet we have little theory to systematically account for these e ff ects. For instance,Fourier analysis is a standard powerful tool in signal processing, which is used to synthesize optimal filters meetingcertain output requirements. If we want to synthesize a filter subject to monitorability requirements, how would wego about it? Conversely, how can we synthesize a monitor that accounts for the modifications introduced by a filterearlier in the processing chain? Today we have no way of answering these questions systematically, because we lackan account of the frequency content of temporal logic. That is, we lack a Fourier representation of the operation of atemporal logic monitor. Lacking such a uniform formalism, we remain in the awkward position of having to study theimpact of frequency-domain operations in the time domain, using ad hoc assumptions like the availability of Lipschitzconstants, and combining them somehow with the time-domain representation of monitors.This paper presents a way of analyzing the frequency content of temporal logic, or equivalently, of analyzing tem-poral logic monitors in the Fourier domain. Because monitors are nonlinear operators, we resort to Volterra series,which generalize the convolution representation of time-invariant linear operators to time-invariant nonlinear opera-tors, and have generalized Fourier representations. To apply the machinery of Volterra series, we work exclusivelywith robustness monitors, which output analog robustness signals.As soon as we start thinking in the Fourier domain, we also realize that the basic measurement model in temporallogic is broken from a physical, and therefore, cyber-physical, perspective: in almost all temporal logics, it is assumedthat one can make an instantaneous measurement. I.e., that it is possible to measure x ( t ) exactly at t , and this is used todetermine the truth value of an atomic proposition, e.g. ‘ x ( t ) ≥ (cid:48) . However, it is well-known that instantaneous mea-surements of analog signals are impossible! Any measurement device has a finite resolution, so at best we can measuresome aggregate of infinitely many signal values. For instance, in a camera the value recorded by a pixel equals the av-erage illumination incident on that pixel, not the illumination in a specific point in space. This matters to us because an a r X i v : . [ ee ss . S Y ] S e p ig. 1: Frequency response of the temporal logic formula Historically p . H ( ω ) captures the linear, or first-order,transfer function, while H ( ω , ω ), H ( ω , ω , ω ), etc, are higher-order generalized frequency responses that capturethe non-linear e ff ects. By studying these response functions we can determine which frequencies of the monitoredsignal a ff ect the monitor output. (Color in digital copy).Fig. 2: Monitoring-safe compression enabled by Logical Signal Processing. (Left) Signal x (red) is compressed byeliminating frequencies above the formula’s cut-o ff frequency (1.5Hz). Resulting signal is in blue. (Second panel)Robustness of original and compressed signals relative to a formula φ . Despite the marked di ff erence between themonitored signals, the robustness signals are almost identical, validating the fact that the compression was monitoring-safe. Right two panes show the order-1 and -2 frequency responses of the formula. (Color in digital copy).instantaneous measurement requires infinite bandwidth, thus rendering the entire frequency analysis trivial or useless.Instantaneous measurements also produce mathematical complications when working with sup norms and Lebesgueintegrals which ignore sets of measure zero. We therefore re-define atomic propositions (and the measurement model)on the basis of the theory of distributions, and demonstrate that the resulting robust semantics are still sound and stillyield robustness tubes that can be used in falsification.Fig. 1 shows an example of the results made possible by the methods of this paper, which we refer to as LogicalSignal processing . The figure shows the first four Generalized Frequency Response Functions (GFRFs) of the formula [1 , . p , read ‘Historically p ’. Here, p is a given atomic proposition and is the Historically operation, the past equiv-alent of Always / Globally. The way to interpret Fig. 1 is, roughly, as follows: let X be the Fourier transform of the inputmonitored signal, and Y be the Fourier of the output robustness signal computed by the monitor. Then H ( ω ) weighs thecontribution of X ( ω ) to Y ( ω ), H ( ω , ω ) weighs the contribution of the product X ( ω ) X ( ω ), H ( ω , ω , ω ) weighsthe contribution of X ( ω ) X ( ω ) X ( ω ), etc. (Product terms appear because the monitor is a non-linear operator, as willbe explained later). Using these GFRFs H n , we can, for example, calculate frequencies ω s.t. X ( ω ) contributes verylittle to the output Y ( ω ), that is, frequencies of the signal that are irrelevant to the monitor. This is shown in Fig. 2:using the methods of this paper, we obtained the frequency responses of formula [0 . , . p , read ‘Once p ’. From theresponses we determined (roughly) that the H n ’s are negligible above ω = π rad / s. Therefore, setting X ( ω ) to 0 abovethe cut-o ff should change the monitoring output very little. This analysis is confirmed by the experiment: Fig. 2 showsthat there’s a marked di ff erence between original and compressed signals, and a priori there is no reason to assumethat their robustnesses would be similar. And yet, the calculated robustness signals are almost indistinguishable, con-firming that compression was done in a monitoring-safe manner. Thus, a codec can suppress all frequencies above thecut-o ff before transmitting the signal, thus reducing the amount of transmitted bits in a monitoring-safe manner.2n this paper, our contributions are: – a definition of atomic propositions on the basis of distributions to ensure measurements have finite bandwidth,such that the resulting logic has sound robust semantics. The new logic di ff ers from standard Signal TemporalLogic only in the atomic propositions. – a modeling of temporal logic monitors using Volterra series – the first frequency analysis of temporal logic formulas and their robustness monitors.The proposed analysis using Volterra series can be used as a signal processing tool for frequency domain analysisof temporal logic monitors but it is not supposed to replace the monitors themselves. It can be used for example todesign filters which respect the monitorability requirements. Related Work.
There are few works dealing with the dual aspects of time and frequency in temporal logic. In [8]the authors introduce a variant of Signal Temporal Logic (STL), called Time-Frequency Logic (TFL), to explicitlyspecify properties of the frequency spectrum of the signal - e.g. one formula might read “ | X ( ω, t ) | > X ( ω, t ) is obtained by a windowed Fourier transform and ω is given). What we do in thispaper is orthogonal to [8]: given a standard STL formula, we want to analyze which frequencies of signal x contributeto the monitoring verdict. We do not require the formula to tell us that explicitly, as in TFL. In fact, our techniquesare applicable to TFL itself: a TFL formula like the one above has a ‘hidden’ frequency aspect, namely the temporalrestriction “in the next three time units”. Our methods allow an automatic extraction of that.The works [15] and [11] provide a general algebraic framework for the semantics of temporal logic based on semi-rings, opening the way to producing new semantics automatically by concretizing the semi-ring operations. In thespecial case of Metric Temporal logic, [15] shows a formal similarity between convolution and the classical semanticsof some logical operations. However, no frequency analysis is made (and indeed that is not the objective of [15]).In [17], the logic SCL is introduced which replaces the usual temporal operators (like Eventually) with a cross-correlation between the signal and a kernel, which resembles our use of it for atomic propositions (we preserve alltemporal operators). The objective of [17] is to measure the fraction of time a property is satisfied, and no frequencyanalysis is made. We also mention [9] which derives continuous-time verdicts from discrete-time reasoning. Themodified measurement in this paper should allow us to connect that work to the sampling process and thereforeoptimize it for monitoring (but we do not carry out that connection here).Finally, the works [1,2] consider the impact of representation basis and out-of-order reconstruction on monitor-ing verdicts, thus also contributing to the study of how signal processing a ff ects runtime verification. The work [2]implicitly replaces instantaneous measurements in the temporal operators with a scrambled, or locally orderless, mea-surement, but maintains instantaneous measurements in atoms. No frequency analysis is made. Terminology.
Let R = ( −∞ , ∞ ) , R + = [0 , ∞ ) , R = R ∪ {±∞} , N = { , , , . . . } . We write C for the set of complexnumbers and i = √−
1. Given an interval I ⊂ R and t ∈ R , t − I : = { t (cid:48) | ∃ s ∈ I . t (cid:48) = t − s } . E.g. t − [ a , b ] = [ t − b , t − a ].For a vector x ∈ R d , | x | is its 2-norm and if x ∈ C , | x | is its modulus (i.e. | a + ib | = √ a + b ) . For a function f , (cid:107) f (cid:107) is its essential supremum, defined as (cid:107) f (cid:107) : = inf { a | | f ( t ) | ≤ a on all sets of non-zero measure in the domain of f } .Given two sets A and B , A B is the set of all functions from B to A . The space of bounded continuous functions f : R n → R m is denoted by C ( R n , R m ). An n -dimensional function is one whose domain is R n . The Fourier transformof function h will be written F { h } . We adopt the convention of using capitalized letters for the transform, e.g. F { h } = H , F { g } = G , etc. The magnitude | H | of a Fourier transform will be called the spectrum .Dirac’s delta distribution is δ . It is common to abuse notation and treat δ as a function; in that case recall that forany continuous function f and τ ∈ R , (cid:82) f ( t ) δ ( t − τ ) dt = f ( τ ), δ (0) = ∞ and δ ( t ) = ∀ t (cid:44) operator to mean a function N that maps signals to signals, e.g. N : C ( R , R ) →C ( R , R ). The composition of operators N and M is N ◦ M . A logical operation , on the other hand, refers to anoperation of a temporal logic, like negation, disjunction, Since and Once. Logical True is (cid:62) , False is ⊥ .3 .1 Signal Temporal Logic (STL) Signal Temporal Logic (STL) [13,7] is a logic that allows the succinct and unambiguous specification of a wide varietyof desired system behaviors over time, such as “The vehicle reaches its destination within 10 time units while alwaysavoiding obstacles” and “While the vehicle is in Zone 1, it must obey that zone’s velocity constraints”.We use a variant of STL which uses past temporal operators instead of future ones. For simplicity in this paper wework with scalar-valued signals. Formally, let
X ⊆ R be the state-space and let E ⊆ R + be an open interval (whichcould be all of R + ). A signal x is a continuous bounded function. Let { µ , . . . , µ L } be a set of real-valued functions ofthe state: µ k : X → R . Let AP = { p , . . . , p L } be a set of atomic propositions. Definition 1 (Past Bounded STL).
The syntax of the logic STL pastbdd is given by φ : = (cid:62) | p | ¬ φ | φ ∨ φ | I φ | I φ | φ S I φ where p ∈ AP and I ⊂ R is a compact interval. The semantics are given relative to signals as follows. ( x , t ) | = (cid:62) i ff (cid:62)∀ p k ∈ AP , ( x , t ) | = p k i ff µ k ( x ( t )) ≥ x , t ) | = ¬ φ i ff ( x , t ) (cid:54)| = φ ( x , t ) | = φ ∨ φ i ff ( x , t ) | = φ or ( x , t ) | = φ ( x , t ) | = I φ i ff ∃ t (cid:48) ∈ t − I . ( x , t (cid:48) ) | = φ ( x , t ) | = I φ i ff ∀ t (cid:48) ∈ t − I ( x , t (cid:48) ) | = φ ( x , t ) | = φ S I φ i ff ∃ t (cid:48) ∈ t − I . ( x , t (cid:48) ) | = φ and ∀ t (cid:48)(cid:48) ∈ ( t (cid:48) , t ] , ( x , t (cid:48)(cid:48) ) | = φ It is possible to define the Once ( ) and Historically ( ) operations in terms of Since S , but we make them baseoperations because we will work extensively with them.The requirement of bounded intervals and past time is needed to enable the Volterra approximation of temporallogic operations, as will be made explicit in Section 4. The robust semantics of an STL pastbdd formula give a quantitative measure of how well a formula is satisfied by a signal x . Usually, robustness is thought of as a functional that maps a signal x and time instant t to a real value ρ φ ( x , t ) whichis the robustness of x relative to φ at t . In this paper, instead, we will think of it as an operator mapping signals x torobustness signals ρ φ ( x ). This forms the starting point of our Volterra approximation and frequency domain modeling. Definition 2 (Robustness[9,7]).
Let φ be a STL pastbdd formula. The robustness ρ φ of φ is an operator which maps signalsx : E → X to signals ρ φ ( x ) : R → R , and is defined as follows: for any t ∈ R , ρ (cid:62) ( x )( t ) = + ∞ ρ p k ( x )( t ) = µ k ( x ( t )) ∀ p k ∈ AP ρ ¬ φ ( x )( t ) = − ρ φ ( x )( t ) ρ φ ∨ φ ( x )( t ) = max { ρ φ ( x )( t ) , ρ φ ( x )( t ) } ρ I φ ( x )( t ) = max t (cid:48) ∈ t − I ρ φ ( x )( t (cid:48) ) ρ I φ ( x )( t ) = min t (cid:48) ∈ t − I ρ φ ( x )( t (cid:48) ) ρ φ S I φ ( x )( t ) = max t (cid:48) ∈ ( t − I ) (cid:40) min { ρ φ ( x )( t (cid:48) ) , min t (cid:48)(cid:48) ∈ ( t (cid:48) , t ] ρ φ ( x )( t (cid:48)(cid:48) ) } (cid:41) The following soundness property allows us to use robustness to monitor signals.
Theorem 1 (Soundness [9]).
For any signal x and STL pastbdd formula φ , if ρ φ ( x )( t ) < then x violates φ at time t, andif ρ φ ( x )( t ) > then x satisfies φ at t. Moreover, for every signal y s.t. (cid:107) x − y (cid:107) < ρ φ ( x )( t ) , ( y , t ) | = φ if ( x , t ) | = φ and ( y , t ) (cid:54)| = φ if ( x , t ) (cid:54)| = φ . .3 Fourier Analysis We give a brief overview of the Fourier transform and LTI systems; readers familiar with this material can skip thissection without loss of continuity. Fourier analysis allows us to decompose a signal into its constituent frequencies,e.g. by decomposing it into a weighted sum of sinusoids or complex exponentials. We can then also compute howmuch energy is placed in a given frequency band.The Fourier transform X : R → C of an input signal x is defined as: X ( ω ) = (cid:90) ∞−∞ x ( t ) e − i ω t dt (1)The real variable ω is the angular frequency measured in rad / sec and relates to the usual frequency f in Hz as ω = π f .The magnitude | X | is called the amplitude spectrum of x ; the energy in frequency band [ ω , ω ] is (cid:82) ω ω | X ( ω ) | d ω . TheFourier transform is invertible and x can be obtained using the inverse Fourier transform: x ( t ) = π (cid:90) ∞−∞ X ( ω ) e i ω t d ω (2)Thus we can see that x at t is a weighted sum of complex exponentials, in which e i ω t is weighted by X ( ω ). For a quickimplementation on computers, the discrete version of Fourier transform is evaluated using the Fast Fourier Transform(FFT) algorithm.The Fourier transform is a powerful tool for studying linear time-invariant (LTI) systems. An LTI system is char-acterized by its impulse response h : R → R . For an input signal x , the system’s output signal y is given by theconvolution (represented by operator ∗ ) of x with the impulse response as follows y ( t ) = ( x ∗ h )( t ) = (cid:90) ∞−∞ h ( τ ) x ( t − τ ) d τ (3)The Fourier Transform reduces this convolution to simply the product of the Fourier transforms: y ( t ) = ( x ∗ h )( t ) ↔ Y ( ω ) = X ( ω ) H ( ω ) (4)Thus if we choose an LTI system such that H ( ω ) = ω c , we would get y ( t ) without highfrequency noise. Hence, the Fourier domain can be done for designing filters that pass or block specific frequencycomponents of the input signal.But Eq. (4) holds only for LTI systems, because complex exponentials are the eigenfunctions for linear, timeinvariant systems. Since robustness operators used in monitoring temporal logic monitors are nonlinear, they requireseparate treatment. A nonlinear extension is necessary which is provided by Volterra series. A finite Volterra series operator N is one of the form( N x )( t ) : = h + N (cid:88) n = (cid:90) . . . (cid:90) h n ( τ , . . . , τ n ) · x ( t − τ ) . . . x ( t − τ n ) d τ . . . d τ n (5)where x is the input signal. A Volterra series generalizes the convolution description of linear time-invariant (LTI)systems to time-invariant (TI) but nonlinear systems. We will drop the parentheses to simply write N u for the outputsignal of N . The n -dimensional functions h n : R n → R , n ≥
1, are known as
Volterra kernels , and their Fouriertransforms H n : C n → C are know as Generalized Frequency Response Functions (GFRFs): H n ( Ω ) : = (cid:90) τ ∈ R n exp( − i Ω T τ ) h n ( τ ) d τ
5e will use Volterra series to approximate the robustness nonlinear operator because there exists a well-developedtheory for studying their output spectra using the GFRFs. For instance, the Fourier of the output signal y = N x is Y ( ω ) = (cid:80) n Y n ( ω ) where [12] Y n ( ω ) = √ n (2 π ) n − (cid:90) ω + ... + ω n = ω H n ( ω , . . . , ω n ) X ( ω ) . . . X ( ω n ) d ω . . . d ω n (6)Eq. 6 gives one way to determine which frequencies of signal x a ff ect the output robustness signal. If a frequency ω ∗ is s.t. for almost all ω , ω , ω , . . . , all the following spectra are below some user-set threshold H ( ω ∗ ) , H ( ω ∗ , ω ) , H ( ω , ω ∗ ) , H ( ω ∗ , ω , ω ) , H ( ω , ω ∗ , ω ) , H ( ω , ω , ω ∗ ) , etc (7)then X ( ω ∗ ) contributes very little to the formation of the monitoring output, and can be safely discarded.Volterra series can approximate time-invariant (TI) operators with fading memory. Intuitively, an operator hasfading memory if two input signals that are close in the near past, but not necessarily the distant past, yield presentoutputs that are close. Definition 3 (Fading memory).
Operator N has fading memory on a subset K of C ( R , R ) if there is an increasingfunction w : ( −∞ , → (0 , , lim t →−∞ w ( t ) = s.t. for each u ∈ K and (cid:15) > there is a constant δ > s.t. ∀ v ∈ K , sup t ≤ | u ( t ) − v ( t ) | w ( t ) < δ → |N u (0) − N v (0) | < (cid:15) Such a w is called a weight function for N . Theorem 2 (Volterra approximation [5]).
Let K M , M : = { u ∈ C ( R , R ) | (cid:107) u (cid:107) ≤ M , (cid:107) u ( · − τ ) − u (cid:107) ≤ M τ ∀ τ ≥ } forsome constants M , M , and let (cid:15) > . Let R be any TI operator with fading memory on K M , M . Then there is a finiteVolterra series operator N such that for all u ∈ K M , M , (cid:107)R u − N u (cid:107) < (cid:15) In practice, how one obtains the Volterra approximation of a given non-linear operator depends on the operator. Theprobing method [12] can be used for systems given by ODEs or auto-regressive equations. However, it is not applicablein our case because it requires feeding complex exponentials to the operator, whereas our robustness operators can onlybe applied to real-valued signals. If the operator’s behavior is given by a set of input-output pairs of signals, one canfirst fit a function to the data, then obtain the Volterra representation of that function - see [12,5]. D ( E ) and D (cid:48) A measurement device is modeled in classical Physics using the theory of distributions. Giving even a cursory overviewof this theory is beyond the scope of this paper. We will provide the necessary mathematical definitions and refer thereader to [10] for a more comprehensive CS and Engineering-oriented introduction to this topic.Let D ( E ) be the space of infinitely di ff erentiable functions with compact support in E . A measurement kernel inthis paper is a non-zero function f : R → R with L norm at most 1, i.e., (cid:107) f (cid:107) : = (cid:82) | f ( t ) | dt ≤
1. Let D (cid:48) be the spaceof all such functions.Note that f ∈ D (cid:48) i ff − f ∈ D (cid:48) and that the shifted kernel f ( · − t ) is in D ( E ) for every t whenever f ∈ D ( E ). The measurement signal y is then obtained by taking the following inner product: y ( t ) = (cid:104) f ( · − t ) , x (cid:105) : = (cid:90) ∞−∞ f ( τ − t ) x ( τ ) d τ ∀ t (8)One can think of the measurement device as taking an f -weighted average of the values of x centered on t . Informally,the width of f dictates the resolution of the measurement: the narrower f , the higher the resolution. Di ff erent mea-surement devices use di ff erent filters f ∈ D (cid:48) . Note that Dirac’s δ is not in D (cid:48) , but can be approximated arbitrarily wellwith narrow integrable functions. 6 Bounded-Bandwidth STL
The semantics of an atomic proposition, given in Def. 1, requires the ability to measure the instantaneous value x ( t ).However, it is physically impossible to do an instantaneous measurement [10]: in (8), y ( t ) = x ( t ) i ff f = δ , Dirac’sdelta. But δ is not realizable because it has infinite energy: (cid:82) δ ( t ) dt = δ (0) = ∞ . In this paper, we must pay closerattention to how measurements are actually made in the physical world for two reasons: – we are interested in analyzing runtime monitors when they are a part of a signal processing chain. If something isnot physically possible, e.g., because it requires infinite energy, it makes little sense to model how other compo-nents in the chain will process its output. – We are interested in analyzing the input-output relation of a temporal logic monitor in the frequency domain (itstransfer function, as it were). Even if we kept using instantaneous measurements in the theory for conveniencesake, we’d end up with the trivial result that all robustness monitors have infinite essential bandwidth [18] since
F { δ } ( ω ) = ∀ ω . I.e., all frequency bands are relevant - clearly a useless analysis.This motivates our introduction of a new logic, Bounded-Bandwidth STL (BB-STL, pronounced ‘baby Steel’), thatdoes away with punctual measurements while preserving the logical relations and the soundness of robust semantics.BB-STL formulas are interpreted over signals in D ( E ), defined in Section 2.5. Let AP be a set of atomic proposi-tions s.t. there exists a bijection between D (cid:48) and AP . We write p f for the atom corresponding to filter f . Definition 4 (Bounded-Bandwidth STL).
The syntax of BB-STL is identical to that of STL pastbdd : φ : = (cid:62) | p f | ¬ φ | φ ∧ φ | I φ | I φ | φ S I φ where p f ∈ AP and I ⊂ R is a compact interval. Its boolean semantics are identical to those of STL pastbdd except for theatomic proposition case given here: ( x , t ) | = p f i ff (cid:104) f ( · − t ) , x (cid:105) ≥ Its robust semantics are identical to those of STL pastbdd except for the base case below. ρ p f ( x )( t ) = (cid:104) f ( · − t ) , x (cid:105) The robustness of any signal relative to any atomic proposition is finite: letting S x be the compact support of signal x ,it holds that (cid:104) f , x (cid:105) ≤ (cid:82) S x | f | dt · (cid:82) S x | x | dt , which is finite since f is absolutely integrable and x is continuous and thereforebounded on any compact set. Thus ρ φ ( x ) ≤ ρ (cid:62) ( x ) for any φ , as required for an intuitive interpretation of robustness.The following theorem establishes that BB-STL can be monitored via its robust semantics. Theorem 3 (Soundness of robust semantics).
For every signal x ∈ D ( E ) and BB-STL formula φ , if ρ φ ( x )( t ) < thenx violates φ at time t, and if ρ φ ( x )( t ) > then x satisfies φ at t. Moreover, for every signal y s.t. d ( x , y ) < ρ φ ( x )( t ) , ( y , t ) | = φ if ( x , t ) | = φ and ( y , t ) (cid:54)| = φ if ( x , t ) (cid:54)| = φ . Before proving the theorem, we make several remarks about the definition of BB-STL and the various restrictionswe placed on the signal and kernel spaces. The measurement process x → (cid:104) x , f ( · − t ) (cid:105) can be written as a convolution( x ∗ f − )( t ), where f − ( t ) = f ( − t ). So F { f − } is the transfer function of the measurement process. By selecting anappropriate set of filters, we get rid of the problem of infinite bandwidth measurements. In particular, we make surethat δ is not in D (cid:48) .STL and STL pastbdd use arbitrary functions µ k in their atoms, which allows arbitrary processing of the signal. E.g. ifsome x is 1-D, and we want to express the requirement x − e x ≥ ∧ x ≥
1, we can do that by using µ ( x ) = x − e x and µ ( x ) = x −
1. BB-STL does not have that expressiveness, but we are nevertheless able to compute arbitrarylinear functionals of x and compare them. E.g. the requirement (cid:104) x , f (cid:105) ≥ (cid:104) x , g (cid:105) is captured as (cid:104) x , f − g (cid:105) ≥
0. So thedi ff erence between STL and BB-STL, at the level of atomic propositions, is in the ability to generate auxiliary signalsin a non-linear vs linear fashion.The Volterra approximation of an operator requires the latter to be causal and have fading memory (causality isimplied by the conditions of Thm. 2 [5]). Causality requires working with past time operations, and fading memoryrequires working with bounded temporal operators. This is why we derived BB-STL from STL pastbdd rather than STL.To prove Thm. 3, we will first need to define a distance function d : D ( E ) × D ( E ) → R : d ( x , y ) : = sup {(cid:104) x − y , f (cid:105) | f ∈ D (cid:48) } (9)7 emma 1. Function d is a metric on D ( E ) .Proof. d is non-negative: indeed for all x ∈ D ( E ) and g ∈ D (cid:48) , sup f (cid:104) x , f (cid:105) ≥ max( (cid:104) x , g (cid:105) , (cid:104) x , − g (cid:105) ) = |(cid:104) x , g (cid:105)| . Since x − y ∈ D ( E ) whenever x , y ∈ D ( E ), the conclusion follows. d is symmetric: d ( x , y ) = sup f (cid:104) x − y , f (cid:105) = sup f (cid:104) y − x , − f (cid:105) = sup f ∈−D (cid:48) (cid:104) y − x , f (cid:105) = sup f ∈D (cid:48) (cid:104) y − x , f (cid:105) = d ( y , x ). d satisfies the triangle inequality: for any x , y , z ∈ D ( E ), d ( x , y ) = sup {(cid:104) x + z − z − y , f (cid:105) | f ∈ D (cid:48) } ≤ sup D (cid:48) {(cid:104) x − z , f (cid:105)} + sup D (cid:48) {(cid:104) z − y , f (cid:105)} = d ( x , z ) + d ( z , y ) d separates points: that is, if d ( x , y ) = x = y . We will argue by contradiction. Define function ε by ε ( t ) = x ( t ) − y ( t ). Assume x (cid:44) y so there exists a t (cid:48) ∈ E s.t. ε ( t (cid:48) ) (cid:44) ε ( t (cid:48) ) > − ε ∈ D ( E )) and that t (cid:48) =
0. Since ε is continuous, there exists a neighborhood I of 0 over which ε ( t ) >
0. Sopick g ∈ D (cid:48) s.t. g ( t ) > I and 0 elsewhere. It then holds that (cid:104) g , ε (cid:105) >
0, contradicting d ( x , y ) =
0. Therefore ε = x = y .Metric d takes the distance between signals to be the largest measurement that can be made of their di ff erence;this is consistent with the view that we have no access to a signal without a measurement device . The only way todi ff erentiate between signals is to measure a di ff erence between them. (Eq. (2.6) in [3] gives a more widely applicablemetric but d above is much more interpretable). We can now proceed with the proof of Thm. 3. Proof.
Let L t ( φ ) be the set of all x s.t. ( x , t ) | = φ and for a subset S ⊂ D ( E ) let dist ( x , S ) = inf y ∈ S d ( x , y ). By conventionset dist ( x , ∅ ) = ∞ . Following [9], and given that d is a metric, it su ffi ces to show that the following inequality holdsfor the base cases φ = (cid:62) and φ = p f : − dist ( x , L t ( φ )) ≤ ρ φ ( x )( t ) ≤ dist ( x , D ( E ) \ L t ( φ ))The remaining cases then follow by structural induction on φ . φ = (cid:62) Then x ∈ L t ( φ ) for any x and so dist ( x , L t ( φ )) = ≤ ∞ = ρ φ ( x )( t ) = dist ( x , ∅ ) = dist ( x , D ( E ) \ L t ( φ )). φ = p f . Suppose x ∈ L t ( φ ). For all y ∈ D ( E ) \ L t ( φ ) d ( x , y ) ≥ (cid:104) x − y , f ( · − t ) (cid:105) since f ( · − t ) ∈ D (cid:48) = (cid:104) x , f ( · − t ) (cid:105) − (cid:104) y , f ( · − t ) (cid:105)≥ (cid:104) x , f ( · − t ) (cid:105) since (cid:104) y , f ( · − t ) (cid:105) < = ρ p f ( x )( t )Thus, dist ( x , L t ( φ )) = ≤ ρ φ ( x )( t ) ≤ dist ( x , D ( E ) \ L t ( φ )).Now suppose x (cid:60) L t ( φ ). As before inf y ∈L t ( φ ) d ( x , y ) ≥ inf y ∈L t ( φ ) (cid:104) y , f ( · − t ) (cid:105) − (cid:104) x , f ( · − t ) (cid:105) so dist ( x , L t ( φ )) ≥−(cid:104) x , f ( · − t ) (cid:105) . Thus, − dist ( x , L t ( φ )) ≤ (cid:104) x , f ( · − t ) (cid:105) = ρ p f ( x )( t ) < = dist ( x , D ( E ) \ L t ( φ )). Having defined the logic BB-STL, we are now in a position to answer the question: what is the frequency content oftemporal logic? The strategy will be to show that the robustness of each logical operation ( p f , ¬ , ∨ , I , I , S I ) can beapproximated by a Volterra series, and derive its GFRF. Then using a composition theorem, we can derive the GFRFof entire formulas to deduce which frequencies are given significant weight by the GFRF, and which aren’t.We note at the outset that the robustness operator for (cid:62) , ρ (cid:62) , maps any signal to the improper constant function t (cid:55)→ + ∞ . Because this function is not in C ( R , R ), ρ (cid:62) is not approximable by a finite Volterra series on the basis ofThm. 2. This is not a serious impediment, since it is highly unlikely that an engineer would explicitly include (cid:62) in aspecification (e.g. φ = p ∨ (cid:62) ), so there is no need to approximate ρ (cid:62) to begin with. As for formulas that accidentallyturn out to be tautologies, like p ∨ ¬ p , their STL robustness is not infinite, and neither is their BB-STL robustness.8 .1 Approximability by Volterra Series We state and prove the main technical result of this paper.
Theorem 4.
For any BB-STL formula φ that does not explicitly include (cid:62) , the robustness operator ρ φ : D ( E ) → R R can be approximated by a finite Volterra series. Recall the set K M , M from Thm. 2, and recall that for a function f , (cid:107) f (cid:107) is its essential supremum. We will firstshow that ρ φ is TI and has fading memory. However, the domain of ρ φ is not a set of the form K M , M so we can’t applyThm. 2 directly. So we show how to roughly decompose D ( E ) into sets of the form K M , M and leverage Thm. 2 toconclude. In all that follows, it is understood that φ does not explicitly include (cid:62) . Lemma 2.
The operator ρ φ is TI and has fading memory.Proof. Time invariance is immediate. To prove fading memory we argue by induction on the structure of φ . Base cases.
Fix an arbitrary p f . We must exhibit a weight function s.t. for all ε > u , v ∈ D ( E ), sup t (cid:48) ≤ | u ( t ) − v ( t ) | w ( t ) < δ = ⇒ | N f u (0) − N f v (0) | = | (cid:82) f ( τ )( u ( τ ) − v ( τ )) d τ | < ε . Fix ε >
0, and let w be a continuous increasingfunction from ( −∞ ,
0] to (0 , u , v ∈ D ( E ), g : = u − v is in D ( E ); let C be its compact support. Ifsup t (cid:48) ≤ | g ( t (cid:48) ) | w ( t (cid:48) ) < δ then (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:90) f ( t ) g ( t ) dt (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:90) C f ( t ) g ( t ) dt (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ≤ (cid:90) C ∩ ( −∞ , | f ( t ) || g ( t ) | dt < δ (cid:90) C ∩ ( −∞ , | f ( t ) | / w ( t ) dt The integral is finite and non-zero so choosing δ = ε/ ( (cid:82) | f ( t ) | / w ( t ) dt ) yields the desired result. Inductive cases.
The case of ¬ φ is immediate. • For φ ∨ φ : by the induction hypothesis there exist weight function w k for ρ φ k , k = , ε > t ≤ | u ( t ) − v ( t ) | w k ( t ) < δ = ⇒ | ρ φ k ( u )(0) − ρ φ k ( u )(0) | < ε . Then w = max { w , w } is easily shown to be a weightfunction for ρ φ ∨ φ . • For I φ : By the induction hypothesis, there exists a weight function w s.t. for all ε > u , v ∈ D ( E ) thereexists δ > t ≤ | u ( t ) − v ( t ) | w ( t ) < δ = ⇒ | ρ φ ( u )(0) − ρ φ ( v )(0) | < ε/ Fact.
If sup t ≤ | u ( t ) − v ( t ) | w ( t ) < δ then sup τ ≤ | ρ φ ( u )( τ ) − ρ φ ( v )( τ ) | ≤ ε/ t ≤ | u ( t ) − v ( t ) | w ( t ) < δ then it holds that for all τ ≥
0, sup t ≤− τ | u ( t ) − v ( t ) | w ( t ) < δ , which is equivalent(by a change of variables) to sup t ≤ | u ( t − τ ) − v ( t − τ ) | w ( t − τ ) < δ . But w ( · − τ ) ≤ w sosup t ≤ | u ( t − τ ) − v ( t − τ ) | w ( t − τ ) < δ = ⇒ sup t ≤ | u ( t − τ ) − v ( t − τ ) | w ( t ) < δ Since u ( · − τ ) , v ( · − τ ) are in D ( E ) it follows that | ρ φ ( u )( − τ ) − ρ φ ( v )( − τ ) | < ε/ τ ≥
0, and thereforesup τ ≤ | ρ φ ( u )( τ ) − ρ φ ( v )( τ ) | ≤ ε/ w is a weight function for ρ I φ . Indeed | ρ I φ ( u )(0) − ρ I φ ( v )(0) | = | max t ∈− I ρ φ ( u )( t ) − max t ∈− I ρ φ ( v )( t ) | . Let t u = argmax − I ρ φ ( u )( t ) and t v = argmax − I ρ φ ( v )( t ); both exist since I is compact and ρ φ is contin-uous. Assume the left-hand side of Eq. (10) holds. Then we finally find the string of inequalities − ε < ε/ ≤ ρ φ ( u )( t v ) − ρ φ ( v )( t v ) ≤ max t ∈− I ρ φ ( u )( t ) − max t ∈− I ρ φ ( v )( t ) ≤ ρ φ ( u )( t u ) − ρ φ ( v )( t u ) ≤ ε/ < ε Therefore | ρ φ ( u )(0) − ρ φ ( v )(0) | < ε as desired. • The case of I φ is similar. • For ψ = φ S I φ : suppose there exist weight functions w u and w v for u and v respectively. Write ρ k = ρ φ k , k = ,
2. Set w = max { w u , w v } : this will be the weight function for ρ ψ . Given ε >
0, there exists a δ > t ≤ | u ( t ) − v ( t ) | w ( t ) < δ = ⇒ | ρ k u (0) − ρ k v (0) | < ε . By the above Fact, it also follows that | ρ k u ( t (cid:48) ) − ρ k v ( t (cid:48) ) | < ε ∀ t (cid:48) ≤ , k = , | ρ ψ u (0) − ρ ψ v (0) | < ε , where ρ ψ u (0) = max t (cid:48) ∈− I { min { ρ u ( t (cid:48) ) , min t (cid:48)(cid:48) ∈ ( t (cid:48) , ρ u ( t (cid:48)(cid:48) ) }} . Given t (cid:48) ≤
0, define t u : = argmin ( t (cid:48) , ρ u ( t (cid:48)(cid:48) ) , t v : = argmin ( t (cid:48) , ρ v ( t (cid:48)(cid:48) ). The following inequalities are immediate: ρ v ( t v ) − ε ≤ ρ v ( t u ) − ε < ρ u ( t u ) ≤ ρ u ( t v ) < ρ v ( t v ) + ε Therefore | ρ u ( t u ) − ρ v ( t v ) | < ε (12)From Eqs. (12) and (11) it follows that ∀ t (cid:48) ∈ − I , | min { ρ u ( t (cid:48) ) , min t (cid:48)(cid:48) ∈ ( t (cid:48) , ρ u ( t (cid:48)(cid:48) ) } (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) a ( t (cid:48) ) − min { ρ v ( t (cid:48) ) , min t (cid:48)(cid:48) ∈ ( t (cid:48) , ρ v ( t (cid:48)(cid:48) ) } (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) b ( t (cid:48) ) | < ε With t a : = argmax t (cid:48) ∈− I a ( t (cid:48) ) , t b : = argmax t (cid:48) ∈− I b ( t (cid:48) ) b ( t b ) − ε ≤ b ( t a ) − ε < a ( t a ) ≤ a ( t b ) < b ( t b ) + ε and we get the desired conclusion: | a ( t a ) − b ( t b ) | = | ρ ψ u (0) − ρ ψ v (0) | < ε .We continue with the main proof. A signal x in D ( E ) is infinitely di ff erentiable and compactly supported, so thereexist M and M s.t. x ∈ K M , M . Moreover for every M (cid:48) ≥ M and M (cid:48) ≥ M , K M , M ⊆ K M (cid:48) , M (cid:48) . Thus if we take anyascending sequence ( M , a , M , a ) a ∈ N with first element (0 ,
0) and which is unbounded in both dimensions, we have that D ( E ) ⊂ ∪ a ∈ N K M , a , M , a . (The lexicographic order is used: ( M , a , M , a ) ≤ ( M , a , M , a ) i ff M , a ≤ M , a (cid:48) and M , a ≤ M , a (cid:48) ).For conciseness write K a : = K M , a , M , a . Lemma 3.
The restriction of ρ φ to any K a is an operator over K a , i.e. ρ φ : K a → K a .Proof. Take x ∈ K a , we show that y = ρ φ ( x ) ∈ K a . For any kernel f and t ∈ R , H¨older’s inequality gives (cid:104) f ( · − t ) , x (cid:105) ≤ (cid:82) | f ( τ ) | d τ · (cid:107) x (cid:107) ≤ M , a , so (cid:107) ρ p f ( x ) (cid:107) ≤ M , a . Since the robustness of any formula other than (cid:62) is obtained by taking maxand min of atomic robustness values, (cid:107) ρ φ ( x ) (cid:107) ≤ M , a . Moreover for all t , s ∈ E | y ( t ) − y ( s ) | = (cid:12)(cid:12)(cid:12)(cid:12)(cid:12)(cid:90) f ( τ )[ x ( τ + t ) − x ( τ + s )] d τ (cid:12)(cid:12)(cid:12)(cid:12)(cid:12) ≤ (cid:107) f (cid:107) · (cid:107) x ( · + t ) − x ( · + s ) (cid:107) ≤ M , a | t − s | This shows that y ∈ K a . Lemma 4.
Consider an operator N : D ( E ) → R R such that its restriction N a to K a is an operator over K a . If N is TIand with fading memory, then it has a finite Volterra series approximation (cid:98) N over D ( E ) .Proof. It is immediate that if N is TI and with fading memory, then so is every N a . Thus, fixing ε > N a has a finiteVolterra series approximation over K a by Thm. 2, call it (cid:98) N a , so that for all x ∈ K a , (cid:107) (cid:98) N a x − N a x (cid:107) < ε .For every signal x ∈ D ( E ), let x (cid:48) be its time derivative. Then x ∈ K (cid:107) x (cid:107) , (cid:107) x (cid:48) (cid:107) , and for all M (cid:48) < M and M (cid:48) < M , x (cid:60) K M (cid:48) , M (cid:48) . (The first part of the last statement is immediate; for the second part, note first that there exists t ∗ in thesupport of x s.t. M = | x (cid:48) ( t ∗ ) | , so pick b , c s.t. b ≤ t ∗ ≤ c and x ( c ) = x ( b ) + x (cid:48) ( t ∗ )( c − b ), or | x ( c ) − x ( b ) | = | x (cid:48) ( t ∗ ) | ( c − b ) > M (cid:48) ( c − b )). So there exists a unique smallest pair ( M , a , M , a ) s.t. x ∈ K a , namely the smallest pair s.t. M , a ≥ (cid:107) x (cid:107) and M , a ≥ (cid:107) x (cid:48) (cid:107) . For a given x let a ( x ) be the index in N corresponding to this smallest pair.Define the operator (cid:98) N : D ( E ) → R R by (cid:98) N x : = (cid:98) N a ( x ) x . Then for all x ∈ D ( E ), (cid:107) (cid:98) N x − N x (cid:107) = (cid:107) (cid:98) N a ( x ) x − N a ( x ) x (cid:107) < ε ,which establishes that (cid:98) N is a finite Volterra approximation of N over D ( E ).Combining the three lemmas allows us to conclude the main proof. Even though it is only strictly correct to speakof the Volterra kernels of the Volterra series that approximates the robustness operator ρ φ , we will often abuse languageand speak directly of the ‘Volterra kernels of φ ’. 10 .2 Calculating the Volterra Approximations and their GFRFs We seek the Volterra series that approximates ρ φ for a given formula in the sense of Thm. 2. Operator ρ φ is built bycomposing a few basic operators. The strategy will be to first approximate each basic operator by a Volterra series,then use a composition theorem to compose these into a Volterra series for the entire formula. We exclude the Sinceoperation from the remainder of this discussion because, even though its robustness is approximable by Lemma 4, wedon’t currently have the tools to compute that approximation. We expand on the technical di ffi culty of performing thatapproximation in Section 4.3. Basic operators.
Fix an interval [ a , b ] ⊂ R + , ε > f ∈ D (cid:48) , let u , v denote arbitrary signals in D ( E ). The basicoperators are: N f u ( t ) = (cid:104) f ( · − t ) , u (cid:105) N − u ( t ) = − u ( t ) (cid:116) ( v , u )( t ) = max { v ( t ) , u ( t ) } (cid:117) [ a , b ] u ( t ) = min t − b ≤ t (cid:48) ≤ t − a u ( t (cid:48) ) (cid:116) [ a , b ] u ( t ) = max t − b ≤ t (cid:48) ≤ t − a u ( t (cid:48) ) (13)The following relations hold: ρ p f = N f ρ ¬ φ = N − ◦ ρ φ ρ φ ∧ φ = (cid:117) ( ρ φ , ρ φ ) ρ [ a , b ] φ = (cid:117) [ a , b ] ◦ ρ φ ρ [ a , b ] φ = (cid:116) [ a , b ] ◦ ρ φ We approximate each basic operator, on a representative set of signals, using a structure made of delays followedby a read-out polynomial; this structure can then be represented exactly with Volterra series. It is shown in [5] thatthis structure (delays followed by polynomial) can approximate any discrete-time operator and is a special case of astructure for approximating any continuous-time operator on C ( R , R ).There are many ways to derive Volterra approximations. Here we give a practical and simple way of computingsuch an approximation numerically. The first two operators can be represented exactly as Volterra series. •N f u ( t ) = (cid:104) f ( · − t ) , u (cid:105) . Then h = , h ( t ) = f ( − t ) , h n ≡ n > •N − u ( t ) = − u ( t ). Then h = , h ( t ) = − δ ( t ) , h n ≡ n >
1. Note that N − is never applied directly to a sourcesignal (i.e. monitored signal x ) but only to robustness signals. Robustness signals are produced by previous monitorsand their values are stored (perhaps symbolically) in computer memory, so it is possible to access their instantaneousvalues. So this does not contradict our earlier point about the inability to instantaneously sample an analog sourcesignal . •(cid:116) [ a , b ] u . We approximate this operator by a polynomial P ( u ( t − t ) , . . . , u ( t − t D )) for a given choice of polynomialdegree d and delays t j , a ≤ t j ≤ b . P is of the form (cid:80) r α r u ( t − t ) r . . . u ( t − t D ) r D where the sum is over all integervectors r = ( r , . . . , r D ) s.t. 0 ≤ r j ≤ d , (cid:80) j r j ≤ d , and the α r ’s are the unknown polynomial coe ffi cients. Then givena set of L signals u (cid:96) and the corresponding output signals (cid:116) [ a , b ] u (cid:96) , and given a set T of sampling times, we setup thelinear system in the α r ’s: (cid:88) r α r u (cid:96) ( t − t ) r . . . u (cid:96) ( t − t D ) r D = (cid:116) [ a , b ] u (cid:96) ( t ) , ≤ (cid:96) ≤ L , t ∈ T (14)A least-squares solution yields the α ’s. We force α = (cid:116) operator has 0 response to 0 input. Therefore h =
0. Given this approximation we seek the kernels h n s.t. P ( u ( t − t ) , . . . , u ( t − t D )) = (cid:88) r α r u ( t − t ) r . . . u ( t − t D ) r D = N (cid:88) n = (cid:90) τ ∈ R n h n ( τ ) n (cid:89) j = u ( t − t j ) d τ Define ∆ Dd ( n ) = { r = ( r , . . . , r D ) ∈ N D | ≤ r j ≤ d , (cid:80) j r j = n } and let ∆ Dd = ∪ ≤ n ≤ d ∆ Nd ( n ). For a given r ∈ ∆ Dd ( n ), u ( t − t ) r . . . u ( t − t D ) r D = (cid:90) δ ( τ − t ) . . . δ ( τ r − t ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) r terms δ ( τ r + − t ) . . . δ ( τ r + r − t ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) r terms . . . δ ( τ n − r D + − t D ) . . . δ ( τ n − t D ) (cid:124) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:123)(cid:122) (cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32)(cid:32) (cid:125) r D terms n (cid:89) j = u ( t − τ j ) d τ h r n ( τ , . . . , τ n ) : = α r (cid:81) r j = δ ( τ j − t ) . . . (cid:81) nj = n − r D + δ ( τ j − t D ). We can now express P ( u ( t − t ) , . . . , u ( t − t D )) = (cid:88) r ∈ ∆ Dd α r u ( t − t ) r . . . u ( t − t D ) r D = d (cid:88) n = (cid:88) r ∈ ∆ Dd ( n ) (cid:90) τ ∈ R n h r n ( τ ) n (cid:89) j = u ( t − τ j ) d τ = d (cid:88) n = (cid:90) (cid:88) r ∈ ∆ Dd ( n ) h r n ( τ ) n (cid:89) j = u ( t − τ j ) d τ : = d (cid:88) n = (cid:90) h n ( τ ) n (cid:89) j = u ( t − τ j ) d τ Therefore H = n th -order GFRF is H n ( Ω ) = (cid:88) r ∈ ∆ Dd ( n ) F { h r n } ( Ω ) = (cid:88) r α r exp( − i · t r (cid:88) j = ω j ) . . . exp( − i · t D n (cid:88) j = n − r D + ω j ) (15)The same approach is used with (cid:117) [ a , b ] . •(cid:117) ( u , v )( t ) = min { u ( t ) , v ( t ) } . Here we must use a separable approximation of the form (cid:117) ( u , v ) ≈ U u + V v . Thisavoids product terms involving u and v which cause the loss of the nice GFRF representation of Volterra kernels [4].The Volterra operators U and V are obtained, again, by polynomial fitting. Specifically, U u ( t ) = R ( u ( t )) for a poly-nomial R and V v ( t ) = Q ( v ( t )) for a polynomial Q . Both polynomials have a 0 constant term since zero inputs producea zero output from (cid:117) . Note also that only the present value of the signal, u ( t ), is used, since it doesn’t make sense touse past values u ( t − τ ) when approximating the instantaneous min operator. The coe ffi cients of the polynomials areobtained by least-squares as before. Once the coe ffi cients of R and Q are calculated, the following easily establishedproposition gives the kernels of the equivalent Volterra series. Proposition 1.
The polynomial operator defined by Nu ( t ) = (cid:80) ≤ k ≤ d α k u ( t ) k has an exact Volterra representation givenby h = α , h n ( τ , . . . , τ n ) = α n δ ( τ ) . . . δ ( τ n ) , n ≥ . The corresponding GFRFs are H = πα δ ( ω ) , H n ( Ω ) = α n ∀ Ω . This concludes the derivation of Volterra series for the basic operators. The following theorem allows us to cal-culate the GFRFs of entire formulas. Given Ω ∈ R n and m ∈ ∆ kn ( n ), we can divide Ω into k sub-vectors, Ω = ( Θ , Θ , . . . , Θ k ), s.t. sub-vector Θ j has length m j . Define the mixing matrix S ( k , n ) of dimensions k -by- n whose j th row is ( × ( m + ... + m j − ) , × m j , × ( m j + + ... + m k ) ), so S ( k , n ) Ω = ( (cid:80) m j = ω j , (cid:80) m + m j = m + ω j , . . . , (cid:80) nj = n − m k + ω j ) T . Theorem 5 ([6]).
Let A , B be Volterra operators with GFRFs { A n } n A n = and { B n } n B n = respectively. Then the operator H : = B ◦ A has Volterra GFRFs given byH = n B (cid:88) k = B k (0) A k H n ( Ω ) = n B (cid:88) k = (cid:88) m ∈ ∆ kn ( n ) B k ( S ( k , n ) Ω ) k (cid:89) j = A m j ( Θ j ) , n ≥ φ = [0 , . g for some atom g , we derive the GFRF { B k } of (cid:116) [0 , . and { A k } of g , then compute the GFRF of N φ using Thm. 5. φ S I φ Di ff erent? For convenience, we write ψ = φ S I φ . The robustness ρ ψ is an operator on C ( R , R ), and we have shown that it isapproximable by a Volterra series. However it is constructed out of operators that change the dimensions of the signals,which adds di ffi culties to the actual computation of the approximation.Specifically: fix an interval [ a , b ] ⊂ R + , ε > f ∈ D (cid:48) ; let u denote an arbitrary signal in D ( E ) and let y ∈ C ( R , R ), i.e. a continuous bounded function from R to R . We define three operators: (cid:116) → : C ( R , R ) → C ( R , R ), (cid:117) → : C ( R , R ) → C ( R , R ), and (cid:117) → : C ( R , R ) → C ( R , R ). They are:12 → y ( t ) = max t − b ≤ t (cid:48) ≤ t − a y ( t (cid:48) , t ) (cid:117) → u ( t (cid:48) , t ) = min t (cid:48) < t (cid:48)(cid:48) ≤ t u ( t (cid:48)(cid:48) ) (cid:117) → ( u , y )( t (cid:48) , t ) = min { u ( t (cid:48) ) , y ( t (cid:48) , t ) } The following relation holds: ρ φ S [ a , b ] φ = (cid:116) → ◦ (cid:117) → ( ρ φ , (cid:117) → ρ φ )The approximation of ρ ψ by Volterra series therefore requires the approximation of the above basic operators,then composing them. Multi-dimensional Volterra series exist (i.e., Volterra operators over C ( R n , R )), e.g., see [19].However what we have above are operators that change the dimensions of the signals. Sandberg [16] provides ageneralization of [5] which allows the approximation of certain operators that map C ( R n , R m ) to C ( R m , R ). Howeverthis still falls short of our needs because of the presence of (cid:117) → .A ‘quick-and-dirty’ way to produce a Volterra series representation of a given formula ψ with Since - that is, withgiven atoms and structure - is to approximate its input-output relation on a representative set of signals by fittingVolterra kernels. However this requires a new fit every time we change atoms or formula structure. It does not providea generic approximation that can be composed with others, as we did in Section 4. We implemented the above calculations in a toolbox which we’ll make available with the paper. In this section wedemonstrate the derivation of Generalized Frequency Response Functions for temporal logic robustness operators. Inall experiments, the GFRFs were generated by solving appropriate versions of (14) with degree-4 polynomials and testsignals generated as random combinations of sinusoids. Sinusoids are dense in C ( R , R ) so approximating the operatorson sinusoids is a sensible thing to do. The approximation error in all cases was in the order of 10 − . That said, ourobjective here is not to provide the most e ffi cient or the most general approximation scheme - that is for future work.We reiterate that the Volterra approximations are not meant to replace the monitoring algorithms that exist. Theyare used as analysis tools that provide a rigorous quantitative Fourier analysis of temporal logic: one that does notdepend on intuition, is automatic, and such that once the GFRFs of a formula are obtained, the formula (and itsmonitor) are treated as just another signal processing box.In what follows, g = G ( µ, s ) means that g is a Gaussian measurement kernel with mean µ and standard deviation s . • We first consider the spectra of [1 , T ] p g shown in Fig. 3a, with g = G (0 , . T has a first-ordere ff ect (observed in H ) of distributing the energy more uniformly over the range [0, 2.5]Hz, and suppressing less thehigher frequencies. | H | on the other hand shows a more complex picture: while there’s an increase of magnitude athigher values of f or f (top left and bottom right corners), the increase at higher f and f is less marked. • Consider next the formula [0 , T ] p for a fixed atom p , shown Fig. 3b. As T increases, H becomes more low-pass, but H becomes more high-pass! This emphasizes the need to study all orders of the response, not only the linearfirst-order response. • We now study the e ff ect of using non-instantaneous measurements. Fig. 4a shows the spectra H of [0 , . p g and [0 , . p g where g = G (0 , s ) for three values of s . As s increases, the Gaussian atom acts more like a low-passfilter (the measurement is lower resolution) and the overall formula has a more low-pass nature. By the same token,high-frequency noise is ignored by the formula and does not a ff ect the monitoring verdict. Similarly, the 2 nd -orderspectra for these two formulas are shown in Fig. 4b with increasing s .In practice, the filter f used in atomic propositions is imposed by the application and is derived from first-principlesmodeling of the physics of the system. This Fourier analysis allows us to trace these e ff ects quantitatively. • Consider now the more complex formula φ T , which says that a is true, preceded by b T units earlier, precededby c T units earlier than that.. Here a , b and c are atoms with Gaussian filters of various widths. φ T = a ∧ ( [0 , T ] ( b ∧ [0 , T ] c )) (16)It is not possible to read, from the formula, how the frequency responses of the various sub-monitors (for the sub-formulas) interact or cancel each other out. By contrast, Fig. 5 shows the signal block diagram for computing this13 a) | H | and | H | for Historically [1 , T ] p g with T = . T = . g = G (0 , . | H | and | H | for Once [1 , T ] p g for four values of T ∈ { . , . , . , . } . The atom g = G (0 , . Fig. 3: GFRFs with varying temporal intervals. Color in digital copy.formula’s Volterra series. This can be read as just another signal processing chain with non-linear filters. On top ofeach box, we display the GFRF H of the entire chain up to and including that box. This shows how the relevantfrequencies evolve with the addition of each monitoring component). ff ect on monitoring We now illustrate what happens if attention is not paid to the frequency representation of temporal logic formulaswhen designing compression or filtering algorithms. In Fig. 2, we The proposed method can be used as a signalprocessing tool for frequency domain analysis of temporal logic monitors but it is not supposed to replace the monitorsthemselves. Thus,it is more of an o ffl ine analysis tool that can be used to design filters which respect the monitorability14 a) | H | of Once [1 , . p g (left) and Historically [0 , . p g with g = G (0 , s ) , s = . , . , . | H | of Once [1 , . p g (left) and Historically [1 , . p g (right) with g = G (0 , s ) , s = . , . Fig. 4: E ff ect of support size s for the atomic proposition filters. In (a), s is larger than temporal interval width, whichis 0.1. In (b) s is much smaller. (Color in digital copy)requirements. had shown how knowledge of the GFRFs allows us to perform monitoring-safe compression: eventhough the post-compression signal is markedly di ff erent from the original x , the monitoring results for the two signalswere almost identical.By contrast, in Fig. 6, we show the same signal but now compressed (via low-pass filtering) without regard to theGFRF or the monitored formula. The resulting monitoring result (in blue) is significantly a ff ected, and the truth value(determined by checking where ρ φ is positive or negative) is modified. We have presented a Fourier analysis of temporal logic using Volterra approximations of the robustness operators.Doing so has necessitated re-defining the semantics of atomic propositions using bounded-bandwidth filters, whichled us to introduce the logic Bounded-Bandwidth STL. Using this analysis, it is possible to incorporate temporal logicmonitors into signal processing chains as ‘just another’ signal processing box.Future work will seek to relax the constraints on the signal space. In particular, we conjecture that it is possible toremove the compact-support requirement. We will also seek more general approximations of the basic operators andextend them to Since. Finally, the frequency representation in this paper presents a unifying formalism which we will15ig. 5: Block diagram of the Volterra representation of ϕ T given in 16. Every displayed | H | is the first-order spectrumof the entire composite formula up to that point. Un and Vn are the GFRFs of the separable Volterra operators U , V that approximate (cid:116) (Section 4.2.)Fig. 6: Filtering signals without accounting for downstream logic monitors leads to incorrect monitoring results. Thefrequency responses (right two panels) indicate a safe cut-o ff frequency around 1.5Hz. If an upstream low-pass filterapplies a cut-o ff of 0.5Hz (left panel), the robustness signal is significantly changed (second panel). In particular, thetruth values di ff er between red (original) and blue (post-filtering). (Colors in digital copy).leverage for optimal filter design that is monitoring-safe, i.e., that does not remove any signal content that is relevantto the output robustness signal. Acknowledgments
The authors would like to thank the anonymous reviewers for helpful comments regarding the definition of kernelspace.
References
1. Abbas, H., Mangharam, R.: Generalized robust mtl semantics for problems in cardiac electrophysiology. In: 2018 AnnualAmerican Control Conference (ACC). pp. 1592–1597 (2018)2. Abbas, H., Pant, Y.V., Mangharam, R.: Temporal logic robustness for general signal classes. In: Proceedings of the 22ndACM International Conference on Hybrid Systems: Computation and Control. p. 4556. HSCC 19, Association for Computing achinery, New York, NY, USA (2019). https: // doi.org / / https://doi.org/10.1145/3302504.3311817
3. Blanchard, P., Br¨uning, E.: Mathematical Methods in Physics. Progress in Mathematical Physics, Birkh¨auser (2003)4. Boaghe, O.M., Billings, S.A.: Subharmonic oscillation modeling and MISO Volterra series. IEEE Transactions on Circuits andSystems I: Fundamental Theory and Applications (7), 877–884 (2003)5. Boyd, S., Chua, L.: Fading memory and the problem of approximating nonlinear operators with volterra series. IEEE Transac-tions on Circuits and Systems CAS-32 (11) (Nov 1985)6. Carassale, L., Kareem, A.: Modeling nonlinear systems by volterra series. Journal of Engineering Mechanics (6) (Nov2010)7. Donz´e, A., Maler, O.: Robust satisfaction of temporal logic over real-valued signals. In: Proceedings of the InternationalConference on Formal Modeling and Analysis of Timed Systems (2010)8. Donz´e, A., Maler, O., Bartocci, E., Nickovic, D., Grosu, R., Smolka, S.: On temporal logic and signal processing. In:Chakraborty, S., Mukund, M. (eds.) Automated Technology for Verification and Analysis. pp. 92–106. Springer Berlin Heidel-berg, Berlin, Heidelberg (2012)9. Fainekos, G., Pappas, G.: Robustness of temporal logic specifications for continuous-time signals. Theoretical Computer Sci-ence (2009)10. Florack, L.: Image Structure. Kluwer Academics (2013)11. Jaksi´c, S., Bartocci, E., Grosu, R., Nickovi´c, D.: An algebraic framework for runtime verification. IEEE Transactions onComputer-Aided Design of Integrated Circuits and Systems (11), 2233–2243 (2018)12. Jing, X., Lang, Z.: Frequency Domain Analysis and Design of Nonlinear Systems based on Volterra Series Expansion. Springer(2015)13. Maler, O., Nickovic, D.: Monitoring Temporal Properties of Continuous Signals. Springer Berlin Heidelberg (2004)14. Pfander, G.E., Walnut, D.F.: Sampling and reconstruction of operators. IEEE Transactions on Information Theory (1), 435–458 (2016)15. Rodionova, A., Bartocci, E., Nickovic, D., Grosu, R.: Temporal logic as filtering. In: Proceedings of the 19th InternationalConference on Hybrid Systems: Computation and Control. p. 1120. HSCC 16, Association for Computing Machinery, NewYork, NY, USA (2016). https: // doi.org / / https://doi.org/10.1145/2883817.2883839
16. Sandberg, I.W., Xu, L.: Uniform approximation of multidimensional myopic maps. IEEE Transactions on Circuits and SystemsI: Fundamental Theory and Applications (6), 477–500 (1997)17. Silvetti, S., Nenzi, L., Bartocci, E., Bortolussi, L.: Signal convolution logic. In: Lahiri, S.K., Wang, C. (eds.) AutomatedTechnology for Verification and Analysis. pp. 267–283. Springer International Publishing, Cham (2018)18. Slepian, D.: On bandwidth. In: Proceedings of the IEEE. vol. 64 (March 1976)19. Thurnhofer, S., Mitra, S.K.: A general framework for quadratic volterra filters for edge enhancement. IEEE Transactions onImage Processing (6), 950–963 (June 1996). https: // doi.org / /83.503911