Bringing Runtime Verification Home -- A Case Study on the Hierarchical Monitoring of Smart Homes using Decentralized Specifications
BBringing Runtime Verification Home (cid:63)
A Case Study on the Hierarchical Monitoring of Smart Homesusing Decentralized Specifications
Antoine El-Hokayem and Yli`es Falcone Univ. Grenoble Alpes, CNRS, Grenoble INP, VERIMAG, 38000 Grenoble, France Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LIG, 38000 Grenoble, France [email protected]
Abstract.
We use runtime verification (RV) to check various specifications in a smart apartment. The specifica-tions can be broken down into three types: behavioral correctness of the apartment sensors, detection of specificuser activities (known as activities of daily living), and composition of specifications of the previous types. Thecontext of the smart apartment provides us with a complex system with a large number of components with twodifferent hierarchies to group specifications and sensors: geographically within the same room, floor or globallyin the apartment, and logically following the different types of specifications. We leverage a recent approach todecentralized RV of decentralized specifications, where monitors have their own specifications and communi-cate together to verify more general specifications. We leverage the hierarchies, modularity and re-use affordedby decentralized specifications to: (1) scale beyond existing centralized RV techniques, and (2) greatly reducecomputation and communication costs.
Sensors and actuators are used to create “smart” environments which track the data across sensors and human-machine interaction. One particular area of interest consists of homes (or apartments) equipped with a myriad ofsensors and actuators, called smart homes [21]. Smart homes are capable of providing added services to users. Theseservices rely on detecting the user behavior and the context of such activities [16], typically detecting activities ofdaily living (ADL) [49,18] from sensor information. Detecting ADL allows to optimize resource consumption (suchas electricity [1]), improve the quality of life for the elderly [44] and users suffering from mild impairment [50].Relying on information from multiple sources and observing behavior is not just constrained to activities. It isalso used with techniques that verify the correct behavior of systems.
Runtime Verification (RV) [37,42,6,7,8] isa lightweight formal method which consists in verifying that a run of a system is correct wrt a specification. Thespecification formalizes the behavior of the system typically in logics (such as variants of Linear Temporal Logic,LTL [46]) or finite-state machines. Based on the provided specification, monitors are automatically synthesizedto run alongside the system and verify whether or not the system execution complies with the specification. RVtechniques have been used for instance in the context of automotive [20] and medical [43] systems. In both cases,RV is used to verify communication patterns between components and their adherence to the architecture and theirformal specifications.While RV can be used to check that the devices in a smart home are performing as expected, we show it can beextended to monitor ADL, and complex behavior on the activities themselves. We identify three classes of specifi-cations for applying RV to a smart home. The first class pertains to the system behavior. These specifications areused to check the correct behavior of the sensors, and detect faulty sensors. Ensuring that the system is behavingcorrectly is what is generally checked when performing RV. However, it is also possible to use RV to verify otherspecifications. The second class consists of specifications for detecting ADL, such as detecting when the user iscooking, showering or sleeping. The third class pertains to user behavior. These specifications can be seen as meta-specifications for both system correctness and ADL, they can include safety specifications such as ensuring that theuser does not sleep while cooking, or ensuring that certain activities are only done under certain conditions.However, standard RV techniques are not directly suitable to monitor the three classes of specifications. This ismainly due to scalability issues arising from the large number of sensors, as typically RV techniques rely on a largeformula to describe specifications. Synthesizing centralized monitors from certain large formulas considered in thispaper is not possible using the current tools. Instead, we make use of RV with decentralized specifications [26,29],as it allows monitors to reference other monitors in a hierarchical fashion. The advantage of this is twofold. First, (cid:63)
This work is supported by the French national program “Programme Investissements d’Avenir IRT Nanoelec” (ANR-10-AIRT-05). The authors thank the Amiqual4Home (ANR-11-EQPX-0002) team, in particular S. Borkowski and J. Crowley forassisting in the case study and J. Cumin, for providing the collected data. a r X i v : . [ c s . S E ] S e p Antoine El-Hokayem and Yli`es Falcone it provides an abstraction layer to relate specifications to each other. This allows specifications to be organizedand changed without affecting other specifications, and even to be expressed with different specification languages.Second, it leverages the structure and layout of the devices to organize the hierarchies. On the one hand, we havea geographical hierarchy resulting from the spacial structure of the apartment from a given device, to a room, afloor, or the full apartment. On the other hand, we have a logical hierarchy defined by the interdependence betweenspecifications, i.e. ADL, specifications that use other ADL specifications, and specifications that combine sensorsafety with ADL specifications. For example, informally, consider checking two activities: sleeping and cooking,which can be expressed using formulae ϕ s and ϕ c respectively. A monitor that checks whether the user is sleepingand cooking requires to check ϕ s ∧ ϕ c and as such will replicate the monitoring logic of another monitor that checks ϕ s alone, instead of re-using the output of that monitor. The formula will be written twice, and changing the formulafor detecting sleeping requires changing the formula for the monitor that checks both specifications.Overall, we see our contributions as follows :- We apply decentralized RV to analyze traces of over 36,000 timestamps spanning 27 sensors in a real smartapartment (Sect. 1.1).- We show how to go beyond system properties, to specify ADL using RV, and more complex interdependent speci-fications defined on up to 27 atomic propositions (Sect. 1.2).- We leverage the hierarchies, modularity and re-use afforded by decentralized specifications (Sect. 2) to both be ableto synthesize monitors and to reduce overhead when monitoring complex interdependent specifications (Sect. 5.1).- We improve the existing data structures used for monitoring decentralized specifications, to account for large traces(Sect. 4).- We use RV to effectively monitor ADL and identifying some insights and limitations inherent to using formal LTLspecifications to determine user behavior (Sect. 5.2).- We elaborate on the advantages of modularity by adapting parts of the specification to the Activity Recognitionwith Ambient Sensing (ARAS) [2] dataset (Sect. 5.3).This paper extends existing work published in the proceedings of the the international conference on RuntimeVerification (RV 2018) [28] with the following:- Providing a more detailed explanation of decentralized specifications and their dependency hierarchies (Sect. 2.2);- Providing full details on trace generation, sensor polling, and trace replay using
THEMIS (Sect. 3);- Enhancing the existing data structures of [26] to support large traces, by elaborating on data structures, theiroperations, and strategies for garbage collection and lazy evaluation in Sect. 4;- Extending the evaluation section to include additional days where the trace is replayed, to illustrate changes in userbehavior in Sect. 5.2, adding more details for modifying the specification to improve precision and recall, and alsoillustrating adaptability to new environments by porting the specification to the ARAS dataset in Sect. 5.3.
We consider an actual apartment, with multiple rooms, where activities are logged using sensors. Amiqual4Home [41]is an experimental platform consisting of a smart apartment, a rapid prototyping platform, and tools for observinghuman activity.
Overview of Amiqual4Home.
The Amiqual4Home apartment is equipped with 219 sensors and actuators spreadacross 2 floors. Amiqual4Home uses the OpenHab 6 integration platform for all the sensors and actuators installed.Sensors communicate using the KNX, MQQT and UPnP protocols sending measurements to OpenHab over thelocal network, so as to preserve privacy. The general layout of the apartment consists of 2 floors: the ground andfirst floors. On the ground floor (resp. first floor), we have the following rooms: entrance , toilet , kitchen , and livingroom (resp. office , bedroom , and bathroom ). Between the two floors, there is a connecting staircase .This layout reveals a tree-like geographical hierarchy of components, where we can see the rooms at the leaves,grouped by floors then the whole apartment. While in effect all device data is fed to a central observation point, it isreasonable to consider the hierarchy in the apartment as a simpler model to consider hierarchies in general, as one isbound to encounter a hierarchy at a higher level (from houses, to neighborhoods, to smart cities, etc.). Furthermore,hierarchies appear when integrating different providers for devices in the same house. Reusing the Orange4Home dataset.
Amiqual4Home has been used to generate multiple datasets that recordall sensor data, this includes an ADL recognition dataset [41] (ContextAct@A4H), and an energy consumption An artifact [25] that contains data, documentation, and software, is provided to replicate and extend on the work.ringing Runtime Verification Home 3
Entering
Entrance Up Staircase
Showering
Bathroom
Sink
Bathroom
Down
Staircase TV Livingroom Up Staircase
Computing
Office
Down
Staircase
Preparing
Kitchen
Cooking
Kitchen
Eating
Livingroom
Dishes
Kitchen
Cleaning
Kitchen Up Staircase
Sink
Bathroom
Dressing
Bedroom
Reading
Bedroom
Napping
Bedroom
Dressing
Bedroom
Computing
Office TV Office
Down
Staircase
Leaving
Entrance
Fig. 1: Suggested Schedule (Tuesday, Jan 31 2017)dataset [22] (Orange4Home). In this paper, we reuse the dataset from [22]. The case study involved a person livingin the apartment and following (loosely) a schedule of activities spread out across the various rooms. The schedulewas set out by the authors of [22]. Figure 1 displays the suggested schedule of activities for Tuesday, Jan 31 2017.This allows us to nicely reconstruct the schedule from the result of monitoring the sensors. Furthermore, the personliving in the home provided manual annotations of the activities done, which helps us assess our specifications. Wechose to use the Orange4Home dataset over the ContextAct@A4H one as it involves only one person living in thehouse at a time which simplifies specifying and validating specifications.
Monitoring environment.
In total, we formalize 22 specifications that make use of up to 27 sensors, and evaluatethem over the course of a full day of activity in the apartment. That is, we monitor the house (by replaying the trace)from 07:30 to 17:30 on a given day, by polling the sensors every 1 second, creating a trace of a total of 36,000timestamps. Specifications are elaborated in Sect. 1.2 and expressed as decentralized specifications [26] (recalled inSect. 2.2). Traces are replayed using the
THEMIS tool [27] which supports decentralized specifications and providesa wide range of metrics. We elaborate on the trace replay in Sect. 3.
We now express the specifications that describe different behaviors of components in the smart apartment. Speci-fications can be subdivided into 3 groups: system-behavior specifications, user-behavior specifications, and meta-specifications on both system and user behavior. The considered specifications are listed in Table 1.
System behavior.
The first group of specifications consists in ensuring that the system behaves as expected. That is,verifying that the sensors are working properly. These properties are the subject of classical RV techniques [31,15]applied to systems. For the scope of this case study, we verify light switches as system properties. We verify that fora given room i , whenever the switch is toggled, then the light must turn on until the switch is turned off. We verifythe property at two scopes, for a given room, and the entire apartment. While this property appears simple to check,it does highlight issues with existing centralized techniques applied in a hierarchical way. We develop the propertyin Sect. 2.1, and show the issues in Sect. 2.2. ADL.
The second group of specifications is concerned with defining the behavior of the user inferred from sensors.The sensors available in the apartment provide us with a wealth of information to determine the user activities. Thelist of activities of interest is detailed in [40] and includes activities such as cooking and sleeping. By correctly iden-tifying activities, it is possible to decide when to interact with the user in a smart setting [1], provide custom caresuch as nursing for the elderly [44], or help users who suffer from mild impairment [50]. Inferring activities done bythe user is an interesting problem typically addressed through either data-based or knowledge-based methods [18].The first method consists in learning activity models from preexisting large-scale datasets of users behaviors by uti-lizing data mining and machine learning techniques. The built models are probabilistic or statistical activity modelssuch as Hidden Markov Model (HMM) or Bayesian networks, followed by training and learning processes. Data-driven approaches are capable of handling uncertainty, while often requiring large annotated datasets for training andlearning. The second method consists in exploiting prior knowledge in the domain of interest to construct activitymodels directly using formal logical reasoning, formal models, and representation. Knowledge-driven approachesare semantically clear, but are typically poor at handling uncertainty and temporal information [18]. We elaborateon such limitations in Sect. 5.2. Writing specifications can be seen as a knowledge-based approach to describe thebehavior of sensors. As such, we believe that runtime verification is useful to describe an activity as a specification
Antoine El-Hokayem and Yli`es Falcone
Table 1: Specifications considered in this paper. (*) indicates added ADL specifications. G indicates specificationgroup: system (S), ADL (A), and meta-specifications (M). | AP | d (resp. ( | AP | c ): atomic propositions needed tospecify specification in decentralized (resp. centralized) specifications. d is the maximum depth of monitor depen-dencies. G Scope Name Description | AP | d | AP | c d S Room sc light ( i ) light switch turns on light ( i ∈ [0 .. ). 2 2 1M House sc ok All light switches are ok. 4 8 2A Toilet toilet ∗ Toilet is being used. 1 1 0A Bathroom sink usage
Sink is being used. 1 2 1A Bathroom shower usage
Shower is being used. 1 2 1A Bedroom napping
Tenant is sleeping on the bed. 1 1 1A Bedroom dressing
Tenant is dressing, using the closet. 2 3 1A Bedroom reading
Tenant is reading. 3 5 2A Office office tv
Tenant is watching TV. 1 1 1A Office computing
Tenant is using the computer. 1 1 1A Kitchen cooking
Tenant is cooking food. 2 2 1A Kitchen washing dishes
Tenant is cleaning dishes. 2 3 1A Kitchen kactivity ∗ Using cupboards and fridge. 4 9 1A Kitchen preparing
Tenant is preparing to cook food. 2 11 2A Living livingroom tv
Tenant is watching TV. 2 2 1A Floor 0 eating
Tenant is eating on the table. 2 2 1M Floor 0 actfloor (0)
Activity triggered on floor 0. 6 16 3M Floor 1 actfloor (1)
Activity triggered on floor 1. 7 11 3M House acthouse
Activity triggered in house 2 27 4M House notwopeople
No 2 simultaneous activities on different floors. 2 27 4M House restricttv
No watching TV for more than 10s. 2 3 3M House firehazard
No cooking while sleeping. 2 3 2 over sensor outputs. We formalize a specification for the following ADL activities described in [22] (see Table 1).We re-use the traces to verify that our detected activities are indeed in line with the proposed schedule. Figure 2 dis-plays the reconstructed schedule after detecting ADL with runtime verification. Each specification is represented bya monitor that outputs (with some delay) for every timestamp (second) verdicts (cid:62) or ⊥ . To do this, the monitor findsthe verdict for a timestamp t then respawns to monitor t + 1 . Verdict (cid:62) indicates that the specification holds, that is,the activity is being performed. The reconstructed schedule shows the eventual outcome of a specification for a giventimestamp ignoring delay. In reality some delay happens based on the specification itself, and the dependencies onother monitors. Meta-specifications.
Specifications of the last group are defined on top of the other specifications. That is, werefer to a meta-specification as a specification that defines the interactions between various specifications. Whileone can easily define specifications by defining predicates over existing ones, such as checking that the light switchspecification holds in all rooms or whether or not detecting an activity was performed on a specific floor or globallyin the house, we are more interested in specifications that relate to each other. We consider a meta-specification thatreduces fire hazards in the house. In this case, we specify that the tenant should not cook and sleep at the same time,as this increases the risk of fire. In addition to mutually excluding specifications, we can also constrain the behaviorof existing specifications. For example, we can specify a specification regulating the duration of watching TV to beat most timestamps. We show how we monitor the apartment using decentralized specifications, while highlighting their advantages.
To monitor the apartment, we use LTL3 monitors [15]. LTL3 [13,14] is a variant of the standard Linear TemporalLogic (LTL) [46] giving a semantics to finite traces. An LTL3 monitor is a complete and deterministic Mooreautomaton where states are labeled with the verdicts in a domain B = {(cid:62) , ⊥ , ? } . Verdicts (cid:62) and ⊥ respectivelyindicate that the current execution complies and does not comply with the specification, while verdict ? indicates that ringing Runtime Verification Home 5 Fig. 2: Detected ADL for Tuesday, Jan 31 2017. Time is in hours starting from 7:30.the verdict has not been determined yet. Verdicts (cid:62) and ⊥ are called final, as once the monitor outputs (cid:62) or ⊥ fora given trace, it cannot output a different verdict for any suffix of that trace. Using LTL3 monitors for representingspecifications allows us to take advantage of the multiple RV tools that convert different specification languages toLTL3 monitors. For our monitoring, we use the THEMIS tool [27] which is able to use both ltl2mon [15] and
LamaConv [38] to generate monitors. ltl2mon generates LTL3 monitors from LTL formulae, while
LamaConv supports a wider range of languages such as Regular Expressions, Omega Regular Expressions, LTL, LTL with past(pLTL), Regular LTL (RLTL) and RLTL with past (pRLTL), and Structured Assertion Language for Temporal Logic(SALT) [12].
Example 1 (Check light switch).
Let us consider property sc light ( i ) (sensor check light): “Whenever a lightswitch is triggered in a room i at some timestamp t , then the light must turn on at t + 1 until the switch is turnedoff again”. Figure 3a shows the Moore automaton that represents the property. Starting from q with verdict ? , theautomaton verifies that the property is falsified (as it is a safety property). That is, upon reaching q the verdict willbe ⊥ for all possible extensions of a trace.For the scope of this paper and for clarity, we use LTL extended with two (syntactic) operators, mostly to strengthenand relax time constraints. We consider the operator eventually within t ( ♦ ≤ t ) which considers a disjunction of nextoperators. It is defined as: ♦ ≤ t ap def = ap ∨ (cid:13) ap ∨ (cid:13) (cid:13) ap ∨ ... (cid:13) t ap , where ap is an atomic proposition. Intuitively,the eventually within states that ap holds within a given number of timestamps. Operator ♦ ≤ t allows us to relax thetime constraints for a given atomic proposition. Similarly, we consider the operator globally within t ( (cid:3) ≤ t ) which isthe dual of the previous operator: (cid:3) ≤ t ap def = ap ∧ (cid:13) ap ∧ (cid:13) (cid:13) ap ∧ (cid:13) t ap . Example 2 (Check light switch modalities).
The property expressed in Ex. 1 can be expressed in LTL as: sc light ( i ) def = (cid:3) (s i = ⇒ (cid:13) ( (cid:96) i U ¬ s i )) . The property can be modified with the extra operators relax or constrain the timeon the light. The relaxed property sc light (cid:48) ( i ) def = (cid:3) (s i = ⇒ ♦ ≤ ( (cid:96) i U ¬ s i )) allows the right-hand side ofthe implication to hold within any of the next 3 timestamps instead of immediately after. The bounded property Antoine El-Hokayem and Yli`es Falcone q start q q s i ¬ s i ¬ s i s i ∧ (cid:96) i (cid:62) s i ∧ ¬ (cid:96) i (a) Centralized q start q q s i ¬ s i ¬ s i s i ∧ light i (cid:62) s i ∧ ¬ light i q (cid:48) start q (cid:48) q (cid:48) (cid:96) i ¬ (cid:96) i (cid:62) (cid:62)A sc light i A light i (b) Decentralized Fig. 3: Monitor(s) for sc light ( i ) , for a given room i in the house. The verdicts associated with the states are ⊥ :dotted red , (cid:62) : double green, and ? : single yellow. sc light (cid:48)(cid:48) ( i ) def = (cid:3) (s i = ⇒ (cid:3) ≤ ( (cid:96) i )) states that the light is on starting from the timestamp the switch is turnedon and the subsequent two (for a total of 3). An example of such a property is the restriction on watching TV for aspecific duration (Table 1) where restricttv def = (cid:3) ( tv = ⇒ ♦ ≤ ¬ tv ) . While simple specifications can be expressed with both LTL and automata, it quickly becomes a problem to scalethe formulae or account for hierarchies (see Sect. 2.3). As such, we use decentralized specifications [26].
Overview.
Decentralized specifications consider a system of multiple components C = {C . . . C n } , where the setof all atomic propositions (noted AP ) (i) has a partition over all components, i.e., AP = AP ∪ . . . ∪ AP n such that ∀ i, j ∈ [1 ..n ] , i (cid:54) = j = ⇒ AP i ∩ AP j = ∅ , and (ii) each component has at least one atomic proposition to monitor(i.e., ∀ i ∈ [1 ..n ] , AP i (cid:54) = ∅ ). Details for assigning sensor information as atomic propositions for this case studyare presented in Sect. 3.2. Furthermore, we have a set of monitor labels AP mons (called monitor references ), thatassociates each monitor with a label. For this case study, each specification in Table 1 is assigned a monitor labeled byits name. Each monitor A lbl ( lbl ∈ AP mons ) is a Moore automaton (detailed in Sect. 2.1) and is assigned to a singlecomponent. A monitor A lbl assigned to component C j ∈ C utilizes the alphabet AP lbl = AP j ∪ ( AP mons \ { lbl } ) .That is, it contains the atomic propositions local to the component (in AP j ), and the references to all dependentmonitors excluding itself ( AP mons \ { lbl } ). A decentralized trace is a partial function that assigns each componentand timestamp with an event. A monitor reference is evaluated as if it were an oracle. That is, to evaluate a monitorreference lbl at a timestamp t , the monitor referenced ( A lbl ) is executed starting from the initial state on the tracestarting at t . The atomic proposition lbl at t takes the value of the final verdict reached by the monitor. Example 3 (Decentralized light switch).
Figure 3b shows the decentralized specification for the check light propertyfrom Ex. 1. We have two monitors A sc light i and A light i . They are respectively attached to the light switch and lightbulb components. In the former, the atomic propositions are either related to observations on the component ( s i ,switch on), or references to other monitors ( light i ). The light switch monitor first waits for the switch to be on toreach q . In q , at some timestamp t , it needs to evaluate reference light i by running the trace starting from t onmonitor A light i . Monitor A light i then reads the value of (cid:96) i at t from the trace, and moves to q (cid:48) or q (cid:48) depending onits value, and sends the verdict (cid:62) or ⊥ respectively back to monitor A sc light i . The returned verdict is associatedwith the reference light i for timestamp t allowing monitor A sc light i to evaluate its own transition at t . Assumptions.
The assumptions of decentralized specifications on the system are as follows: no monitors sendmessages that contain wrong information; no messages are lost, they are eventually delivered in their entirety butpossibly out-of-order; all components share one logical discrete clock marked by round numbers indicating relevanttransitions in the system specification. While security is a concern in the smart apartment setting, the first twoassumptions are met in this case study as the apartment sensor network operates on the local network, and we expectmonitors to be deployed by the sensor providers, and users of the apartment. The last assumption is also met in thesmart setting, as all sensors share a global clock.
Hierarchical dependencies.
Decentralized specifications allow us to analyze the dependencies between variousmonitors, and organize them in logical hierarchies represented as directed acyclic graphs (DAGs). The DAGs helpus relate specifications to other specifications and analyze the inter-dependent behavior of monitors. We elaborateon the benefits of the hierarchical dependencies in Sect. 2.3.
Example 4 (Hierarchical dependencies).
Figure 4 presents the dependency DAG of specification preparing . Wecan see that specification preparing depends directly on both specifications kactivity and cooking . Specifica- ringing Runtime Verification Home 7 preparingkactivity cookingsink water fridge door presencecupboard cooktop ovenoven ∗ cook ∗ pres ∗ fdoor ∗ cold ∗ hot ∗ c2 ∗ c3 ∗ c4 ∗ c5 ∗ c1 ∗ Fig. 4: Dependencies for preparing . * indicates an atomic proposition of a component.tion kactivity depends on specifications cubpoard , sink water , presence , and fridge door , as it dependson the tenant being present in the kitchen, opening or closing cupboards or the fridge, or using the sink. The laterspecifications do not depend on other specifications but on direct observations from the components. We note thatwhile presence is not used in this case study to determine the cooking activity, since a tenant can start cookingand leave the kitchen. One could imagine that specifications can share dependencies, as such the hierarchy is indeedbest represented as a DAG. Let us consider the monitor checking specification cupboard . Since we have 5 cupboarddoors, we have 5 sensors in total (1 for each door). The monitor observing the 5 different observations simply checksif one is open and relays its verdict upwards, transmitting only the summary of observations instead of the totality.In this example, the hierarchy can be seen starting from different sensors on the same component, and expandinggeographically to the different components in the room (kitchen). Monitor references in decentralized specifications allow specifications writers to modu-larize behavior. Given that a monitor represents a specific specification, this same monitor can be re-used to definemore complex specifications at a higher level, without consideration for the details needed for this specification.This allows specification writers to reason at various levels about the system specification.Let us consider the ADL specification cooking (resp. sleeping ) which specifies whether the tenant is cooking(resp. sleeping) in the apartment. One can reason about the meta-specification firehazard using both cooking and sleeping specifications without considering the lower level sensors that determine these specifications, that is: firehazard def = (cid:3) ( sleeping = ⇒ ¬ cooking ) . While we can define cooking as: cooking def = kitchen presence ∧ ♦ ≤ ( kitchen cooktop ∨ kitchen oven ) . Additionally, any specification that requires either sleeping or cooking specifications can re-use the verdict out-putted by their respective monitors. For example, specifications actfloor (0) and actfloor (1) require the verdictsfrom monitors associated with cooking and sleeping , respectively, since cooking happens on the ground floorwhile sleeping on the first floor. Furthermore, we can disjoin actfloor (0) and actfloor (1) to easily specify thatthere is some activity in the house, acthouse def = actfloor (0) ∨ actfloor (1) . While specification acthouse can be seen as a quantified version of actfloor ( i ) , we can use modular specifications for behavior, for exam-ple we can verify the triggering of an alarm in the house within 5 timestamps of detecting a fire hazard, i.e. checkalert def = firehazard = ⇒ ♦ ≤ ( firealert ) .In addition to providing a higher level of abstraction and reasoning about specifications, the modular structureof the specifications present three additional advantages.1. The first is that sub-specifications can change without affecting the meta-specifications, that is if the sub-specification cooking is changed (possibly to account for different sensors), no changes need to be propagatedto specifications firehazard , actfloor (0) , acthouse , and checkalert .2. The second advantage is controlling duplication of computation and communication, as such sensors do not haveto send their observations constantly to all monitors that verify the various specifications. Specification cooking requires knowledge from the kitchen presence sensor, the kitchen cooktop (being enabled) and the kitchen oven.Without any re-use these three sensors (presence, cooktop, and oven) need to send their information to monitorschecking: firehazard , actfloor (0) , acthouse , and checkalert . Antoine El-Hokayem and Yli`es Falcone
3. The third advantage is a consequence of modeling explicitly the dependencies between specifications. Thisallows the monitoring to take advantage of such dependencies and place the monitors that depend on each othercloser depending on the hierarchy, either geographically (i.e., in the same room or floor) or logically (i.e., closeto the monitors of the dependent sub-specifications). Furthermore, knowing the explicit dependencies betweenspecifications allows the user to choose a placement for their monitors, adjusting the placement to the systemarchitecture. In the case a placement is not possible, it is possible to create intermediate specifications thatsimply relay verdicts of other monitors, to transitively connect all components that are not connected.
Abstraction from implementation.
One setback for learning-based techniques to detect ADL is their specificityto the environment. That is, the training set is specific to a house layout, user profile (i.e., elderly versus adults) [39].Decentralized specifications define modular specifications that can be composed together to form bigger andmore complex specifications. By using references to monitors, we leave the implementation of the specificationto be specific for the house or user profile. Using our existing example, cooking is implemented based on theavailable sensors in the house, which would change for different houses. However, the meta-specifications such as firehazard can be defined independently from the implementation of both cooking and sleeping .Furthermore, using monitor references, which are treated as oracles, opens the door to utilizing existing tech-niques in the literature based on other formalisms (not based on automata). That is, as a reference is expected toeventually evaluate to (cid:62) or ⊥ , any decision procedure can be incorporated to form more complex specifications.For example, one can use the various machine learning techniques [16,39,49] to define monitors that detect specificADLs, then reference them in order to define more complex specifications. Scalability.
Decentralized specifications allow for a higher level of scalability when writing specifications, and alsowhen monitoring. By using decentralized specifications, we restrict the atomic propositions of monitors to (i) thelocal atomic propositions of the components they are attached to and (ii) references to other monitors (see Sect. 2.2).This greatly reduces the number of atomic propositions to consider when synthesizing the monitor and reduces itssize, as the sub-specifications are offloaded to another monitor.For example, let us consider writing specifications using LTL formulae. The classical algorithm that convertsLTL to Moore automata is doubly exponential in the size of the formula counted in terms of atomic propositions (toform events) [15]. Therefore, reducing both the size of the formula and the number of atomic propositions used inthe formula helps significantly when synthesizing the monitors, allowing us to scale beyond the limits of existingtools. For a large formula, and the larger formulas considered in this paper, it becomes impossible to generate acentral monitor using the existing synthesis techniques. Decentralized specifications provide a way to manage thelarge formula by subdividing it into subformulas. The decomposition ensures that the formula evaluates to the sameverdict given the same observations, at the cost of added delay.
Example 5 (Synthesizing the check light monitor).
Recall the system property sc light ( i ) in Ex. 2 responsible forverifying that in a room i a light switch does indeed turn a light bulb on until it is turned off. We recall the LTLspecification sc light ( i ) def = (cid:3) (s i = ⇒ (cid:13) ( (cid:96) i U ¬ s i )) . To verify the property across n rooms of the house, weformulate a property sc ok def = (cid:86) i ∈ [0 ..n ] sc light ( i ) . In the case of a decentralized specification the formula willreference each monitor in each room, leading to a conjunction of at n atomic propositions. However, in the case of acentralized specification, the specification needs to be written as: sc ok cent def = (cid:86) i ∈ [0 ...n ] (cid:3) (s i = ⇒ (cid:13) ( (cid:96) i U ¬ s i )) ,which is significantly more complex as a formula consisting of n operators (to cover the sub-specification), along n conjunctions, and defined over each sensor and light bulb atomic propositions ( n ). Given that monitor synthesisis doubly exponential, both ltl2mon [15] and lamaconv [38] require significant resources and time to generatethe minimal Moore automaton (in our case , both tools where unable to generate the monitor for n = 3 after an hourto timeout). THEMIS
To perform monitoring we use
THEMIS [27] which is a tool for defining, handling, and benchmarking decentralizedspecifications and their monitoring algorithms. For replaying the trace, we perform monitoring by defining a starttime, an end time and a polling interval. For this case study, for a given date, we use 07:30 as start time, 17:30 as anend time, and a 1-second polling interval.We first overview
THEMIS in Sect. 3.1. Then, in Sect. 3.2, we elaborate on the trace format provided in thepublic dataset, and our adaptation for replay to perform the monitoring. In brief, the process consists of extractingeach sensor data converting it to observations (atomic propositions and verdicts), and passing the observation to a On an Intel(R) Core(TM) i7-6700HQ CPU, using 16GB RAM, and running openjdk 1.8.0 172, with ltl2mon 0.0.7.ringing Runtime Verification Home 9 logical component for multiple related sensors. Later in Sect. 4, we introduce extra considerations when monitoringlarge traces.
THEMIS
Overview.
THEMIS [27] is a tool to facilitate the design, development, and analysis of decentralized monitoringalgorithms; developed using Java and AspectJ. It consists of a library and command-line tools.
THEMIS provides anAPI, data structures, and measures for decentralized monitoring. These building blocks can be reused or extendedto modify existing algorithms, design new algorithms, and elaborate new approaches to assess existing algorithms.
THEMIS encompasses existing approaches [11,19] that focus on presenting one global formula of the system fromwhich they derive multiple specifications, and in addition supports any decentralized specification [29].
Monitoring.
THEMIS defines two phases for a monitoring algorithm: setup and monitor. In the first phase, thealgorithm creates and initializes the monitors, connects them to each other so they can communicate, and attachesthem to components so they receive the observations generated by components. In the second phase, each monitorreceives observations at a timestamp based on the component it is attached to. The monitor can then perform somecomputation, communicate with other monitors, abort monitoring or report a verdict. The two distinct phases sepa-rate the monitor generation (monitor synthesis) problem from the monitoring [26], giving algorithms the freedom togenerate monitors and deploy them on components, while integrating with existing tools for monitor synthesis suchas [15,38]. The monitors used in this case study use similar logic than choreography [19], as they are defined overa shared global clock. All monitors start monitoring at t = 0 . A monitor checks the compliance of the specificationfor a given timestamp t , which could take a fixed delay d to check. After reaching the delay at t + d , the monitorreports the verdict for t to all other monitors that depend on it, and starts monitoring the specification again for t + 1 (i.e., it respawns ). As such, the communication between monitors consists of sending verdicts for given timestamps. The trace from [22] is given as a database with a table for each sensor. We extract each tableas a csv file for each sensor. The provided sensor data is stored as entries of values associated with timestamps,representing the changes in the sensor data across time. Typically, a new entry is provided whenever a change in thesensor data occurs. The provided data range over Boolean-like, integer, or real domains.
Generating atomic propositions.
The sensor data needs to be processed to create observations, as LTL3 monitors(see Sect. 2.1) operate on atomic propositions. Each sensor is implemented as an input (
Periphery in THEMIS ) toa logical component. For example, for the shower water, we use both cold and hot water sensors but define onlya single component (“shower water”), from an RV perspective, “hot” and “cold” are multiple observations passedto the “shower water” component. To process different sensor data, we implemented two peripheries:
SensorBool and
SensorThresh . The first periphery parses Boolean values from the csv file associated with timestamps. Theprocessing assigns Boolean values (cid:62) (resp. ⊥ ) based on sensor data such as: ”ON” (resp. ”OFF”), and ”OPEN”(resp. ”CLOSED”). The second periphery reads real (double) values, and returns a Boolean based on whether thenumber is below or above a certain threshold. Both peripheries associate each atomic proposition with the generatedBoolean to generate an observation. Synchronizing traces.
The provided dataset only provides sensor updates, that is, the data only contains timestampsand values for a sensor when the value changes. Our monitoring strategy, however, requires polling the devices atgiven fixed time intervals. Since the system has a global clock, to synchronize observations, our periphery imple-mentations synchronize on a date at the start and an increase (in our case 1 second) and a default Boolean valuefor the observation. When polled, the periphery returns the default value if nothing is observed yet, or the last valueobserved otherwise. The last value observed is updated when changes occur in the csv file. In short, we interpolatevalues between changes to return the oldest value before a change.
Determining the polling rate.
We leverage the global clock of the system to evaluate the specification syn-chronously for all components. As such, we need a fixed interval to poll the monitors in order to evaluate thespecification, that is, we take the necessary transition in each of the automata. We refer to this interval as the pollingrate . The polling rate determines the frequency of evaluation of the specification; the higher the rate, the morerounds, and the more monitors process and communicate. To determine the minimal rate, we consider the rate ofchange for all sensors involved in the specification. We are interested in ensuring that no sensor changes twice inbetween the evaluation of the specification. To do so, we write a simple program that processes the trace files foreach sensor in an input specification, to determine the rate of change. Listing 1 shows an example output on the27 sensors used for ADL detection. It shows the atomic proposition associated with the sensor, the sensor type, thetrace file, the fastest change rate (min), and the slowest change rate (max), and whether or not it is skipped. The rates
Listing 1
Rates of change for sensor data. The highlighted sensors are skipped since their data never change. livingroom_table SensorBool 28.csv Min: 3000 Max: 230704000 (ms) [OK] kitchen_dishwasher SensorThresh 167.csv Min: 2190810000 Max: 2190810000 (ms) [SKIP] office_deskplug SensorThresh 119.csv Min: 6000 Max: 231159000 (ms) [OK] office_tv SensorBool 283.csv Min: 420000 Max: 343980000 (ms) [OK] livingroom_couch SensorBool 45.csv Min: 3000 Max: 247031000 (ms) [OK] kitchen_presence SensorBool 269.csv Min: 2000 Max: 230702000 (ms) [OK] kitchen_c1 SensorBool 300.csv Min: 1000 Max: 259080000 (ms) [OK] kitchen_c2 SensorBool 315.csv Min: 1000 Max: 431493000 (ms) [OK] kitchen_c3 SensorBool 316.csv Min: 1000 Max: 259095000 (ms) [OK] kitchen_c4 SensorBool 317.csv Min: 1000 Max: 259051000 (ms) [OK] kitchen_c5 SensorBool 355.csv Min: 1000 Max: 779361000 (ms) [OK] kitchen_sink_hotwater SensorThresh 184.csv Min: 12000 Max: 260085000 (ms) [OK] kitchen_sink_coldwater SensorThresh 189.csv Min: 12000 Max: 260501000 (ms) [OK] bedroom_closet_door SensorBool 339.csv Min: 7000 Max: 605093000 (ms) [OK] bedroom_luminosity SensorThresh 120.csv Min: 1000 Max: 254250000 (ms) [OK] kitchen_cooktop SensorThresh 36.csv Min: 7000 Max: 260333000 (ms) [OK] bathroom_shower_coldwater SensorThresh 22.csv Min: 12000 Max: 345139000 (ms) [OK] bathroom_shower_hotwater SensorThresh 201.csv Min: 12000 Max: 345066000 (ms) [OK] kitchen_fridge_door SensorBool 314.csv Min: 1000 Max: 260749000 (ms) [OK] livingroom_tv SensorBool 282.csv Min: 840000 Max: 344040000 (ms) [OK] toilet SensorThresh 254.csv Min: 12000 Max: 518222000 (ms) [OK] bathroom_sink_coldwater SensorThresh 86.csv Min: 12000 Max: 260437000 (ms) [OK] bathroom_sink_hotwater SensorThresh 264.csv Min: 25000 Max: 25000 (ms) [SKIP] kitchen_oven SensorThresh 232.csv Min: 2191235000 Max: 2191235000 (ms) [SKIP] bedroom_drawer_1 SensorBool 357.csv Min: 1000 Max: 345825000 (ms) [OK] bedroom_drawer_2 SensorBool 358.csv Min: 2000 Max: 515617000 (ms) [OK] bedroom_bed_pressure SensorThresh 349.csv Min: 1000 Max: 342361000 (ms) [OK] (Detected Rate) Min: 1000 Max: 779361000 (ms) are provided in milliseconds. Then, we aggregate over all sensors by computing the fastest and slowest. Sensors arenot included in the aggregate computation (i.e., skipped) if no change appears in their entire trace file. In this case,we choose 1 second as our polling rate, as no sensor will change twice within a second. Managing the trace length (36,000) is an issue for the monitoring techniques presented in [26]. Since the associatedmonitors rely on eventual consistency [47], in some cases, they wait for input for the length of the trace, whichrequires a lot of memory. This was not an issue for the small traces (of length 100) used to compare algorithmsoriginally, but becomes a significantly larger issue when monitoring a real apartment.Two data structures are introduced in [26] to support monitoring decentralized specifications: memory and ex-ecution history encoding ( EHE ). We briefly review them in Sect. 4.1 along with their key operations so we can wepresent a garbage collection strategy for the memory data structure in Sect. 4.2 and an expansion strategy for the
EHE in Sect. 4.3. The memory footprint for monitors consists of the sizes of their memory and
EHE . Both our im-provements aim at reducing the size of the data structures for long traces. Theoretical details for the data structuresand monitoring are in [26].
The data structures memory and
EHE operate over atoms , where an atom is an encoding of atomic propositions. Theencoding used for monitoring the appartment consists of a pair of timestamp and atomic proposition. For example,the atom (cid:104) , s (cid:105) , is used to refer to the truth value of switch at timestamp . Memory.
The memory buffers all observations the monitor received from the component it is associated with,and the monitors it depends on. The memory is a partial function (noted M ) that associates atoms with verdicts.For example, the memory M = [ (cid:104) , s (cid:105) (cid:55)→ (cid:62) , (cid:104) , s (cid:105) (cid:55)→ ⊥ ] states that at timestamp 23, switch 1 was enabledwhile switch 2 was disabled. An underlying operation used to perform monitoring is denoted by eval , which takesa Boolean expression of atoms, and a memory. Function eval attempts to rewrite the expression by replacing thevalue of the atoms present in the memory by their associated verdict, then simplifies the expression (using Booleansimplification). The memory stores all observations and is used to rewrite expressions when performing monitoring. Example 6 ( eval ). For the expression e = (cid:104) , s (cid:105) ∨ (cid:104) , (cid:96) (cid:105) and memory M = [ (cid:104) , s (cid:105) (cid:55)→ (cid:62) ] , applying eval( e, M ) will first rewrite e to (cid:62) ∨ (cid:104) , (cid:96) (cid:105) , which is then simplified to (cid:62) . ringing Runtime Verification Home 11 Execution History Encoding.
We recall from Sect. 2.1 that monitors are Moore automata that check decentralizedtraces. Since we are dealing with partial information due to the decentralized nature of monitors, the
EHE encodes theexecution of the underlying automaton, keeping track of potential states when receiving partial observations. In brief,an
EHE can be modeled as a partial function ( I ) that associates a timestamp t and a state q of the automaton with aboolean expression e . Whenever e holds (i.e., I ( t, e ) ), we are sure that the automaton is in state q at timestamp t . TheBoolean expression e is evaluated using the content of the monitor’s memory data structure using eval . The size ofthe EHE grows to account for timestamps and potential reachable states as the system executes (as time passes). Themain function that extends the
EHE to new timestamps is mov . Function mov takes the current
EHE , along with itslast stored timestamp, and an arbitrary timestamp in the future, and expands the entries by generating the expressionsup to the future timestamp using the structure of the automaton and reachability. As such, to create an
EHE I (cid:48) fromanother one I containing current information at timestamp t cur with information up to timestamp t future , we use I (cid:48) = mov( I , t cur , t future ) . Expanding the EHE when information is missing leads to large expressions in the
EHE which require a larger memory to store and a longer time to simplify. As such, it is important to ensure that mov iscalled when sufficient information is present to resolve the
EHE . We optimized data structure memory (which is used to store observations) to add garbage collection. To do so wehave created a new implementation (
MemoryIndexed ) that indexes observations by timestamp. When the monitorconcludes with a final verdict for timestamp t , and respawns to monitor timestamp t + 1 , all observations associatedwith a timestamp lesser than or equal to t are removed from the memory. That is, the new memory M (cid:48) is constrainedto dom( M (cid:48) ) = dom( M ) \ {(cid:104) t (cid:48)(cid:48) , ap (cid:105) ∈ dom( M ) | t (cid:48)(cid:48) < t } (where dom indicates the domain of the partialfunction). This ensures that older information is discarded as the monitoring moves with time. EHE
Expansion
The
EHE data structure is designed to be as general as possible, and keeps expanding while it has not detected thestate the automaton is in. For large trace sizes, this can cause an
EHE to grow quickly to consume all availablememory and prevents monitoring from completion. That is, the monitor expands the
EHE using mov , causing theexpressions to grow exponentially [29], when no information is provided to the monitor.This is prominently the case when monitoring safety properties. Safety properties such as p def = (cid:3) ( ap ) will onlyconclude when the value of ap is ⊥ . So long as the value of ap is (cid:62) , the monitor checking p does not reach afinal verdict, and does not report it to its parent. Consequently, a monitor that checks a safety property that is neverviolated, incurs a delay that is as long as the trace size. One approach is to limit the expansion of the EHE to a fixedlength (assuming a fixed maximal delay), and use a sliding window to maintain the limit. This approach, however,may cause monitoring not to conclude in cases where monitoring requires more time than that of the window. Tosolve this issue and provide the user with more control, we allow the user to specify the expansion condition for the
EHE as an additional Boolean formula that is determined by communication. This allows us to expanding the
EHE based on the communication patterns between monitors.
Scope.
We recall from Sect. 2.2 that, for a given monitor labeled lbl , its alphabet AP lbl consists of atomic proposi-tions of dependent monitors and the alphabet of the attached component. For this enhancement, we consider monitorswhich only depend on other monitors, i.e., when AP lbl ⊆ AP mons . We can see, when looking at dependencies inFig. 4, that most monitors eventually rely only on lower-level monitors which themselves rely on component obser-vations. As such, most high-level specifications for the smart home, and in particular safety properties (formulated asmeta-specifications in Table 1), rely on other monitors which evaluate different specifications, and thus only dependon monitors. Communication AP.
For a monitor that only depends on other monitors, its alphabet consists of monitor references(i.e., AP lbl ⊆ AP mons ). For each dependent monitor (labeled dep ), we create two atomic propositions, one if thereceived verdict is (cid:62) (noted (cid:62) dep ) and one if it is ⊥ (noted ⊥ dep ). The resulting alphabet is AP comlbl = {(cid:62) dep , ⊥ dep | dep ∈ AP lbl } . The expansion condition (noted ϕ triggerlbl ) is thus a Boolean expression over the alphabet AP comlbl . Evaluating the expansion condition.
To evaluate the added atomic propositions, we define function resolve whichtakes as input an expansion condition ϕ triggerlbl , a memory M , and a timestamp t as follows: resolve( ϕ triggerlbl , M , t ) = match ϕ triggerlbl with | (cid:62) dep ∈ AP comlbl → eval( (cid:104) t, dep (cid:105) , M ) = (cid:62)| ⊥ dep ∈ AP comlbl → eval( (cid:104) t, dep (cid:105) , M ) = ⊥ Function resolve performs pattern matching to convert the communication atomic proposition to an expressioncapable of being evaluated using eval , checking if the monitor returned verdict (cid:62) and ⊥ at timestamp t for (cid:62) dep and ⊥ dep , respectively. We note that when the atom is not found in the memory, both (cid:62) dep and ⊥ dep do not hold. Triggering the expansion.
Given a current time t cur for which we last expanded the EHE , we determine themaximum possible expansion for the
EHE by looking for the atom in the memory with the highest timestamp,noted t max . Next, we define function resolved , which takes as input an expansion condition, a memory, a currenttimestamp and a maximum timestamp and generates the timestamps for which the EHE must be expanded. resolved( ϕ trigger , M , t cur , t max ) = { t cur < t ≤ t max | resolve( ϕ trigger , M , t ) = (cid:62)} Finally, we pick the maximum of the timestamps and expand the
EHE accordingly.
Remark 1 (Wildcard Trigger.).
It is common to observe a expansion condition that involves, for a given monitor(labeled lbl ), all the atoms found in the checked specification. The expansion condition is then a disjunction of allatoms (i.e., (cid:87) ap ∈ AP comlbl (ap) ). To avoid evaluating such large expression, particularly when many dependencies exist(for example, meta-specifications actfloor ( ) and actfloor ( ) ), we provide an optimization flag for a monitorto only trigger expansion upon receiving messages from other monitors. Example 7 (Combination of safety properties and expansion).
Consider the three monitors m , m and m thatcheck for the following specifications: (cid:3) ( ¬ firehazard ) , (cid:3) ( ¬ notwopeople ) and m ∧ m . We can see that in thiscase m and m only output verdicts when the property is falsified. That is, monitor m which depends on both,has to normally expand its own EHE as time passes awaiting information that will only become available when thespecification of either is falsified (i.e., firehazard or notwopeople evaluate to true in either monitors m or m ,respectively). As such, we can specify the expansion condition for monitor m to be ⊥ m ∨ ⊥ m : so long as no ⊥ is communicated from either m or m , the EHE is not expanded, as it cannot be falsified. When using lazy
EHE expansion using m , m and m with the unbounded operator (cid:3) , we are able to reduce the maximal observed sizeof an EHE during the simulation by as much as 58%.
Monitoring the smart apartment requires leveraging the interdependencies between specifications to be able to scale,beyond monitoring system properties, to more complex meta-specifications (as detailed in Sect. 1.2). We assess usingdecentralized specifications to monitor the apartment by conducting three scenarios. The first scenario (Sect. 5.1)evaluates the scalability and re-use advantages of using decentralized specifications presented in Sect. 2.3 by lookingat the complexity of monitor synthesis, and communication and computation costs when adding more complexspecifications that re-use sub-specifications. The second scenario (Sect. 5.2) evaluates the effectiveness of detectingADL by looking at various detection measures such as precision and recall. The third scenario (Sect. 5.3) portraysthe advantages of modularity by (i) adapting specification napping to use different sensors without modifyingdependencies, and (ii) porting specification firehazard to a completely different environment (using the ARASdataset [2]).
Table 1 displays the number of atomic propositions referenced by each specification for thedecentralized ( | AP d | ) and the centralized ( | AP c | ) settings. Column d indicates the maximum depth of the directedacyclic graph of dependencies. We use the depth to assess how many levels of sub-specifications need to be com-puted. When d = 0 , it indicates that the specification can be evaluated directly by the monitor placed on thecomponent, while d = 1 indicates that the monitor has to poll at most 1 monitor for its verdict (which typicallyrelays the component observations). More generally, when d = n , it indicates that the specification depends on amonitor that has at most depth n − . The atomic propositions indicate either direct references to sensor observations(in the centralized setting) or references to either sensor observations or dependent monitors (in the decentralizedsetting). For certain specifications such as toilet which relies only on the water sensor in the toilet to be detected,there is no difference between using a centralized or decentralized specification, as it resolves to the observations.Reduction becomes more pronounced when specifications re-use other specifications as sub-specifications. For ex-ample, specification acthouse def = actfloor (0) ∨ actfloor (1) , when decentralized, uses only 2 references (foreach of the sub-specifications). However, when expanded, it references all 27 sensors used to detect activities. Ad-ditionally, specification notwopeople def = ¬ ( actfloor (0) ∧ actfloor (1)) would not re-use the sub-specificationsif expanded, requiring all sensors again. Henceforth, re-use greatly reduces the formula size and allows us to syn-thesize the monitors needed to check the formulas, as the synthesis algorithm is doubly exponential as mentioned inSect. 2.3. ringing Runtime Verification Home 13 Properties M s g s ( N o r m a li z e d ) (a) Communication. Properties S i m p li f i ca t i on s ( N o r m a li z e d ) (b) Computation. Fig. 5: Scalability of communication and computations in decentralized specifications.
Assessing re-use and scalability.
Reducing the size of the atomic propositions needed for a specification not onlyaffects monitor synthesis, but also runtime performance, as atomic propositions represent the information neededto determine the specification (Sect. 2.3). To assess re-use and scalability, we perform two tasks and gather twomeasures pertaining to computation and communication, and present results in Fig. 5. The first task compares acentralized (SW-C) and a decentralized (SW-D) version of specification sc ok presented in Example 5 using only 2rooms. The second task introduces large meta-specifications on top of the ADL specifications to check scalability.Firstly, we measure the communication and computation for monitoring ADL specifications (ADL). Secondly, weintroduce specifications actfloor (0) , actfloor (1) and acthouse (ADL+H) as they require information about allsensors for ADL. Thirdly, we add specification notwopeople (ADL+H+2), as it re-uses the same sub-specificationsas specification acthouse . Lastly, we show all measures for all meta-specifications in Table 1 (ADL+M). We re-usetwo measures from [26]: the total number of simplifications the monitors are doing, and the total number of messagestransferred. These measures are provided directly with THEMIS [27]. The total number of messages abstracts thecommunication ( ), as our messages are of fixed length, they also represent the total data transferred. The totalnumber of simplifications ( ) abstracts the computation done by the monitors, as they attempt tosimplify Boolean expressions that represent automaton states, which are the basic operations for maintaining themonitoring data structures in [26]. Both measures are normalized by the number of timestamps in the execution(36,000). The resulting normalized measures represent the number of simplifications and messages per round.
Results.
Figure 5a shows the normalized number of messages sent by all monitors. For the first task, we noticethat the number of messages is indeed lower in the decentralized setting, SW-D sends on average 2 messages pertimestamp less than SW-C, which corresponds to the difference in the number of atomic propositions referenced (6for SW-D and 8 for SW-C). For the second task, we notice that on the baseline for ADL, we observe 24 messagesper timestamp, a smaller number than the sensors count (27). This is because some ADL like toilet are directlyevaluated on the sensor without communicating, and other ADL like preparing , re-use other ADL specificationslike kactivity . By introducing the 3 meta-specifications stating that an activity occurred on a floor or globally inthe appartment, the number of messages per round only increases by 15. This also coincides with the number ofatomic propositions for the specifications (6 for actfloor (0) , 7 for actfloor (1) , and 2 for acthouse ) as thosemonitors depend in total on 15 other monitors to relay their verdicts. This costs much less than polling 16 sensors todetermine actfloor (0) , 11 sensors to determine actfloor (1) , and 27 (a total of 54) to determine acthouse . Toverify this, we notice that the addition of notwopeople (ADL+H+2) that needs information from all 27 sensors, onlyincreases the total number of messages per timestamp by 2. The specification notwopeople reuses the verdicts ofthe two monitors associated with each actfloor specification. After adding all the meta-specifications (ADL+M),
Table 2: Precision, Recall, and F1 scores of monitoring all ADL specifications on three days with different schedules.
Tuesday, Jan 31 2017 Monday, Feb 20 2017 Tuesday, Feb 21 2017
Specification Precision Recall F1 Precision Recall F1 Precision Recall F1 computing office tv cooking shower usage washing dishes livingroom tv dressing toilet ∗ sink usage eating napping preparing reading the total number of messages per timestamp is 46, whihc is less than the number needed to verify adding actfloor ,and acthouse in a centralized setting (54). We notice a similar effect for computation (Fig. 5b). Table 2 displays the effectiveness of using RV to detect all ADL specifications on the trace ofthree days with different schedules. To assess the effectiveness, we compared with the provided self-annotated datafrom [22], where the user annotated the start and end of each activity. We measure precision, recall and F1 (thegeometric mean of precision and recall). To measure precision, we consider a true positive when the verdict (cid:62) ofa monitor for a given timestamp fell indeed in the self-annotated interval for the activity. To measure recall, wemeasure the proportion of the intervals for which the monitors have determined (cid:62) (using RV). This approach ismore fine-grained than the approach used in [41] where the precision and recall are computed for the start and endof intervals.
Results.
The effectiveness of detection depends highly on the specification. Our approach performs well for thespecifications computing , cooking , office tv , as it exhibits high precision and high recall. The second groupof specifications contains specifications such as shower usage , and livingroom tv . It exhibits high precision butmedium recall, that is, we were able to determine around 40 to 50% of all the timestamps where the specifica-tions held according to the person annotating, without any false positives. The third group is similar to the secondgroup but has very low recall (13-18%) and contains the specifications toilet and sink usage . We notice that for sink usage specific user behavior can throw it off, as seen for the trace of Feb 21, we elaborate on the limitationsin the next paragraph. The fourth group, which includes the specifications napping and preparing , shows highrecall but a high rate of false positives. And finally, specification reading is not properly detected, as it has a highrate of false positives and covers almost no annotated intervals. Limitations of RV for detecting ADL.
The limitations of using RV to detect ADL are due to the modeling. Asmentioned in Sect. 1.2, RV can be seen as a knowledge-based approach to activity detection, as such it suffers fromsimilar weaknesses and limitations [18]. The activity is described as a rigid formal specification over the sensor data,and this has two consequences. Firstly, since RV relies purely on sensor data, activities which cannot be inferredfrom existing sensors will be poorly detected or not detected at all. This is the case for reading , as there are nosensors to indicate that the tenant is reading. We infer reading by checking that the light is on in the room and noother specified activity holds. Secondly, given that specifications are rigid, we expect the user to behave exactlyas specified for the activity to be detected, any minor deviation results in the activity not being detected (as seenon Feb 21). To illustrate this point, the specification computing relies on the power consumption of the plug inthe office. Had the tenant been charging his phone instead of computing, the recall would have suffered greatly.Another great example of this is the shower usage specification, that is captured by inspecting the water usage ofthe shower. The time the tenant spends getting into the shower and out of the shower will not be considered, whichgreatly impacts recall. The above issues are further compounded by the annotation being carried out by a person. ringing Runtime Verification Home 15
Table 3: Modifying the decentralized specification to improve detection, and adapt to new environment. (a) Refining napping using the bedroom sensors: bed pressure( weight ), presence ( pres ), and light ( (cid:96) ). Formula Precision Recall F1 (cid:3) ≤ ( weight ) (cid:3) ≤ ( weight ) ♦ ≤ ( weight ) (cid:3) ≤ ( pres ∧ weight ) (cid:3) ≤ ( ¬ (cid:96) ∧ weight ) firehazard in ARAS. Specification Formula preparing ♦ ≤ ( m kdrawer ∨ m fridge ∨ m cupboard ) cooking preparingbeds bed1 ∨ bed2beds (cid:48) bed1 ∧ bed2napping (cid:3) ≤ ( beds ) firehazard napping = ⇒ ¬ cooking The annotator can for example take a few seconds to annotate some events which could impact recall, especiallyfor short intervals of activity. However, even with the inherent limitations of using knowledge-based approaches,our observed groups and results fall within the expected range, of knowledge-based approaches such as [41], andalso have similar effectiveness as model-based SVM approaches such as [17]. We elaborate on how the introducedmodularity from decentralized specifications can alleviate some of these issues in Sect. 5.3.
Decentralized specifications introduce numerous advantages (see Sect. 2.3) for monitoring hierarchical systems thatcan change. We illustrated in Sect. 5.1 the scalability of decentralized specifications with hierarchies. Decentral-ized specifications allows specifications to be written with references to other specifications. The references allowspecifications to be modular, changing the referenced specification is transparent with no modification to the speci-fications that depend on it. In this section, we illustrate the advantages of modularity in two cases. In the first case,we improve the detection of the activity napping by adding relevant sensors. The change only requires changingthe monitor for napping , and no change is necessary for the remaining dependent specifications. In the second case,we apply the specification firehazard and all its dependencies on a completely different environment using theARAS dataset [2].
Improving activity detection.
We modify the specification napping to better capture the activity. This requires nochange to specifications that depend on napping . Table 3a shows the changes in precision and recall, for variousversions of the specification napping . We modify the formula to relax the time constraints on the output of thebed pressure sensor. We notice, that while this could slightly improve recall (0.95 to 1), it does not translate toany precision improvement (it remains at 0.43). We explore using additional sensors in the room to capture thespecification better. Using the presence sensor proves to be detrimental as it reduces precision to 0.34 and recall to0.14. This is reasonable, as the presence sensor is a motion detector, and when someone is sleeping there may be nomotion at all. However, people typically tend to turn the lights off when sleeping. Using the additional light sensorto detect lights are off, helps us increase precision to 1 and recall to 0.99. One could see that the effect of ADLdetection is behavior specific, a tenant that sleeps with lights on will have undetected sleep using our specification.Being able to change to specific parts of the specification without impacting the rest of the it provides the flexibilityto tune the ADL detection to specific users and behaviors.
Adapting to new environments.
In Sect. 1.2 we mentioned that ADL can be challenging as the detection of thespecification does not only depend on the user behavior, but also on the environment in which it is monitored. In thecontext of learning techniques, using information learned from one environment to apply it to detection of ADL inother environments is discussed in [39]. Since decentralized specifications provide both a hierarchical and modularapproach to designing specifications, it is possible to adapt specifications to new environment, by only changing therelevant parts or dependencies, and reasoning at the appropriate level. For instance, while specifications specifyingADL may change depending on the sensors and user behavior, meta-specifications do not necessarily change. Weadapt specification firehazard and all its dependencies in the ARAS [2] dataset. The ARAS dataset featurescontact, pressure, distance, and light sensors, recording the interactions of two tenants with the sensors over a periodof 30 days.Table 3b shows the changes in the decentralized specification compared with that of Amiqual4Home found inAppendix A. For activity preparing , we follow a similar pattern, looking at the usage of cupboards, fridge, andkitchen drawers. Thus, we adapt the formula to reflect the available sensors in the kitchen. However, the ARAS dataset does not provide any electricity sensors for appliances, nor any way to detect heat being turned on. As suchit is impossible to detect cooking using any sensors. Since we cannot tell preparing and cooking apart, we define cooking to simply be equivalent to preparing . Notice how in this case, we inverted the dependency from Fig. 4(in ARAS, cooking depends on preparing ). The ARAS dataset records the behavior of two people, instead of just one . As such, activity napping needs to be adjusted for the two beds . There are two ways to do so, the first assumeseither one of the tenants is napping ( beds ), and the second assumes both are napping simultaneously ( beds (cid:48) ). Wenotice that the meta-specification firehazard remains unchanged. However, it has two different interpretations.If we use beds , then it is possible to trigger firehazard when one tenant is cooking while the other is sleeping.We verify that, and notice that it is indeed falsified in 8 days (7, 9, 16, 17-19, 24, 27). Using beds (cid:48) , allows us toonly capture firehazard when both tenants are sleeping. It is then possible to refer napping to allnapping and anynapping , then using firehazard on allnapping , which would apply in both scenarios. Discussion.
We see that modularity provides several advantages. It allows us to make local change to specificationsthat do not need to be propagated upwards. It also makes it possible to generalize and abstract the specificationto adapt to multiple environments. Decentralized specifications allow specifications to be written in a modular andadaptable fashion, allowing specifications to be adapted to target changes in user behavior and environment. It canbe seen much like component-based design [48], which separate the implementation of each component in software,from its interaction with other components.
We present similar or useful techniques for detecting ADL in a smart apartment that use log analysis and complexevent processing. Then, we present techniques from stream-based RV that can be extended for monitoring smartapartments.
ADL detection using log analysis.
Detecting ADL can be performed using trace analysis tools. The approachin [41] defines parametric events using Model Checking Language (MCL) [45] based on the modal mu-calculus(inspired by temporal logic and regular expressions). Traces are read and transformed into actions, then actionsare matched against the specifications to determine locations in the trace that match ADL. Five ADL (sleep, usingtoilets, cooking, showering, and washing dishes) are specified and checked in the same smart apartment as our work.While this technique is able to detect ADL activities, it amounts to checking traces offline, and a high level of post-processing is required to analyze the data. In [10], the authors describe an approach for log analysis at very largescale. The specification is expressed using Metric First Order Temporal Logic (MFOTL), and logs are expressed asa temporal structure. The authors develop a
MapReduce monitoring algorithm to analyze logs generated by morethan 35,000 computers, producing approximately 1 TB of log data each day. While this approach is designed fordistributed systems, does not map dependencies, and works offline, it could be used to process and monitor richspecifications over sensor data seen as log files.
ADL detection using Complex Event Processing.
Reasoning at a much higher level of abstraction than sensordata, the approach in [36] attempts to detect ADL by analyzing the electrical consumption in the household. To doso, it employs techniques from Complex Event Processing (CEP), in which data is fed as streams and processedusing various functions to finally output a stream of data. In this work, the ADL detection is split into two phases,one which detects peaks and plateaus of the various electrical devices, and the second phase uses those to indicatewhether or not an appliance is being used. This illustrates a transformation from low-level data (sensor signal)to a high-level abstraction (an appliance is being used). The use of CEP for detecting ADL is promising, as itallows for similar scalability and abstraction. However, CEP’s model of named streams makes it hard to analyze thespecification formally, making little distinction between specification and implementation of the monitoring logic.
ADL detection using Runtime Verification.
Similarly to CEP but focusing on Boolean verdicts, various stream-based RV techniques have been elaborated such as LOLA [23] which are used to verify correctness propertiesfor synchronous systems such as the PCI bus protocol and a memory controller. A more recent approach uses theTemporal Stream-Based Specification Language (TeSSLa) to verify embedded systems using FPGAs [24]. Stream-based RV is particularly fast and effective for verifying lengthy parametric traces. However, it is unclear how theseapproaches handle monitor synthesis for a large number of components and account for the hierarchy in the system.
Discussion.
Stream-based systems such as stream-based RV and CEP are bottom-up. Data in streams is eventuallyaggregated into more complex information and relayed to a higher level. Decentralized specifications also supporttop-down approaches, which would increase the efficiency of monitoring large and hierarchical systems. To illus-trate the point, consider the decentralized specification in Fig. 3b. In the automaton A sc light i , the evaluation of thedependent monitor A (cid:96) i only occurs when reaching q , so long as the automaton is in q , no interaction with the de- ringing Runtime Verification Home 17 pendent monitor is necessary. This top-down feedback can be used to naturally optimize dependencies and increaseefficiency. Because of the oracle-based implementation of decentralized specifications, it is possible to integrate anymonitoring reference that eventually returns a verdict. One could imagine integrating other stream-based monitorsor even data-driven ADL detection approaches. The integration works both ways, as monitors can be considered a(blocking) stream of verdicts for the other techniques. Monitoring a smart apartment presents RV with interesting new problems as it requires a scalable approach that iscompositional, dynamic, and able to handle a multitude of devices. This is due to the hierarchical structure imposedby either limited communication capabilities of devices across geographical areas or the dependencies between vari-ous specifications. Attempting to solve such problems with centralized specifications is met with several obstacles atthe level of monitor synthesis techniques (as we are presented with large formulae), and also at the level of monitor-ing as one needs to model interdependencies between formulae and re-use the sub-specifications used to build morecomplex specifications. We illustrate how decentralized specifications tackle such systems by explicitly modelingof interdependencies between specifications. Furthermore, we illustrate monitoring specifications that detect ADLin addition to system properties and even more specifications defined over both types of specifications.
We believe that the use of decentralized specifications could be further extended to bring monitoring closer todata (collected on sensors), and make RV a suitable verification technique for edge computing. One challengeof the case study was to determine the correct sampling period for monitor to operate. Further investigation isrequired to layout the tradeoffs between the sampling period, communication overhead, and energy consumption.Also, decentralization is only supported by specifications based on the standard (point-based) LTL3 semantics. Webelieve that the use and decentralization of richer specification languages are desirable. For instance, we consider (i)using a counting semantics able to compute the number of steps needed to witness the satisfaction or violation of aspecification [5] (ii) using techniques allowing to deal with uncertainty (e.g., in case of message loss) [9] (iii) usingspatio-temporal specifications (e.g. [35]) to reason on physical locations in the house, and (iv) using a quantitativesemantics possibly with time [4]. Finally, we consider using runtime enforcement [30,34,33] techniques (especiallythose for timed specifications [32]) to guarantee system properties and improve safety in the house (e.g., disablingcooking equipment whenever specification firehazard is violated). This requires to define the foundations fordecentralized runtime enforcement on the theoretical side, and provide houses and monitors with actuators on thepractical side.
References
1. Aimal, S., Parveez, K., Saba, A., Batool, S., Arshad, H., Javaid, N.: Energy optimization techniques for demand-side manage-ment in smart homes. In: Advances in Intelligent Networking and Collaborative Systems, The 9th International Conferenceon Intelligent Networking and Collaborative Systems, INCoS-2017. Lecture Notes on Data Engineering and Communica-tions Technologies, vol. 8, pp. 515–524. Springer (2017)2. Alemdar, H. ¨O., Ertan, H., Incel, ¨O.D., Ersoy, C.: ARAS human activity datasets in multiple homes with multiple residents.In: 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, PervasiveHealth2013. pp. 232–235. IEEE (2013)3. Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, Santa Barbara, CA,USA, July 10 - 14, 2017. ACM (2017)4. Bakhirkin, A., Ferr`ere, T., Maler, O., Ulus, D.: On the quantitative semantics of regular expressions over real-valued sig-nals. In: Abate, A., Geeraerts, G. (eds.) Formal Modeling and Analysis of Timed Systems - 15th International Conference,FORMATS 2017, Berlin, Germany, September 5-7, 2017, Proceedings. Lecture Notes in Computer Science, vol. 10419, pp.189–206. Springer (2017)5. Bartocci, E., Bloem, R., Nickovic, D., R¨ock, F.: A counting semantics for monitoring LTL specifications over finite traces.CoRR abs/1804.03237 (2018)6. Bartocci, E., Falcone, Y. (eds.): Lectures on Runtime Verification - Introductory and Advanced Topics, Lecture Notes inComputer Science, vol. 10457. Springer (2018)7. Bartocci, E., Falcone, Y., Bonakdarpour, B., Colombo, C., Decker, N., Havelund, K., Joshi, Y., Klaedtke, F., Milewicz, R.,Reger, G., Rosu, G., Signoles, J., Thoma, D., Zalinescu, E., Zhang, Y.: First international competition on runtime verification:rules, benchmarks, tools, and final results of crv 2014. International Journal on Software Tools for Technology Transfer (Apr2017)8. Bartocci, E., Falcone, Y., Francalanza, A., Reger, G.: Introduction to runtime verification. In: Bartocci, E., Falcone, Y. (eds.)Lectures on Runtime Verification - Introductory and Advanced Topics, Lecture Notes in Computer Science, vol. 10457, pp.1–33. Springer (2018), https://doi.org/10.1007/978-3-319-75632-5_1 gitlab.inria.fr/monitoring/themis-rv18smarthome
26. El-Hokayem, A., Falcone, Y.: Monitoring decentralized specifications. In: Antoine El-Hokayem and Yli`es Falcone [3], pp.125–13527. El-Hokayem, A., Falcone, Y.: THEMIS: a tool for decentralized monitoring algorithms. In: Antoine El-Hokayem and Yli`esFalcone [3], pp. 372–37528. El-Hokayem, A., Falcone, Y.: Bringing runtime verification home. In: Colombo, C., Leucker, M. (eds.) Runtime Verification -18th International Conference, RV 2018, Limassol, Cyprus, November 10-13, 2018, Proceedings. Lecture Notes in ComputerScience, vol. 11237, pp. 222–240. Springer (2018), https://doi.org/10.1007/978-3-030-03769-7
29. El-Hokayem, A., Falcone, Y.: On the monitoring of decentralized specifications. ACM Transactions on Software Engineeringand Methodology (TOSEM) (2019), to appear.30. Falcone, Y.: You should better enforce than verify. In: Barringer, H., Falcone, Y., Finkbeiner, B., Havelund, K., Lee, I., Pace,G.J., Rosu, G., Sokolsky, O., Tillmann, N. (eds.) Runtime Verification - First International Conference, RV 2010, St. Julians,Malta, November 1-4, 2010. Proceedings. Lecture Notes in Computer Science, vol. 6418, pp. 89–105. Springer (2010)31. Falcone, Y., Havelund, K., Reger, G.: A tutorial on runtime verification. In: Engineering Dependable Software Systems,NATO science for peace and security series, d: information and communication security, vol. 34, pp. 141–175. ios press(2013)32. Falcone, Y., J´eron, T., Marchand, H., Pinisetty, S.: Runtime enforcement of regular timed properties by suppressing anddelaying events. Sci. Comput. Program. 123, 2–41 (2016)33. Falcone, Y., Mariani, L., Rollet, A., Saha, S.: Runtime failure prevention and reaction. In: Bartocci and Falcone [6], pp.103–134ringing Runtime Verification Home 1934. Falcone, Y., Mounier, L., Fernandez, J., Richier, J.: Runtime enforcement monitors: composition, synthesis, and enforcementabilities. Formal Methods in System Design 38(3), 223–262 (2011)35. Haghighi, I., Jones, A., Kong, Z., Bartocci, E., Gros, R., Belta, C.: Spatel: A novel spatial-temporal logic and its applicationsto networked systems. In: Proceedings of the 18th International Conference on Hybrid Systems: Computation and Control.pp. 189–198. HSCC ’15, ACM, New York, NY, USA (2015)36. Hall´e, S., Gaboury, S., Bouchard, B.: Activity recognition through complex event processing: First findings. In: ArtificialIntelligence Applied to Assistive Technologies and Smart Environments, Papers from the 2016 AAAI Workshop. AAAIWorkshops, vol. WS-16-01. AAAI Press (2016)37. Havelund, K., Goldberg, A.: Verify your runs. In: Meyer, B., Woodcock, J. (eds.) Verified Software: Theories, Tools, Ex-periments, First IFIP TC 2/WG 2.3 Conference, VSTTE 2005, Zurich, Switzerland, October 10-13, 2005, Revised SelectedPapers and Discussions. Lecture Notes in Computer Science, vol. 4171, pp. 374–383. Springer (2005)38. Institute for Software Engineering and Programming Languages: LamaConv - Logics and Automata Converter Library,
39. van Kasteren, T., Englebienne, G., Kr¨ose, B.J.A.: Transferring knowledge of activity recognition across sensor networks. In:Pervasive Computing, 8th International Conference, Pervasive 2010. Proceedings. Lecture Notes in Computer Science, vol.6030, pp. 283–300. Springer (2010)40. Katz, S.: Assessing self-maintenance: Activities of daily living, mobility, and instrumental activities of daily living. Journalof the American Geriatrics Society 31(12), 721–727 (1983)41. Lago, P., Lang, F., Roncancio, C., Jim´enez-Guar´ın, C., Mateescu, R., Bonnefond, N.: The ContextAct@A4H real-life datasetof daily-living activities - activity recognition using model checking. In: Modeling and Using Context - 10th Internationaland Interdisciplinary Conference, CONTEXT 2017, Proceedings. Lecture Notes in Computer Science, vol. 10257, pp. 175–188. Springer (2017)42. Leucker, M., Schallhart, C.: A brief account of runtime verification. J. Log. Algebr. Program. 78(5), 293–303 (2009)43. Leucker, M., Schmitz, M., `a Tellinghusen, D.: Runtime verification for interconnected medical devices. In: LeveragingApplications of Formal Methods, Verification and Validation: Discussion, Dissemination, Applications - 7th InternationalSymposium, ISoLA 2016, Proceedings, Part II. Lecture Notes in Computer Science, vol. 9953, pp. 380–387 (2016)44. Majumder, S., Aghayi, E., Noferesti, M., Memarzadeh-Tehran, H., Mondal, T., Pang, Z., Deen, M.J.: Smart homes for elderlyhealthcare - recent advances and research challenges. Sensors 17(11), 2496 (2017)45. Mateescu, R., Thivolle, D.: A model checking language for concurrent value-passing systems. In: FM 2008: Formal Methods,15th International Symposium on Formal Methods, Proceedings. Lecture Notes in Computer Science, vol. 5014, pp. 148–164. Springer (2008)46. Pnueli, A.: The temporal logic of programs. In: 18th Annual Symposium on Foundations of Computer Science, Providence,Rhode Island, USA, 31 October - 1 November 1977. pp. 46–57. IEEE Computer Society (1977)47. Shapiro, M., Preguic¸a, N.M., Baquero, C., Zawirski, M.: Conflict-free replicated data types. In: D´efago, X., Petit, F., Villain,V. (eds.) Stabilization, Safety, and Security of Distributed Systems - 13th International Symposium, SSS 2011, Grenoble,France, October 10-12, 2011. Proceedings. Lecture Notes in Computer Science, vol. 6976, pp. 386–400. Springer (2011)48. Szyperski, C.A., Gruntz, D., Murer, S.: Component software - beyond object-oriented programming, 2nd Edition. Addison-Wesley component software series, Addison-Wesley (2002)49. Tapia, E.M., Intille, S.S., Larson, K.: Activity recognition in the home using simple and ubiquitous sensors. In: PervasiveComputing, Second International Conference, PERVASIVE 2004, Vienna, Austria, April 21-23, 2004, Proceedings. LectureNotes in Computer Science, vol. 3001, pp. 158–175. Springer (2004)50. Thapliyal, H., Nath, R.K., Mohanty, S.P.: Smart home environment for mild cognitive impairment population: Solutions toimprove care and quality of life. IEEE Consumer Electronics Magazine 7(1), 68–76 (2018)
A List of Properties
Table 4 shows all property definitions used in this case study. We ommitted the smaller monitors that are trivial suchas m kitchen cupboard which is a disjunction of all cupboard doors observations in the kitchen.
Table 4: Definitions of the specifications used in the case study. A specification with name prefixed with m is suchthat the corresponding monitor is directly deployed on the component. Name Formula sc light ( i ) (cid:3) (switch i = ⇒ (cid:13) (light i U ¬ switch i ) , i ∈ [0 .. sc ok (cid:86) i ∈ [0 .. sc light ( i ) m toilet toilet watersink usage (cid:3) ≤ ( m bathroom sink water ) m bathroom sink water bathroom sink cold ∨ bathroom sink hotshower usage (cid:3) ≤ ( m bathroom shower water ) napping (cid:3) ≤ ( m bedroom bed pressure ) dressing ♦ ≤ ( m bedroom closet door ∨ m bedroom drawers )) reading m bedroom light ∧ ♦ ≤ ( ¬ dressing ∧ ¬ napping ) office tv ♦ ≤ ( m office tv ) computing ♦ ≤ ( m office deskplug ) cooking ♦ ≤ ( m kitchen cooktop ∨ m kitchen oven ) washing dishes ♦ ≤ ( m kitchen dishwasher ∨ m kitchen sink water ) kactivity m kitchen presence ∧ ♦ ≤ ( m kitchen sink water ∨ m kitchen fridgedoor ∨ m kitchen cupboard ) preparing kitchen activity ∧ ¬ cookinglivingroom tv ♦ ≤ ( m livingroom tv ∧ m livingroom couch ) eating ¬ m kitchen presence ∧ (cid:3) ≤ ( m livingroom table ) actfloor (0) cooking ∨ preparing ∨ eating ∨ washing dishes ∨ livingroom tv ∨ m toiletactfloor (1) computing ∨ dressing ∨ napping ∨ office tv ∨ reading ∨ shower usage ∨ sink usageacthouse actfloor (0) ∨ actfloor (1) notwopeople ¬ ( actfloor (0) ∧ actfloor (1)) restricttv office office tv = ⇒ ♦ ≤ ( ¬ office tv ) restricttv living livingroom tv = ⇒ ♦ ≤ ( ¬ livingroom tv ) restricttv restricttv living ∧ restricttv officefirehazard napping = ⇒ ¬⇒ ¬