Joost-Pieter Katoen

Model-checking algorithms for continuous-time Markov chains

Continuous-time Markov chains (CTMCs) have been widely used to determine system performance and dependability characteristics. Their analysis most often concerns the computation of steady-state and transient-state probabilities. This paper introduces a branching temporal logic for expressing real-time probabilistic properties on CTMCs and presents approximate model checking algorithms for this logic. The logic, an extension of the continuous stochastic logic CSL of Aziz et al. (1995, 2000), contains a time-bounded until operator to express probabilistic timing properties over paths as well as an operator to express steady-state probabilities. We show that the model checking problem for this logic reduces to a system of linear equations (for unbounded until and the steady-state operator) and a Volterra integral equation system (for time-bounded until). We then show that the problem of model-checking time-bounded until properties can be reduced to the problem of computing transient state probabilities for CTMCs. This allows the verification of probabilistic timing properties by efficient techniques for transient analysis for CTMCs such as uniformization. Finally, we show that a variant of lumping equivalence (bisimulation), a well-known notion for aggregating CTMCs, preserves the validity of all formulas in the logic.

A probabilistic extension of UML statecharts: specification and verification

This paper is the extended technical report that corresponds to a published paper [14]. This paper introduces means to specify system randomness within UML statecharts, and to verify probabilistic temporal properties over such enhanced statecharts which we call probabilistic UML statecharts. To achieve this, we develop a general recipe to extend a statechart semantics with discrete probability distributions, resulting in Markov decision processes as semantic models. We apply this recipe to the requirements-level UML semantics of [8]. Properties of interest for probabilistic statecharts are expressed in PCTL, a probabilistic variant of CTL for processes that exhibit both non-determinism and probabilities. Verification is performed using the model checker Prism. A model checking example shows the feasibility of the suggested approach.

Model-Based Testing of Reactive Systems, Advanced Lectures

Testing of Finite State Machines.- I. Testing of Finite State Machines.- 1 Homing and Synchronizing Sequences.- 2 State Identification.- 3 State Verification.- 4 Conformance Testing.- II. Testing of Labeled Transition Systems.- Testing of Labeled Transition Systems.- 5 Preorder Relations.- 6 Test Generation Algorithms Based on Preorder Relations.- 7 I/O-automata Based Testing.- 8 Test Derivation from Timed Automata.- 9 Testing Theory for Probabilistic Systems.- III. Model-Based Test Case Generation.- Model-Based Test Case Generation.- 10 Methodological Issues in Model-Based Testing.- 11 Evaluating Coverage Based Testing.- 12 Technology of Test-Case Generation.- 13 Real-Time and Hybrid Systems Testing.- IV. Tools and Case Studies.- Tools and Case Studies.- 14 Tools for Test Case Generation.- 15 Case Studies.- V. Standardized Test Notation and Execution Architecture.- Standardized Test Notation and Execution Architecture.- 16 TTCN-3.- 17 UML 2.0 Testing Profile.- VI. Beyond Testing.- Beyond Testing.- 18 Run-Time Verification.- 19 Model Checking.- VII. Appendices.- Appendices.- 20 Model-Based Testing - A Glossary.- 21 Finite State Machines.- 22 Labelled Transition Systems.

Process algebra for performance evaluation

This paper surveys the theoretical developments in the field of stochastic process algebras, process algebras where action occurrences may be subject to a delay that is determined by a random variable. A huge class of resource-sharing systems - like large-scale computers, client-server architectures, networks - can accurately be described using such stochastic specification formalisms. The main emphasis of this paper is the treatment of operational semantics, notions of equivalence, and (sound and complete) axiomatisations of these equivalences for different types of Markovian process algebras, where delays are governed by exponential distributions. Starting from a simple actionless algebra for describing time-homogeneous continuous-time Markov chains, we consider the integration of actions and random delays both as a single entity (like in known Markovian process algebras like TIPP, PEPA and EMPA) and as separate entities (like in the timed process algebras timed CSP and TCCS). In total we consider four related calculi and investigate their relationship to existing Markovian process algebras. We also briefly indicate how one can profit from the separation of time and actions when incorporating more general, non-Markovian distributions.

A Markov reward model checker

This short tool paper introduces MRMC, a model checker for discrete-time and continuous-time Markov reward models. It supports reward extensions of PCTL and CSL, and allows for the automated verification of properties concerning long-run and instantaneous rewards as well as cumulative rewards. In particular it supports to check the reachability of a set of goal states (by only visiting legal states before) under a time and an accumulated reward constraint. Several numerical algorithms and extensions thereof are included in MRMC.

The ins and outs of the probabilistic model checker MRMC

The Markov Reward Model Checker (MRMC) is a software toolfor verifying properties over probabilistic models. It supports PCTL and CSL model checking, and their rewardextensions. Distinguishing features of MRMC are its support for computing time- and reward-bounded reachability probabilities, (property-driven) bisimulation minimization, and precise on-the-fly steady-state detection. Recent tool features include time-bounded reachability analysis for uniform CTMDPs and CSL model checking by discrete-event simulation. This paper presents the tools current status and its implementation details.

Approximate Symbolic Model Checking of Continuous-Time Markov Chains

of Invited Talk Research in the specification and verification of concurrent systems falls into two general categories. The temporal logic school advocates temporal logic as a language for formulating system requirements, with the semantics of the logic being used as a basis for determining whether or not a system is correct. The process-algebraic community focuses on the use of “higher-level” system descriptions as specifications of “lower-level” ones, with a refinement relation being used to determine whether an implementation conforms to a specification. From a user’s perspective, the approaches offer different benefits and drawbacks. Temporal logic supports “scenario-based” specifications, since formulas may be given that focus on single aspects of system behavior. On the other hand, temporal logic specifications suffer from a lack of compositionality, since the language of specifications differs from the system description language. In contrast, compositional specification is the hallmark of process algebraic reasoning, but at the expense of requiring what some view as overly detailed specifications. Although much research has studied the connections between the temporal logic and process algebra, a truly uniform formalism that combines the advantages of the two approaches has yet to emerge. In my talk I present preliminary results obtained by Gerald Lüttgen, of ICASE, and me on the development of such a formalism. Our approach features a process-algebra-inspired notation that enriches traditional process algebras by allowing linear-time temporal formulas to be embedded in system descriptions. We show how the combined formalism may be given a uniform operational semantics in Plotkin’s Structural Operational Semantics (SOS) style, and we define a refinement relation based on Denicola/Hennessy testing and discuss its congruence properties. We then demonstrate that traditional temporal-logic-style arguments about system correctness can be naturally captured via refinement; we also illustrate how the combination of logical and system operators allows users to define systems in which some “components” remain specified only as formulas. ? Research supported by NSF grants CCR-9257963, CCR-9505562 and CCR-9804091, AFOSR grant F49620-95-1-0508, and ARO grant P-38682-MA. Jos C.M. Baeten, Sjouke Mauw (Eds.): CONCUR’99, LNCS 1664, pp. 1–1, 1999. c

Comparative branching-time semantics for Markov chains

This paper presents various semantics in the branching-time spectrum of discrete-time and continuous-time Markov chains (DTMCs and CTMCs). Strong and weak bisimulation equivalence and simulation preorders are covered and are logically characterized in terms of the temporal logics Probabilistic Computation Tree Logic (PCTL) and Continuous Stochastic Logic (CSL). Apart from presenting various existing branching-time relations in a uniform manner, this paper presents the following new results: (i) strong simulation for CTMCs, (ii) weak simulation for CTMCs and DTMCs, (iii) logical characterizations thereof (including weak bisimulation for DTMCs), (iv) a relation between weak bisimulation and weak simulation equivalence, and (v) various connections between equivalences and pre-orders in the continuous-and discrete-time setting. The results are summarized in a branching-time spectrum for DTMCs and CTMCs elucidating their semantics as well as their relationship.

Model Checking Continuous-Time Markov Chains by Transient Analysis

The verification of continuous-time Markov chains (CTMCs) against continuous stochastic logic (CSL) [3,6], a stochastic branching-time temporal logic, is considered. CSL facilitates among others the specification of steady-state properties and the specification of probabilistic timing properties of the form \({\cal P}_{\bowtie p}(\Phi_1 \, {\cal U}^{I} \, \Phi_2)\), for state formulas Φ1 and Φ2, comparison operator ⋈, probability p, and real interval I. The main result of this paper is that model checking probabilistic timing properties can be reduced to the problem of computing transient state probabilities for CTMCs. This allows us to verify such properties by using efficient techniques for transient analysis of CTMCs such as uniformisation. A second result is that a variant of ordinary lumping equivalence (i.e., bisimulation), a well-known notion for aggregating CTMCs, preserves the validity of all CSL-formulas.

Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes

A continuous-time Markov decision process (CTMDP) is a generalization of a continuous-time Markov chain in which both probabilistic and nondeterministic choices co-exist. This paper presents an efficient algorithm to compute the maximum (or minimum) probability to reach a set of goal states within a given time bound in a uniform CTMDP, i.e., a CTMDP in which the delay time distribution per state visit is the same for all states. It furthermore proves that these probabilities coincide for (time-abstract) history-dependent and Markovian schedulers that resolve nondeterminism either deterministically or in a randomized way.