Holger Hermanns

Model-checking algorithms for continuous-time Markov chains

Continuous-time Markov chains (CTMCs) have been widely used to determine system performance and dependability characteristics. Their analysis most often concerns the computation of steady-state and transient-state probabilities. This paper introduces a branching temporal logic for expressing real-time probabilistic properties on CTMCs and presents approximate model checking algorithms for this logic. The logic, an extension of the continuous stochastic logic CSL of Aziz et al. (1995, 2000), contains a time-bounded until operator to express probabilistic timing properties over paths as well as an operator to express steady-state probabilities. We show that the model checking problem for this logic reduces to a system of linear equations (for unbounded until and the steady-state operator) and a Volterra integral equation system (for time-bounded until). We then show that the problem of model-checking time-bounded until properties can be reduced to the problem of computing transient state probabilities for CTMCs. This allows the verification of probabilistic timing properties by efficient techniques for transient analysis for CTMCs such as uniformization. Finally, we show that a variant of lumping equivalence (bisimulation), a well-known notion for aggregating CTMCs, preserves the validity of all formulas in the logic.

A probabilistic extension of UML statecharts: specification and verification

This paper is the extended technical report that corresponds to a published paper [14]. This paper introduces means to specify system randomness within UML statecharts, and to verify probabilistic temporal properties over such enhanced statecharts which we call probabilistic UML statecharts. To achieve this, we develop a general recipe to extend a statechart semantics with discrete probability distributions, resulting in Markov decision processes as semantic models. We apply this recipe to the requirements-level UML semantics of [8]. Properties of interest for probabilistic statecharts are expressed in PCTL, a probabilistic variant of CTL for processes that exhibit both non-determinism and probabilities. Verification is performed using the model checker Prism. A model checking example shows the feasibility of the suggested approach.

Process algebra for performance evaluation

This paper surveys the theoretical developments in the field of stochastic process algebras, process algebras where action occurrences may be subject to a delay that is determined by a random variable. A huge class of resource-sharing systems - like large-scale computers, client-server architectures, networks - can accurately be described using such stochastic specification formalisms. The main emphasis of this paper is the treatment of operational semantics, notions of equivalence, and (sound and complete) axiomatisations of these equivalences for different types of Markovian process algebras, where delays are governed by exponential distributions. Starting from a simple actionless algebra for describing time-homogeneous continuous-time Markov chains, we consider the integration of actions and random delays both as a single entity (like in known Markovian process algebras like TIPP, PEPA and EMPA) and as separate entities (like in the timed process algebras timed CSP and TCCS). In total we consider four related calculi and investigate their relationship to existing Markovian process algebras. We also briefly indicate how one can profit from the separation of time and actions when incorporating more general, non-Markovian distributions.

The ins and outs of the probabilistic model checker MRMC

The Markov Reward Model Checker (MRMC) is a software toolfor verifying properties over probabilistic models. It supports PCTL and CSL model checking, and their rewardextensions. Distinguishing features of MRMC are its support for computing time- and reward-bounded reachability probabilities, (property-driven) bisimulation minimization, and precise on-the-fly steady-state detection. Recent tool features include time-bounded reachability analysis for uniform CTMDPs and CSL model checking by discrete-event simulation. This paper presents the tools current status and its implementation details.

Approximate Symbolic Model Checking of Continuous-Time Markov Chains

of Invited Talk Research in the specification and verification of concurrent systems falls into two general categories. The temporal logic school advocates temporal logic as a language for formulating system requirements, with the semantics of the logic being used as a basis for determining whether or not a system is correct. The process-algebraic community focuses on the use of “higher-level” system descriptions as specifications of “lower-level” ones, with a refinement relation being used to determine whether an implementation conforms to a specification. From a user’s perspective, the approaches offer different benefits and drawbacks. Temporal logic supports “scenario-based” specifications, since formulas may be given that focus on single aspects of system behavior. On the other hand, temporal logic specifications suffer from a lack of compositionality, since the language of specifications differs from the system description language. In contrast, compositional specification is the hallmark of process algebraic reasoning, but at the expense of requiring what some view as overly detailed specifications. Although much research has studied the connections between the temporal logic and process algebra, a truly uniform formalism that combines the advantages of the two approaches has yet to emerge. In my talk I present preliminary results obtained by Gerald Lüttgen, of ICASE, and me on the development of such a formalism. Our approach features a process-algebra-inspired notation that enriches traditional process algebras by allowing linear-time temporal formulas to be embedded in system descriptions. We show how the combined formalism may be given a uniform operational semantics in Plotkin’s Structural Operational Semantics (SOS) style, and we define a refinement relation based on Denicola/Hennessy testing and discuss its congruence properties. We then demonstrate that traditional temporal-logic-style arguments about system correctness can be naturally captured via refinement; we also illustrate how the combination of logical and system operators allows users to define systems in which some “components” remain specified only as formulas. ? Research supported by NSF grants CCR-9257963, CCR-9505562 and CCR-9804091, AFOSR grant F49620-95-1-0508, and ARO grant P-38682-MA. Jos C.M. Baeten, Sjouke Mauw (Eds.): CONCUR’99, LNCS 1664, pp. 1–1, 1999. c

Optimal state-space lumping in Markov chains

We prove that the optimal lumping quotient of a finite Markov chain can be constructed in O(mlg n) time, where n is the number of states and m is the number of transitions. Our proof relies on the use of splay trees (designed by Sleator and Tarjan [J. ACM 32 (3) (1985) 652-686]) to sort transition weights.

Comparative branching-time semantics for Markov chains

This paper presents various semantics in the branching-time spectrum of discrete-time and continuous-time Markov chains (DTMCs and CTMCs). Strong and weak bisimulation equivalence and simulation preorders are covered and are logically characterized in terms of the temporal logics Probabilistic Computation Tree Logic (PCTL) and Continuous Stochastic Logic (CSL). Apart from presenting various existing branching-time relations in a uniform manner, this paper presents the following new results: (i) strong simulation for CTMCs, (ii) weak simulation for CTMCs and DTMCs, (iii) logical characterizations thereof (including weak bisimulation for DTMCs), (iv) a relation between weak bisimulation and weak simulation equivalence, and (v) various connections between equivalences and pre-orders in the continuous-and discrete-time setting. The results are summarized in a branching-time spectrum for DTMCs and CTMCs elucidating their semantics as well as their relationship.

CONCUR 2006: 17th International Conference on Concurrency Theory, CONCUR 2006

Invited Contributions.- Modeling Timed Concurrent Systems.- Some Remarks on Definability of Process Graphs.- Sanity Checks in Formal Verification.- Invited Tutorials.- Welcome to the Jungle: A Subjective Guide to Mobile Process Calculi.- Probability and Nondeterminism in Operational Models of Concurrency.- Model Checking.- A Livelock Freedom Analysis for Infinite State Asynchronous Reactive Systems.- Proving Liveness by Backwards Reachability.- Model Checking Quantified Computation Tree Logic.- Process Calculi.- Liveness, Fairness and Impossible Futures.- Checking a Mutex Algorithm in a Process Algebra with Fairness.- A Complete Axiomatic Semantics for the CSP Stable-Failures Model.- Transition Systems of Elementary Net Systems with Localities.- Minimization and Equivalence Checking.- Operational Determinism and Fast Algorithms.- Minimization, Learning, and Conformance Testing of Boolean Programs.- A Capability Calculus for Concurrency and Determinism.- Types.- A New Type System for Deadlock-Free Processes.- Sortings for Reactive Systems.- Dynamic Access Control in a Concurrent Object Calculus.- Semantics.- Concurrent Rewriting for Graphs with Equivalences.- Event Structure Semantics for Nominal Calculi.- Encoding ?Duce in the ??-Calculus.- Probability.- A Complete Axiomatisation of Branching Bisimulation for Probabilistic Systems with an Application in Protocol Verification.- Probabilistic I/O Automata: Theories of Two Equivalences.- Reachability in Recursive Markov Decision Processes.- Strategy Improvement for Stochastic Rabin and Streett Games.- Bisimulation and Simulation.- Weak Bisimulation Up to Elaboration.- Generic Forward and Backward Simulations.- On Finite Alphabets and Infinite Bases III: Simulation.- Real Time.- Inference of Event-Recording Automata Using Timed Decision Trees.- Controller Synthesis for MTL Specifications.- On Interleaving in Timed Automata.- Formal Languages.- A Language for Task Orchestration and Its Semantic Properties.- Finding Shortest Witnesses to the Nonemptiness of Automata on Infinite Words.- Second-Order Simple Grammars.

Model Checking Continuous-Time Markov Chains by Transient Analysis

The verification of continuous-time Markov chains (CTMCs) against continuous stochastic logic (CSL) [3,6], a stochastic branching-time temporal logic, is considered. CSL facilitates among others the specification of steady-state properties and the specification of probabilistic timing properties of the form \({\cal P}_{\bowtie p}(\Phi_1 \, {\cal U}^{I} \, \Phi_2)\), for state formulas Φ1 and Φ2, comparison operator ⋈, probability p, and real interval I. The main result of this paper is that model checking probabilistic timing properties can be reduced to the problem of computing transient state probabilities for CTMCs. This allows us to verify such properties by using efficient techniques for transient analysis of CTMCs such as uniformisation. A second result is that a variant of ordinary lumping equivalence (i.e., bisimulation), a well-known notion for aggregating CTMCs, preserves the validity of all CSL-formulas.

Probabilistic CEGAR

Counterexample-guided abstraction refinement (CEGAR) has been en voguefor the automatic verification of very large systems in the past years. When trying to apply CEGAR to the verification of probabilistic systems, various foundational questions arise. This paper explores them in the context of predicate abstraction.