Frank Ciesinski

PROBMELA: a modeling language for communicating probabilistic processes

Building automated tools to address the analysis of reactive probabilistic systems requires a simple, but expressive input language with a formal semantics based on a probabilistic operational model that can serve as starting point for verification algorithms. We introduce for probabilistic parallel programs with shared variables, message passing via synchronous and (perfect or lossy) fifo channels and atomic regions and provide a structured operational semantics. Applied to finite-state systems, the semantics can serve as basis for the algorithmic generation of a Markov decision process that models the stepwise behavior of the given system.

On Probabilistic Computation Tree Logic

In this survey we motivate, define and explain model checking of probabilistic deterministic and nondeterministic systems using the probabilistic computation tree logics PCTL and PCTL *. Juxtapositions to non-deterministic computation tree logic are made and algorithms are presented.

LiQuor: A tool for Qualitative and Quantitative Linear Time analysis of Reactive Systems

LiQuor is a tool for verifying probabilistic reactive systems modelled Probmela programs, which are terms of a probabilistic guarded command language with an operational semantics based on (finite) Markov decision processes. LiQuor provides the facility to perform a qualitative or quantitative analysis for omega-regular linear time properties by means of automata-based model checking algorithms

Partial order reduction for probabilistic systems

In the past, several model checking algorithms have been proposed to verify probabilistic reactive systems. The techniques to combat the state-explosion problem have mainly concentrated on symbolic methods with variants of decision diagrams or abstraction methods. In this paper, we show how partial order reduction with a variant of Peleds ample set method can be applied in the context of LTL model checking for probabilistic systems modelled by Markov decision processes.

Reduction Techniques for Model Checking Markov Decision Processes

The quantitative analysis of a randomized system, modeled by a Markov decision process, against an LTL formula can be performed by a combination of graph algorithms, automata-theoretic concepts and numerical methods to compute maximal or minimal reachability probabilities. In this paper, we present various reduction techniques that serve to improve the performance of the quantitative analysis, and report on their implementation on the top of the probabilistic model checker \LiQuor. Although our techniques are purely heuristic and cannot improve the worst-case time complexity of standard algorithms for the quantitative analysis, a series of examples illustrates that the proposed methods can yield a major speed-up.

Model Checking Linear-Time Properties of Probabilistic Systems

This chapter is about the verification of Markov decision processes (MDPs) which incorporate one of the fundamental models for reasoning about probabilistic and nondeterministic phenomena in reactive systems. MDPs have their roots in the field of operations research and are nowadays used in a wide variety of areas including verification, robotics, planning, controlling, reinforcement learning, economics and semantics of randomized systems. Furthermore, MDPs served as the basis for the introduction of probabilistic automata which are related to weighted automata. We describe the use of MDPs as an operational model for randomized systems, e.g., systems that employ randomized algorithms, multi-agent systems or systems with unreliable components or surroundings. In this context we outline the theory of verifying ω-regular properties of such operational models. As an integral part of this theory we use ω-automata, i.e., finite-state automata over finite alphabets that accept languages of infinite words. Additionally, basic concepts of important reduction techniques are sketched, namely partial order reduction of MDPs and quotient system reduction of the numerical problem that arises in the verification of MDPs. Furthermore we present several undecidability and decidability results for the controller synthesis problem for partially observable MDPs.

Quantitative Analysis under Fairness Constraints

It is well-known that fairness assumptions can be crucial for verifying progress, reactivity or other liveness properties for interleaving models. This also applies to Markov decision processes as an operational model for concurrent probabilistic systems and the task to establish tight lower or upper probability bounds for events that are specified by liveness properties. In this paper, we study general notions of strong and weak fairness constraints for Markov decision processes, formalized in an action- or state-based setting. We present a polynomially time-bounded algorithm for the quantitative analysis of an MDP against *** -automata specifications under fair worst- or best-case scenarios. Furthermore, we discuss the treatment of strong and weak fairness and process fairness constraints in the context of partial order reduction techniques for Markov decision processes that have been realized in the model checker LiQuor and rely on a variant of Peleds ample set method.

ProbMela and verification of Markov decision processes

Markov decision processes (MDP) can serve as operational model for probabilistic distributed systems and yield the basis for model checking algorithms against qualitative or quantitative properties. In this paper, we summarize the main steps of a quantitative analysis for a given MDP and formula of linear temporal logic, give an introduction to the modelling language ProbMela which provides a simple and intuitive way to describe complex systems with a MDP-semantics and present the basic features of the MDP model checker LiQuor.

Controller Synthesis for Probabilistic Systems (Extended Abstract)

Controller synthesis addresses the question of how to limit the internal behavior of a given implementation to meet its specification, regardless of the behavior enforced by the environment. In this paper, we consider a model with probabilism and nondeterminism where the nondeterministic choices in some states are assumed to be controllable, while the others are under the control of an unpredictable environment. We first consider probabilistic computation tree logic as specification formalism, discuss the role of strategy-types for the controller and show the NP-hardness of the controller synthesis problem. The second part of the paper presents a controller synthesis algorithm for automata-specifications which relies on a reduction to the synthesis problem for PCTL with fairness.

Generating Compact MTBDD-Representations from Probmela Specifications

The purpose of the paper is to provide an automatic transformation of parallel programs of an imperative probabilistic guarded command language (called Probmela ) into probabilistic reactive module specifications. The latter serve as basis for the input language of the symbolic MTBDD-based probabilistic model checker PRISM , while Probmela is the modeling language of the model checker LiQuor which relies on an enumerative approach and supports partial order reduction and other reduction techniques. By providing the link between the model checkers PRISM and LiQuor , our translation supports comparative studies of different verification paradigms and can serve to use the (more comfortable) guarded command language for a MTBDD-based quantitative analysis. The challenges were (1) to ensure that the translation preserves the Markov decision process semantics, (2) the efficiency of the translation and (3) the compactness of the symbolic BDD-representation of the generated PRISM -language specifications.