Damjan Škulj

Discrete time Markov chains with interval probabilities

The parameters of Markov chain models are often not known precisely. Instead of ignoring this problem, a better way to cope with it is to incorporate the imprecision into the models. This has become possible with the development of models of imprecise probabilities, such as the interval probability model. In this paper we discuss some modelling approaches which range from simple probability intervals to the general interval probability models and further to the models allowing completely general convex sets of probabilities. The basic idea is that precisely known initial distributions and transition matrices are replaced by imprecise ones, which effectively means that sets of possible candidates are considered. Consequently, sets of possible results are obtained and represented using similar imprecise probability models. We first set up the model and then show how to perform calculations of the distributions corresponding to the consecutive steps of a Markov chain. We present several approaches to such calculations and compare them with respect to the accuracy of the results. Next we consider a generalisation of the concept of regularity and study the convergence of regular imprecise Markov chains. We also give some numerical examples to compare different approaches to calculations of the sets of probabilities.

Finite Discrete Time Markov Chains with Interval Probabilities

A Markov chain model in generalised settings of interval probabilities is presented. Instead of the usual assumption of constant transitional probability matrix, we assume that at each step a transitional matrix is chosen from a set of matrices that corresponds to a structure of an interval probability matrix. We set up the model and show how to obtain intervals corresponding to sets of distributions at consecutive steps. We also state the problem of invariant distributions and examine possible approaches to their estimation in terms of convex sets of distributions, and in a special case in terms of interval probabilities.

Imprecise Markov chains with absorption

We consider convergence of Markov chains with uncertain parameters, known as imprecise Markov chains, which contain an absorbing state. We prove that under conditioning on non-absorption the imprecise conditional probabilities converge independently of the initial imprecise probability distribution if some regularity conditions are assumed. This is a generalisation of a known result from the classical theory of Markov chains by Darroch and Seneta [6].

Efficient computation of the bounds of continuous time imprecise Markov chains

Abstract When the initial distribution and transition rates for a continuous time Markov chain are not known precisely, robust methods are needed to study the evolution of the process in time to avoid judgements based on unwarranted precision. We follow the ideas successfully applied in the study of discrete time model to build a framework of imprecise Markov chains in continuous time. The imprecision in the distributions over the set of states is modelled with upper and lower expectation functionals, which equivalently represent sets of probability distributions. Uncertainty in transitions is modelled with sets of transition rates compatible with available information. The Kolmogorov’s backward equation is then generalised into the form of a generalised differential equation, with generalised derivatives and set valued maps. The upper and lower expectation functionals corresponding to imprecise distributions at given times are determined by the maximal and minimal solutions of these equations. The second part of the paper is devoted to numerical methods for approximating the boundary solutions. The methods are based on discretisation of the time interval. A uniform and adaptive grid discretisations are examined. The latter is computationally much more efficient than the former one, but is not applicable on every interval. Therefore, to achieve maximal efficiency a combination of the methods is used.

Under Interval and Fuzzy Uncertainty, Symmetric Markov Chains Are More Difficult to Predict

Markov chains are an important tool for solving practical problems. In particular, Markov chains have been successfully applied in bioinformatics. Traditional statistical tools for processing Markov chains assume that we know the exact probabilities pij of a transition from the state i to the state j. In reality, we often only know these transition probabilities with interval (or fuzzy) uncertainty. We start the paper with a brief reminder of how the Markov chain formulas can be extended to the cases of such interval and fuzzy uncertainty. In some practical situations, there is another restriction on the Markov chain-that this Markov chain is symmetric in the sense that for every two states i and j, the probability of transitioning from i to j is the same as the probability of transitioning from j to i: pij = pji. In general, symmetry assumptions simplify computations. In this paper, we show that for Markov chains under interval and fuzzy uncertainty, symmetry has the opposite effect: it makes the computational problems more difficult.

Jeffrey’s conditioning rule in neighbourhood models☆

Abstract Neighbourhoods of classical probability measures, presented in the form of interval probabilities, are studied in the paper. The main goal is a characterization of two important classes, convex and bi-elastic neighbourhoods. Those two classes are equivalently characterized through closure conditions with respect to Jeffrey’s rule of conditioning. Moreover, some other interpretations of the closure property are given, including a description of behaviour of conditional expectation under the lower and upper expectation operators. This description is useful for a better understanding of some models in the theory of choice under risk. Further, closure under Jeffrey’s rule can serve as an extension rule for partially determined interval probabilities.

A classification of invariant distributions and convergence of imprecise Markov chains

Abstract We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes , that are the minimal sets capable of containing and preserving the whole probability mass of the chain. These classes generalise the essential classes known from the classical theory. We then define a class of extremal imprecise invariant distributions and show that they are uniquely determined by the values of the upper probability on minimal permanent classes. Moreover, we give conditions for unique convergence to these extremal invariant distributions.

The use of Markov operators to constructing generalised probabilities

A new approach to constructing generalised probabilities is proposed. It is based on the models using lower and upper previsions, or equivalently, convex sets of probability measures. Our approach uses sets of Markov operators in the role of rules preserving desirability of gambles. The main motivation being the operators of conditional expectations which are usually assumed to reduce riskiness of gambles. Imprecise probability models are then obtained in the ways to be consistent with those desirability preserving rules. The consistency criteria are based on the existing interpretations of models using imprecise probabilities. The classical models based on lower and upper previsions are shown to be a special class of the generalised models. Further, we generalise some standard extension procedures, including the marginal extension and independent products, which can be defined independently of the existing procedures known for standard models.

The Use of Sets of Stochastic Operators to Constructing Imprecise Probabilities

A new approach to constructing sets of probabilities is presented. We use sets of stochastic operators that represent rules that preserve desirability of gambles. We also provide a set of criteria that allow constructing imprecise probability models consistent with the desirability preserving rules. The model is more general than the standard imprecise probability models using lower and upper previsions. The greater generality means that credal sets and therefore lower previsions can be understood as a special case. Some results on extensions of such models are also provided that generalise the corresponding results from the theory of lower and upper previsions.

Products of capacities on separate spaces through additive measures

An approach to product of capacities is proposed. It extends our previous approach which works only for a special class of capacities. The main advantage of our approach, as opposed to most approaches given so far, is that it is designed for general, i.e. not necessarily discrete, monotone capacities. The product is not defined uniquely but rather, a lower and upper bound are given, which in case where both capacities are additive measures both coincide with the usual additive product.