Tugrul Dayar

Comparison of Multilevel Methods for Kronecker-based Markovian Representations

Abstract.The paper presents a class of numerical methods to compute the stationary distribution of Markov chains (MCs) with large and structured state spaces. A popular way of dealing with large state spaces in Markovian modeling and analysis is to employ Kronecker-based representations for the generator matrix and to exploit this matrix structure in numerical analysis methods. This paper presents various multilevel (ML) methods for a broad class of MCs with a hierarchcial Kronecker structure of the generator matrix. The particular ML methods are inspired by multigrid and aggregation-disaggregation techniques, and differ among each other by the type of multigrid cycle, the type of smoother, and the order of component aggregation they use. Numerical experiments demonstrate that so far ML methods with successive over-relaxation as smoother provide the most effective solvers for considerably large Markov chains modeled as HMMs with multiple macrostates.

Bounding the equilibrium distribution of Markov population models

SUMMARY We propose a bounding technique for the equilibrium probability distribution of continuous-time Markov chains with population structure and infinite state space. We use Lyapunov functions to determine a finite set of states that contains most of the equilibrium probability mass. Then we apply a refinement scheme based on stochastic complementation to derive lower and upper bounds on the equilibrium probability for each state within that set. To show the usefulness of our approach, we present experimental results for several examples from biology. Copyright

Quasi Lumpability, Lower-Bounding Coupling Matrices, and Nearly Completely Decomposable Markov Chains

In this paper, it is shown that nearly completely decomposable (NCD) Markov chains are quasi-lumpable. The state space partition is the natural one, and the technique may be used to compute lower and upper bounds on the stationary probability of each NCD block. In doing so, a lower-bounding nonnegative coupling matrix is employed. The nature of the stationary probability bounds is closely related to the structure of this lower-bounding matrix. Irreducible lower-bounding matrices give tighter bounds compared with bounds obtained using reducible lower-bounding matrices. It is also noticed that the quasi-lumped chain of an NCD Markov chain is an ill-conditioned matrix and the bounds obtained generally will not be tight. However, under some circumstances, it is possible to compute the stationary probabilities of some NCD blocks exactly.

Block SOR Preconditioned Projection Methods for Kronecker Structured Markovian Representations

Kronecker structured representations are used to cope with the state space explosion problem in Markovian modeling and analysis. Currently, an open research problem is that of devising strong preconditioners to be used with projection methods for the computation of the stationary vector of Markov chains (MCs) underlying such representations. This paper proposes a block successive overrelaxation (BSOR) preconditioner for hierarchical Markovian models (HMMs; throughout the paper, the HMM acronym stands for hierarchical Markovian models and should not be confused with the HMM that is sometimes used for hidden Markov models.) that are composed of multiple low-level models and a high-level model that defines the interaction among low-level models. The Kronecker structure of an HMM yields nested block partitionings in its underlying continuous-time MC which may be used in the BSOR preconditioner. The computation of the BSOR preconditioned residual in each iteration of a preconditioned projection method becomes the problem of solving multiple nonsingular linear systems whose coefficient matrices are the diagonal blocks of the chosen partitioning. The proposed BSOR preconditioner solves these systems using sparse LU or real Schur factors of diagonal blocks. The fill-in of sparse LU factorized diagonal blocks is reduced using the column approximate minimum degree (COLAMD) ordering. A set of numerical experiments is presented to show the merits of the proposed BSOR preconditioner.

On the effects of using the Grassmann-Taksar-Heyman method in interative aggregation-disaggregation

Iterative aggregation–disaggregation (IAD) is an effective method for solving finite nearly completely decomposable (NCD) Markov chains. Small perturbations in the transition probabilities of these chains may lead to considerable changes in the stationary probabilities; NCD Markov chains are known to be ill-conditioned. During an IAD step, this undesirable condition is inherited by the coupling matrix and one confronts the problem of finding the stationary probabilities of a stochastic matrix whose diagonal elements are close to l. In this paper, the effects of using the Grassmann–Taksar–Heyman (GTH) method to solve the coupling matrix formed in the aggregation step are investigated. Then the idea is extended in such a way that the same direct method can be incorporated into the disaggregation step. Finally, the effects of using the GTH method in the IAD algorithm on various examples are demonstrated, and the conditions under which it should be employed are explained.

TRANSFORMING STOCHASTIC MATRICES FOR STOCHASTIC COMPARISON WITH THE ST-ORDER

We present a transformation for stochastic matrices and analyze the effects of using it in stochastic comparison with the strong stochastic (st) order. We show that unless the given stochastic matrix is row diagonally dominant, the transformed matrix provides better st bounds on the steady state probability distribution.

Iterative disaggregation for a class of lumpable discrete-time stochastic automata networks

Stochastic automata networks (SANs) have been developed and used in the last 15 years as a modeling formalism for large systems that can be decomposed into loosely connected components. In this work, we concentrate on the not so much emphasized discrete-time SANs. First, we remodel and extend an SAN that arises in wireless communications. Second, for an SAN with functional transitions, we derive conditions for a special case of ordinary lumpability in which aggregation is done automaton by automaton. Finally, for this class of lumpable discrete-time SANs we devise an efficient aggregation-iterative disaggregation algorithm and demonstrate its performance on the SAN model of interest.

Computing Moments of First Passage Times to a Subset of States in Markov Chains

This paper presents a relatively efficient and accurate method to compute the moments of first passage times to a subset of states in finite ergodic Markov chains. With the proposed method, the moment computation problem is reduced to the solution of a linear system of equations with the right-hand side governed by a novel recurrence for computing the higher-order moments. We propose using a form of the Grassmann--Taksar--Heyman (GTH) algorithm to solve these linear equations. Due to the form of the linear systems involved, the proposed method does not suffer from the drawbacks associated with GTH in a row-wise sparse implementation.

Lumpable continuous-time stochastic automata networks

Abstract The generator matrix of a continuous-time stochastic automata network (SAN) is a sum of tensor products of smaller matrices, which may have entries that are functions of the global state space. This paper specifies easy to check conditions for a class of ordinarily lumpable partitionings of the generator of a continuous-time SAN in which aggregation is performed automaton by automaton. When there exists a lumpable partitioning induced by the tensor representation of the generator, it is shown that an efficient aggregation-iterative disaggregation algorithm may be employed to compute the steady-state distribution. The results of experiments with two SAN models show that the proposed algorithm performs better than the highly competitive block Gauss–Seidel in terms of both the number of iterations and the time to converge to the solution.

Infinite level-dependent QBD processes and matrix-analytic solutions for stochastic chemical kinetics

Systems of stochastic chemical kinetics are modeled as infinite level-dependent quasi-birth-and-death (LDQBD) processes. For these systems, in contrast to many other applications, levels have an increasing number of states as the level number increases and the probability mass may reside arbitrarily far away from lower levels. Ideas from Lyapunov theory are combined with existing matrix-analytic formulations to obtain accurate approximations to the stationary probability distribution when the infinite LDQBD process is ergodic. Results of numerical experiments on a set of problems are provided.