Bernd Heidergott

Taylor series expansions for stationary Markov chains

We study Taylor series expansions of stationary characteristics of general-state-space Markov chains. The elements of the Taylor series are explicitly calculated and a lower bound for the radius of convergence of the Taylor series is established. The analysis provided in this paper applies to the case where the stationary characteristic is given through an unbounded sample performance function such as the second moment of the stationary waiting time in a queueing system.

Max-plus linear stochastic systems and perturbation analysis

Max-Plus Algebra.- Max-Plus Linear Stochastic Systems.- Ergodic Theory.- Perturbation Analysis.- A Max-Plus Differential Calculus.- Higher-Order D-Derivatives.- Taylor Series Expansions.

Towards a (Max,+) Control Theory for Public TransportationNetworks

We consider the modelling and analysisof public transportation networks, such as railway or subwaynetworks, governed by a timetable. Specifically, we study a (max,+)-linearmodel of a generic transportation network and thereby give aself-contained introduction to the key ideas underlying the (max,+)algebra. We elaborate on the algebraic structure implied by the(max,+)-model to formulate (and solve) the control problem inthe deterministic as well as in the stochastic case. The controlproblem is here whether a train should wait on a connecting trainwhich is delayed. Our objective is then to minimise the propagationof the delay through the network while maintaining as many connectionsas possible. With respect to the deterministic control problem,we present some recent ideas concerning the use of (max,+)-techniquesfor analysing the propagation of delays. Moreover, we show howone can use the (max,+)-algebra to drastically reduce the searchspace for the deterministic control problem. For the stochasticcontrol problem, we consider a parameterised version of the controlproblem, that is, we describe the control policy by means ofa real-valued parameter, say Θ. Finding theoptimal control is then turned into an optimisation problem withrespect to Θ. We address the problem by incorporatingan estimator of the derivative of the expected performance withrespect to Θ into a stochastic approximationalgorithm.

Series Expansions for Continuous-Time Markov Processes

We present update formulas that allow us to express the stationary distribution of a continuous-time Markov process with denumerable state space having generator matrix Q* through a continuous-time Markov process with generator matrix Q. Under suitable stability conditions, numerical approximations can be derived from the update formulas, and we show that the algorithms converge at a geometric rate. Applications to sensitivity analysis and bounds on perturbations are discussed as well. Numerical examples are presented to illustrate the efficiency of the proposed algorithm.

Series Expansions For Finite-State Markov Chains

This article provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

A characterisation of (max ,+) -linear queueing systems

The (max,+)-algebra has been successfully applied to many areas of queueing theory, like stability analysis and ergodic theory. These results are mainly based on two ingredients: (1) a (max,+)-linear model of the time dynamic of the system under consideration, and (2) the time-invariance of the structure of the (max,+)-model. Unfortunately, (max,+)-linearity is a purely algebraic concept and it is by no means immediate if a queueing network admits a (max,+)-linear representation satisfying (1) and (2). In this paper we derive the condition a queueing network must meet if it is to have a (max,+)-linear representation. In particular, we study (max,+)-linear systems with time-invariant transition structures. For this class of systems, we find a surprisingly simple necessary and sufficient condition for (max,+)-linearity, based on the flow of customers through the network.

Gradient estimation for discrete-event systems by measure-valued differentiation

In simulation of complex stochastic systems, such as Discrete-Event Systems (DES), statistical distributions are used to model the underlying randomness in the system. A sensitivity analysis of the simulation output with respect to parameters of the input distributions, such as the mean and the variance, is therefore of great value. The focus of this article is to provide a practical guide for robust sensitivity, respectively, gradient estimation that can be easily implemented along the simulation of a DES. We study the Measure-Valued Differentiation (MVD) approach to sensitivity estimation. Specifically, we will exploit the “modular” structure of the MVD approach, by firstly providing measure-valued derivatives for input distributions that are of importance in practice, and subsequently, by showing that if an input distribution possesses a measure-valued derivative, then so does the overall Markov kernel modeling the system transitions. This simplifies the complexity of applying MVD drastically: one only has to study the measure-valued derivative of the input distribution, a measure-valued derivative of the associated Markov kernel is then given through a simple formula in canonical form. The derivative representations of the underlying simple distributions derived in this article can be stored in a computer library. Combined with the generic MVD estimator, this yields an automated gradient estimation procedure. The challenge in automating MVD so that it can be included into a simulation package is the verification of the integrability condition to guarantee that the estimators are unbiased. The key contribution of the article is that we establish a general condition for unbiasedness which is easily checked in applications. Gradient estimators obtained by MVD are typically phantom estimators and we discuss the numerical efficiency of phantom estimators with the example of waiting times in the G/G/1 queue.

Measure-Valued Differentiation for Stationary Markov Chains

We study general state-space Markov chains that depend on a parameter, say, . Sufficient conditions are established for the stationary performance of such a Markov chain to be differentiable with respect to . Specifically, we study the case of unbounded performance functions and thereby extend the result on weak differentiability of stationary distributions of Markov chains to unbounded mappings. First, a closed-form formula for the derivative of the stationary performance of a general state-space Markov chain is given using an operator-theoretic approach. In a second step, we translate the derivative formula into unbiased gradient estimators. Specifically, we establish phantom-type estimators and score function estimators. We illustrate our results with examples from queueing theory.

Sensitivity estimation for Gaussian systems

In this paper we address the construction of efficient algorithms for the estimation of gradients of general performance measures of Gaussian systems. Exploiting a clever coupling between the normal and the Maxwell distribution, we present a new gradient estimator, and we show that it outperforms both the single-run based infinitesimal perturbation analysis (IPA) estimator and the score function (SF) estimator, in the one-dimensional case, for a dense class of test functions. Next, we present an example of the multi-dimensional case with a system from the area of stochastic activity networks. Our numerical experiments show that this new estimator also has significantly smaller sample variance than IPA and SF. To increase efficiency, in addition to variance reduction, we present an optimized method for generating the Maxwell distribution, which minimizes the CPU time.

OPTION PRICING VIA MONTE CARLO SIMULATION: A WEAK DERIVATIVE APPROACH

Using a weak derivation approach to gradient estimation, we consider the problem of pricing an American call option on stock paying dividends at discrete times. Similar simulation-based sensitivity estimators were introduced earlier by Fu and Hu (1995) who used smoothed perturbation analysis. We improve upon their results by presenting an estimator with a uniformly lower variance. In addition, we reduced the multidimensional optimization problem of pricing an option with multiple ex-dividend dates to a one-dimensional one. Numerical examples indicate that this approach saves a considerable amount of computation time. Our estimator holds uniformly for a class of payoff functions, and applications to other types of options will be addressed in the article.