Henk Tijms

Stochastic modelling and analysis: a computational approach

The main concern of this text is the application of stochastic models to practical situations involving uncertainty and dynamism. A unique feature is the integrated treatment of models and computational methods for stochastic design and stochastic optimization problems. The book uses realistic examples to explore a wide variety of applications, such as inventory and production control, reliability, maintenance, queueing computer and communication systems. Exercises and suggestions for further reading are provided at the end of each chapter. The book was written with advanced students in mind, however, as it contains a wealth of material not found in other texts, it will also be of considerable interest to practitioners and researchers in operations research, statistics, computer science and engineering.

Simple approximations for the reorder point in periodic and continuous review (s, S) inventory systems with service level constraints

Abstract In this paper we consider a general class of (s, S) inventory systems including periodic review and continuous review systems. We allow for stochastic lead times for replenishment orders provided that the probability of orders crossing in time is negligible for the relevant (s, S) control rules. In accordance with common practice we emphasize on service level constraints rather than assuming given stockout costs. In particular we consider the service measure requiring that a specified fraction of the demand is met directly from stock on hand. The purpose of this paper is to present practically useful approximations for the recorder point s such that the required service level is achieved. By a simple and direct approach, a unifying treatment of the general class of (s, S) inventory systems considered is given. We obtain for the first time tractable approximations for the continuous review (s, S) inventory system with undershoots of the reorder point. Also, we discuss 2-moments approximations obtained by fitting normal respectively gamma distributions to the empirical demand distributions. Extensive numerical experience with the approximations is reported, including results about the sensitivity of the reorder point to the higher moments of the demand distributions.

Denumerable Undiscounted Semi-Markov Decision Processes with Unbounded Rewards

This paper establishes the existence of a solution to the optimality equations in undis-counted semi-Markov decision models with countable state space, under conditions generalizing the hitherto obtained results. In particular, we merely require the existence of a finite set of states in which every pair of states can reach each other via some stationary policy, instead of the traditional and restrictive assumption that every stationary policy has a single irreducible set of states. A replacement model and an inventory model illustrate why this extension is essential. Our approach differs fundamentally from classical approaches; we convert the optimality equations into a form suitable for the application of a fixed point theorem.

A Markov decision algorithm for optimal inspections and revisions in a maintenance system with partial information

Abstract This paper deals with a gradually deteriorating equipment whose actual degree of deterioration can be revealed by inspections only. An inspection can be succeeded by a revision depending on the systems degree of deterioration. In the absence of inspections and revisions, the working condition of the system evolves according to a Markov chain whose changes of state are not observable with the possible exception of a breakdown. Examples of this model include production machines subject to stochastic breakdowns, and maintenance of communication systems. The cost structure of the model consists of inspection, revision and operating costs. It is intuitively reasonable that in many applications a simple control-limit rule will be optimal. Such a rule prescribes a revision only when inspection reveals that the degree of deterioration has exceeded some critical level. A special-purpose Markov decision algorithm operating on the class of control-limit rules is developed for the computation of an average cost optimal schedule of inspections and revisions.

Denumerable state semi-Markov decision processes with unbounded costs, average cost criterion

This paper establishes a rather complete optimality theory for the average cost semi-Markov decision model with a denumerable state space, compact metric action sets and unbounded one-step costs for the case where the underlying Markov chains have a single ergotic set. Under a condition which, roughly speaking, requires the existence of a finite set such that the supremum over all stationary policies of the expected time and the total expected absolute cost incurred until the first return to this set are finite for any starting state, we shall verify the existence of a finite solution to the average costs optimality equation and the existence of an average cost optimal stationary policy.

Exponential convergence of products of stochastic matrices

Abstract This paper considers a finite set of stochastic matrices of finite order. Conditions are given under which any product of matrices from this set converges to a constant stochastic matrix. Also, it is shown that the convergence is exponentially fast.

Approximations for the single product production-inventory problem with compound Poisson demand and service level constraints

We consider a production-inventory problem in which the production rate can be continuously controlled in order to cope with random fluctuations in the demand. The demand process for a single product is a compound Poisson process. Excess demand is backlogged. Two production rates are available and the inventory level is continuously controlled by a switch-over rule characterized by two critical numbers. In accordance with common practice, we consider service measures such as the average number of stockouts per unit time and the fraction of demand to be met directly from stock on hand. The purpose of the paper is to derive practically useful approximations for the switch-over levels of the control rule such that a pre-specified value of the service level is achieved.

Computation of the stationary distribution of the queue size in an M/G/1 queueing system with variable service rate

This paper presents a simple and computationally tractable method which recursively computes the stationary probabilities of the queue size in an M/G/1 queueing system with variable service rate. For each service two possible service types are available and the service rule is characterized by two switch-over levels. The computational approach discussed in this paper can be applied to a variety of queueing problems.

A heuristic rule for routing customers to parallel servers

A practically important problem is the assignment of stochastically arriving service requests to one of several parallel service groups so as to minimize the long-run average sojourn time per service request. An exact solution of this multi-dimensional optimization problem is computationally infeasible. A simple heuristic solution method yielding a good suboptimal rule will be given for the case of server groups with different and generally distributed service times. This solution method is based on a decomposition approach and first principles from Markov decision theory. The main idea of the heuristic method is to apply one step of policy improvement to the best Bernoulli-splitting rule.

A two-moment approximation for a buffer design problem requiring a small rejection probability

Abstract A useful model for buffer capacity design in communication systems is the single server queueing model with restricted accessibility where arriving customers are admitted only if their waiting plus service times do not exceed some fixed amount. A two-moment approximation for the buffer capacity in order to achieve a specific rejection probability is proposed for the case of Poisson arrivals and general service requirements. This approximation is a weighted combination of exact results for the special cases of deterministic and exponential service requirements where the weights use only the coefficient of variation of the general service requirement. Numerical experiments show an excellent performance of the approximation.