J. van der Wal

Discounted Markov games; successive approximation and stopping times

This paper presents a number of successive approximation algorithms for the repeated two-person zero-sum game called Markov game using the criterion of total expected discounted rewards. AsWessels [1977] did for Markov decision processes stopping times are introduced in order to simplify the proofs. It is shown that each algorithm provides upper and lower bounds for the value of the game and nearly optimal stationary strategies for both players.

Monotonicity of the throughput of a closed exponential queueing network in the number of jobs

SummaryIt is shown that the throughput of a closed exponential queueing network is nondecreasing in the number of jobs in the system if the service rate in each station is nondecreasing in the number of jobs in that queue. The line of proof seems to be extendable to a variety of networks without a product form equilibrium.ZusammenfassungEs wird gezeigt, daß der Durchsatz eines geschlossenen exponentiellen Warteschlangen-Netzwerks hinsichtlich der Anzahl der Aufträge im System nicht fällt, wenn die Bedienungsrate in jeder Station hinsichtlich der Anzahl der Aufträge in dieser Warteschlange nicht fällt. Die Beweisführung scheint erweiterbar auf eine Vielzahl von Netzwerken ohne Produktionsgleichgewicht.

Successive approximations for average reward Markov games

This paper considers two-person zero-sum Markov games with finitely many states and actions with the criterion of average reward per unit time. Two special situations are treated and it is shown that in both cases the method of successive approximations yields anε-band for the value of the game as well as stationaryε-optimal strategies. In the first case all underlying Markov chains of pure stationary optimal strategies are assumed to be unichained. In the second case it is assumed that the functional equation Uv=v+ge has a solution.

The maximization of CP-utilization in an exponential CP-terminal system with different think times and different job sizes

This paper deals with the optimization of CP utilization in a CP-Terminal system with exponential job sizes and exponential think times. It is shown that if preemptions of the resume type are allowed, the best static priority rule is to give priority to the fastest thinker and is independent of the expected job size.

The response time distribution in a multi-processor database with single queue static locking

A transaction scheduling mechanism is designed for a shared memory, multiprocessor database system. The scheduler used is a variant of static locking, adapted for real time and more than one processor. It is assumed that transactions arrive according to a Poisson process, execution times of transactions are independent and exponentially distributed and all transactions use the same number of data items. The system is then represented as a Markov model. A steady state is derived from this model. By examining the path through the system of a single transaction, a recursive relation that describes all moments of a transactions response time is derived. The response time distribution is approximated by fitting a distribution to the first two moments. Simulation shows that this approximation gives excellent results.

The M/G/∞ Queueing Model for Optimistic Concurrency Control

The M/G/∞ queue with Optimistic Concurrency Control (OCC) is a model for a special form of parallel transaction processing in a real-time database. Transactions arrive according to a Poisson process and require some generally distributed execution time. One of the differences with ordinary multi- and infinite-server queueing models is that under OCC the successful completion of one transaction may immediately cause the failure of one or more of the other transactions. This happens if the completing transaction has overwritten a data-item that is in use by another transaction in progress. As soon as this failure is detected the failed transaction is restarted. So the total service time of a transaction consists of its final successful run and the time spent on unsuccessful runs. n n n nIn this study we develop an approximation for the distribution of the total service time, and test the approximation against simulation. Although in practice the number of servers is never unlimited, this study provides valuable insight in the asymptotics with respect to the number of servers. The approximation clearly demonstrates the very limited performance gain from an increase of the number of servers.

Two Approximations for the steady state probabilities and the sojourn time distribution of the M/D/c queue with state dependent feedback

In the M/D/cqueue with state-dependent feedback, a customer is only allowed to depart from the system if his service has been successful. Otherwise, the customer must be re-serviced immediately. The probability that a customers service is successful depends on the number of customers in service at the moment the service is finished. The application behind this type of feedback queue is a real-time database where transactions must be rerun if their data was changed by other transactions during the execution. In this paper, two different approximations for the steady-state probabilities and the sojourn-time distribution of the M/D/cqueue with state-dependent feedback are studied. The first approximation is based on an embedded Markov chain and uses the well-known residual-life approximation for the remaining service times of the customers in service. The second approximation is similar to the exact analysis of the ordinary M/D/cqueue. Comparison with simulation shows, that both approximations are very accu...