Stanley R. Pliska

Martingales and stochastic integrals in the theory of continuous trading

This paper develops a general stochastic model of a frictionless security market with continuous trading. The vector price process is given by a semimartingale of a certain class, and the general stochastic integral is used to represent capital gains. Within the framework of this model, we discuss the modern theory of contingent claim valuation, including the celebrated option pricing formula of Black and Scholes. It is shown that the security market is complete if and only if its vector price process has a certain martingale representation property. A multidimensional generalization of the Black-Scholes model is examined in some detail, and some other examples are discussed briefly.

A stochastic calculus model of continuous trading: optimal portfolios

The problem of choosing a portfolio of securities so as to maximize the expected utility of wealth at a terminal planning horizon is solved via stochastic calculus and convex analysis. This problem is decomposed into two subproblems. With security prices modeled as semimartingales and trading strategies modeled as predictable processes, the set of terminal wealths is identified as a subspace in a space of integrable random variables. The first subproblem is to find the terminal wealth that maximizes expected utility. Convex analysis is used to derive necessary and sufficient conditions for optimality and an existence result. The second subproblem of finding the admissible trading strategy that generates the optimal terminal wealth is a martingale representation problem. The primary advantage of this approach is that explicit formulas can readily be derived for the optimal terminal wealth and the corresponding expected utility, as is shown for the case of an exponential utility function and a risky security modeled as geometric Brownian motion.

A stochastic calculus model of continuous trading: Complete markets

A paper by the same authors in the 1981 volume of Stochastic Processes and Their Applications presented a general model, based on martingales and stochastic integrals, for the economic problem of investing in a portfolio of securities. In particular, and using the terminology developed therein, that paper stated that every integrable contingent claim is attainable (i.e., the model is complete) if and only if every martingale can be represented as a stochastic integral with respect to the discounted price process. This paper provides a detailed proof of that result as well as the following: The model is complete if and only if there exists a unique martingale measure.

Controlled jump processes

Finite and infinite planning horizon Markov decision problems are formulated for a class of jump processes with general state and action spaces and controls which are measurable functions on the time axis taking values in an appropriate metrizable vector space. For the finite horizon problem, the maximum expected reward is the unique solution, which exists, of a certain differential equation and is a strongly continuous function in the space of upper semi-continuous functions. A necessary and sufficient condition is provided for an admissible control to be optimal, and a sufficient condition is provided for the existence of a measurable optimal policy. For the infinite horizon problem, the maximum expected total reward is the fixed point of a certain operator on the space of upper semi-continuous functions. A stationary policy is optimal over all measurable policies in the transient and discounted cases as well as, with certain added conditions, in the positive and negative cases.

A martingale characterization of the price of a nonrenewable resource with decisions involving uncertainty

Abstract This paper presents a general model of nonrenewable resource consumption and exploration decisions involving uncertainty about the time of occurrence of an event such as exhaustion, stock discovery, or a substitute development. The resulting price process is characterized in terms of necessary and sufficient conditions under which the price is expected to rise at a rate equal to, greater than, or less than the discount rate. The general model is illustrated and the price process and the optimal decisions are characterized by examining the three types of uncertainty indicated above.

Optimal Control of Single-Server Queuing Networks and Multi-Class M/G/1 Queues with Feedback

We consider a queuing network with Poisson arrivals at each node. At each service completion epoch, a reward is received and the serviced customer changes nodes or leaves the system according to specified probabilities. In addition, linear holding costs are incurred. The problem is to schedule the server so as to maximize the expected discounted reward over an infinite planning horizon. This model is equivalent to a single-server, multi-class queuing system with feedback of the customers. We study two cases: general service times with a non-preemptive service discipline and exponential service times with a preemptive service discipline. For each case we show that a modified static policy of priority form is optimal and we provide an algorithm for computing an optimal policy.

Optimal Consumption of a Nonrenewable Resource with Stochastic Discoveries and a Random Environment

We present a general model for the optimal consumption of a nonrenewable resource under two kinds of uncertainties. One source of uncertainty is in the resource discovery process and the other is in the economic environment that affects resource supply and demand conditions, such as exhaustion and development of a substitute. The problem is formulated as one of optimally controlling a storage process with Markov additive discoveries. The optimal value of the resource stock is characterized as the solution of a functional equation and the existence of an optimal consumption policy is established. It is shown that, in a given environment, the optimal consumption rate is increasing and the resource price is decreasing in the level of proven reserves. A counterexample is provided to show that better environments may in fact mean higher prices and lower consumption rates. Finally, a variety of examples is given to illustrate the scope and applicability of the general model.

Choosing the maximum from a sequence with a discount function

Choosing the maximum value from a sequence ofN independent values is a well known problem often called the candidate problem or secretary problem. This paper treats the above problem with a discount penaltyα (0<α<1) for each additional observation taken. It is shown that asN increases indefinitely, the optimal stopping policy is bounded although the maximum expected payoff goes to zero, and that there exists a sequence 0=α0<α1<α2<⋯<1, such that the asymptotic optimal stopping rule is the same for allαi−1<α⩽αi.

ON THE TRANSIENT CASE FOR MARKOV DECISION CHAINS WITH GENERAL STATE SPACES

The transient case for discrete time dynamic programs is extended to allow for fairly general state and action spaces. Most of the finite state space results hold, including that the transient case exists if and only if there exists a certain condition involving just the stationary policies. These results are applied to a problem of controlling a multitype branching process.

Accretive Operators and Markov Decision Processes

The dynamic programming functional equation for an abstract, continuous parameter, Markov decision process is shown to involve an operator which is m-accretive, thus giving rise to a nonlinear semigroup, called the Bellman semigroup. A class of controls is specified for which the maximum expected reward over a finite planning horizon is given by this semigroup. This theory is applied to controlled jump, diffusion, and storage processes.