Sudhakar D. Deshmukh

A martingale characterization of the price of a nonrenewable resource with decisions involving uncertainty

Abstract This paper presents a general model of nonrenewable resource consumption and exploration decisions involving uncertainty about the time of occurrence of an event such as exhaustion, stock discovery, or a substitute development. The resulting price process is characterized in terms of necessary and sufficient conditions under which the price is expected to rise at a rate equal to, greater than, or less than the discount rate. The general model is illustrated and the price process and the optimal decisions are characterized by examining the three types of uncertainty indicated above.

The role of external search in bilateral bargaining

We consider the problem of bargaining between two individuals who can also search for possible alternatives to each others offers. We explicitly incorporate the search activity into the process of bargaining: a disagreement in bargaining will lead each individual to search for an outside offer in the hope of improving his future bargaining position and attaining a more favorable outcome. We model the interlaced phases of bargaining and search over time as an extensive noncooperative game. We characterize the associated subgame-perfect equilibrium outcome and strategies. Finally, we show that superior search ability yields an individual a more favorable outcome and hence represents a greater bargaining strength.

Optimal Consumption of a Nonrenewable Resource with Stochastic Discoveries and a Random Environment

We present a general model for the optimal consumption of a nonrenewable resource under two kinds of uncertainties. One source of uncertainty is in the resource discovery process and the other is in the economic environment that affects resource supply and demand conditions, such as exhaustion and development of a substitute. The problem is formulated as one of optimally controlling a storage process with Markov additive discoveries. The optimal value of the resource stock is characterized as the solution of a functional equation and the existence of an optimal consumption policy is established. It is shown that, in a given environment, the optimal consumption rate is increasing and the resource price is decreasing in the level of proven reserves. A counterexample is provided to show that better environments may in fact mean higher prices and lower consumption rates. Finally, a variety of examples is given to illustrate the scope and applicability of the general model.

A Controlled Birth and Death Process Model of Optimal Product Pricing Under Stochastically Changing Demand

We consider the problem of product pricing when the firms market share is changing stochastically according to a birth and death process. The current market share together with the price prevailing determine the current rate of profit made as well as the birth and death rates. The optimal pricing policy must balance the immediate advantage of setting a high price in terms of increased current profit against the disadvantage in terms of a possible erosion of the future market share. We formulate a continuous-time Markov decision model and analyse it using a recent technique developed by Lippman [6] for optimization of exponential queueing systems. The optimal pri6ing policy is characterized as having a sort of monotonicity property. We also analyse the dependence of the optimal policy on the problem parameters and indicate further extensions of the model. BIRTH AND DEATH PROCESS; QUEUE CONTROL; OPTIMAL PRICING STRATEGY; DYNAMIC PROGRAMMING

Stochastic Control of Competition Through Prices

We assume that the price of a product set by a firm affects its immediate profit rate as well as the probabilistic rate of arrival of new firms into the industry. Therefore, the firms optimal dynamic pricing strategy must balance the increased current profits from setting a high price against the expected dilution of future profits due to additional competition. We provide a continuous-time Markov decision model and characterize the structure of the optimal control strategy and its sensitivity to the problem parameters. We also indicate the relationship of our problem to the queue control literature.

Capacity Design and Service Quality Control in a Queuing System

We consider a finite-capacity single-server queuing system in which the problems of capacity design and service quality control are integrated. The amount of time spent on servicing a customer is used as a measure of the quality of service provided. The expected reward from servicing is assumed to be a nondecreasing function of the service duration, while a linear holding cost is assumed for customers waiting in the system. We maximize the long-run average return per unit time by optimally controlling the quality of service to be provided as a function of the workload facing the server for different waiting-room capacities. We show that the optimal control policy for a given capacity is monotone and also examine the effect of varying the system capacity on the optimal control policy. Furthermore, we analyze the design problem of selecting an optimal capacity by assuming that an optimal service control policy will be followed for each system capacity.

A Zero-Sum Stochastic Game Model of Duopoly

We consider a discrete time zero-sum stochastic game model of duopoly and give a partial characterization of each firms optimal pricing strategy. An extension to a continuous time model is also discussed.

Storage Decisions in Information Systems

Abstract The problem is to dynamically store different data records in different storage devices in each period so as to minimize the total expected discounted cost over a planning horizon. Each device has a fixed total capacity, each record has a given storage space requirement, while the number of requests for each record per period is changing stochastically through time. Given an allocation, the total cost per period consists of the storage cost (depending on the storage requirements and device), the access cost including update and retrieval costs (depending on the number of requests) and the transfer cost (depending upon the change of allocation from the previous period). A dynamic programming model is presented to yield optimal strategies. The special case of independent identically distributed demands is completely solved, using a generalized transportation algorithm while a heuristic procedure is indicated for the general problem using parametric analysis.

Computational Delays

Decision-making processes usually involve time-consuming and costly operations of observation and communication of the state of the environment and generation and implementation of appropriate actions. Collectively these activities may be called computational and the procedure required to carry them out may be called an algorithm. If the environment is changing stochastically the fixed computational delay involved in operating the algorithm yields obsolete actions resulting in reduced expected return. The loss as a function of delay is a measure of efficiency of the algorithm which depends upon the stochastic properties of the environment. If it is possible to reduce the loss by employing a faster algorithm at a higher cost, the algorithm may be optimally designed and selected from a given family.

Optimal delays in decision and control

Computations involved in controlling a system or a decision process are time-consuming in practice. The problem of optimally choosing the estimation and control delays is formulated in the dynamic programming framework and illustrated by examples. Selection of optimal estimation and control algorithms is outlined conceptually.