Savas Dayanik

A Knowledge-Gradient Policy for Sequential Information Collection

In a sequential Bayesian ranking and selection problem with independent normal populations and common known variance, we study a previously introduced measurement policy which we refer to as the knowledge-gradient policy. This policy myopically maximizes the expected increment in the value of information in each time period, where the value is measured according to the terminal utility function. We show that the knowledge-gradient policy is optimal both when the horizon is a single time period and in the limit as the horizon extends to infinity. We show furthermore that, in some special cases, the knowledge-gradient policy is optimal regardless of the length of any given fixed total sampling horizon. We bound the knowledge-gradient policys suboptimality in the remaining cases, and show through simulations that it performs competitively with or significantly better than other policies.

Optimal Multiple Stopping of Linear Diffusions

Motivated by the analysis of financial instruments with multiple exercise rights of American type and mean reverting underlyers, we formulate and solve the optimal multiple-stopping problem for a general linear regular diffusion process and a general reward function. Instead of relying on specific properties of geometric Brownian motion and call and put option payoffs as in most of the existing literature, we use general theory of optimal stopping for diffusions, and we illustrate the resulting optimal exercise policies by concrete examples and constructive recipes.

Adaptive Poisson disorder problem

We study the quickest detection problem of a sudden change in the arrival rate of a Poisson process from a known value to an unknown and unobservable value at an unknown and unobservable disorder time. Our objective is to design an alarm time which is adapted to the history of the arrival process and detects the disorder time as soon as possible. In previous solvable versions of the Poisson disorder problem, the arrival rate after the disorder has been assumed a known constant. In reality, however, we may at most have some prior information about the likely values of the new arrival rate before the disorder actually happens, and insufficient estimates of the new rate after the disorder happens. Consequently, we assume in this paper that the new arrival rate after the disorder is a random variable. The detection problem is shown to admit a finite-dimensional Markovian sufficient statistic, if the new rate has a discrete distribution with finitely many atoms. Furthermore, the detection problem is cast as a discounted optimal stopping problem with running cost for a finite-dimensional piecewise-deterministic Markov process. This optimal stopping problem is studied in detail in the special case where the new arrival rate has Bernoulli distribution. This is a nontrivial optimal stopping problem for a two-dimensional piecewise-deterministic Markov process driven by the same point process. Using a suitable single-jump operator, we solve it fully, describe the analytic properties of the value function and the stopping region, and present methods for their numerical calculation. We provide a concrete example where the value function does not satisfy the smooth-fit principle on a proper subset of the connected, continuously differentiable optimal stopping boundary, whereas it does on the complement of this set.

An Adaptive Bayesian Replacement Policy with Minimal Repair

In this study, an adaptive Bayesian decision model is developed to determine the optimal replacement age for the systems maintained according to a general age-replacement policy. It is assumed that when a failure occurs, it is eithercritical with probabilityp ornoncritical with probability 1 --p, independently. A maintenance policy is considered where the noncritical failures are corrected with minimal repair and the system is replaced either at the first critical failure or at age t, whichever occurs first. The aim is to find the optimal value of t that minimizes the expected cost per unit time. Two adaptive Bayesian procedures that utilize different levels of information are proposed for sequentiallyupdating the optimal replacement times. Posterior density/mass functions of the related variables are derived when the time to failure for the system can be expressed as a Weibull random variable. Some simulation results are also presented for illustration purposes.

Bayesian Sequential Change Diagnosis

Sequential change diagnosis is the joint problem of detection and identification of a sudden and unobservable change in the distribution of a random sequence. In this problem, the common probability law of a sequence of i.i.d. random variables suddenly changes at some disorder time to one of finitely many alternatives. This disorder time marks the start of a new regime, whose fingerprint is the new law of observations. Both the disorder time and the identity of the new regime are unknown and unobservable. The objective is to detect the regime-change as soon as possible, and, at the same time, to determine its identity as accurately as possible. Prompt and correct diagnosis is crucial for quick execution of the most appropriate measures in response to the new regime, as in fault detection and isolation in industrial processes, and target detection and identification in national defense. The problem is formulated in a Bayesian framework. An optimal sequential decision strategy is found, and an accurate numerical scheme is described for its implementation. Geometrical properties of the optimal strategy are illustrated via numerical examples. The traditional problems of Bayesian change-detection and Bayesian sequential multi-hypothesis testing are solved as special cases. In addition, a solution is obtained for the problem of detection and identification of component failure(s) in a system with suspended animation.

MULTISOURCE BAYESIAN SEQUENTIAL CHANGE DETECTION

Suppose that local characteristics of several independent compound Poisson and Wiener processes change suddenly and simultaneously at some unobservable disorder time. The problem is to detect the disorder time as quickly as possible after it happens and minimize the rate of false alarms at the same time. These problems arise, for example, from managing product quality in manufacturing systems and preventing the spread of infectious diseases. The promptness and accuracy of detection rules improve greatly if multiple independent information sources are available. Earlier work on sequential change detection in continuous time does not provide optimal rules for situations in which several marked count data and continuously changing signals are simultaneously observable. In this paper, optimal Bayesian sequential detection rules are developed for such problems when the marked count data is in the form of independent compound Poisson processes, and the continuously changing signals form a multi-dimensional Wiener process. An auxiliary optimal stopping problem for a jump-diffusion process is solved by transforming it first into a sequence of optimal stopping problems for a pure diffusion by means of a jump operator. This method is new and can be very useful in other applications as well, because it allows the use of the powerful optimal stopping theory for diffusions.

Poisson Disorder Problem with Exponential Penalty for Delay

We solve the Poisson disorder problem when the delay is penalized exponentially. Our objective is to detect as quickly as possible the unobservable time of the change (or disorder) in the intensity of a Poisson process. The disorder time delimits two different regimes in which one employs distinct strategies (e.g., investment, advertising, manufacturing). We seek a stopping rule that minimizes the frequency of false alarms and an exponential (unlike previous formulations, which use a linear) cost function of the detection delay. In the financial applications, the exponential penalty is a more apt measure for the delay cost because of the compounding of the investment growth. The Poisson disorder problem with a linear delay cost was studied by Peskir and Shiryaev [2002. Solving the Poisson Disorder Problem. Advances in Finance and Stochastics. Springer, Berlin, Germany, 295--312], which is a limiting case of ours.

Optimal Stopping of Linear Diffusions with Random Discounting

We propose a new solution method for optimal stopping problems with random discounting for linear diffusions whose state space has a combination of natural, absorbing, or reflecting boundaries. The method uses a concave characterization of excessive functions for linear diffusions killed at a rate determined by a Markov additive functional and reduces the original problem to an undiscounted optimal stopping problem for a standard Brownian motion. The latter can be solved essentially by inspection. The necessary and sufficient conditions for the existence of an optimal stopping rule are proved when the reward function is continuous. The results are illustrated on examples.

Compound Poisson Disorder Problem

In the compound Poisson disorder problem, arrival rate and/or jump distribution of some compound Poisson process changes suddenly at some unknown and unobservable time. The problem is to detect the change (or disorder) time as quickly as possible. A sudden regime shift may require some countermeasures be taken promptly, and a quickest detection rule can help with those efforts. We describe complete solution of the compound Poisson disorder problem with several standard Bayesian risk measures. Solution methods are feasible for numerical implementation and are illustrated by examples.

Sequential multi-hypothesis testing for compound Poisson processes

Suppose that there are finitely many simple hypotheses about the unknown arrival rate and mark distribution of a compound Poisson process, and that exactly one of them is correct. The objective is to determine the correct hypothesis with minimal error probability and as soon as possible after the observation of the process starts. This problem is formulated in a Bayesian framework, and its solution is presented. Provably convergent numerical methods and practical near-optimal strategies are described and illustrated on various examples.