Matthew D. Bailey

The Optimal Time to Initiate HIV Therapy Under Ordered Health States

The question of when to initiate HIV treatment is considered the most important question in HIV care today. Benefits of delaying therapy include avoiding the negative side effects and toxicities associated with the drugs, delaying selective pressures that induce the development of resistant strains of the virus, and preserving a limited number of treatment options. On the other hand, the risks of delayed therapy include the possibility of irreversible damage to the immune system, development of AIDS-related complications, and death. We use Markov decision processes to develop the first HIV optimization models that aim to maximize the expected lifetime or quality-adjusted lifetime of a patient. We prove conditions that establish structural properties of the optimal solution and compare them to our data and results. Model solutions, based on clinical data, support a strategy of treating HIV earlier in its course as opposed to recent trends toward treating it later.

Modeling Medical Treatment Using Markov Decision Processes

Medical treatment decisions are often sequential and uncertain. Markov decision processes (MDPs) are an appropriate technique for modeling and solving such stochastic and dynamic decisions. This chapter gives an overview of MDP models and solution techniques. We describe MDP modeling in the context of medical treatment and discuss when MDPs are an appropriate technique. We review selected successful applications of MDPs to treatment decisions in the literature. We conclude with a discussion of the challenges and opportunities for applying MDPs to medical treatment decisions.

Oligopoly models for market price of electricity under demand uncertainty and unit reliability

Abstract Several oligopoly models have been proposed for representing strategic behavior in electricity markets, which include Bertrand, Cournot, and Supply Function Equilibrium (SFE). For the most part, these models are deterministic, with the exception of the SFE originally developed by Klemperer and Meyer. However, their model does not include supply side uncertainties. In this paper, we consider both load and supply side uncertainties (resulting from generator availabilities). We obtain Nash equilibrium solutions for Cournot and SFE models, in which asymmetric firms (whose generating units have different costs and capacities) submit their bids so that each firm’s expected profit is maximized.

Eliciting Patients' Revealed Preferences: An Inverse Markov Decision Process Approach

Estimating patient preferences over various health states is an important problem in health care decision modeling. Direct approaches, which involve asking patients various abstract questions, have significant drawbacks. We propose a new approach that infers patient preferences based on observed decisions via inverse optimization techniques. We illustrate our methods on the timing of a living-donor liver transplant.

SPAR: stochastic programming with adversarial recourse

We consider a general adversarial stochastic optimization model. Our model involves the design of a system that an adversary may subsequently attempt to destroy or degrade. We introduce SPAR, which utilizes mixed-integer programming for the design decision and a Markov decision process (MDP) for the modeling of our adversarial phase.

Modeling hospital discharge policies for patients with pneumonia-related sepsis

Sepsis, the tenth-leading cause of death in the United States, accounts for more than

A modeling framework for replacing medical therapies

16.7 billion in annual health care costs. A significant factor in these costs is hospital length of stay. The lack of standardized hospital discharge policies and an inadequate understanding of sepsis progression have resulted in unnecessarily long hospital lengths of stay. In this paper, a general model of when to discharge a patient with pneumonia-related sepsis from the hospital is presented. The model is parameterized using patient-based disease progression data from a large clinical study in order to characterize optimal discharge policies for various problem instances. In the presented experiments, patient health is represented by SOFA scores, which are commonly used to assess sepsis severity. Control-limit policies for specific patient cohorts defined by age and race are demonstrated.

Dynamic Air Tasking Evaluation in a Simulated Network-Centric Battlespace

A common application of Markov Decision Processes (MDPs) is to determine when to replace machines that stochastically degrade over time. Typical assumptions are that there exists an infinite supply of identical replacements, each of which, upon installation, immediately renews the system to the best state, from which stochastic deterioration proceeds. This paper considers a situation for which these assumptions no longer apply: the replacement of medical therapies so as to maximize a patients expected lifetime (or quality-adjusted lifetime). For example, upon taking HIV therapy, levels of the virus gradually decrease; however, viral resistance accrues over time, leading to an eventual increase of viral levels. This prompts a switch in therapy; however, there are only a finite number of effective therapies. A model is presented that addresses these challenges to a stationary MDP framework and a general algorithm for scheduling therapies is discussed.

What Are the Odds? A Structured Approach for Unstructured Problems

Joint Air Operations (JAO) are historically designed through centralised planning using static air tasking that assigns air assets to mission packages for the purpose of achieving campaign objectives. The current methodology cannot anticipate changes in the battlespace nor take advantage of real-time information. In this study, we develop a simulation model of the battlespace that utilises a Multi-Criteria Decision Analysis (MCDA) model to assign vehicles to targets by considering four competing objectives (effort, effectiveness, efficiency, and connectivity). A Voronoi network is used to determine the paths of vehicles to their assigned targets and network optimisation is used to validate the quality of the assignments. The results indicate that dynamic air tasking is considerably more effective and more efficient than static air tasking.

Game—Introduction to Reverse Auctions: The BucknellAuto Game

Decision makers are often faced with ill-defined problems such as determining the odds of unlikely occurrences. However, class settings often explicitly provide such values within problem sets or case studies. In this paper, we discuss an incident of a waitress at a bar being shown her own stolen identification. Using this problem, we motivate the use of mathematical modeling to decompose a problem to determine the odds of a rare event. Similar problem decomposition methods can be used in a variety of business and engineering problems.