Martin L. Puterman

Dynamic Multipriority Patient Scheduling for a Diagnostic Resource

We present a method to dynamically schedule patients with different priorities to a diagnostic facility in a public health-care setting. Rather than maximizing revenue, the challenge facing the resource manager is to dynamically allocate available capacity to incoming demand to achieve wait-time targets in a cost-effective manner. We model the scheduling process as a Markov decision process. Because the state space is too large for a direct solution, we solve the equivalent linear program through approximate dynamic programming. For a broad range of cost parameter values, we present analytical results that give the form of the optimal linear value function approximation and the resulting policy. We investigate the practical implications and the quality of the policy through simulation.

Mixed Poisson regression models with covariate dependent rates.

This paper studies a class of Poisson mixture models that includes covariates in rates. This model contains Poisson regression and independent Poisson mixtures as special cases. Estimation methods based on the EM and quasi-Newton algorithms, properties of these estimates, a model selection procedure, residual analysis, and goodness-of-fit test are discussed. A Monte Carlo study investigates implementation and model choice issues. This methodology is used to analyze seizure frequency and Ames salmonella assay data.

The Censored Newsvendor and the Optimal Acquisition of Information

This paper investigates the effect of demand censoring on the optimal policy in newsvendor inventory models with general parametric demand distributions and unknown parameter values. We show that the newsvendor problem withobservable lost sales reduces to a sequence of single-period problems, while the newsvendor problem withunobservable lost sales requires a dynamic analysis. Using a Bayesian Markov decision process approach we show that the optimalin ventory level in the presence of censored demand ishigher than would be determined using a Bayesian myopic policy. We explore the economic rationality for this observation and illustrate it with numerical examples.

Analysis of Patent Data—A Mixed-Poisson-Regression-Model Approach

Count-data models are used to analyze the relationship between patents and research and development spending at the firm level, accounting for overdispersion using a finite mixed Poisson regression model with covariates in both Poisson rates and mixing probabilities. Maximum likelihood estimation using the EM and quasi-Newton algorithms is discussed. Monte Carlo studies suggest that (a) penalized likelihood criteria are a reliable basis for model selection and can be used to determine whether continuous or finite support for the mixing distribution is more appropriate and (b) when the mixing distribution is incorrectly specified, parameter estimates remain unbiased but have inflated variances.

Improving resource utilization for diagnostic services through flexible inpatient scheduling: A method for improving resource utilization

We present a method to increase the utilization of and reduce the waiting times for an under-capacitated diagnostic resource in the presence of uncertain demand with several priority levels. We consider the case of a computed tomography (CT) scanning department that services both high-priority in-patients and lower priority outpatients. Current practice calls for all in-patient demand to be met on the day of the request. Our proposal looks at the benefit of reserving space for carrying over a percentage of non-emergency in-patient demand to the next day and utilizing a pool of on-call outpatients who can respond quickly to available capacity. We formulate and solve an optimization problem that returns a reservation policy that minimizes unused capacity subject to an overtime constraint. We use a simulation to demonstrate a significant reduction in the growth rate of outpatient waiting time resulting from using the proposed method and investigate the sensitivity of results to several model assumptions.

On the Convergence of Policy Iteration in Stationary Dynamic Programming

The policy iteration method of dynamic programming is studied in an abstract setting. It is shown to be equivalent to the Newton-Kantorovich iteration procedure applied to the functional equation of dynamic programming. This equivalence is used to obtain the rate of convergence and error bounds for the sequence of values generated by policy iteration. These results are discussed in the context of the finite state Markovian decision problem with compact action space. An example is analyzed in detail.

Dynamic multi-appointment patient scheduling for radiation therapy

Seeking to reduce the potential impact of delays on radiation therapy cancer patients such as psychological distress, deterioration in quality of life and decreased cancer control and survival, and motivated by inefficiencies in the use of expensive resources, we undertook a study of scheduling practices at the British Columbia Cancer Agency (BCCA). As a result, we formulated and solved a discounted infinite-horizon Markov decision process for scheduling cancer treatments in radiation therapy units. The main purpose of this model is to identify good policies for allocating available treatment capacity to incoming demand, while reducing wait times in a cost-effective manner. We use an affine architecture to approximate the value function in our formulation and solve an equivalent linear programming model through column generation to obtain an approximate optimal policy for this problem. The benefits from the proposed method are evaluated by simulating its performance for a practical example based on data provided by the BCCA.

Age-Related Changes in Humoral and Cell- Mediated Immunity in Down Syndrome Children Living at Home

ABSTRACT. Abnormalities of humoral and cell-mediated immunity have been described in Down syndrome but reported findings have been inconsistent. Confounding factors have included age, institutional versus home life, hepatitis B antigenemia, and zinc deficiency. To clarify this problem, we studied 64 children with Down syndrome (DS) compared with an age-matched control group. All children had always lived at home. All the DS children were negative for hepatitis B surface antigen. Serum zinc concentration in the DS group was on average 12 μg/dl lower than age-matched control children. They also had significantly lower levels of immunoglobulin M, total lymphocyte count, T and B lymphocytes, and T helper and suppressor cells. In vitro lymphocyte response to phytohemagglutinin and concanavalin A was significantly reduced at all ages in the DS group. Lymphocyte response to pokeweed mitogen increased with age in control children but decreased in the DS children. By 18 yr, the mean response for DS was 60000 cpm lower than controls. The DS group had significantly higher concentrations of immunoglobulins A and G than controls and the difference increased with age. Complement fractions C3 and C4 were also higher in the DS group at all ages. The number of HNK-1 positive cells was higher in the DS group than controls at all ages. When hepatitis and institutionalization are excluded as confounding factors, DS children still differ in both humoral and cell-mediated immunity from an age-matched control group.

Increased compliance in response to salbutamol in premature infants with developing bronchopulmonary dysplasia

We compared the effect of salbutamol and placebo in a double-blind study of preterm infants with bronchopulmonary dysplasia, using a randomized, crossover design with several replicates per subject. Sixty-two tests were performed on 20 ventilator-dependent infants weighing less than 1500 gm. Patients were entered as early as the first week of life and studied for at least 4 weeks or until extubation. Each subject was his own control subject and was randomly assigned to a placebo-salbutamol or salbutamol-placebo sequence administered on 2 consecutive days of each week. Static compliance, expiratory resistance of the respiratory system, and changes in transcutaneous oxygen and carbon dioxide tension were measured. Static compliance improved by 0.240 ml/cm H2O/kg (35.3%) after salbutamol and by 0.010 ml/cm H2O/kg (2.8%) after placebo (p less than 0.0001). The presence of a predetermined decrease in carbon dioxide tension correlated with large changes in static compliance per kilogram and with the need for a high level of fractional inspired oxygen. The magnitude of the clinical and physiologic improvement observed, and the early response suggest that long-term bronchodilator therapy starting as early as the second week of life may be beneficial for very low birth weight infants with early bronchopulmonary dysplasia.

Perturbation theory for Markov reward processes with applications to queueing systems

We study the effect of perturbations in the data of a discrete-time Markov reward process on the finite-horizon total expected reward, the infinite-horizon expected discounted and average reward and the total expected reward up to a first-passage time. Bounds for the absolute errors of these reward functions are obtained. The results are illustrated for a finite as well as infinite queueing systems (MIM/1/S and M/M/1/oo). Extensions to Markov decision processes and other settings are discussed.