Edilson F. Arruda

Stability and optimality of a multi-product production and storage system under demand uncertainty

This work develops a discrete event model for a multi-product multi-stage production and storage (P&S) problem subject to random demand. The intervention problem consists of three types of possible decisions made at the end of one stage, which depend on the observed demand (or lack of) for each item: (i) to proceed further with the production of the same product, (ii) to proceed with the production of another product or (iii) to halt the production. The intervention problem is formulated in terms of dynamic programming (DP) operators and each possible solution induces an homogeneous Markov chain that characterizes the dynamics. However, solving directly the DP problem is not a viable task in situations involving a moderately large number of products with many production stages, and the idea of the paper is to detach from strict optimality with monitored precision, and rely on stability. The notion of stochastic stability brought to bear requires a finite set of positive recurrent states and the paper derives necessary and sufficient conditions for a policy to induce such a set in the studied P&S problem. An approximate value iteration algorithm is proposed, which applies to the broader class of control problems described by homogeneous Markov chains that satisfy a structural condition pointed out in the paper. This procedure iterates in a finite subset of the state space, circumventing the computational burden of standard dynamic programming. To benchmark the approach, the proposed algorithm is applied to a simple two-product P&S system.

Accelerating the convergence of value iteration by using partial transition functions

This work proposes an algorithm that makes use of partial information to improve the convergence properties of the value iteration algorithm in terms of the overall computational complexity. The algorithm iterates on a series of increasingly refined approximate models that converges to the true model according to an optimal linear rate, which coincides with the convergence rate of the original value iteration algorithm. The paper investigates the properties of the proposed algorithm and features a series of switchover queue examples which illustrates the efficacy of the method.

Approximate Dynamic Programming Based on Expansive Projections

We present a general method to obtain convergent approximate value iteration algorithms with function approximation. The result is applicable to any arbitrary approximation architecture and generalizes existing results in the literature derived for particular approximation schemes. Additionally, we show how to obtain a convergent approximate mapping whose fixed point is the projection in the approximation space of a fixed point of the exact dynamic programming mapping with regards to a suitable subset norm. This result relies on evaluating the difference between successive iterates in the selected subset norm, which provides convergent procedures for any arbitrary approximation architecture

Long-term integrated surgery room optimization and recovery ward planning, with a case study in the Brazilian National Institute of Traumatology and Orthopedics (INTO)

Abstract This paper proposes an integrated approach for the long-term planning and surgery allocation problem with downstream constraints. It is motivated by a case study in the Brazilian National Institute of Traumatology and Orthopedics, which provides elective high complexity surgeries for patients from the Brazilian public health system. We introduce an optimization problem that designs a periodic surgery allocation schedule as well as a recovery ward utilization plan, with a view at balancing patient arrivals and releases in the long term, in such a way that all surgeries are performed in a timely manner.

Difficulties in access and estimates of public beds in intensive care units in the state of Rio de Janeiro

ABSTRACT OBJECTIVE To estimate the required number of public beds for adults in intensive care units in the state of Rio de Janeiro to meet the existing demand and compare results with recommendations by the Brazilian Ministry of Health. METHODS The study uses a hybrid model combining time series and queuing theory to predict the demand and estimate the number of required beds. Four patient flow scenarios were considered according to bed requests, percentage of abandonments and average length of stay in intensive care unit beds. The results were plotted against Ministry of Health parameters. Data were obtained from the State Regulation Center from 2010 to 2011. RESULTS There were 33,101 medical requests for 268 regulated intensive care unit beds in Rio de Janeiro. With an average length of stay in regulated ICUs of 11.3 days, there would be a need for 595 active beds to ensure system stability and 628 beds to ensure a maximum waiting time of six hours. Deducting current abandonment rates due to clinical improvement (25.8%), these figures fall to 441 and 417. With an average length of stay of 6.5 days, the number of required beds would be 342 and 366, respectively; deducting abandonment rates, 254 and 275. The Brazilian Ministry of Health establishes a parameter of 118 to 353 beds. Although the number of regulated beds is within the recommended range, an increase in beds of 122.0% is required to guarantee system stability and of 134.0% for a maximum waiting time of six hours. CONCLUSIONS Adequate bed estimation must consider reasons for limited timely access and patient flow management in a scenario that associates prioritization of requests with the lowest average length of stay.

Optimal approximation schedules for iterative algorithms with application to dynamic programming

Many iterative algorithms rely on operators which may be difficult or impossible to evaluate exactly, but for which approximations are available. Furthermore, a graduated range of approximations may be available, inducing a functional relationship between computational complexity and approximation tolerance. In such a case, a reasonable strategy would be to vary tolerance over iterations, starting with a cruder approximation, then gradually decreasing tolerance as the solution is approached. In this article, it is shown that under general conditions, for linearly convergent algorithms the optimal choice of approximation tolerance convergence rate is the same linear convergence rate as the exact algorithm itself, regardless of the tolerance/complexity relationship. We illustrate this result by presenting a partial information value iteration (PIVI) algorithm for discrete time dynamic programming problems. The proposed algorithm makes use of increasingly accurate partial model information in order to decrease the computational burden of the standard value iteration algorithm. The algorithm is applied to a stochastic network example and compared to value iteration for the purpose of benchmarking.

Stochastic economic lot sizing and scheduling problem with pitch interval, reorder points and flexible sequence

This paper presents a solution for a class of the stochastic economic lot sizing scheduling problem that is typical of the replenishment pull system proposed by the lean manufacturing approach. In this class, lots of any product are produced in fixed intervals called pitch. The proposed solution uses flexible production sequences and reorder points that are compatible with the concepts of supermarket and level production. It adopts the queuing discipline obtained from a fluid model that approximates the stochastic process of arrival and production orders. Given the queuing discipline, an iterative algorithm returns a near-optimal solution for the system. The proposed approach allows us possible to differentiate inventory cost and service levels by product, and the stock required is lower than that required by the discipline ‘first stock out, first out’. The algorithm is fast and stable, allowing its frequent use in real-world instances.

Solving average cost Markov decision processes by means of a two-phase time aggregation algorithm

This paper introduces a two-phase approach to solve average cost Markov decision processes, which is based on state space embedding or time aggregation. In the first phase, time aggregation is applied for policy optimization in a prescribed subset of the state space, and a novel result is applied to expand the evaluation to the whole state space. This evaluation is then used in the second phase in a policy improvement step, and the two phases are then alternated until convergence is attained. Some numerical experiments illustrate the results.

An application of convex optimization concepts to approximate dynamic programming

This paper deals with approximate value iteration (AVI) algorithms applied to discounted dynamic (DP) programming problems. The so-called Bellman residual is shown to be convex in the Banach space of candidate solutions to the DP problem. This fact motivates the introduction of an AVI algorithm with local search that seeks an approximate solution in a lower dimensional space called approximation architecture. The optimality of a point in the approximation architecture is characterized by means of convex optimization concepts and necessary and sufficient conditions to global optimality are derived. To illustrate the method, two examples are presented which were previously explored in the literature.

Oil industry value chain simulation with learning agents

Abstract Simulation is an important tool to evaluate many systems, but it often requires detailed knowledge of each specific system and a long time to generate useful results and insights. A large portion of the required time stems from the need to define operational rules and build valid models that represent them properly. To shorten this model construction time, a learning-agent-based model is proposed. This technique is recommended for cases where optimal policies are not known or hard and costly to unequivocally determine, as it enables the simulation agents to learn good policies “by themselves”. A model is built with this technique and a representative case study of oil industry value chain simulation is presented as a proof of concept.