Michael Jong Kim

Optimal maintenance policy for a multi-state deteriorating system with two types of failures under general repair

We present a general repair model for a multi-state deteriorating system subject to major and minor failures. The process is modeled as a semi-Markov decision process with the optimality criterion being the minimization of the long-run expected average cost per unit time. A modified policy-iteration algorithm using the embedded technique is developed as the computational approach used to find the optimal maintenance policy. The advantage of using the embedded technique is that it reduces the size of the linear system in the value-determination step of the algorithm resulting in reduced computational efforts for large state spaces. Numerical examples are given which illustrate the implementation of the computational approach.

Joint Optimization of Sampling and Control of Partially Observable Failing Systems

Stochastic control problems that arise in reliability and maintenance optimization typically assume that information used for decision-making is obtained according to a predetermined sampling schedule. In many real applications, however, there is a high sampling cost associated with collecting such data. It is therefore of equal importance to determine when information should be collected and to decide how this information should be utilized for maintenance decision-making. This type of joint optimization has been a long-standing problem in the operations research and maintenance optimization literature, and very few results regarding the structure of the optimal sampling and maintenance policy have been published. In this paper, we formulate and analyze the joint optimization of sampling and maintenance decision-making in the partially observable Markov decision process framework. We prove the optimality of a policy that is characterized by three critical thresholds, which have practical interpretation a...

Maximum Likelihood Estimation for a Hidden Semi-Markov Model with Multivariate Observations†

In this paper, a parameter estimation procedure for a condition-based maintenance model under partial observations is presented. The deterioration process of the partially observable system is modeled as a continuous-time homogeneous semi-Markov process. The system can be in a healthy or unhealthy operational state, or in a failure state, and the sojourn time in the operational state follows a phase-type distribution. Only the failure state is observable, whereas operational states are unobservable. Vector observations that are stochastically related to the system state are collected at equidistant sampling times. The objective is to determine maximum likelihood estimates of the model parameters using the Expectation–Maximization (EM) algorithm. We derive explicit formulae for both the pseudo likelihood function and the parameter updates in each iteration of the EM algorithm. A numerical example is developed to illustrate the efficiency of the estimation procedure. Copyright

Optimal Control of a Partially Observable Failing System with Costly Multivariate Observations

We model a partially observable deteriorating system subject to random failure. The state process follows an unobservable continuous time homogeneous Markov chain. At equidistant sampling times vector-valued observations having multivariate normal distribution with state-dependent mean and covariance matrix are obtained at a positive cost. At each sampling epoch a decision is made either to run the system until the next sampling epoch or to carry out full preventive maintenance, which is assumed to be less costly than corrective maintenance carried out upon system failure. The objective is to determine the optimal control policy that minimizes the long-run expected average cost per unit time. We show that the optimal preventive maintenance region is a convex subset of Euclidean space. We also analyze the practical three-state version of this problem in detail and show that in this case the optimal policy is a control limit policy. An efficient computational algorithm is developed for the three-state problem, illustrated by a numerical example.

A Bayesian model and numerical algorithm for CBM availability maximization

In this paper, we consider an availability maximization problem for a partially observable system subject to random failure. System deterioration is described by a hidden, continuous-time homogeneous Markov process. While the system is operational, multivariate observations that are stochastically related to the system state are sampled through condition monitoring at discrete time points. The objective is to design an optimal multivariate Bayesian control chart that maximizes the long-run expected average availability per unit time. We have developed an efficient computational algorithm in the semi-Markov decision process (SMDP) framework and showed that the availability maximization problem is equivalent to solving a parameterized system of linear equations. A numerical example is presented to illustrate the effectiveness of our approach, and a comparison with the traditional age-based replacement policy is also provided.

Robust Control of Partially Observable Failing Systems.

This paper is concerned with optimal maintenance decision making in the presence of model misspecification. Specifically, we are interested in the situation where the decision maker fears that a nominal Bayesian model may be miss-specified or unrealistic, and would like to find policies that work well even when the underlying model is flawed. To this end, we formulate a robust dynamic optimization model for condition-based maintenance in which the decision maker explicitly accounts for distrust in the nominal Bayesian model by solving a worst-case problem against a second agent, “nature,” who has the ability to alter the underlying model distributions in an adversarial manner. The primary focus of our analysis is on establishing structural properties and insights that hold in the face of model miss-specification. In particular, we prove (i) an explicit characterization of nature’s optimal response through an analysis of the robust dynamic programming equation, (ii) convexity results for both the robust va...

A Stochastic Model for Highway Accident Predictions with Winter Data

In this paper, we consider the problem of modeling and predicting highway accidents in the presence of randomly changing winter driving conditions. Unlike most accident prediction models in the literature, which are typically formulated in a static (e.g. regression models) or discrete time (e.g. time-series models) setting, we propose a continuous- time stochastic model to describe the relation between highway accidents and winter weather dynamics. We believe this to be a more natural way to describe discrete-event highway accidents that occur in continuous-time. In particular, the accident counting process is viewed as a non-homogeneous Poisson process (NHPP) with an intensity function that depends on a (Markovian) weather process. Such a model is known in the stochastic process literature as a Markov- modulated Poisson process (MMPP) and has been successfully applied to queuing and telecommunications problems. One main advantage of such an approach, is its ability to provide explicit closed-form prediction formulae for both weather and accidents over any future time horizon (i.e. short or long-term predictions). To illustrate the effectiveness of the proposed stochastic model, we study a large winter data set provided by Ministry of Transportation of Ontario (MTO) that includes motor vehicle accidents on Highway 401, the busiest highway in North America.

Residual life prediction for systems subject to condition monitoring

This paper presents a parameter estimation and residual life prediction method for a system subject to condition monitoring. We suppose the deterioration process of a system is evolving according to a continuous-time homogeneous Markov chain, including unobservable good state 0 and warning state 1 and observable failure state 2. Multivariate observations which are stochastically related to the system state are collected at equidistant sampling epochs through condition monitoring techniques and they are used to assess the deterioration level of the system. Using the EM algorithm, parameters for the state and observation processes are estimated in the hidden Markov model framework and prediction of system residual life is addressed. A numerical example is provided to illustrate the entire procedure of this approach.

Computational Method and a Numerical Algorithm for Finding the Optimal Control Policy for a Partially Observable System

In this paper a computational method and numerical algorithm is developed for finding the optimal control policy for a partially observable system subject to random failure. The condition of the system is modeled by an unobservable continuous‐time homogeneous Markov chain. Multivariate data stochastically related to the system state is collected at equidistant sampling times. The system is controlled by a multivariate Bayesian control chart, i.e. full inspection followed by maintenance is performed when the posterior probability that the system is in the warning state exceeds a control limit. A statistical constraint is considered which bounds the probability of a true alarm given by the control chart. The objective is to minimize the long‐run expected average cost per unit time by determining the optimal values of the control limit and sampling interval subject to the statistical constraint. The stochastic evolution of the posterior probability process is analyzed and computational algorithm is developed...