Markov Decision Processes: Discrete Stochastic Dynamic Programming
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Finite State Markovian Decision Processes
Finite State Markovian Decision Processes
OPTIMAL CONTROL OF A SIMPLE IMMIGRATION PROCESS THROUGH THE INTRODUCTION OF A PREDATOR
Probability in the Engineering and Informational Sciences
Markov decision models for the optimal maintenance of a production unit with an upstream buffer
Computers and Operations Research
| Hi-index | 0.00 |
Control limit type policies are widely discussed in the literature, particularly regarding the maintenance of deteriorating systems. Previous studies deal mainly with stationary deterioration processes, where costs and transition probabilities depend only on the state of the system, regardless of its cumulative age. In this paper, we consider a nonstationary deterioration process, in which operation and maintenance costs, as well as transition probabilities “deteriorate” with both the system's state and its cumulative age. We discuss conditions under which control limit policies are optimal for such processes and compare them with those used in the analysis of stationary models.Two maintenance models are examined: in the first (as in the majority of classic studies), the only maintenance action allowed is the replacement of the system by a new one. In this case, we show that the nonstationary results are direct generalizations of their counterparts in stationary models. We propose an efficient algorithm for finding the optimal policy, utilizing its control limit form. In the second model we also allow for repairs to better states (without changing the age). In this case, the optimal policy is shown to have the form of a 3-way control limit rule. However, conditions analogous to those used in the stationary problem do not suffice, so additional, more restrictive ones are suggested and discussed.