Asymptotic theory of statistical inference
Asymptotic theory of statistical inference
Importance sampling for stochastic simulations
Management Science
Sensitivity of the Stationary Distribution of a Markov Chain
SIAM Journal on Matrix Analysis and Applications
Importance sampling for the simulation of highly reliable Markovian systems
Management Science
Reinforcement learning with replacing eligibility traces
Machine Learning - Special issue on reinforcement learning
Using permutations in regenerative simulations to reduce variance
ACM Transactions on Modeling and Computer Simulation (TOMACS) - Special issue on modeling and analysis of stochastic systems
Passage time distributions in large Markov chains
SIGMETRICS '02 Proceedings of the 2002 ACM SIGMETRICS international conference on Measurement and modeling of computer systems
Neuro-Dynamic Programming
| Hi-index | 0.00 |
We consider the problem of estimating passage times in stochastic simulations of Markov chains. Two types of estimator are considered for this purpose: the “simple” and the “overlapping” estimator; they are compared in terms of their asymptotic variance. The analysis is based on the regenerative structure of the process and it is shown that when estimating the mean passage time, the simple estimator is always asymptotically superior. However, when the object is to estimate the expectation of a nonlinear function of the passage time, such as the probability that the passage time exceeds a given threshold, then it is shown that the overlapping estimator can be superior in some cases. Related results in the Reinforcement Learning literature are discussed.