A guide to simulation (2nd ed.)
A guide to simulation (2nd ed.)
Importance sampling for stochastic simulations
Management Science
Notes: conditions for the applicability of the regenerative method
Management Science
Asynchronous Stochastic Approximation and Q-Learning
Machine Learning
Analysis of an importance sampling estimator for tandem queues
ACM Transactions on Modeling and Computer Simulation (TOMACS)
Fast simulation of rare events in queueing and reliability models
ACM Transactions on Modeling and Computer Simulation (TOMACS)
Importance sampling for Markov chains: computing variance and determining optimal measures
WSC '96 Proceedings of the 28th conference on Winter simulation
Asynchronous Stochastic Approximations
SIAM Journal on Control and Optimization
Accelerated simulation for pricing Asian options
Proceedings of the 30th conference on Winter simulation
Analytical Mean Squared Error Curves for Temporal DifferenceLearning
Machine Learning
The O.D. E. Method for Convergence of Stochastic Approximation and Reinforcement Learning
SIAM Journal on Control and Optimization
Actor-Critic--Type Learning Algorithms for Markov Decision Processes
SIAM Journal on Control and Optimization
Eligibility Traces for Off-Policy Policy Evaluation
ICML '00 Proceedings of the Seventeenth International Conference on Machine Learning
On the Lock-in Probability of Stochastic Approximation
Combinatorics, Probability and Computing
Efficient simulation of buffer overflow probabilities in jackson networks with feedback
ACM Transactions on Modeling and Computer Simulation (TOMACS)
Analysis of state-independent importance-sampling measures for the two-node tandem queue
ACM Transactions on Modeling and Computer Simulation (TOMACS)
ACM Transactions on Modeling and Computer Simulation (TOMACS)
STEWARD: demo of spatio-textual extraction on the web aiding the retrieval of documents
dg.o '07 Proceedings of the 8th annual international conference on Digital government research: bridging disciplines & domains
Estimating the probability of a rare event over a finite time horizon
Proceedings of the 39th conference on Winter simulation: 40 years! The best is yet to come
Reinforcement learning in the presence of rare events
Proceedings of the 25th international conference on Machine learning
A New Learning Algorithm for Optimal Stopping
Discrete Event Dynamic Systems
Approximate zero-variance simulation
Proceedings of the 40th Conference on Winter Simulation
On the inefficiency of state-independent importance sampling in the presence of heavy tails
Operations Research Letters
Markov chains, Hamiltonian cycles and volumes of convex bodies
Journal of Global Optimization
Zero-Variance Importance Sampling Estimators for Markov Process Expectations
Mathematics of Operations Research
A vision for a stochastic reasoner for autonomic cloud deployment
Proceedings of the Second Nordic Symposium on Cloud Computing & Internet Technologies
| Hi-index | 0.00 |
For a discrete-time finite-state Markov chain, we develop an adaptive importance sampling scheme to estimate the expected total cost before hitting a set of terminal states. This scheme updates the change of measure at every transition using constant or decreasing step-size stochastic approximation. The updates are shown to concentrate asymptotically in a neighborhood of the desired zero-variance estimator. Through simulation experiments on simple Markovian queues, we observe that the proposed technique performs very well in estimating performance measures related to rare events associated with queue lengths exceeding prescribed thresholds. We include performance comparisons of the proposed algorithm with existing adaptive importance sampling algorithms on some examples. We also discuss the extension of the technique to estimate the infinite horizon expected discounted cost and the expected average cost.