Likelilood ratio gradient estimation: an overview

Authors:
Peter W. Glynn
Affiliations:
Department of Industrial Engineering, University of Wisconsin, Madison, WI
Venue:
WSC '87 Proceedings of the 19th conference on Winter simulation
Year:
1987

Citing 2
Cited 37

Sensitivity analysis via likelihood ratios

WSC '86 Proceedings of the 18th conference on Winter simulation
Stochastic approximation for Monte Carlo optimization

WSC '86 Proceedings of the 18th conference on Winter simulation

Likelihood ratio gradient estimation for stochastic systems

Communications of the ACM - Special issue on simulation
Performance interpolation for computer simulation models: a simulation-based approach

CSC '92 Proceedings of the 1992 ACM annual conference on Communications
A tutorial on simulation optimization

WSC '92 Proceedings of the 24th conference on Winter simulation
Metamodels for simulation input-output relations

WSC '92 Proceedings of the 24th conference on Winter simulation
Identifying policies that are most likely to produce a desirable outcome

WSC '94 Proceedings of the 26th conference on Winter simulation
A set of extensions to the SIMAN/ARENA simulation environment

WSC '95 Proceedings of the 27th conference on Winter simulation
Stochastic optimization applied to a manufacturing system operation problem

WSC '95 Proceedings of the 27th conference on Winter simulation
Gradient/sensitivity estimation in discrete-event simulation

WSC '93 Proceedings of the 25th conference on Winter simulation
Variance property of discontinuous perturbation analysis

WSC '96 Proceedings of the 28th conference on Winter simulation
An overview of derivative estimation

WSC '91 Proceedings of the 23rd conference on Winter simulation
Simulation optimization methodologies

Proceedings of the 31st conference on Winter simulation: Simulation---a bridge to the future - Volume 1
Simulation optimization: a survey of simulation optimization techniques and procedures

Proceedings of the 32nd conference on Winter simulation
On improving the performance of simulation-based algorithms for average reward processes with application to network pricing

Proceedings of the 33nd conference on Winter simulation
First and Second Derivative Estimators for Closed Jackson-LikeQueueing Networks Using Perturbation Analysis Techniques

Discrete Event Dynamic Systems
Sample Path Analysis for Continuous Tandem Production Lines

Discrete Event Dynamic Systems
Approximate Gradient Methods in Policy-Space Optimization of Markov Reward Processes

Discrete Event Dynamic Systems
The semi-regenerative method of simulation output analysis

ACM Transactions on Modeling and Computer Simulation (TOMACS)
Sensitivity analysis for transient single server queuing models using an interpolation approach

WSC '04 Proceedings of the 36th conference on Winter simulation
An efficient performance extrapolation for queuing models in transient analysis

WSC '05 Proceedings of the 37th conference on Winter simulation
Geometric Variance Reduction in Markov Chains: Application to Value Function and Gradient Estimation

The Journal of Machine Learning Research
Policy Gradient in Continuous Time

The Journal of Machine Learning Research
2008 Special Issue: Reinforcement learning of motor skills with policy gradients

Neural Networks
Some topics for simulation optimization

Proceedings of the 40th Conference on Winter Simulation
Geometric variance reduction in Markov chains: application to value function and gradient estimation

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Pricing and hedging of financial derivatives using a posteriori error estimates and adaptive methods for stochastic differential equations

Journal of Computational and Applied Mathematics
Kernel Estimation of the Greeks for Options with Discontinuous Payoffs

Operations Research
Strong points of weak convergence: a study using RPA gradient estimation for automatic learning

Automatica (Journal of IFAC)
“What-if” analysis in computer simulation models: A comparative survey with some extensions

Mathematical and Computer Modelling: An International Journal
Uniformization and performance sensitivity estimation in closed queueing networks

Mathematical and Computer Modelling: An International Journal
Variance reduction for sensitivity estimates obtained from regenerative simulation

Operations Research Letters
Techniques for simulation response optimization

Operations Research Letters
Discrete-time 'inversion' and derivative estimation for Markov chains

Operations Research Letters
An infinitesimal perturbation analysis algorithm for a multiclass G/G/1 queue

Operations Research Letters
A New Stochastic Derivative Estimator for Discontinuous Payoff Functions with Application to Financial Derivatives

Operations Research
Improved methods and measures for computing dynamic program slices in stochastic simulations

Proceedings of the Winter Simulation Conference
Reinforcement learning in robotics: A survey

International Journal of Robotics Research
Counterfactual reasoning and learning systems: the example of computational advertising

The Journal of Machine Learning Research

Quantified Score

Hi-index	0.01

Visualization

Abstract

The likelihood ratio method for gradient estimation is briefly surveyed. Two applications settings are described, namely Monte Carlo optimization and statistical analysis of complex stochastic systems. Steady-state gradient estimation is emphasized, and both regenerative and non-regenerative approaches are given. The paper also indicates how these methods apply to general discrete-event simulations; the idea is to view such systems as general state space Markov chains.