Percentile Optimization for Markov Decision Processes with Parameter Uncertainty

Authors:
Erick Delage;Shie Mannor
Affiliations:
Department of Management Science, HEC Montréal, Montréal, Quebec H3T 2A7, Canada;Department of Electrical and Computer Engineering, McGill University, Montreal, Quebec H3A 2A7, Canada
Venue:
Operations Research
Year:
2010

Citing 10
Cited 10

Robust Convex Optimization

Mathematics of Operations Research
Bounded-parameter Markov decision process

Artificial Intelligence
Markov Decision Processes: Discrete Stochastic Dynamic Programming

Markov Decision Processes: Discrete Stochastic Dynamic Programming
Neuro-Dynamic Programming

Neuro-Dynamic Programming
L2-Gain and Passivity Techniques in Nonlinear Control

L2-Gain and Passivity Techniques in Nonlinear Control
Robust Dynamic Programming

Mathematics of Operations Research
Bias and Variance Approximation in Value Function Estimates

Management Science
Robust Control of Markov Decision Processes with Uncertain Transition Matrices

Operations Research
Convex Approximations of Chance Constrained Programs

SIAM Journal on Optimization
Model based Bayesian exploration

UAI'99 Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence

Theory and Applications of Robust Optimization

SIAM Review
A Distributional Interpretation of Robust Optimization

Mathematics of Operations Research
Distributionally Robust Markov Decision Processes

Mathematics of Operations Research
Probabilistic goal Markov decision processes

IJCAI'11 Proceedings of the Twenty-Second international joint conference on Artificial Intelligence - Volume Volume Three
Optimization Under Probabilistic Envelope Constraints

Operations Research
A framework for computing bounds for the return of a policy

EWRL'11 Proceedings of the 9th European conference on Recent Advances in Reinforcement Learning
Adaptive planning for markov decision processes with uncertain transition models via incremental feature dependency discovery

ECML PKDD'12 Proceedings of the 2012 European conference on Machine Learning and Knowledge Discovery in Databases - Volume Part II
Robust Markov Decision Processes

Mathematics of Operations Research
Shortest stochastic path with risk sensitive evaluation

MICAI'12 Proceedings of the 11th Mexican international conference on Advances in Artificial Intelligence - Volume Part I
Robust Modified Policy Iteration

INFORMS Journal on Computing

Quantified Score

Hi-index	0.00

Visualization

Abstract

Markov decision processes are an effective tool in modeling decision making in uncertain dynamic environments. Because the parameters of these models typically are estimated from data or learned from experience, it is not surprising that the actual performance of a chosen strategy often differs significantly from the designer's initial expectations due to unavoidable modeling ambiguity. In this paper, we present a set of percentile criteria that are conceptually natural and representative of the trade-off between optimistic and pessimistic views of the question. We study the use of these criteria under different forms of uncertainty for both the rewards and the transitions. Some forms are shown to be efficiently solvable and others highly intractable. In each case, we outline solution concepts that take parametric uncertainty into account in the process of decision making.