Risk-averse dynamic programming for Markov decision processes

  • Authors:
  • Andrzej Ruszczyński

  • Affiliations:
  • Rutgers University, Department of Management Science and Information Systems, 08854, Piscataway, NJ, USA

  • Venue:
  • Mathematical Programming: Series A and B - 20th International Symposium on Mathematical Programming – ISMP 2009
  • Year:
  • 2010

Quantified Score

Hi-index 0.00

Visualization

Abstract

We introduce the concept of a Markov risk measure and we use it to formulate risk-averse control problems for two Markov decision models: a finite horizon model and a discounted infinite horizon model. For both models we derive risk-averse dynamic programming equations and a value iteration method. For the infinite horizon problem we develop a risk-averse policy iteration method and we prove its convergence. We also propose a version of the Newton method to solve a nonsmooth equation arising in the policy iteration method and we prove its global convergence. Finally, we discuss relations to min–max Markov decision models.