Brief On terminating Markov decision processes with a risk-averse objective function

Authors:
Stephen D. Patek
Affiliations:
Department of Systems Engineering, School of Engineering and Applied Science, University of Virginia, Olsson Hall, Charlottesville, VA 22903, USA
Venue:
Automatica (Journal of IFAC)
Year:
2001

Citing 7
Cited 4

Discounted MDP's: distribution functions and exponential utility maximization

SIAM Journal on Control and Optimization
State-space formulae for all stabilizing controllers that satisfy and H∞ norm bound and relations to risk sensitivity

Systems & Control Letters
An analysis of stochastic shortest path problems

Mathematics of Operations Research
Risk sensitive control of Markov processes in countable state space

Systems & Control Letters
Stochastic Shortest Path Games

SIAM Journal on Control and Optimization
Optimal Control: Basics and Beyond

Optimal Control: Basics and Beyond
Brief Risk-sensitive and minimax control of discrete-time, finite-state Markov decision processes

Automatica (Journal of IFAC)

An exact algorithm for solving MDPs under risk-sensitive planning objectives with one-switch utility functions

Proceedings of the 7th international joint conference on Autonomous agents and multiagent systems - Volume 1
Functional value iteration for decision-theoretic planning with general utility functions

AAAI'06 proceedings of the 21st national conference on Artificial intelligence - Volume 2
Risk-sensitive planning with one-switch utility functions: value iteration

AAAI'05 Proceedings of the 20th national conference on Artificial intelligence - Volume 2
Shortest stochastic path with risk sensitive evaluation

MICAI'12 Proceedings of the 11th Mexican international conference on Advances in Artificial Intelligence - Volume Part I

Quantified Score

Hi-index	22.14

Visualization

Abstract

We consider a class of terminating Markov decision processes with an exponential risk-averse objective function and compact constraint sets. We assume the existence of an absorbing cost-free terminal state @W, positive transition costs and continuity of the transition probability and cost functions. Without discounting future costs in the argument of the exponential utility function, we establish (i) the existence of a real-valued optimal cost function which can be achieved by a stationary policy and (ii) the convergence of value iteration and policy iteration to the unique solution of Bellman's equation. We illustrate the results with two computational examples.