Brief Risk-sensitive and minimax control of discrete-time, finite-state Markov decision processes

Authors:
Stefano P. Coraluppi;Steven I. Marcus
Affiliations:
ALPHATECH, Inc., 50 Mall Road, Burlington, MA 01803, USA;Department of Electrical Engineering and Institute for Systems Research, University of Maryland at College Park, MD 20742, USA
Venue:
Automatica (Journal of IFAC)
Year:
1999

Citing 5
Cited 4

Discounted MDP's: distribution functions and exponential utility maximization

SIAM Journal on Control and Optimization
State-space formulae for all stabilizing controllers that satisfy and H∞ norm bound and relations to risk sensitivity

Systems & Control Letters
Risk sensitive control of Markov processes in countable state space

Systems & Control Letters
Risk-Sensitive Control of Finite State Machines on an Infinite Horizon I

SIAM Journal on Control and Optimization
Risk-Sensitive Control of Finite State Machines on an Infinite Horizon II

SIAM Journal on Control and Optimization

Robust Reinforcement Learning

Neural Computation
Risk-sensitive reinforcement learning applied to control under constraints

Journal of Artificial Intelligence Research
Brief On terminating Markov decision processes with a risk-averse objective function

Automatica (Journal of IFAC)
Safe exploration of state and action spaces in reinforcement learning

Journal of Artificial Intelligence Research

Quantified Score

Hi-index	22.15

Visualization

Abstract

This paper analyzes a connection between risk-sensitive and minimax criteria for discrete-time, finite-state Markov decision processes (MDPs). We synthesize optimal policies with respect to both criteria, both for the finite horizon and the discounted infinite horizon problem. A generalized decision-making framework is introduced, which includes as special cases a number of approaches that have been considered in the literature. The framework allows for discounted risk-sensitive and minimax formulations leading to stationary optimal policies on the infinite horizon. We illustrate our results with a simple machine replacement problem.